L.D.
It IS normal for kids to behave that way - but they need to have adults that correct them and teach them the right way to behave.
To me, the problem isnt with the little boy, it is with the sitter.
I would not let her bring the the little boy if she is not going to address his behavior.
I have a two and a half year old son and a one year old daughter, and my son DOES swat at her, push, shove, etc to keep her away from his toys and we address his behavior, give timeouts, praise when he shares, teach him other options (such as giving her an alternate toy). We do NOT say "Oh, its normal" and let get pushed around.