My 3.5 yo LOVES to play Candyland, and it has really helped him learn how to take turns in games. I love it because there are not a TON of pieces and the set-up is extremely easy.
We have tried Chutes and Ladders, but he's not that into it yet. Maybe another 6 mos before I try it again.
He also loves Hi-Ho-Cherry-Oh, which is a great game for learning simple counting skills.
We also play Go Fish with a deck of Mickey Mouse cards.
Memory (ours has Toy Story pictures on it) is great and can be used for many years. When I started it with my son he was about 3 and I really limited the number of tiles that I would set out. As he got older I'd set out more and more tiles. We are not yet using all of the tiles, but almost there.
We have Cooties, but there are a TON of pieces, and he wasn't very interested.
Another great game is Elefun, which is basically an elephant-shaped fan with a long trunk that blows out L. tulle butterflies. It is good for hand-eye coordination, since the kiddos have to catch the butterflies in their nets.
LOTS of puzzles. We buy a bunch of the ones that come in L. square boxes from Target or the dollar stores. They are about $4 and are more challenging than the ones that have frames.
Lincoln Logs - my son likes to see how high he can build it before it topples. He is not quite ready for some of the "plans" that they provide, but it is definitely something that we can work on together for long periods of time.
Megablocks or larger Legos are great, but he prefers for me to do it with him than on his own.
I just bought some Moon Sand, but haven't tried it yet. I am planning on getting an indoor sensory table so that we can work with some of this stuff in a designated area (like the basement) for easy clean-up.