DC is perfect for kids - especially on a budget - nearly everything is free! I lived there for a year when I was pregnant with my first, and I was not working (we were there for my husbands job) so I did all the sights by myself and with a newborn :)
My favorites for kids that age would be:
Natural History Museum - GREAT Dinosaur exhibit and other animals too
Air and Space Museum - of course your son will love it
The National Zoo is great for all your kids. The metro will take you pretty close, but you'll have to walk a few blocks once you get off.
Places to eat with kids -
I agree if you can pack snacks (I assume you'll have a stroller?) that will lessen the cost and pain of restaurant eating.
If you are heading to the Zoo - when you get off the metro at Woodley Park, there are restaurants there at are kid friendly. If you walk just south from the stop there is an american place on the opposite side of the street that was never busy, and had burgers/fries, etc. There are also numerous ethnic restaurants at that location. From that station you have to walk up a hill a bit to get to the zoo. There is also a Starbucks and another small restaurant right across from the zoo.
If you take the metro to one stop past the Woodley Park stop (I can't remember the name) there are restaurants there as well. There is also a Cold Stone Creamery ;)
If you get off the redline at Dupont Circle - there is a fantastic greek place there called Zorba's. There greek pizza is super tasty.
http://www.zorbascafe.com/
If you are down on the mall, and have walked toward the monuments there are places to eat that are not hotdog vendors. I often stopped at Potbellys:
Potbelly Sandwich Shop - 17th & Penn
Potbelly Sandwich Shop - 1400 NY Ave
And, in the lowest level of the National Art Museum, there is a link between the old building and the new Modern building with a decent cafe style eatery. It's a pretty place to sit too - with a cool wall fountain and good bathrooms.
Things to think about as you travel with a stroller - there are elevators in most of the metro stops, but check the website to confirm that they are all working. Sometimes they were out of order.
Plan extra time to get from place to place. Sometimes the elevators spit you out in a different location than the main metro exit. Make sure you have good maps, and don't hesitate to ask for help. I found people to be very nice, especially when you have a baby with you.
You can send me a message if you have other questions!
J.