Could you rotate and do a party at home one year, and then go to grandma's house the next year? That's what we used to do for Christmases growing up, and it was fine.
I personally wouldn't mind having family and friends show up. I had about 40+ guests for my son's first birthday party. The difference between his party and a lot of others we've attended: instead of paying for a place that limits how many you can invite, or what they can do, and buying gift bags and favors and all that, we rented a nice ampitheater at a park with a nice little playground, that also has a little spray park, had access to LOTS folding chairs and tables, and private bathrooms for like $100 for 4 hours. (That gave us time for the men to set up tables/chairs while the women laid out the buffet and gift table). We brought a stereo, had a pull string pinata, and a few games planned for the bigger kids, but don't can't remember if we even did them or not. I spent like $80 on food (we made a his cake, jambalaya which was AWESOME, white beans, made my own veggie tray, bought some french bread, and friends brought drinks-waters, sodas, juices-people actually stood in line for leftovers to take home!)We had so much fun just visiting, watching the children run and play, remeniscing about what all's happened in a year, played, etc). It ended up being much longer than expected. We had plenty of help: the men put up the chairs/tables, someone else swept up real quick, someone else took the bags of trash out, etc. We closed up the ampitheater as the contract agreed, and just stepped over to the park and continued hanging out. I did schedule the party right after his regular nap, and his grandma watched him while we set up. I sent Joseph home for his bath and to rest up with grandma as we hung out a little longer because noone wanted to go. It's just once a year, and instead of buying material stuff, we just made it all about loving on everyone. Whatever you do, I hope you have a good time! We would do that EVERY year if we could get people together...that party had people from Ca, Ga, Al, La, and Tx, old soccer buddies, highschool friends, family, aquaintances, some peoples' dates, all their kids, "whoever". Throwing some hotdogs on a grill is also a cheap way to feed lots of kids if they're not into jambalaya. (Louisiana kids are ALLLLLL into it though) :)