My advice is to keep it small! You're already finding out the problems with expenses even for small groups. The larger it gets, the more unmanageable it is anyway, and the kids get overstimulated. Think also about what a "special party" means. For me, it means very good friends and a fun activity, with a few gifts. Once you invite the whole class, it's not a special invitation for any of the guests (they all feel you were obligated to invite them), and now it's 20 or 25 gifts, and your child learns that a birthday is a big gift fest rather than a celebration of her special day with only special people. Then you get into the whole thing of parents feeling that they don't have to RSVP because, gee, there are 25 kids invited and so what's the big deal about one more person. Mamapedia is just full of understandable rants from party-giving moms who planned for 12 and got 30, or planned for 30 and got 5. It's nothing but frustrating.
Siblings are another problem. If the kids are roughly the same age and play together, then invite them. The older the kids get, the more likely the parent is to drop off the invited child and then come back. But if you hold a party at a venue like a Chuck E. Cheese, then parents feel they can stay. If you have a huge gathering, they often feel like they SHOULD stay to help supervise. Then you are faced with feeding and entertaining all these extra people.
So our policy was always to invite only those kids my son would really enjoy - not the children of co-workers, not those with whom he played occasionally, and not siblings. We also followed the "age rule" - he got to invite the number of kids for the age he was turning. When he was 5, he invited 5. When he was 7, he invited 7. By the time he was 11, he wanted to do expensive things like lunch and a movie, so he only got to invite the number of kids who could fit in our car! It was great - he got to really talk to the few kids he invited, he got a few gifts (and had a manageable number of thank you notes), and he got to open the gifts in front of those giving them. So everyone got to see the pleasure in the gifts, receive heartfelt thank yous to their face, and move on.
If you have 20 kids, they don't want to sit there for the gift-opening because it takes forever, and then the gift becomes more like the price of admission to the party. It's just a terrible idea. So be careful before you start down that road when she goes to kindergarten - it's hard to change your mind once your child gets used to huge parties. And all you will do is lament the costs and hassles and all the extra people who show up!
Life is about making choices. You feel guilty leaving people out but really, no one goes to everything. We also turned down invitations to parties if he didn't really know the kid. We didn't want to spend every other weekend working around a birthday party with 25 kids in the class all going to each other's things, adding in parties for neighbors and "occasional" friends and the children of our own friends. My son had different friends than we did and that's okay.
And when people ask you what you are doing for her birthday, it may be sincere interest from her good friends, but it can also be a bit of trolling for info about venues and sucking you into the competition of "whose parents are giving the best parties?" - you don't have to play that game.