I know exactly how you feel! When I had my eldest daughter's fifth birthday and started inviting a few school friends for the first time, I had no idea the parents would all stay. It's not a show! I was most uncomfortable having all those adult eyes watching everything I was doing. The worst thing, I found though, at that age, were that the invitees would sometimes cling to mommy or want the parent involved in the games and it completely changed the dynamics of the way the day went. I have worked with kids forever and I knew that I could handle the group and get them having fun if it were just a group of birthday party kids without the audience but instead it was so awkward. I expected a kid's birthday party like when I was a kid. Parents never came that I remember.
We once got an invitation for my son that said "this is a drop off party". Fortunately I knew the parents well enough to be comfortable with that being that it was just a 2-3 hour party. When I arrived at the house, I realized that it was very small and they had all kinds of fun activities planned all over the house and parents really just didn't belong. All the parents obeyed and the party was a smashing success. The kids were only five.
My kids have since asked, if there is any doubt, that we write on the invitation "this is a drop off party". I usually add, "due to lack of space" if that is the reason. I don't want parents wondering why they are not welcome. One time we had a magic show at the end of the party and I invited all the parents to join us for that part. That seemed appropriate. Other times, I just explain all the things we are doing at the party. That is always helpful anyway so parents know how much food is being served or if it's just snack stuff. I like the invitations to be really clear so everyone knows what's going on.
Ironically, when you are friends with all the parents, you may want to have a birthday party where the entire families are invited, but if you don't even know them, who wants them hanging around? Of course those are the people, though, who more likely will want to stick around!
Okay admittedly I am a very protective parent and now that my kids have all been invited to so many different parties, including some where I didn't know the parents, I get the whole "I'm not just dropping my kid off at a stranger's house". In some situations I have asked other parents whom I trusted if they knew the parents of the birthday child and usually I could get a feeling that way.I would not let my kids sleep over unless I knew the family well. I have been very surprised at the few families who have no issue whatsoever with sending their kid anywhere, even sleep overs without even talking to the parents hosting! One birthday party was with a very flaky mom who took them all to Six Flaggs by herself. I later found out she had lost one of the kids for some time. Thankfully I had trusted my instincts on that one and my kid was "busy" that day! Uggh!
Since yours is only two families, I would get to know them ahead of time and invite them over to meet you and see your safe house explaining the reason for doing so. Good luck.