Personally, I think that you should try to talk to the pastor. Let them know that you would like to have some classes more geared towards families. Even offer to lead one to kind of get some interest going. Maybe then all of the parents could take turns leading it or someone who has grown children would be willing to lead it.
As far as the kids programs goes. Someone has to be willing to lead it. If no one is then it won't get done. Maybe you could talk to your pastor about starting a children's program. It will be difficult and a lot of work at first but once you get it off of the ground, then you will get more volunteers and the load will get lighter.
It sounds like you're involved in a small church. My husband (and kis) and I have been involved in churches ranging from a start up church that started with 10 or 15 people to a church that had regular attenders in excess of 1200 every week. The start up church we were involved in, I believe was as successful as it was because of the children's program. That was the first thing that was launched. The kids loved coming and the parents brought them then liked the church and stayed. The really big church had a great kids program as well.
We actually just changed churches because we were attending the large church. We felt very lost in the crowd there and had a hard time figuring out area's to get involved in. We tried to contact the youth pastors about some programs that we would have loved to run and we never heard back, which was very discouraging. Our kids loved that church but we needed something smaller where we could know people. We tried several churches. To be honest, anything that didn't have a kids program, we didn't return to. We're at a church that probably has a couple hundred right now and love it. It's got a great kids program and everyone is very nice!
My kids are older than yours so we asked them every time we went to a new church how they liked it and we weighed that with how we liked it. I think that the decision as to when to leave is something totally up to you and your husband and something you should pray about. You will have peace and you will find something else out there if you're supposed to go. Personally, my husband and I would take into consideration the fact that there's not much of a children's program, because as our kids get older we want them to actually want to attend instead of having to attend. And we would take into consideration how connected we felt.
Sorry this is so long. Good luck to you!