Listen to your instinct- it doesn't feel good to leave your children that way for a reason! When you honor and respect your children's need for attachment and staying close to you, they paradoxically develop a genuine sense of security and independence in their own unique and gradual way. This way also optimizes Trust and Relationship.
Be willing to stay with them as long as they need you (which may be the entire service), making it a place they are free to come and go from with you (ie, spend a little time all together in the service, then come play for a bit with them in the nursery, go back and forth a bit). Eventually, it will become a place that feels comfortable and enjoyable and they'll be willing to stay and hang out while you leave. At this point you may agree to come back and check on them periodically and/or ask the attendant to get you if they need you. Eventually, they'll be running in there saying "bye, mom!" Again, Trust & Relationship are required here, which take time and consideration to build. But when you make that investment by honoring their needs, it will pay back in dividends. Take heart, this phase of their lives will not last forever!
You also might appreciate the wisdom at gentlechristianmothers.com.