Speaking as a divorced mom with a 10 year old ( we got divorced when my son was 4) I would recommend seeing a mediator for a few reasons:
1) If your ex is paying child support, he is entitled to his visitation. BUT- he is also supposed to have a set schedule, which is better for your daughter as well. You are supposed to both make an effort to meet each other's needs, but it sounds like he is demanding that your daughter's schedule just be moved around ever month or week and that is definitely not what is best and most stable for her.
2) If you ended up going to family court over this, most likely the FIRST thing a judge will do is ask if you've been to mediation. If you haven't, he will probably require you to go anyway- so you may as well do it now. It is less expensive than paying for a lawyer and court costs as well. We have found it very effective in settling disputes over vacation, etc. Your attorney can probably recommend a bunch of local mediators who specialize in divorce/family issues or you can look online.
Normally when we have seen a mediator we each paid on our own for our private meeting, then split the cost halfway for the meeting together.
3) The good thing about the mediator is that not only are they totally familiar with all the legalities of your visitation agreement and how it is supposed to work, they are a non-interested party who will not take sides. Often the mediator will meet once with you and once with your husband on your own, and then you will meet together to hash out what to do.
Also, he will look at your Joint Parenting Agreement- and if you don't have one YOU NEED ONE RIGHT NOW. If you haven't got one, the mediator is the person who will help you work out those other issues as well as the schedule. Then the mediator will write it all up into one legal document for you BOTH to sign off on and you can have your attorney file it (or just a scheduling agreement as an addendum to your original parenting agreement or divorce arrangements) in court. This way, it is legally binding and you have something to fall back on if your ex keeps trying to change things on you.
Seriously, no matter how friendly the divorce was or how nice things seem to be now, or how much your ex loves his daughter- get EVERYTHING in writing. I learned the hard way - please take my advice on this and realize that just saying " Oh sure, we can do that" means nothing legally if your ex changes his mind or his circumstances change.
4) Be prepared- make sure the mediator knows that you are not trying to keep your daughter from seeing her daddy- but that you want things to be stable for her and this hodge-podge schedule will not be predictable and comforting for a 4 year old! Also, you are willing to accommodate him reasonably- but come on, to be realistic, you have a work schedule too and he needs to meet you halfway here.
Ahead of time, figure out your MAIN goals- what is it you really want to get out of this mediation? It will keep things on track ( the mediator is also good at that- they will keep you from getting bogged down in old issues, other problems, etc, so you can focus on this one thing) and better insure a good outcome for you.
For example, if your main concern is how to schedule this December trip, make that a priority. Say " Although I think it would be more reassuring for Susie if she saw her dad on the same day each week, if there is just NO way to do that, then we will have to figure something else out. But I need to buy plane tickets for our trip to her grandparents in December ASAP to get the best rate. Can we agree that I will have Susie for those two weeks and my ex can have her for 4 extra days in January when we get back?"
I know it's difficult- but crazy planning ahead and scheduling is ABSOLUTELY the way to go here. Not only will it be MUCH more reassuring for you daughter if you can point to the calendar and say " see? Next week you are at Daddy's house on Monday and Tuesday. I will pick you up on Wednesday morning for preschool." or to show her ahead of time where she will be on various holidays, etc. Trust me- it will make her feel much more in control of the situation just to be able to see that ahead of time.
But also- I have to tell you, it will save you a TON of arguing and bickering, etc. to have as much of the schedule hashed out ahead of time as possible. Honestly, the way your ex wants to do it is just a fight waiting to happen.
My ex and I have our basic schedule, in which he has our son roughly every other weekend ( we used to do it so he had a weeknight over night as well, but somehow after a month or so, it never works out, so I have stopped even trying to maintain it and he has never requested it back again) and we do holidays every other year ( if I have Thanksgiving, he has Christmas, etc). He gets my son for one week vacation in the summer (I've offered him 2, but 1 is all he says he can take off work).
So that is MOSTLY worked out. We both try to be flexible when special events come up or weekends need to change, etc. and that helps a lot. BUT- in addition to all that- about twice a year, in spring and in fall, we have a giant go-around of emails back and forth with calendars and dates and schedules, just to set up summer ( day camp, swimming lessons, vacations, visiting grandparents, etc) and then again to set up the school year schedule.
It is important to understand that you are not you ex's 'secretary' here. Honestly, its sad sometimes, but all you can do is set up the schedule and stick to it as well as you can and be there for your daughter if he drops the ball- whether because of work or whatever. Some things are just not going to work out.
but- see a mediator about this and let them help you. It absolutely will help and hopefully your ex will see that this is about your daughter feeling safe and secure- not about you being hostile!
good luck and happy scheduling!!