C.S.
Does she give a reason for these last minute visits?
If not, I would warn her in a nice way that you need to put her visits on your calendar to avoid conflicts. Otherwise it will be catch as catch can.
Next you need to break her of asking what you are doing that you cannot cancel. This is done with firmness and generalities. Instead of, "I made lunch plans with my old friend Linda," you might try, "I am sorry but I made plans for Friday weeks ago." If she inquires further, be ready to change the subject in a friendly way. Perhaps you can even mention the two of you planning something for her next visit. It sounds like you have a fairly good relationship with her, but do not assume for one minute that whatever you tell her is not going to get back to your ex. It will no matter what she might be telling you. Count on it.
You are young and this is all part of taking control of your life and making it go the way you want it to, but I am impressed that you have maintained relations with his family. That is great for your daughter.
Good luck,
C. S.