I, too, talked up the "pacifier fairy" and said he is big enough now to give it to her so another baby who really needs it can have it. I talked about this for about 3 weeks and said when he was ready to let me know. Well, we were also potty training at the time (my son was 2 1/2) and as he was waving "bye-bye" to his poop as it was being flushed, the pacifier fell out of his mouth and, yep, went down the toilet. He looked at me and I said, "what happened?" He just pointed and said, "my pat!" (that is what he called it). This was in the afternoon and I debated the rest of the day, "should I just go with it?" "He saw what happened to it, I didn't take it away". Hubby no help, said it was up to me. I felt so guilty, but decided to give it a try. So, I hid all the extra "pats" and told him it was gone, which he understood. He did cry a bit that first night and the second, but was fine by the 3rd.
I know it is hard, but I agree, sooner is better than later. There is no "perfect time".
I also agree that your hubby can come up with a plan and implement it if he is so concerned about it.
Good luck!