I wouldn't be annoyed. Now if you'd said no gluten... that's even more difficult! LOL
But, no, I think if your child has a need to avoid it, it shouldn't be an annoyance to parents who are bringing things into the classroom. Personally, I would view it just like a child with a peanut allergy... Yes, you have to pay closer attention to what you're bringing to school, but as a parent, it is our choice to bring that into school to start with.
Now if you were attending another child's birthday party (not at school) and requested the mom not to use the dye, that would be different. In that circumstance, the inconvenience would have to be on you - to either provide an alternative for your child, discuss with your child why he couldn't have what the others were having, or not attend the event. That is an optional event for your child. School is not.
I think most parents would be understanding. The biggest problem I would expect, would be if the class is constantly sending home lists for parents to provide things to class parties and the lists are filled with items typically containing red dye... (Kool-Aid drinks, candy, etc). If the lists are health minded to start with -- mini-water bottles, carrot sticks and dressing, grapes, cheese sticks, etc... then that is a non-issue anyway.
I personally wish that ALL teachers would send home a note at the beginning of the year indicating what types of "treats" are acceptable to send in or not... In the early years it seems the kids are inundated with TONs of JUNK all year long!
Probably no one said anything, because it's not that big of a deal, and people are likely sympathetic to your situation. I know someone who's child has problems with the dye. It's a little more planning to work around it, but it keeps me mindful that I should be grateful that I only have to work that way SOME of the time... not daily! :)
God Bless!