I see the last two responses were what I would recommend, don't allow your son there unless you are too. If this seems to be better, you can try allowing him a short visit without you but you also have to remember that even when grandparents are "normal," your child is going to do this.
My mother does not allow any child in her house that is not well behaved but one of the reasons that kids are holy terrors when they get back from grandma's is because grandma is able to give the child undivided attention. Even if you are a SAHM, you have to cook and clean. Grandma knows she can do that stuff when the child is not around so he goes from an environment where he is truly the center of the universe to home, where he of course receives a lot of attention but in a different way.
Grandparents also do not have to deal with the proclivities of children on a daily basis so they truly do not get irritated when the kid takes 20 minutes to put on his shoes because he decides to unlace them first. Whoohoo, this is amusing, let's put them back and then put on the shoes and go. Mom's reaction, probably not so much even if she tries not to be aggravated, the kid senses the change underneath.
So, while you may be absolutely right about why your child is so difficult when he comes home, it is also pretty much inherent in the grandparent dynamic. If grandpa is crocked much of the time, I would not allow my child there without me, or I would make sure that it was early in the day when hopefully he had not been drinking already.
As another woman suggested, if the grandparents wonder why the change, be honest with them and ask your husband to be the one to tell them since he made the decision to bar visits and you seem to have experienced the backlash for that.
You know, grandparents are wonderful but if they are countermanding everything you are trying to teach your child, you do not have to allow them to see the child. As much as it would hurt to make that decision for more than a temporary fix to the situation, if that's what it takes, you do it. Nobody said that parenting was easy, some of the decisions are very hard and/or painful.