Well, his grandparents raised him. So in HIS MIND they're his parents. Take that for what you think it's worth but it's a fact. In HIS MIND you have taken him from his home, made him come live in your home, demand he bond to you and think of you as his parents, and more. He bonded with the in laws and in his mind they are his parents.
Sounds like your whole family, husband, you, son, and grandparents need to do some family counseling. To work through all this.
I am a grandparent raising grandchildren and I know, from counseling, that my grand kids tell others that we're their parents. In therapy, in school, in writing assignments, and more, they call us their parents. But in real life they call us grandma and grandpa. But for all intents and purposes we are their parents from the day they came into our home we've been that parent role to them.
We sent one of the grand kids back to his mom. It devastated him. He cried for days and days and begged his mom to let him go home. He told his teachers at school he wanted to go home, he told the therapists he would be good and mind if we'd just let him come home.
The school district we'd moved into was horrible for him in first grade. If I could do it over I'd just home school him but his mom lived in a town nearby that has some of the best schools in Oklahoma. I felt he would succeed there and excel. He has too, once he calmed down and got used to coming over all the time and spending a lot of time here.
My point is that the kid comes first. You have to allow him as much time with them as possible. He's going to hate you and act out and turn on you more and more and more if you take them away from him.
He needs them. You and your husband need to go to therapy with him, and the grandparents sometimes so they can have input into what they see going on. They know him, better than you do because they've been there for all his developmental years. They have insight you don't have and might never have.
They are a resource and you're treating them like they are thieves taking something away from you.
So bring your family together, fix this by working together instead of pulling it all apart.