Have you talked with your husband about this? I have been in the same situation, but my husband is my son's step-dad. It has always seemed to me that it might just be a male-bonding thing. I don't know but I also get jealous of my husband.
My son thinks that the only person to run to when he gets into trouble by me is his step-dad. But what he doesn't realize is that is that is not the solution for that. I sometimes have to warn him that if he doesn't listen to me, he will have to deal with his step-dad or his own father when he sees him.
If I were you, I wouldn't get stressed out about it, because it will hurt your body when it hits. I've been through that. Just maybe you need to work outside of the home and see if you can place your son in a child care facility to see if that would work. Then, you will notice that your son will not always cling on to his father.
I have been in your shoes and I still am. I really know what you are going through. My husband works nights, so I have a lot more time with my son than he does. But, when we are both home, the same things that are happening to you, happen to me.
Have a talk with your husband to let him know how you really feel. If you have and there has been no progress, think about yourself and talk with a close family member that might be able to give you some pointers.
I would tell your husband that he needs to help out with taking care of your son and to at least let you do the work that you need to get done without any problems. It should be a couple thing, not an individual. He should know that already.
But,if you do need anything, just give me a ____@____.com name is S. Brooks. (I have a 6 year-old son, named Zachariah.