A.V.
It makes sense, but try to use it as a positive. Since the birth of my great niece, I've been able to see her father more and have a better relationship with him because of it. Consider doing things with the older kids to get them out of the house while SIL is dealing with baby sleep deprivation. You might initially go to see the baby, but you might find that you can find new ways to relate to the older kids because you see all of them more. Does that make sense? Maybe take the kids out to lunch or do a "just the girls" day while your DH does a "just the boys" day.
And if you have different relationships with different kids, is that so bad? We're all individuals, right? So long as no one is being shunned or treated poorly, then it's OK to be closer to one or another person.