Both my daughters were adopted at ages at which there was no "hiding" the fact from them (one was 4, one was 9).
My parents were divorced and both remarried before my children were born/we adopted, so they have both grandparents and step-grandparents. My oldest son was adopted by my husband when he was young, so he's technically only a half-brother to my youngest son and neither are, of course, "blood related" to their sisters.
We don't hide anything from any of the children. The use of the term "real" is what's damaging in these situations, in putting too much emphasis on blood relationships that, in my experience, are really quite irrelevant. Use the term "blood related" or "biological" but never, ever use "real" to describe a biological relation to a child when you're trying to differentiate relationships. That's what's damaging and confusing. "Real" relations are those that are involved in your life, that have an impact, that act in a manner you expect of that role in your family.
Eventually your kids are going to be asked to make a family tree in school. They always are, for some reason. You could wait until then or you could just explain it now. For some reason some children feel very betrayed to find out in their teenage years that some relative isn't biological. I don't know why, but some do. Given that it's almost always best to explain earlier rather than later. You might want to start by finding some interesting story about an animal that's adopted a different animal - dogs and kittens, cats and puppies, ducks and X...there's tons of them out there. And use that as a starting point to discuss family relationships and how sometimes a mom or dad just isn't able to be a mom or dad and that someone else is.
You don't think it's a big deal because you've lived with the knowledge all your life. Remember it's new to them no matter when they discover the situation and that it can be a lot more complicated than it seems with the perspective of experience and age. So yes, I think earlier is better simply because when it's discovered in their teenage years they equate it with lying and hiding information. That's not the case, of course, but teenagers tend to be paranoid, egocentric creatures that are certain whatever you're doing revolves around them personally.
Just my opinion, but I'd tell them sooner rather than later. My youngest is 20 months, but when he's old enough to understand - 6, 7 - then we'll explain all the crazy relationships in our family to him because he deserves to know what everyone else knows. As you say, it isn't a big deal, so why hide it?
*** I asked my oldest son about his feelings on the subject and he agreed with most of the posters - there's no reason to ever tell your son "the truth". He says it just doesn't matter, but if you were going to tell him to wait until he's older. He thinks it would be better as a teenager to find out if he was going to find out again.
The only other factor in this is medical history. A lot of disease/problems are genetic, so knowing an accurate family medical history is important. If he doesn't know the truth, he's giving an inaccurate history that may lead doctors down the wrong path in the future. ****
L.