First, congratulations on the new little one and completing your family! And no, you shouldn't feel guilty.
However, I'm so glad to see you are sensitive to those who are struggling with infertility around you. You seem to be very conscious and thoughtful and coming from years of infertility treatments myself I appreciate that consideration.
For the 13 years we went though infertility treatments, I have to say it was the most emotionally draining experience I've ever had to endure. I was angry when I saw a pregnant woman. I was angry when I saw kids on the news who had been beaten, abused, forgotten, killed.
I wanted nothing to do with anyone who was pregnant. It had absolutely nothing to do with the person who was pregnant as a person and everything to do with my own misery.
The thing that would really upset me was to hear someone say how they were so "blessed" that they didn't have trouble having kids. This upset me because it made me feel even smaller and more broken and so much more "un-blessed". As if somehow, because another woman could have kids they were so much more in the graces of God than I. Looking back that kind of sounds kooky but when you are searching for reasons within yourself and around you things start to take on extreme changes in meaning.
Just hearing how "blessed" everyone else was I couldn't see my own blessings. I had plenty of them....they just didn't happen to be child related.
Now, after 13 years, I have 2 beautiful girls. One by adoption and one by birth. And still today, I am very careful around those I know are struggling and I'm thankful that at least others are too. Because until you've been in those shoes, you truly have no idea how it affects someone.
Best of luck!