First of all, your husband should not be having communication with her. Period. IF you and your husband need advice for your child, then I would suggest consulting your pediatrician. This will eliminate the need for your husband to talk to this woman about your child.
Secondly, I cannot for the life of me understand why this woman's husband would want his wife's old boyfriend to work on plumbing in their house. This is extremely bizarre to me. I think ANY man would feel uncomfortable having his wife's ex-boyfriend working on something that he should be doing himself.
Thirdly, your husband needs to stop communicating with her. Period. They need to remove themselves from each others fb page. This relationship that has "rekindled" is inappropriate (1) because he is married to you and (2) she is married to someone else. Talking about your child is one thing...talking about their past relationship is inappropriate. Old feelings will most certainly arise from this and I don't care how much you "trust" your husband, temptation is NOT something to mess with.
Do not believe he will never cheat on you. I know this may sound harsh but you need to understand that there are many forms of adultery and often it starts out on an emotional level, much like you have described. A line has already been crossed when your husband didn't want you to see him texting with her and you had to find out by picking up his phone on your own. If this happened to me, I would sit my husband down and tell him that this "relationship" that he has with this woman makes you feel uncomfortable and that you want him to stop communicating with her, that you want him to block her from his fb and block her number so he doesn't receive texts, if that is possible. Otherwise, you explain to him that there is no room in your marriage for the insecurity this new found relationship is instilling in you. You have EVERY right and reason to protect your marriage and if it means putting your foot down and telling him what you want and how this makes you feel, then you do it. You have to protect your marriage. This woman is an intrusion and it needs to be blocked immediately. IF your husband refuses to stop communicating with her, then I would be so bold to contact her myself. Maybe she is unhappily married. Maybe she is pursuing your husband because she has regret that they broke up so long ago. Whatever her reasons, this relationship needs to end today.
And, I don't like that your husband stated that "you may invite yourself," to her house so he can do the plumbing job. How in the world do you know for sure that her husband will indeed be there? Tell your husband to cancel it and tell her to hire a professional plumber. Otherwise, I would not allow my husband to step foot in that house, with or without me.