K.P.
It's "too close for comfort" for me. I don't think a married man should be at all concerned with any woman's sex life except his own wife. Period. End of story. For one thing, if it's at work, he could open himself up to charges of sexual harassment; and whether at work or not, he is skating on thin ice, and it looks like it's pretty easy to move from "innocent" flirting and/or talking about her sex life into thinking about her, lusting after her, and then finally sleeping with her. Best to keep himself far away from the temptation, than to see how close he can get to it without falling in.
And as another commenter said, if the man's wife or children are made uncomfortable by the man's flirting, then it needs to stop out of respect for them, even if "nothing is going on." And, if the man wouldn't tell his wife or wouldn't want her to know, then that's a huge red flag that he knows it's not right.