I was in your position a few months ago. We "agreed" years ago that I would work and he would stay home. He was of the opinion that he was "doing his job" by taking care of our child. Okay, fine... yes, taking care of a 3 yr old is work, but our "deal" was that he would be a stay-at-home parent. To me, that encompasses running the household. We had to actually get into an argument to get it worked out. My problem with him was not that he didn't have everything clean all the time. (I never did when I stayed home) My problem with him was that he COMPLAINED about the place being a mess all the time. (and acutally called me a poor housekeeper!)
The first thing I did, was stop cleaning. I let our daughter's room go a whole month! That helped some. Eventually, I had to break it down to him... when he worked, he expected me to take care of everything at home. (kids, cleaning, bills, etc.) Why should it be any different when the roles are reversed? I was by no means a "perfect" housekeeper, but I don't expect him to be perfect either. I just expect him not to complain about the place being a mess or dishes or laundry being piled up when he's home every day.
I don't have a problem with toys being out or a few dishes needing to be washed, (and he does not cook dinner, I do that when I get home) but when things get too messy and he starts complaining about it (implying that I should've cleaned everything up), it infuriates me. (and he wondered why I was often too tired to have any "alone time")
I think you'll just have to explain to him that you have little time with the kids, and you don't want to spend that available time cleaning. And... if he were at work all day, wouldn't he expect you to pick up? PLUS, you'll be too tired to perform any "wifely duties" if you spend all day at work and then start cleaning the moment you walk in the door! :)