M.P.
Have you tried helping him get to sleep while breast feeding before he's overly tired? It may be he's just too tired to focus on anything including feeding.
Walking to help him go to sleep is a common way to help babies get to sleep. I wouldn't be concerned. You have him most of the time. Why not let his Dad have success with this one thing? I also suggest that he would probably go to sleep if you walked him without trying to breast feed him. Try holding him in a way so that he's not facing the breast.
Sounds to me that you're doing just fine.