L.D.
When my children were that age, I used to have them carry around a toddler sized backpack (the kind with the handle and wheels) filled with their favorite toys, crayons and coloring books. I found that, they couldn't run very far or very fast when they were loaded down like that. You will also find that the backpack will come in very handy when you go out to eat because it contains all that your child needs to keep them entertained until the food arrives. I also made sure that I had the latest Oriental Trading company catalog in there as well. The pictures are very fun for kids to look at and there are great labeling opportunities for you to share with your child.
Also, make a game of your outings with him. When you are at the store, make it a mission to try to locate as many things as possible that are a particular shape (circle, square), number (3, 7) or color (orange, blue). Not only is this educational, it is very fun for your child to learn new things on in a adventurer/explorer type of game. Also, it keeps your child occupied and he'll have less time or opportunity to think about waundering off from you.
The last thing I would suggest is setting up something that is like a positive behavior plan for your child. When your child does something that you don't like, it is not enough to tell him what he is doing wrong or punish him for it, it really helps if you do what I call "do-overs," which is basically recreating the situation right away and modeling for him initially the appropriate way that he should have acted or responded and having him "do it over". He should pick this up fairly quickly so he should know what a "do over" and do it automatically when you tell him to. You will probably have a lot of doing it over before he begins to automatically do it right the first time without the need for a do over but be patient-- it takes a lot of work to civilize children.
You may also want to set up a reward system for him when you see him engage is in a desired behavior spontaneously at random times throughout the day. Pick one or two behaviors that you would like to see from him like "saying please and thank you" or "using his indoor voice" and, throughout the day, if you catch him doing just that without having to be told to do so or without him doing it and expecting a reward, give him a small reward like a sticker or whatever to encourage him to keep doing what he just did.
Now, when you are on the phone or talking to someone else, you are going to be interrupted because that's what kids do and that's what they do best. But, if you keep a basket of toys, books, whatever on hand that you can pull out for your kid to play with for just those particular occassions, it may give him something to do while you have your conversation. The key to this working is that the toys and items that are in this basket have to be unlike anything that he has available for him to play with on a regular basis. These have to be special toys that he will really, really want to play with so that it will keep him entertained while you talk for a bit.
And the last thing that I wanted to mention to you is that he's not too young for time out in the naughty corner if he keeps on being a repeat offender or if he is acting too wild and out of control. Find a special spot that is quiet and not where the tv is, and that can be your naughty corner. If he gets put there, have him stay a full 3 minutes (a minute for each year of his life), and each time he tries to leave before the time is over, the timer gets set back again so that he spends the full 3 minutes in the corner. When the time is up, explain to him why he was put in the naughty corner, have him repeat what you just told him and have him apologize to you and then hug him and tell him that you love him.
I hope this gives you some inspiration on how to help your son learn and behave when you are with him. Wishing you all the best.