Your son sounds like a very well-behaved child for his age. It's good that you want to stop this inappropriate behavior. You can do that gradually, however, without resorting to hitting your child for hitting. That can be confusing to many kids – they hear you say no, but then you do what you tell them not to do. Some kids get it, others just don't.
But your son is at a very frustrating age – he probably gets told "NO" many times during the day. He tries to do things that he's not yet able to manage. He doesn't get to touch and manipulate many of the things he sees around him and finds interesting. And parents get to make him do things on their own schedule, without thinking much about how exasperating this might be for him. If you realize how frustrating this all is, you might be able to give him advance warnings when you'll need him to do something.
His brain isn't developed enough yet to consistently realize his behaviors result in consequences. He'll get there – almost all kids do by the time they reach 4 or so, and his impulsive behavior will gradually improve until he's in control of himself more or less consistently. But you'll need to come up with a consistent response for him to get there more quickly – he'll just be confused if you react differently every week.
If I were in your shoes, I would use the hand-holding approach – take his hands immediately, gently but firmly EVERY time. If he's holding something he intended to throw, calmly remove it from his hand and put it out of reach. Tell him with a firm voice something like "We don't hit people." Give him words to use instead of actions: "I feel mad!" "I want that, please." "I want to finish doing this, please." "Will you help me?" If it seems appropriate, let him pound on a pillow to get his frustration out, or even get him into a giggly mood.
He's learning by your example how to respond to upset and frustration. No parent is perfect, but do your best to stay in a calm, cheerful place. He will get there eventually. Two-year-olds are well-known for being unable to control their moods well. By four or five, children have learned to be much more civilized.
Marda's suggestion of the book How to Talk… is a good one. I've used those easy-to-learn techniques with my grandson since he was 2.5, and it's been very good for our communication.