I agree with Valerie. It sounds like this may be attention seeking behavior, especially when you say that if you give him what he wants while throwing a fit he continues to hit himself. At this point, he has your full attention which is what he wants and this is called function of behavior. To start with, I would put him in time out and set the timer for 2 minutes. I would have a certain chair that is only used for that purpose. Don't let him play in it otherwise. THE SECRET TO THIS WORKING IS THAT YOU MUST REMAIN CONSISTENT IN PUTTING HIM IN TIMEOUT WHEN HE IS INAPPROPRIATE every stinking single time he gets up until you think you are going to scream if it happens again. You have to reset the timer each time he starts over. Second, once you can get him to stay in time out then WALK AWAY until the bell rings. Make sure dad is in on the plan as well as anyone else that is caring for him. I know it is unsettling for him to be hitting himself but when you try to stop him you are reinforcing the function of his behavior. After he has successfully completed his time out, I would reward him with a little hug and I would say something like "there is my good boy. Mommy does not like it when you act like that." Third, I would focus on his good behavior and only reward those. This is hard because it is so easy to fall back into those habits we already have deep within us.
If you choose to use the walk away method, then the most I would say is "that is not appropriate behavior, when you are done let me know so we can have fun" when he acts up. The first few times you do walk away it will be the hardest walk of your life. Be prepared, when you do this, it usually gets worse before it gets better so you will have to make sure that though you walk away, you keep an eye on him to keep him out of dangerous situations like pulling over shelves or running outside without supervision. It is a very effective technique for some children but if you do not see any success in consistent application after several weeks, try the time out method.
I found the time out method more effective with my son. Once he figured out that I was serious about time out he got it and now he is 10 and knows that his timeout does not start until he gets to a designated spot and is quiet. From that point, he has also learned that each time I have to remind him that his time out hasn't started I add one minute to his time. I then give him 5 minute updates and praise him for how well he is doing. I might say "you are doing a good job standing in time out. you have completed 5 minutes of your time" I know your son is younger but if you get him in the habit of this now, when your new baby comes along, he might regress but it will be easier to get in place because he is already used to this technique. I like this one also because you can do this anywhere. You can use a bench at the mall, a chair in the shoe store while you browse, a blanket on the floor at someone else's house. Once he gets the concept of time out down, he will learn the time portion of it quickly. I didn't start adding verbal minutes though until my son was 8 or 9 for not complying with a time out request though the minutes might have been added if he was really noncompliant or not if he went straight to timeout and started.
The important thing is consistency. If you do this today and not tomorrow or you do this for 4 days and miss the 5th, then he knows your holdout point is 4 days and that if he keeps the pressure up for long enough he will again gain control of the situation. And yes, believe me these little bitty, innocent children know how to break you down girl...quite young infact.
Figure out what you think will work for you and be consistent with it for at least 3 or 4 weeks before modifying or changing what you do. It will also be helpful for other adults to have a written copy of the plan you are working with so he will see consistency there as well.
It takes at least 21 consistent days to break a habit and 3 days to make one, even in children.
Good luck
C.