I will be the first to admit that some kids, once you issue punishment, will comply, while others will continue to push the envelope. I have witnessed both and they all turn out okay - but the parents suffer in the interim.
So, first of all, make sure you are consistent, and very clear on what behaviors will merit punishment as well as what type of punishment. Then, issue the punishment the FIRST time the undesired behavior is exhibited. If you dont, then the child is not really sure you are serious about it, and then if after several weak warnings the punishment is finally issued, the child is very frustrated (as are you) and the child is angry because by that point, they feel the punishment is unjust, which leads to temper tantrums, etc. (Inconsistent punishment sets your child up for failure - no wonder they are angry.)
My own experience (which is supported by Supper Nanny's advice) has proven that "time out" works the best (not slapping hands or spanking). A young child dearly loves their parents and will do anything to please them and mostly, be with them and spend time with them. Any attention, even negative attention, will increase a behavior. Therefore, have a "time-out" spot and take them there (haul if necessary) immediately and close the door. Keep time-out short - but if they KNOW you will remove them everytime, without variance, they will learn.
Secondly, observe your child to watch for his "cues" as to when he is tired or getting frustrated. Put them in "quiet time or nap time before they reach that "over stimulated" point where the behavior deteriorates quickly.
Lastly, your child is 4 yrs old and will be going to school soon. Try to talk with him and find out what gets him frustrated and then find ways to put elements of the world within his control that are reasonable. (As a healthcare designer, studies show that in-patients improve faster when elements of their care are within their control - even simple things like what time and what they eat, the room temperature, choice of artwork in the room, etc). My own "Mom" experience has proven that finding ways for your child to control his own environment, within reason, not only teaches them responsibility but also shows that their behavior does impact what they do and do not receive.
It is very important to master this concept now because, teens are nothing more that bigger toddlers (they grow and develop and mature up til about 12 yrs old - and then those darn hormones kick in- and it seems like you are back to 4 yrs old again - only bigger). Kids nowadays are much smarter and know how to manipulate the system - and the parents. So - if they are not TOTALLY convinced at age 4 that you will follow through on the established guidelines - you wont have a chance at age 14.
sorry this is so long, but i read the other comments. Channa S had great advice about giving child permission to have the tantrum - but somewhere else! This gives them some choice and control - while still following your rules. Also, Heather F advice about James Dobson book, "the Strong Willed Child". i also read this - and followed most of this advice - as does Super Nanny.
Good luck, hang in there - and watch the Super Nanny weekly for moral support. She really does know what she is talking about.
About me: a 53 yr old working mom of two grown children, a grandmother on a 3 yr old, married 30 yrs. An interior design major and child psychology minor.