You are doing the right thing to intervene now. As he progresses in school, so much will be expected of him as of every child.
He will need to be able not only to listen and follow directions but also to remember what to pack in his backpack to bring home; what form to give you that the teacher says his parents just have to complete tonight; what to do for homework (and to put it iin his backpack when he completes it, so it's not sittiing at home the next day while he says to the teacher, "But I did do it!"); where he left his lunch box (so he's not sitting at lunch starving, and therefore grumpy and even less focused all afternoon); which bus to take on the one day he has a different schedule and needs to go somewhere other than home; etc. It sounds like he's not yet on track for that kind of responsibility and remembering, and it will come upon him, and you, quickly now that he's in elementary school.
Get him evaluated by a psychiatrist or psychologist or counselor (have you started with the school counselor??) who specializes in child behavior. Emphasize that he doesn't have issues with anger or violence and doesn't seem to be ADD or ADHD, but a typical kid who is "scattered." You should get specific advice and techniques for helping him at home with improving his attention, his ability to do tasks as asked and when asked, etc.
Do try the school counselor first; this is why they're there, and if he or she thinks your son needs more help outside school, ask for referrals to professionals. But do it soon -- the school year is nearly over and you need to start working on this now, in the school setting, before summer comes and he's relaxed and listening even less.
One small thing to help might be lists, lists, lists. We knew a boy with this issue of keeping things together, bringing home what he neede to bring home from school, taking the right things to school, doing what he had to remember to do, etc., and the counselor told his folks to POST lists everywhere: On his bathroom mirror for his morning get-ready tasks, on the back of the front door for his "what must go to school" list, on top of his desk at school for what must come home....It was a good start on organizing him. Make sure you are there to go over the home lists with your son each day for a long time, so he gets used to following them.
And work with his teachers on getting him out of trouble and more able to follow directions at school. Are there certain times of day he has more issues with following directions? Is he more wound up and less able to do as he's told, for instance, right after recess? Before lunch? Late in the day? Pick apart the circumstances that seem to be toughest for him and talk with the teacher about whether things can be done to "set him up for success" and make it easier for him to follow directions. And praise him a LOT when he does follow them and when he is able to calm himself down appropriately too.
And one person asked what consequences you were using along with charts and rewards. Whatever you do both for rewards and consequences, settle on something and be consistent -- don't shift from one consequence to another etc. He needs to know exactly what happens EVERY time he does X, or what the reward is every time he does Y.