Most teachers have some "extra help" days. Why doesn't he start with that, working with the same teacher? It will help to cement their relationship. He should tell her that the class seems to go too fast for him (it won't sound like a criticism of HER but it will tell her where his problem lies). By looking at his tests and homework, she should be able to see where he's missed something. He'll also, very likely, see that other kids are in extra help. It's early in the year and a lot of kids have trouble getting off to a good start. If, together, they decide that he'd be better off in the other group, then so be it. It will help his self-esteem to work with the teacher, get some one-on-one or small group help, and perhaps master what he missed. The longer you wait, the more he will miss and the worse he will feel.
The labels of "accelerated" and "slow" are very damaging. I think it's more important to say that people, both kids and adults, learn in different ways. There are different class groupings to gear the work to different learning styles. You should not assume that everyone in the "slow" group has a behavior problem! You can also tell your son that different teachers have different teaching styles. They communicate differently, and so on. Just because he doesn't adapt to this one teacher's style doesn't mean there is anything wrong with him. Really reinforce that with him!
I also don't see the value in pushing kids beyond their level - if they're working hard, there's nothing wrong with an occasional C in middle school. If they can get A's & B's, well, fine. I also don't like measuring kids against each other because of who took algebra in 8th and who took it in 9th. There's plenty of time for kids to achieve in various areas and to move into different levels through high school.
My son did not take Advanced Placement math, for example, in high school, he had a great 4 years, and he got into a great college. Our neighbor took AP everything, was stressed beyond belief to get all A's, and got into a mid-level college. He had a miserable high school existence and complained for 4 solid years about how hard everything was as he stayed up late every night to get A's. So there are many factors involved. Activities, leadership opportunities, community service & volunteering, summer jobs, recommendations - these all matter much more than certain grades.
Your child probably has many skills and many strong areas. Is he a good reader? A creative writer? A history buff? Celebrate those. While he needs to work at all his subjects, maybe math will turn out not to be his "thing." Make sure he is making friends and adapting to middle school in other ways. We need to try not to stress our kids out in middle school.