If he's not happy I would have second thoughts, but I would not move him if he's happy, especially if it has kids he'll go to kindergarten with, that's a good benefit. Unfortunately there will be times when either you or your child don't click with the teacher. It's a real problem if your kid is unhappy, not learning, being bullied, etc. But short of that, it's part of life.
I'm acutally there with you, my son's preschool teacher and I do not click. But he is there for free through the town, and it will be the elementary school he attends, so the benefit of him getting used to the building and routine are great.
I had a problem with her witholding recess the second day of school for talking, esp for age 4. She just said to me, well, we like to be firm early to establish rules. So I gave it another chance. She then sent home a note asking me to re0inforce that he could not talk in class. I beleive that school is also where kids learn social skills. So I approached her with my son and the note the next day. She was with another parent, I waited hanging back, but she motioned me over to address the issue in front of the other parent. I have all sorts of issues with this, but decided it was not private to me so I asked her my question in front of the other parent. I told her that we talked about not talking while the teacher was teaching, or doing work. But then I told her that at our house when I reinforced when the kids cannot do something, I also let them know when they can, so I needed for her to let my son and I know when it IS ok for the kids to be talking. You could tell that at that moment she felt very much put on the spot in front of the other parent. It got her to tell me that the kids could talk quietly before class, at snack, when on the rug, etc. My son insists that before this he was not allowed to talk before class or at snack, but now he is. At this point he likes school, and like I said for him there are extra benefits. But honestly, if he was still miserable I would have second thoughts about keeping him there.
I would just keep talking to the teacher about specific issues when they come up, try to keep things calm and ask what the two of you can do to resolve the issue together.
Good luck!