I know that you have already made the choice to go back, but I wanted to let you know that schooling with a toddler underfoot is probably one of the hardest things you will ever do. Since you have only 2 children its hard to find someone else to distract the child while you accomplish something with the older child. I've definitely gone through this frustration. We are not so worried about whether we are accomplishing objectives "on time" because we do not have to compare our children with other children as in the school world. If we need to deal with a younger sibling who needs just as much attention as a school age child we spend the time doing this.
I'm surprised that it is taking 3-4 hours to get through a 1st grader's work (unless he's just having so much fun he can't stop :) When we talk to 1st year homeschoolers in a support group that I'm in, we encourage them to break the subjects into tiny chunks and work diligently for short amount of time to help them to learn to use time efficiently. We recommend seatwork types of work be kept at about 30 min. per grade level with a max of about 5 hours in high school. The rest of the day can be spent helping around the house, playing, mom reading to child, fun school type projects, practicing reading to self, taking up an instrument, doing art projects. Many of these things can include little ones in tow. Maybe you are expecting him to sit still too long?
Sure the older child has to learn to tolerate and maybe even enjoy the younger child, but that ends up being a lesson that can't be manufactured in a school setting.
If he's doing what you've described, you really don't have to worry too much about placing him in the school environment. I think that academically you already know his strengths and weaknesses and can do a few things over the holidays to recall to his mind and encourage him for the new way of schooling. Be careful as you prepare to go back about putting pressure to perform perfectly before going off to school. He'll do fine. You've provided him a good education so far (even though you may not think so.)
Enjoy Christmas as a family. This will do more to prepare him than you think.