E.T.
If your son -- at age 4 -- has trouble understanding how to answer questions and communicates mostly by pointing or pulling, then he's developmentally delayed for his age. It's not because you didn't put him in daycare. It's probably something going on in his brain that therapy might help. Please take him to the pediatrician and ask to have him evaluated.
What do the preschool teachers say?
Edited to add: Nannette - you can call your local school district and ask that he be evaluated. If your preschool is part of the public school district, they might be able to get the evaluation started for you, but if it's a private school, all you have to do is call the public school and ask how to get some testing done to see what therapy he needs. It's great that the school he's at now suggested therapy - just don't forget to contact the public school to do a full evaluation. You may find that the therapy he's in now is all he needs, but the evaluation might find he needs speech, or sensory, or play therapy, or tons others. It sounds like you want the best for your son and that you're working very hard to help him become a confident learner. Hugs to you!