Go to a Mother's Day out program. Many churches run them and they are for 1 or 2 days a week usually 9-2 or so. It is great for moms because you get some free time to run errands, take care of yourself, etc and it is great for the children because they start socializing, making friends, etc.
I uunderstand you don't want him in a daycare because you don't want him sick but at some point, he needs to be with other children to learn socialization skills, etc.
Other options are a music class like Kindermusic
Activity class involving gymnastics, etc
Our community Parks and Rec dept has great things for little ones to do with and without parents.
Remember that you need time as well to recharge so you can be the best mom you can be.
Have fun... they grow up WAY too fast.