As others have said... it all depends.
For bootcamp & MCT (Marine Combat Training), no. For his A or C-school... it depends on how long his school is. Some are only a few months (or less) some are over a year. For the longer ones, yup, you can go with, and the 2 of you can get offbase housing (the onbase housing list is always very very long, and it goes by a system that prioritizes by rank, length of duty assignment, and number of kids. Regardless, he'll be getting extra pay from the moment you two wed and he files you as a dependent... and then once he's actually posted (or in some schools).
For deployements... it also depends. If he's being deployed into action (Iraq, Afghanistan, etc.), no. If he's being deployed overseas to a stageing area (like germany)... yes. If it's just regular orders (anywhere in country, or overseas not in action) yes. As RM said... there are accompanied tours and unaccompanied tours.
Most of my married friends' spouses joined them when we got to our schooling (but we were in "long" schools... I was training for almost my entire first 2 years in). As a child, my dad had already been in for over 15 years before I was born (he enlisted at 17)... and we ALWAYS went with him wherever he was stationed. He wasn't usually with us (he was typically at sea 6+ months out of the year), but it meant that when he WASN'T at sea, we were right there. That doesn't always equal a lot of time. Some postings he would leave at 4am and not get home until late in the night. Others he was in at 8am and off at 4pm. Some posts you're working 10+ hour shifts 5,6,or7 days a week... some you're working 8 hours 5 days a week and have weekends. Sometimes you're pulling 72 hours, other times you're cycling between days and nights. Shifts are based on a lot of factors... and you never know what you're going to get. It depends what's needed. As soon as you sign on the line, you're property... and you go where you're told when you're told.
A lot will depend on what his MOS is, and where he's ordered to.