Consider renting kitchen time from a restaurant or another bakery.. This way you will not have to worry about getting your own kitchen in the beginning..
If you want to do it out of your home and deliver baked goods to businesses in the beginning to save up cash, look into city codes and permits. See if your own kitchen will work. Also see if you are allowed to have this business in your neighborhood. Some city codes are pretty tough.
I have a friend that has started with a "Pie Wagon". It is a trailer that she sells pies out of.. She does the baking out of a rented space. She has a few clients that she keeps stocked with her pies and then she actually sells to the public out of her trailer.. She has applied for a loan to take over an established bakery. The owner is going to also help finance it in the beginning for her.
http://www.cutiepiewagon.com/
You may want to look into actually working in a bakery to see if it is what you really want to to. The hours are really tough in that business. I know the bakeries in town begin baking at 2:00 am - 3:00 am to open at 6:30 for morning customers..
There is also a very successful cupcake business in town. This young man started by selling out of an airstream trailer, he now also has his own cupcake bakery in a storefront.
http://www.heycupcake.com/story.html
The other new and up coming bakery is Sugar Mama's Bakery. It is pretty tiny, but her product is so good, she has quickly gained a following.. I love her "icing shots" and I do not normally even like icing..
http://www.sugarmamasbakeshop.com/
Also I read, Gesine Bullock-Prado’s memoir, "Confections of a Master Baker". This is Sandra Bullocks sister and she has her own bakery. She tells about the ups and downs of owning a bakery.
Good luck to you. Any new business will take up 90% of your time, so be sure you and your family are prepared.