It's wonderful that you're being so dedicated to breastfeeding. Here's the good news. You're almost out of the woods. What you're describing is the hardest part of it, and you're learning fast.
Try starting from the other end. It will require a lot less measuring and a lot less math :) How many wet diapers is he making? (Check his diaper often so you know it's only one pee, not two or three that you're looking at.) I don't remember how much he needs to be making, minimum, at this age, but a quick check at either llli.org (la leche league international) or askdrsears.com will give you the information you need.
Another thing is, you're being really wise to prioritize his cues over anything else. Keep on doing this: if he acts hungry, feed him.
He's probably spitting up much less than you think (at least relatively speaking). And that stuff spreads a lot. Pour a tablespoon of milk from the fridge on a blanket and you'll see how big it looks on the fabric. And all babies spit up. It might not be too much feeding, but how you pick him up, lay him down or hold him afterwards. Sometimes when I pick my daughter up after a feeding (even 1/2 hr later), I accidentally squeeze her belly in just the right way, and I've got to sponge off a big puddle on the floor.
If it were me, I'd stop with the bottle feeding and focus on the breast. The nipple on the bottle is too easy for him (I don't care what the manufacturer says). So he's getting used to that and his latch on your nipple is staying shallow.
WHat i did was, I let my baby find her own way. I gave her a minute or two, and even though her first attempt was a shallow latch - just my nipple - that was just her starting point. It was part of her process. Her real intention was to go from there to get a big mouthful. Wait and see if this is what your son does. If not, tickle above his mouth and wait till he opens really wide, like a yawn, then swiftly but gently move him onto the breast. Let his head move down, like taking a big bite of sandwich. Don't work the jaw, but the face (same as when you tkae that bite of sandwich.) You can help him and encourage that good, big latch by gently compressing your breast with one hand to make it flat parallel to his mouth. Just like you eat a sandwich horizontally, not vertically relative to your mouth :)
You can also fix the lip by gently going in a bit with one finger and dragging it out as you withdraw your finger.
Hope this helps. Keep trying! My daughter and I had way worse issues, we kept trying and once we learned the right tricks, we settled right in and it's SO easy to breastfeed. I sleep really well overnight even though she wakes to feed. She sleeps in bed with us and I just lift up my shirt, she latches and I drift right back. COzy. DOn't get up during those cold winter nights to make formula!!!
L.