I prioritize what we buy organic, based on how much we consume the food (milk, for example, every day) and whether it's on the "organic dirty dozen" list, which you can google for or here's a link: http://www.msnbc.msn.com/id/13737389/page/2/
Unfortunately, I don't live in an area that's great for grocery store choices nearby, and don't have the time to drive all the way to the less expensive stores every week. All the stores mentioned already seem to have decent prices. Whole Foods would bankrupt us although it's nearby and I buy some stuff there - but we eat tons of fresh produce, which is expensive. We stock up when we do see deals, and we also subscribe to one of those organic farmer things thru our PTO (http://www.tcforganics.net/) where they deliver a box of mixed produce every couple of weeks. (Which is fun, because then you can play the "what the heck is this root vegetable and how do I cook it" game!)
So I guess, short answer - spend more on the things that have the biggest impact, and generally buy less processed foods and convenience foods. I don't have any money-saving secrets, though!