It is true that old fruit will attract the fruit flies into your house, but once they are there you will need to starve them to extinction before they will go away. So, now that you already have them, take out the garbage every day, and even with that, don't throw anything into it that they would eat (not just fruit, but vegetable peels, meat, anything organic). Put ALL of your fruit, vegetables, and anything sweet (cupcakes?) into the refrigerator. Clean off the counters and clean out the sink and the garbage disposal, and the floor and refrigerator front if it needs it. Whew - now that that is done, keep it this way for 3-7 days (lifespan). They will go away; if they don't, you are feeding them somewhere and should look around to see where they hang out. Once they are gone, you will probably be able to leave unripe or just ripe bananas and other fruit on the counter again, but keep an eye out for the buggers, and nip it in the bud if you see any!