N.W.
I heard a baker who uses silicone bakeware say that you should not use cooking spray on the silicone cookware as it's naturally non-stick and the cooking spray makes a gummy mess that makes it harder to clean and this stuff you are having trouble removing is probably overspray from where you tried to spray the muffin holes. If you don't spray it should release the muffins just fine after you finally get all the non-stick spray off them. The non-stick spray puts a sticky gummy mess on them and defeats the purpose of the silicone which is in itself non-stick. I have all sorts of silicone bakeware and per instructions never use a cooking spray for any of it it is not designed for use with cooking spray.