It's totally normal for toddlers to do that. He's just learning about his environment and the behavior will stop once he's finished with this learning stage. By telling him "no" and giving him a reaction, you could very well be prolonging this stage so I recommend just ignoring it.
A great example I have is of the wood chips in playgrounds. I see parents constantly telling their children to stop putting them in their mouths from a very young age all the way through to 3-4 years old. I, however, let both my girls put wood chips in their mouths (with no big reaction from me - even though it kinda freaks me out that they've probably been treated with goodness knows what...) from the beginning. They did it a little during the first 3-4 times we went to the park and then stopped. They satisfied their curiosity about what wood chips taste/feel like in their mouths and didn't want to do it anymore.
I personally draw the line at garbage (cigarette butts, food waste, packaging that isn't ours and the like) and excrement and things like bird feathers and of course, poisonous plants. Pretty much everything else, I have let my children explore with all their senses and the "tasting" stage passed very quickly for my girls, right around 1 year of age, though mine both started walking at 9 months so they were early toddlers :)
This will pass - best wishes to you!
~B.