Right from the start, even before he totally understood us, we always made colors a regular part of our everyday conversation.
Do you want to wear the red shirt or the green shirt?
Look at this pink flower!
I like your blue hat.
Here is the red ball.
etc.
Whenever we talked about anything it was the color, then the name of the item.
The book "Brown Bear, Brown Bear What Do You See?" was also a big help - get the large one. It's so colorful and wonderful for reinforcing both colors and animal names.
Don't stress, just normalize it! The more he hears the colors and sees their associations, the more naturally he'll assimilate the knowledge - no drills or flashards needed! Use the world around you to engage his awareness...it's perfect, natural, no-stress learning.
Most of all, have fun! Kids can pick up on anxiety and may shut down for fear of being wrong or disappointing you and this can happen as early as toddlerhood. You definitely don't want this as it will result in a lifetime of frustration with learning and / or school.
Good luck!