My son just turned 4 and has never really liked eating (I guess he liked baby food okay, as a baby, but not table food). I only give him snacks if he eats his 3 meals/day well. Otherwise, he gets no snacks. I don't necessarily tell him this, though I might sometimes. I know, it seems kind of opposite- that if he didn't eat a meal, he should have snacks or there's nothing in his belly. If he gets snacks, though, he much less likely to eat his meals. I know a lot of people give their kids snacks, but mine just wouldn't eat the good stuff at the table if he got to snack all the time. He gets breakfast (generally 6 oz yogurt, and 1/2 piece toast or something) and lunch (I'm lucky if he eats half a sandwhich, usually a whole piece of fruit, though, and Cheetos or something after that, only if he's eaten his sandwich) and then dinner (his worst meal of the day- sometimes it's 2 bites or so). I'm not quite sure how he survives on such little food or how he grows, but he's always been at the top of the chart for height and 50% for weight. He's about 38 lbs now, which is definitely avg among his friends.
So, I say the pedi is right and to cut the snacks out. My son does turn into a crying, raving lunatic when he's hungry, but if he is hungry, then he gets a meal, not snacks. If your son doesn't eat the meal save it for later and try again (same or next day).
I know it's hard, trust me- but he'll always be a snacker and refuse the food that's best for him if you don't do it. The sooner he gets used to it, the better. My sister had the pickiest eater and I remember as a pre or young teen he was hiding cereal and crackers under his bed so he could snack when he wanted, because she had put her foot down. She waited way too long to enforce the "food rules" and suggests to everyone she knows to "make them eat what you eat- they get that or nothing". It went better with her second kid.
I make my son try at least one or two bites of whatever is on his plate and if he doesn't eat his food (like at dinner) we tell him his "belly will be mad at him and hurt" when he goes to bed. Your son is old enough to understand this- make it their decision and he'll get it eventually. You can't force food on them. All you can do it lay down the rules.
I really wish my son liked to eat, but he doesn't. He has rarely, in his 4 years of life, said "I'm hungry"- those usually only happen in growth spurt times- maybe 2x/year and for a week. Love that time, but it's rare.
Sorry I don't have more encouraging news.