That sounds like a tough decision. For me I think it would depend heavily on how much he enjoys the program. If it's very similar to what's been going on during the school year, you can get an idea of that. Hopefully he enjoys it enough that that he would prefer to be there than to be at home. If it's not excited about the school, then anything to pressure him to go more could be very counter-productive. You don't want the poor kid hating school before he even gets to first grade.
The most important thing he needs to be doing is running around like a maniac and burning off some of that boy energy. If the summer school means a lot of sitting still and being quiet, it could make him quite miserable. If they have things like centers where he can build castles, have pretend play, and spend LOTS of time with free play with the other kids, especially outside, then that sounds pretty cool. (Scratched knees, mosquito bites, and a little sunburn means he's having a good summer.)
I wonder if the "developmentally delayed" refers to his development academically, socially, or emotionally. I think he's a bit young to be pushing too much of an academic agenda on him. Make sure they won't be drilling him on flashcards, numbers and letters all summer.
If he doesn't do the summer school program are they likely to hold him back a year?
It also depends on what other plans you had for the kids this summer. If you've found a great summer camp then maybe you can explain to the teachers that you think he will get many of the same benefits at a camp. Or even if you keep the kids at home and just do lots of activities with them ("Camp Mom" I've heard it called) with trips to libraries, nature centers, museums, etc. Doing that with 3 boys would be guarateed to wear you out, though.
And I certainly would make sure that any summer school wouldn't keep you from taking them on a vacation somewhere. If you want to take them somewhere for vacation, make sure he won't be penalized for being out that time.
Good luck.