I hope you'll go back and re-read Pam R.'s good response. Please talk with your son -- NOT when he is grumpy or has been told he must go somewhere and is peeved! Wait until he's mellowed out! He should be able to express, if you can ask in the right way, the "why" behind this. If it is all about gaming and videos, that's one issue. But there may be more going on: Maybe he feels that at the pool, he's simply bored because he doesn't really like swimming or even horsing around in water, and he doesn't like being in the heat out of the water (that's how I am -- the pool is the last place I want to be, frankly). Or he may feel secretly that he's not as good at swimming or tricks as the other kids -- have you observed closely when he's in the water to see if maybe he's holding back compared to other kids, even though he gets candy etc.? Or on other outings, he may get quickly overwhelmed if there is any kind of crowd; does he do better in very small groups or just with family present?
Regarding the pool, that is something I wouldn't force with him, but it sounds like you don't want him home alone which I sure understand; I'd just tell him, your sibling and I want to go and you can't stay home alone, so bring a book and I don't mind if you go read in the shade or go in the water. Maybe do twice a week at the pool rather than three times and then cheerily tell him, "You got through OUR pool time without sulking so today YOU get to choose what we all do as long as it's outside the house." Sounds fair to me.
What about in school? It's summer now, but during the school year, how much did you know about whether he interacted well there? Did he participate freely in class or did teachers have to "drag it out of him" to get him to participate? How was he about doing group projects, which are increasing by age 10? I would really want to know if the impulse to be alone and the tendency to check out or be irritable are also present in school; in scouts or church youth group or martial arts or sports or whatever else he does. Talking in detail and privately with his teachers, coaches, leaders, etc. could help you figure it out.
He's in camp next week - does he WANT to be there? Pay attention next week: Does he fuss and fume each day about being "forced" to go? Does he do that but then come home talking positively and happily about what went on that day at camp? Assess whether he's turning off the sulkiness at camp or if it seems to continue there. Don't be afraid to ask to talk to whatever counselor sees him most of the day -- by midweek the counselor may be able to tell you something useful, and you too can pay attention to how your son is before and after each camp day. If it's sleep-away camp, it's harder to gauge, but the counselors will know him better so don't hesitate to tell the director you want to talk to his counselor at the end of the week.
I would not just dismiss this as "Oh, he's an introvert, it's just the way he is." That can be the case, but I know introverted kids who are not grousy and grumpy and who don't sulk.
Is the "he always wants more game time" a summer phenomenon or does it go on during the school year as well? You mention sports practice but no other activities. During the school year, I might get him into a few more things outside school -- let him help choose so he feels an interest. Investigate what he likes best; does he really like art class at school? Find a summer AND school year extracurricular art class for fun. Or if he's into building and programming, get him into Lego Robotics (serious robotics, not little-kid Legos! Schools have chapters of this club and some libraries have robotics clubs or classes). And so on. You have to be proactive to find these things but they are out there.
You can't force him to be social but you can work on the sulkiness -- a trait which will be increasingly unappealing to teachers and other kids too. Getting him into things that really interest him but also involve other people and leaving the house could help. An engaged and interested kid won't be sulky at that time.
I hope you limit him to weekends only for game time in the school year. That can help a LOT. I have friends who do that with their sons and after some initial distress the sons quickly were so occupied with their other activities that they fell in line with the "games only on weekends" rule in the school year.