It sounds like a complex problem you have. First of all, your son is old enough to know what he's doing and very well may be working you. Yet, I'm hearing evidence of a life transition that could be a justifiable root cause; the move from a small apartment to a larger house. I'm wondering if he manifests his discomfort in other ways? Is he anxious at school? has his teacher noticed anything?
My first thought is that at 6.5 years old, he should be able to articulate his feelings and pehaps even strategize with you for more reasonable soloutions. I will often engage my 5 & 7 year old in conversation about behaviors that need to change and why.
"It's important that you learn to do --x-- instead of --y--. What do you think would help you be able to do that?"
My kids are fully aware that when they go to sleep is when Mom & Dad do their "homework". So we simply cannot stay with them all night.
Also, I will say, I am an advocate of using sibling responsibilty as a tool for personal growth. Stay firm on keeping him in his room for bed time. He needs to "set a good example for his brother". But then ask him, "where do I need to be tonight until you go to sleep?" each night, urge him to allow you to be further away.
as for the middle of the night visits; perhaps you can get him to agree to move out of your bed in stages.
Stage 1 - he can come to the room, but has to stay in a rocker/chair or in a sleeping bag on the floor.
Stage 2 - he can come in and talk to you but must return to the sleeping bag in the hallway.
Eventually, his bed becomes the much better option and he will return to it.
Finally, I know that I once saw a 2-way baby monitor and thought it was a great idea. But you could use a set of walkie talkies for the same purpose. This way he could stay in his room but still be able to talk to you wherever you may be in the house.
Bottom line, I think he may just need some time to adjust to his new spacial/spacious surroundings. Eventually, he will not need your physical closeness to feel connected.
Good Luck
Working-married mom of 3 energetic children ages 7, 5 and 2.75.