You say he gets more worked up when you stay with him, do you talk at all while you're there? are you making visual contact? You might want to try staying there with him reassuring him but telling it's nighty night, i want to go to sleep and pretend you're asleep in a mattress next to his bed. No talking, you ahve to be BORING, you just need to let him know it's ok and you're by his side but is no chatting time. Everytime he leaves his bed or tries to, you put him down, shsh him or pat him and off to "pretend to sleep" you go. It might be exhausting the first night and you and him might not sleep a blink but you need to establish the point that he's sleeping there at night time. The first night is the hardest and sometimes we give up cause we're afraid lo won't sleep enough, but just one night of "permission" is enough for them to start a bad pattern.
HTH and remember is just a phase till he gets used to his new environment. Good luck!