We have also struggled with some very early mornings off and on. When he wakes early I go to him and pick him up if he wants, and tell him it is still night time and time to go to sleep. He will continue to ask to go downstairs, or for milk, or daddy, etc.. I continue to tell him no, and that it is time to go to sleep, until he eventually goes back into his crib. Some days he does go back to sleep, and others I have to go back in again, but we do not leave his room until 6 at the earliest.
I think you are doing the right thing in not leaving the room. I think if you leave their room with them that early it just reinforces it.