As I thought, I layed him down at 9:00 and he was fussing for me. I went and got him, brought him downstairs, he drank 2 sips of milk. 5 minutes later, I put him to bed and he was out.
I need to start limiting his nap, because I think he plays catch up. It is such so nice to have that break during the day since I am pregnant, but then again, I like some a lone time at night also.