I would highly recomend a change in routine. 11pm is way to late for bedtime and having him sleep all morning is not healthy for his development. I say this because we need to teach our children how to live in the adult world. Starting her on a patern of late nights and mornings is going to cause many problems in the future. Added to that that by 11pm babies are overtired and do not sleep well. When they don't sleep well they don't grow as children do the most growing in their sleep. At 4 months old bedtime should be 7-7:30 at the latest. Wake time 7-7:30 this way they get all the high quality sleep and daytime learning they need. Your LO is inconsolable die to tiredness.
The question is id the 11pm bedtime simply a convinence for you? I know I wish I could sleep to 11 in the morning like I used to
I recomend you try moving bedtime to 7-7:30 and match his routine to meet that. If you requier more sleep in the am do what I do
I go for a nap when my DD does at 8:30-9 am when I can. Its an ajustment, but its worth it.