We've been going through this off and on for a looooong time. Tyler will have a few days in a row where he naps well and falls asleep quickly at night, then a few days where it goes all wacky. Since I never know what he'll need that particular day, I put him down for his nap at the same time each day. If he naps, we do wake him (we're still tinkering with how long to let him sleep, I'm thinking no more than 1 hr might be about right). If he doesn't, then he has quiet time up in his room reading books or talking to himself. I plan to stick with quiet time as long as humanly possible...even on days that he doesn't need a nap, I've found that he still needs some quiet time alone to recharge.
Like Emma said, you might think about waking your LO from his nap earlier...4:30 is pretty late. This is how our day looks:
730 - wake
1 - nap/quiet time
3 - wake (if he slept)
830 - bed (nap days...if no nap, 730)