If he isn't interested in feeding I would try holding him off, but I would shorten his A time accordingly. On days where we get off, I count her awake time from end of sleep/nap to start of next nap, even if E is in between so it may be more AEAS or even AES (as long as he is not being fed to sleep). For example if you have 1.5 hours A time if he wakes at 6:15 he should be asleep at 7:45. It messes up the "routine" but he will be better rested that way.
Looks like your nap times have been good!
HTH,
Jodi