I completely understand you being hesitant to wake a sick baby, and you should be. When my kids are sick, I let them sleep until they're ready to wake. However once he is healthy do him, and yourself, a favour and get him back into his routine.
When I say pick a time, I'm referring to the A time. I do think part of the problem is that the days are often inconsistent from one day to the next. It's ok if the A times are different (eg. DS2 is awake for 3hrs 15 mins in the morning but only 2.5 in the afternoon) but it should pretty much always be the same each day, baring illness etc. If your LO is routinely getting the stimulation he needs and the right amount of sleep, he's going to fall into his rhythm naturally and you'll likely start getting much better naps and night times.