Ok, first of all don't panic! 12 weeks is a long time away for your baby and he won't be on the same routine by then anyway! If he isn't happy eating first thing can whoever he is going to while you work feed him when you get there?
The routines in the book are only suggestions, if your baby is on a different routine time wise it doesn't matter as the order of events eat/activity/sleep is the important bit. Can you post your routine as it is now and we will see if we can help!
Not all babies take to the DF, you can try it a bit earlier or later and see if you catch him at a different part of his sleep cycle and see if that helps. It was only 3 hours since his last feed though and at 4 months he really needs 4 hours so he may just not have been hungry enough.
Laura