Have you tried going back to 3h? Or even extending it more slowly, so trying 3h10-15m? What happens? I ask because ebf babies often need more frequent feeds than bottle fed babies because breastmilk is so easily digested, so they don't move to a 3.5h or a 4h feeding routine as quickly. Dylan was on a 3h routine until 4.5mo, then a 3.5h routine until about 5.5mo when we started solids, and that's fairly normal in an ebf lo.
It could be either of the things you mentioned - he could be really hungry by the time you feed, and is impatient for the letdown so fussing because there's no milk there right away. He could also be fussing because of a stronger letdown/faster flow because of more milk. Or on the opposite end, he could be fussing because the sudden change in frequency has decreased your milk supply a bit.
When you say fussing, what do you mean? Is he bobbing on and off, crying, whining, squirming, etc.? I would try feeding him more frequently (ie back to 3h or just a bit more) and see if that helps.