Is it possible that he's going through a growth spurt? It has been going on for a week.
Yes, there's a big one at 4 months
Supplementing with formula might make it longer too - your body is not getting the message that it needs to make more milk if you're giving him milk some other way. What's happening is absolutely normal. Your supply hasn't dropped necessarily, his needs are higher and he's feeding more to stimulate more production so you're both doing your jobs perfectly, its just difficult when things have been going to well.
What do I do?
Drink lots of water yourself
Rest in the afternoon as evening is your lowest supply time
Breast compressions when LO is frustrated with not getting much milk
Sometimes the stimulation of hand expressing helps more than a pump if you're not getting much - perhaps give that a go.
WRT solids - they're lower in calories than milk, so its not ideal if he's really hungry. Recommendations differ between 4-6 months depending on the country, etc. but its worth making sure he can sit unsupported and has some interest. Initially, LO won't eat much in the way of solids anyway.