I think if it is pretty serious allergy, you would see within hours. If it is less serious like Max's intolerance was, I usually saw it the next day. It takes about a week to get allergens out of your milk if you're doing an elimination diet.
Fussy babies can sometimes not tolerate prenatal vitamins.