Sadly they aren't always predictable and babies have their own patterns but roughly roughly:
Common times for growth spurts are during the first few days at home and around 7-10 days, 2-3 weeks, 4-6 weeks, (so yes, that does pretty much mean a permanent growth spurt for some for the first couple of months!!)
Then maybe 3 months, 4 months, 6 months and 9 months (more or less), a year.
Then perhaps every few months during the toddler years (around 2 is often reported) and periodically thereafter on through the teenage years.
It may not necessarily precede a period of physical growth but also a developmental advance.
As you say - they can take you by surprise but that can happen anyway even if you do know the 'schedule'. The best thing is just to keep an eye on baby's cues as you are. If you stick too much by the calendar you could drive yourself loopy - some babies have growth spurts that are fairly subtle, others have a crazy week.