admittedly, i know nothing about the weather where you are... is it likely to be cold & miserable in november? if not, perhaps a park? you could do an entire picnic theme. hold it mid-morning or later after naptime, and you won't have to do a full-on meal, just snacks.
since you won't be at home, cupcakes will be way easier than a single large cake. there are all sorts of nifty designs you could do.
and since you are having performance anxiety
i'd suggest inviting a good friend or relative (whether she has a child or not) to come & help with all the party stuff & keeping you sane.