I don't think you are being unreasonable, I try to take ds out often, just to make sure he's used to it if nothing else.
I hope this will be some small comfort to you - ds went through a very similar phase at this age. I think it was to do with his desire to do things being stronger than his ability. He wanted to walk everywhere, but he couldn't, so I think he was frustrated and felt out of control. Now he can (tentatively) toddle about holding onto things, he's much happier.
I do things like take him out of the pram/highchair whenever he didn't need to be in, b/c he gets fed up in there, sit him on your lap, or on the floor if it's clean/safe enough, and save the pram/chair for when you need your hands free. Also, can he nap in the pram? If you go out during one of his nap times, and he takes it in the pram, then you get a good chunk of time to wander around the shops. As long as you don't do this every day (I do it about once a week), then it shouldn't upset his routine. Then, when he wakes up is the time to go and eat. We always try to eat together, with him in the highchair, it takes a bit longer b/c you're faffing about with his food a lot, but I think it's good for him to learn that families eat together, and it saves him getting bored whilst you eat.
Can you save certain toys just for when you are out? Or even give some forbidden toys (like your mobile phone, etc). This might help keep his interest for longer.
I also take him for a walk around the restaurant while we are waiting for our meals (either carrying or him toddling holding onto my hands). This helps keep his interest, and it helps to make the restaurant not such an unfamiliar environment to him so he relaxes a bit. Point things out and tell him about things, and if people look friendly, say hello.
HTH, hopefully it's just a phase