have you tried him with some less stimulating toys? i was given a beautiful mamas and papas baby gym for my dd and she HATED it!!
for a start the mat was very padded so she couldn't roll around and all the brightly coloured stuff hanging over her head really annoyed her
she also hates her bouncy chair! it was just a plain one but she always got frustrated with it after a few minutes (i think she hated being strapped in). we just bought her a bumbo seat and she loves that
her favourite pastime is just lying on a rug on the floor with a couple of little toys that she can grab and put in her mouth! she spends ages playing herself (in addition to being entertained by myself and dh!), but i always make sure she knows i'm close by