Well that's just silly of course you don't need to have swimming lessons in order not to develop a water phobia! We stopped swimming lessons with Stan when he was 15 months (just because it was the end of the course) and since then have taken him to the pool ourselves every week. We splash about and have a great time, he is still SO confident in the water and has no problem going under, jumping in, diving off the side, lying on his back, kicking. We don't try to follow any structure except we do 15 mins with no arm bands at the beginning, then 15-20 mins with armbands, and another 5-10 with no armbands at the end. Practising things like climbing out of the pool unaided etc. I think we'll look for another class around the time he's 3ish.
It sounds like you've given it a fair try, and if he's not enjoying it then why persevere?