Hello! I don't have any experience with milk allergies, but I thot I'd just share that my DD didn't take to rice cereal at all. Just wasn't interested and although it's so hard to tell that early it seemed to give her tummy troubles. If he likes oat, tho, I'd just stick with that for now and try rice in a month or so. I've also heard that bannanas can be troublesome for the tummy too unless they're super ripe. Have you tried peaches? That was favorite in our house at that age. I don't think it's unusual to not be too interested in solids at 5 months. Maybe just stick with what is working for now and add in a few new things at a time in a few weeks when he's more used to digesting the solids.
Good luck!