Lots of things going on there! The milestone of rolling over is wreaking havoc on his sleep. This is normal. They get used to it and get back to napping as they were before within a couple of weeks - if you can try keeping him on his routine (which, I can see, is very difficult, especially with another child).
Does your lo have a wedge in his crib (or is the 'upper' portion of his crib raised), so that he is sleeping at an incline? This should help with the reflux (depending on the severity), even if he is on his back. If that can help the reflux while he is on his back, he will get used to sleeping on his back (many babies have the opposite issue - they need to get used to sleeping on their tummies once they learn to roll over, so it shouldn't be a problem). If putting him on his side gets him to sleep, I suggest doing that and then going in to put him on his tummy once he is asleep.
The other issue is that it doesn't sound as if your lo is sleeping enough hours if you are forgoing the nap because he is crying. This only makes him more tired and could be affecting his eating (though I'm not going to say that's the issue with the solids since I don't know). You really do need to put in the time to help your lo learn to sleep, either with PU/PD or pat/shush (if that still works for you). Since you have another child, is there any way you can do some training on the weekend when someone else can help you, like a dp or family member?
Regarding solids, maybe he has a physical issue with the gagging. Can you have his pedi check the back of his throat? His actions say he is ready for solids but the gagging makes me wonder. Tracy does say it takes a while for a baby to get used to this new feeling of the food in the mouth. Is there a way to make the cereal more watery (add less cereal) and be careful about how far back it goes? I'm really not sure about that issue... I'd suggest posting that particular question in the Eating forum for a better response.
HTH