Yes, that makes sense, but sometimes the comfort sucking actually gets out more of the fatty hindmilk than the "nutritive" does, if that makes any sense?
If it were me, I'd switch to the other side once he started getting sleepy or do some breast compressions to try to keep him a bit more interested.
Has he been checked for anything (like a tongue tie) that could be affecting milk transfer?