It's not too-too much milk, advice in most countries is 16 ounces from this age, but it's two extra "meals" that he should start weaning off of at this age. From one, solids is the main form of nutrition and milk in large volumes should be avoided; the intention is to have LOs drink milk as a "beverage" that they have through the day and/or with meals in smaller quantities. So instead of having milk 2x per day and in volumes where he is filling his tummy and using it as a large calorie meal, it would be good to start moving him towards having a breakfast of solid food with a few ounces of milk as a drink. He can have a few more with his morning snack, a few more with lunch, some for afternoon snack, with dinner, and a few before bed...would all add up to his 16-20 ox per day, iyswim. It isn't necessarily something that he will take to in a couple of days, but might take a few weeks to transition him to the idea of it over a couple of weeks.
HTH.