I was like that from the third week on. Dd didn't want to bother with my left one so after some struggle, I decided to give up and just feed from one breast only. As it was in the very beginning of my nursing I had no problems with supply, I always offered the right one only and let dd work on supply. The left one dried up eventually, I didn't pump and I was OK, no engorgement there. I guess, it is because I had so little milk in there, dd has never really latched on well on that one. To tell the truth, after two years, I still occasionally leak from that dried up breast, maybe once in six months, can you believe it? So this is my experience.
Now, to answer your question, you can absolutely do it with one breast only. And I can understand why you don't want to bother with offering and pumping. If you keep offering one breast only, you may need to offer it initially a little more often if possible, treat it lilke a growth spurt. Eventually your body will start producing an adequate amount of milk in your left one and everything will be fine.
The downside is that you will end up lopsided but I figured that I didn't really care that much about it. At first I put a pad in my left bra but now I don't even remember to do this. My left boob is a pre-pregnancy size while the right one is hmmm, bigger LOL. I think I would be horrified ten years ago to see myself so careless about it but now I just don't bother. And I am a person that likes to be well put together, if you kwim.
Good luck with your one sided nursing, it is not so unusual, there are plenty of other moms doing it. I know that many extended nursing moms sooner or later end up with one preferred breast only.