Of course it's fine! I doubt there are many babies who don't take naps in their stroller or pram! Babies need fresh air and sunlight and to look around and see the world just as much as moms (and dads!) need to get out for the same!
Independent sleeping isn't that a baby needs to be at home in their bedroom in their own bed three or four naps a day plus every night - that's not reasonable for a family (or baby) at all! As PP said, if your LO can't sleep in their cot at all, and you are putting him/her in the stroller at every sleep time and walking to GET them asleep and have to keep walking or baby wakes up, then that's different! Using walks as props because your baby needs that movement and motion to get to sleep in the first place would be something you would be encouraged to break. But, allowing baby's nap to be in their stroller while you are going out and doing things (like exercising) is totally fine!! And, you've got to make sure that your LO is flexible as well, and willing to take a sleep in their stroller when you need them to, because over the next months you will have many times when there's no option but for that to happen, because you NEED to go out at nap time!
I always found different benefits to the different times of day - if we went out first thing in the morning, I felt more "accomplished" - like I had gotten something done in the day and started my day off right, and we'd both had the fresh air that our lungs needed after a long night trapped in the house. But, I also found that on days when we'd had a rough morning and things had all gone haywire, it was a good time to go for a nap late afternoon to get me calmed and relaxed. And of course, the weather made the final decision!