Well.... in an ideal world, sure, I'd go all-organic 'cause it would be available and affordable.
In real life, though, I got my frozen veggies, even though Super Baby Food said not to use frozen, then cook them and freeze them again, and we were all fine. Now our local grocery chain carries their own brand of organic foods, including frozen veggies, so I get them when I can, or I get fresh organic when I can, and I try not to sweat it too much.
Yogurt - I got around to doing that around 6-7 months, maybe 8 with Natalie? She started solids a bit later than Josie. I used Stonyfield Farms organic whole milk - SOOO rich and creamy!!!! YUM!!!