We get lots of healthy fats....we're good. 
Ooookay.

Definitely good to make sure plenty of the good stuff: sat fats (excluding trans fats), coconut oil (very stable for high heat cookinf ans frying), butter, oily fish and dark meat chicken, olive oil (for room temp or lower heat cooking, never for frying), fish oil and flaxseed oil (never heat those), avocadoes. Lard, tallow (beef fat) and chicken and duck fat are great if you can get them from grassfed or pasteured sources, much less so from CAFOs. Sesame oil isn't bad; I do a blend of equal parts olive,nsesame, and coconut oil for most basic stuff i need to have a pourable oil for (coconut solidifies at 76F).
The less-good and even bad (inflammatory) stuff is the processed veg and seed oils, like corn, canola, and soybean. Way high in omega-6s and often oxidized by the time they're purchased.

So related to the OP, if there are dairy issues or if you just don't like the taste of full fat dairy like some people (my mom is one), those are other options for good sources of healthy fats. Otheriwse, no need to switch to lowfat.