It really depends on how your SharePoint site is branded. An even more necessary dependency is weather or not it should be flexible for both desktop and mobile-device diplay. In which case I recomment making 2 stylesheets or use CSS Media Query (if you're a rather experienced SharePoint brander).
I usually disregard the default SharePoint masterpages and CSS files, as they have caused me WAY too much trouble for minor quite needless element modifications.
EM and % (Percentage) is the primary font-size handlers to use when you need to optimize the SharePoint site for mobile-devices, while PX or PT are more suitable for desktop display only.
EM is in my experience a bit hard to fool around with when toying with SharePoint branding, as this font-size is rather annoying as it double scales and you end up in many situations where you'll have you write font-size: X.[1-9]EM;. With that said, EM is preferred for mobile-device optimized SharePoint sites. Pixels and Points is honestly a NO-GO on mobile-devices.
When I change the font-size on my clients SharePoint sites I tend to only override 3 classes.
body, .ms-WPBody, .ms-WPBody TD
There is no need what so ever to change it on every single container which has its own set font-size as it usually contains child elements which has its own anyways, which is either an inheritance of body or its content resides within a container inheriting from .ms-WPBody in most cases.