Do I need to install the CU on both WFEs (we have 2 wfe) and CA Server?
I installed it on CA server and ran the Config wizard and it's complaining about the hotfixes are not available on WFEs.
You need to install the update on all servers. Then run the configuration wizard on each server after it has been installed on all of them. You can install it on all of them at the same time, however you can only run the config wizard on once at a time.