Is it possible to have web parts on application pages. In other words having a web parts page in the _layouts catalog? Have tried inheriting from webpartpage, no errors but nothing happens when clicking add web part to zone.
You can certainly have a web part statically in an application page. After all, it is just a control with extra behaviour to support browser-based customization.
Whether you can have a web part zone is another matter. The application pages don't support customization (in the sense of un-ghosting), so I don't think they will have the wiring to persist your web part configuration either.
You need to have an SPWebPartManager in your page - this will usually come from the master page. But I don't think it will be able to do anything.
Application Pages don't support "Edit" mode, so you wouldn't be able to add web parts to a Zone anyway. You can freely add static web parts, however, as @SPDoctor said, they are just Controls.