Standard SharePoint web parts on one page
I would add the webparts by code using a feature eventreceiver. Create a SharePoint solution in Visual Studio, add a feature and on the "onActivated" event of the feature you add the web parts to the page by code.
If needed a user/site admin can still remove the webparts
Standard SharePoint web parts on all pages
Create a SharePoint solution with a master page. You can put web parts on your master page by adding the XML configuration xml on the master page. It's a huge topic, google it for some tutorials. (you can also do this by using SharePoint Designer)
Custom functionality on all the pages
Create a SharePoint solution in Visual Studio. Create a master page. Create your custom functionality in a user control. Add this usercontrol to the master pages. I would recommend putting the master page and the user controls in the same solution. If the user controls referenced in the master page are not available you get a nasty error.
Site admins and users cannot remove user controls from their (master) pages.
I've done this for custom footers and