There are many ways that you can go about provisioning these artifacts to your site(s). It comes down to a declarative (CAML-based) vs imperative (through code, like PowerShell) apprach. Heck, 3 of the 4 things you described are able to be accomplished through the SharePoint User Interface by a Site Owner...
The first, and probably most important, question is, do you need this provisioning to be repeatable across various site(s) and/or farms (i.e. Development, QA, PRODUCTION)? For example, if you created these via PowerShell (or any of the Object Models, such as the server-side object model), you don't have control over the IDs used, and can have limited control around the internal naming of objects (well, without writing the logic to provision, then renaming the display name). This can make it hard to write code later on top of these artifacts and have it work consistently between environments without full content database refreshes. In general, if you need this to be repeatable across various sites/farms, go with the declarative approach (CAML-based). There are SPI templates in Visual Studio 2010 (and also the Beta of Visual Studio vNext) that help greatly here. If CAML is too complicated for you, go with the Server Side APIs and a FeatureActivated event receiver. If its a one time provisioning, you can be fine with PowerShell, but any powershell you write, unless you've downloaded some SharePoint specific cmdlets from the web, will be basically the same code you'd write in a FeatureActivate event receiver. The bonus with Feature Receiver is that you get all the future "upgrade" plumbing for free. Here are some other considerations for choosing declarative vs imperative: https://www.nothingbutsharepoint.com/sites/devwiki/Playbook/Pages/SharePoint%20Declarative%20(CAML)%20vs%20Imperative%20(Object%20Model)%20Provisioning.aspx
I'll add to the above, that sometimes the situation/artifact dictates which approach you are allowed to take. For example, sometimes environments will mandate that all SharePoint customization be made through the UI or through CAML. In this case, you are forced into creating things using a declarative approach. Also, not sure you can create a List Template from code, so you may be forced to use CAML here. There is no clear cut approach here. Microsoft, for example, in many of there features leverage a CAML approach, but in some like the WikiPageHomePage feature, it's mostly a code based approach. It just comes down to requirements and constraints.
When determining whether these things should be in the same feature, or different features, think about future needs or possibilities and the scope of the objects you define. For example, Site Columns, Content Types, and List Templates are generally provisioned at the Site Collection level (Feature scope = "Site") but can be provisioned at the Site level (Feature scope = "Web"). This is so that they can be used in any subsite of the site collection, which is usually what you want. List Instances are generally provisioned via a Web scoped feature. Refer to this article about the different elements and feature scope: http://msdn.microsoft.com/en-us/library/ms454835.aspx (which may not be entirely accurate as I think Web Templates are only available at Site Collection scope...at least I received an error from Visual Studio 2010 the other day along these lines, but truthfully haven't revisted to confirm this).
If you are developing for a company and there are future plans to customize SharePoint more into the future, you might create a separate feature for each of these types of artifacts: "MyCompany.SharePoint.SiteColumns", "MyCompany.SharePoint.ContentTypes", "MyCompany.SharePoint.ListTemplates", and "MyCompany.SharePoint.ListInstances". Using ActivationDependency elements, you can create references from one feature to another. You might, for example, create a consolidated feature that simply has 4 feature activation dependencies, one to each of the 4 mentioned above. This is how Microsoft does it with some of the publishing features/functionality. Just understand how Activation Dependencies work, at first, it's tricky with regards to automatic activation with Hidden vs Visible features...
Breaking out your features in this way will all you to upgrade each independent of the others. This said, if this is your first forray into SharePoint Development and customization, there is nothing technically wrong for lumping all these into a single feature. Just know that this may make it harder for you to change things down the line...
Installing and activating your features is a little different depending on whether you are in the Sandbox or Farm solution. I'll assume, based on the context of your original posting, that you are in a Farm Solution. In this case refer to these for guidance:
Installation and Deployment of Farm Solutions in SharePoint 2010: http://msdn.microsoft.com/en-us/library/aa544500.aspx
Installing or Uninstalling Features: http://msdn.microsoft.com/en-us/library/ms442691.aspx
In general you must:
1. Add Solution.
2. Activate Solution
3. Install Feature
4. Activate Feature
All this can be done via powershell and I'd recommend PowerShell for an automated/consistent deployment...Or you can do portions of it through the UI (activating).
In summary, I'd avoid PowerShell approach for creating the artifacts; it's likely whatever code you write for PowerShell would be very similar to that needed for a feature. I'd only be considering Server Side Object Model + FeatureActivated event or a CAML based approach. If you are not comfortable with CAML (declarative) approach, go with Feature Activated event + code. If you can invest the time in CAML approach, this will be beneficial to you in the long run (assuming you'll be creating other SharePoint artifacts in the future). Just be warned that the CAML approach can be hard to learn/troubleshoot, especially if you are new to it. This page appears to accomplish all the things you are trying to do using a declarative approach and should be a good starting point: http://msdn.microsoft.com/en-us/library/ms466023.aspx
Hope this helps!