What I want to achieve is to set a term that makes an item appear in a SharePoint list. Now I want only certain users to be able to use this term. All the other terms in the term set should be still available for other users.
No, this is not possible out of the box in SharePoint. If your users can see the term set, they can see all of the terms.
You can create term sets scoped to a site collection and put your secured terms in there. It's best to respect the security boundaries put in place by the platform rather than trying to "roll your own". If you want unsecured terms and secured terms mingled together in the same term set, no, you cannot do that.
For documents tagged with a term you could technically implement custom security via remote event receivers. But that is messy and I'd stay away from that. Like I said, respect the natural security boundaries.