I want to Set the permission to enable the users in a particular group to Only view the Documents in a specific library and anything beside that should not be accessible to them (Site Contents and Other Sharepoint Basic Elements. How to do that ?
Detach the permission of said files and apply security of View only for that specific group of users.
However by experience I can attest it is going to be frustrating for the View only users if they click on anything else (including deleting portions of the URL to go 'up one level') to get access denied.
You should architect a site that is fairly accessible by anybody (because there is a generic description/nothing confidential) and then detach that said document library and detach single files.