If you need to restrict users from accessing SharePoint site other than the document library, consider using custom unique permission to grant the users direct access to this library.
This is what users can see when we put a document library web part on a modern page. It hides several buttons on the menu, but users still have access to the full menu in the library by clicking “See all”.
As far as I know, we cannot hide the buttons from the menu since modern experience doesn’t accept customization.
- Users need View Application Pages permission to access the All Items view of the document library, which will give them access to
the Switch view options.
- To hide Power Apps and Power Automate buttons from the menu, you need
to disable these features on the site level.
In conclusion, even users with View Only permission have access to the Switch view options, Filters, Details and Power Automated buttons with full functions enabled in this site.