In SharePoint online modern experience, you can use SPFx extensions to create custom navigation - Specifically Application Customizer.
You can find more information & sample codes to get started with at:
SPFx Side Navigation Project for Modern Sites
Navigation Bar In SPFx With Current User Greetings
I will recommend you to check SharePoint out of the ...
You can edit the left navigation for your lists. Just like this doc said.
And in Team site, a newly created list will have a navigation link created by default on the left navigation. Are you handling a communication site? Would you like to say a bit more about your current site and your goal?
SharePoint Groups are not supported for audience targeting in SharePoint online.
Azure Active Directory (Azure AD) groups (including Security groups and Microsoft 365 groups) are supported.
Source: Target navigation, news, and files to specific audiences
If you want Microsoft to allow using SharePoint groups, considering voting on below related UserVoice: