Using SharePoint out of the box approach:
You can use "Text" web part with headings (heading 1, heading 2, etc.) or sections headings which gives you direct hyperlink to heading/section.
Then you can use these hyperlinks in another text web part or quick links web part to create a table of contents on a page. To create links at right side of page, use Vertical section.
However, this will not create a fixed/sticky navigation and navigation will be hidden when you scroll down to the bottom of longer page.
Reference: What is an Anchor Link in SharePoint Online?
Using SharePoint Framework (SPFx) approach:
If you want fixed/sticky header even when you scroll down on longer page, there are few custom SPFx solutions available to achieve your requirements.
Check below links for detailed information and code. You can use these solutions as it is or you can enhance them as per your requirements:
Table of Contents Web Part

Table Of Contents for SharePoint

Page Navigator

Page Sections Navigation