You can try the approach below, but bear in mind that SharePoint Online:
- Requires authentication. Simply calling Invoke-WebRequest will just return an access denied page.
- Modern pages use lots of client scripts that run after the page is served to the client. These scripts first pull in data and then render the page contents. Since PowerShell will simply get the initial bear-to-bone page, you won't get what you probably expect to get.
Install prerequisite PowerShell module
Install-Module PnP.PowerShell -Scope CurrentUser
Authenticate and perform a query to a page of your choice
Connect-PnPOnline -url "https://contoso.sharepoint.com" -UseWebLogin
$HTML_Page_Content = Invoke-PnPSPRestMethod -url /_layouts/15/viewlsts.aspx -Raw
Invoke-PnPSPRestMethod can be used together with the
-Raw switch to get contents of any SharePoint page.
Invoke-PnPSPRestMethod is not restricted to calling only endpoints located under
So, instead of doing what you are doing, I suggest simply getting what you want from SharePoint by using PowerShell PnP commandlets.
For example, to get a list of lists and libraries, simply use:
No need to suffer through the unnecessary web scraping.