To change a document library from classic to modern experience.
Go to library settings->advanced settings->scroll down, “List experience” section, select “New experience” for “Display this list using the new or classic experience?”.
To change the SharePoint site from classic experience to modern experience, we can connect the classic SharePoint site to office 365 group.
Go to Gear->select “connect to new office 365 group”.
Then it will create a modern home page for the classic site collection.
It is possible to go back to classic experience after switching the site to modern experience.
You could go to the site pages library and click the original home page.
The Home.aspx is the original home page.
The Home(1).aspx is the modern home page for the classic site collection.