I do not want to do a whole migration as to avoid overwriting the current SharePoint Online configuration.
Is there a way to migrate only the libraries from the on-premise SharePoint Server to the new Online one?
Yes, you can using the 3rd party tool i.e. Sharegate or metalogix etc.
another workaround which depend upon the migration requirement, size of libraries and number of documents. You can try to manual migration, for which.
here is another example for manual migration if you want do some scripting:HOW TO MIGRATE AN ON-PREMISE SHAREPOINT DOCUMENT LIBRARY TO SHAREPOINT ONLINE WITH POWERSHELL