I have a document library with ~20k Folders.
SharePoint Online is telling me, that the list threshold is reached even if I navigate into a folder which only 4 documents in it.
But Manage large lists and libraries in SharePoint Support Site tells, that we can use folders to organize the documents:
Using folders to organize You can use folders to organize data effectively, though you should be careful not to have a query on the folder return more than the List View Threshold. For more info, see Create a folder in a list or Create a folder in a document library.