I have several site collections in SharePoint online tenant and few hundred documents uploaded in them. When I go to Yammer and try to post and click on "SharePoint" icon, I am not able to get those documents in dialog box. However, when I click on "Recent" tab, I am able to see some recently uploaded documents on SharePoint. Otherwise only documents I can see is of "One Drive" and not SharePoint.
EDIT: Adding screenshot below. This is the first screen which I get when I click on "Select a file on SharePoint". These folders are from Onedrive. There is no option to enter SharePoint URL too. When I click on "Recent" from left bar, it shows document from SharePoint which is uploaded recently.