I am able to reproduce this issue on our SharePoint site. Here are the results:
I am unable to see the Format this column option to SharePoint default "Title" column in the document library (Title column is added to view using classic experience view settings page):

I opened the Title column settings from list settings page, added the JSON formatting and saved the column settings. But when I opened the Title column settings again, the JSON was automatically removed from there:

I tried settings custom JSON column formatting for this field using PnP PowerShell as well but it didn't work.
Maybe there is some limitation or bug for the SharePoint default Title column in the document library (it is a site column inherited from "Item" content type). You can raise a support ticket with Microsoft directly and report this behavior at: Get M365 Support - Online
Meanwhile, you can try this workaround:
- Create a new column in your document library.
- Apply JSON column formatting for this new column. Use this JSON:
{
"$schema": "https://developer.microsoft.com/json-schemas/sp/v2/column-formatting.schema.json",
"elmType": "div",
"txtContent": "=if(indexOf([$FileLeafRef],'.')>-1,substring([$FileLeafRef],0,lastIndexOf([$FileLeafRef],'.')),[$FileLeafRef])"
}
Where FileLeafRef
is the internal name of SharePoint default "Name" column in document libraries. You can get the exact internal name of your column by following this article: How to find the Internal name of columns in SharePoint Online?
Output:

"
after=
in txtContent property, try removing it.