We have a highly customized SharePoint site in which we have used server name mappings to override the search url from (http://xxxxxx/Product/Forms/DispForm.aspx) to (http://xxxxxx/Pages/DocumentInfo.aspx). we show the document info based on the id. Initially all the documents were shown in the search results. But recently we started loosing most of the documents in the search. I've removed the Crawl rule that picks up only DispForm.aspx, then I can see all the documents but pointing directly to the document library, when I enforce this rule again I see only few. All the documents are published and we don't have the concept of draft. As soon as the user uploads a document through the site it is published. We have also noticed that most of the documents have more than 1 version which are either renamed or have some other data modified.
Further analysis shows that all the office file types are excluded and search result shows only pdf, txt and other formats when the server mappings are used.
Any suggestions are most welcome.
I checked for similar questions to avoid redundancy before posting but didn't find any.