So I'm working with a client that is using Fast Search to crawl a SharePoint 2010 Publishing site. For the most part it is working fine, except they have "news" and "events" pages that are dynamically generating content from lists. The content is a list of news articles or events with complex urls like protocol://server/site/sub-site/pages/news-details.aspx?List=SOMEGUID&Item=4. The "news" and "events" pages are being crawled, but it appears to be ignoring the links generated within the web-part as I'm not seeing any of the referenced pages in the crawl log or search results.
I've checked everything I know to check:
- Made sure Library Settings > Advanced Settings > Allow items from this document library to appear in search results is set to Yes.
- Made sure Site Settings > Search and Offline Availability > Indexing ASPX Page Content is set to Always index all Web Parts on this site
- Tried several iterations of crawl rules (broad to specific) with Crawl complex URLs and Crawl SharePoint content as http pages both set.
- Made sure the pages in question were published.
I'm stumped. Any suggestions on what I may be missing are greatly appreciated.
UPDATE:
I found out what was causing the problem. My client is making heavy use of the URL Rewrite module in IIS. I went through and disabled all the rules and SharePoint proceeded to crawl complex urls. Still need to reconcile the two, but at least I know what was causing the problem. Going to close this question...will see if I have to ask a new one. :)
Just a FYI for anyone who comes across this, here is Microsoft's official supportability statement on SharePoint and Rewrites/Redirects:
