The site I'm working on uses folders inside the Pages library (yeah, I know and agree.. but they pay the bills). When searching for key terms, I'm not getting the pages within these folders in my results.
Initially I think I was, but I added some crawl rules to hide list items, list forms, etc and having reset my index and done a full crawl, I only get pages that live in the Pages library itself, rather than within folders in that Pages library.
I removed my crawl rules, reset the index again (thankfully this is a dev site, not production) and I still only get pages, not folder/pages.
This is true with osssearchresults.aspx as well as my custom results.aspx file (which is essentially the ootb page with a different master)
Any ideas of what I can do to allow folder/page style results? I swear this was working before I add/removed the rules.
[Updated to provide more info]
My site structure matches this example
.../Pages/default.aspx .../us/Pages/default.aspx .../us/Pages/products/default.aspx .../us/Pages/products/allproducts.aspx .../us/Pages/products/product1.aspx
Each of the pages within the us Pages library mentioned product1 in some way. So, when I search for product1 I would expect to see results for all 4 of the pages within that library, listed above.
What I see is just
no /us/Pages/default.aspx or any pages within the products folder inside that library.