SharePoint Stack Exchange is a question and answer site for SharePoint enthusiasts. It only takes a minute to sign up.
Sign up to join this community
We have a Sharepoint 2010 farm. There are a few pages that come up in SEO search results that I need to quash. How do I go from a URL to the source of the content in my farm? The URL has the form:
https://mydomain/sites/level1/level2/level3.aspx
I want to hide (possibly delete) anything at level1 or lower.
External crawlers are supposed to follow your robots.txt file. There is nothing "forcing" them to do so, but Google, Bing, Yahoo etc all "should". In SharePoint 2010 you create the text file and deploy it to the root of your IIS site. (In 2013/2016 the server does this automatically.)
I always have to play with it but I would start with:
User-agent: *
Disallow: /sites/level1/level2/level3.aspx
Once deployed you can try a validator like http://tool.motoricerca.info/robots-checker.phtml
You "hide" things by managing Site, list/library, folder, item/page permissions.
In your example "level1" is a Site Collection. You delete these from Central Administration. "level2" could be either a subsite or a library. Site Owners and Site Collection Administrators can delete subsites. Site Owners can delete libraries. "level3.aspx" is a web page. It can either be deleted from the library named "level2" or from the root of the site using SharePoint Designer.
If you just want to hide content from search, there is an option in each site's Advanced Settings to exclude the site from search. Libraries have a similar option.
