Changes to permission take effect in search results only with next full crawl

Question

I'm facing the problem that changes made to users' permissions within a site collections take effect only after a full crawl in search results.

Given the following SharePoint 2013 Server environment.

Search configuration

• approx. 10 content sources
• full crawl: once a week per content source
• incremental crawl: every 2 hours per content source
• continuous crawl: not configured

Permission scenario

The permissions are very granular and very restrictive due customer request. Within the site collections permissions are assigned to SharePoint Groups. Each document library has two "associated" SharePoint groups:

+-- document library A
       +-- document library A Visitors (Read permission)
       +-- document library A Members (Contribute permission)

+-- document library B
       +-- document library B Visitors (Read permission)
       +-- document library B Members (Contribute permission)

+-- etc.

• Individual users are added to these groups.
• Permissions on document libraries are not assigned directly to users.
• AD-groups cannot be used: Group membership limit of approximately 1,015 would be exceeded certainly. (see TechNet article: Active Directory Maximum Limits)

New user added to the site collection

A new user is added to a group (e.g. document library A Visitors) for the first time. He / she had never access to this site collection before.

The user can access all the items in document library A with Read permission immediatly via the GUI.

But when the user uses search to find some documents from library A zero documents are returned, until the next full crawl for the corresponding content source has been finished. Waiting for the next incremental crawl has no effect to the search results.

From what I've read, a full crawl is only necessary when permissions have changed on a file share not within SharePoint itself. (see TechNet article: Plan crawling and federation in SharePoint Server 2013 / Reasons to do a full crawl)

To sum it up

• Is it really necessary to wait for a full crawl so that permission changes in SharePoint take effect in the search results?
• Could there be a configuration issue within the search application (crawl user, ...)?
• Could a continuous crawl strategy be a solution to this problem?

Update 2015-10-14

After Steven Mulkerrins pointed out in his answer that an Incremental Crawl would suffice to update permissions, I’ve installed a fresh test system. Everything worked fine on an uncustomized site with just using Incremental Crawl. Also applying our “corporate standard modifications” (branding, broken permission inheritance, etc.) did not “break” anything.

So I contacted the admins of our production farm to confirm me the crawl details they gave me before I started my tests.

Here comes the embarrassing part: The admins changed the incremental crawl during my tests without proper authorization or notification from “every 2 hours” to “once a day”. So I was heading into the wrong direction.

What Steven Mulkerrins pointed out is correct: Incremental Crawl updates permission changes. Shame on me for bugging the community :)

Answer 1

I am surprised that an incremental crawl is not updating the permissions, I definitely believe that it should, but I haven't had a similar scenario to yours in some time, so can't be sure.

The only similar scenario I've seen recently was unfortunately with SharePoint Online, so it may not be exactly the same for on-prem, but my experiences of that were

• Using continuous crawl (the only option available in SPO) does definitely update permissions. (typically happens within 15 mins)
• When using individual users as your principles in SharePoint Groups, this works great up to about 5,000 users, but begins to fail for users beyond this. This is a search engine limitation and I would expect it to be the same for on-prem, so if your number of users in a group approaches or exceeds 5,000 this may be related to your issue.
• Putting users into Azure AD groups, and using the Azure AD groups as your principles for SharePoint groups works perfectly with 10,000+ users in the Azure AD group (I think our largest Azure AD group was close to 12,000 users).

One other thing I would point out is that I think you have misinterpreted the Active Directory limitation that you linked. Your link relates to how many groups a principle many belong to, rather than how many principles a group may contain. From Windows 2003 onwards, there is no known recommended limit for how many principles an AD group may have. Microsoft's scalability testing reached 500 million users: https://technet.microsoft.com/en-us/library/active-directory-maximum-limits-scalability(v=ws.10).aspx#BKMK_Users

So I would recommend that you investigate both enabling continuous crawling, and refactoring your permissions to use AD groups.