Moving SharePoint Index location to another drive but gatherer still uses default location

Question

I'm in the process of moving the index location for SharePoint 2016 from C drive to a dedicated drive. I used the method to clone the topology, modify and then activate it but I have conflicting results.

When I check the search components I can see that the index component has the new root location (S:\SearchData) but when I check it using different commands I get different results.

As you can see from the search service instance it tells me that it's still set to the old default location. The same when listing components from the search service application. BUT when I view the actual active topology, the index component seems to be set correctly to the S drive. Currently I'm running a full crawl and I can see the S drive filling slowly but the size of the default location increases dramatically during crawl and then it releases the space. Does anyone has an explanation for this or I'm just doing something wrong? Also is it possible to change this location as wel? I couldn't find anything on the internet, just these articles describing the same issue but with no answer.

https://sharepoint.stackexchange.com/questions/254307/change-default-index-location-of-search-service-instance-on-sharepoint-2013] https://social.technet.microsoft.com/Forums/en-US/35490a80-0caa-45b2-b40d-cc173e544fe3/moving-the-index-location-in-sharepoint-2013?forum=sharepointgeneral

 $ssi = Get-SPEnterpriseSearchServiceInstance
 $ssi.Components

Id : 4e6ec89d-b6da-4fc7-bf47-3575d7936f98

ServerName : FrontEnd

IndexLocation : C:\Program Files\Microsoft Office Servers\16.0\Data\Office Server\Applications

State : Ready

DesiredState : Ready

IndexLocation : C:\Program Files\Microsoft Office Servers\16.0\Data\Office Server\Applications

Initialized : True

Name : cdd83138-e432-4561-b796-2148fd2dda6a

ServerName : FrontEnd

ServerId : c1676a68-43c4-4653-a80f-86b3b50ee93f

Server : SearchServiceInstance

$ssa = Get-SPEnterpriseSearchServiceApplication 
$ssa.AdminComponent.IndexLocation

C:\Program Files\Microsoft Office Servers\16.0\Data\Office Server\Applications

  Get-SPEnterpriseSearchComponent -SearchTopology $ssa.ActiveTopology

ComponentId : 9bf74bfd-4235-4cc4-a9f1-21a75e623a02

TopologyId : 22c97d72-2969-43dc-922f-09813f232f3a

ServerId : c1676a68-43c4-4653-a80f-86b3b50ee93f

Name : AdminComponent1

ServerName : FrontEnd

ExperimentalComponent : False

ComponentId : 5358ddcc-7c66-490e-802a-2fba96f8ebba

TopologyId : 22c97d72-2969-43dc-922f-09813f232f3a

ServerId : c1676a68-43c4-4653-a80f-86b3b50ee93f

Name : QueryProcessingComponent1

ServerName : FrontEnd

ExperimentalComponent : False

IndexPartitionOrdinal : 0

RootDirectory : S:\SearchData

ComponentId : 27c6772b-ed60-4ec6-86af-35195de0320e

TopologyId : 22c97d72-2969-43dc-922f-09813f232f3a

ServerId : c1676a68-43c4-4653-a80f-86b3b50ee93f

Name : IndexComponent1

ServerName : FrontEnd

ExperimentalComponent : False

ComponentId : f793411d-443c-47bb-9eed-6eaea7788919

TopologyId : 22c97d72-2969-43dc-922f-09813f232f3a

ServerId : c1676a68-43c4-4653-a80f-86b3b50ee93f

Name : ContentProcessingComponent1

ServerName : FrontEnd

ExperimentalComponent : False

ComponentId : 14f80f1e-dfe4-411a-ab1a-7fe21bed2350

TopologyId : 22c97d72-2969-43dc-922f-09813f232f3a

ServerId : c1676a68-43c4-4653-a80f-86b3b50ee93f

Name : CrawlComponent0

ServerName : FrontEnd

ExperimentalComponent : False

Answer 1

Maybe you can update the DefaultIndexLocation first and then update the topology as you're doing

$server = "ServerName"
$IndexLocation= "S:\SearchData" 

$server | % { 
    $svcInst = (Get-SPServer -Identity $_).serviceinstances | ? { $_.GetType().FullName -eq "Microsoft.Office.Server.Search.Administration.SearchServiceInstance" } 
    $svcInst.DefaultIndexLocation = $IndexLocation 
    $svcInst.Update() 
    
}

Answer 2

If you configure a new DefaultIndex Location, SharePoint will perform a process called Index Merge. This is poorly documented, but it will require up to 30% of the existing Index-Space on the source drive. If the merge completed, you should also get rid of the old locations automatically.

If you do not have or cannot provide enough space on C for index merge, you have to recreate search. A new topology will always cause the merge. An alternative would be to delete the Index before the topology change, so a merge won't consume much space.

Answer 3

I ended finding this thread answered by a MS tech: https://social.technet.microsoft.com/Forums/Lync/en-US/6dfdbc41-48c3-48e6-a6d3-2a171fd49d27/problems-with-drive-c-running-out-of-space-after-moving-sharepoint-search-index-to-d-drive

Basically it's possible by changing some registry settings but not supported. The index location from drive C in my case was were the search gatherer kept the files during crawling. They were deleted after being processed and copied to index on S drive. We ended up just monitoring the C drive space with perfmon overnight during full crawl and gauged how much we needed and just expanded C drive....

Here is more from the link in case it goes dead:

Hello.. I work with MS on the search team. I will suggest a couple of things to look at to get an understanding of where your data is being written to

on your search servers, when you install SP for the first time, there are 2 registry keys that gets created @ HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\15.0\Search\Setup, they are "DataDirectory" and "InstallDirectory" This pretty much sets the precedence of where data will live from the get go when you create SSAs Eventhough you may have defined like a D:\SearchIndex for the directory that the search index will live on the Index Components, there are still some other directories that come into play when the mssdmn process gathers the data, it will temporarily write its "documents" into the path defined in an example reg key like: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\15.0\Search\Global\Gathering Manager\TempPath Once they have been processed on this path, the mssearch process will then run a windows operation on those files in the TempPath and rewrite and move them to the path defined in this reg key as its base: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\15.0\Search\Components\66f3af0e-4772-46e2-b205-aaf103d0d62b-crawl-0\GathererDataPath Once mssearch.exe has done this, the Content Processing Component should then fetch the files from this path. It will fetch it via a network share ( ex: \AJCNSV5APP\gthrsvc_66f3af0e-4772-46e2-b205-aaf103d0d62b-crawl-0\ ) After its done doing this, these documents should disappear and no longer be stored in either the TempPath or this file share. If they are still hanging out in either, then something has an open handle on these files and not being allowed to be deleted. (I would ensure that AV is excluded from scanning these directories ) There is a "way" to move these temp paths and directories, but it is not the recommended way. I would actually recommend opening a support case if you want to do this. The recommended method to properly move these paths is to remove SharePoint from those search nodes and at install time, select a Drive that has plenty of space on another drive..

Anthony Casillas

First is a quick high level flow of what happens at crawl time:

1. As the mssdmn.exe (gatherer) goes out and fetches content, it will store the data in the TempPath defined at this registry key:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\15.0\Search\Global\Gathering Manager\TempPath

2. The mssearch.exe process will then perform a windows operation to Rename\Move the file to the path defined here at this registry key:

( Your "guid-crawl-x " will differ )

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\15.0\Search\Components\52ccea97-e96c-47ca-bcc4-737823da6e85-crawl-0\GathererDataPath

3. Once the Content Processing Component Processes this data, the mssearch should then delete the file from this path.

This is OK to use this method to move the locations where the "gatherer" writes it data to

• $ssa.TempPath cmdlet has been removed • To modify application path where mssdmn\mssearch writes to, the following registry keys should be modified:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\15.0\Search\Global\Gathering Manager\DefaultApplicationPath

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\15.0\Search\Components{Both admin and crawl components}\LocalStoragePath

( your "admin" component key will be a GUID folder without anything like, xxxxx-crawl-xx, following it )

• After that search service should be restarted

net stop osearch15

net start osearch15

• and original folders:

C:\Program Files\Microsoft Office Servers Debug\15.0\Data\Office Server\Applications{both admin and crawl components}

• should be removed • ONLY modify these keys and on the Crawl Servers and Admin components..