Index Server Recovery

Question

Our staging environment (1xSQL, 2X WFE 1x indexer / application running MOSS 2007 SP1 with some hotfixes) went bad (vmware mayhem) and we lost our search indexer. No form of recovery is possible.

Our plan is to rebuild from scratch which is nearly complete. The configuration (disks etc) is all identical.

Should I remove the old server from farm and then add the newly built machine with the original name?

Or run the configuration wizard and try to add it into the farm with the same name?

Or add it into the farm as a new indexer with a new name?

There is also a crazy plan to use a clone from a server that does exactly the same thing in our live environment. How does this sound to peeps (same domain, different name)?

Any comments appreciated.

Update 1:

Thanks, I've added the new server and got it setup crawling ok (although its taking for ever).

Should I reset all crawled content and start again? When should I remove the dead servers (old indexer and 2nd WFE) from the farm?

Update 2:

I did remove the server from the farm and did a new crawl and it's all happy now.

Answer 1

I think there is a guid in the registry that uniquely identifies a machine to the farm. It's located here:

HKEY_LOCAL_MACHINE -> SOFTWARE -> Microsoft -> Shared Tools -> Web Server Extensions -> 12.0 -> ServerId

Whenever I've cloned a VM I always change that to a new guid as well as delete the string value located at:

HKEY_LOCAL_MACHINE -> SOFTWARE -> Microsoft -> Shared Tools -> Web Server Extensions -> 12.0 -> Secure -> Configdb -> dsn

But i've only cloned WFE's, never the indexing server. So I don't know how that might affect things or whether you will need to build a new SSP from scratch or not.

One thought might be to assign your new machine the same serverid guid and computer name that the old one had and see what happens.

Since your old machine is down, i'm assuming you can probably find the serverid that you would need in the sharepoint config db somewhere. I think the objects table and then find the computer name using the name column and the id column is the serverid guid.

Answer 2

I would always add a new server with a new name as there is always a chance that the old server could have some reference somewhere screwed up - so always use a new server.

Answer 3

I've never actually had to deal with this, but my strong recommendation would be to add your new machine to the farm with a new name, then assign it the indexer role. This would effectively be the process you'd follow if you were swapping out the box which was the indexer under 'normal' circumstances (in which case you certainly wouldn't be trying to match up the machine name anyway). The sequence of removing the old machine/adding the new shouldn't matter at all.

Since SharePoint stores machine SIDs in the config db, it's not wise to pursue any of the other options as you'd then have a mismatch. I guess Steve Lineberry's suggestion of actually setting the SID of the new machine to be the same as the old machine (presumably with NewSid.exe or similar) would get you round that, but I certainly don't see any published guidance which recommends anything like that. The server roles are what's important, not matching machine names or SIDs.

Treat it as if you were swapping the machine out as business as usual and you'll be fine.

Answer 4

I would always deploy a replacement server as a new name then rebuild the indexes.

Chris brings up a really good point with the cloned image too, with the SIDs stored in SharePoint, using a clone could be risky. That said, we use a sysprep tool right before we take system images that scrubs the SID from the box. When you redeploy the image there's then a reverse process to enter the Windows product key as part of the deployment. This allows us to deploy system images with software/configuration already done without running into SID conflicts. I'm not sure how much that could apply to your situation if you're trying to use VM clones/snapshots. We do most of our imaging from physical hardware and use WinPE or Acronis to create the images.