1

After recovering from a recent hardware failure on our SharePoint server (single server farm), all the SQL DBs were in suspect mode, to change the mode back to normal, we ran the consistency checks on all DBs and successfully changed back to normal mode. However, one particular database i.e. SharePoint_AdminContent_ is still causing SQL crashes with messages like:

The Database ID 6, Page (1:11812), slot 22 for LOB data type node does not exist. This is usually caused by transactions that can read uncommitted data on a data page. Run DBCC CHECKTABLE.

dbcc checkdb with REPAIR_ALLOW_DATA_LOSS fails and does not complete successfully.

I have set the DB to single user mode for now, the central admin works when I set to multi user mode but the SQL logs very quickly fill up the hard drive with crash dumps. I suspect that the hardware failure has caused some serious damage to the DB which cannot be repaired.

I tried to move central admin site to a new content db using move-spsite but it fails with the error given above.

Now, in an attempt to repair central admin, I have tried to unprovision the central admin and tried to re-create the central admin using both Configuration Wizard GUI and PowerShell, one by one but both these methods return the same error that I have specified above, while trying to create new central admin.

I have tried to backup the corrupted DB and restore it to a new DB to see if it works, but it does not. The corruption transfers to restored DB as well.

I have also tried to detach the corrupted AdminContent DB from SQL and then tried to create a new central admin site (hoping that it will create a new admin content DB) but it complains that it cannot find the old admin content db (I suppose SharePoint_Config DB holds the references to old AdminContent DB), anyways this method fails as well because the old DB detached, and is not available.

Then, I have tried to create a new content database under central admin web application, unprovisioned central admin site, removed corrupted AdminContent DB (through central admin) and tried to create new central admin site using psconfiggui, it did not open the site until I attached corrupted admin content DB through powershell (mount-spcontentdatabase)

I have a full farm backup taken using SharePoint native tools through powershell. It has central admin backup but it cannot be restored individually, I will need to restore the whole farm somewhere to even try to see if restored admin content DB will work and even if it works, how would I transfer it back to original farm because it will have a new guid and how would I associate it with original farm? I cannot restore it to original farm because the backup is 3-4 days older and I can only restore admincontent if I perform a full farm restore which will overwrite all the content as well.

Is there any way I can setup a new Admin Content DB and create a new central admin site using that DB? or anything I can do to fix this? Any help will be appreciated.

Improve this question
9
  • You said hardware failure, which piece failed harddrive, CPU, Ram, board? your SQL and SharePoint is on same server? how you restored the SQL db after recover from Failure? DO you have any other server, which you can attach to the farm just for temp and provision the Central admin over there...once working then move Central admin back and remvoe the server from farm? what about other sites/web app are these working?
    – Waqas Sarwar MVP ♦
    Commented Sep 8, 2014 at 13:29
  • We had two physically different drives, DB files were placed on one and transaction logs were placed on the other. The one with transaction logs failed but then came alive and the log files were recovered but that failure caused all SQL DBs to go to suspect mode.
    – JTag
    Commented Sep 8, 2014 at 17:05
  • did you clear the Config cache on the server? and do you have backup of the Admin content DB, if yes try to restore it.
    – Waqas Sarwar MVP ♦
    Commented Sep 8, 2014 at 17:16
  • I have full farm backup, as specified in my original post, but it does not allow restoring just the central admin DB alone, the whole farm has to be restored which we cannot afford since it will overwrite everything.
    – JTag
    Commented Sep 8, 2014 at 17:19
  • All other Sites/Apps are functional and did not have any consistency error that could not be fixed by dbcc checkdb. Yes I can arrange another server but the problem is that only 1 centraladmin db (which in my case is already corrupt) is allowed per farm, so even if I create centraladmin on another server it will use the same DB. What is config cache and how can i clear it? can you please explain?
    – JTag
    Commented Sep 8, 2014 at 17:20

1 Answer 1

Reset to default
1

If all above steps not fix the issue, then i think you need to think about 2 ways.

  1. As you said you have the full farm backup, then build a temp farm and restore it on the new farm. Once recovered and CA working then you can backup & restore the admin DB. Your millage may vary may be Config db complain about it.
  2. 2nd option, you can take the full backup of your content Database, note down all the customization, AAM settings, keep the solutions handy. Then below up the farm and rebuild it. after rebuilding, create Service Apps, Web Apps and then mount the database to relative web apps. Apply the AAM settings, or any customization.
Improve this answer
2
  • I did unprovision the CA but did not delete old admin content db, just detached it from SQL. I don't think I can create a new admin db AFTER I have unprovisioned CA. Anyways, I am going to try to create a new DB, attach it to CA, then unprovision CA, remove old DB and try to provision new CA again on a test server and see what happens. If it doesn't work, I think I will go with the rebuilding the whole farm option.
    – JTag
    Commented Sep 8, 2014 at 18:48
  • if it is not a hardware/ software related issue then you can also try SharePoint Database Recovery Tool to fix content database
    – Jason Clark
    Commented Sep 12, 2017 at 9:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.