Export Crawl Log with PowerShell

Question

I try to export the crawl log with PowerShell to analyze the errors and most importantly the warnings:

Add-PSSnapin microsoft.sharepoint.powershell -ea 0

$ssa = Get-SPEnterpriseSearchServiceApplication | Where-Object {$_.Name -eq "Search Service Application"} 

$id = Get-SPEnterpriseSearchCrawlContentSource -SearchApplication $ssa | ? { $_.Name -eq "Filer"}

$log = New-Object Microsoft.Office.Server.Search.Administration.CrawlLog $ssa

#GetCrawledUrls(bool getCountOnly, long maxRows, string urlQueryString, bool isLike, int contentSourceID, int errorLevel, int errorID, DateTime startDateTime, DateTime endDateTime)

Write-Host "Success:"
$dt = $log.GetCrawledUrls($false, 1000000, $null, $false, $id.Id, 0, -1, [System.DateTime]::MinValue, [System.DateTime]::MaxValue)
$dt.Rows.Count | Out-String

Write-Host "Warnings:"
$dt = $log.GetCrawledUrls($false, 1000000, $null, $false, $id.Id, 1, -1, [System.DateTime]::MinValue, [System.DateTime]::MaxValue)
$dt.Rows.Count | Out-String

Write-Host "Errors:"
$dt = $log.GetCrawledUrls($false, 1000000, $null, $false, $id.Id, 2, -1, [System.DateTime]::MinValue, [System.DateTime]::MaxValue)
$dt.Rows.Count | Out-String

Write-Host "Deleted:"
$dt = $log.GetCrawledUrls($false, 1000000, $null, $false, $id.Id, 3, -1, [System.DateTime]::MinValue, [System.DateTime]::MaxValue)
$dt.Rows.Count | Out-String

Write-Host "Everything:"
$dt = $log.GetCrawledUrls($false, 1000000, $null, $false, $id.Id, -1, -1, [System.DateTime]::MinValue, [System.DateTime]::MaxValue)
$dt.Rows.Count | Out-String

$group = $dt | Group-Object "ErrorID"
$errors = $log.GetErrorMessages() # <= this guy returns an internal Type: Microsoft.Office.Server.Search.Administration.ErrorInfo

I tried to get the Warning/Error Messages with $log.GetErrorMessages() - but that does not return anything. I assume this is because the return type is Dictionary and ErrorInfo is an internal class.

Anyone can give me a pointer how to get the error messages?

Answer 1

The reason it fails is due to a code bug in Microsoft.Office.Server.Search.Administration.GetErrorMessages.

In the loop it checks if allKnownErrors has the key, and of course it does as it's looping that dictionary. The code should check if dictionary2.ContainsKey instead.

    public Dictionary<int, string> GetErrorMessages()
    {
        IDictionary<int, ErrorInfo> allKnownErrors = this.GetAllKnownErrors();
        Dictionary<int, string> dictionary2 = new Dictionary<int, string>();
        foreach (KeyValuePair<int, ErrorInfo> pair in allKnownErrors)
        {
            if (!allKnownErrors.ContainsKey(pair.Key))
            {
                dictionary2.Add(pair.Key, pair.Value.Message);
            }
        }
        return dictionary2;
    }

To get all possible error messages you can use code similar to this:

CrawlLog log = new CrawlLog(searchApp);
MethodInfo methodInfo = log.GetType().GetMethod("GetAllKnownErrors", BindingFlags.NonPublic|BindingFlags.Instance);
var allKnownErrors = methodInfo.Invoke(log, null) as IDictionary;
Dictionary<int, string> dictionary = new Dictionary<int, string>();
Assembly searchDll = Assembly.Load("Microsoft.Office.Server.Search, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c");
Type errorInfoType = searchDll.GetType("Microsoft.Office.Server.Search.Administration.ErrorInfo");
var propertyInfoMessage = errorInfoType.GetField("Message");
foreach (DictionaryEntry dictionaryEntry in allKnownErrors)
{
    int key = (int)dictionaryEntry.Key;
    string message = (string)propertyInfoMessage.GetValue(dictionaryEntry.Value);
    if (!dictionary.ContainsKey(key))
    {
        dictionary.Add(key, message);
    }
}