2

User Profile Service Application - Feed Cache Repopulation Job fails each time trying to run. Wondering if this is dependent on the farm using the SharePoint User Profile Sync Service (and FIM) vs. Active Directory User Import... (We are configured for AD User Import). If there is not dependencies what steps should I perform to trouble shoot and fix this issue.

Thanks. SG.

Here is the output from the commands in the comments below:

PS C:\scripts> Use-CacheCluster

PS C:\scripts> Get-CacheHost
Get-CacheHost : ErrorCode<ERRCAdmin032>:SubStatus<ES0001>:Invalid operation encountered on APP01.LAN.MYDOMAIN.COM:AppFabricCachingService : Cannot open
AppFabricCachingService service on computer 'APP01.LAN.MYDOMAIN.COM'.
At line:1 char:1
+ Get-CacheHost
+ ~~~~~~~~~~~~~
+ CategoryInfo          : InvalidOperation: (:) [Get-AFCacheHostStatus], DataCacheException
    + FullyQualifiedErrorId :     ERRCAdmin032,Microsoft.ApplicationServer.Caching.Commands.GetAFCacheHostStatusCommand

Get-CacheHost : ErrorCode<ERRCAdmin032>:SubStatus<ES0001>:Invalid operation     encountered on APP01:AppFabricCachingService : Cannot open     AppFabricCachingService service
on computer 'APP01'.
At line:1 char:1
+ Get-CacheHost
+ ~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [Get-AFCacheHostStatus], DataCacheException
    + FullyQualifiedErrorId : ERRCAdmin032,Microsoft.ApplicationServer.Caching.Commands.GetAFCacheHostStatusCommand


HostName : CachePort                  Service Name            Service Status Version Info
--------------------                  ------------            -------------- ------------
APP01.LAN.MYDOMAIN.COM:22233 AppFabricCachingService UNKNOWN        0 [0,0][0,0]
APP01:22233                   AppFabricCachingService UNKNOWN        0 [0,0][0,0]
WEB01.LAN.MYDOMAIN.COM:22233 AppFabricCachingService UP             3 [3,3][1,3]
WEB01:22233                   AppFabricCachingService UP             3 [3,3][1,3]

PS C:\scripts> Get-CacheClusterHealth

Cluster health statistics

HostName = WEB01.LAN.MYDOMAIN.COM

NamedCache = default
    Healthy               = 3.03
    UnderReconfiguration  = 0.00
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedActivityFeedCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 3.03
    UnderReconfiguration  = 0.00
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedBouncerCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 3.03
    UnderReconfiguration  = 0.00
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedLogonTokenCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 3.03
    UnderReconfiguration  = 0.00
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedSecurityTrimmingCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 3.03
    UnderReconfiguration  = 0.00
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedAccessCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 3.03
    UnderReconfiguration  = 0.00
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedSearchCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 3.03
    UnderReconfiguration  = 0.00
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedViewStateCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 3.03
    UnderReconfiguration  = 0.00
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedServerToAppServerAccessTokenCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 3.03
    UnderReconfiguration  = 0.00
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedActivityFeedLMTCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 3.03
    UnderReconfiguration  = 0.00
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedDefaultCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 3.03
    UnderReconfiguration  = 0.00
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

HostName = WEB01.LAN.MYDOMAIN.COM

NamedCache = default
    Healthy               = 0.00
    UnderReconfiguration  = 3.03
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedActivityFeedCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 0.00
    UnderReconfiguration  = 3.03
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedBouncerCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 0.00
    UnderReconfiguration  = 3.03
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedLogonTokenCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 0.00
    UnderReconfiguration  = 3.03
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedSecurityTrimmingCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 0.00
    UnderReconfiguration  = 3.03
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedAccessCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 0.00
    UnderReconfiguration  = 3.03
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedSearchCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 0.00
    UnderReconfiguration  = 3.03
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedViewStateCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 0.00
    UnderReconfiguration  = 3.03
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedServerToAppServerAccessTokenCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 0.00
    UnderReconfiguration  = 3.03
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedActivityFeedLMTCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 0.00
    UnderReconfiguration  = 3.03
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedDefaultCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 0.00
    UnderReconfiguration  = 3.03
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

Unallocated named cache fractions

NamedCache = default
    Unallocated fraction     = 3.03

NamedCache = DistributedAccessCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Unallocated fraction     = 3.03

NamedCache = DistributedActivityFeedCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Unallocated fraction     = 3.03

NamedCache = DistributedActivityFeedLMTCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Unallocated fraction     = 3.03

NamedCache = DistributedBouncerCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Unallocated fraction     = 3.03

NamedCache = DistributedDefaultCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Unallocated fraction     = 3.03

NamedCache = DistributedLogonTokenCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Unallocated fraction     = 3.03

NamedCache = DistributedSearchCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Unallocated fraction     = 3.03

NamedCache = DistributedSecurityTrimmingCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Unallocated fraction     = 3.03

NamedCache = DistributedServerToAppServerAccessTokenCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Unallocated fraction     = 3.03

NamedCache = DistributedViewStateCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Unallocated fraction     = 3.03

Here is the newest output for the DC Health command: PS C:\scripts> Get-CacheClusterHealth

Cluster health statistics
=========================

HostName = DSASHAREWEB01.LAN.ODOD.OHIO.GOV
-------------------------

NamedCache = DistributedAccessCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 9.09
    UnderReconfiguration  = 0.00
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = default
    Healthy               = 9.09
    UnderReconfiguration  = 0.00
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedActivityFeedCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 9.09
    UnderReconfiguration  = 0.00
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedActivityFeedLMTCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 9.09
    UnderReconfiguration  = 0.00
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedBouncerCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 9.09
    UnderReconfiguration  = 0.00
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedLogonTokenCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 9.09
    UnderReconfiguration  = 0.00
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedSecurityTrimmingCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 9.09
    UnderReconfiguration  = 0.00
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedSearchCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 9.09
    UnderReconfiguration  = 0.00
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedViewStateCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 9.09
    UnderReconfiguration  = 0.00
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedServerToAppServerAccessTokenCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 9.09
    UnderReconfiguration  = 0.00
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00

NamedCache = DistributedDefaultCache_c4f975aa-4f68-4847-b470-55e95ff7a5d7
    Healthy               = 9.09
    UnderReconfiguration  = 0.00
    NotPrimary            = 0.00
    InadequateSecondaries = 0.00
    Throttled             = 0.00


Unallocated named cache fractions
---------------------------------
  • It isn't. But it is dependent on Distributed Cache. Is that healthy? Use-CacheCluster;Get-CacheHost;Get-CacheClusterHealth. – Trevor Seward Aug 16 '17 at 14:50
  • Should I try removing the Distributed Cache and rebuild it on all servers or perhaps just one? I tried running the below commands and that is when the FQDN's showed up, but the timer-job issue has been there before and after running those commands: Remove-SPDistributedCacheServiceInstance Add-SPDistributedCacheServiceInstance – Shane Gib. Aug 17 '17 at 14:44
  • Unallocated fractions are a sign of an unhealthy Distributed Cache instance. Those fractions should be reallocated after a short period of time. How long have you waited between starting DC and watching the unallocated fraction count? – Trevor Seward Aug 17 '17 at 14:54
  • Servers have been up and running for about 5 days now before I ran the: Get-CacheClusterHealth I also ran this command a few minutes ago with same results: Get-AFCacheClusterHealth I installed the August 2017 CU's on the servers got rid of a lot of issues but this one has been lingering for awhile now and the last issue I have found in this "Hand me down" farm I have been given to work with... Thanks for the help btw much appreciated! SG. – Shane Gib. Aug 17 '17 at 15:34
  • @TrevorSeward, take a look at the newest Get-CachClusterHealth command output above. The Unallocated fractions as well as, the UnderReconfiguration duplicate issues are gone. Also the original problem of the User Profile Service Application - Feed Cache Repopulation Job - Failed is now fixed, if this looks more correct now let me know and I will post the solution, otherwise just let me know if the Healthy 9.09 is off target... All PowerShell errors are gone now as well (from the Get-Cachehost). Thanks. – Shane Gib. Aug 17 '17 at 16:56
0

NOTE: When working with the DC in PS (using the SP Shell Admin) you need to run as administrator and you need to load the connection from SP Shell Admin to the App Fabric Cache Service by using the following PS Command (Do this both on the APP01 and WEB01 Servers first):

Use-CacheCluster

THEN the Simple answer to my particular use case was as follows on the APP01 Sever I ran the following PS Command:

Remove-SPDistributedCacheServiceInstance 

Then on the WEB01 Server I ran the following PS Commands AFTER THE ABOVE:

Unregister-CacheHost –Provider SPDistributedCacheClusterProvider –ConnectionString "Data Source=SQL01;Initial Catalog=SPConfig2013;Integrated Security=True;Enlist=False" –HostName APP01.LAN.MYDOMAIN.COM
Unregister-CacheHost –Provider SPDistributedCacheClusterProvider –ConnectionString "Data Source=SQL01;Initial Catalog=SPConfig2013;Integrated Security=True;Enlist=False" –HostName APP01
Unregister-CacheHost –Provider SPDistributedCacheClusterProvider –ConnectionString "Data Source=SQL01;Initial Catalog=SPConfig2013;Integrated Security=True;Enlist=False" –HostName WEB01

Then ran the following commands (on WEB01) to recheck the health status of the Distributed Cache:

Get-CacheHost
Get-AFCacheHostConfiguration -ComputerName WEB01.LAN.MYDOMAIN.COM -CachePort "22233"
Get-CacheClusterHealth

The output for the Distributed Cache before it was fixed and the output for the Distributed Cache after it was fixed is listed in the question itself above

*NOTE: Before you run these PS Commands I suggest you look at this TechNet Article and understand the repercussions of running them (its all in the fine print literally...) Especially if it is in your production environment, I would suggest testing them in another environment i.e. TESTAPP01 or DEVWEB01 first just to see if they have an unacceptable side effect!

Happy SharePointing. SG.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.