Symptoms

Unable to access ViewFlow with the following error message:

"System.DirectoryServices.ActiveDirectory.ActiveDirectoryServerDownException: The server is not operational. Name: "..."

We receiving this despite there are multiple domain controllers available.
 

Diagnoses

Usually a K2 server machine restart resolves the issue.
 

Resolution

Usually K2 server machine restart resolves the issue.

This is how AD handles LDAP requests: The DC Locator Service has been re-designed in Windows Server 2008 and later to include a new mechanism. When a client computer finds a preferred domain controller, it sticks to this domain controller unless that domain controller stops responding or the client computer is restarted. This is generally called Domain Controller Stickiness. If you take this domain controller offline for maintenance purpose or it goes down, the clients that were connected to it will look for another domain controller to shift their connections to new domain controller. But when the domain controller comes online again, these connections are not shifted back because client computers do not refresh themselves to check to see if domain controller is back again. This can cause load-balancing issues because client computers remain connected to same domain controller.

For the cases when multiple DCs available but one of them (to which K2 happens to be connected) fails following is applicable. K2 performs bind with the DirectoryEntry class e.g: new DirectoryEntry(?LDAP://DC=Domain,DC=COM?, ??, ??,AuthenticationTypes.ReadOnly) This process relies on Domain Controller Locator which an algorithm that runs in the context of the Net Logon service. Essentially it is a sort of AD DS client part which is responsible for selecting specific DC for specific domain. Domain Controller Locator has its own cache. The Net Logon service caches the domain controller information so that it is not necessary to repeat the discovery process for subsequent requests. Caching this information encourages the consistent use of the same domain controller and, thus, a consistent view of Active Directory. Refer to the Microsoft documentation for details: Domain Controller Location Process https://technet.microsoft.com/en-us/library/cc978011.aspx Domain Controller Locator https://technet.microsoft.com/en-us/library/cc961830.aspx Sometimes this cache may keep a reference to unavailable DC for specific domain, and the following command can be used to force reset of this cache: nltest /dsgetdc:DomainName /force Note sometimes it is necessary to run this a few times till another DC is selected. To check which DC you are using at the moment for specific K2 server you can use: nltest /dsgetdc:domain_name You may also look into your the following group policy setting "Specify positive periodic DC Cache refresh for non-background callers," according to documentation it determines when a successful DC cache entry is refreshed. This policy setting is applied to caller programs that do not periodically attempt to locate DCs and it is applied before the returning the DC information to the caller program. Default value (when it is not configured) is 30 minutes (1800). I am not sure if it influence K2 or not, but if it does it means that with default settings your K2 issue should be resolved by itself in 30 minutes. It is not recommended to set very low values for this policy, but in case you have this policy configured to a significantly higher values you may revert back to default or even slightly lesser values for it on your K2 servers. As Domain Controller Locator runs in the context of the Net Logon service restart of this service likely clear locator cache too. Note: in link failure scenario when no DCs available for specific domain are available for specific domain (all DCs are behind WAN link which is temporarily failed) there is nothing that can be done from K2 side apart from restoring WAN link or placing locally available DC/RODC to mitigate against this scenario. In such scenario you will likely to get slightly different exception - "A referral was returned from the server"

Current recommendations: 1) Reconfigure K2 to use GC instead of LDAP. The global catalog is a distributed data repository that contains a searchable, partial representation of every object in every domain in a multidomain Active Directory Domain Services (AD DS) forest. So essentially your GC placed in local domain can serve part of the queries which otherwise should go to DCs in another domain, potentially over WAN link. From purely AD DS side GC has the following benefits: - Forest-wide searches. The global catalog provides a resource for searching an AD DS forest. - User logon. In a forest that has more than one domain GC can be used during logon for universal group membership group enumeration (Windows 2000 native DFL or higher) and for resolving UPN name when UPN is used at logon. - Universal Group Membership Caching: In a forest that has more than one domain, in sites that have domain users but no global catalog server, Universal Group Membership Caching can be used to enable caching of logon credentials so that the global catalog does not have to be contacted for subsequent user logons. This feature eliminates the need to retrieve universal group memberships across a WAN link from a global catalog server in a different site. Essentially you may enable this feature to make use of GC even more efficient. To reconfigure K2 to use GC you have to edit RoleInit XML field of HostServer.SecurityLabel table and replace "LDAP://" to "GC://" with subsequent restart of K2 service. From K2 prospective it should improve responsiveness of AD SmartObjects as well as slightly decrease reliance on WAN link/number of queries to DCs outside of local domain. 2) Try to use Domain Locator cache refresh clear up if the same issue will occur again (see details above, nltest /dsgetdc:DomainName /force) - and verify whether it is viable workaround. It should work. You may also leverage "nltest /dsgetdc:DomainName" to confirm which specific DC is being used by K2 server to verify status and availability of this specific DC with your infrastructure team.