Technical Issues

Yes

Identity Synchronization and Caching

The Identity Sync Service, introduced with K2 Five (5.2), manages identity synchronization and caching in K2. Use the buttons below to download the installer for your version of K2.

Installing the sync service changes the way K2 syncs and caches identities (users, groups and group memberships) from your Identity Provider (IdP), such as Active Directory or Azure Active Directory.

Installing the sync service provides the following benefits:

Faster, differential syncing of identities after initial (full) sync for most IdPs
Removes the auto-expiration of cached identities
Avoids the direct resolution of identities with the provider
Addresses performance issues present in the previous way identities were cached

The installer supports Microsoft SQL Server 2019.
K2 uses the same identity cache as before when enabling the new sync service, but the way the cache is populated is based on the sync service. The new service provides better performance and predictability for populating the identity cache.
The sync service does not change the way system URM SmartObjects query and return data from IdPs, which means that enabling the new sync service will not affect your solutions that use these SmartObjects.

Who should choose this functionality?

This new identity synchronization is recommended for all K2 Five (5.3) and later customers. The main goals are:

Consistent and reliable identity data through frequent, proactive caching – populate the K2 identity cache for users, groups, and group membership, independent of someone logging or refreshing their worklist. Also, the identity information cached always represents the latest information from the IdP. For example, a cached identity is active in the K2 identity cache until it is disabled or deleted from the provider.
Improve runtime performance through local identity resolution – anytime the K2 platform needs to know attributes about a user (for example their email or manager) or group membership, K2 only queries the local K2 identity cache, avoiding 'expensive' direct queries of the IdP to retrieve user or group data.
Scalable with better performance by leveraging differential queries – when possible, differential queries get changes since the last sync with supported identity stores (AD, AAD, and SharePoint).
Only AD, AAD and SharePoint support differential syncs. Other providers such as SQLUM, LDAP, and custom UMs, do a full sync every time.

The identity caching model manages the same identity attributes as in the previous model. These include user information (name, login name, email address, manager), group information, and group membership. It is important to remember that passwords are never synchronized or stored in the K2 identity cache. The identity cache is the single source for identities in K2 which the sync service updates on a regular basis to maintain consistency with the provider.

Hybrid SharePoint environments cannot use the sync service. If you use SharePoint in a hybrid environment, in other words using both SharePoint online and on-premises, the sync service only syncs the most recently registered SharePoint environment. So, if you first registered SharePoint on-premises and then later registered SharePoint Online, only SharePoint Online will be synced.

Terminology

Identities	Users, groups, and group memberships
Identity Provider (IdP)	Source of identity record Examples include AD, AAD, and SharePoint
Sync providers	Execute queries against an IdP Stores identity data in an intermediary table
Sync types	Full - gets all users, groups and group memberships from an IdP Differential (diff) - gets only the changes since the last sync

Installing and using the sync service

The time it takes to complete the first sync depends on the number of identities in your Identity Provider (IdP) and the complexity of the membership of the users & groups synced. The first sync can take a few hours to complete because all users, groups, and group memberships are synced. While the sync is running, the identities that are not yet cached are not available to K2, which means they will not have access to K2 sites and tools.
See the Run the first sync below for instructions about to run your first sync.
If you have multiple provider instances, sync one at a time.

High-level steps

Install or upgrade to K2 Five 5.3 or later and apply the latest fix pack, if applicable. The K2 installation creates identity sync providers for each security label in your environment. These sync providers are referenced when using SmartObjects to manage your syncs.
Download and run the Identity Sync Service installer for your version of K2 on your K2 server.
Run the first sync for each provider instance you want to sync.
Configure recurring scheduled differential syncs.
Only AD, AAD, and SharePoint support differential syncs. Other providers such as SQLUM, LDAP, and custom UMs, do a full sync every time.

Sync service installer

The sync service installer makes changes in the database to turn off the legacy identity resolution behavior and use the sync service instead. Use the Sync Service SmartObjects in the K2 Management site to manage identity syncs. Download and run the installer on your K2 server. If you have a distributed installation, you must restart the K2 service on each node, but you do not have to run the installer again on each physical K2 server in the farm. Use the Change button to select your K2 database if the installer does not automatically choose the correct database.

Run the first sync

The time it takes to complete the first sync depends on the number of identities in your Identity Provider (IdP) and the complexity of the membership of the users & groups synced. The first sync can take a few hours to complete because all users, groups, and group memberships are synced. While the sync is running, the identities that are not yet cached are not available to K2, which means they will not have access to K2 sites and tools.

For AD, AAD, and SharePoint IdPs, all syncs started after the initial full sync run as differential syncs. This retrieves only changes in identity information since the last sync and, as a result, is significantly faster than the initial full sync. See the Sync Service topic in the User Guide for information on re-executing a full sync.

If you enable the sync service, you must run a sync for each of the providers in your environment. If you do not, you may see errors like the following:
Could not connect to server [k2.denallix.com]:[5555]
Inner Exception : An identity for the user K2:DENALLIX\Administrator could not be found.
Source : SourceCode.Categories.Runtime

Follow these steps to run the first full sync of identities for each IdP:

How to sync Active Directory

Open the K2 Management site and browse to the Sync Service object in the System category.
Select the Operation SmartObject and execute the Start Sync method.
- The Provider Name matches the security labels configured on your system. Active Directory has a default value of K2.
- The Provider Instance Name is the name of the provider instance and is the domain section of a fully qualified domain name. Use your domain name if you have Active Directory. The AAD provider does not have a Provider Instance Name so leave it blank. SharePoint requires a Provider Instance Name which you can find by executing the List Provider Instances method of the Provider Instance SmartObject.
- Enter a value of Yes or No in the Skip Deleted box to exclude (skip) or include deleted identities from your provider. On your first sync, K2 recommends skipping deleted identities because you most likely do not want deleted items in your K2 cache. The sync service does not retrieve skipped items in future syncs unless they change.
  After the first sync, set the Skip Deleted option to No (or leave it blank) so that identities you delete after the first sync are also marked as deleted in your K2 cache.
If you are upgrading your K2 environment to K2 Five 5.3, set the Skip Deleted to No (or leave it blank) to sync items in the cache that you’ve recently deleted. Doing this also syncs all deleted items from in your IdP but ensures that your K2 cache is up to date.
You must do this initial sync for each IdP (each label) in your environment.

How to sync Azure Active Directory

See the Known Issues section for important information on using K2 Five and AAD.

Open the K2 Management site and browse to the Sync Service object in the System category.
Select the Operation SmartObject and execute the Start Sync method.
- The Provider Name matches the security labels configured on the system. Azur Active Directory has a default value of AAD.
- The Provider Instance Name is the name of the provider instance and is the domain section of a fully qualified domain name. The AAD provider does not have a Provider Instance Name so leave it blank.
- Enter a value of Yes or No in the Skip Deleted box to exclude (skip) or include deleted identities from your provider. On your first sync, K2 recommends skipping deleted identities because you most likely do not want deleted items in your K2 cache. The sync service does not retrieve skipped items in future syncs unless they change.
  After the first sync, set the Skip Deleted option to No (or leave it blank) so that identities you delete after the first sync are also marked as deleted in your K2 cache
You must do this first-time sync for each IdP (and hence, each label) in your environment.

After one of the cumulative updates mentioned in the Nintex K2 migration to Microsoft Graph article is installed on your system, you must run the following script for the Sync Service to work with AAD. The script works by changing the AAD provider Type to Graph and registering an oAuthResourceId for Graph in the Sync Engine.

This script assumes that the provider name and your security label name are AAD, if not, edit the script to reflect your environment. Contact support if you are not sure what values to change.

DECLARE @ProviderTypeID AS int
SELECT @ProviderTypeID =ID
FROM [SyncEngine].[ProviderType]
WHERE Type = 'MSGraph'

UPDATE [SyncEngine].[Provider]
SET ProviderTypeID = @ProviderTypeID
WHERE Name = 'AAD'

DECLARE @ProviderInstanceID AS int

SELECT @ProviderInstanceID = ID
FROM [SyncEngine].[ProviderInstance]
Where ProviderID = (SELECT ID
FROM [SyncEngine].[Provider]
WHERE Name = 'AAD')

DECLARE @ProviderInstanceIDGuid AS nvarchar(max)

SELECT @ProviderInstanceIDGuid = AuthInit.value('(/AuthInit/OAuthResourceID/node())[1]','nvarchar(max)')
FROM [HostServer].[SecurityLabel]
Where SecurityLabelName = 'AAD'

Insert into [SyncEngine].[ProviderInstanceRuntimeConfig]([ProviderInstanceID],[ConfigKey],[ConfigValue])
VALUES (@ProviderInstanceID,'msgraph.oAuthResourceId',@ProviderInstanceIDGuid)

How to sync SharePoint

Open the K2 Management site and browse to the Sync Service object in the System category.
Select the Operation SmartObject and execute the Start Sync method.
The Provider Name matches the security labels configured on your system. SharePoint has a default value of SP.
SharePoint requires a Provider Instance Name which you can find by executing the List Provider Instances method of the Provider Instance SmartObject.
Enter a value of Yes or No in the Skip Deleted box to exclude (skip) or include deleted identities from your provider. On your first sync, K2 recommends skipping deleted identities because you most likely do not want deleted items in your K2 cache. The sync service does not retrieve skipped items in future syncs unless they change.
After the first sync, set the Skip Deleted option to No (or leave it blank) so that identities you delete after the first sync are also marked as deleted in your K2 cache.

After the first sync, remember to set up a sync schedule for SharePoint as described in the Configure sync schedules section below.

How to sync SQL, LDAP and Custom providers

Open the K2 Management site and browse to the Sync Service object in the System category.
Select the Operation SmartObject and execute the Start Sync method.
- The Provider Name matches the security labels configured on the system. Use the name of the label you want to sync.
- Leave the Provider Instance Name blank for legacy providers.
- Enter a value of Yes or No in the Skip Deleted box to exclude (skip) or include deleted identities from your provider. On your first sync, K2 recommends skipping deleted identities because you most likely do not want deleted items in your K2 cache. The sync service does not retrieve skipped items in future syncs unless they change.
  After the first sync, set the Skip Deleted option to No (or leave it blank) so that identities you delete after the first sync are also marked as deleted in your K2 cache
You must do this first-time sync for each IdP (and hence, each label) in your environment.

When syncing with AD FS, only the first 1000 records (identities or groups) are returned by default. To sync all AD FS identities, change your configuration as described in KB002721 - How To Configure ADFS To Return More Than The Default 1000 Records.

For every sync (full and differential), identities are cached to the intermediary cache and then transformed and placed in the main identity cache one minute after the sync. A maximum of two syncs can run at a time. Once they finish, additional scheduled syncs are run.

New identities added to your IdP after the initial sync, and changes to existing identities, are not synced to K2 unless you configure a sync schedule as detailed in the next section.

Configure sync schedules

Follow these steps to configure the identity sync schedule:

Open the K2 Management site and browse to the Sync Service object in the System category.
Select the Operation SmartObject and execute the Set Provider Schedule method.

You can schedule an amount of time to sync between three minutes and seven days. You must enter it in the following format: DD:HH:MM:SS. For example, 00:07:00:00 is seven hours. In this case, the next scheduled sync starts seven hours after the previous one finished syncing.

The AD, AAD, and SharePoint providers perform differential syncs on schedule but other IdPs (SQLUM, ADFS, LDAP, and custom) do a full sync at the scheduled time. Differential syncs are quick and efficient but if you have a large number of identities in SQLUM, for example, each scheduled sync takes about as long to run as the initial sync. Take that into consideration when configuring schedules for these IdPs. You may want to schedule syncs less frequently for IdPs that do not have volatile user data that changes often.

Sync run history

Use the Run History SmartObject method to see when your last sync ran and its status. Follow these steps to see the sync history.

Open the K2 Management site and browse to the Sync Service object in the System category.
Select the Run History SmartObject and execute one of the methods such as Get Run History Entries By Date Range as shown here.

See the Sync Service broker topic in the K2 Five User Guide for information on the Sync Service SmartObjects.

Identity Sync Service Configuration Settings

This section lists the various configuration settings of the Identity Synchronization Service. All settings are stored in the K2 Database.

Identity Sync Service Configuration Settings Reference

This section lists the various configuration settings of the Identity Synchronization Service. All settings are stored in the K2 Database. See the following sections for details on the various configuration categories:

ETL Configuration
SyncEngine Configuration
Provider Instance Configuration

ETL Configuration

Configuration Location: [SyncEngine].[RepositoryConfig]
Configuration ID: 579E84D3-758B-42F6-A6DB-AB1BD63C0296
Configuration Settings:

Configuration Name	Description	Valid and Default Value	Change permitted
CommandTimeoutSeconds	The wait time (in seconds) before terminating the attempt to execute a command and generating an error.	Value should be greater or equal to 600. Any value less than 600 will be ignored and the default value will be used. Default : 600	Yes
DTUBackOffIntervalSeconds	The amount of time (in seconds) to wait before continuing with processing if DTULimitPercentage configured value is reached. Only applicable when using Azure SQL Database	Value must be non-negative and greater than zero. Default: 30	Yes
DTULimitPercentage	The DTU percentage before applying the DTU wait period configured for DTUBackOffIntervalSeconds . Only applicable when using Azure SQL Database	Valid Value range is 0 to 100 Default: 95	Yes
EnableAudit	Enables audit logging. The following information will be logged into the table [SyncEngine].[_IdentityAudit] Timestamp DTU end time (only when using Azure SQL Database) DTU percentage (only when using Azure SQL Database) Task Row returned Default: false	Valid values: true false	Yes
IntervalSeconds	The frequency (in seconds) of checking for any new Identities\Membership to ETL from the SyncEngine schema to Identity Cache.	Value must be non-negative and greater than zero. Default : 60	Yes
IdentityBatchSize	The maximum number of identities to process before committing to the database and continuing processing. Setting this value to small will result in higher number of SQL Connections. Setting this value too large will result in a high memory usage	Value must be non-negative and greater than zero. Default: 5000	Yes
IdentityState	Used to store the state (Watermark) of the last Identity proceed. Setting this value to small will result in higher number of SQL Connections. Setting this value too large will result in a high memory usage		Only if instructed by K2 Support
MembershipBatchSize	The maximum number of identity members to process before committing to the database and continuing processing. Setting this value to small will result in higher number of SQL Connections. Setting this value too large will result in a high memory usage	Value must be non-negative and greater than zero. Default: 25000	Yes
MembershipState	Used to store the state (Watermark) of the last Membership proceed. Setting this value to null will result in the processing of all the membership records in SyncEngine Schema to the Identity Cache		Only if instructed by K2 Support
ServerId	For future use		No

SyncEngine Configuration

Configuration Location: [SyncEngine].[RepositoryConfig]
Configuration ID: BB31888E-7115-4EE0-895E-5A0E89336A2F
Configuration Settings:

Configuration Name	Description	Valid and Default Value	Change permitted
CommandTimeoutSeconds	The wait time (in seconds) before terminating the attempt to execute a command and generating an error. This setting is only applicable to K2 Five Installation(s)	Value should be greater or equal to 600. Any value less than 600 will be ignored and the default value will be used. Default : 600	Yes
SyncBatchSize	The maximum number of identities/Identity Properties/Identity Links to process before committing to the database and continuing processing during flushing of Instructions. This setting is only applicable to K2 Five Installation(s). Setting this value to small will result in higher number of SQL Connections. Setting this value too large will result in a high memory usage	Value must be non-negative and greater than zero. Default: 1000	Yes
SyncProviderInstructionPageSize	The maximum number of Instructions to build during the Provider Sync Stage before flushing to the repository and continuing processing. This setting is only applicable to K2 Five Installation(s)	Value must be non-negative and greater than zero. Default: 500	Yes
SyncProviderSlots	The maximum number of Provider Syncs to run on a K2 Server Instance concurrently. This setting is only applicable to K2 Five Installation(s). If you have a large amount of enabled Provider Instances configured on your environment, Setting this value too large will result in a high memory usage	Value must be non-negative and greater than zero. Default: 2	Yes
SyncProviderInstanceDefaultIntervalMinutes	Indicates the default schedule value to use when Provider instances are created via the Security Label component upon SharePoint K2 Activation per site collection. This setting is only available from K2 Five 5.2 CU1 Fix Pack 22 and K2 Five 5.3 Fix Pack 17	Value must be non-negative and greater than zero. Default: 15	Yes

Provider Instance Configuration

Configuration Location: [SyncEngine].[ProviderInstanceRuntimeConfig]
SmartObject Category: System\Sync Service
SmartObject Name: Provider Instance Runtime Config

AD Provider Runtime Configuration

Configuration Settings:

Configuration Name	Description	Valid and Default Value	Change permitted
Path	This is the LDAP path of the target domain (effectively a connection string). The exact value you need to enter will depend on your AD configuration; check with your AD administrator to determine the LDAP path for the target domain. This is a required field	Value must be in the format of LDAP://[DistinguishedName].	Yes
NetBiosName	This is the NETBIOS name of the domain. Locate this name by querying the general properties of the domain using the Active Directory Domains and Trusts tool. This is a required field		Yes
UserName	If your LDAP directory does not support integrated connections or you want to use an account different from the K2 service account, specify a user to query the directory. This is an optional field. Also see the Connect to a domain using a specific user KBT for more information		Yes
Password	The password for the user specified for UserName configuration setting. This is an optional field		Yes
ADCache	Not used and can be ignored/excluded when registering new AD Provider Instance. Currently only used by the Legacy Resolver Manager		Yes
IgnoreForeignPrincipals	Not used and can be ignored/excluded when registering new AD Provider Instance. Currently only used by the Legacy Resolver Manager		Yes
IgnoreUserGroups	Not used and can be ignored/excluded when registering new AD Provider Instance. Currently only used by the Legacy Resolver Manager		Yes
LDAPPath	Not used and can be ignored/excluded when registering new AD Provider Instance. Currently only used by the Legacy Resolver Manager		Yes
MultiDomain	Not used and can be ignored/excluded when registering new AD Provider Instance. Currently only used by the Legacy Resolver Manager		Yes
OnlyUseSecurityGroups	Not used and can be ignored/excluded when registering new AD Provider Instance. Currently only used by the Legacy Resolver Manager		Yes
ResolveNestedGroups	Not used and can be ignored/excluded when registering new AD Provider Instance. Currently only used by the Legacy Resolver Manager		Yes

AAD Provider Runtime Configuration

Configuration Settings:

Configuration Name	Description	Valid and Default Value	Change permitted
aad.tenantDomain	Tenant Domain Identifier. This is a required field		No
aad.oAuthResourceId	OAuth Resource ID for the OAuth calls. This is a required field		No
aad.oAuthResourceAudience	OAuth Resource Audience for the OAuth calls.		No
aad.jsonMaxPageSize	Value indicating the max amount of JSON pages to retrieve before building and writing instructions. This is an optional field	Value must be non-negative and greater than zero. Default: 1024	Yes
aad.logJsonResponse	Log JSON from the Query Response during sync. This is an optional field and used for debugging purposes	Valid values: true false Default: false	Only to be enabled if requested by K2 Support
aad.httpClientTimeoutInMinutes	This is to overwrite the default AAD HttpClient timeout. This is an optional field		Yes

SP Provider Runtime Configuration

Configuration Settings:

Configuration Name	Description	Valid and Default Value	Change permitted
sp.oAuthResourceId	OAuth Resource ID for the OAuth calls. This is a required field		No
sp.oAuthResourceAudience	OAuth Resource Audience for the OAuth calls.		No
sp.siteUrl	The URL to the root of the SharePoint site. This is a required field		No
sp.callTimeoutMs	This value determines the length of time in milliseconds before the SharePoint client context disconnects.	Value must be non-negative and greater than zero. Default: 5000(K2 Five) / 20000 (K2 Cloud)	Yes
sp.changeLogRetentionPeriod	The changelog retention period is configured in the Resource Throttling section for Web Application properties in Central Administration. Valid configuration values range: 1 - 18250 This is an optional field	Valid Value range is 1 to 18250 Default: 60	Yes
sp.credentials	For future use		No
sp.version	Not used and can be ignored/excluded when registering new SP Provider Instance. Currently only used by the Legacy Resolver Manager		No
sp.isOnline	Not used and can be ignored/excluded when registering new SP Provider Instance. Currently only used by the Legacy Resolver Manager		Yes

Identity Sync Service: Deep Dive

This section offers supplementary information including, an overview, best practices and troubleshooting.

Deep Dive

High-level overview

Components

When you are working with the Identity Sync service in K2, you should understand the components and functions of each component.

Providers	Responsible for querying the Identity Providers (IdP) that store the identity information (for example AD, AAD, and SharePoint) and sending the results to the Sync Engine.
Sync Engine	Executes the Providers, and then transforms and stores the results in an intermediary table (intermediate cache).
ETL	Transforms the Sync Engine data to identities (for example users, groups, group membership and managers) that K2 can consume, and stores them in the Identity schema.
Identity Schema	The Identity Schema is the runtime cache of identities that are consumed by K2.

Full Sync vs Differential Sync

A full sync (or the initial Sync), syncs all IdP data at that moment in time. The system saves a change token (also known as a delta link/watermark) to keep track of this state. If the Sync is successful the system saves the watermark, ensuring that no IdP data is incorrect because of failures.

Subsequent syncs will be differential syncs, sending the change token to the IdP to request only the changes made after the previous Sync. Differential syncs are fast and efficient if they occur frequently, and K2 recommends that the sync schedule should be as short as possible (the default is 15 minutes).

Only AD, AAD and SharePoint support differential syncs. Other providers such as LDAP, custom User Managers and SQLUM do a full sync every time. If you have many identities in SQLUM for example, each scheduled sync takes as long to run as the initial sync. Take that into consideration when configuring schedules for IdPs that do not support differential syncs. You may want to schedule syncs less frequently for IdPs that do not have volatile user data.

SmartObjects

The SmartObjects used by the Identity Sync Service are available in K2 Management under System > Sync Service.

SmartObject	Notes
Run History	Use the Run History SmartObject method to see the status and time that your last sync ran.
Operation	Use this SmartObject to manually start a full sync, to set an ongoing sync schedule or to get a list of existing schedules. When performing the initial sync, setting the Skip Deleted property to yes will not sync users deleted from the IdP. Performing the initial sync and setting the Skip Deleted property to no will sync deleted users, but in a disabled state. If these users' accounts change, the changes will sync when the sync schedule triggers.
Provider Instance	Use the methods of the Provider Instance SmartObject to list existing provider instances and see associated information about them. Execute the List Provider Instances method to see provider instance names and additional information. Use the provider instance name to start a full sync and to scheduled ongoing sync.
Provider	Use the methods of the Provider SmartObject to list existing providers and to see associated information. Execute the List Providers method to see provider names, types, and if they are enabled.

Best practices

K2 recommends doing the following SQL maintenance before and after the initial Sync:

Database backup and shrink
Check indexes and re-index/reorganize index.

Database backup and shrink

Backup

Right-click K2 database and select Tasks > Back Up
Ensure that the Backup type is on Full and click OK to complete the backup.

Shrink

Right-click K2 database and select Tasks > Shrink > Database
Ensure that it’s the correct database and click OK to complete the shrink.

Check indexes and re-index / reorganize index

Although K2 does have the built-in capability to re-index data, we suggest doing the re-indexing/reorganizing of indexes one-by-one, especially on PROD environments, to minimize the performance impact on the environment.

SQL query to check index fragmentation:

SELECT dbschemas.[name] AS [Schema]
       ,dbtables.[name] AS [Table]
       ,dbindexes.[name] AS [Index]
       ,indexstats.avg_fragmentation_in_percent
       ,indexstats.page_count
       ,indexstats.index_type_desc
       ,[ActionRequired] = CASE
              WHEN indexstats.avg_fragmentation_in_percent > 5
                     AND indexstats.avg_fragmentation_in_percent < 30
                     THEN 'Reorganize Index'
              WHEN avg_fragmentation_in_percent > 30
                     THEN 'Rebuild Index'
              ELSE 'None'
              END
FROM sys.dm_db_index_physical_stats(DB_ID(), NULL, NULL, NULL, NULL) AS indexstats
INNER JOIN sys.tables dbtables ON dbtables.[object_id] = indexstats.[object_id]
INNER JOIN sys.schemas dbschemas ON dbtables.[schema_id] = dbschemas.[schema_id]
INNER JOIN sys.indexes AS dbindexes ON dbindexes.[object_id] = indexstats.[object_id]
       AND indexstats.index_id = dbindexes.index_id
WHERE indexstats.database_id = DB_ID()
       AND dbschemas.[name] = 'SyncEngine'
ORDER BY indexstats.avg_fragmentation_in_percent DESC
       ,[Schema]
       ,[Table]
       ,[Index]

Reorganize Indexes

For the above query, for every ActionRequired that states Reorganize Index, you need to run the following query:

ALTER INDEX [<Index name>] ON [SyncEngine].[<Table name>] REORGANIZE;

Re-Indexes

For the above query, for every ActionRequired that states Rebuild Index you need to run the following query:

DBCC DBREINDEX ('SyncEngine.<Table name>', '<Index name>');

Troubleshooting

When troubleshooting issues with the Identity Sync service, try to identity where the error is occurring in the stack of components. The table below lists some common issues or items to check on each component in the stack.

Provider	Is there a change token error/unauthorised error from SharePoint, AAD, AD, etc.? The issue is in the Provider if the Sync schema and the IdP data does not match up. For on-prem: check the SyncEngine.RunHistory table for Sync failures (or use the Management SMO - Run History) For Cloud Ops: check Badger.SyncAudit table for any failures
Sync	Is Sync not completing? Is it a more generic error while syncing – Sync engine bug For on-prem: check SyncEngine.RunHistory for Sync failures (Management SMO) For cloud Ops: check Badger.SyncAudit table for failures Check HostServer Logs for errors Check ETW logs for errors
ETL	Run the Script to verify that ETL has completed (SyncEngineGetETLStatus) Check ETW logs for errors
When something is wrong, but there are no errors	Exactly what happened before running a Sync? Was the opt-in installer run? Was it run before/after? Are the flags correct (ResolvingEnabled = true/false)? Compare data from IdP to SyncEngine to Identity.Identity, are there differences?

When logging a ticket:

Run through the troubleshooting steps above and provide the info you discovered.
Provide the HostServer.UpdateHistory data
If there are deadlocks, provide deadlock graphs (in SQL profiler)
If known, provide the Environment flavor and setup (multi-node/single domain/on-prem using AAD/etc.)?
What are the Provider types or combinations of provider types you are using?
Screenshots from the IdP vs screenshots of Sync schema/Identity schema
SyncEngine schema and Identity schema (obfuscation script if required)
Script to check for stale data:
EXEC [SyncEngine].[CleanIdentityStaleData]
Script to check status of ETL
EXEC [SyncEngine].[GetETLStatus]

Considerations

When opting-in to make use of the new sync service, make sure that you are on the latest fix pack and CU.
If you add a new provider to your system, you must run an initial first sync on it, and then configure a sync schedule for it.
By default, K2 limits the active sync jobs to two. When a sync finishes, the next one starts. You may want to stagger scheduled syncs based on the average runtime of scheduled sync tasks.
After the first sync, there may be a discrepancy between the number of identities in the runtime cache compared to the intermediary cache. This is normal because there may be identities cached from other IdPs like SharePoint that you have not yet synced.
Data in the intermediary table is transformed and added to the identity cache one minute after a sync.
If you install the sync service but do not run an initial sync, you may get errors in K2 because identities from the IdP are not in the cache. A typical error is "An identity for the user {FQN} could not be found." where FQN is the fully-qualified name of the identity with which you're logging into K2 sites and tools.
If possible, you may want to run the initial sync at a time when the K2 environment is used less frequently by your users. This can help to mitigate users experiencing the "An identity for the user {FQN} could not be found" error while the initial sync runs.
If you decide to revert to the prior method of identity syncing, you must log a support ticket.
- If you have been using the new Sync service for a while and you switch back to the old way of resolving – all identities expire
- You will need to perform a full sync again
- This is not recommended
When using K2's Identity Synchronization and Caching with a user who has both AD and ADFS security labels, if the user account gets disabled in AD, the ADFS label stays enabled. However, when this happens, the user cannot be authenticated or access anything in K2. This is expected behavior, as the AD label deals with the authentication.
If you upgrade to K2 Platform Classic (5.4) and are using the Identity Sync Service, you may notice error messages logged to the Host Server log file relating to SharePoint. This is especially true if you sync an ADFS provider. For more information see the KB article KB003618 Code Fix: When running a sync on an ADFS provider, multiple errors are logged.
When you want to use ADFS users in SharePoint groups, you need to complete a few steps to ensure that the K2 SyncEngine handles their membership correctly. For more information see the KB article: KB003638: How to map MLM Mappings for SharePoint to ADFS from K2Sync.

Known Issues

If you are using K2 Five and Azure Active Directory as an identity provider, contact K2 product support before running the first sync. A known issue exists where calls to the UMUser SmartObject return incorrectly formatted values for the Manager field. Support will check if your environment needs the fix and provide the fix if necessary.
If your K2 environment has a cumulative update containing the Microsoft Graph changes installed (EG: K2 Five (5.5) November 2021 CU), you must run an additional script to use the Sync Service with AAD.
Managers remain after delete - When you delete a user who is also a manager of another user, the manager-direct report relationship still exists in the identity cache. A future release will address the removal of the manager-direct report relationship when the manager is deleted.
When syncing with AD FS, only the first 1000 records (identities or groups) are returned by default - To sync all the AD FS identities, change your configuration as described in KB002721 - How To Configure ADFS To Return More Than The Default 1000 Records
After installing the sync service and executing synchronizations, the K2 database file size may grow excessively. This is most likely due to the transaction log in SQL. In this case, consider performing a SQL database backup and shrink (or other SQL maintenance tasks) to reduce the size of the transaction log file.
Groups in AAD with the same name are cached as a combined single group in K2. See the KB003504 article for more information.