I am in the process of cleaning up our workflow history list which was previously at 4.7 million items. Down to 1.4 now woohoo. Anyways, this cleanup is completed through a backend powershell script that Microsoft assisted with.
With recently getting Nintex I came across the "purge" action and was hoping I could learn a little more about it. When going through Site Settings > Nintex > Manage workflow history lists I became curious if this may be easier than using the powershell script.
Would someone please explain to me what purging actually does in this instance? How would this affect the performance of the site? How long does it take?
Also noticed that the "Enabled for Nintex Workflow" is disabled for the SharePoint Workflow History list, would purging still work on this or is it only supposed to be utilized for Nintex workflows.
Appreciate the insight.
Solved! Go to Solution.
I'll encourage you to hit the blue Help button you see there to bring up the details you are looking for. Let me post a few things from the help document that can provide a few answers you are looking for.
You are on the Manage Workflow History Lists settings for a site.
The Manage workflow history lists provides the ability to create new SharePoint workflow history lists and enable existing SharePoint workflow history lists for use with Nintex Workflow within a team site.
A SharePoint workflow history list stores messages generated by a workflow. Everything from approval activity (approve/reject), updates to a workflow, comments, cancellations and even workflow errors.
By default Nintex Workflow creates a SharePoint workflow history list that is enabled for Nintex Workflow. The enablement deploys event receivers into the list for workflow reporting. The standard SharePoint workflow history list cannot be used with workflows in Nintex. To use a standard SharePoint workflow history list it must first be enabled for Nintex Workflow.
Purge items from the workflow history lists
The Manage workflow history lists also provides the option to delete workflow history messages from the workflow history list on a site. This option is also available as an NWAdmin operation. This option can be used for maintenance on the history list to prevent the number of history items exceeding 2000 which can have performance implications. Deleting history messages does not affect the operation of a running workflow.
Check out the full article in the help guide for the full instructions on how to perform the purging actions
Purge Workflow Data
I wanted to point out that there are other purging methods beyond the history lists. You can also manage the Nintex Workflow Databases.
The Purge workflow data provides the option to delete historic data from the Nintex Workflow database. This option is also available as a NWAdmin operation. Reasons that data may need to be removed:
Workflow data can be purged globally for the web farm, a specific site collection or individual sites.
So after you remove history list information, eventually you will also want to manage the databases. Especially if you only have one Nintex workflow content database and not multiples for your site collections.
Other important blogs you may be interested in: Demystifying Workflow History (Part 1) Defensive Workflow Design Part 1 - Workflow History Lists Database Capacity Planning: Nintex Workflow Database Mapping
Thanks Andrew to write the definitive explanation of purging workflow data.
I have experienced catastrophic performance issues with large history lists leading to the crash on some low perf servers.
The purge management should be mandatory part of any workflow implementation.
Do you have ps1 or bat to regularly purge with nwadmin ? Do you base your purge on dates, status, other ?
Yes, it should definitely be in the initial planning. I don't have the powershell scripts on me, but I did make a few for clients. You can run the NWAdmin commands right from a powershell shell, so it's simple to do. We created a windows scheduled task that ran the set interval we wanted, never more than once a month. If anything you would run it less often, maybe once in two months. The only real PowerShell I used was to fetch all of the site collections from a web app, then iterate through them so I can run the nwadmin commands reusing code.
Run these two commands
PurgeHistoryListData using the -lastActivityBefore and set the today to what you need. That way you have an expected date range. We kept the last 6 months, and the item count was always under 5000 on any lists. There are a lot of other switches for this command, check them out.
PurgeWorkflowData - make sure you use the purge history first. There is a side effect using this command, as you will no longer be able to have history list items associated with them purged using the ‘PurgeHistoryListData’ command unless the–clearall switch is used. This command also has a switch -lastActivityBeforeLocal
One other note, We found there were a few workflows generating a lot history that was necessary, but they were sharing the same history list. So we split them up to use different lists. This helped a lot.
Also make sure you explicitly limit deletions to Completed workflow instances. We've had a serious incident when PurgeWorkflowData deleted workflow log records from NW database for running instances, which caused all of those to fail in turn on re-hydration (trying to continue after a pause or on an event).