Hello I have been searching the community but haven’t found anything to help me.
I am working on a project where we have migrated a very large document Library (around 75,000 items) from another system into SharePoint. There is a requirement to set Item level permissions for each document in the library.
The rules which determine the correct group/permission level to grant are held in a separate list. I have created a Nintex Workflow 2013 list workflow to set permissions on the document library items based on the rules defined in the list. This workflow performs as expected.
Then to use that workflow to set permissions for every item in the library I created a site workflow that loops through the whole document library and starts the list workflow on each item using a call web service action. This worked well in my initial tests with a small document library.
When I moved my test to a large 75,000 item library I started getting timeout errors in the ULS logs with varying degrees of success. The workflow would run through several loops setting permissions correctly then I would get a message saying workflow failed to start in the workflow history.
I was initially using the StartWorkflowOnListItem Web method without using error handling and permissions were correctly set on between 300 to 500 items per test run.
Next I configured Error handling on the call web service action. My next tests successfully set permissions on around 8,000 items before the same error occurred.
Looking at the history list it looks like the workflow is failing at the call web service action. Even though I get the message that the workflow failed to start, the state of the workflow remains as in progress until I terminate the workflow.
Next I added a pause to the loop so the workflow would pause after every 500 items. Now I can’t get through more than 25 items before getting the same error.
I Changed the Web method in the Call web service action to StartWorkflow. And my current test is at around 2000 items without failing but it has taken about 4 hours.
Does anyone have any tips that could help me out? Even if my current version works it will take years to finish.
Any help would be greatly appreaciated. I will add some screenshots of the workflows below.
this one is the site workflow that starts the list workflow on each item.
This one is the list workflow.
Solved! Go to Solution.
you may be running into the ACL limit for the list, if you haven't yet, you will. See the Security scope section of https://technet.microsoft.com/en-us/library/cc262787.aspx#ListLibrary
Thank you for the response.
I see the max of 5000 ACEs per ACL but wouldn't each item in the Library have a separate ACL?
My current test run has correctly set permissions on about 13,000 items so the 5,000 ACE limit would have been hit a while ago if it is per library.
that's very common that if one generate too big batch to process by sharepoint workflow fails.
your approach of splitting overall set into smaller chunks is correct. you need to commit and/or pause within loop after each reasonable sized chunk.
apart from that, I would try to replace web service call with start workflow action with 'wait for workflow finished' option activated. this way you serialize execution of child workflows. it will definitely run longer, but there will run less workflows in parallel hence lower (over)load on sharepoint.
reg. performance - set permission operation is slow on its own.
I assume you start site workflow manually, so I'd suggest to add a pause action at the very beginning.
maybe some other approach (eg. some powershell script) might be more efficient
Thank you for your advice.
I have given up on using the Nintex site workflow and created a powershell script that seems to be working well.
It can be hard to wrap your head around it, but the hard limit is 50,000 and they suggest that you keep it to 5,000 or less for performance. Individual permissions in a large list or library will lead to performance problems. Batch size becomes an issue and as the batches get larger it takes increasingly more time. Monitor SQL while running this in either a workflow or Powershell and you will see the issues. Securing things at a folder level helps reduce the number of securable objects, as does defining additional libraries.
See my comment above, while the powershell run time may have different constraints than a Nintex workflow you still have some underlying hard limits. You will need to find another solution long before you get to the 75,000 items.
You are correct,
My PowerShell worked great up to 50,000 items. (Yes the performance on the library was very poor with 50,000 individual item permissions.)
I tried changing the scope limit with this PowerShell,
$webapp = Get-SPWebApplication http://<serverName>
$webapp.MaxUniquePermScopesPerList = <Number of scope limit>
After changing the scope limit to 100,000 items I can no longer open the library.
My next step will be re-structuring the library using folders and multiple libraries.