First, I'm fairly certain I mostly understand the issue, I'm looking for information on working around, or fixing it.
The environment is an extranet that uses mix authentication (Windows NTLM and ADFS Claims) to allow internal and external users to access. We are having an issue in Nintex Workflows that attempt to use the "Call Web Service" or "Web Request" actions to start a workflow on a list item.
We have a scheduled Site Workflow that is batch processing child items and is supposed to start a list workflow in the item if conditions are met. This requires us to use the "Call Web Service" or "Web Request" actions to start the workflow on the items. This action is failing whenever the workflow is ran and returning a error that reads;
Error returned from server: The remote server returned an error: (401) Unauthorized.
We've tried using a user account, as well as a service account and they all return this error. The weird thing is, when using the "Test Connection" button in the action returns a "OK" message, and the "Refresh" button in the action returns the list of available web services to populate and be selected. It is only when the actual call is attempted that it returns with a (401) Unauthorized response.
When researching, we found a few things pointing out that Web Service calls on a SharePoint environment using mixed or claims based authentication may not work.
This second post (https://community.nintex.com/docs/DOC-3221) gives the option to use a X-FORMS_BASED_AUTH_ACCEPTED header to force Windows Authentication, but when I attempt to use it I still get a 401 Unauthorized response.
I finally found this write up (http://www.sharepointblogs.be/blogs/vandest/archive/2014/09/12/calling-web-services-in-nintex-workflow-and-different-authentication-mechanisms.aspx) and tested using a Web Request action and passing a FedAuth Cookie value as a header. This Worked! The action ran successfully. However, since we are using a service account and scheduling the Site Workflow, I'm not sure if using this workaround would be a valid solution. How would a FedAuth Cookie be retrievable for a service account when the workflow is activated by a schedule and not a user's browser?
The other option of using a different URL Zone is a big change to the client, and something that I'm hesitant to recommend.
Has anyone else solved this sort of issue? Did you use one of the above methods or is there a better way to fix this?