Ever unintentionally create a recursive/infinite loop in Automation Cloud?

  • 23 February 2024
  • 12 replies
  • 76 views

Badge +2

Hi all. Wondering if others have had the misfortune of creating a recursive workflow/infinite loop in Automation cloud?  If so, how did you detect it and how many workflows did it trigger?  


12 replies

Userlevel 1
Badge +5

Hi Mark

 

The easiest would be to look at the Analytics : https://analytics.nintexcloud.com/analytics/workflows/instances


 

Badge +2

Yeh, but if you don’t know you’ve created it you don’t know to look at Analytics.  Is there any way to be warned when a workflow goes rogue?

Userlevel 4
Badge +14

Hi all. Wondering if others have had the misfortune of creating a recursive workflow/infinite loop in Automation cloud?  If so, how did you detect it and how many workflows did it trigger?  

Is this from a SharePoint start event workflow? 

If so we usually put a condition on it to not trigger if modified by user is equal to the account using the connection. However we force everyone to use service accounts for all our connections in NAC so we can control that. Designers don't have the ability to create their own connections. 

If you don't there Is another method in the sharpeoint list you can use but before i explain I'd want to know if its SharePoint related. ;)

Userlevel 4
Badge +14

FYI ...My response only prevents it from happening. 

 

I would not know how to detect it though in NAC other than checking instances as a global admin every morning like we do.

Userlevel 1
Badge +5

I may stand corrected, but I dont think there is a notification for this OOB.

We can certainly work on a daily instance report(scheduled workflow) which would be a more proactive way to determine if this is happening.

Userlevel 4
Badge +14

I may stand corrected, but I dont think there is a notification for this OOB.

We can certainly work on a daily instance report(scheduled workflow) which would be a more proactive way to determine if this is happening.

That would be helpful i would think for alot of customers. As we experience the same issue. :(

Badge +2

Yeh, it was a SharePoint list.  In addition to not repeating the mistake, my concern is we only picked it up coincidentally and it had already run 6,000 executions over a few days - using more than half of our licensed quota.  What would have happened if we didn’t notice it?

Badge +2

Thanks SteveBarnard.  A daily instance report would certainly be helpful and help manage the risk of rogue workflows.  Do you have something you could share?

Badge +2

 

I may stand corrected, but I dont think there is a notification for this OOB.

We can certainly work on a daily instance report(scheduled workflow) which would be a more proactive way to determine if this is happening.

Thanks SteveBarnard.  A daily instance report would certainly be helpful and help manage the risk of rogue workflows.  Do you have something you could share?

Userlevel 6
Badge +22

Yeh, it was a SharePoint list.  In addition to not repeating the mistake, my concern is we only picked it up coincidentally and it had already run 6,000 executions over a few days - using more than half of our licensed quota.  What would have happened if we didn’t notice it?

I am unsure what you mean by “using more than half of our licensed quota.”.

The only quota for a license is the number of workflows that can be published.

There is a limitation of workflow executions based on the license type, but if you have 6000 executions over a few days, you are nowhere near this limitation.

Badge +2

Yeh, it was a SharePoint list.  In addition to not repeating the mistake, my concern is we only picked it up coincidentally and it had already run 6,000 executions over a few days - using more than half of our licensed quota.  What would have happened if we didn’t notice it?

I am unsure what you mean by “using more than half of our licensed quota.”.

The only quota for a license is the number of workflows that can be published.

There is a limitation of workflow executions based on the license type, but if you have 6000 executions over a few days, you are nowhere near this limitation.

Hi Simon.  We are on the “new” license structure whereby we are not restricted to the number workflows but to a set number of ‘executions’ across our account.  We were told all accounts will be transitioning to the per-execution license model, that we can’t move to a per-workflow license, and that the new model was in response to “feedback gathered from the install base”.

Apparently, we are also one of the only unlucky (or silly) customers who have had problems with errant workflows that resulted in large numbers of executions.  The first time we ran up a bill in excess of $100,000.  We caught the second one purely by coincidence.

The challenge for us is how to mitigate the risk of rogue workflows when:

  1. there is no ability to set limits on how many times a workflow executes – either overall, or over a time period;
  2. there is no warning when a workflow may be executing excessively;
  3. there is no warning/notice when a set number of workflows is executed against an account;

  4. there is no warning/notice when you approach or exceed license allocations (in fact it appears there is no upper limits to executions);

  5. there is no ability to receive alerts/automated updates on how many workflows have been executed across an account (note we have had two Nintex specialist try to construct a workflow that does this, and neither has succeeded); and

  6. the only way to check how many workflows have been executed across an account is to manually check in Nintex Analytics.

Any ideas or advice is appreciated as we otherwise love the product and don’t want to ditch it, but the risk at present is very high. I’m surprised others haven’t had the same problem, but apparently we are the only ones to have raised it so far.

Userlevel 4
Badge +10

We’ve not had this problem before, but if it has cost you that much in the past, it may be worth developing something with the the NAC API. Could just be a simple script that queries a list of all workflow instances for the current day, and if it’s above a certain amount, alerts you that something may be going haywire. 

Reply