Help! K2 Workflow freaked out on me when restarted!

  • 3 January 2018
  • 8 replies
  • 52 views

Badge +8

I have a simple workflow to send reminder emails to external clients.  A screenshot of it is below.  

 

Running it today for the first time in PROD on the 4th record of the "For Each" event, there was a missing date field which caused the workflow to fail.  (as part of the for/each parameters it is looking for records with a date = TODAY()).

 

So...I went ahead and updated the date field and put a date in there, then clicked "RETRY" on the Management console. 

 

Well, BIG MISTAKE!  The workflow went into an infinite loop and sent out multiple identical emails over and over again.  There seemed to be little rhyme or reason as to what records were getting repeated. I had to stop our PROD service instance (whcih got me into huge trouble with the bosses) (and by the way I think is a HORRIBLE bug in K2 where clicking "Stop" on a a workflow really does not stop it...) in order to stop the workflow...plus dozens of external customers got multiple emails. If I had not been moniitoring the box that got the "CC" emails, this potentially could have sent THOUSANDS of emails.  

 

Anyway, I'm a little freaked out and I hesitate to try this again for fear of getting fired if it gets screwed up again.

 

Any idea what caused this to happen?  

 

When I run the workflow in our test environment with the offending date field NOT blank, it runs perfectly...sends the required number of emails...1 per record...is retstarting a stopped workflow with a "for each" type loop in it a bad idea?  

 

Guidance is appreciated.

 

And seriously....that stop workflow button on Management console SERIOUSLY needs to work...I hope this is fixed in 5.

 

 

Rob

 

16740iE88511CA5EC3D0A4.jpg

 


8 replies

Badge +15

Hi,


 


I have come across the same thing in the past. If I have a workflow that has some sort of loop in it, if it is stuck in that loop, you can never be able to stop it from the Management console. There is a reason for this, but I can't recall it. Maybe someone here can help explain this.


 


Anyway, if you ever enconter the same thing again, you can try the following:


 


1. Stop your K2 blackpearl service instance


 


2. Go to your K2 database, look for the [Server].[ProcInst] table


 


3. If your process instance is stuck in a loop, it will likely be in the running state (Status = 1). So you can use this script to narrow down the list:


 


SELECT *


FROM [Server].[ProcInst]


WHERE [Status] = 1


 


4. Once you find the process instance with the issue, change the status to 4.


 


5. Restart your K2 blackpearl service instance, then check your Management console if the process instance has stopped. 

Badge +8

Yes, this has happened to me before as well, and K2 tech support turned me on to the same solution to get the workflow to stop.  Horrible bug...needs to be fixed.

 

 

Userlevel 2
Badge +9

I can tell you why clicking the "stop" button in Management doesn't work to stop a looping workflow, I think. Basically, the workflow instance is "open" and using a worker thread until it gets to a stopping point. I believe this effectively locks the record in SQL. Clicking the "stop" button is supposed to use the API to stop the instance, but it is locked to the worker thread that is trying to finish executing its instructions.


 


This is why we end up having to stop the K2 Host Server and update the instance directly in the Database to a non-running status (per "boringNerd's post).


Let me look in our systems and verify that this is on the backlog to be addressed. I agree that being able to stop a runaway instance without having to stop the K2 service is a very good thing.


 


Best regards,


Gail

Userlevel 2
Badge +9

To follow up on my post last week, here is a feature request link you can "up-vote" to allow you a UI-based means of stopping runaway workflows:


https://ideas.k2.com/ideas/APIT-I-109


 


I can also confirm that this topic has been discussed with our Development and Product teams, and they understand the need. It is something they are working on, but I always recommend people adding their weight to things on the IDEAS site. 


 


Thanks, and regards,


Gail

Badge +8

Thanks, Gail. I appreciate the input.

 

I don't, however, understand why one has to 'up-vote' getting bugs fixed.  This is not a feature request...this is getting something to work that currently does not, and it should be addressed regardless of how many votes it gets, no?

 

Rob

 

 

Userlevel 2
Badge +9

Rob,


The bug is that we allow you to design workflows that can infinitely loop, and that is not anything you need to vote on. 


The "up-vote" is for the feature request to present you with a UI-based mechanism of stopping such runaway workflows.


At least, that's how I understand it is currently classified.


 


Basically, we have bugs filed for various scenarios to try and prevent people from creating designs that are likely to cripple the server. The feature request is to make it easier for you to stop things, should that actually make it through.


 


I hope that helps clarify. I know that significant work has already gone in on the new HTML Designer in K2 Five/Cloud that significantly reduces likelihood of creating looping workflows, and more work is planned.


 


Best regards,


Gail

Badge +8

I'm probably just being too simple here, but the managemnet console has a "stop workflow" button that does NOT work when the workflow is in an infintie loop.  To me, that's a bug.  The button doesn't say, "Stop Workflow UNLESS the workflow is in a an infinite loop".  The UI for this is already there...it just doesn't work.

 

To me, that's a bug.

 

 

Userlevel 2
Badge +9

Rob, 


I see your point. Thank you for the feedback. I believe my wording was also not ideal to convey the correct information before. There is definitely work being actively done in both K2 Cloud and K2 Five to address this very scenario. The Idea was a secondary means of "getting around" the issue.


 


Thank you again for your input. It is invaluable in driving improvements in our products and services. 


 


Best regards,


Gail

Reply