Check that the URL for the tab you are trying to close does not end in "?"

It has got something to do with your IE security settings. Please read the following blog posts for the same.

https://social.technet.microsoft.com/Forums/ie/en-US/db7ef036-e4*personal details removed*e7-bcee-1a245b8cc80f/ie-11-the-webpage-you-are-viewing-is-trying-to-close-the-tab?forum=ieitprocurrentver

https://answers.microsoft.com/en-us/ie/forum/ie8-windows_other/webpage-you-are-viewing-is-trying-to-close/9adc4ec8-6d62-e0*personal details removed*dfc-68b599b31bf5

https://stackoverflow.com/questions/17300705/ie-stop-message-the-web-page-you-are-viewing-is-trying-to-close-the-tab

Please try to do following and see if that helps.

I created a ticket for K2 and this is what they said:

The 'Close the browser window' rule has a small flaw, HTML 5 standards dictate that windows that have not been opened by a script cannot be closed by that script: https://www.w3.org/TR/html51/browsers.html#dom-window-close

Here's a community post regarding this issue: http://community.k2.com/t5/K2-blackpearl/Rule-quot-close-the-browser-window-quot-with-different-results/m-p/63350#M17734

This label for the rule does cause confusion and we currently have a feature request to change it to 'Close the Form'.

We also have a workaround:

Create a Literal, Hidden Data Label on your form, and when you wish to close the window, create a Transfer Data rule that transfers the following script into the Data Label:

<script>window.open('', '_self', ''); window.close();</script>

This script opens a blank window on the current page and then closes it, getting around the aforementioned issue.