The display mode has only close button. There shouldn't be any other button because it is display mode.

The Cancel button will be automatically changed to close button on display mode

The name of the button does change to close in display mode, however it does not close the form. This is the issue.

Which version of the form are you using ?

Or did you change the button action

It's version 75.0. Can this issue arise when there are too many versions?

The issue preceded any changes to button actions. The changes were part of testing to see if other actions would work, as I have had issues similar to this which were resolved by activating the save button. However, on the same site I created a new library with a new form (currently version 3.0) which is presenting with the same issue.

I can't find anything online about this kind of issue, nor can I find any coding to stop the form from being closed, but I can't work it out either.

I need to know the nintex version , not the form version . There is no problem if you have many versions for the form.

Did you check the setting for the button. Can you post the screen shot for the buttons.

It's Nintex 2013. The settings for the cancel/close button are their default values. 

Any changes were made to new buttons which have been deleted.

Ummmm, I tried it and reproduced the issue, seems it is bug.

Awesome...time for Nintex support!

Just to confirm, when you recreated the issue, was it on a picture library as well?

Yes, it is for picture library