I had someone report a similar issue today.  They were unable to create a new form or edit an existing form.  When clicking the Nintex Forms button in the ribbon, the forms builder workspace would try to load, but a prompt for windows credentials would appear.  If credentials were entered, the person received this Ajax error:

I tried it and instead of entering credentials, I cancelled at the credential prompt and received this error:

After some troubleshooting, we found that there was a lookup column on the list, which pointed to a list that had unique permissions.  The person building the form did not have permission to the lookup list.  Once he was added to the lookup list permissions, he was able to create the form without any credential prompting or error messages.

Thank You Brian.