Nintex forms - how prevent user submit same data

You should be able to use the lookup function with a rule to determine whether or not the current user has submitted an item already. 

I had a form where the user was only supposed to submit one form, and any changes should be made to the already existing form. They should not submit a new one. So, to accomplish this, I setup a rule that disabled the Submit button and all fields while showing a red text message that said something like "You've already submitted a form. Please modify your previous form." This forced them to click the Cancel button or simply close the form.

Here's the rule I used on the Submit button and fields to disable them :

not(isNullOrEmpty(lookup("Employee Work Schedule","OriginatorSig",userProfileLookup(Current User,"PreferredName"),"OriginatorSig")))&&Is New Mode

Here's the rule for the red text to hide unless a form already exists for the current user:

isNullOrEmpty(lookup("Employee Work Schedule","OriginatorSig",userProfileLookup(Current User,"PreferredName"),"OriginatorSig"))||not(Is New Mode)

Now, in your situation, I don't know if your users can only submit one form, OR if they can only submit one form per unique value in a specific field. If the latter is the case, your rules will be a bit more complicated, but you should still be able to achieve your goal.

Thanks for quick reply, my users can submit one form per unique value in a specific field.

Any ideas how going on? ;)

Honestly, that's going to be interesting. I think you're going to need a rule similar to mine used for disabling the Submit button. But, you're going to need one for each of the fields, and it's going to need to do a look on the list for that field's value and for the current user. 

Off the top of my head, I think the rule is going to need to be a nested lookup, because the lookup function can only filter on one field. So, it would probably be best to first lookup based off the current user to find all the forms they've submitted and return that specific field. Then, put that inside another lookup that filters the results of the first lookup for the current control's text value to see if it exists in the results.

That's just in theory though. I haven't tried it.