I do not believe that the Check In/Out system is available for SharePoint Lists, and is only enabled for Libraries, which might throw a wrench into your plans if you're strictly working with a list.
So if you are working with a List then I would maybe just create a new column called something like "Claimed By", which can be populated *only* by a workflow ran on the List Item itself. The workflow would just check to see if the column was blank, and if it was, insert the name of the user who was invoking the WF.
In the Form for the item I would then create validation to invalidate the form is the:
{CurrentUser} !== Claimed By
You could also add a little Formatting Rule to make a message appear if the two values are not matched.
Lastly you can make the Submit Button clear the value from the Claimed By column upon submission so that there's no additional work needed to "release" the item.
This also lets people see at-a-glance if the Form is being worked on by someone else.
Let me know if this sounds like a good solution or if you need help implementing it.