I am new to the field of k2 so please have some patience with me.
My requirement is to add a button on to K2 form and this button is made using other programming language such as .NET etc.
The purpose of this button is to start scanning of documents and get the resultant documents onto K2 form.
In short I need to integrate the scanner with K2.
I hope my query is clear and kindly do provide help on this requirement.
You can use the endpoints assembly to call into .NET code - https://help.nintex.com/en-US/k2five/userguide/5.0/default.htm#ServiceBrokers/EndPoints/EndPoints_As... and then just call a SmartObject method from a normal K2 button.
But again all of this will be running on the server and not the end users machine.
If you have a .NET app then you can start a workflow using C# code on the client side, no need to have an endpoint assembly.
If I understand you correctly, you have created a win form app that runs on the users PC? As this is a different approach you can use a different solution.
In this scenario you don't need to use a service broker and can instead reference the K2 client API in your app, passing it the name of the server and the process you want to start. You could store the k2 server name and process name in the app.config and point to your dev / prod server as appropriate.
The thing is though you won't be able to run that client app on the server so unless the app scans across the network I don't think it'll do what you want.
Also if you take a look at the assembly endpoint documentation you'll see that you can only consume static methods in static classes in a dll only so the winform thing cannot be used in that manner.