Product: Kryon RPA
Product Version: 20.3 and above
Components: Kryon Studio
Article Contributors: Darren Cheng
Prerequisites: You'll need Kryon RPA v20.3 or above and a MS Azure account.
Steps:
https://docs.microsoft.com/en-gb/azure/cognitive-services/form-recognizer/quickstarts/label-tool?tabs=v2-0
After following the above guide you'll have the necessary items to be able to use Form Recognizer with Kryon:
Introduction
Kryon RPA can easily work with MS Azure Cognitive Services through the use of Kryon API advanced commands. The Form Recognizer service in Azure is an AI-based service that enables the extraction of field data from structured and semi-structured documents for use in your wizards. For example, invoice data or form data.Initial Setup
If you're new to Form Recognizer then there is a great quickstart guide from Microsoft that you can follow. You'll be taken though how to set up the necessary resources in Azure, gather sample documents, mark fields for extraction and train a model based on those documents.https://docs.microsoft.com/en-gb/azure/cognitive-services/form-recognizer/quickstarts/label-tool?tabs=v2-0
After following the above guide you'll have the necessary items to be able to use Form Recognizer with Kryon:
- An API key (e.g. "b0becd21af744124812f81f483f42263")
- An endpoint (e.g. "https://myendpointname.cognitiveservices.azure.com/")
- A model ID (e.g. "9427ebce-2a4c-4606-8ad1-f1c35ffc1e9c")
Analyzing a Form Using Kryon
To do this you need to use the cURL advanced command in Kryon Studio with this command (note that Studio already prefixes the command with "curl"):-i --location --request POST "$fr endpoint$formrecognizer/v2.0/custom/models/$model id$/analyze?includeTextDetails=false" --header "Content-Type: $content type$" --header "Ocp-Apim-Subscription-Key: $fr api key$" --data-binary "@$input filepath$" $fr endpoint$ the endpoint you configured in the MS Azure portal.
$model id$ the model ID which was created when you trained your model with the sample training documents.
$content type$ is typically "application/pdf" as you are submitting PDF documents to the service.
$fr api key$ is your API key you configured in the MS Azure portal.
$input filepath$ is the filepath to the PDF which you are uploading to the service.
$submit response$ is the result that the Form Recognizer service sends back to you.
$submit exit code$ is the result of the actual cURL command. You will want this to be 0 otherwise there was an error in your command.
The JSON-formatted submit response from Azure will contain a document ID which can extracted using a regular expression. Here is one example that works well:
$submit exit code$ is the result of the actual cURL command. You will want this to be 0 otherwise there was an error in your command.
The JSON-formatted submit response from Azure will contain a document ID which can extracted using a regular expression. Here is one example that works well:
The document id is simply a reference to your file submission. It doesn't mean that the extraction is complete. You must periodically query Azure to find the status and result of the document's extraction operation. Here is an example code snippet that will help with this. This code will keep querying the document very second until it is reported as complete. When it is complete, the extracted data returned by Azure will be contained in the variable below called $get response$.
The configuration of the cURL AC:
The configuration of the Query JSON AC:
$get response$ contains all the data Azure has extracted from your PDF document. It is JSON formatted so after writing it to disk, you'll have to parse it into the format you require. Here is a sample invoice and sample output of what you might expect.
If you have Kryon v20.9 a wizard sample has been posted in the Automation Hub on KryoNet. If you supply an API key and endpoint, you can train new models, analyze documents, parse JSON results into Excel files, and manage you set of trained models.
https://community.kryonsystems.com/KryoNet/s/question/0D50600007Flr72CAB/sample-wizard-for-ms-azure-form-recognizer-service
(Link to be updated)
Further Information
If you need to test your Azure service outside of Kryon before implementing you can use the provided online tester from Microsoft:
https://uksouth.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/GetCustomModels/console
Microsoft also offer a pretrained model specifically for invoices, a guide which you can find here. Note that at the time of writing while this guide uses the Form Recognizer API v2.0, the prebuilt invoice model is only present in v2.1 Preview, and as such the API URLs are different than those mentioned in this article.
https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/concept-invoices
https://westcentralus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2-1-preview-2/operations/5ed8c9843c2794cbb1a96291
You can test trained models easily (including the prebuilt Invoice model) using this online tool before implementing in Kryon:
https://fott-preview.azurewebsites.net/
I hope this helps get you started with Form Recognizer - please post in KryoNet if you need any advice!
https://uksouth.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/GetCustomModels/console
Microsoft also offer a pretrained model specifically for invoices, a guide which you can find here. Note that at the time of writing while this guide uses the Form Recognizer API v2.0, the prebuilt invoice model is only present in v2.1 Preview, and as such the API URLs are different than those mentioned in this article.
https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/concept-invoices
https://westcentralus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2-1-preview-2/operations/5ed8c9843c2794cbb1a96291
You can test trained models easily (including the prebuilt Invoice model) using this online tool before implementing in Kryon:
https://fott-preview.azurewebsites.net/
I hope this helps get you started with Form Recognizer - please post in KryoNet if you need any advice!
