In this article, I will:

Microsoft Cognitive Services

Image Visual Content Analysis: Overview

To analyze the image content, you no longer need to be a Ph.D. in computer science or be a computer scientist or machine learning expert. Tech giants like Microsoft, Google, and Amazon have developed machine learned artificial intelligence cloud-based products to analyze the visual content of the image.

It allows the developer to add image processing capability to application easily.

As developers, we just need to integrate the API into the application simply by REST API call using HttpClient to extract the image content.

For articles on other Microsoft Cognitive Service, please visit the below link https://www.mysimpletips.com/category/cognitive-services/

Available Image Content Analysis API

The following Vision APIs are available to extract the visual content of the image:

Microsoft Computer Vision API overview

It is a part of Microsoft Cognitive Service - a suite of Artificial Intelligent products built using Machine Learning.

Microsoft Computer Vision API is cloud-based pre-trained machine learning model. Its advanced algorithm enables the developer to integrate the image processing capability in the application.

By analyzing the image, Vision API extracts the following visual content of the images

Apart from the above information, Computer Vision API performs the following

Prerequisite

Microsoft Computer Vision API In Action

To view the Vision API in action to extract text from image,

Please go to https://azure.microsoft.com/en-in/services/cognitive-services/computer-vision/ and upload the image.

In the right side of the panel, it will display the extracted text. Here I have uploaded the image that I created for the article. It extracted almost every text except the "adult/racy" text.

Microsoft Cognitive Services
Microsoft Computer Vision API In Action [/caption]

Please note the below terms and conditions by Microsoft while uploading and testing:

By uploading data for this demo, you agree that Microsoft may store it and use it to improve Microsoft services, including this API. To help protect your privacy, we take steps to de-identify your data and keep it secure. We shall not publish your data or let other people use it.

Approach for API integration

Get Vision API and Key from the Azure portal

Go to Azure portal and to create Cloud Vision API, please follow the below steps

Or click on the below link to create the Cloud Vision API in the Azure portal.

https://portal.azure.com/#create/Microsoft.CognitiveServicesComputerVision

Microsoft Cognitive Services
Microsoft Cognitive Service: Create Computer Vision API [/caption]

There are 2 pricing tiers, Free and Standard, available to create Vision API.

Here I have created API using a Free pricing tier. The below screen shows the available Keys to access Cloud Vision API.

Microsoft Cognitive Services
Microsoft Cognitive Service: Computer Vision API Keys[/caption]

Code Snippet

It required 2 REST API Calls:

  1. const string subscriptionKey = "dac6066364fd4a83bd7a4f300632fde1";
  2. const string uriBase = "https://southcentralus.api.cognitive.microsoft.com/vision/v1.0/recognizeText";
  3. string imagePath = @ "Image.JPG";
  4. string imageTextContent;
  5. HttpClient httpclient = new HttpClient();
  6. // Add Subscription Key in Request headers.
  7. httpclient.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", subscriptionKey);
  8. //Set "handwriting" to true in case handwritin text else true for printed text.
  9. string requestParams = "handwriting=false & detectOrientation=true";
  10. // Final URI
  11. string uri = uriBase + "?" + requestParams;
  12. HttpResponseMessage httpresponse = null;
  13. string resultStorageLocation = null;
  14. // Get the image as byte array; this method is defined below
  15. byte[] imagebByteData = GetByteArrayOfImage(imagePath);
  16. ByteArrayContent imageContent = new ByteArrayContent(imagebByteData);
  17. //Set content type: "application/octet-stream" or "application/json"
  18. imageContent.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
  19. //The 1st REST APT call to start the async process by submitting the image.
  20. httpresponse = await httpclient.PostAsync(uri, imageContent);
  21. //Get location of result from response
  22. if (httpresponse.IsSuccessStatusCode) resultStoragelocation = httpresponse.Headers.GetValues("Operation-Location").FirstOrDefault();
  23. //2nd REST API call to get the text content from image
  24. httpresponse = await httpclient.GetAsync(resultStorageLocation);
  25. imageTextContent = await httpresponse.Content.ReadAsStringAsync();
  26. //TO DO: This imageTextContent is raw JSON string; Need to format this JSON string for further processing.
  27. //Returns the byte array of input image
  28. private byte[] GetByteArrayOfImage(string imagePath) {
  29. FileStream filestreamObj = new FileStream(imagePath, FileMode.Open, FileAccess.Read);
  30. BinaryReader binaryreaderObj = new BinaryReader(filestreamObj);
  31. return binaryreaderObj.ReadBytes((int) filestreamObj.Length);
  32. }

Microsoft Computer Vision API Use Case

Please subscribe to my blog www.mysimpletips.com to get the latest articles on Azure, Chatbot, Cognitive Service. Please go to "Email Subscription" section and provide name/email address and submit.