Introduction

CCTV cameras are now everywhere. Large-scale video processing is a grand challenge representing an important frontier for analytics, with videos from factory floors, jewelry showrooms, traffic intersections, police vehicles, and retail shops. It’s the golden era for computer vision, AI, and machine learning – it’s a great time now to extract value from videos to impact science, society, and business!
I would like to share one recent incident that happened in India, Tamil Nadu. One of the famous jewelry stores in Trichy has more than 20 security personnel, and 60 CCTV cameras in surveillance. Two individuals were wearing animal masks and packing all the jewels in a sack, just like a scene from a movie. The burglars entered the showroom by drilling a hole in the wall which was large enough for a man to enter, in the backside of the building. They stole almost 100 kgs of gold which is worth Rs 50 crore.
Video surveillance is incredibly useful for helping track down a suspect. Often burglars are intelligent enough to mask their faces, so facial recognition is not a possibility. However, video surveillance will help identify what the perpetrator looked like by recording height build, hair color, skin color, and the human body and object recognition and it will alert to showroom manager.
Jewellery Showroom Theft Detection Using Microsoft Cognitive Services
In this article, I will share how to implement computer Vision and Face API Analyze Videos in real-time.

How to use Microsoft cognitive services

Performing real-time analysis on frames taken from a live video stream and identify the face with extracted the content using vision API, the application will follow the below.
Jewellery Showroom Theft Detection Using Microsoft Cognitive Services
Let's start to implement Live Video analysis, in the following steps

Create a Computer Vision API

The Computer Vision API will help developers to identify the objects with access to advanced algorithms for processing images and returning image metadata information.
Step 1
Microsoft offers Azure cognitive services free for 12 months, it's fantastic news. Create a free trial Azure subscription account from the Azure portal.
Jewellery Showroom Theft Detection Using Microsoft Cognitive Services
Step 2
Create New resource and select the AI+ Machine learning and click on Computer Vision API
Jewellery Showroom Theft Detection Using Microsoft Cognitive Services
Step 3
After this, fill in the following information for creating Cognitive Service and click on "Create".
Jewellery Showroom Theft Detection Using Microsoft Cognitive Services
Step 4
The resource successfully completes after a few minutes and will get a confirmation message, you can click on "Go to Resource" Button.
Jewellery Showroom Theft Detection Using Microsoft Cognitive Services
Step 5
You can capture/copy the Azure commuter vision API Endpoint URL and key from All resources.
Jewellery Showroom Theft Detection Using Microsoft Cognitive Services

Create a Computer Vision API

The Face API will detect one or more human faces in an image and get back face rectangles for where in the image the faces are, along with face attributes which contain machine learning-based predictions of facial features.
Step 1
You have already logged in into the Azure portal. Click on Create on resource > Select AI + Machine Learning and click on Face and follow step 3 and step 4.
Jewellery Showroom Theft Detection Using Microsoft Cognitive Services
Step 2
You can capture/copy the Azure Face API Endpoint URL and key from All resources.
Jewellery Showroom Theft Detection Using Microsoft Cognitive Services

Demo Source code

Microsoft created and released a sample POC application for Video Analysis, it lets you download and replace the cognitive service API Endpoint URL and key. Application contains a library, along with two applications, for analyzing video frames from a webcam in near-real-time using APIs from Microsoft Cognitive Services and uses the OpenCvSharp package for webcam support.
Step 1
Open the sample in Visual Studio 2019++.
Step 2
Restore all required nuget packages. Right-click on solutions and select the restored nuget packages.
Jewellery Showroom Theft Detection Using Microsoft Cognitive Services
Step 3
Select the solutions and build the applications. You can change the key and Endpoint URL from the following files.
Step 4
Run the application. Select your camera and Mode of event.
Jewellery Showroom Theft Detection Using Microsoft Cognitive Services

Summary

In this article, you learned how to run real-time analysis on live video streams using the Face, Computer Vision APIs, and how to use Microsoft sample code to get started. Start building your app with free API keys and reference sample code.