Azure AI – Vision Services

Microsoft’s suite of Azure AI services offers countless opportunities to automate everyday tasks that formerly required a human set of eyes.

Azure AI Vision allows your apps to analyze images, read text, and detect faces with prebuilt image tagging, text extraction with optical character recognition (OCR), and facial recognition.

Overview

Microsoft’s Azure Vision is a cloud service set that uses machine learning to analyze images.  It’s broken into 4 major components:

• Optical Character Recognition (OCR) – extracts printed and handwritten text from images (not documents) with varied languages and writing styles. Examples include:
  • Pulling serial #s from a picture of equipment or component boards
  • Getting the text from a label within an image
  • Reading the license plate of a car
• Image Analysis – pulls from more than 10,000 concepts and objects to detect, classify, caption, and generate insights. Examples include:
  • Identifying objects within an image
  • Recognizing products on shelves
  • Searching through photos
  • Removing backgrounds from images
  • Identifying sensitive content within images
• Facial Recognition – creates intelligent applications that recognize and verify human identity. Examples include:
  • Recognizing faces within an image.
  • Ensuring the image is an actual, live face before the camera.
  • Comparing 2 facial images to determine if they are the same person.
• Video Analysis – recognizes people’s presence and movements within physical areas in real-time. Examples include:
  • Summarize the video feed and main points (e.g. there is a spill on aisle 3)
  • Tracking movements of people within a video feed (e.g. # of people in an area at a given time, when someone crosses into an area, etc.)

How does it work?

Training a model – Machine learning requires a “model” to be trained, a process that teaches the system to find patterns and make decisions. 

Pre-Trained Models – Microsoft has pre-trained the Vision Services models to analyze images and video. 

License Plate Example—If you need to track cars entering a secure facility regularly, a picture snapped of the back of each vehicle could train the model to identify the license location and extract the license plate number from the image as text. Once trained, the application could record all license plates entering and leaving.

This image shows the application pulling the license plate data, car model, and more from three cars:

This AI Vision model identified a person falling in the data center but not in the warehouse.

In this employee safety example, the app recognized an employee falling off a ladder.

Here, the model goes beyond “cleanup on aisle three” and identifies “milk on the floor.”

Using Vision AI in Manufacturing

For manufacturers, the ability to detect material spills on a production line or warehouse could speed clean-up.

The facial models can be used to compare two images of a person – similar to facial recognition on your phone.

For TwinEngines’ Whitaker client, we used this technology to monitor the “clocking in” of employees at a remote job site. The facial model for each employee ensures that the employee who clocked in matches the roster. If the model wasn’t 80% confident, a supervisor manually reviewed the clock-in.

Putting Vision AI to Work

A REST API is used to submit images or videos to the model.  Results are then returned to the application, typically in a JSON format

Here is a snippet of the JSON data returned by the model that extracted texts for the license plates example above.

(Note: The facial model also leverages developer libraries that simplify dealing with REST APIs.)

Real World Examples

Explore these case studies where Azure Vision AI Services streamline commercial applications:

• VIEW CASE STUDY: TwinEngines used Azure AI Vision to help Formetco remotely monitor billboards and quickly identify when one is down.
• VIEW CASE STUDY: TwinEngines used Azure AI Vision to help Whitacre Rebar ensure the accuracy of employee timekeeping.

Implementing your application

Designed for developers – Azure AI Vision is a developer’s tool.  A seasoned developer must set up and configure Azure resources, make API calls, and analyze JSON results.  Although Vision Studio makes experimenting with models relatively easy, integrating these services within your application requires IT expertise. 

Bottom line

Azure AI Vision allows you to analyze images and videos in ways that would otherwise require worker resources. In any scenario requiring inordinate amounts of time examining images or videos, Azure AI Vision can reduce this tedious workload.

TwinEngines is committed to delivering on artificial intelligence’s promise by developing Real-World AI applications for manufacturers, suppliers, and specialty contractors using Microsoft’s leading-edge suite of Azure AI Services.

To learn how we can automate manual processes in your workflow, schedule an online meeting with a TwinEngines Solution Architect.