How to implement automated license plate recognition in C# using OCR

This documentation is built up from two major parts: the first part consists of a rather theoretical and general description of license plate recognition and the Computer Vision technology in order to implement the example successfully; while the second part is a more detailed and programming- oriented documentation.

Happy coding! :)

Contents:

Part 1: Introduction to the technology

• What is Computer Vision

• What is OCR (Optical Character Recognition)

• What is ‘license plate recognition’ and how you can benefit from it

Part2:  C# programming, implementation of the example

• Creating the C# code for OCR

• Implement GUI example

Conclusion and References

Sources to download:

Microsoft Visual Studio and .NET Framework

USB camera

Camera SDK

Part1: Introduction to the technology 

What is Computer Vision

People have a chance to use their brains and eyes in order to visually sense the things which are all around them. But what about the computers? Nowadays, the computers are also able to detect and sense the world with the help of computer vision technology. This visual understanding technology can be used in a lot of fields, just to mention some of them:

• security and surveillance

• image restoration

• character recognition

• medical image analysis

• agriculture

• augmented reality

• geosciences

• robotics

These are just a few from the applications of the computer vision technology.

Let’s quote a kind of general description of Computer Vision:

“Computer vision is a field that includes methods for acquiring, processing, analyzing, and understanding images and, in general, high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions.[…] As a scientific discipline, computer vision is concerned with the theory behind artificial systems that extract information from images. The image data can take many forms, such as video sequences, views from multiple cameras, or multi-dimensional data from a medical scanner. As a technological discipline, computer vision seeks to apply its theories and models to the construction of computer vision systems.” [1]

As it can be seen, the applications which are connected to this technology extend from business and corporate use of machine vision functionalities (such as image processing, storing and producing data etc.) to researches and studies into artificial intelligence and robotics.

The license plate recognition is an interesting and widely used function which is closely related to Computer Vision technology, moreover, the functionality which makes the plate recognition possible (OCR) is a part of computer vision.

What is OCR (Optical Character Recognition)

Optical Character Recognition is a mechanical or electronic convert of images of typewritten and printed text into a machine-encoded text.[2] 

Optical Character Recognition is a quite complex function which can make you able to convert images. Into what? You can convert them into formats which are digital and most importantly, editable. Just to mention some examples for these: you can create files with .doc, .txt and .pdf formats. 

The Optical Character Recognition function can be used for developing surveillance systems or in the field of logistics and to make systems automate. Moreover, this function can be used for security monitoring. 

Optical Character Recognition function is widely used nowadays on several corporate fields. For example, in medication labeling or converting printed data into digital form. Moreover, OCR can be used to detect license plates which can serve a lot of purposes well.

License plate detection

The license plate recognition is an extremely useful component of OCR. With the help of this function it becomes possible to make logistics automated or implement effective security monitoring. The LPR (license plate recognition) makes it possible to read the license number of different vehicles. Basically, the following happens: the pixels of the image (the digital one) are converted into an ASCII format of the license plate. (ASCII is an abbreviation which stands for the American Standard Code for Information Interchange. It is a scheme which responsible for character-encoding. The ASCII consists of 128 characters and it is based on English alphabet). With other words: the camera scans the area where a device is placed. There are cars from a quite huge distance and their license plates cannot be seen. The application is able to detect texts on digital sufcaes/images (in this case, to detect texts on the license plates of the cars. (Figure 1.) This detection function is easy-to-use and extremely useful in the case of surveillance systems.

Figure 1- How OCR works in a Machine Vision application

Part2: How to accomplish OCR in C#?

First, it is important to determine the needed environment and the tools and devices. As it was provided above it is recommended to download the Microsoft Visual Studio which is an integrated development environment and can be used for developing programs which are designed to operate on Windows operating systems. Microsoft Visual Studio can handle managed and native codes as well. Such as the other integrated development environments, the Microsoft Visual Studio has a code editor. Furthermore, you will find the download link of the Microsoft .NET Framework which will also be an essential part of the development. As you may know, this is a partly open source freeware software framework. To build up the solution you will need an USB camera which is mostly up to you which device you would like to use. Also, I shared the download link of the camera SDK which was used for the implementation.

Some people can say “Why would I implement OCR?” Well, there are several advantages of this function which may not come to one’s mind at first. First, in the case of  long documents, it would be expensive to hire someone who may take the job to enter the whole amount. Secondly, it is also faster than manually typing the text. Finally, a more advanced software is able to recreate tables.

To be honest, at the beginning it felt like this:

But guys, don’t let all these codes to confuse you. After finishing the project it will be a fantastic feeling to create something great and useful. I tried to be as systematic as it was possible and I hope that the following will be helpful. Let’s start!

To begin, let’s see the MainForm.cs class. 

The MainForm.cs class is a file which is for defining the background code. With the help of the MainForm.cs class the necessary objects can be initialized and the pre-defined controls can be applied to the GUI. The events which belong to the different controls and objects are defined here. 

The first step is to add some using lines to your code in order to use the SDK methods and components. With the help of these “using lines” you can make reference to the namespaces. This using directive can be used in two cases: it is good for allowing types of namespaces and for creating a using alias directive (alias for a namespace).

It is important to mention that the these using lines are limited which means that they can be used only in the files in which they appear. They will not provide access to namespaces that are embedded in the specified namespace. However, these lines are excellent for making the qualification of an identifier to a namespace easier. 

Let’s see the code: 

using System;
using System.Drawing;
using System.Windows.Forms;

Before the actual implementation: there are several good solutions out there from which you can choose, basically it is totally subjective. On my side, I used the Ozeki Camera SDK since this was the easiest to configure and the solution worked for me with this product. Also, on the webpage you can find detailed documentation and free source codes which is a great help in implementing projects.

In this case the namespace will be the following: Ozeki.Media.MediaHandlers.Video.CV.Processer so the code became the following:

using System;
using System.Drawing;
using System.Windows.Forms;
using Ozeki.Media.MediaHandlers;
using Ozeki.Media.MediaHandlers.Video;
using Ozeki.Media.MediaHandlers.Video.CV;
using Ozeki.Media.MediaHandlers.Video.CV.OCRData;
using Ozeki.Media.MediaHandlers.Video.CV.Processer;

Below the code detail can be found which stands for the connection of the camera device.

Speaking of devices, in this project an USB camera was used in order to build up the solution. At the beginning of this tutorial you can find a link which leads you to several USB devices to choose which is the most appropriate for you. 

Now back to the code. :) The connection to the web camera and the displaying of the image take place in the buttonConnect_Click function:

void buttonConnect_Click(object sender, EventArgs e)

As a next step, in the buttonStartDetect_Click function it is possible to connect the camera with the frame capture and connect the frame capture with the ImageProcesserHandler.

void buttonStartDetect_Click(object sender, EventArgs e)

To the ImageProcesserHandler those detectors should be added which process the incoming camera images. 

It is possible to add detectors to the ImageProcesserHandler in the following way:

.AddProcesser(_characterDetector);

Moreover, the handler and the video channel of the camera should be connected:

connector.Connect(_webCamera, frameCapture);

                connector.Connect(_frameCapture, imageProcesserHandler);

                connector.Connect(_imageProcesserHandler, imageProvider);

As you can see, in this example the frame capture is also inserted between the ImageProcesserHandler and the webcamera. The aim of frame capture is that in this way it will be possible to determine which frames should be processed by the ImageProcesserHandler.

At the end of this process, the processed image will be sent to the Image Provider. The image will be sent to that image provider which is set to the videoviewer in order to display the image. Here’s the code:

videoViewer.SetImageProvider(_imageProvider);

One of the advantages of this function is that you can detect the texts in more languages. If you wish to detect the text in English, all you have to do is to insert the “eng” word in the code to the appropriate location. The constructor part contains different parameters which help us to determine the language which we wish to detect the optical characters and the location of the file. Here’s a code to demonstrate this…

ICharacterDetector _characterDetector =

ImageProcesserFactory.CreateCharacterDetector(@"", "eng",

… and the more complex code:

_imageProvider = new DrawingImageProvider();

            _connector = new MediaConnector();

            _imageProcesserHandler = new ImageProcesserHandler();

            _frameCapture = new FrameCapture();

            _characterDetector = ImageProcesserFactory.CreateCharacterDetector(@"", "eng", OcrEngineMode.FASTEST);

As it was mentioned before, after the CreateCharacterDetector the language can be determined (here it is “eng” so English). This function is not too essential in the case of license plate since the text of the license plate is not a comprehensive text. However, this can be useful in the case of situations where complete and linked texts have to be detected. After this an “OcrEngineMode.FASTEST” can be seen which stands for the speed of character recognition. 

Regarding the detection:

You can subscribe to the event of the “characterdetector” with the help of the following code line:

_characterDetector.DetectionOccurred += _characterDetector_DetectionOccurred;

By finishing this, it is recommended to determine different properties. The first property which you can set is the DrawColor property which stands for the color of the detected characters and text. The color of the detected text is up to you. If you wish the detected text to be green you can do it, that will be fine.  As it appears in the code:

_characterDetector.DrawColor = Color.DarkGreen;

The next property which can be determined is the DrawThickness property. This can be used to determine the value of thickness. With other words, how thick you wish the characters to be detected.  The code form of this is the following:

_characterDetector.DrawThickness = 3;

So far so good, let’s see the whole code block in its whole splendor! :)

void buttonStartDetect_Click(object sender, EventArgs e)

        {

            try

            {

                if (_webCamera == null) return;

 

                _characterDetector.DetectionOccurred += _characterDetector_DetectionOccurred;

                _characterDetector.DrawColor = Color.DarkGreen;

                _characterDetector.DrawThickness =3;

Now it is set to detect different texts with dark green color and the thickness of the detected text will be “3”. 

When the DetectionOccured event takes place it can be handled in the a_characterDetector_DetectionOccured function. In this case the content of the founded text will be written down with the help of the following code:

textBoxDetectedText.Text = e.DetectedText.Content

And the code block:

void _characterDetector_DetectionOccurred(object sender, TextDetectedEventArgs e)

        {

            InvokeGuiThread(() =>           

                textBoxDetectedText.Text = e.DetectedText.Content

            );

        }

After the text is detected, there are properties which are available. The “Content” is the detected text itself and the “Characters” which is for keeping the information which is carried by the detected text.

It is also possible to create different lists on which the detected characters can be placed. These lists can be useful if somebody wishes to set the text to display. Also, it is possible to create a list for digit recognition. With other words, this list will only display recognized digits if digits (1234567890) are specified.

The frame of the code to create lists is the following:

void SetVariable(CharacterFilterMode variableName, String value, bool isBetterAccuracy);

In this frame there are some parameters which can be examined and modified. In the first parameter the White/Black list should be specified (variableName). The second part (String value) is the part where the characters which you wish to put on the lists should be specified. Finally, the third part which stands for a better accuracy.

Let’s see it in our code block:

void Filter_Changed(object sender, EventArgs e)

        {

            if (radioButtonBlack.Checked)

                _characterDetector.SetVariable(CharacterFilterMode.BlackList, textBoxFilter.Text, true);

            else

                _characterDetector.SetVariable(CharacterFilterMode.WhiteList, textBoxFilter.Text, true);

        }

To sum up, a code is written here which is able to detect a text which is black, the thickness of the text is “3”, the color of the detection in green (Figure 2). 

Figure 2.

So far, we are fantastic. Now, let’s take a look on the GUI code of this application.

Implement GUI example

Now the code comes which stands for the interface, the GUI code. While reading this you are looking at the graphical user interface of your browser. It would be nice if your applications would also have one, right? :) Without the graphical user interface, it would be quite harsh to handle your application. A lot of commands and codes would have to be used which is more uncomfortable than create the GUI which will provide a clean and easy-to-use surface. The programming of the GUI takes place in the MainForm.Designer.cs class as the following:

namespace NVA_OCR_USB_Camera_WF

{

    partial class Form1

    {

In the GUI you can create the interface of the Connect and Disconnect button, the color of the buttons and so on. After creating the buttons,a simple click on the button will be enough to execute a command. In this case there are some extremely important buttons. Below we will observe some examples with codes. 

Let’s see an important element, the button which is used to connect to the device. Here you can find the example code for implementing that:

this.buttonConnect.Location = new System.Drawing.Point(7, 18);

            this.buttonConnect.Name = "buttonConnect";

            this.buttonConnect.Size = new System.Drawing.Size(85, 23);

            this.buttonConnect.TabIndex = 0;

            this.buttonConnect.Text = "Connect";

            this.buttonConnect.UseVisualStyleBackColor = true;

            this.buttonConnect.Click += new System.EventHandler(this.buttonConnect_Click);

In this case on top left of the screen you will see a “Connect” button.

After this, we have to mention the other useful button,the Disconnect button for which here you can also find an example code:

this.buttonDisconnect.Enabled = false;

            this.buttonDisconnect.Location = new System.Drawing.Point(109, 19);

            this.buttonDisconnect.Name = "buttonDisconnect";

            this.buttonDisconnect.Size = new System.Drawing.Size(85, 23);

            this.buttonDisconnect.TabIndex = 1;

            this.buttonDisconnect.Text = "Disconnect";

            this.buttonDisconnect.UseVisualStyleBackColor = true;

            this.buttonDisconnect.Click += new System.EventHandler(this.buttonDisconnect_Click);

After implementing this, a “Disconnect” button will show up next to your “Connect button”, which is quite fun.

As we talk about detection function, it would be nice to implement a button which starts the detection function, right? :) So here it is:

this.buttonStartDetect.Enabled = false;

            this.buttonStartDetect.Location = new System.Drawing.Point(7, 20);

            this.buttonStartDetect.Name = "buttonStartDetect";

            this.buttonStartDetect.Size = new System.Drawing.Size(85, 23);

            this.buttonStartDetect.TabIndex = 0;

            this.buttonStartDetect.Text = "Start detection";

            this.buttonStartDetect.UseVisualStyleBackColor = true;

            this.buttonStartDetect.Click += new System.EventHandler(this.buttonStartDetect_Click);

From this code is can be seen that the name of the button will be “Start detection”. The name of the button can be determined with the this.buttonStartDetect.Text =” “;

After this:

this.buttonStopDetect.Enabled = false;

            this.buttonStopDetect.Location = new System.Drawing.Point(109, 19);

            this.buttonStopDetect.Name = "buttonStopDetect";

            this.buttonStopDetect.Size = new System.Drawing.Size(85, 23);

            this.buttonStopDetect.TabIndex = 1;

            this.buttonStopDetect.Text = "Stop detection";

            this.buttonStopDetect.UseVisualStyleBackColor = true;

            this.buttonStopDetect.Click += new System.EventHandler(this.buttonStopDetect_Click);

… and here is the pair of the detection button, the Stop detection! Just like in the previous example the name of the button can be determined by providing the this.buttonStopDetect.Text = " “ line.

Another essential element is the video viewer. Of course, it also needs a created button on the graphical user interface. If you wish to implement the video viewer on your graphical user interface follow the below:

this.videoViewer.BackColor = System.Drawing.Color.DarkGray;

            this.videoViewer.FlipMode = Ozeki.Media.Video.FlipMode.None;

            this.videoViewer.Location = new System.Drawing.Point(6, 19);

            this.videoViewer.Name = "videoViewer";

            this.videoViewer.RotateAngle = 0;

            this.videoViewer.Size = new System.Drawing.Size(543, 323);

            this.videoViewer.TabIndex = 0;

            this.videoViewer.Text = "videoViewerWF1";

After creating the GUI you will see something like this on your screen (Figure 3) :

Figure 3

There can be controls and objects which are not pre-defined. If they have an event which occurs you can modify the property of the control of the GUI in the body of the event. You can implement this in the Invoke method:

void InvokeGuiThread(Action action)

        {

            BeginInvoke(action);

These are the relatively most important parts, but on the webpage of the used Camera SDK you can find a complete example code free of charge for implementing the graphical user interface of your application which makes your job much a more easier. 

Conclusion

After providing the necessary examples, codes and details this tutorial aims to help implementing the Optical Character Recognition (OCR) function in C#. In this tutorial everybody can find useful help in order to achieve success in the field of character recognition. After examining the examples you will be able to detect texts from digital images, you can determine the language of the detection and you will be able to modify the color and the thickness of the detected text. This solution can be useful for those who wish to develop surveillance systems or work on the field of logistics and wish to automatize the system. Moreover, this example can be used for security monitoring. A detailed part about the license plate recognition also can be found here. 

Hope this tutorial will be helpful and please do not hesitate to use it and provide your opinion on it. Thank you for reading this! :)

References

• http://en.wikipedia.org/wiki/Computer_vision

• http://en.wikipedia.org/wiki/Optical_character_recognition

• http://en.wikipedia.org/wiki/Automatic_number_plate_recognition

• http://www.camera-sdk.com/p_13-download-onvif-standard-ozeki-camera-sdk-for-webcam-and-ip-camera-developments-onvif.html

• http://en.wikipedia.org/wiki/.NET_Framework

• http://en.wikipedia.org/wiki/Microsoft_Visual_Studio