Detect vehicles from a CCTV image

vehicle detection from a CCTV image

Much of the unstructured data that exists today and will be created in the future is image or video data. Instagram, Facebook, CCTV etc. As an example of how to analyze image data this post will explain how to detect vehicles from a CCTV image using a cascade of boosted classifiers based on HAAR like features.

We will use the opencv library. It is a C++ library written for computer visualization and contains lots of useful functions for processing and manipulating images. We will use the python wrapper to this library called cv2.

Installing opencv and cv2

You can download the opencv library from the OpenCV Website.

The python wrapper is called and is included in the install. You may need to check that the PYTHONPATH environmental variable includes the path to the file.

To include the library in your python scripts just import cv2, e.g.

Importing the opencv library wrapper
$ ipython

import cv2


Help on Module cv2:



The four steps needed to detect vehicles from a CCTV image

  1. Asset Preparation
  2. Create sample images and generate a .vec file
  3. Train classifier
  4. Test classifier, if not accurate enough go back to step 1

Step 1: Asset Preparation

To identify objects such as vehicles we need to train a classifier to recognize the objects within the image. To train the classifier we need two types of images, positive images, that contain the object we are trying to detect and negative images that don’t.

We may need hundreds of objects of both types to train a classifier depending on the accuracy we want.

Unfortunately acquiring the images is a manual process. I looked at CCTV traffic images and manually cropped the vehicles from them. For the negative images you can use any image that doesn’t contain the object you are trying to identify. I used CCTV images that did not contain any vehicles or I manipulated the images to remove any vehicles present.

Below are some of the cropped positive images manually extracted from CCTV images (I have collected them together for this blog article but they are stored as separate images).

car montage

Below are some of the cropped negative images manually extracted from CCTV images (As with the positive images the negative images are stored separately).

negative car montage

One trick you can use to increase the number of images is to cut an image into tiles or flip the image horizontally. Here is a simple script I use to increase the number of negative images:

NOTE: Install imagemagick to get the convert command

rm -f *tile*.jpg *flop*.jpg

convert -flop *.jpg flop_%02d.jpg
convert *.jpg -crop 3x3+20+20@ +repage +adjoin tile_%02d.jpg

Now you have many more images that can be used to train your classifier!

Step 2: Create Sample Images and generate a .vec file

The next step is to generate a metadata file called collection.dat. This file contains the file name, number of objects, position and dimensions of the objects within each image. I only have one object per image and crop the image to only contain the object so the position is (0,0) and the dimension of the object is the width and height of the image.

This is what the collection.dat metadata file looks like:

traffic/positive/taxi9_flop.jpg 1 0 0 14 12
traffic/positive/car16_flip.jpg 1 0 0 13 13
traffic/positive/car42_flip.jpg 1 0 0 8 9
traffic/positive/truck19.jpg 1 0 0 27 26
traffic/positive/car99_flip.jpg 1 0 0 16 15
traffic/positive/car36_flip.jpg 1 0 0 9 10
traffic/positive/van7_flop_flip.jpg 1 0 0 17 16
traffic/positive/truck20.jpg 1 0 0 72 73
traffic/positive/truck22_flop_flip.jpg 1 0 0 39 48
traffic/positive/car8_flop.jpg 1 0 0 17 18

Once you have generated the collection.dat metadata file you then need to create a samples.vec file that will be used by the classifier. This file contains all positive images within a single file.

After we have generated the samples.vec file and the collection.dat file we can invoke the script provided by the opencv library to train the classifier.

Step 3: Train the classifier

This is the script I use to generate the intermediate files and to train the classifier:

Train classifier script: After the script has completed you should find a file called cascade.xml in the classifier directory. This is the file you need for object detection.  

Step 4: Test the accuracy of the classifier

Here is code to use the classifier to find vehicles within an image and to highlight each detected vehicle:

Object detection script:

Cascade.xml file: cascade.xml

Running this code gives the following output:

Vehicles detected: 6

vehicle detection from a CCTV image

It’s not perfect, some cars that are obscured by trees are not detected and cars in the distance aren’t detected either. Still this is a very useful technique for extracting data from images or video frames.

Additional Files:

See Also: