Detect vehicles from a CCTV image
Much of the unstructured data that exists today and will be created in the future is image or video data. Instagram, Facebook, CCTV etc. As an example of how to analyze image data this post will explain how to detect vehicles from a CCTV image using a cascade of boosted classifiers based on HAAR like features.
We will use the opencv library. It is a C++ library written for computer visualization and contains lots of useful functions for processing and manipulating images. We will use the python wrapper to this library called cv2.
Installing opencv and cv2
You can download the opencv library from the OpenCV Website.
The python wrapper is called cv2.so and is included in the install. You may need to check that the PYTHONPATH environmental variable includes the path to the cv2.so file.
To include the library in your python scripts just import cv2, e.g.
Importing the opencv library wrapper
$ ipython import cv2 help(cv2) Help on Module cv2: NAME cv2 FILE /usr/lib64/python2.7/site-packages/cv2.so
The four steps needed to detect vehicles from a CCTV image
- Asset Preparation
- Create sample images and generate a .vec file
- Train classifier
- Test classifier, if not accurate enough go back to step 1
Step 1: Asset Preparation
To identify objects such as vehicles we need to train a classifier to recognize the objects within the image. To train the classifier we need two types of images, positive images, that contain the object we are trying to detect and negative images that don’t.
We may need hundreds of objects of both types to train a classifier depending on the accuracy we want.
Unfortunately acquiring the images is a manual process. I looked at CCTV traffic images and manually cropped the vehicles from them. For the negative images you can use any image that doesn’t contain the object you are trying to identify. I used CCTV images that did not contain any vehicles or I manipulated the images to remove any vehicles present.
Below are some of the cropped positive images manually extracted from CCTV images (I have collected them together for this blog article but they are stored as separate images).
Below are some of the cropped negative images manually extracted from CCTV images (As with the positive images the negative images are stored separately).
One trick you can use to increase the number of images is to cut an image into tiles or flip the image horizontally. Here is a simple script I use to increase the number of negative images:
NOTE: Install imagemagick to get the convert command
rm -f *tile*.jpg *flop*.jpg convert -flop *.jpg flop_%02d.jpg convert *.jpg -crop 3x3+20+20@ +repage +adjoin tile_%02d.jpg
Now you have many more images that can be used to train your classifier!
Step 2: Create Sample Images and generate a .vec file
The next step is to generate a metadata file called collection.dat. This file contains the file name, number of objects, position and dimensions of the objects within each image. I only have one object per image and crop the image to only contain the object so the position is (0,0) and the dimension of the object is the width and height of the image.
This is what the collection.dat metadata file looks like:
traffic/positive/taxi9_flop.jpg 1 0 0 14 12 traffic/positive/car16_flip.jpg 1 0 0 13 13 traffic/positive/car42_flip.jpg 1 0 0 8 9 traffic/positive/truck19.jpg 1 0 0 27 26 traffic/positive/car99_flip.jpg 1 0 0 16 15 traffic/positive/car36_flip.jpg 1 0 0 9 10 traffic/positive/van7_flop_flip.jpg 1 0 0 17 16 traffic/positive/truck20.jpg 1 0 0 72 73 traffic/positive/truck22_flop_flip.jpg 1 0 0 39 48 traffic/positive/car8_flop.jpg 1 0 0 17 18
Once you have generated the collection.dat metadata file you then need to create a samples.vec file that will be used by the classifier. This file contains all positive images within a single file.
After we have generated the samples.vec file and the collection.dat file we can invoke the script provided by the opencv library to train the classifier.
Step 3: Train the classifier
This is the script I use to generate the intermediate files and to train the classifier:
Train classifier script: train_classifier.sh After the script has completed you should find a file called cascade.xml in the classifier directory. This is the file you need for object detection.
Step 4: Test the accuracy of the classifier
Here is code to use the classifier to find vehicles within an image and to highlight each detected vehicle:
Object detection script: object_detection.py
Cascade.xml file: cascade.xml
Running this code gives the following output:
Vehicles detected: 6
It’s not perfect, some cars that are obscured by trees are not detected and cars in the distance aren’t detected either. Still this is a very useful technique for extracting data from images or video frames.