Detect vehicles from a CCTV image

vehicle detection from a CCTV image

Much of the unstructured data that exists today and will be created in the future is image or video data. Instagram, Facebook, CCTV etc. As an example of how to analyze image data this post will explain how to detect vehicles from a CCTV image using a cascade of boosted classifiers based on HAAR like features.

We will use the opencv library. It is a C++ library written for computer visualization and contains lots of useful functions for processing and manipulating images. We will use the python wrapper to this library called cv2.

Installing opencv and cv2

You can download the opencv library from the OpenCV Website.

The python wrapper is called cv2.so and is included in the install. You may need to check that the PYTHONPATH environmental variable includes the path to the cv2.so file.

To include the library in your python scripts just import cv2, e.g.

Importing the opencv library wrapper
$ ipython

import cv2

help(cv2)

Help on Module cv2:

NAME
   cv2

FILE
   /usr/lib64/python2.7/site-packages/cv2.so

The four steps needed to detect vehicles from a CCTV image

  1. Asset Preparation
  2. Create sample images and generate a .vec file
  3. Train classifier
  4. Test classifier, if not accurate enough go back to step 1

Step 1: Asset Preparation

To identify objects such as vehicles we need to train a classifier to recognize the objects within the image. To train the classifier we need two types of images, positive images, that contain the object we are trying to detect and negative images that don’t.

We may need hundreds of objects of both types to train a classifier depending on the accuracy we want.

Unfortunately acquiring the images is a manual process. I looked at CCTV traffic images and manually cropped the vehicles from them. For the negative images you can use any image that doesn’t contain the object you are trying to identify. I used CCTV images that did not contain any vehicles or I manipulated the images to remove any vehicles present.

Below are some of the cropped positive images manually extracted from CCTV images (I have collected them together for this blog article but they are stored as separate images).

car montage

Below are some of the cropped negative images manually extracted from CCTV images (As with the positive images the negative images are stored separately).

negative car montage

One trick you can use to increase the number of images is to cut an image into tiles or flip the image horizontally. Here is a simple script I use to increase the number of negative images:

NOTE: Install imagemagick to get the convert command

rm -f *tile*.jpg *flop*.jpg

convert -flop *.jpg flop_%02d.jpg
convert *.jpg -crop 3x3+20+20@ +repage +adjoin tile_%02d.jpg

Now you have many more images that can be used to train your classifier!

Step 2: Create Sample Images and generate a .vec file

The next step is to generate a metadata file called collection.dat. This file contains the file name, number of objects, position and dimensions of the objects within each image. I only have one object per image and crop the image to only contain the object so the position is (0,0) and the dimension of the object is the width and height of the image.

This is what the collection.dat metadata file looks like:

traffic/positive/taxi9_flop.jpg 1 0 0 14 12
traffic/positive/car16_flip.jpg 1 0 0 13 13
traffic/positive/car42_flip.jpg 1 0 0 8 9
traffic/positive/truck19.jpg 1 0 0 27 26
traffic/positive/car99_flip.jpg 1 0 0 16 15
traffic/positive/car36_flip.jpg 1 0 0 9 10
traffic/positive/van7_flop_flip.jpg 1 0 0 17 16
traffic/positive/truck20.jpg 1 0 0 72 73
traffic/positive/truck22_flop_flip.jpg 1 0 0 39 48
traffic/positive/car8_flop.jpg 1 0 0 17 18

Once you have generated the collection.dat metadata file you then need to create a samples.vec file that will be used by the classifier. This file contains all positive images within a single file.

After we have generated the samples.vec file and the collection.dat file we can invoke the script provided by the opencv library to train the classifier.

Step 3: Train the classifier

This is the script I use to generate the intermediate files and to train the classifier:

Train classifier script: train_classifier.sh After the script has completed you should find a file called cascade.xml in the classifier directory. This is the file you need for object detection.  

Step 4: Test the accuracy of the classifier

Here is code to use the classifier to find vehicles within an image and to highlight each detected vehicle:

Object detection script: object_detection.py

Cascade.xml file: cascade.xml

Running this code gives the following output:

Vehicles detected: 6

vehicle detection from a CCTV image

It’s not perfect, some cars that are obscured by trees are not detected and cars in the distance aren’t detected either. Still this is a very useful technique for extracting data from images or video frames.


Additional Files:

See Also:

  • Serbanescu Bogdan

    Hey mark ! Great post ! Can you share your classifier ? I would really appreciate ! Thanks !

    • Hi Serbanescu,

      I modified the article to include a link to the cascade.xml file generated in Step 3.

      • Dodoso Zangu

        hello Mark! can you help me on how to organize those source code and get the script running? i mean where to place those folders?

  • Dodoso Zangu

    hello Mark! can you help me on how to organize those source code and get
    the script running? i mean where to place those folders?

    • I created and executed all the scripts within the same directory. For the positive and negative images used to train the classifier I created a subdirectory called traffic and within that directory I created a “positive” and “negative” folder. See lines 3-4 in the Step 3 script to see where the script expects to find the images. Everything else can go in your current working directory.

      • Dodoso Zangu

        Am a new beginner in python and openCV. Where do i create that working directory? is it in python? or in openCV? or just in my desktop? help please

        • Create the working directory anywhere on your machine, e.g.

          mkdir some_directory
          cd some_directory

          # TODO: copy scripts to this directory

          mkdir -p traffic/positive
          mkdir -p traffic/negative

          # TODO: unzip positive and negative images to the appropriate directory just created

          # now run the scripts, you will need a properly installed system with python, opencv, ipython
          # and for the asset preparation step you will need image magick installed

  • Dodoso Zangu

    Am a new beginner in python and openCV. Where do i create that working
    directory? is it in python? or in openCV? or just in my desktop? help
    please

    • Osama Abbas ♑️

      First of all after setting up Python (Using Defaults), you should add the Environment Variable of the directory you have the data in to PYTHONPATH you can do this easily (Google it). Secondly after installing OpenCV you should copy “cv2.pyd” from C:opencvbuildpython2.7 and paste it in C:Python27Libsite-packages. After that open Python IDLE from start, and type “import cv2” if no errors appear, then you are done and you are able to use OpenCV with Python. Note: Try to install everything in 32-bit even if you are having a PC with 64-bit Architecture, because most of the Packages and Libraries are made for in 32-bit first.

      To get the best of it, I recommend using Python 2.7.11 and OpenCV 2.4.11. There is OpenCV 3.X version available but I am still using 2.4.11 because it has everything I need, and 3.X is still under progress till the moment of writing this post.

      I hope I helped you.

    • Osama Abbas ♑️

      Oh, I forgot to mention something, try to keep all your work in one folder to avoid annoying extra requirements.

  • Miha Susteric

    Hi, first thanks for the great work you did here, i have to say it’s a really nice tutorial.
    Second would you be so kind and give a link to cctv image you used to test classifier. I had to change few things in the script as i’m using Mac OS and would like to try recognition with my .xml file. Thanks!

    • mark

      In the additional files section at the bottom of the post I’ve included a link to a zip file containing a large number of webcam images you can use to test your classifier. The images are not all for the same location, some are in colour and some in black and white so you probably want to test with only a subset of the images.

    • In the additional files section at the bottom of the post I’ve included a link to a zip file containing a large number of webcam images you can use to test your classifier. The images are not all for the same location, some are in colour and some in black and white so you probably want to test with only a subset of the images.

      • Miha Susteric

        Thanks!

  • jj pan

    Hi Mark, is it possible to import cv2 using jyupter?

    Kept getting : ImportError: No module named ‘cv2’

  • Rahul Agnihotri

    Hi Mark,

    First of all, let me thank you article.

    However, i am not good at image processing and computer vision. just learning. I have some sample images of car as part of my project in the degree course. The output should be as follows:

    1) Identify the vehicle (car, bus or truck) and then identify the color of the same.

    Using your code i am unable to detect the vehicle. The issue is in some images, we might come up with multiple vehicles but we need to focus on the vehicle that covers most part of the image.

    Help is really appreciated and required. Would be good if you can share the complete code where I can just update the link of the image in which vehicle needs to be identified.

    Regards,
    Rahul

    • Osama Abbas ♑️

      Hi Rahul,

      Mark’s code is good, but not that good. He did not enough cars while creating the XML file he uploaded “cascades.xml”, that’s why you sometimes get some cars “virtual cars” in the image that do not exist. You should do your own data set of cars for training. If you don’t have enough time for that you can google “data set of objects” where you can find free downloadable vehicle data sets, and the only thing you need to do is to put the data set you will download instead of the “cascades.xml” one.

      The process of creating positives and negatives needs a lot of images ex. 5000 image each if possible for more accurate results. Unfortunately, if you want to detect all vehicles in an image you should do one by yourself, because not all of the data sets available on the internet are not good enough.

      I recommend using https://github.com/AALEKH/Python-Based-Image-Recogntion/blob/master/cars3.xml , it’s somehow good and fulfilled my purpose.

      I hope I helped. If you want to ask anything feel free to ask.

  • Osama Abbas ♑️

    Hi. I am an intermediate engineering student in DIP. I want to mention something at one point the camera won’t detect the car far away from it, because each camera has a property called Field of View (FOV), when the object exists this field the camera won’t be able to detect it, it’s not a problem in the code, it’s the camera itself.

    Thanks a lot for the great post (Y)

  • Vikas Kr.

    Hi All,

    I have 3 problem statements, or I can say I wanna do PoC on following given topics,
    Could you please help me for the same ?

    1. Shape detection in any pic
    2. Objects detection in any pic
    3. How to predict from 1000 images that its cat not anything else.

    The solution of above problems should be in Python (Jupyter), Opencv and with window 7 only (In rare case I can do practice on Ubuntu also).
    Request you to please send me any link/links, from your blog or any other blogs, which I can follow easily. My assignment deadline is this saturday.

    Thanks for your support.
    Regards,
    Vikas
    (vikas.reply@gmail.com)

  • Mina César

    Hello ,
    How could I extract the coordinates of the rectangle around the car from the image as I would like to use these coordinates to crop images

  • Valki

    Hi, how do i run the python script? im getting error at %matplotlib inline, what am i doing wrong?

  • TheEasyWay

    I get an error when I run

    find traffic/positive/ -type f -name ‘*.jpg’ > positives.dat

    find traffic/negative/ -type f -name ‘*.jpg’ > negatives.dat

    I can’t find any documentation on these commands. How can I run them?

  • anbu

    hi
    how to getting xml file

  • Simon Burfield

    When I run your bash script I get

    OpenCV Error: Bad argument (Can not get new positive sample. The most possible reason is insufficient count of samples in given vec-file.
    ) in get, file /Users/travis/miniconda3/conda-bld/work/opencv-3.1.0/apps/traincascade/imagestorage.cpp, line 157
    libc++abi.dylib: terminating with uncaught exception of type cv::Exception: /Users/travis/miniconda3/conda-bld/work/opencv-3.1.0/apps/traincascade/imagestorage.cpp:157: error: (-5) Can not get new positive sample. The most possible reason is insufficient count of samples in given vec-file.
    in function get

    classifier/train_classifier.sh: line 29: 8519 Abort trap: 6 opencv_traincascade -data classifier -featureType HAAR -vec samples.vec -bg negatives.dat -numPos 600 -numNeg 470 -numStages 20 -precalcValBufSize 6000 -precalcIdxBufSize 6000 -minHitRate 0.999 -maxFalseAlarmRate 0.5 -mode ALL -w 20 -h 20

    Process finished with exit code 134 (interrupted by signal 6: SIGABRT)

    • Simon Burfield

      I am now getting Train dataset for temp stage can not be filled. Branch training terminated.

    • hossein Hojjat Ansari

      your problem:
      you must change -numPos 600 to -numPos 196 (becuse in your postive folder you have 196)
      and -numNeg 470 to -numNeg 54

  • Priyanka Kochhar

    Another interesting approach is to use OpenCV HOG + SVM classifier to detect vehicles. Check out this blog:
    https://medium.com/@priya.dwivedi/automatic-vehicle-detection-for-self-driving-cars-8d98c086b161#.6x2k4szf5

  • Harsh Shah

    Can you please share your cascades.xml file. I have been trying to train a classifier with several datasets but haven’t got good results. Anyone who has a well trained classifier, please share! Thanks.

    • Marcin Wałach

      I’m seeing the same 27 vehicles with the same files. But when I change maxSize=(20,20) result is similar, 7 cars + sign