OpenCV librtary - detection of human body

[FlyingDutch]

Hello,

I am trying to write application (using OpenCV library) in C++ for detecting in room human bodies (in many various positions). I am using "openCV" version 4.5.1 on Windows 10 OS with USB camera attached to my laptop. As a compiler (IDE) I am using "Visual Studio 2019" (which is working fine). As a target hardware (for detection) I am going to use "Raspberry Pi Compute Module 4" - see links:

https://www.raspberrypi.org/products/compute-module-4/?variant=raspberry-pi-cm4001000

https://datasheets.raspberrypi.org/cm4/cm4-datasheet.pdf

with a MIPI Camera attached to it.

First I tried to write a face detection program using "Haar cascade classifier". I used pre-trained classifiers from "OpenCV" folder: "\opencv\build\etc\haarcascades". Face detection is working efficiently - I test it on few video files downloaded from "YouTube". Then I tried to detect a bodies with this method ("Haar cascade classifier") and pre-trained classifiers from "OpenCV" folder: "\opencv\build\etc\haarcascades". I used for detection such classifiers:

1) haarcascade_fullbody.xml
2) haarcascade_upperbody.xml
3) haarcascade_lowerbody.xml

Results in detection human bodies are disappointing. So I started to wonder if "Cascade Harr Classifier" is proper method for detecting human bodies. Maybe someone could to point me better method for this task - I mean different algorithm/method. I would care the algorithm would be implemented using OpenCV library (I prefer C++, but Python is also OK).

I have also second question related to comparison "OpenCV" and "Keras" (together with Tensorflow). Has somebody an exprierience and can compare a results achieved in full human body detection using OpenCV or Keras framework.

Thanks in advance and Regards

 

[BradtheRad]

I imagine it's far easier to make an algorithm that detects any slight movement of a limb, rather than an algorithm to recognize a motionless form. Or else use an infra-red camera, or a filter which converts infra-red to a temperature reading.

Motion-capture software follows positions/movement of lights attached to a body and limbs (of dancers, etc.). It's an expensive process.

There are also less sophisticated low-cost programs which recognize movements of a plain-clothed body and limbs. I suppose the user must try to avoid confusing the program unnecessarily. It may be fruitful for you to look into the workings of such programs.

 

[FlyingDutch]

Hello @BradtheRad,

thanks for your answer. I am using infra-red motion detection sensor in this system I am designing, and these sensor work OK. The camera is added to this system as a protection and have to work basically on still images. Camera has to for example detect people which lose consciousness. This is system for sterilization of elevator by UVC lamp (ultra-violet radiation). So I have to get certainty that all people to leave elevator before system to switch-on UVC lamp.

Regards

 

[andre_luis]

Perhaps if you show a sample of the pictures it could help to give a better insight of the suited approach, most often, each kind of pattern ask for a different algorighm.

 

[FlyingDutch]

Perhaps if you show a sample of the pictures it could help to give a better insight of the suited approach, each kind of pattern ask for a different algorighm.

Hello @andre_teprom,

I just working on it, didn't finish yet. I tested detecting human bodies on video downloaded from Youtube (subject: gym exercises). I used every fourth frame from video-stream, but results of detection had been very weak (I read that "cascading Haar classifier" is not good choice for human body detection). Few years ago I was using a SVM algorithm:

Support vector machine - Wikipedia

en.wikipedia.org

for similiar purpose, and results had been encouraging. But I used that time MATLAB and now I have to use "OpenCV" library. Maybe I will try also "Keras" framework (for training ANN) and "Tensorflow" for running that on target device.

Regards

 

[andre_luis]

AFAIK, SVM is quite strict of being scale and shift invariant so that even the smallest change on the picture could lead to mistake. As said, each pattern require a distinct approach; we don't know what is the angle and distance you are taking the pictures from ( eg.: from top or side view ? How many bodies fit each picture ? ).

 

[FlyingDutch]

AFAIK, SVM is quite strict of being scale and shift invariant so that even the smallest change on the picture could lead to mistake. As said, each pattern require a distinct approach; we don't know what is the angle and distance you are taking the pictures from ( eg.: from top or side view ? How many bodies fit each picture ? ).

There is camera with fish eye lens (angle approximately 160 degrees in both axies). Camera is located two meters above floor on the wall opposed to entry to elevator. Image from camera embracess floor and part of walls. People can stand near to camera and in extreme to lie on the floor, so the scale of images can differ significantly. If people in elevator are in movement then two infra-red move sensors detects them with great certainty, the camera is additional protection for cases in which humans are not in movement.

Regards

 

[BradtheRad]

Suppose you take a photograph of the empty compartment and store it in computer memory? That becomes your reference for comparison when nobody is in the elevator. Any discrepancy in the live image means someone is present.

I imagine, however, this method can be defeated by a smaller object such as a glove dropped on the floor. So perhaps a 'fuzzy logic' algorithm is useful for making the decision.

 

[FlyingDutch]

Suppose you take a photograph of the empty compartment and store it in computer memory? That becomes your reference for comparison when nobody is in the elevator. Any discrepancy in the live image means someone is present.

I imagine, however, this method can be defeated by a smaller object such as a glove dropped on the floor. So perhaps a 'fuzzy logic' algorithm is useful for making the decision.

Hello @BradtheRad,,

if I only detect objects in elevator and for example not differentiate marks of ketchup on the floor or wall from human it would be an error in system operasting. I am still in phase of thinking how to build algorithm for this system.

Best Regards

 

[andre_luis]

The approach that seems most promising in this case could be to segment the images against the background and track the movement of the center of mass of "objects" into the elevator, considering those that move coordinately within a certain tolerance and proximity as a single "person" (for example someone carrying a large suitcase)

 

[FlyingDutch]

Hello again,

I have few days to investigate and try few approaches to solve my problem - detection of human bodies in a room. I focused on solutions implemented in OpenCV library, and this time I didn't take into consideration solutions based on ANNs (Artificial Neural Networks), especially implemented in Keras framework.

First I decided to take into account two different methods - one for face detection and second for human body detection. In target solution both methods will be operating on image frames from camera and if one of them would give positive result it will be adopted that human was detected.

For face detection I decided to use "Face detection using Haar cascade Classifier". See this link:

https://www.bogotobogo.com/python/O...n_Face_Detection_Haar_Cascade_Classifiers.php

This pretrained Haar classifier supplied with OpenCV - 'haarcascade_frontalface_default.xml' properly detect on more or less than sixty percent of images (on which it hasn't be trained). I am going to train it additionally on more samples made by me (photos of my colleagues) - like descripted in this page:

https://docs.opencv.org/2.4/doc/user_guide/ug_traincascade.html

The second method (algorithm) for detection human bodies is based on HOG descriptor (Histogram of Oriented Gradients) connected with "Support Vector machines" (SVM). See these links:

https://learnopencv.com/histogram-of-oriented-gradients/

https://www.pyimagesearch.com/2015/11/09/pedestrian-detection-opencv/

https://www.thepythoncode.com/article/hog-feature-extraction-in-python

**broken link removed**

It also has proper detection of humaqn bodies ratio on level about sixty percent, and I also would like to train SVM with HOG Descriptor (pretrained form OpenCV). I think this page is a good start point for doing it:

https://stackoverflow.com/questions/57598104/training-of-svm-and-prediction-using-opencv-with-python

After additional training of this two methods and checking it inference (prediction) on new images I would like implement CNN(Convolutional Neural Networks) with Keras framework. After implementation I would like to compare OpenCV methods (first two described above) with its Keras analogue.

See this links to familiarize yourself with CNN:

https://towardsdatascience.com/a-co...nal-neural-networks-the-eli5-way-3bd2b1164a53

https://cs231n.github.io/convolutional-networks/

Best Regards