Visual Detection of Motion - Part I

(Part II)

Most motion detection schemes employ passive IR sensors to detect local changes in heat relative to ambient background. This can work well as far as it goes, but it is limited in range and lacks the ability to discriminate and identify. I wanted to experiment with using image processing algorithms to achieve detection allowing the possibility of identification as well.

I was interested in detecting motion in low-light conditions as well as daytime, so I started by employing an inexpensive CCD camera with superior low-light sensitivity. I mounted it looking out one of my north-facing second story windows, down at my front yard. The road in front of my house is within its field of view, as well as part of my driveway.

My first approach employed an algorithm which looked at the the probability densities of the separate RGB color intensities of successive image differences. For example, foliage moving in the wind tends to produce higher green and blue densities than red, so detecting on red thresholds while normalizing by green and blue works better than detecting based on pure grayscale intensities.

The downside of this approach is the lack of discrimination based on size of an object where pixels are moving en masse. In otherwords, a lot of motion distributed evenly over the image, such as occurs during wind moving branches of trees or during rain, will generate characteristics difficult to discriminate from those of a car driving by on the road or an elk walking across the front lawn.

In order to achieve the highest possible processing throughput I coded this algorithm in C1. It turns out this was not necessary, as the tentpole was the camera itself. USB-1 is too slow for full-resolution real-time motion — though it has some advantages in terms of reducing the processing load on the CPU.

I only store images when motion is detected. So, even though the algorithm is processing imagery continuously, the stored images go from one detection event to the next with nothing being stored during the quiet intervals. And due to camera throughput constraints, the number of images I could capture per second is about 1/3 what would be necessary to attain realtime video. This results in montages of motion with the character of silent movies— everything speeded up about 3x normal.

The first day I ran the camera any length of time I was surprised how well it worked2. Improvements would be gained by playing with the many parameters, but basically it did just what it was supposed to do.

Unfortunately, when I sent the link out to some friends I quickly discovered most of them couldn't view my movies3. I built it on a Linux desktop using a standard, modern, good-quality video codec. But Windows continues to eschew standards, singing "I gotta be me" like the plumbing contractor in The Money Pit— an accidentally apt metaphor for owning Windows. Windows users who had installed quicktime could play it, and a few others who had downloaded special plug-ins or players, but no one else could.

I continued to run the algorithm pretty much around the clock while modifying and tweaking its details. Most of the sequences looked pretty much like the first. This one4 turned out to be more interesting than most.

captured elk


(1) Note: I also modified the timing loop of the PWC (Philips Web Camera) driver to provide a reduced latency mode of operation.

(2) First test:

movers_20071124 (avi)NIL Mb
movers_20071124 (mpg)NIL Mb

Throughput effects make the cars look like they are driving real fast when in fact they seldom go over 25mph. This effect is most obvious with the walkers. Note how well it picks up small movers like the cat and the squirrel.

(3) Note: you may or may not be able to play the file above. It is a standard mpeg-4 video in an AVI container that plays fine on any Linux desktop or Mac I've tried. If you can't (and want to) the quicktime plugin for Windows will—I believe— play it. And newer versions of Windows may also.

(4) Workers were scheduled to come fix a problem with our skylight sometime in the middle of the day. Ever want to know what workers work like when you're not around?

movers_20071205 (mpg)NIL Mb

Watch one guy throw garbage in our bushes, and talk on his cellphone non-stop while his the other guy does all the work. Must be the supervisor—or at least, on the clock, while the other guy acts like he's getting paid by the job. I know which one I'd keep.

Video compatibility Issues

The codec incompatibility issue with Windows became a nuisance. While trying to solve the problem I came across an article by Gary Steele containing some very useful information. I subsequently wrote up details of my video processing to remind myself later. The videos I generated as a result seemed to work everywhere they had failed before.

The downside I found was that none of the Windows compatible codecs I found do as good a job: they are all either less efficient (make bigger files) or lower quality (grainy). I ended up opting for quality with the result that the Windows-compatible videos are at least twice the size of their AVI counterparts. Tough luck, Windows users.

Results

One day I discovered the motion detector had captured a herd5  of elk making their way across my front yard from a meadow to the right (out of view), where they often come through and graze. Stay with it— they'll keep coming right up to the end.

All four of the linked clips were made with the same sequence of images. For the first I used the original mpeg-4 avi technique. The other three are different attempts at Windows compatibility. I adjusted the video compression parameters so that all four have about the same visual quality to my eye, but where any greater compression causes degradation. The original is by far the smallest — less than half (almost a third) the size of the other three. Most Windows users will only be able to play the last two. The second plays on some versions of Windows and not on others. I don't know why and don't have it to experiment with.

The wind and rain poses the problem of filling up my hard drive with clutter real fast. Some kind of discrimination is necessary to prevent this6 — which will be the focus of my next set of improvements.

(5) Its windy as all get out and many of the frames were triggered by false-alarms. But the algorithm is also very sensitive — if you watch carefully through the trees across the street, you will see the larger part of the herd. On some days I've counted as many as 37 elk in the herd that lives in this area.

movers_20071203 (avi)NIL Mb
movers_20071203_msmpeg4v2 (mpg)NIL Mb
movers_20071203_wmv1 (mpg)NIL Mb
movers_20071203_wmv2 (mpg)NIL Mb

(6) I know where of I speak. One day it rained hard all day. When I returned that evening my /home partition had 0% space left. Absolutely 0-bytes. Linux was still humming along, but I had to play some interesting games to free up space without rebooting it (its glorious not having to reboot your OS when weird stuff happens).