Motion vecotrs in MPEG represent essentially "free" data. They are computationally very expensive to compute from a series of frames, but MPEG files come with them already available. Thus, if the motion vectors already encoded in MPEG could be utilized to perform a simple object tracking function, we would have a method of object tracking that could be performed in real-time on virtually any platform.

The first step is, of course, actually extracting the motion vectors from the MPEG file. We accomplished this using the mpeg_stat utility downloadable from Berkeley's BMRC site. With a little work, you can load the motion vectors into MATLAB and then the work is relatively easy.

It is important to note, however, that the B motion vectors are not useful for our purpose, so we are sub-sampling the video in a sense. We only have motion vectors for P frames. This means that we are looking at less than 10 frames per second (because we also have no motion vectors for I frames), but for our purpose, 10 frames per second will suffice.

To fill in the gaps left by I frames in the motion vector sequence, a simple method which works as well as any other we tried was to simply average the vectors from the last P frame and the next P frame.

With the resulting vectors, we could use MATLAB to produce movies showing the vector fields changing over time. Also, we produced a 3-D movie which showed only the magnitude (not direction) of each vector as time passes. To reduce the effects of erratic motion vectors, a filter which smoothes the motion vector at a given point over time was also applied. (The final movie shows the filtered product).