Video Matching


This paper describes a method for bringing two videos (recorded at different times) into spatiotemporal alignment, then comparing and combining corresponding pixels for applications such as background subtraction, compositing, and increasing dynamic range. We align a pair of videos by searching for frames that best match according to a robust image registration process. This process uses locally weighted regression to interpolate and extrapolate high-likelihood image correspondences, allowing new correspondences to be discovered and refined. Image regions that cannot be matched are detected and ignored, providing robustness to changes in scene content and lighting, which allows a variety of new applications.

Full Citation

Peter Sand and Seth Teller. 2004. Video Matching. ACM Transactions on Graphics (TOG) 22, 3, 592-599.

Please see our Prototype Change Detection System based on this method.

More Information

Paper (PDF, 32.6MB)
Video (MPEG1, 39.2MB)
Video (Windows Media 9, 21.6MB)
Stata Pan Demo (MPEG1, 11.1MB)
Stata Pan Demo (MPEG2, 14.4MB)
Stata Pan Demo (MPEG4, 6.4MB)
Driving along Broadway sequences
Technical Report

Fan2 dataset
Fan3 dataset