Sharing digital and physical contents on and above distributed surfaces using active infrared keying

This paper presents SyncSurface, a system framework for distributed collaboration, which shares digital contents and physical contents on and above distributed surfaces. In such sharing, one important technique is to segment each frame, in a video stream from a camera placed above a surface, into its foreground (e.g., objects and users’ bodies) and its background (i.e., displays). This paper describes a simple and low-cost segmentation technique called “active infrared keying” that uses a display that emits infrared light from its surface. We have implemented two systems using the framework to examine the feasibility of the technique. The results of our pilot studies indicate that the technique enables users to communicate with each other using physical miniatures on the surfaces, gestures for pointing out physical objects, and facial expressions.