Virtual tangible widgets: Seamless universal interaction with personal sensing devices

Using a single personal device as an universal controller for diverse services is a promising approach to solving the problem of too many controllers in ubiquitous multimodal environments. However, the current approaches to universal controllers cannot provide intuitive control because they are restricted to traditional mobile user interfaces such as small keys or small touch panels. We propose Vidgets, which is short for virtual tangible widgets, as an approach to selecting and controlling ubiquitous services with virtually implemented tangible user interfaces based on a single sensing personal device equipped with a digital camera and several physical sensors. We classify the use of the universal controller into the three stages: (a) searching for a service, (b) grasping the service and (c) using the service. User studies with our prototype implementation indicate that the smooth transition and integration of the three stages improve the overall interaction with our universal controller.