Exploring augmented reality interaction for everyday multipurpose wearable robots

Multipurpose wearable robots are an emerging class of devices, which offer intriguing interaction potential. The interoperability of multipurpose wearable robots and other types of wearable devices remains largely uninvestigated. We take the first steps to bridge this gap by presenting a framework for integrating augmented reality (AR) and multipurpose wearable robots. Our framework uses the publisher-subscriber model to expose different robot functionalities as services on a network. These can be invoked by the AR system. This model is advantageous in coping with different robot morphologies and various interaction methods. We implemented a prototype system using our framework by integrating an AR head-mounted display (HMD) and a wrist-worn robot, and demonstrate four experiences utilizing our prototype solution: 1) Robot status display, 2) shape-changing menus, 3) a media player and 4) a robot pose controller. To evaluate our approach, we performed a user study, which gauged user impressions and usability of developed experiences. Results indicate that our approach was well received, though participants highlighted a number of challenges in AR tracking when interacting within some of the experiences. Lastly, we discuss limitations and future research direction for our project.