When Smart Devices Interact With Pervasive Screens: A Survey

The meeting of pervasive screens and smart devices has witnessed the birth of screen-smart device interaction (SSI), a key enabler to many novel interactive use cases. Most current surveys focus on direct human-screen interaction, and to the best of our knowledge, none have studied state-of-the-art SSI. This survey identifies three core elements of SSI and delivers a timely discussion on SSI oriented around the screen, the smart device, and the interaction modality. Two evaluation metrics (i.e., interaction latency and accuracy) have been adopted and refined to match the evaluation criterion of SSI. The bottlenecks that hinder the further advancement of the current SSI in connection with this metrics are studied. Last, future research challenges and opportunities are highlighted in the hope of inspiring continuous research efforts to realize the next generation of SSI.