Isaac Wang and Jaime Ruiz. 2021. International Journal of Human–Computer Interaction (Mar. 2021), 1–26. DOI: https://doi.org/10.1080/10447318.2021.1898851
Back to ProjectsExamining the Use of Nonverbal Communication in Virtual Agents
It’s a Joint Effort: Understanding Speech and Gesture in Collaborative Tasks
Diana’s World: A Situated Multimodal Interactive Agent
User-Aware Shared Perception for Embodied Agents
Exploring Virtual Agents for Augmented Reality
Cooperating with Avatars Through Gesture, Language and Action
EASEL: Easy Automatic Segmentation Event Labeler
EGGNOG: A Continuous, Multi-modal Data Set of Naturally Occurring Gestures with Ground Truth Labels
Exploring the Use of Gesture in Collaborative Tasks
Communicating and Acting: Understanding Gesture in Simulation Semantics
Communication Through Gestures, Expression and Shared Perception
Personal assistants such as Siri have changed the way people interact with computers by introducing virtual assistants that collaborate with humans through natural speech-based interfaces. However, relying on speech alone as the medium of communication can be a limitation; non-verbal aspects of communication also play a vital role in natural human discourse. Thus, it is necessary to identify the use of gesture and other non-verbal aspects in order to apply them towards the development of computer systems. The goal of this project is to understand how systems can model non-verbal communication as a means to mimic human-human communication.
People
Funding
Communication through Gestures, Expression and Shared Perception
Related Publications
Isaac Wang, Pradyumna Narayana, Dhruva Patil, Rahul Bangar, Bruce Draper, Ross Beveridge, and Jaime Ruiz. 2021. In Human-Computer Interaction. Interaction Techniques and Novel Applications (Lecture Notes in Computer Science), 159–178. https://doi.org/10.1007/978-3-030-78465-2_13
Nikhil Krishnaswamy, Pradyumna Narayana, Rahul Bangar, Kyeongmin Rim, Dhruva Patil, David McNeely-White, Jaime Ruiz, Bruce Draper, Ross Beveridge, and James Pustejovsky. 2020. Proceedings of the AAAI Conference on Artificial Intelligence 34, 09: 13618–13619. https://doi.org/10.1609/aaai.v34i09.7096
David White, Ross Beveridge, Bruce Draper, Francisco Ortega, Rahul Bangar, Dhruva Patil, James Pustejovsky, Nikhil Krishnaswamy, Kyeongmin Rim, Jaime Ruiz, and Isaac Wang. 2019. In 2019 IEEE International Conference on Humanized Computing and Communication (HCC), Laguna Hills, CA, USA, 2019, pp. 46-51.
Isaac Wang, Jesse Smith, and Jaime Ruiz. 2019. In 2019 CHI Conference on Human Factors in Computing Systems Proceedings (CHI 2019), May 4–9, 2019, Glasgow, Scotland, UK. ACM, New York, NY, USA. Paper 281, 10 pages. https://doi.org/10.1145/3290605.3300511
Pradyumna Narayana, Nikhil Krishnaswamy, Isaac Wang, Rahul Bangar, Dhruva Patil, Gururaj Mulay, Kyeongmin Rim, Ross Beveridge, Jaime Ruiz, James Pustejovsky, and Bruce Draper. 2018. In Intelligent Systems and Applications (Advances in Intelligent Systems and Computing), 272–293.
Isaac Wang, Pradyumna Narayana, Jesse Smith, Bruce Draper, Ross Beveridge, and Jaime Ruiz. 2018. In Proceedings of the 23rd International Conference on Intelligent User Interfaces (IUI ’18). ACM, New York, NY, USA. DOI: https://doi.org/10.1145/3172944.3173003
Isaac Wang, Mohtadi Ben Fraj, Pradyumna Narayana, Dhruva Patil, Gururaj Mulay, Rahul Bangar, J. Ross Beveridge, Bruce A. Draper, and Jaime Ruiz. 2017. In 2017 12th IEEE International Conference on Automatic Face Gesture Recognition (FG 2017), 414–421. DOI: https://doi.org/10.1109/FG.2017.145
Isaac Wang, Pradyumna Narayana, Dhruva Patil, Gururaj Mulay, Rahul Bangar, Bruce Draper, Ross Beveridge, and Jaime Ruiz. 2017. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA ’17). Association for Computing Machinery, New York, NY, USA, 2990–2997. https://doi.org/10.1145/3027063.3053239
Nikhil Krishnaswamy, Pradyumna Narayana, Isaac Wang, Kyeongmin Rim, Rahul Bangar, Dhruva Patil, Gururaj Mulay, Ross Beveridge, Jaime Ruiz, Bruce Draper, and James Pustejovsky. 2017. In 12th International Conference on Computational Semantics (IWCS 2017)–Short papers.