Diana's World: A Situated Multimodal Interactive Agent
Nikhil Krishnaswamy, Pradyumna Narayana, Rahul Bangar, Kyeongmin Rim, Dhruva Patil, David McNeely-White, Jaime Ruiz, Bruce Draper, Ross Beveridge, and James Pustejovsky
State of the art unimodal dialogue agents lack some core aspects of peer-to-peer communication—the nonverbal and visual cues that are a fundamental aspect of human interaction. To facilitate true peer-to-peer communication with a computer, we present Diana, a situated multimodal agent who exists in a mixed-reality environment with a human interlocutor, is situation- and context-aware, and responds to the human's language, gesture, and affect to complete collaborative tasks.
Citation
Nikhil Krishnaswamy, Pradyumna Narayana, Rahul Bangar, Kyeongmin Rim, Dhruva Patil, David McNeely-White, Jaime Ruiz, Bruce Draper, Ross Beveridge, and James Pustejovsky. 2020. Diana’s World: A Situated Multimodal Interactive Agent. Proceedings of the AAAI Conference on Artificial Intelligence 34, 09: 13618–13619. https://doi.org/10.1609/aaai.v34i09.7096
BibTex
@article{krishnaswamy_dianas_2020, title = {Diana's World: A Situated Multimodal Interactive Agent}, volume = {34}, issn = {2374-3468, 2159-5399}, url = {https://aaai.org/ojs/index.php/AAAI/article/view/7096}, doi = {10.1609/aaai.v34i09.7096}, shorttitle = {Diana's World}, abstract = {State of the art unimodal dialogue agents lack some core aspects of peer-to-peer communication—the nonverbal and visual cues that are a fundamental aspect of human interaction. To facilitate true peer-to-peer communication with a computer, we present Diana, a situated multimodal agent who exists in a mixed-reality environment with a human interlocutor, is situation- and context-aware, and responds to the human's language, gesture, and affect to complete collaborative tasks.}, pages = {13618--13619}, number = {9}, journaltitle = {Proceedings of the {AAAI} Conference on Artificial Intelligence}, shortjournal = {{AAAI}}, author = {Krishnaswamy, Nikhil and Narayana, Pradyumna and Bangar, Rahul and Rim, Kyeongmin and Patil, Dhruva and {McNeely}-White, David and Ruiz, Jaime and Draper, Bruce and Beveridge, Ross and Pustejovsky, James}, date = {2020-04-03}, }