Selected Publications

2021

Expanding Explainability: Towards Social Transparency in AI systems
Upol Ehsan, Q. Vera Liao, Michael Muller, Mark Riedl, Justin D. Weisz
To appear in CHI 2021: ACM Conference on Human Factors in Computing Systems
* Best Paper Award, Honorable Mention

Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
Umang Bhatt, Yunfeng Zhang, Javier Antorán, Q Vera Liao, Prasanna Sattigeri et al.
To appear in AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES2021)

Model LineUpper: Supporting Interactive Model Comparison at Multiple Levels for AutoML
Shweta Narkar, Yunfeng Zhang, Q. Vera Liao, Dakuo Wang, Justin D. Weisz
To appear in IUI 2021: ACM International Conference on Intelligent User Interface

Facilitating Knowledge Sharing from Domain Experts to Data Scientists for Building NLP Models
Soya Park, April Yi Wang, Ban Kawas, Q. Vera Liao, David Piorkowski, Marina Danilevsky
To appear in IUI 2021: ACM International Conference on Intelligent User Interface

Why or Why Not? The Effect of Justification Styles onChatbot Recommendations
Daricia Wilkinson, Öznur Alkan, Q. Vera Liao, Massimiliano Mattetti, Inge Vejsbjerg, Bart P. Knijnenburg, Elizabeth Daly
To appear in ACM Transactions on Information Systems (TOIS)

Doc2Bot: A Document Grounded Bot Framework
Kshitij Fadnis, Pankaj Dhoolia, Li Zhu, Q Vera Liao, Steven Ross, Nathaniel Mills, Sachindra Joshi, Luis Lastras
AAAI 20221 Demonstration Track:AAAI Conference on Artificial Intelligence

Question-Driven Design Process for Explainable AI User Experiences
Q. Vera Liao, Milena Pribić, Jaesik Han, Sarah Miller, Daby Sow
Working paper (welcome feedback :)

How Much Automation Does a Data Scientist Want?
Dakuo Wang, Q. Vera Liao, Yunfeng Zhang, Udayan Khurana, Horst Samulowitz, Soya Park, Michael Muller, Lisa Amini
Working paper

Uncertainty Quantification 360: A Holistic Toolkit for Quantifying and Communicating the Uncertainty of AI
Soumya Ghosh, Q. Vera Liao, Karthikeyan Natesan Ramamurthy, Jiri Navratil, Prasanna Sattigeri, Kush R Varshney, Yunfeng Zhang
Working paper

2020

Questioning the AI: Informing Design Practices for Explainable AI User Experiences
Q. Vera Liao, Daniel Gruen, Sarah Miller
CHI 2020: ACM Conference on Human Factors in Computing Systems
* Best Paper Award, Honorable Mention
The talk is online!

Explainable Active Learning (XAL): Toward AI Explanations as Interfaces for Machine Teachers
Bhavya Ghai, Q. Vera Liao, Yunfeng Zhang, Rachel Bellamy, Klaus Mueller
CSCW 2020: ACM Conference on Computer Supported Cooperative Work and Social Computing

Human-AI Collaboration in a Cooperative Game Setting: Measuring Social Perception and Outcomes
Zahra Ashktorab, Q. Vera Liao, Casey Dugan, James Johnson, Qian Pan, Wei Zhang, Sadhana Kumaravel, Murray Campbell
CSCW 2020: ACM Conference on Computer Supported Cooperative Work and Social Computing

Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making
Yunfeng Zhang, Q. Vera Liao, Rachel Bellamy
FAT* 2020: ACM Conference on Fairness, Accountability, and Transparency

Conversational Interfaces for Information Search (book chapter)
Q. Vera Liao, Werner Geyer, Michael Muller, Yasaman Khazaen
Springer Human-Computer Interaction Series

Tell Me About Yourself: Using an AI-Powered Chatbot to Conduct Conversational Surveys
Ziang Xiao, Michelle X Zhou, Q Vera Liao, Gloria Mark, Changyan Chi, Wenxi Chen, Huahai Yang
TOCHI: ACM Transactions on Computer-Human Interaction

AI Explainability 360: An Extensive Toolkit for Understanding Data and Machine Learning Models
Vijay Arya, Rachel KE Bellamy, Pin-Yu Chen, Amit Dhurandhar, Michael Hind, Samuel C Hoffman, Stephanie Houde, Q. Vera Liao, et al.
Journal of Machine Learning Research

Doc2Dial: a Framework for Dialogue Composition Grounded in Documents
Song Feng, Kshitij Fadnis, Q. Vera Liao, Luis A. Lastras
AAAI 20220 Demonstration Track:AAAI Conference on Artificial Intelligence

Business (mis)Use Cases of Generative AI
Stephanie Houde, Vera Liao, Jacquelyn Martino, Michael Muller, David Piorkowski, John Richards, Justin Weisz, Yunfeng Zhang
IUI 2020 Workshop on Human-AI Co-Creation with Generative Models

2019

Resilient Chatbots: Repair Strategy Preferences for Conversational Breakdown
Zahra Ashktorab, Mohit Jain, Q. Vera Liao, Justin D. Weisz
CHI 2019: ACM Conference on Human Factors in Computing Systems

How Data Science Workers Work with Data: Discovery, Capture, Curation, Design, and Creation
Michael Muller, Ingrid Lange, Dakuo Wang, David Piorkowski, Jason Tsay, Q. Vera Liao, Casey Dugan, Thomas Erickson
CHI 2019: ACM Conference on Human Factors in Computing Systems

Explaining Models: An Empirical Study of How Explanations Impact Fairness Judgment
Jonathan Dodge, Q. Vera Liao, Yunfeng Zhang, Rachel K.E. Bellamy, Casey Dugan
IUI 2019: ACM International Conference on Intelligent User Interfaces
* Outstanding Paper Award

FarmChat: A Conversational Agent to Answer Farmer Queries
Mohit Jain, Pratyush Kumar, Ishita Bhansali, Q. Vera Liao, Khai Truong, Shwetak Patel
UbiComp 2019/Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Computing (IMWUT)

Bootstrapping Conversational Agents With Weak Supervision
Neil Mallinar, Abhishek Shah, Rajendra Ugrani, Ayush Gupta, Manikandan Gurusankar, Tin Kam Ho, Q. Vera Liao, Yunfeng Zhang, Rachel K.E. Bellamy, Robert Yates, Chris Desmarais, Blake McGregor
IAAI 2019: AAAI Conference on Innovative Applications of Artificial Intelligence

Human-AI Collaboration: Towards Socially-Guided Machine Learning
Q. Vera Liao, Rachel K.E. Bellamy, Michael Muller, Heloisa Candello
CHI 2019 Workshop on Human-Centered Machine Learning Perspectives
The talk is online!

Doc2Dial: a Framework for Dialogue Composition Grounded in Business Documents
Song Feng, Kshitij Fadnis, Q. Vera Liao, Luis A. Lastras
NeuIPS 2019 Workshop on Document Intelligence

Enabling Value Sensitive AI Systems through Participatory Design Fictions
Q. Vera Liao, Michael Muller
Working paper

2018

All Work and no Play? Conversations with a Question-and-Answer Chatbot in the Wild
Q. Vera Liao, Muhammed Mas-ud Hussain, Praveen Chandar, Matthew Davis, Marco Crasso, Dakuo Wang , Michael Muller, N. Sadat Shami, Werner Geyer
CHI 2018: ACM Conference on Human Factors in Computing Systems
The talk is online!

Face Value? Exploring the Effects of Embodiment for a Group Facilitation Agent
Ameneh Shamekhi , Q. Vera Liao, Dakuo Wang, Rachel Bellamy, Thomas Erickson
CHI 2018: ACM Conference on Human Factors in Computing Systems

Towards an Optimal Dialog Strategy for Information Retrieval Using both Open-ended and Close-ended Questions
Yunfeng Zhang, Q. Vera Liao, Biplav Srivastava
IUI 2018: ACM International Conference on Intelligent User Interfaces

Ways of Knowing for AI: (Chat)bots as Interfaces for Machine Learning [ Slides ]
Q. Vera Liao, Michael Muller
HCIC 2018: HCI and AI

The Central Role of Cognitive Computations in Human-Information Interaction (book chapter)
Wai-Tat Fu, Jessin Chin, Q. Vera Liao
Computational Interaction (Oulasvirta et al., editor)

2017

What’s in it for me? Self-serving versus Other-oriented Framing in Messages Advocating use of Prosocial Peer-to-peer Services
Rajan Vaish, Q. Vera Liao (co-first author), Victoria Bellotti
International Journal of Human-Computer Studies

Leveraging Conversational Agent to Assist New Hires during Onboarding
Praveen Chandar, Yasaman Khazaeni, Matthew Davis, Michael Muller, Marco Crasso, Q. Vera Liao, N. Sadat Shami, Werner Geyer
INTERACT 2017: International Conference on Human-Computer Interaction

Exploring AI Values and Ethics through Participatory Design Fictions
Michael Muller, Q. Vera Liao
HCIC 2017: Design Futures

Tailoring Conversational UX through the Lens of Dialogue Complexity
Q. Vera Liao, Biplav Srivastava, Pavan Kapanipathi
CHI 2017 Workshop on Conversational UX Design

2016

#Snowden: Understanding Biases Introduced by Behavioral Differences of Opinion Groups on Social Media
Q. Vera Liao, Wai-Tat Fu, Markus Strohmaier
CHI 2016: ACM Conference on Human Factors in Computing Systems

What Can You Do? Studying Social-Agent Orientation and Agent Proactive Interactions with an Agent for Employees
Q. Vera Liao, Matthew Davis, Werner Geyer, Michael Muller, N. Sadat Shami
DIS 2016: ACM Conference on Designing Interactive Systems

Improvising Harmony: Opportunities for Technologies to Support Crowd Orchestration
Q. Vera Liao, Victoria Bellotti, Michael Youngblood
GROUP 2016: ACM Conference on Supporting Group Work

2015

It Is All About Perspective: An Exploration of Mitigating Selective Exposure with Aspect Indicators
Q. Vera Liao, Wai-Tat Fu, Sri Shilpa Mamidi
CHI 2015: ACM Conference on Human Factors in Computing Systems

Emerging Media to Promote Physical Activity: Understanding Exposure, Attention and Behavior Change from Psychological and Computational Perspectives (book chapter)
Dolores Albarracin, Vera Liao, Jessica Yi, Chengxiang Zhai
Sedentary Behavior and Health: Concepts, Assessment and Intervention (Weimo Zhu, editor)

2014

Expert Voices in Echo Chambers: Effects of Source Expertise Indicators on Exposure to Diverse Opinions
Q. Vera Liao & Wai-Tat Fu
CHI 2014: ACM Conference on Human Factors in Computing Systems
* Best Paper Award, Honorable Mention

Can You Hear Me Now? Mitigating the Echo Chamber Effect by Source Position Indicators
Q. Vera Liao & Wai-Tat Fu
CSCW 2014: ACM Conference on Computer Supported Cooperative Work and Social Computing

Age Differences in Credibility Judgment of Online Health Information
Q. Vera Liao & Wai-Tat Fu
TOCHI journal: ACM Transactions on Computer-Human Interaction

2013

Beyond the Filter Bubble: Interactive Effects of Perceived Threat and Topic Involvement on Selective Exposure to Information
Q. Vera Liao & Wai-Tat Fu
CHI 2013: ACM Conference on Human Factors in Computing Systems

Aggregating Personal Health Messages for Scalable Comparative Effectiveness Research
Jason Cho, Q. Vera Liao, Yunliang Jiang, Bruce Schatz
BCB HIS 2013: ACM Conference on Bioinformatics, Computational Biology and Biomedical Informatics

2012

Understanding Experts’ and Novices’ Expertise Judgment of Twitter Users
Q. Vera Liao, Claudia Wagner, Peter Pirolli, Wai-Tat Fu
CHI 2012: ACM Conference on Human Factors in Computing Systems

An ACT-R Model of Credibility Judgment of Micro-blogging Web Pages
Q. Vera Liao, Peter Pirolli, Wai-Tat Fu
ICCM 2012: International Conference on Cognitive Modeling

Designing and Evaluating a Clustering System for Organizing and Integrating Patient Drug Outcomes in Personal Health Messages
Yunliang Jiang, Q. Vera Liao, Qian Cheng, Richard Berlin, Bruce Schatz
AMIA 2012: Annual Symposium of the American Medical Informatics Association

It’s Not in Their Tweets: Modeling Topical Expertise of Twitter Users
Claudia Wagner, Q. Vera Liao, Peter Pirolli, Les Nelson, Markus Strohmaier
SocialCom 2012: ASE/IEEE International Conference on Social Computing

Information and Attitude Diffusion in Networks
Wai-Tat Fu & Q. Vera Liao
SBP 2012: International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction

2011

Medtable™ An EMR-Based Tool to Support Collaborative Planning for Medication Use
Q. Vera Liao, Julian Chin, Stacey McKeever, Katie Kopren, Daniel Morrow, Katherine Davis, Elizabeth A.H. Wilson, Darren Kaiser, Michael Wolf, Thembi Conner-Garcia and James Graumlich
HFES 2011: Human Factors and Ergonomics Society Annual Meeting 2011

Crowdsourcing Quality Control of Online Information: a Quality-based Cascade Model
Wai-Tat Fu & Q. Vera Liao
SBP 2011: International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction

Invited Talks and Panels

"Questioning the AI: Towards Human Centered Explainable AI" Software Engineering for Machine Learning Application Symposium (SEMLA2021), June 2021

"Questioning the AI: Towards Human Centered Explainable AI" Academic Fringe Festival at TU Delft, June 2021

"Human Centered Explainable AI" Human-AI Interaction Summer School Guest Lecture at University of Toronto, June 2021

"Human Centered Explainable AI" Applied & Interactive Machine Learning Guest Lecture at UCLA, May, 2021

"Human Centered Explainable AI" Human-AI Interaction Guest Lecture at University of Texas at Austin, April 2021

"Questioning the AI: Towards Human Centered Explainable AI" J.P Morgan AI Research, March 2021

"Questioning the AI: Towards Human Centered Explainable AI" Tsinghua University Students Forum, March 2021

"Panel on Explainable AI in Industry" AAAI 2021 Workshop on Explainable Agency in Artificial Intelligence, February 2021

"Towards Human Centered Interpretable Machine Learning" Workshop on Interpretable Machine Learning& Causal Inference, Royal Statistical Society, December 2020

"Towards Conversational Interfaces for Information Search" Keynote at International Workshop on Chatbot Research, November 2020

"Questioning the AI: Towards Human Centered Explainable AI" Department of Computer Science and Engineering, New York University, November 2020

"Questioning the AI: Towards Human Centered Explainable AI" Spotify Research, June 2020

Editorial Board

Associate Editor for International Journal of Human-Computer Studies

Guest Editor for ACM Transactions on Interactive Intelligent Systems (TiiS) Special Issue on Conversational Agents for Healthcare and Wellbeing

Distinguished Reviewer for ACM Transactions on Interactive Intelligent Systems (TiiS)

Organizing Committee

ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW 2021)

ACM Conference on Interactive Media Experiences (IMX 2021)

CHI Workshop on Operationalizing Human-Centered Perspectives in Explainable AI (CHI 2021)

CHI Workshop on Conversational Agents for Health and Wellbeing (CHI 2020)

IUI Workshop on User-Aware Conversational Agent (USER2AGENT 2019, 2020)

ACM Conference on Intelligent User Interface (IUI 2019)

CSCW Panel on Human-Agent Communication: Connecting Research and Development in HCI and AI (CSCW 2019)

IEEE International Conference on Healthcare Informatics 2016 (ICHI 2016)

ICHI 2015 Workshop on Health Information Quality (HealthIQ 2015)

Senior Program Committee

ACM Conference on Intelligent User Interface– Senior Program Committee (IUI2022)

ACM Conference on Human Factors in Computing Systems – Associate Chair (CHI 2018, CHI 2019, CHI 2020, CHI 2021, CHI2022)

ACM Conference on Computer Supported Cooperative Work and Social Computing – Associate Chair (CSCW 2018)

Research Interns Mentored or Co-mentored

Jiao Sun (University of Sothern California) 2021

Charvi Rastogi (Carnegie Mellon University) 2021

Upol Ehsan (Georgia Institute of Technology) 2020

Soya Park (MIT) 2020

Shweta Narkar (Rensselaer Polytechnic Institute) 2020

Bhavya Ghai (Stony Brook University) 2019

Daricia Wilkinson (Clemson University) 2019

Ana Smith (Cornell University) 2019

Jonathan Dodge (Oregon State University) 2018

Ameneh Shamekhi (Northeastern, dissertation committee, now at Nuance) 2017