Research Statement (2022)

I am broadly interested in human-robot interaction (HRI), robotics, AI, augmented reality, and HCI. My overarching goal is to develop capable and understandable robotic systems that communicate efficiently and explain themselves so they fluently collaborate and interact with humans. As robots stand to benefit society, providing solutions to industry, our daily life, and the aging population, my goal is essential for robots to gain more acceptance and retain these benefits.

I reach my goal with a multidisciplinary approach:

  1. I draw insights from psychology and cognitive science to enable natural human-robot communication.
  2. I develop computational cognitive models and AI algorithms to structure robot explanations.
  3. I investigate natural language generation techniques and novel visualization methods for effective robot communication, such as concise referring forms and projector-based augmented reality.
  4. I conduct user studies using quantitative and qualitative methods in realistic real-world task settings.

I. Accomplished Work

During my Ph.D. study, I have independently proposed and developed expertise loosely related to the funded grants of my advisor. My current postdoctoral research further broadens my horizons.

A. Fluent Collaboration via Proactive Handover Release

I started my career working on enabling robots to hand objects, an essential daily activity for collaborative and assistive tasks. Seemingly straightforward, it is challenging for robots to achieve the human level of fluency, especially during object transfer. In my HRI paper [C4], I attached force sensors to a robot’s gripers and developed proactive release algorithms to analyze the haptic data to detect object grasp and pull. A user study revealed a reduction of handover time from 3.7(threshold approach) to 2.7s, achieving higher fluency and ease of taking.

B. Humans’ Preferences in Robot Explanations

During the handover study, participants wanted the robot to explain when it refused to release. In my ICRA handover workshop [W2] and ACM THRI [J3] journal papers, I examined the need and properties of preferred robot explanations with a failure scenario where a robot was asked to hand an almost-reachable cup. The robot conveyed the unreachability non-verbally: doing nothing, looking at it, looking with repeated pointing. Results showed that non-verbal behaviors must accompany verbal explanations and participants prefer them to be in situ and concise.

C. Explanation Generation Algorithms with Behavior Tree

Given these user preferences, I then proposed algorithms with behavior tree (BT) to generate the structure of explanations. BT encapsulates behaviors in a more interpretable tree structure compared to finite state machines with intertwined states, and scales well with its modular and reusable behavior nodes. However, the free-form BT does not have the human explanation structure. In my THRI journal paper [J1], I focused on organizing and representing complex tasks to make them readily explainable. I added structure to BT by framing it as a set of semantic sets and proposed hierarchical explanation generation algorithms. I also proposed algorithms to explain failures and correct them by extracting and inserting a self-contained, dependency-free subtree.

D. Real-World Collaborative Mobile Manipulation Testbed

To improve the generalization of my algorithms, I evaluated them with a multi-task, multi-step collaborative mobile manipulation kitting task, in which a Fetch robot builds kits of gearbox parts for human coworkers to assemble. The robot navigates in a confined environment and collects irregular parts from different stations, with machining involved. As a competition task, my team won second place and published the effort at ICRA 2020 [C5].

With both manipulation and navigation subtasks, the task is rich in challenges that can lead to realistic and robust robotic solutions. Compared to open-space navigation, it helps tackle autonomous near-obstacle navigation (e.g., table) required for proximity manipulation. It also helps develop robust detection algorithms for objects in manipulation. Finally, it advances obstacle avoidance algorithms in confined spaces, such as gearing machines. Besides the ICRA paper, I collaborated with Tufts University to replace the task sequence component with a cognitive architecture to prove the architecture’s adaptability, published at IROS [C6].

E. Explaining Past Behavior in Changing Environment via Replay

The situated and temporal nature of human-robot interaction creates unique challenges for robot explanation generation. Robots and humans physically move objects that will no longer be present at explanation time. In my THRI journal paper [J5], I proposed robot replays and ways to indicate the missing objects by physically replaying past behaviors, with verbal and projection indicators. Participants were asked to tell where a misrecognized object was picked, where a ground obstacle was to explain the detour, and where an object was misplaced. Results showed that physical replay with verbal and projection indicators best conveyed all three missing information.

F. Projector-Based Augmented Reality (AR) for Scalability

For the replay experiment, I found projection markers alone remarkably efficient in communicating navigation goals. I have architected and implemented this spatial AR technology on a Fetch robot with a projector and a pan-tilt unit overhead [C9]. It can also visualize manipulation intent [W3]. Compared to head-worn AR, projector-based AR does not require wearing special hardware, and the visualizations are visible in group and crowd contexts.

G. Cognitive Status-Informed Referential Choice Model

In my postdoc research, I have focused on concise referring forms [W4, C10, C11] for easy-to-follow language and explanations. Precisely, I computationally model and evaluate cognitive status-informed referring form selection. For data collection and human evaluation, I went beyond simple tabletop scenarios and designed a novel interaction task where an instructor teaches a learner a series of construction tasks in four separate quadrants, leading to repeated references to a mixture of present and non-present objects. The task design [W4] won the Best LBR Award, Third Prize at HRI 2022. The human evaluation [C10] won the Best Long-Paper Award at INLG 2022.

H. Reality Mismatch in Augmented Reality (AR) Robot Appendage

Besides projector-based AR, I have expanded to head-worn AR to add virtual body parts to physically limited robots, e.g., enabling armless robots to gesture. Yet, they might have negative impacts due to being virtual. With student researchers, we evaluated AR and physical arms for gesturing [W7C13]. Results showed that AR body parts have the same benefits as physical morphological components in task performance and subjective experience.



Zhao Han, Daniel Giger, Jordan Allspaw, Michael S Lee, Henny Admoni, and Holly A Yanco. “Building the foundation of robot explanation generation using behavior trees”. In: ACM Transactions on Human-Robot Interaction (THRI) 10.3 (2021). Top HRI Journal, pp. 1–31.


Zhao Han, Elizabeth Phillips, and Holly A Yanco. “The need for verbal robot explanations and how people would like a robot to explain itself”. In: ACM Transactions on Human-Robot Interaction (THRI) 10.4 (2021). Top HRI journal, pp. 1–42.


Zhao Han and Holly A Yanco. “Communicating Missing Causal Information to Explain a Robot’s Past Behavior”. In: ACM Transactions on Human-Robot Interaction (THRI) (2022). Top HRI journal; Just accepted.


Zhao Han and Holly Yanco. “The effects of proactive release behaviors during human-robot handovers”. In: 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI). Top HRI conference; 28% acceptance rate. 2019, pp. 440–448.


Zhao Han, Jordan Allspaw, Gregory LeMasurier, Jenna Parrillo, Daniel Giger, S Reza Ahmadzadeh, and Holly A Yanco. “Towards mobile multi-task manipulation in a confined and integrated environment with irregular objects”. In: 2020 IEEE International Conference on Robotics and Automation (ICRA). Top robotics conference; 1,4833,512 = 42% acceptance rate. 2020, pp. 11025–11031.


Tyler Frasca*, Zhao Han*, Jordan Allspaw, Holly Yanco, and Matthias Scheutz. “Going cognitive: A demonstration of the utility of task-general cognitive architectures for adaptive robotic task performance”. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). *Equal contribution; Top Robotics conference; 1,4092,996 = 47% acceptance rate. IEEE. 2020, pp. 8110–8116.


Zhao Han, Jenna Parrillo, Alexander Wilkinson, Holly A Yanco, and Tom Williams. “Projecting Robot Navigation Paths: Hardware and Software for Projected AR”. In: Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction. 2022, pp. 623–628.


Zhao Han, Polina Rygina, and Tom Williams. “Evaluating Referring Form Selection Models in Partially-Known Environments”. In: Proceedings of the 15th International Conference on Natural Language Generation. 2022.


Kevin Spevak*, Zhao Han*, Tom Williams, and Neil T Dantam. “Givenness Hierarchy Informed Optimal Document Planning for Situated Human-Robot Interaction”. In: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). *Equal contribution; Top Robotics conference; 1,7403,579 = 48% acceptance rate. 2022.


Zhao Han*, Yifei Zhu*, Albert Phan, Fernando Sandoval Garza, Amia Castro, and Tom Williams. “Crossing Reality: Comparing Physical and Virtual Robot Deixis”. In: Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction. *Equal contribution; Top HRI conference; 25.1% acceptance rate; Just accepted. 2023.


Zhao Han and Holly A Yanco. “Reasons People Want Explanations After Unrecoverable Pre-Handover Failures”. In: ICRA 2020 Workshop on Human-Robot Handovers. 2020.


Zhao Han, Alexander Wilkinson, Jenna Parrillo, Jordan Allspaw, and Holly A Yanco. “Projection Mapping Implementation: Enabling Direct Externalization of Perception Results and Action Intent to Improve Robot Explainability”. In: The Artificial Intelligence for Human-Robot Interaction Symposium at AAAI Fall Symposium Series 2020 (AI-HRI). 2020.


Zhao Han and Tom Williams. “A Task Design for Studying Referring Behaviors for Linguistic HRI”. In: Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction. 2022, pp. 783–786.


Zhao Han, Boyoung Kim, Holly A Yanco, and Tom Williams. “Causal Robot Communication Inspired by Observational Learning Insights”. In: 2022 AAAI Spring Symposium on Closing the Assessment Loop: Communicating Proficiency and Intent in Human-Robot Teaming. 2022.


Zhao Han, Albert Phan, Amia Castro, Fernando Sandoval Garza, and Tom Williams. “Towards an Understanding of Physical vs Virtual Robot Appendage Design”. In: The International Workshop on Virtual, Augmented, and Mixed-Reality for Human-Robot Interactions at HRI 2022. Full paper is under review. 2022.


Zhao Han. Course: Augmented Reality. 2022.


Mines Asian Community Alliance. 2022.


Mines Community Alliances. 2022.