HRI 2025
Expertise Augmented Reality (AR)
Projection Mapping, Projected AR
Projection mapping, or project AR, in robotics is an augmented reality (AR) or extended reality (XR) method that directly projects a robot’s intent onto its physical operating environment (e.g., highlighting an object to grasp or navigation path to follow on the floor).
Compared to virtual reality (VR), it frees humans from wearing extra headsets, making observation of projections scalable beyond a single person (i.e., to a crowd).
Other benefits include improved visibility (e.g., no need to be close to a monitor to see), drastic decrease in mental inference of robot intent (unlike the vague pointing behaviors through arm movement or eye gaze), and no error-prone mapping from the media itself to the operating environment.
As opposed to the non-verbal methods and verbal explanations found among humans, projection mapping is a method that allows for direct and accurate externalization. This projection completely removes the need for mental inference as the perceived objects or the objects to be manipulated are directly externalized. The directness is similar to the use of a display screen, but projection is more salient because bystanders or robot coworkers can also see the projection from farther away, instead of requiring people to stop their work in order to walk to a monitor to examine the robot’s states. Direct projection onto the operating environment also eliminates mental mapping from another media such as a monitor, which can cause misjudgment and lead to undesired consequences.
HRI 2022
Projecting Robot Navigation Paths: Hardware and Software for Projected AR
For mobile robots, mobile manipulators, and autonomous vehicles to safely navigate around populous places such as streets and warehouses, human observers must be able to understand their navigation intent. One way to enable such understanding is by visualizing this intent through projections onto the surrounding environment. But despite the demonstrated effectiveness of such projections,… Continue reading Projecting Robot Navigation Paths: Hardware and Software for Projected AR
Paladyn, 2021
Design Guidelines for Human-Robot Interaction with Assistive Robot Manipulation Systems
The design of user interfaces for assistive robot systems can be improved through the use of a set of design guidelines presented in this paper. As an example use, the paper presents two different user interface designs for an assistive manipulation robot system. We explore the design considerations from these two contrasting user interfaces.… Continue reading Design Guidelines for Human-Robot Interaction with Assistive Robot Manipulation Systems
AI-HRI 2020
Projection Mapping Implementation: Enabling Direct Externalization of Perception Results and Action Intent to Improve Robot Explainability
Existing research on non-verbal cues, e.g., eye gaze or arm movement, may not accurately present a robot’s internal states such as perception results and action intent. Projecting the states directly onto a robot’s operating environment has the advantages of being direct, accurate, and more salient, eliminating mental inference about the robot’s intention. However, there… Continue reading Projection Mapping Implementation: Enabling Direct Externalization of Perception Results and Action Intent to Improve Robot Explainability