THRI, 2021 — ACM Transactions on Human-Robot Interaction (THRI), 10(3), July, 2021

Building The Foundation of Robot Explanation Generation Using Behavior Trees

Zhao Han, Daniel Giger, Jordan Allspaw, Michael S. Lee (CMU), Henny Admoni (CMU), and Holly A. Yanco

, , , , ,
Screw placing, a representative subtask of the gearbox kitting task, modeled as a Behavior Tree. The leftmost node’s ancestors are not shown. For more detail, refer to Section 5.
Behavior tree, robot task representation
News
  • Mar 31, 2021

    I just gave an invited talk at Georgia Tech! Here are my slides. Feel free to email me about your questions.

  • Nov 19, 2020

    Our journal paper on robot explanation algorithms using Behavior Trees is accepted to the top HRI journal ACM Transactions on Human-Robot Interaction (THRI)!

Abstract

As autonomous robots continue to be deployed near people, robots need to be able to explain their actions.

In this paper, we focus on organizing and representing complex tasks in a way that makes them readily explainable. Many actions consist of sub-actions, each of which may have several sub-actions of their own, and the robot must be able to represent these complex actions before it can explain them.

To generate explanations for robot behavior, we propose using Behavior Trees (BTs), which are a powerful and rich tool for robot task specification and execution. However, for BTs to be used for robot explanations, their free-form, static structure must be adapted.

In this work, we add structure to previously free-form BTs by framing them as a set of semantic sets {goal, subgoals, steps, actions} and subsequently build explanation generation algorithms that answer questions seeking causal information about robot behavior. We make BTs less static with an algorithm that inserts a subgoal that satisfies all dependencies.

We evaluate our BTs for robot explanation generation in two domains: a kitting task to assemble a gearbox, and a taxi simulation. Code for the behavior trees (in XML) and all the algorithms is available at github.com/uml-robotics/robot-explanation-BTs.

Figures

Behavior Trees

Classical Behavior Tree BT Formulation Nodes and Return Statuses
Classical Behavior Tree (BT) Formulation – Nodes and Return Statuses.
Notation of and changes to Behavior Trees as used in this paper
Notation of Behavior Trees (BTs) used in this paper.

Modelling a Mobile Manipulation Kitting Task in Behavior Trees

The detail of this task was published in ICRA 2019. See the paper →

The FetchIt arena where the gearbox kitting task is carried out by a Fetch robot. Rendered in Gazebo, the main goal is to place a specified set of parts into the correct sections of the caddy, then to transport the caddy to the inspection table.
The FetchIt arena where the gearbox kitting task is carried out by a Fetch robot. Rendered in Gazebo, the main goal is to place a specified set of parts into the correct sections of the caddy, then to transport the caddy to the inspection table.
Parts to be collected: (a) Large gear (b) Gearbox top (c) Gearbox bottom (d) Screw (e) Small gear. Note that the large gear (a) is meant to be machined to have threads; the large gear must be inserted into a machine for this process to occur.
Parts to be collected: (a) Large gear (b) Gearbox top (c) Gearbox bottom (d) Screw (e) Small gear. Note that the large gear (a) is meant to be machined to have threads; the large gear must be inserted into a machine for this process to occur.
Assembled gearbox using the required mechanical parts that the robot placed into the caddy and delivered to the inspection table.
Assembled gearbox using the required mechanical parts that the robot placed into the caddy and delivered to the inspection table.
The top level of the gearbox kitting task represented as a sequence of subtrees in a Behavior Tree. For readability, six tasks—go {pick | place} {small gear | gearbox top | gearbox bottom}—are represented by “. . . ”. Note that the root node on the top is merely a pointer to the real root node in the middle, which is why depth 0 is at the sequence node.
The top level of the gearbox kitting task represented as a sequence of subtrees in a Behavior Tree. For readability, six tasks—go {pick | place} {small gear | gearbox top | gearbox bottom}—are represented by “. . . ”. Note that the root node on the top is merely a pointer to the real root node in the middle, which is why depth 0 is at the sequence node.

Go pick screw

The representative screw picking subtask modeled in Behavior Trees. The leftmost dot indicates that the fallback node has a parent, but the subtree parent node and other ancestor nodes are hidden here, as they are shown in the previous figure.
The representative screw picking subtask modeled in Behavior Trees. The leftmost dot indicates that the fallback node has a parent, but the subtree parent node and other ancestor nodes are hidden here, as they are shown in the previous figure.
 Simplified, semantic sets for the screw picking subtask.
Simplified, semantic sets for the screw picking subtask.

Go place screw

Screw placing, another representative subtask of the gearbox kitting task, modeled as a Behavior Tree. Similar to the previous figure, the leftmost node’s ancestors are not shown. For more detail, refer to Section 5.
Screw placing, another representative subtask of the gearbox kitting task, modeled as a Behavior Tree. Similar to the previous figure, the leftmost node’s ancestors are not shown. For more detail, refer to Section 5.
Simplified, semantic sets for the screw placing subtask.
Simplified, semantic sets for the screw placing subtask.

Large gear insertion

The large gear insertion subtask modeled in Behavior Trees.
The large gear insertion subtask modeled in Behavior Trees.
The framed BT for the large gear insertion subtask.
The framed BT for the large gear insertion subtask.

Dynamic Behavior Insertion as Subgoal

The behavior tree for the kitting task after answering “Can you insert large gear?” Most subgoals in Figure 4 are collapsed for readability. The empty sequence node at depth 1 is dynamically inserted to ensure all input ports are satisfied by output ports. See Section 7.1.2 for more.
The behavior tree for the kitting task after answering “Can you insert large gear?” Most subgoals in Figure 4 are collapsed for readability. The empty sequence node at depth 1 is dynamically inserted to ensure all input ports are satisfied by output ports. See Section 7.1.2 for more.

Robot Explanation Algorithms with Behavior Trees

Questions to Be Answered during Hierarchical Robot Explanation Supported by Behavior Trees
Questions to Be Answered during Hierarchical Robot Explanation Supported by Behavior Trees
Robot Explanation Algorithm 8 Find Self contained Behavior Node for Supporting Dynamic Behavior Insertion as Subgoal
Robot Explanation Algorithm 8 – Find Self-contained Behavior Node for Supporting Dynamic Behavior Insertion as Subgoal

Robot Failure Explanation Algorithm with Behavior Trees

Robot Failure Explanation Algorithm with Behavior Trees
Robot Failure Explanation Algorithm with Behavior Trees

Taxi domain

While this work was originally designed for the context of the kitting task, we show here the potential of this work in a non-manipulation domain.

Taxi domain environment
An environment of the Taxi domain, in which the agent delivers a passenger to the destination.
Simplified behavior tree representation of a taxi reinforcement agent to go pick up a passenger annotated
The behavior tree representation of an optimal policy in the non-manipulation Taxi domain. Given that it is less complex than a kitting subtask, there is no need for simplification.
An reinforcement agent’s optimal policy vs. a humans’s expected policy for the taxi domain
An reinforcement agent’s optimal policy vs. a humans’s expected policy for the taxi domain environment. Divergences in actions (some leaf nodes) between the two policies are colored in dark blue and their immediate parents (some sequence nodes) are colored in blue-gray.

Slides

Robot Explanation with BT Invited Talk at Georgia Tech
Slides for invited talk at Georgia Tech

Leave a comment

Your email address will not be published.