ICRA 2020 — 2020 IEEE International Conference on Robotics and Automation (ICRA)

Towards Mobile Multi-Task Manipulation in a Confined and Integrated Environment with Irregular Objects

Zhao Han, Jordan Allspaw, Gregory LeMasurier, Jenna Parrillo, Daniel Giger, S. Reza Ahmadzadeh and Holly A. Yanco

42% Acceptance rate
Fetch inserted a large gear 16x9 1
  • Dec 1, 2019

    Our paper is accepted to the 2020 IEEE International Conference on Robotics and Automation (ICRA), the top conference in robotics! This is our publication output from the FetchIt competition.


The FetchIt! Mobile Manipulation Challenge, held at the IEEE International Conference on Robots and Automation (ICRA) in May 2019, offered an environment with complex and integrated task sets, irregular objects, confined space, and machining, introducing new challenges in the mobile manipulation domain.

Here we describe our efforts to address these challenges by demonstrating the assembly of a kit of mechanical parts in a caddy.

In addition to implementation details, we examine the issues in this task set extensively, and we discuss our software architecture in the hope of providing a base for other researchers.

To evaluate performance and consistency, we conducted 20 full runs, then examined failure cases with possible solutions. We conclude by identifying future research directions to address the open challenges.


successful kit delivery
Snapshots of a successful kit delivery in our test course from the accompanying video

Fetch inserted a large gear
The competition arena, where the Fetch robot autonomously drives
close to the gear station, grasps a large gear, inserts it into the lathe chuck, and retreats its arm to prepare for navigation.
fetchit arena in simulation
The FetchIt! Mobile Manipulation Challenge environment in
simulation. The main goal is to place a specified set of parts into the correct sections of the caddy, then to transport the caddy to the inspection table.
parts in the FetchIt manipulation
Parts to be collected: (a) Large gear (b) Gearbox top (c) Gearbox
bottom (d) Screw (e) Small gear.
detection outputs
Images from RViz showing three detection outputs. Left: In-hand
large gear obstacle, two waypoints and extracted chuck pose. Middle: Poses
of the three caddy compartments. Right: poses of four gearbox bottom parts.
perception challenges
A set of perception challenges. See Section VII-A for more details.
navigation challenges
A set of navigation challenges. See section VII-C for more details.
diagram fetch mobile manipulation
The Unified Modeling Language (UML) diagram of our architec-
ture. Yellow, blue, cyan, gray and red blocks indicate the main, manipulation,
perception, navigation and common code components respectively.


Video Presentation


Leave a comment

Your email address will not be published.