Expertise

Natural Language Generation

XRDS, 2023

The Importance of Memory for Language-Capable Robots

Robots need to be able to communicate with people through natural language. But how should their memory systems be designed to facilitate this communication? Tags: Cognitive robotics, Cognitive science, Natural language generation Introduction As robots become more widely available to the public and play more prominent roles in people’s day-to-day routines, those robots will… Continue reading The Importance of Memory for Language-Capable Robots

Published

SemDial 2023

Toward Open-World Human-Robot Interaction: What Types of Gestures Are Used in Task-Based Open-World Referential Communication?

Gestures play a critical role in human-human and human-robot interaction. In task-based contexts, deictic gestures like pointing are particularly important for directing attention to task-relevant entities. While most work on task-based human-human and human-robot dialogue focuses on closed-world domains, recent research has begun to consider open-world tasks, where task-relevant objects may not be known to interactants a priori. In open-world tasks, we argue that… Continue reading Toward Open-World Human-Robot Interaction: What Types of Gestures Are Used in Task-Based Open-World Referential Communication?

Published

INLG 2023

Exploring the Naturalness of Cognitive Status Informed Referring Form Selection Models

Language-capable robots must be able to efficiently and naturally communicate about objects in the environment. A key part of communication is Referring Form Selection (RFS): the process of selecting a form like it, that, or the N to use when referring to an object. Recent cognitive status-informed computational RFS models have been evaluated in… Continue reading Exploring the Naturalness of Cognitive Status Informed Referring Form Selection Models

Published

CogSci 2023

Evaluating Cognitive Status-Informed Referring Form Selection for Human-Robot Interactions

Robots must be able to communicate naturally and efficiently, e.g., using concise referring forms like it, that, and the ⟨N’⟩. Recently researchers have started working on Referring Form Selection (RFS) machine learning algorithms but only evaluating them offline using traditional metrics like accuracy. In this work, we investigated how a cognitive status-informed RFS computational… Continue reading Evaluating Cognitive Status-Informed Referring Form Selection for Human-Robot Interactions

Published

IROS 2022

Givenness Hierarchy Informed Optimal Document Planning for Situated Human-Robot Interaction

Robots that use natural language in collaborative tasks must refer to objects in their environment. Recent work has shown the utility of the linguistic theory of the Givenness Hierarchy (GH) in generating appropriate referring forms. But before referring expression generation, collaborative robots must determine the content and structure of a sequence of utterances, a… Continue reading Givenness Hierarchy Informed Optimal Document Planning for Situated Human-Robot Interaction

Published

INLG 2022

Evaluating Referring Form Selection Models in Partially-Known Environments

This paper won the Best Long-Paper Award at INLG 2022: For autonomous agents such as robots to effectively communicate with humans, they must be able to refer to different entities in situated contexts. In service of this goal, researchers have recently attempted to model the selection of referring forms on the basis of cognitive… Continue reading Evaluating Referring Form Selection Models in Partially-Known Environments

Published

HRI 2022 LBR

A Task Design for Studying Referring Behaviors for Linguistic HRI

Two of four quadrants of the task environment to study referring behaviors for linguistic HRI. Two rule cards are placed on each table, with learners' on the left hand side and instructors' (dashed box) on the right hand side (solid box). Objects are intentionally placed at the intersections of a 3×3 grid to encourage use of different referring forms whose use varies according to distance. Instructors teach learners to construct buildings whose constituents blocks are distributed across the visible and non-visible quadrants.

In many domains, robots must be able to communicate to humans through natural language. One of the core capabilities needed for task-based natural language communication is the ability to refer to objects, people, and locations. Existing work on robot referring expression generation has focused nearly exclusively on generation of definite descriptions to visible objects.… Continue reading A Task Design for Studying Referring Behaviors for Linguistic HRI

Published