Member Login

Project

Project 2.5: Interactive (and Collaborative) Robot Programming using Language

Project based at

Lead Partner Organisation

PhD scholarship available for this project. Submit an Expression Of Interest today

Interactive (and Collaborative) Robot Programming using Language

Programming robots to carry out desired tasks is difficult and time-consuming. This PhD project focuses on collaborative and instructional dialogue agents to help human operators program robot tasks.

In this collaborative scenario, a human operator converses with an AI agent to explain the steps that are to be performed, using high-level references and abstractions that make sense to the human, as opposed to simple verbal instructions corresponding to rudimentary robot movements. The AI agent must interpret the high-level instructions and translate these into a robot program based on advanced AI path  planning and goal planning capabilities. Importantly, the human operator’s perception of the shared environment may differ from that of the AI’s sensors, leading to scenarios where instructions may be  ambiguous, and their disambiguation requires a collaborative dialogue.

The PhD candidate will research and develop methods to situate a collaborative dialogue agent, focusing on the core research question of how to tackle ambiguities in instruction-to-code translation within a grounded robotics scenario. The project will lead to the development of a system – tools, algorithms and data – that allows a human operator to interactively program a robot using language. The research will lead to a better understanding of the design space in human-robot collaboration and will generate an effective example of collaborative intelligence based on both human and robot capabilities. The PhD candidate will be expected to publish this research in relevant high-quality peer-reviewed journals and conferences.

The project will be part of the Data61’s collaborative intelligence (CINTEL) Future Science Platform (FSP) that will develop the science that enables human intelligence and technology to work together across multiple domains, driving sustainable productivity growth and improving both the quantity and quality of jobs for human workers.


Associated Researchers

Pavan Sikka

Associate Investigator
CSIRO CINTEL
View Bio

Cécile Paris

Associate Investigator
CSIRO CINTEL
View Bio

Stephen Wan

Associate Investigator
CSIRO CINTEL
View Bio

Stine S. Johansen

Postdoctoral Research Fellow (Human-Robot-Interaction Program)
Queensland University of Technology
View Bio

Jared Donovan

Research Program Co-lead (Human-Robot Interaction program)
Queensland University of Technology
View Bio

Markus Rittenbruch

Research Program Co-lead (Human-Robot Interaction program)
Queensland University of Technology
View Bio