Member Login

Project

Project 2.3: Multi-modal HRC

Lead researcher

Project based at

Lead Partner Organisation

Multimodal Human Robot Collaboration

To improve the flexibility and adaptability of HRC, novel multimodal interfaces and gesture control devices are required. In these approaches conventional interaction techniques are augmented by vision and audio-based and haptic input/output technologies. Recent advances in speech and gesture interfaces are now widely used in domestic settings. Paired with vision-based and haptic interaction systems already in use in industrial settings, these technologies make possible new ways to control collaborative robots.

In this project you will develop a vocabulary of multimodal HRC interaction techniques to accomplish tasks such as specifying the intended path for the robotic tool to follow by manually drawing onto the work surface, instructing the robot through naturalistic speech and gesture, and directly sensing feedback on the state of a robotic task through visual, auditory, and haptic feedback. You will also study the needs for human-robot interaction in an authentic work context to make sure that the interaction techniques are appropriate for use in that setting.

Research activities

This research project may involve the following activities:

  • Context studies to establish candidate modalities for human robot interaction
  • Development of sensing and output capabilities for robotic equipment to detect, interpret and respond to human interaction across a range of modalities
  • Deployment and evaluation of the multimodal HRC interaction vocabulary in an applied setting

Outcomes

Upon completing this research, we expect to develop:

  • New tools for designing and prototyping multimodal human robot interactions
  • Development of sensing and output capabilities for robotic equipment to employ multimodal interactions
  • A framework to guide the design of multi-modal HRI including informing choices of modalities

Skills and experience

To be considered for this project, you must have a background in human-robot interaction, interaction design, human-computer interaction, or a related field particularly with skills in development of interactive prototypes. Ideally, you will have experience carrying out applied design-led research with end users and an ability to interpret qualitative data.

 


Associated Researchers

Markus Rittenbruch

Research Program Co-lead (Human-Robot Interaction program)
Queensland University of Technology
View Bio

Jared Donovan

Research Program Co-lead (Human-Robot Interaction program)
Queensland University of Technology
View Bio

Stine S. Johansen

Postdoctoral Research Fellow (Human-Robot-Interaction Program)
Queensland University of Technology
View Bio