Member Login

Author: Merryn Ballantyne

6 Reasons Why We Need a Prototyping Toolkit for Designing Human-Robot Collaboration

Posted on by Merryn Ballantyne

Written by Postdoctoral Research Fellow, Dr Stine S Johansen and PhD Researcher, James Dwyer

In this short article, we will share 6 benefits of having a prototyping toolkit for designing human-robot collaboration (HRC). We will lift the curtain on our planned activities to work towards this in Program 2 of the Australian Cobotics Centre.

What type of human-robot collaboration are we talking about?

The Australian Cobotics Centre focuses on cobots in manufacturing settings. In these settings, robots are most often big and locked away in cages for safety reasons. They are useful for highly defined and repeatable tasks that require strength. In contrast, cobots are typically smaller and allow for people to safely carry out a task by handing over items to the robot or even by physically handling the robot.

Cobots address an increasing need for more adaptable robotic systems for customised and bespoke products. These types of products still require people in the manufacturing line to accommodate changes from product to product.

So, what could a prototyping toolkit look like?

Imagine a toolbox with screwdrivers, a hammer, cutters, etc. Similar to that, we already have tools in our design toolbox that work at a generic level or are appropriated to suit particular problems. But a toolkit for prototyping human-robot collaboration is still left for us to investigate. In Program 2, James Dwyer (PhD student) will contribute to our knowledge about how different prototyping tools can facilitate design processes of HRC. The goal is to develop a practical and affordable toolkit that can be used to enable designers, engineers, and end-users to work together towards human-robot collaboration in manufacturing settings and beyond.

What are the benefits of having a prototyping toolkit?

Knowing how a cobot can fit into an existing or new manufacturing setting requires substantial research. What if we had a way to make that process easier and more efficient for designers and clients as well as more accommodating for the final end-users of the cobot? This is the broad aim of a HRC prototyping toolkit. Here are 6 concrete benefits that we aim to support through our work in Program 2.

1) Accessible end-user engagement

Manufacturers often lack the expertise to define how a cobot could be used. They are, however, experts in their respective domain. Domain knowledge is not always something that can be documented in written reports. It is also the tacit knowledge that workers build through years of experience. A prototyping toolkit can enable that knowledge to play a role very early in the design and development process by lowering the currently high technical barriers to understand how a robot works. In Program 2, we rely on principles from participatory design which is a design practice to produce tangible outcomes together with end-users.

2) Cost and time efficiency

Facilitating a cobot integration project can require substantial costs and time which makes it non-viable for some manufacturers. The hardware investments require committing to a particular setup, but there are risks associated with such investments if feasibility of the concept has not been investigated early on. Therefore, it will be beneficial to have prototyping tools to conduct such investigations without the necessity of actual hardware. Prototyping tools can furthermore allow for quick and cheap iterations. Subsequently, there is a need for tools that facilitate the transition from early concepts to implementation and testing.

3) Flexibility

Given the opportunity for cobots to assist in manufacturing of customised products, there is a high need for flexible solutions. Crucial to realising flexibility is the establishment of design processes that bridge the gap between early stage conceptual development and technical integration. For cobots to effectively contribute to customised production, they must follow a rich understanding of work practices, production methods, and customisation requirements entailed in the manufacturing. This understanding can be developed through iterative design and a holistic approach, covering all aspects from conceptualisation, prototyping, and implementation. This will ensure that the cobots are versatile, adaptable, and able to meet changing production needs.

4) Risk mitigation

Even though cobots are generally equipped with safety measures such as a safe stop button and sensors to detect and stop collisions with people, it is still possible to get hurt by a faulty cobot that has not been adapted to its environment. Prototyping tools allow us to mitigate this risk in two ways. First, it is possible to create virtual models of the environment and cobot, meaning that we can simulate tasks and clarify potential safety risks we might not otherwise have detected purely from prior experience and safety standards. This allows us to develop safety measures long before anyone gets hurt. Second, while engaging end-users in the design process has many benefits, people with non-technical backgrounds are not necessarily comfortable interacting with a robot – especially an unfinished robot solution. Therefore, prototyping tools can support our engagement with end-users by removing the potential fear of getting hurt.

5) Enhanced creativity

As design researchers, we often engage in generative ideation activities to address research questions. Prototypes enable us to see facets of an idea that were not previously obvious. This is sometimes referred to as ‘filtering’ (for further reading on this topic, see our list of references). It’s like putting on special glasses that highlight the specific qualities we want to explore further while still capturing the essence of the entire concept. In order to use prototypes as filters, it is necessary to have a holistic understanding of the context within which the cobot will operate and how that context can change with the introduction of the cobot. A prototyping toolkit can help give us different lenses to explore facets of the context in early prototypes, thereby becoming a creative extension for designers. This could include prototyping tools such as facilitating Wizard-of-Oz methods, video prototyping, or virtual simulations.

6) Facilitating internal communication

Prototyping is an activity that allows us to both internalise and externalise ideas. In other words, prototypes enable us to internally reflect on what works and what does not work as well as communicate ideas to team members, clients, or anyone interacting with them. Prototypes have always had that role in design research, but with the technical barriers to quick prototyping for human-robot collaboration, there is a need to identify new ways to facilitate this role of prototypes.

We look forward to sharing our progress throughout the next few years. Please reach to us for further discussion, questions, or other inquiries.

Further reading:

Lim, Y. K., Stolterman, E., & Tenenberg, J. (2008). The anatomy of prototypes: Prototypes as filters, prototypes as manifestations of design ideas. ACM Transactions on Computer-Human Interaction (TOCHI), 15(2), 1-27.

Wensveen, S., & Matthews, B. (2014). Prototypes and prototyping in design research. In The routledge companion to design research (pp. 262-276). Routledge.

William Odom, Ron Wakkary, Youn-kyung Lim, Audrey Desjardins, Bart Hengeveld, and Richard Banks. 2016. From Research Prototype to Research Product. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ’16). Association for Computing Machinery, New York, NY, USA, 2549–2561. https://doi.org/10.1145/2858036.2858447

Gopika Ajaykumar. 2023. Supporting End-Users in Programming Collaborative Robots. In Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’23). Association for Computing Machinery, New York, NY, USA, 736–738. https://doi.org/10.1145/3568294.3579969

Two joint CSIRO & Australian Cobotics Centre scholarships now available!

Posted on by Merryn Ballantyne

We are currently recruiting for two PhD Scholarships funded by CSIRO’s Data61‘s Collaborative Intelligence (CINTEL) Future Science Platform.

Both scholarships are based at QUT in Brisbane and students will also be part of the Australian Cobotics Centre. They will receive a scholarship of $36,161 per annum (indexed annually) with additional funding for travel and collaboration available.

*** Interactive (and Collaborative) Robot Programming using Language ***

The first scholarship is working with our Human Robot Interaction team, supervised by Jared DonovanDonovan and Markus Rittenbruch and CSIRO’s Data61‘s Dr Andrew Reeson.

The Project, “Interactive (and Collaborative) Robot Programming using Language” focuses on collaborative and instructional dialogue agents to help human operators program robot tasks. The PhD candidate will research and develop methods to situate a collaborative dialogue agent, focusing on the core research question of how to tackle ambiguities in instruction-to-code translation within a grounded robotics scenario. The project will lead to the development of a system – tools, algorithms and data – that allows a human operator to interactively program a robot using language.

Find our more: Project 2.5: Interactive (and Collaborative) Robot Programming using Language

*** Collaborative robot adoption across industries ***

The second scholarship is based with our Designing Socio-Technical Robotic System program, supervised by A/Prof Glenda Caldwell & Dr Matthias Guertler and working with researchers from the CSIRO’s Data61 team, Dr Cécile Paris, Dr Stephen Wan and Dr Pavan Sikka.

The Project, “Collaborative robot adoption across industries” will examine other industries across Australia to identify to what extent they have adopted cobots. The project would identify:

  • The drivers of adoption, along with barriers to it, and how any such barriers were overcome.
  • To what extent have these various industries adapted their processes to accommodate cobot technology, and what was required for them to do this?
  • What (re)-training was required for workers? How specific or generalisable are the required skills?

Find our more: Project 3.7:  Collaborative robot adoption across industries

If you are interested in these scholarships, submit an expression of interest here:

PhD Expressions of Interest

 

 

Centre-wide mid-year meeting

Posted on by Merryn Ballantyne

On 18th July, we held our mid-year meeting. This year each university hosted an in-person event and we joined together via zoom.

It was fantastic to reflect on our achievements over the past 6months and look forward to what we want to get done by the end of the year.

So far, in 2023, our group has:

and much, much more.

For the QUT group, we combined the meeting with a tour of Stryker‘s R&D lab in Herston and finished off the afternoon with a game of mini golf!

Thank you all for coming along! We are now looking forward to the end of year symposium where we can all meet in person and share our progress!

Australian Manufacturing Week 2023

Posted on by Merryn Ballantyne
PhD researcher Jagannatha Pyaraka and research program co-lead, Dr Michelle Dunn

Australian Manufacturing Week was held in Melbourne from 9-12 May 2023.

It provided an incredible opportunity for me (PhD student, Jagannatha Pyaraka) to gain firsthand experience and insights into the world of cobotics in the manufacturing industry. Special thanks to Cornelius van Niekerk, Business Development Manager from Weld Australia for giving us the opportunity to share the stand. Representing the Australian Cobotics Centre (ACC) Jagannatha Pyaraka, Mats Isaksson, Michelle Dunn, Christopher McCarthy, and Anushani Bibile, showcased a demonstration featuring our new UFactory Xarm6 cobot.

Our demo aimed to demonstrate the potential of collaborative robots (cobots) in enhancing manufacturing processes. We highlighted how cobots, designed to work alongside humans, can improve productivity while ensuring safety. By employing the Media-pipe technology in conjunction with D435 RealSense camera, the cobots capability to accurately capture and follow the tip of presenter’s palm was
demonstrated.

Throughout the event, we had the opportunity to interact with a diverse audience. People were intrigued by the capabilities of cobots and had a keen interest in the ACC’s work. We engaged in discussions on various aspects, including safety, efficiency, integration, and the return on investments associated with implementing cobots in manufacturing processes. These conversations provided valuable insights into the practical challenges and applications of this emerging technology.

One particular highlight was the interaction with other exhibitors, who showcased their own advancements and trends in the industry. The most widely shown cobot application was welding. This exchange of ideas allowed us to broaden our perspectives and gain a better understanding of the future developments in cobotics.

The experience at Australian Manufacturing Week 2023 has not only provided exposure to practical applications but has also deepened my understanding of the challenges and opportunities that arise when implementing cobots. Witnessing the enthusiasm and interest of the audience reaffirmed the importance of the work carried out by the ACC and boosted my passion for further research in this field.

Looking ahead, it is evident that cobotics will continue to revolutionize the manufacturing industry. As we strive to improve safety, efficiency, and productivity, the ACC will play a vital role in driving innovation and shaping the future of cobotics. I am excited to be able pursue my research in this field and look forward to witnessing the continued growth and impact of collaborative robots in manufacturing
processes.

Overall, Australian Manufacturing Week 2023 was a remarkable experience that not only allowed us to showcase our demo but also provided valuable insights, connections, and inspiration for the future of cobotics and the ACC.

Weld Australia CEO Geoff Crittenden
PhD Researcher, Jagannatha Pyaraka & Postdoc, Dr Anushani Bibile
Swinburne University Lead, A/Prof Mats Isaksson

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

ACM/IEEE International Conference on Human-Robot Interaction March 13-16, 2023 Stockholm, SE

Posted on by Merryn Ballantyne

Postdoctoral Research Fellow, Dr Stine Johansen was at the IEEE Human Robot Interaction Conference this week where she presented her paper, Illustrating Robot Motions.

Stine created a video presentation of the paper which gives an overview of the survey and examples of findings relating to how robot movements are illustrated in the ACM/IEEE Conference on Human-Robot Interaction proceedings from 2016 to 2022.

Download the paper and Stine’s video here: Illustrating Robot Movements | Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction

We congratulate Stine, Jared and Markus on their paper acceptance especially given the conference’s average acceptance rate of 24% (last 5 years 24%).

Dr Stine Johansen
Associate Professor Jared Donovan
Associate Professor Markus Rittenbruch

ARTICLE: Try-a-Trade, Gladstone

Posted on by Merryn Ballantyne

By Melinda Laundon, Postdoctoral Research Fellow, and Jacqueline Greentree, PhD Researcher, Human-Robot Workforce Program, Australian Cobotics Centre

Almost 200 Grade 9 and 10 girls from Gladstone and surrounds gathered at CQUniversity for Try-a-Trade on 15th March. This event brought together manufacturing, aerospace, energy, mining, engineering and construction industry businesses and stakeholders from government and education to encourage female high school students to learn about a range of careers in STEM and try out some practical activities that they might encounter in STEM jobs.

The event was organised by the Gladstone Engineering Alliance in partnership with the National Association of Women in Construction and the Queensland Government‘s Department of Employment, Small Business and Training and Gladstone Manufacturing Hub.

It is important to engage high school students early to consider careers in manufacturing. The lower numbers of women in manufacturing make it particularly important to provide female students with opportunities to consider and experience diverse manufacturing career options. In Queensland, 29% of the current manufacturing workforce are women. While the proportion of women employed in manufacturing has grown dramatically over the past decade, most are in clerical or administrative roles. Only 11% of women in manufacturing are technicians and trade workers[1].

Australian Cobotics Centre industry partner Weld Australia hosted a stall at Try-a-Trade with two Soldamatic welding simulators. This popular activity allows people to experience welding in a safe environment. It also provided a fun competition between girls to compare their welding accuracy. Weld Australia’s Regional Training Coordinator, Adam Coorey said:

“To address the skills shortage, we need to give a greater range of access to the full available workforce. By utilising augmented reality technology, students who would normally shy away from the heat and sparks of a welding bay can try welding in a safe environment.  This accessible technology gives students the opportunity to experience a career that they may thrive in”.

At the Cobotics Centre stall, we discussed the impact of cobots and other advanced technology on future work. We also asked students to think about the skills and attributes that would be required in the future to work with a cobot as a team member. They came up with many creative insights, including:

  • Problem solving skills
  • Patience
  • Understanding human interactions
  • Coding
  • Good communication
  • Independence
  • Curiosity
  • Designing
  • Digital technology education
  • To be able to build
  • Understanding of mechanics

[1] Queensland Department of Regional Development, Manufacturing and Water (2023) Women in Manufacturing Strategy.

Meet our E.P.I.C. Researcher, Dr Melinda Laundon

Posted on by Merryn Ballantyne

The Australian Cobotics Centre has some incredibly E.P.I.C. researchers. Each month we will be profiling a different researcher. Dr Melinda Laundon is a Postdoctoral Research Fellow with our Human Robot Workforce program and is based at QUT.

We sat down with Melinda recently to find out more about why she does what she does.

  • Tell us a bit about yourself and your research with the Centre?

I’m part of the Human-Robot Workforce research program. We have a broad scope to research the changing skills, capabilities, and composition of the Australian manufacturing workforce, including the influence of cobots on jobs, workers and organisations. My research is investigating attraction and retention of manufacturing workers, from the perspective of sector stakeholders, managers and the workers themselves. I hope it will help to understand how advanced technologies can change the way we attract people to careers in manufacturing, and help to address some of the crucial skills shortage issues in Australian manufacturing.

  • Why did you decide to be a part of the Australian Cobotics Centre?

I jumped at the chance to work with great researchers from many different disciplines, as well as the opportunity to engage with industry partners on issues that are immediately important and relevant. I could also see strong parallels between the Cobotics Centre’s commitment to sustainable growth and quality jobs in manufacturing, and QUT’s Centre for Decent Work & Industry, where I co-lead a research stream on sustainable transitions between education and work.

  • What project are you most proud of throughout your career and why?

I’m proud of my work with the Australian Research Council and universities to help academics and PhD students to plan for and achieve an impact on society from their research.  I’m also proud of my roles as a mentor and assessor for Higher Education Academy (HEA) Fellowship schemes at QUT and other universities. HEA Fellowship emphasises the value of reflection, professional development to improve student learning, and sharing good practice with colleagues – all of which apply to research as well as university teaching.

  • What do you hope the long-term impact of your work will be?

I came to academia after a policy career in the Australian Public Service, including stints with the Australian Communications and Media Authority and the Australian Research Council. My PhD in management examined employees’ fairness perceptions at work, especially in relation to reward and recognition. I started my academic career with the intention of not only doing good scholarly research but also having a practical impact on organisations and public policy. I hope that my research helps to inform policy and practice by contributing deeper understandings of workers’ perceptions and needs.

  • Aside from your research, what topic could you give an hour-long presentation on with little to no preparation?

This is a tough one! I might have to break it up into a few smaller lectures on different topics including Scottish and Australian crime fiction, Australian slow fashion labels, and music played on Triple J in the late ‘90s and early 2000s.

 

QUT’s Vacation Research Experience Scheme (VRES)

Posted on by Merryn Ballantyne

The Centre has been lucky enough to have two undergraduate students from QUT, Sabrin Daniel and Melanie Lay, working on projects with us over the last couple of months as part of QUT’s Vacation Research Experience Scheme (VRES).

VRES allows students the opportunity to participate in a research project for approximately 6-10 weeks. This opportunity is particularly valuable if they’re interested in pursuing a research degree! 

We sat down with Sabrin and asked her a little more about her experience:

 

Why did you decide to participate in QUT’s VRES?

Throughout my degree, I’ve had a particular interest in Social and Organisational Psychology. As a result, I thought this was the best way to gain insight into organisational issues such as teamwork and productivity, as well as gaining experience in conducting a literature review. Moreover, it is clear that post COVID-19, advancements in technology is changing the way businesses meet client and consumer expectations, therefore I was curious to understand how this would impact me as a HR professional. Specifically, how HR will need to adapt its policies and processes to support companies in this transition.

What did you learn that you think will be most useful in your future work or studies? 

The process in identifying patterns within the literature and being able to make appropriate conclusions about their impact to the topic at hand was a great skill that I learnt. I believe that this will help me in my future studies. As well, this is a skill that helps to identify what exists in the literature and what the gaps are to be able to implement appropriate policies and HR practices. Specifically in my research, a common consensus is that human-robot collaboration (HRC) requires commitment, capability and fulfilment and issues such as communication, trust and interdependency can hinder HRC. As a result, reading the literature allowed me to understand how HR professionals can address these issues which will be useful for me in the future.

What did you enjoy most about your project? 

I thoroughly enjoyed attending the Australian Cobotics Centre (ACC) launch, as I was able to hear industry partners speak about how cobots present opportunities for their industries, along with the anticipated challenges. Additionally, I was fortunate enough to be able to do a site visit at Urban Arts Precinct (UAP) with my mentors, Melinda Laundon and Greg Hearn. Both of these events helped in providing context for me to understand my research. Finally, I was fortunate enough to have mentors that provided me with resources to understand an efficient and systematic approach to conducting a literature. I believe their feedback and guidance throughout the process made this VRES project thoroughly enjoyable.

What would be the next steps in your project if it was to continue? 

The next step in the process is to identify and focus research questions based on the findings of my research.

Meet the VRES Students

Sabrin Daniel worked with the Human-Robot Workforce research program.

Her research focuses on Work crews and cobots: the future of work. Specifically, a literature review on, trends in cobotic adoption, how team dynamics will be impacted and the possible advantages and challenges to cobotic adoption trends in.

She is currently an undergraduate at QUT studying a dual degree; a Bachelor of Behavioural Science (Psychology) and a Bachelor of Business (Human Resource Management Major).

 Melanie Lay worked with the Designing Socio-Technical Robotic Systems Program.

Her research aims to create digital twins to support human-robot collaboration. The outcome of this will include modelled 3D assets which can be used to digitally simulate a real-world manufacturing environment.

She is currently an undergraduate student at QUT, studying a double degree of Bachelor of Design (Architecture)/Bachelor of Engineering (Mechatronics).

 

ARTICLE: Human-Robot Collaboration Challenges and Opportunities in Australia

Posted on by Merryn Ballantyne

At the 2022 annual Australian conference for the Computer-Human Interaction Special Interest Group, OzCHI, 20 researchers from leading Australian robotics research environments met up for the workshop “Empowering People in Human-Robot Collaboration”. This workshop was co-organised by members from the Australian Cobotics Centre and the Collaborative Intelligence Future Science Platform (CSIRO). In this short overview, we offer insights from the workshop that show challenges and opportunities for building this research in Australia.

An Australian Perspective on Human-Robot Collaboration

Discussions that considered “The Tyranny of Distance”, the classic account of how Australia’s geographical remoteness has been central to shaping the people’s values and available resources, identified the lack of robotics manufacturing facilities as one major challenge. While this limits the opportunity to conduct fundamental research that contributes to the hardware design of robots, this workshop discussed how this limitation enables Australian HRC researchers to focus more on application-specific and ‘in the wild’ research. Discussions around the cultural mindset that tends to lack trust in automation highlighted that this mindset can be used in answering critical research questions such as what work should and should not be automated. Workshop participants also called attention to one of the most fundamental problems that is not only relevant to Australian context or HRC research. Currently in research gatherings, we focus on sharing success stories. We need to start facilitating “failure tracks” to share the lessons learnt from failures if we want to truly expedite the advancement of HRC research.

Beyond the Australian perspective, the researchers offered several directions for future work in human-robot collaboration:

Human role in HRC: Participants discussed the terms we use for people in human-robot collaboration. Suggestions included collaborator, partner, and helper. Similarly, the role of the robot in relation to the person was discussed, e.g., robot as leader, follower, co-artist, etc. Examples of appropriate and creative ways of partnering included mimicking and training.

Improving awareness: An emerging research interest in enhancing robots’ and human collaborators’ awareness of each others’ states and actions was observed. Specifically, some participants were interested in studying how to model human internal states (e.g., cognitive workload) and individual differences using multimodal analytics, so robots can use related outcomes to adapt to human needs and augment humans. Another area of interest was to implement intelligent user interfaces to assist humans in predicting states and actions of robots and understanding reasons behind robots’ decisions.

Empathy for robots: While the focus of human-robot collaborative applications has mostly been on fulfilling human needs and achieving human goals, participants also highlighted that putting people in the ‘shoes’ of robots is also important. Topics discussed in this space included interface requirements of robots (considering robot-human as well as robot-robot interactions) and the need to prevent human abuse of robots.

Ethical considerations: Discussion covered the ethical considerations associated with the purpose, design and intended users of cobots, and the ways these may interact. Certain purposes, such as intimate engagements or persuasion, have ethical implications yet to be fully explored or resolved. Given the purpose of many collaborative robots is social, there are special considerations that need to be made when identifying and responding to the needs of vulnerable populations, such as children, elderly people, and people with disabilities. Design matters with broad relevance to ethical robotics include safety, appropriate trust, and transparency. Participants suggested special consideration should be given to the potential consequences of anthropomorphism of robots, including mistreatment of robots, attribution of moral agency, and uncanny valley backlash.

HRC design aims: There is a need for designers to consider how the design of the robot complements people. One example is the opportunity for robots to extend abilities of, or provide access for, people with disabilities. Related to this, participants noted the barriers of robots complementing people, including building and negotiating trust, making explainable robots, and designing collaboration in iterative steps.

HRC design approaches: Several discussions revolved around materials and prototyping approaches for design of collaborative robots. This was supported by questions around how to involve end users in prototyping processes and what kinds of wizard of oz methods could be appropriate. Another discussion around design approaches was centered on robot creativity and how to support human-robot interaction for creative purposes.

Evaluating HRC: A challenge for the research field is how to evaluate human-robot collaboration outcomes. Participants noted that this was mainly due to the dynamic nature of the collaboration. An example is trust which changes over the duration of the collaboration.

 

We thank all the participants for their valuable perspective, and CINTEL for a great workshop collaboration. Let’s continue to build this research community in Australia.

The Australian “The rise of AI is shaking up the world of work”, Dr Penny Williams

Posted on by Merryn Ballantyne

Research Program Lead Dr Penny Williams featured in The Australian today in an article entitled, “The rise of AI is shaking up the world of work”, part of the Women in Education, Cutting Edge Careers special edition.

Penny talked about her research with the Australian Cobotics Centre and the opportunities available for women working with AI and robots in the future. “Women should consider courses that, in addition to giving them a trade or professional qualification, will help them develop digital skills, including basic coding. (They’ll also require) entrepreneurial thinking, strong communication and problem-solving skills, and the ability to collaborate with both humans and machines.”

Read the full article: https://lnkd.in/gGjv_zen