Author: Katherine Christensen

Australian Cobotics Centre researchers have two papers accepted for publication at the upcoming IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2024 in Abu Dhabi. IROS is one of the largest and most important robotics research conferences in the world, attracting researchers, academics, and industry professionals from around the globe.

Postdoctoral Research Fellow, Dr Fouad Sukkar gave is a brief summary of two of the papers appearing at the conference in October this year.

Constrained Bootstrapped Learning for Few-Shot Robot Skill Adaptation, by Nadimul Haque, Fouad (Fred) Sukkar, Lukas Tanz, Marc Carmichael, Teresa Vidal Calleja, proposes a new method for teaching robot skills via demonstration. Often this is a cumbersome and time-consuming process since a human operator must provide a demonstration for every new task. Furthermore, there will inevitably be some discrepancies between how the demonstrator carries out the task versus the robot, for example, due to localisation errors, that need to be corrected for in order for the skill to be successfully transferred. This paper tackles these two problems by proposing a learning method that facilitates fast skill adoption to new tasks that have not been seen by the robot. We do so by training a reinforcement learning (RL) policy across a diverse set of scenarios in simulation offline and then use a sensor feedback mechanism to quickly refine the learnt policy to a new scenario with the real robot online. Importantly, to make offline learning tractable we utilise Hausdorff Approximation Planner (HAP) to constrain RL exploration to promising regions of the workspace. Experiments showcase our method achieving an average success rate of 90% across various complex manipulation tasks compared to state-of-the-art which only achieved 56%.

Coordinated Multi-arm 3D Printing using Reeb Decomposition, by Jayant Kumar , Fouad (Fred) Sukkar, Mickey Clemon, Ramgopal Mettu, proposes a framework for utilising multiple robot arms to collaboratively 3D print objects. For robots to do this efficiently and minimise downtime while printing, they must have the flexibility to work closely together in a shared workspace. However, this dramatically increases problem complexity since there is a need to coordinate the arms so they do not collide with each other or the partially printed object. This is in addition to the planning problem of effectively allocating parts of the object to each robot while respecting the physical dependencies of the print, for example an arm can’t start extruding a contour until all the contours below it are printed first. All these factors make effective coordination a very computationally hard problem and we show that with bad coordination you can end up with even worse utilisation than if a single arm had carried out the same print! In this work we address this by performing a Reeb decomposition of the object model which partitions the model into smaller, geometrically distinct components. This drastically reduces the search space over feasible toolpaths, thus allowing us to plan highly effective allocations to each arm using a tree search-based method. For producing fast collision avoiding motions we utilise Hausdorff Approximation Planner (HAP). Our experimental setup consists of two robot arms with pellet extruders mounted on their end effectors. We evaluate our framework on 14 different objects and show that our method achieves up to a mean utilisation improvement of 132% over benchmark methods.

In advanced industries, the integration of Extended Reality (XR) technologies into Human-Robot Collaboration (HRC) presents unprecedented opportunities and challenges. XR, encompassing Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR), plays a crucial role in overcoming barriers to HRC adoption across various sectors. This article introduces the current applications of XR in HRC, addressing aspects such as types and roles, design guidelines and frameworks, and devices and platforms. It also provides insights into the future direction of XR in HRC, highlighting its potential to enhance collaboration and efficiency in industrial environments.

Extended Reality

In general, Extended Reality (XR) serves as an umbrella term for immersive technologies like Virtual Reality (VR), Mixed Reality (MR), and Augmented Reality (AR). Virtual Reality immerses users in a completely computer-generated environment (including visual, acoustical, tactile information), while Augmented Reality enhances the real-world environment by overlaying digital information or objects onto it. Specifically, Mixed Reality (MR) refers to formats that bridge the gap between reality and Virtual Reality.

In Human-Robot Collaboration (HRC), XR technologies are trending towards enhancing safety, improving workspace design, data visualisation, training operators, and creating more intuitive user interfaces due to their capability to visualise unseen information in the physical world in real time. These applications are closely linked to aiding human decision-making. By enhancing safety, XR technologies reduce the cognitive workload on operators, allowing them to focus on critical decision points. Well-designed XR-enabled workspaces facilitate the seamless integration of human and robotic workflows, boosting collaboration and efficiency. Advanced visualisation and immersive training capabilities provided by XR tools give operators a better understanding and control, leading to higher quality and precision in their decisions. Intuitive XR-based interfaces improve human-robot interactions, resulting in faster and more efficient decision-making. This effective decision-making is crucial in complex and dynamic HRC environments.

Extended Reality in Human-Robot Collaboration

From 2023 onwards, research has explored various types of XR technologies applied in Human-Robot Collaboration (HRC), including Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). Generally, XR is primarily used as an interface. Additionally, XR serves multiple roles such as development environments, learning environments, platforms for design, visualisation, simulation, instruction and guidance, task and motion planning, and more.

Currently, VR is used as an interface, evaluation tool, simulation platform, task and motion planning aid, learning environment, design tool, and for data collection. Conversely, AR overlays digital information onto the real world, making it ideal for enhancing and augmenting real-world interactions. MR blends the physical and digital worlds, providing immersive experiences that enhance real-time interactions and task execution. The distinction between AR and MR is often unclear, with AR considered a subset of MR. Telepresence, achievable by combining VR and MR, allows multi-human-robot teams to collaborate from different locations.

In current research on XR in HRC, various XR devices such as (Head-Mounted Displays) HMDs, mobile devices, and projectors are utilised. While HMDs are commonly employed, projectors are sometimes used for AR-based interfaces in HRC. Additionally, mobile devices like tablets are utilised for AR-based visualisation, instruction and guidance, interfaces, and training.

Regarding software and tools for developing XR in HRC, the game engine Unity is the most popular choice. In specific areas such as HRC fabrication, Building Information Modelling (BIM) platforms, and Computer-Aided Design (CAD) platforms like Rhino 3D and Grasshopper are used. Unity is generally preferred because it is powerful enough to support various platforms and users.

The Future of Extended Reality in Human-Robot Collaboration

Recently released HMDs such as Varjo XR-4 and Apple Vision Pro, AR goggles such as Xreal Air 2 Pro and Viture Pro show considerable promise for future use in HRC. The newest HMDs feature enhanced display resolution, refresh rates, and reduced latency, making them increasingly powerful. Conversely, AR goggles are lightweight while still offering high resolution and refresh rates. Moreover, mobile devices such as tablets and smartphones remain highly accessible and user-friendly for mobile AR applications, continuing to be a viable option for future use. The potential of Unreal Engine and WebGL also warrants further exploration. Unreal Engine provides photorealistic visuals for the most immersive visualisations, while WebGL enables users to interact through web-based applications from various locations and devices, enhancing accessibility and flexibility.

Current designs often focus either on XR or HRC without sufficient attention to user experience and human factors. Therefore, future research should integrate human factors and user-centric approaches to enhance the effectiveness and usability of XR in HRC. This comprehensive analysis highlights the importance of combining advanced XR technologies with human-centric design to optimise human-robot collaboration.

 

 

 

As a researcher deeply invested in advancing knowledge and innovation, I’ve consistently encountered a significant challenge: securing meaningful partnerships with industry. This gap between academia and industry isn’t just a personal observation; it’s a widespread issue that affects the pace of innovation and the practical application of cutting-edge research. Today, I’d like to dig into why this disconnect exists.

The Barriers to Collaboration

1. Time Constraints

In the fast-paced world of industry, time is often equated with money. This perspective can create significant barriers to research collaboration:

• Research Timelines: Academic research often operates on longer timelines, sometimes spanning years. This can clash with the quarterly or annual targets that drive many businesses.
• Production Slowdowns: There’s a prevalent fear that engaging in research might slow down existing production processes or divert resources from immediate business needs.
• Return on Investment (ROI) Concerns: Companies often struggle to see the long-term benefits of research when faced with short-term pressures to deliver results.

1. Financial Considerations

The financial aspect of research collaboration is another major hurdle:

• High Costs: Cutting-edge research often requires significant financial investment in equipment, materials, and personnel.
• Limited R&D Budgets: Many businesses, especially small and medium enterprises, lack dedicated research and development budgets.
• Risk Aversion: There’s an inherent uncertainty in research outcomes, making it a risky investment from a business perspective.
• Funding Complexities: The procedures for securing and managing research funding can be complex and time-consuming for businesses unfamiliar with academic processes.

1. Knowledge Gap

Perhaps the most insidious barrier is the knowledge gap that often exists between academia and industry:

• Technological Unfamiliarity: Many industries are comfortable with their current technologies and processes, making them hesitant to explore new, unproven methods.
• Resistance to Change: There’s often a cultural resistance to change within established industries, making it difficult to introduce new research-based innovations.
• Communication Challenges: Researchers and industry professionals may struggle to communicate effectively due to differences in jargon, priorities, and perspectives.
• Lack of Awareness: Many businesses simply aren’t aware of the potential benefits that academic research could bring to their operations.

The Importance of Collaboration

Despite these challenges, the importance of industry-research collaborations cannot be overstated:

• Innovation Acceleration: When academics and industry professionals work together, it can dramatically speed up the process of turning theoretical knowledge into practical applications.
• Real-World Problem Solving: Industry partners provide researchers with insights into real-world challenges, helping to guide research in the most impactful directions.
• Economic Growth: Successful collaborations can lead to new products, services, and even entirely new industries, driving economic growth.
• Skill Development: These partnerships provide valuable opportunities for skill exchange, benefiting both academic researchers and industry professionals.

While the benefits are clear, bridging the gap between academia and industry remains a complex challenge. In our next article, we’ll explore potential solutions to strengthen these crucial partnerships. Stay tuned for “Bridging the Gap: Solutions for Effective Industry-Academic Collaboration”.

Manufacturing is one of the top 3 engineering-heavy sectors in Australia, employing more than 46,000 qualified engineers. The manufacturing sector currently has a 70% male workforce, as discussed by Australian Cobotics Centre PhD candidate Akash Hettiarachchi in his recent webinar. The importance of gender equity to Australia’s global competitiveness in manufacturing was also highlighted in a recent parliamentary inquiry, which recommended a national strategy to attract and retain under-represented groups (including women) to advanced manufacturing careers. Manufacturing organisations, government departments and industry bodies are making concerted efforts to increase gender balance in the sector so they can achieve the benefits of a diverse workforce. 

At present, only 14% of engineers working in Australia are women. I was recently invited by the Australasian Tunnelling Society and Engineers Australia to present and be part of a panel at an International Women in Engineering Day (INWED) event, Bridging the Gap: Addressing Gender Pay Disparities in Engineering. INWED celebrates women’s contribution to the engineering profession and the 2024 theme is Enhanced by Engineering. However, in all industry sectors and occupations in Australia and most of the world, women’s contribution is still under-valued in terms of pay.  

The current gender pay gap in Australia (the difference between the average earnings of men and women), is 21.7% including full time, part time and casual workers and payments such as bonuses, overtime and commission. This means that on average, for every $1 a male worker makes, a female worker makes 78 cents. The gap is still 13.7% even when only including the base salaries of full-time workers. National statistics, the international Global Gender Gap Index, company reporting, and research show that a gap exists even when considerations such as experience and education are controlled for, and only part of the gap can be attributed to different career choices. A gender pay gap exists across nations, industries, occupations and at different levels of pay. It is however higher in male dominated industry sectors, industries with higher bonus, overtime or commission payments, higher paid roles, and organisations with fewer women in leadership. 

At the Bridging the Gap event, we discussed the gender pay gap, the policy and reporting framework in Australia, and actions that individuals, managers and organisations can take to address pay disparities.  

For the first time in 2024, the Workplace Gender Equality Agency (WGEA) published the gender pay gaps of all private sector employers with 100 or more staff members. The WGEA Data Explorer provides a rich source of data for anyone interested in the gender equity performance, policies and strategies of their own and other organisations. As well as gender pay gap data, policy and action, you can use the WGEA Data Explorer to see and compare industry and employer data on other indicators including the composition of the workforce and boards, access to and use of flexible work and parental leave by men, women and managers, employee consultation and harassment. Initiatives such as conducting and acting on the results of a gender pay audit, making pay more transparent, increasing the proportion of women in leadership, identifying and removing gender bias from recruitment and promotion decisions, and encouraging men to access flexible work and parental leave can all improve the gender pay gap.  

Australian Cobotics Centre Program 5 (The Human-Robot Workforce) has several researchers with experience in researching gender equity. We can assist companies of all sizes to consider how they can evaluate gender equity and realise the benefits for their organisation.  

Hydraulic systems are integral to industrial applications that require significant force, such as mining and manufacturing. Despite their power and efficiency, traditional hydraulic systems pose operational risks, especially when relying on binary controls and low-resolution feedback mechanisms. To address these challenges, a research team from the University of Technology, Sydney, led by Danial Rizvi, explored the potential of multi-modal feedback to enhance safety and performance in hydraulic maintenance operations.

The Challenges of Traditional Hydraulic Systems

In industrial settings, hydraulic systems are essential for tasks like installing and removing bushings and bearings. However, these systems typically use binary controls, limiting operators to simple open or close actions. This lack of precision can lead to operational errors and safety risks. Operators often rely on visual and auditory cues, which can be inconsistent and unreliable, increasing the potential for accidents and equipment failure.

Multi-modal Feedback: A New Approach

The research aimed to improve hydraulic maintenance operations by integrating haptic feedback through an adaptive trigger mechanism. This approach provides operators with tactile feedback, simulating the pressure build-up in hydraulic systems. The study compared the effectiveness of this haptic feedback against traditional visual and auditory cues.

Methodology

The team conducted a user study involving 10 participants operating a simulated hydraulic system using a re-programmed DualSense controller. This controller provided four types of feedback: force (through adaptive trigger resistance), visual (pressure readings), sound (auditory cues), and vibration (tactile cues). Participants performed tasks under different feedback conditions to evaluate the impact on performance and user experience.

Performance Analysis

The study measured three key performance metrics: elapsed time, final pressure (PSI), and extension percentage. The results showed no significant differences in task performance across the different feedback types. However, participants expressed a preference for the adaptive trigger in subjective evaluations, noting that it enhanced their control and reduced cognitive load.

Subjective Ratings

Participants rated their comfort and confidence with each feedback type. The adaptive trigger received the highest median comfort rating, while the vibration feedback was the least preferred. Overall, the study found that while all feedback types enabled participants to achieve the desired hydraulic pressures, the adaptive trigger offered slight advantages in user comfort and perceived control.

Implications for Industrial Maintenance

The integration of haptic feedback into hydraulic systems holds promise for improving safety and efficiency in industrial maintenance. By providing operators with more precise and intuitive control mechanisms, multi-modal feedback systems can reduce reliance on less reliable sensory cues and enhance overall operational safety.

Future Research

Further research is needed to explore the long-term benefits of multi-modal feedback in diverse industrial environments. Expanding the participant pool and incorporating real-world scenarios will help validate these findings and refine the technology for broader application.

Conclusion

The study conducted by the University of Technology, Sydney, demonstrates the potential of multi-modal feedback to enhance hydraulic maintenance operations. While traditional feedback mechanisms remain effective, the adaptive trigger offers additional benefits in user comfort and control. As industries continue to evolve, integrating advanced feedback systems into hydraulic operations can lead to safer and more efficient maintenance practices.

References:

• Danial Rizvi, Dinh Tung Le, Munia Ahamed, Sheila Sutjipto, Gavin Paul. “Multi-modal Feedback for Enhanced Hydraulic Maintenance Operations.” University of Technology, Sydney.

These workshops were organised and run by Swinburne University of Technology’s Factory of the Future and were funded through the Victorian Government’s Digital Jobs for Manufacturing (DJFM) program. 

This article is written by PhD researcher from Swinburne University of Technology, Jagannatha Pyaraka.

In a series of enlightening workshops, Swinburne University of Technology has taken significant step in bridging the gap between industry professionals and the transformative potential of Industry 4.0 technologies. Over the past few weeks, four workshops were organized at strategic locations to maximize outreach and impact. The workshops were held at the VGBO office in Bundoora, Holiday Inn Dandenong, Rydges Geelong, and Mercure Ballarat. These sessions aimed to raise awareness and provide hands-on experience with collaborative robots (cobots), a foundation of modern industrial automation and other Industry 4.0 technologies such as AR, VR and wearable sensors.

The workshops attracted operations managers, CEOs, CFOs, and other key decision-makers eager to understand the practical applications and benefits of cobots in their respective fields. Accompanied by my ACC colleague, Dr. Anushani Bibile, we used the easily portable and cost-effective UFactory xArm6 cobot to demonstrate cobotics functionality.

The workshops commenced with an introduction to collaborative robots. Unlike traditional industrial robots, which often require extensive programming and are confined to specific tasks, cobots are designed to share a workspace with humans. Their ease of programming, adaptability to various tasks, and advanced safety features make them suitable for dynamic and evolving industrial environments.

To illustrate these points, we demonstrated a program involving the stacking of four objects. The objects were placed in predefined positions, and xArm6 was tasked with picking each object and stacking them. This exercise highlighted the cobot’s ability to perform repetitive tasks and its intuitive programming interface. Using Blockly, a visual programming language, participants observed how quickly and easily they could teach the cobot to execute tasks.

Following the demonstration, participants had the opportunity to interact with xArm6. They used Blockly to program the cobot for a simple pick-and-place task. This exercise allowed them to experience the user-friendly interface and the cobot’s responsiveness. The feedback was positive, with many participants noting how quickly they could learn to program and operate the cobot.

The hands-on session helped to remove common misconceptions about the complexity and inflexibility of industrial automation. By the end of the workshop, participants had a better understanding of how cobots can be integrated into their operations to enhance productivity, safety, and cost-effectiveness.

The workshops also emphasized the cost-effectiveness of cobots. Unlike traditional robots that require significant investment in programming and setup, cobots like the xArm6 offer an affordable solution without compromising performance. Their advanced safety systems, which allow them to operate safely alongside human workers, make them a viable option for businesses of all sizes.

Specific feedback from participants highlighted the positive impact and value of these sessions. One attendee noted, “The workshop provided a great insight into how Industry 4.0 can better impact our business and automate our processes.” Another participant appreciated the practical demonstrations, stating, “It was great to see the practical applications during the demonstrations.” Many attendees emphasized that the hands-on experience was invaluable, with one remarking, “Cobots demo was very stimulating. Thoroughly enjoyed the workshop.”

Before the workshop, common reactions included uncertainty about the complexity and applicability of cobots in their operations. After the sessions, many participants expressed confidence in integrating these technologies into their workflows, recognizing the potential for improved efficiency and innovation.

Overall, these workshops effectively bridged the knowledge gap for attendees, providing them with the tools and understanding necessary to embrace Industry 4.0 technologies. As more companies recognize the benefits of automation, the demand for cobots is set to rise, paving the way for a more efficient and innovative industrial landscape.