Visual Intelligence for Robotic Arms in Space

As we continue in pushing the frontiers of technology on Earth, our future forays into space will depend on ambitious missions that will reach for the stars. Remarkable innovations taking shape today will enable us to assemble and service satellites in orbit, extract resources on the Moon, and build a resilient infrastructure beyond our planet. To achieve this, we’ll need space robots that are highly capable, scalable, and intelligent, so they can perform a variety of tasks in challenging environments.

Until recent decades, the space sector was driven mainly by national activity, dominated by government funded agencies and programmes. But a seismic shift has occurred that has seen much greater involvement from private companies. Known as the NewSpace movement, there has been a dramatic progression in the speed at which innovative technologies are being developed, as well as a significant increase in the number of space start-ups. In Luxembourg alone, there are around 70 companies in the space sector – with 20 being launched in 2021. The country has also seen the establishment of a centre dedicated to space resources – the European Space Resources Innovation Centre (ESRIC).

Redwire Space Luxembourg is one of the many ambitious companies in the vibrant NewSpace sector in Luxembourg. They develop cost-effective and modular robotic arms to enable dexterity in orbital and planetary surface operations, hopefully enabling us to service satellites, assemble large telescopes and harness resources on orbit. Leveraging its long-standing heritage and experience of its parent company places the company in a unique position for developing state-of-the-art robotic technology. Right now, they’re focusing on furthering their robotic arm and the underlying technologies that will provide the dexterity and intelligence required to conduct complex servicing, assembly and in-situ resource utilisation tasks in space. To help them with this task, Redwire Space Luxembourg partnered with the Space Robotics (SpaceR) group at the University of Luxembourg’s Interdisciplinary Centre for Security, Reliability and Trust (SnT). The partnership will foster research into state-of-the-art perception algorithms and visual intelligence apparatus that will enable the robotic arm to detect, track and handle objects in space. The project is also supported by the Fonds National de la Recherche (FNR) in the form of a Ph.D. Industrial Fellowship Grant, entitled ‘Modular vision for dynamic grasping of unknown resident space objects’.

Left to Right: Jaroslaw Jaworski, Miguel Olivares Mendez, Kuldeep Barad, Vinayak Vadlamani, Carol Martinez Luna

“In theory, the idea of robots picking and moving objects seems simple – we think nothing of it as humans – but in practice it’s actually a very complex and nuanced problem,” explained Kuldeep Barad, the doctoral researcher on this project. “Especially,” he continued, “If a robot must handle a lot of unknown objects, with different shapes, sizes and materials.” As humans, we have the dexterity and sensory intelligence that makes us remarkably good at grasping random objects, building furniture, and making food. But for a robot, this can be extremely hard to accomplish – without even taking into consideration the extreme environment of space, where a lot of the approaches that are currently used on the ground are either ineffective, unreliable, or intractable. An additional factor is the level of autonomy we’d like robots to have. “Ideally, the space robots should be able to perform complex tasks autonomously, because human intervention may not always be feasible or possible. What we want is to provide robots with the visual intelligence that will allow them to see, understand and reason about their environment,” Barad explained. In this context, his research focuses on a fundamental skill of vision-based robotic manipulation. This research addresses foundational questions that not only allow the robot to grasp objects, but also incrementally to clear debris, assemble spacecraft, build habitats and extract resources.

Within the project, the work has so far focused on a simulation environment, tools and system design, which forms the foundation of the project. In the framework of SnT’s Partnership Programme, Barad additionally shares his presence between SnT and Redwire Europe. This allows for a close integration with the company and steers the course of the research outcomes into a direction that is much closer to industrial competencies and practicalities. In the first few weeks of 2023, the team has also undertaken a series of tests in SnT’s Zero-G Lab – a versatile facility that mimics perception and motion in a space environment. “The fast evolution of the space industry means that state-of-the-art technologies being developed are quickly becoming outdated or obsolete,” said Barad. “It’s a challenging and exciting field to be a part of for this reason, and it means we have to continually adapt our approach with research, so that our technology can remain relevant in the years to come,” he enthused.