Published on: March 12, 2025 / Updated on: March 12, 2025 – Author: Konrad Wolfenstein
Researchers Sepehr Samavi and Prof. Angela Schoellig next to robot Jack – Photo: Astrid Eckert, Munich
Pioneering work in robotics: TUM develops predictive robot
Autonomous Systems: How Robots Learn to Interact with Humans
In a world rapidly evolving towards automation and artificial intelligence, autonomous systems are becoming an increasingly important part of our daily lives. From self-driving cars and intelligent assistive robots to sophisticated industrial plants, the ability of machines to make independent decisions and operate in complex environments is transforming numerous aspects of our lives. A particularly exciting and challenging discipline within robotics is the development of systems that can move safely and efficiently in dynamic, human-populated environments. This involves not only avoiding obstacles but also understanding, predicting, and responding to human behavior to ensure smooth and safe interaction.
Researchers at the renowned Technical University of Munich (TUM) are working intensively at precisely this intersection of robotics, artificial intelligence, and human behavior. In their Learning Systems and Robotics Lab, headed by Professor Angela Schoellig, they have developed an innovative robot named "Jack" that is capable of navigating crowds with remarkable skill and foresight. What distinguishes Jack from many other robots is its ability not only to perceive its immediate surroundings but also to actively consider how people in its vicinity will move and how they might react to its own movements. This anticipatory thinking allows Jack to plan its route through busy spaces not just reactively, but proactively and intelligently.
Suitable for:
The challenge of navigating in crowds
Navigating crowds presents a formidable challenge for robots, one that extends far beyond simply avoiding obstacles. Unlike static or predictable environments, crowds are dynamic, unpredictable, and characterized by complex social interactions. Each person in a crowd moves individually, yet simultaneously influences the movements of others. This interdependence, combined with the natural variability of human behavior, makes it extremely difficult for robots to move safely and efficiently.
Traditional robot navigation algorithms, often based on rigid rules and simple sensor data, quickly reach their limits in such environments. They typically react to obstacles by abruptly stopping or swerving, which can lead to unwanted congestion, inefficient routes, or even dangerous situations in a crowd. To navigate successfully in crowds, robots therefore need a significantly more advanced form of intelligence that allows them to understand and predict human behavior and actively incorporate it into their navigation planning.
Jack's innovative approach: Forward thinking and interaction
The robot Jack, developed by TUM researchers, takes a crucial step beyond traditional approaches. At its core is a sophisticated algorithm that enables it not only to perceive the movements of people in its environment, but also to actively predict them and incorporate them into its own route planning. Professor Schoellig emphasizes the fundamental difference to conventional methods: “Our robot models how people will react to its movements in order to plan its own routes. This is the major difference compared to other approaches that typically ignore this interaction.”
This ability to model interactions is key to Jack's success. Instead of viewing people merely as unpredictable obstacles, Jack understands them as intelligent agents whose behavior he can partially predict and even influence. This allows him to move through crowds in a way that closely resembles human navigation. He doesn't hesitate to move into gaps, anticipates pedestrian movements, and dynamically adjusts his route to avoid collisions while efficiently reaching his destination.
Sensors and computing power in combination
To accomplish this demanding task, Jack is equipped with highly advanced sensors and computing power. A key component is a lidar (light detection and ranging) sensor, which continuously emits laser beams into its surroundings and receives the reflected signals. From this data, the lidar creates a precise 360-degree map of the environment in real time, capturing not only static objects but also, and especially, the position and movement of people. The lidar thus provides the robot with a detailed "picture" of its environment, forming the basis for its navigation decisions.
In addition to lidar, Jack has sensors in its wheels that precisely measure its speed and distance traveled. This information is crucial for accurately determining its position in its surroundings and optimizing navigation efficiency. All sensor data is processed by a powerful onboard computer capable of executing complex algorithms in real time. This computer is Jack's "brain," responsible for analyzing sensor data, predicting human movement, and calculating the optimal route.
Suitable for:
The algorithm in detail: prediction, planning and adaptation
At the heart of Jack's intelligence is the navigation algorithm developed by TUM researchers. This algorithm works in several steps to enable Jack to navigate safely and efficiently through crowds.
1. Perception and data acquisition
First, Jack continuously gathers data about its environment using its sensors. The lidar provides information about the position and movement of people, while the wheel sensors provide data about the robot's own movement.
2. Predicting human movements
Based on the collected data, the algorithm analyzes the movement patterns of people in the vicinity. It attempts to predict the likely paths people will take in the next few seconds. This prediction is based on statistical models learned from extensive datasets of human movement behavior in crowds.
3. Route planning
At the same time, the algorithm plans the optimal route to the robot's destination. In doing so, it considers not only the predicted movements of people, but also the robot's own capabilities and limitations, such as its speed and maneuverability. The goal is to find a route that leads to the destination as quickly and efficiently as possible, without risking collisions with people.
4. Dynamic adaptation
A key aspect of the algorithm is its ability to adapt dynamically. The entire process of data acquisition, prediction, and route planning is continuously repeated approximately ten times per second. This allows Jack to adjust its route in real time to the constantly changing environment. This high adaptation frequency is essential for navigating safely and efficiently in a dynamic environment with many people, as the robot simultaneously recognizes and reacts to people's movements, as TUM researcher Sepehr Samavi explains.
Learning from human behavior: The key to human-like navigation
Another crucial aspect of Jack's intelligence is his ability to learn from human behavior. The TUM researchers didn't simply program Jack with rigid rules and algorithms, but instead gave him the opportunity to continuously improve by analyzing data on human movement behavior.
Professor Schoellig explains that the mathematical model on which the planning algorithm is based was derived from human movements and translated into equations. The algorithm thus relies not on abstract assumptions about human behavior, but directly on real data documenting crowd movements. To make this possible, the researchers collected extensive datasets describing human behavior in various situations and environments, which serve as training material for Jack.
By analyzing this data, Jack learns to recognize and anticipate typical human movement patterns and incorporate them into his own decisions. For example, he learns that people usually swerve when approaching an obstacle or adjust their speed to avoid a collision. This knowledge is fed into the algorithm, allowing Jack to behave in a way that resembles the intuitive behavior of people in crowds.
A concrete example of this learning process is Jack's handling of potential collisions. A traditional robot would typically stop immediately upon detecting an obstacle, such as a person, on a collision course. Jack, however, having learned from human behavior, reacts more subtly. He anticipates that people will usually adapt and swerve to avoid a collision. Therefore, he doesn't stop immediately but continues his movement while simultaneously observing the person's reaction. Only if there are indications that the person will not swerve does Jack adjust his plans and choose an alternative route. This behavior is significantly more efficient and human-like than the abrupt stop of a traditional robot.
Evolutionary development: From reactive to interactive
The development of Jack's navigation skills was an evolutionary process that unfolded in three stages. Each stage represents an advancement in the complexity and intelligence of the algorithm.
Level 1: Reactive navigation.
In the first stage, Jack merely reacted to his environment. He avoided obstacles as soon as he perceived them, without predicting or anticipating human behavior. While functional, this stage was inefficient and often led to abrupt stops and detours.
Level 2: Predictive navigation.
In the second stage, the algorithm was extended to predict the movement of oncoming people. This allowed Jack to navigate more proactively and avoid collisions before they were imminent. This stage already represented significant progress, but was still limited, as it largely ignored the interaction between robot and human.
Level 3: Interactive navigation.
The current version of Jack represents the third and most advanced stage of evolution to date: interactive navigation. At this stage, Jack is not only able to predict people's movements, but also to actively consider how people will react to his own. He is able to influence people's behavior through his own actions while simultaneously avoiding collisions. This interactive capability is the crucial breakthrough that makes Jack a truly intelligent and human-like navigation system.
Researcher Samavi explains that Jack can predict the movements of other people and simultaneously influence their actions through his own behavior, while avoiding collisions. This form of interactive navigation allows Jack to move safely, efficiently, socially acceptably, and intuitively through crowds.
Application areas: From delivery robots to autonomous driving
The innovative technology behind Jack has enormous potential for a wide range of applications. Although Jack was initially developed as a research platform, TUM researchers are already considering concrete applications in the real world.
Delivery robot
One obvious application is delivery robots that can autonomously deliver goods and packages in urban environments. These robots must be able to move safely and efficiently on sidewalks, in pedestrian zones, and in busy city centers. Jack's ability to navigate crowds is crucial for this. In the future, autonomous delivery robots could make a significant contribution to solving "last mile" problems in logistics and reducing urban traffic congestion.
Suitable for:
wheelchairs
Another promising application is the integration of the technology into smart wheelchairs. For people with mobility impairments, navigating busy environments can be a major challenge. A wheelchair equipped with Jack's navigation algorithm could significantly improve their independence and quality of life. The wheelchair could automatically avoid obstacles, move safely through crowds, and autonomously transport the user to their desired destination.
Autonomous driving
Professor Schoellig considers autonomous driving a particularly relevant application area for interactive navigation technology. She emphasizes that these interactive scenarios present a key challenge. In complex traffic situations, such as merging onto highways, turning at intersections, or interacting with pedestrians and cyclists, it is essential not only to plan one's own movements but also to anticipate the behavior of other road users and incorporate it into one's planning. The technology's ability to provide interactive navigation could thus make a significant contribution to the development of safer and more efficient autonomous vehicles. She cites merging onto a highway as an example: When a vehicle is on the acceleration lane of a highway entrance, many drivers approaching from behind change lanes or brake slightly. It is precisely in such situations that the new approach makes it possible to appropriately consider the reactions of other road users.
Humanoid robots
Humanoid robots could particularly benefit from these algorithms, especially in areas like caregiving, service, or manufacturing, where they work closely with humans. For them to be accepted and used effectively, it is essential that they can navigate safely and intuitively in human environments. Professor Schoellig, however, points to a key challenge: while a mobile robot can simply stop when needed, humanoid robots are currently quite unstable and quickly lose their balance. Improving the stability of humanoid robots in dynamic environments is an important area of research that needs further development to unlock the full potential of interactive navigation for humanoid robots.
Advanced robot navigation: How Jack understands human behavior
TUM's research in the field of interactive robot navigation represents a significant advancement toward intelligent and autonomous systems that can operate safely and efficiently in human environments. The robot Jack impressively demonstrates that it is possible to develop machines that can not only perceive their surroundings but also understand and predict human behavior and incorporate it into their decision-making. This ability for interactive navigation opens up new possibilities for a wide range of applications, from delivery robots and smart wheelchairs to autonomous driving.
The development of Jack, however, is just the beginning. Research in robotics and artificial intelligence is progressing rapidly, and we can expect further exciting innovations in the coming years and decades. The integration of robots into our everyday lives will become increasingly commonplace, and autonomous systems will play an ever more important role in our society. It is therefore crucial that we shape the development of these technologies responsibly and consider the ethical and societal aspects from the outset. Only in this way can we ensure that robots and humans can work together for the benefit of all in the future.
Suitable for:
Your global marketing and business development partner
☑️ Our business language is English or German
☑️ NEW: Correspondence in your national language!
Konrad Wolfenstein
I would be happy to serve you and my team as a personal advisor.
You can contact me by filling out the contact form or simply call me on +49 7348 4088 965 (Munich) . My email address is: wolfenstein ∂ xpert.digital
I'm looking forward to our joint project.
