Published on: May 20, 2025 / Update from: May 20, 2025 - Author: Konrad Wolfenstein
The Twist system: Motion Capture Technology (Mocap) revolutionizes the control of humanoid robots-Image: Xpert.digital
Teleoperated Whole-Body Imitation System: Real-time human-robot interaction we change the robot technology
Human movements for robots: The potential of the Twist system
Scientists have achieved a significant breakthrough in the development of teleoperation systems for humanoid robots. By using motion capture technology, humanoid robots can now perform human-like movements in real time. This innovation enables precise and intuitive control of robots, which is an important step towards developing robots with full -body skill. The new system Twist (Teleoperated Whole-Body Imitation System), which transfers the full body movements of a person to a robot, is particularly remarkable and thus heralds a new era of human-robot interaction.
Suitable for:
The basics of motion capture-based telephoto surgery
Teleoperation describes the remote control of machines and is particularly important in the robotics area. Telerobotic systems are used if the work space is too far away, too small, too large or too dangerous for humans. The spatial decoupling between humans (operator) and robot (teleoperator) enables operations in various areas such as minimally invasive surgery, the intelligence description or space applications.
Motion capture technology (Mocap) forms the basis for modern teleoperation systems. This technology enables detailed recordings and simulations of human movement, whereby individual people or entire groups of people can be digitized. The movements recorded are intelligently processed and can be used for the animation of bodies and their movements.
How the motion capture technology works
In the motion capture technology, the body movements of real people are precisely pursued and recorded via a special suit with marking points and optical systems. Movement data of all parts of the body are collected - not only by arms, hands, legs and feet, but also by fuselage, hips and head. These comprehensive data are then converted into commands using artificial intelligence (AI) that can carry out humanoid robots.
The Twist System: A breakthrough in robot telescopic
The Twist system developed at Stanford University and Simon Fraser University is a significant progress in the humanoid robot telephoto.
"We want humanoids to have the same level of full-body skills as people," explains Yanjie Ze, first author of the Twist study. "Imagine an untidy kitchen. People can hold things with two hands and use their feet to move obstacles, such as a basket on the ground. Humans can also open the door with the sides of their body or elbows. We want humanoids to be able to do the same by imitation people directly."
Technical implementation of Twist
The Twist system includes three essential components:
- Data acquisition and retargeting: By offline and online retargeting, human movements are adapted to the robot. This is done by optimized transmission of the 3D joint positions and orientations, whereby body orientation and foot placement are also adapted in real time.
- Controller training in the simulation: Twist uses a two-stage approach with a “Teacher student” methodology:
- The “Teacher” controller has privileged access to future reference movements, which enables it to plan smooth movements.
- The “student” controller is trained by a combination of Reinforcement Learning (RL) and Behavior Cloning (BC) and can only access current movement information.
- Full body controller: The trained controller enables the robot to use all degrees of freedom while the balance is held at the same time. This results in more natural and human -like movements.
In tests with the Humanoid G1 robot from UNTERE, the researchers found that it was sufficient to record full-body movements and to precisely transfer them to the joints of the robot, with ensuring that the movements of the different limbs are carried out.
Suitable for:
Challenges in humanoid telephoto surgery
The development of teleoperation systems for humanoid robots presents researchers with several complex challenges:
Bridge the embodiment gap
A central challenge is the bridging of the “Embodiment gap”-the anatomical differences between humans and robots. Since robots have other proportions, joint configurations and physical properties than humans, direct transfer of human movements is not easily possible.
Balance and full body coordination
Humanoid full-body tracking not only requires the precise control of individual joints, but also the dynamic maintenance of balance during complex movements. In conventional telephoto surgery systems, the focus is often only on isolated movements such as locomotion or manipulation, while Twist enables coordinated full body movements.
Latency and sensory feedback
Teleoperation systems have to overcome problems such as latency (time delay) and restrictions on sensory feedback. These factors can affect the synchronization of human actions with robotic reactions.
Diverse applications of motion capture telephotia
The motion capture-based telephoto surgery of humanoid robots opens up numerous possible uses:
Dangerous situations and rescue operations
In dangerous environments, teleoperated robots can be used instead of people, for example in the defusing of explosives (EOD - Explosive Ordnance Disposal). Between 2015 and 2020 there were around 2,000 EOD operations annually in the United Kingdom alone, which illustrates the need for safe alternatives.
Complex manipulation tasks
Humanoid robots can do complex manipulation tasks through teleoperation, for example in unstructured environments such as kitchens or workshops. The ability to use the whole body including the arms, hands, legs and feet coordinated offers crucial advantages here.
Social robotics and expressiveness
For humanoid social robots, the ability to express expressive movements is essential. The OCRA system developed on the MPI (optimization-based customizable retargeting algorithm) enables real-time movement images between different kinematic chains, which leads to intuitive and human-like movements.
Alternative approaches and comparison of different systems
In addition to Twist, there are various other approaches for motion capture-based telephoto surgery:
IMU-based systems
Some researchers use IMU-based (Inertial Measurement Unit) Motion capture systems that are portable and cheaper than optical systems. This technology is used, for example, for the telephoto operation of loco manipulation tasks in which locomotion and manipulation are combined.
Neuronal network-based approaches
An alternative approach uses neural networks to learn an assignment between the sensor data of the motion capture suit and the angle positions of the robot upholstery. This method does not require a previous analytical or mathematical model of the robot and can therefore be applied to various human-robot pairings.
Systems for specific body parts
In addition to full-body telephoto operating systems, there are also specialized systems that focus on certain parts of the body, such as dual-hand movement recording. These systems play an important role in the precise control of bionic bimanual robots for fine manipulation tasks.
Suitable for:
Recent progress and future prospects
The development of teleoperation systems for humanoid robots is progressing rapidly. In addition to Twist, researchers recently presented further innovative systems:
H2O: Human to Humanoid
The H2O system only enables the real-time telephotia of a complete humanoid robot with an RGB camera. It uses an RL-based frame and a “Sim-to-Data” process to filter and select suitable movements for humanoid robots.
AR-supported telephoto surgery
Researchers also investigate how Augmented Reality (AR) can support Mocap-based telephones. By visualizing a virtual reference of the human arm in addition to the robot arm, users can better understand movement images.
KI and Motion capture: The future of human-robot interaction
The motion capture-based telephoto surgery of humanoid robots has developed considerably in recent years. Systems such as Twist mark significant progress by making it possible for robot to perform human -like full -body movements in real time.
The combination of motion capture technology and advanced AI methods such as Reinforcement Learning and Behavior Cloning opens up new opportunities for human-robot interaction. Humanoid robots can now not only perform isolated movements, but also coordinated full -body actions that enable higher skill and expression.
In the future, these technologies could significantly expand the use of humanoid robots in dangerous environments, with complex manipulation tasks and in social contexts. The continuous improvement of precision, robustness and user -friendliness of teleoperation systems will help to further reduce the gap between human abilities and robotic execution.
Suitable for:
Your global marketing and business development partner
☑️ Our business language is English or German
☑️ NEW: Correspondence in your national language!
Konrad Wolfenstein
I would be happy to serve you and my team as a personal advisor.
You can contact me by filling out the contact form or simply call me on +49 89 89 674 804 (Munich) . My email address is: wolfenstein ∂ xpert.digital
I'm looking forward to our joint project.
