Tactile robotics: Robots with a sense of touch: The new generation from Vulcan and MIT research on haptic object recognition – Image: Xpert.Digital
MIT's object recognition system without special sensors and Amazon's Vulcan robot
Haptic perception for machines: Setting new standards in object recognition
In the field of robotics, the development of tactile sensor and recognition systems marks a crucial advancement, enabling machines for the first time not only to see their environment but also to "feel" it. This development is exemplified by Amazon's new Vulcan robot and MIT's innovative object recognition system. Both technologies significantly expand the applications of robots and enable tasks that previously could only be accomplished by humans with their natural haptic perception.
Related to this:
Amazon's Vulcan robot: A breakthrough in the field of tactile robot gripping
Functioning and technological foundations
The Vulcan robot, developed by Amazon, represents a significant technological advancement in the field of physical artificial intelligence. Amazon itself describes the development as a “breakthrough in robotics and physical AI.” The system consists of two main components: “Stow” for storing objects and “Pick” for retrieving them. Its outstanding feature is its ability to perceive its environment tactilely.
The technological basis for Vulcan's tactile abilities consists of special force-torque sensors, shaped like a hockey puck, that allow the robot to "feel" how much force it can apply to grasp an object without damaging it. Adam Parness, Director of Robotics AI at Amazon, emphasizes the uniqueness of this approach: "Vulcan isn't our first robot that can move objects. But with its sense of touch—its ability to understand when and how it comes into contact with an object—it opens up new possibilities for optimizing workflows and facilities.".
To sort items onto shelves, Vulcan uses a tool resembling a ruler attached to a hair straightener. With this "ruler," he pushes other items aside to make room for new ones. The gripper arms adjust their grip strength depending on the size and shape of the item, while integrated conveyor belts move the item into the container. To retrieve items, Vulcan uses a suction gripper in combination with a camera system.
Current areas of application and performance
The Vulcan robot is currently being tested in two Amazon logistics centers: in Winsen near Hamburg (Germany) and in Spokane, Washington (USA). In Washington, six Stow Vulcan robots are in operation and have already successfully stored half a million items. In Winsen, two Pick Vulcan robots are working and have already processed 50,000 orders.
The system's capabilities are remarkable: Vulcan can currently handle approximately 75 percent of the millions of products offered by Amazon. The smallest object the robot can manipulate is roughly the size of a lipstick or a USB stick. Particularly impressive is the robot's ability to identify objects in real time, as it would be "impossible for it to know all the specifics of the items by heart," as Parness explains.
Future plans and integration into the logistics chain
Amazon plans to significantly increase the number of Vulcan robots in the coming years. This year, the number of Vulcans in Winsen is expected to increase to 60 and in Washington to 50. The long-term plan is to deploy the robots in logistics centers throughout Europe and the USA.
A key aspect of Amazon's strategy is the coexistence of humans and machines. The company's "master plan" envisions humans and machines working side-by-side. The robots are primarily intended to handle those products on the shelves that humans cannot reach without a ladder or that would require excessive bending. This is expected to lead to greater overall efficiency while simultaneously reducing the workload for human employees.
MIT's object detection system through handling: Intelligent "sensing" without special sensors
Innovative approach to object recognition
In parallel to Amazon's Vulcan, researchers from MIT, Amazon Robotics, and the University of British Columbia have developed a system that takes a different approach to giving robots haptic abilities. This technology allows robots to recognize properties of an object, such as weight, softness, or contents, simply by picking it up and gently shaking it—much like humans do when handling unfamiliar objects.
What's special about this approach is that no special tactile sensors are needed. Instead, the system uses the joint encoders already present in most robots – sensors that detect the rotational position and speed of the joints during movement. Peter Yichen Chen, an MIT postdoc and lead author of the research paper, explains the vision behind the project: “My dream would be to send robots out into the world so they can touch and move things and independently discover the properties of everything they interact with.”.
Technical functionality and simulation models
The core of the MIT system consists of two simulation models: one that simulates the robot and its movements, and one that replicates the dynamics of the object. Chao Liu, another MIT postdoc, emphasizes the importance of these digital twins: “An accurate digital replica of the real world is really important for the success of our method.”.
The system uses a technique called “differentiable simulation,” which allows the algorithm to predict how small changes in an object's properties, such as mass or softness, will affect the final position of the robot's joints. Once the simulation matches the robot's actual movements, the system has identified the correct properties of the object.
A key advantage of this method is its efficiency: The algorithm can perform the calculations within seconds and only requires a real-world motion trajectory of the robot to function. This makes the system particularly cost-effective and practical for real-world applications.
Application potential and advantages
The developed technology could be particularly useful in applications where cameras are less effective, such as sorting objects in a dark basement or clearing debris in a partially collapsed building after an earthquake.
Since the algorithm does not require a large dataset for training, unlike some methods that rely on computer vision or external sensors, it is less prone to errors when confronted with unknown environments or new objects. This makes the system particularly robust and versatile.
The broader research landscape on tactile sensors in robotics
Fundamental challenges and current solutions
The development of robots with a sense of touch presents researchers with fundamental challenges. While the human tactile system is extremely complex and nuanced, artificial systems must replicate this using technological means. Ken Goldberg, a roboticist at the University of California, Berkeley, emphasizes the complexity of this task: “The human sense of touch is incredibly nuanced and complex, with a vast dynamic range. While robots are making rapid progress, I would be surprised to see tactile sensors at a human level in the next five to ten years.”.
Despite these challenges, significant progress is being made in research. For example, the Fraunhofer IFF is developing tactile sensor systems that enable reactive gripping, mimicking the human hand, and are ideal for handling fragile or flexible objects. The sensor data is used for gripper adaptation, component and position recognition, and process monitoring.
Innovative research projects in the field of tactile robotics
Besides the developments by Amazon and MIT, there are other significant research projects in the field of tactile robot sensors:
The Max Planck Institute for Intelligent Systems has developed a haptic sensor called Insight that detects touch with high sensitivity. Georg Martius, head of a research group at the institute, emphasizes the sensor's performance: "Our sensor demonstrates outstanding performance thanks to the innovative mechanical design of its casing, the custom-designed imaging system inside, the automatic data acquisition, and state-of-the-art deep learning methods." The sensor is so sensitive that it can even sense its own orientation relative to gravity.
Another interesting project is DensePhysNet, a system that actively performs a sequence of dynamic interactions (e.g., sliding and colliding) and uses a deep predictive model over its visual observations to learn dense, pixel-wise representations that reflect the physical properties of observed objects. Experiments in both simulation and real-world environments show that the learned representations contain rich physical information and can be used directly to decode physical object properties such as friction and mass.
Related to this:
Future prospects for tactile robot systems
Integration of multimodal sensor systems
The future of tactile robotics lies in the integration of different sensory modalities. Researchers at MIT are already working on teaching artificial intelligence to combine senses such as sight and touch. By understanding how these different sensory modalities interact, robots can develop a more holistic understanding of their environment.
The MIT team is already planning to combine their object detection method with computer vision to create a multimodal sensor system that is even more powerful. “This work is not trying to replace computer vision. Both methods have their advantages and disadvantages. But here we have shown that we can already discover some of these properties even without a camera,” explains Chen.
Expanded application areas and future developments
The MIT team researchers also want to explore applications with more complex robotic systems, such as soft robots, and more complex objects, including sloshing liquids or granular media like sand. In the long term, they hope to use this technique to improve robot learning, enabling future robots to quickly develop new manipulation skills and adapt to changes in their environment.
Amazon plans to further develop and deploy Vulcan technology on a larger scale in the coming years. The integration of Vulcan with the company's existing fleet of 750,000 mobile robots suggests a comprehensive automation concept that could fundamentally transform the logistics industry.
Tactile learning: When sensors give robots a sense of touch
The development of robots with a sense of touch, exemplified by Amazon's Vulcan and MIT's object recognition system, marks a crucial turning point in robotics. These technologies enable robots to take on tasks that were previously the sole domain of humans, as they require fine motor skills and tactile understanding.
The different approaches – Amazon's focus on specialized sensors and MIT's concept of using existing sensors for haptic inference – demonstrate the diversity of research directions in this field. Both approaches have their specific strengths and areas of application.
With the increasing integration of tactile capabilities into robotic systems, new opportunities are opening up for the automation of complex tasks in logistics, manufacturing, healthcare, and many other fields. The ability of robots not only to see but also to "feel" their environment brings us a significant step closer to a future in which robots and humans can collaborate even more closely and intuitively.
Related to this:
