Stretch Community News - April 2025
Hello again from Hello Robot!
This month, we were thrilled to be the first ever recipient of the RBR50 Robots for Good award, celebrating robots that make a positive impact on society. If you’re attending the Robotics Summit this week in Boston, stop by the expo floor and say hi!
There has been a lot of interesting new work in the Stretch community - this month we have open-source dynamic semantic mapping, servoing with vision models, zero-shot manipulation from RGB-only human videos, several new human-robot interaction papers, and more! Read on for details. If you’d like to see your work featured in a future newsletter, please let us know!
Georgia Tech researchers have developed an open-source system for mobile manipulators that builds and updates a semantic map of large, dynamic environments such as households, offices and warehouses. By integrating ORB-SLAM2, semantic segmentation, and the Khronos mapping pipeline, this approach tracks moving objects and people over time, enabling long-term scene understanding even when changes occur outside the robot’s immediate view.
Many mobile manipulation tasks require precise interaction with small objects, such as switches or knobs. To address this, researchers at UIUC propose Servoing with Vision Models (SVM), a training-free framework that uses advanced vision models with a wrist-mounted RGB-D camera for computing 3D targets. By out-painting occlusions and combining object detection with point tracking, SVM achieves 85% zero-shot success on unseen tasks, significantly outperforming baseline methods.
VidBot is a new framework from TU-Munich and ETH-Zurich that enables zero-shot robotic manipulation using learned 3D affordances from in-the-wild monocular RGB-only human videos. The system uses a depth foundation model and structure-from-motion techniques to extract 3D hand trajectories from videos, building temporally-consistent and embodiment-agnostic affordance representations. A coarse-to-fine affordance learning model enables zero-shot generalization across 13 manipulation tasks and deployment on real robots in diverse environments.
Many approaches to human-robot communication assume the robot already knows where important objects are in the environment - unrealistic in many real-world scenarios. Researchers at Naval Research Labs have developed the Uncertainty Expression System (UES), which lets users share their knowledge of object locations through precision input, heat map painting, or ranked lists; a user study showed that ranking was the most efficient and user-friendly method, while painting was the least accurate.
A new study from Deakin University compared two robot delivery methods - handing objects directly to users vs. placing them on a table - while the user is either occupied or unoccupied. Results from a 25-person study showed that table placement significantly improved user satisfaction, perceived safety, confidence in the robot, and intuitiveness, especially when the user was busy, supporting table placement as a default delivery strategy in real-world environments.
New work led by Georgia Tech and Emory evaluates a robot-led physical therapy system designed to enhance care for people with Parkinson’s disease, with 11 clinical exercise specialists (ES) assessing its effectiveness and usability. The system was well-received for its potential to enhance patient engagement and adherence to home exercises, and highlighted the importance of human-like feedback systems and general ease of use.
A new journal article from CMU researchers presents a wearable high-density electromyography (HDEMG) device that enables real-time gesture-based control of a mobile manipulator for assistive and household tasks. Tested with 13 participants, the system demonstrated precise control of Stretch during complex self-care and daily activities, highlighting its potential as a non-intrusive, wearable interface for in-home assistance.
The newest Stretch Software drop details a web interface implementation of Firebase networking and new user authentication tools, trajectory mode and waypoint tracking improvements, improved OS upgrade procedure, a new ROS2 debugging tutorial, and fixes for ROS2 action server timeouts and other bugs.
