Developing a Perception Pipeline for MIT Humanoid Robot
Project Information
Semester: Fall 2025
Duration: 1 semester +
* This project is open for MIT Students only
Project Overview
Humanoid robots hold tremendous potential for real-world applications in everyday environments. To perform such tasks effectively, robust and reliable perception systems are essential. The challenge becomes even greater in highly dynamic scenarios such as sports, where perception must be both fast and accurate. This project aims to develop an egocentric perception pipeline for humanoid sports applications (e.g., soccer). The student will work on camera-to-IMU calibration and design a perception software stack for the humanoid robot. The primary outcome will be a ball state estimation framework that processes RGB-D images to infer the ball's position, velocity, and other relevant properties. The project provides an opportunity to collaborate closely with PhD students and postdoctoral researchers, while gaining hands-on experience with state-of-the-art robots, sensors, and algorithms. The main focus will be on developing and integrating the perception module into a full loco-manipulation framework.
Work Packages
- Literature review on humanoid perception and ball tracking
- Camera (e.g., Intel RealSense) to robot IMU calibration
- Develop a ball state estimation module relative to the humanoid
- Integrate the perception module with the robot's controller
Keywords
- Humanoid, Loco-Manipulation, Perception, Camera Calibration, State Estimation
Requirements
- Linux systems
- Python programming
- Linear algebra
- C++ programming
- Machine Learning
- Computer Vision project experience
- Camera calibration
- Robotics communication (ROS, LCM)
- Linear Algebra
- Computer Vision
- Introduction to Programming
Required Skills
Preferred Skills
Relevant Coursework
Contact
-
Ho Jae Lee - hjlee201@mit.edu
Justin Lin - tzuyuan@mit.edu