Developing a Perception Pipeline for MIT Humanoid Robot

Project Information

Semester: Fall 2025

Duration: 1 semester +

* This project is open for MIT Students only

Project Overview

Humanoid robots hold tremendous potential for real-world applications in everyday environments. To perform such tasks effectively, robust and reliable perception systems are essential. The challenge becomes even greater in highly dynamic scenarios such as sports, where perception must be both fast and accurate. This project aims to develop an egocentric perception pipeline for humanoid sports applications (e.g., soccer). The student will work on camera-to-IMU calibration and design a perception software stack for the humanoid robot. The primary outcome will be a ball state estimation framework that processes RGB-D images to infer the ball's position, velocity, and other relevant properties. The project provides an opportunity to collaborate closely with PhD students and postdoctoral researchers, while gaining hands-on experience with state-of-the-art robots, sensors, and algorithms. The main focus will be on developing and integrating the perception module into a full loco-manipulation framework.

Work Packages

  • Literature review on humanoid perception and ball tracking
  • Camera (e.g., Intel RealSense) to robot IMU calibration
  • Develop a ball state estimation module relative to the humanoid
  • Integrate the perception module with the robot's controller

Keywords

  • Humanoid, Loco-Manipulation, Perception, Camera Calibration, State Estimation

Requirements

    Required Skills

    • Linux systems
    • Python programming
    • Linear algebra

    Preferred Skills

    • C++ programming
    • Machine Learning
    • Computer Vision project experience
    • Camera calibration
    • Robotics communication (ROS, LCM)

    Relevant Coursework

    • Linear Algebra
    • Computer Vision
    • Introduction to Programming

Contact

    Ho Jae Lee - hjlee201@mit.edu
    Justin Lin - tzuyuan@mit.edu