Reflexive Control for Manipulation

Overview

Within the field of robotic manipulation, much research focus has been placed on improving perception and planning algorithms, assuming that the actions output by these high-level planners will be easily achieved by the robot systems. However, while improved perception and planning are certainly necessary for robotic systems to surpass human manipulation performance, fast and robust execution of the manipulation plans is also critical. We are implementing a manipulation architecture that decouples low-level, high-bandwidth behaviors, which we call reflexes, from high-level plans. The reflexes are fast, autonomous reactions to local sensing information that are designed to add robustness to high-level manipulation plans while also reducing the necessary complexity of manipulation planning problems.

By designing and using hardware platforms that incorporate high-bandwidth actuation and low-latency tactile sensing, we maximize the reactive capabilities of the system. Our initial studies for teleoperation and autonomous grasping have validated this approach, as our preliminary reflexive controllers have increased manipulation speed and robustness.

Motivation

There are inherent limitations to the current preferred approaches to solving robotic manipulation problems. They generally require precise planning, leading to behaviors that are afraid of premature contacts, and rely heavily on vision data for feedback, which experiences high latency and is prone to occlusions. Overall, these approaches become slow and clunky, and they are not able to perform human-like manipulation skills in common human-centric environments.

Below is an example of a human picking pears one at a time from a larger group. In contrast to robotic manipulation approaches, the hand and fingers are compliant, quickly adapting to local collisions and contact information. Once a pear is enveloped, even if it is not the top-most fruit or is not separated from the rest, it can be easily and securely picked from the pile. As the hand is picking fruits, it reacts and carries out a series of feasible actions, instead of a carefully calculated optimal picking order.

We believe a different approach to designing robotic manipulation systems is necessary, in which we prioritize fast reactions and searching for feasible solutions rather than planning to optimality. The difference in the two approaches is illustrated in the following diagram. In a typical system, a motion planner takes some task specification and uses global sensing information to construct a precise plan over several seconds or even minutes. While the individual motors can be controlled at much higher frequencies, any deviations or inaccuracies can ruin the current plan invalid. By inserting a reflexive control layer, the planning problem becomes simpler due to the decoupling of local sensing information and motor control from global action planning. Additionally, the feedback loops can occur at much higher frequencies, increasing the robustness of the reflexive manipulation behaviors.

A key motivation behind developing this reflexive control architecture is to separate the representation and execution of manipulation plans. This allows for more abstract reasoning at the planning level and less coupling between the overall plan and individual contact events between the fingers and the object. Additionally, fast feedback from tactile sensors enables the system to make multiple attempts per high-level plan in less time than it would take to re-plan from scratch.

Our overall hypothesis is that quickly finding and trying several feasible actions will lead to more successful and robust manipulation behaviors than calculating and executing the optimal action just once.

Projects

Towards Robust Autonomous Grasping with Reflexes Using High-Bandwidth Sensing and Actuation (ICRA 2023)

Abstract:

Modern robotic manipulation systems fall short of human manipulation skills partly because they rely on closing feedback loops exclusively around vision data, which reduces system bandwidth and speed. By developing autonomous grasping reflexes that rely on high-bandwidth force, contact, and proximity data, the overall system speed and robustness can be increased while reducing reliance on vision data. We are developing a new system built around a low-inertia, high-speed arm with nimble fingers that combines a high-level trajectory planner operating at less than 1 Hz with low-level autonomous reflex controllers running upwards of 300 Hz. We characterize the reflex system by comparing the volume of the set of successful grasps for a naive baseline controller and variations of our reflexive grasping controller, finding that our controller expands the set of successful grasps by 55% relative to the baseline. We also deploy our reflexive grasping controller with a simple vision-based planner in an autonomous clutter clearing task, achieving a grasp success rate above 90% while clearing over 100 items.

The paper preprint can be found here.

Fast Reflexive Grasping with a Proprioceptive Teleoperation Platform (IROS 2022)

Supplementary video can be found here.

Abstract:

We present a proprioceptive teleoperation system that uses a reflexive grasping algorithm to enhance the speed and robustness of pick-and-place tasks. The system consists of two manipulators that use quasi-direct-drive actuation to provide highly transparent force feedback. The end-effector has bimodal force sensors that measure 3-axis force information and 2-dimensional contact location. This information is used for anti-slip and re-grasping reflexes. When the user makes contact with the desired object, the re-grasping reflex aligns the gripper fingers with antipodal points on the object to maximize the grasp stability. The reflex takes only 150ms to correct for inaccurate grasps chosen by the user, so the user's motion is only minimally disturbed by the execution of the re-grasp. Once antipodal contact is established, the anti-slip reflex ensures that the gripper applies enough normal force to prevent the object from slipping out of the grasp. The combination of proprioceptive manipulators and reflexive grasping allows the user to complete teleoperated tasks with precision at high speed.

The paper preprint can be accessed at https://ieeexplore.ieee.org/abstract/document/9981383 and https://arxiv.org/abs/2208.04487.