Data-efficient robot learning for decision making in the real world using knowledge of physics and prior experience

Yifan Zhu and Kris Hauser

Summary

Recent breakthroughs have made deep learning a popular approach in various fields of engineering and science, including robotics, but it is data-hungry and transfers poorly to new tasks once trained. Hence, compared to other fields like vision and NLP, learning has not yet made as much of an impact in robotics where data is often scarce and expensive to collect. This project investigates data-efficient learning methods that 1) taking advantage of knowledge of physics and performing "gray-box" learning and 2) using prior experience on similar tasks.

We have applied our approach on 3 robotics tasks in which gray-box learning has substantially improved data-efficiency compared to black-box learning. Our approaches are applied to locomotion granular media, visuo-tactile model learning of deformable objects, and automated heart and lung auscultation. Beyond the problem of learning models, we also address decision-making problems that use such models for dynamics, reward, and/or constraint prediction.

Highlights of this work include the introduction of a novel bilevel trajectory optimization techniques for legged robots to traverse sandy terrain, where the sand-foot interaction behavior is learned with a few hundred touches. We also demonstrate a novel deep Gaussian Process meta-learning technique based on maximizing the deployment gap during meta-training, which enables a robot to use vision and just a few online manipulation attempts to achieve high-quality scooping actions on out-of-distribution granular terrains. This model was transferred in zero-shot fashion to the JPL OWLAT testbed, and outperformed all other models tested.

		Meta-learning few-shot scooping Autonomous lander missions on extraterrestrial bodies will need to sample granular material while coping with domain shift, no matter how well a sampling strategy is tuned on Earth. This paper proposes an adaptive scooping strategy that uses deep Gaussian process method trained with meta-learning to learn on-line from very limited experience on the target terrains. It introduces a novel meta-training approach, Deep Meta-Learning with Controlled Deployment Gaps (CoDeGa), that explicitly trains the deep kernel to predict scooping volume robustly under large domain shifts. Employed in a Bayesian Optimization sequential decision-making framework, the proposed method allows the robot to use vision and very little on-line experience to achieve high-quality scooping actions on out-of-distribution terrains, significantly outperforming non-adaptive methods proposed in the excavation literature as well as other state-of-the-art meta-learning methods. Moreover, a dataset of 6,700 executed scoops collected on a diverse set of materials, terrain topography, and compositions is made available for future research in granular material manipulation and meta-learning.
		Automated Heart and Lung Auscultation in Robotic Physical Examinations This paper presents the first implementation of autonomous robotic auscultation of heart and lung sounds. To select auscultation locations that generate high-quality sounds, a Bayesian Optimization (BO) formulation leverages visual anatomical cues to predict where high-quality sounds might be located, while using auditory feedback to adapt to patient-specific anatomical qualities. Sound quality is estimated online using machine learning models trained on a database of heart and lung stethoscope recordings. Experiments on 4 human subjects show that our system autonomously captures heart and lung sounds of similar quality compared to tele-operation by a human trained in clinical auscultation. Surprisingly, one of the subjects exhibited a previously unknown cardiac pathology that was first identified using our robot, which demonstrates the potential utility of autonomous robotic auscultation for health screening.
		Semi-Empirical Simulation of Learned Force Response Models for Heterogeneous Elastic Objects This paper presents a semi-empirical method for simulating contact with elastically deformable objects whose force response is learned using entirely data-driven models. A point-based surface representation and an inhomogeneous, nonlinear force response model are learned from a robotic arm acquiring force-displacement curves from a small number of poking interactions. The simulator then estimates displacement and force response when the deformable object is in contact with an arbitrary rigid object. It does so by estimating displacements by solving a Hertzian contact model, and sums the expected forces at individual surface points through querying the learned point stiffness models as a function of their expected displacements. Experiments on a variety of challenging objects show that our approach learns force response with sufficient accuracy to generate plausible contact response for novel rigid objects. Summary video ICRA 10-minute talk video
		Contact-Implicit Trajectory Optimization with Learned Deformable Contacts Using Bilevel Optimization Based on the previous project shown below, we present a bilevel, contact-implicit trajectory optimization (TO) formulation that searches for robot trajectories with learned soft contact models. On the lower-level, contact forces are solved via a quadratic program (QP) with the maximum dissipation principle (MDP), based on which the dynamics constraints are formulated in the upper-level TO problem that uses direct transcription. Our method uses a contact model for granular media that is learned from physical experiments, but is general to any contact model that is stick-slip, convex, and smooth. We employ a primal interior-point method with a pre-specified duality gap to solve the lower-level problem, which provides robust gradient information to the upper-level problem. We evaluate our method by optimizing locomotion trajectories of a quadruped robot on various granular terrains offline, and show that we can obtain long-horizon walking gaits of high qualities. Supplemental video
		A Data-driven Approach for Fast Simulation of Robot Locomotion on Granular Media In this paper, we propose a semi-empirical approach for simulating robot locomotion on granular media. We first develop a contact model based on the stick-slip behavior between rigid objects and granular grains, which is then learned through running extensive experiments. The contact model represents all possible contact wrenches that the granular substrate can provide as a convex volume, which our method formulates as constraints in an optimization-based contact force solver. During simulation, granular substrates are treated as rigid objects that allow penetration and the contact solver solves for wrenches that maximize frictional dissipation. We show that our method is able to simulate plausible interaction response with several granular media at interactive rates. ICRA video

NSF NRI Grant #1527826
NSF NRI Grant #1911087
NSF NRI Grant #2025782
NSF NRI Grant #1830366