Contact-based state estimation and policy learning for robotic manipulation tasks

Abstract

In this thesis, we focus on improving the perception capability for robots and addressing the problem of combining the perception capability with action planning and execution for robotic manipulation tasks. Our proposed approach combines Bayesian filtering methods with accurate models of multi-body dynamics for state estimation in the robotic manipulation tasks. In order to understand the design trade-offs of particle filter applications for the state estimation problems, we evalu- ate different particle filter modeling options in both simulation and physical exper- iments. We then propose a contact-based RBPF that samples the discrete contact states and updates the continuous state distribution through Kalman filters. Re- sults show that the contact-based RBPF is more effective and more efficient than the state of the art filters that sample the continuous state space. Finally, we apply reinforcement learning algorithms to learn policies for robotic manipulation tasks with a state space discretized using contact states. This discretized space learning is proven to be more effective than learning with continuous state space. We further propose to combine the learned policies with the contact-based RBPF for online action selection during robotic manipulation tasks.

The robotic manipulation problem is very important in robotic research and appli- cations. In a typical robotic manipulation task, the robot needs to interact with certain objects to accomplish a goal, such as picking up a cup from a table. Without manipulation capabilities, robots will not be able to help humans with their daily tasks such as using tools to fix a broken car. Although the hardware of robots has been significantly improved, the ability to perceive the current state of a robotic manipulation task is still essential for a robot to fully utilize its hardware. However, less effort has been made to address the perception problem for robotic manipulation tasks.