Physics-empowered perception for robot grasping and dexterous manipulation

Abstract

To address such weaknesses, we define the C-SLAM problem to set requirements on perception algorithms for the ultimate goal of human-like Bayesian grasping and manipulation. It requires simultaneous object localization and modeling of its physical properties during manipulating the object. In this thesis, we propose two estimation frameworks to attack the C-SLAM problem. Both frameworks explicitly adopt a dynamic system transition model to characterize real-time interactions between the robot end effector and the object at the center of respective stochastic system transition models. Their main difference is that one framework adopts a computationally complex full-blown dynamic formulation while the other significantly reduces the computational cost by decomposing it into sub-models of contact mode prediction and state propagation, which are originally tightly coupled in physics. Constraints are hence relaxed at certain sampled states. Both frameworks are proven effective in capturing dynamic behaviors from applications to various grasping scenarios and the relaxation model gives better computation performance.

In recent years, robot technology has been greatly advanced by sophisticated perception algorithms which usually combine the advantages of a predetermined system model and up-to-date observations from physical sensors. However, the robot's grasping and manipulation capabilities fall far behind its mobility counterpart. In unstructured environments, state-of-the-art robots grasp only with separate palm and finger motions, which are hence slower than human-like grasp, to overcome unavoidable uncertainties from locating objects and positioning end effectors. Still, well-designed grasps might fail due to inadvertent bumping between the robot hand and the object which causes the object to tumble away from the original planned grasp trajectory. The inability to promptly detect such tumbling movements, which will also be induced during dynamic grasping acquisition, is a primary reason behind the poor state of the art. Lacking knowledge of the object's properties also prevents effective reactive grasping strategy as many properties exhibit heavy effects on the dynamic interactions and hence the grasping actions.