Visual and gesture guided robotic block stacking

Abstract

This thesis focuses on the development of a system that could achieve the complex manipulation of an arbitrary arrangement of objects. The objects being manipulated are Jenga® blocks. The combination of binary marker tags and computer vision provides the basis for determining each block's pose within the robot's workspace. Through the use of tags and robust pickup strategies, the robot is able to observe the block's location, determine the appropriate response necessary to grasp, and sequence to remove blocks from a pile. The integration of tags, robust pickup strategies, and a specially designed gripper forms the basis of a robust and efficient block localization and manipulation system.

Industrial robots are precise and efficient at performing repetitive tasks. Robots in industry have led to the complete automation of tasks that would normally be considered too hazardous, strenuous, or tedious for humans to perform. However, robots do not have the innate ability to recognize and manipulate objects they rely on human operators to translate the desired task into a set of operations it can perform. Object recognition and manipulation are intuitive for humans, but are still difficult problems for robots. For example, bin-picking has long been a research area in the field of robotics that aims to provide robots with the ability to manipulate randomly ordered objects in unstructured environments.

Bin-picking systems must address three system design challenges: scene interpretation, object recognition, and pose estimation. The difficulty in implementing a bin-picking system lies in the development of a robust system that can recognize objects in a cluttered scene, discern objects that are piled together, and address environmental conditions such as lighting. Current bin-picking systems use arrays of expensive sensors and employ computationally intensive algorithms to allow robotic systems to recognize objects in their workspace. This project seeks to simplify object detection and pose estimation through the use of binary marker tags. These tags provide a set of uniquely identifiable features, allowing for fast and efficient object recognition and localization.