Enabling human-machine coexistence using depth sensors

Thumbnail Image
Authors
Jivani, Devavrat Ganesh
Issue Date
2020-12
Type
Electronic thesis
Thesis
Language
ENG
Keywords
Electrical engineering
Research Projects
Organizational Units
Journal Issue
Alternative Title
Abstract
In this thesis, we investigate the application of sensing technology to scenarios including large scale immersive environments, industrial robot workspaces, and domestic environments. We design frameworks that tackle the individual requirements and challenges of each of these application areas. First, we look at expansive immersive environments, where the addition of multiple 3D sensors enables occupant awareness and gesture based interactions through the application of image filtering techniques and shape approximations. This not only allows the environment to react to the presence of its users, but also lets the users interact with its vast screens through pointing and dragging. We evaluate our system in an experimental immersive space. Second, we look at industrial robot workcells, where through a combination of well-placed 3D sensors, joint configuration information of the robot, highly accurate models of its links, and 3D shape primitives we develop a real-time safety solution. Through simulation and a physical testbed, we demonstrate that this system allows human workers to safely cohabit and collaborate with large, powerful industrial robots. Third, we employ robot-mounted sensors and image-based visual servoing on augmented reality markers to enable the flexible assembly of large structures in an industrial manufacturing scenario. We evaluate our approach in a physical testbed and establish that we can exceed metrics achieved by a fully manual process. Lastly, we integrate room and robot mounted sensors with an assistive robot through the use of markers to enable an activity of daily living for quadriplegic individuals, introducing the possibility of restoring some independence in their lives.
Description
December 2020
School of Engineering
Full Citation
Publisher
Rensselaer Polytechnic Institute, Troy, NY
Terms of Use
Journal
Volume
Issue
PubMed ID
DOI
ISSN
EISSN