Enabling human-machine coexistence using depth sensors

Jivani, Devavrat Ganesh
Thumbnail Image
Other Contributors
Radke, Richard J., 1974-
Wen, John T.
Julius, Anak Agung
Fajen, Brett R.
Issue Date
Electrical engineering
Terms of Use
This electronic version is a licensed copy owned by Rensselaer Polytechnic Institute, Troy, NY. Copyright of original work retained by author.
Full Citation
In this thesis, we investigate the application of sensing technology to scenarios including large scale immersive environments, industrial robot workspaces, and domestic environments. We design frameworks that tackle the individual requirements and challenges of each of these application areas. First, we look at expansive immersive environments, where the addition of multiple 3D sensors enables occupant awareness and gesture based interactions through the application of image filtering techniques and shape approximations. This not only allows the environment to react to the presence of its users, but also lets the users interact with its vast screens through pointing and dragging. We evaluate our system in an experimental immersive space. Second, we look at industrial robot workcells, where through a combination of well-placed 3D sensors, joint configuration information of the robot, highly accurate models of its links, and 3D shape primitives we develop a real-time safety solution. Through simulation and a physical testbed, we demonstrate that this system allows human workers to safely cohabit and collaborate with large, powerful industrial robots. Third, we employ robot-mounted sensors and image-based visual servoing on augmented reality markers to enable the flexible assembly of large structures in an industrial manufacturing scenario. We evaluate our approach in a physical testbed and establish that we can exceed metrics achieved by a fully manual process. Lastly, we integrate room and robot mounted sensors with an assistive robot through the use of markers to enable an activity of daily living for quadriplegic individuals, introducing the possibility of restoring some independence in their lives.
December 2020
School of Engineering
Dept. of Electrical, Computer, and Systems Engineering
Rensselaer Polytechnic Institute, Troy, NY
Rensselaer Theses and Dissertations Online Collection
Restricted to current Rensselaer faculty, staff and students. Access inquiries may be directed to the Rensselaer Libraries.