DSpace@RPI
DSpace@RPI is a repository of Rensselaer Polytechnic Institute's theses and dissertations which are available in digital format, largely from 2006 to present, along with other selected resources.
Recent Submissions
Item Towards Computer-Using Personal Agents(Schloss Dagstuhl Leibniz-Zentrum für Informatik GmbH, 2025-01-31)Computer-Using Agents (CUA) enable users to automate increasingly-complex tasks using graphical interfaces such as browsers. As many potential tasks require personal data, we propose Computer-Using Personal Agents (CUPAs) that have access to an external repository of the user's personal data. Compared with CUAs, CUPAs offer users better control of their personal data, the potential to automate more tasks involving personal data, better interoperability with external sources of data, and better capabilities to coordinate with other CUPAs in order to solve collaborative tasks involving the personal data of multiple users. This report is a result of Dagstuhl Seminar 25051 "Trust and Accountability in Knowledge Graph-Based AI for Self Determination", which took place in January 2025Item Reflections From Former EICs: 40 Years of IEEE Intelligent Systems.(IEEE, 2025-02-20)The year 2025 marks the 40th anniversary of IEEE Intelligent Systems, a significant milestone in the magazine’s remarkable evolution. With this issue, we celebrate four decades of the magazine’s insightful contributions to the artificial intelligence (AI) community. Past issues, articles, and summaries are conveniently aggregated at the IEEE Computer Society Digital Library and IEEE Xplore. This anniversary offers a moment for reflection, recollection, and celebration. We invited five former editors-in-chief (EICs)—Daniel Edmund O’Leary, James Hendler, Daniel Zeng, V.S. Subrahmanian, and San Murugesan—to reflect on the magazine’s history, accomplishments, and future and share their memories. Here are their reflections.Item Analytic and numerical investigations of lattice-based statistical mechanical models(Rensselaer Polytechnic Institute, Troy, NY, 2024-12)We study the canonical ensembles of two lattice models which work with vorticity. In a 2D setting vorticity can be treated as a scalar quantity. Certain functions of vorticity, most notably its first moment, are conserved. By choosing a set of conserved quantities appropriate to the problem being studied, and an inverse temperature which allows one to specify whether a high energy or low energy regime is of interest, one can construct a statistical ensemble. An ensemble encodes certain long-term behaviors of the system, but does not require solving the underlying differential equations which govern the dynamics. The first system is studied is based off the Helmholtz-Onsager point vortex gas, and studies the low positive temperature/low energy regime in a multiply-connected domain. In this regime vorticity particles have high probability to be near the domain walls, as the system energy has a self-interaction term for each vortex which is negative in this neighborhood. This behavior was observed in the canonical ensemble. Due to the simplicity of the point vortex dynamics, the results were supplemented with a microcanonical analysis based on simulating the system and analytically solving a mean field equation which gives its long-term density average. The second system studied is the Kac-Berlin spherical model, which conserves the second moment of the site strengths. The low negative temperature/high energy regime of the spherical model has been used to depict wave systems which undergo inverse energy cascade. We approach the model with new analytical techniques and find evidence for a phase transition, as well as an equation for the expectation of energy after the phase transition. These predictions are then examined and verified with numerical simulations on several lattices which encode a particular instance of the spherical model.Item Geometry-based and physics-informed 3d face & eye reconstruction for facial behavior analysis(Rensselaer Polytechnic Institute, Troy, NY, 2024-12)Facial behavior analysis and recognition plays an important role in human-centered AI, boosting the technology development in the areas of human emotion recognition, attention detection and autonomous driving. This research performs 3D facial analysis, focusing on accurate 3D face and eye reconstruction, facial action recognition, and eye gaze estimation.The developments of deep learning models, combined with large benchmark datasets and representative 3D facial models, greatly improved the accuracy of 3D face and eye reconstruction. Despite this progress, existing methods in both areas still suffer from several significant limitations: a) lack of detailed shape modeling for accurately recovering subtle 3D facial motions and eyeball movement; b) over-dependence on a large amount of training data and labels; c) poor generalization across subjects and under different illuminations, distances and large head poses; and d) failure to effectively exploit physically plausible facial dynamics in videos. We introduce methods to address these limitations. For accurate 3D face reconstruction, we combine 3D facial models with Facial Action Unit (FAU) encoding system, where each AU represents a specific local facial motion driven by specific muscle activation.We first present a personalized 3D FAU blendshape learning framework together with a 3D face reconstruction model for recovering AU-interpretable 3D facial details. We also innovatively incorporate general knowledge of AU correlations into the learning process to reduce the amount of expression labels used in training. Our method not only produces a more personalized and detailed 3D face model but also yields improved facial action recognition. For 3D eye reconstruction, we create a deformable eye shape basis for representing detailed 3D eye structure. Different from existing approaches, we incorporate the 3D eye shape basis into a learning-based eye gaze estimation framework, inducing a geometry-based weak supervision in training the deep model. Our model is superior to others in terms of recovering 3D eye shape, eye rotation and gaze simultaneously from an image and is less dependent on full training labels while still maintaining the gaze accuracy. To further address the generalization and to exploit the facial dynamics for both facial actions and eye movement, we propose dynamic 3D face action and eye gaze tracking methods from monocular videos. The intuitive idea is based on the facial anatomy that all the facial motion components are activated by certain muscle contractions, so the reconstructed 3D motion should match with the physical laws of motion (Newton’s second law). Different from our frame-based method, we design different physically plausible models for facial action units and eyeball movement. For facial action units, we design a physics-informed model by constraining the reconstructed sequence to satisfy the underlying physics laws. For dynamic gaze tracking, we propose a physics-informed gaze tracking system by subjecting the eyeball movements to certain physical constrains and biomechanical laws. Furthermore, we propose to leverage human interactions and hand-eye coordination to reduce 3D eye gaze annotation using weakly supervised eye gaze tracking models. Our methods are evaluated against state-of-the-art methods both quantitatively and qualitatively, including 3D face reconstruction accuracy, facial action unit detection accuracy, and gaze estimation accuracy, both within and across datasets.Item The logic of bias: using cognitive architecture to explore interactions between cognitive abilities and decision errors(Rensselaer Polytechnic Institute, Troy, NY, 2024-12)Viewed as cognitive imperfections, biases have been thought to be responsible for hinderinghumans from fully making use of their reasoning abilities. Several recent strains of research, however, have begun to emphasize the positive aspect of biases. For example, PSI architec- ture (Dörner & Güss, 2013) views these as engineered by evolution to prevent dissatisfaction, and promote subsequent satisfaction of human needs. Crucially, PSI assumes higher skills and reasoning capacities to enable a higher degree of introspection and cognitive flexibility, thus alleviating the effects of biases. Recent work by Kahan et al. (2017) called this general assumption into question: sub- jects with higher numeracy skills were not better protected from a polarized interpretation of statistical data, if the data contradicted their political beliefs — instead, the effect of bias was increased. It is unclear, however, how to situate these finding within the PSI framework, as they could be attributed to being A. a general cognitive fallacy caused to a large extent by modulations of perceptional and attentional processes not specific to group integrity; B. rooted in the long-term forming of stable, habituated action patterns, associated with the subject’s beliefs; or C. an effect indeed specific to groups with strong affiliative connections. Each of the accounts above would warrant varying revisions to the architecture. To test the account A above, I conducted a controlled experiment examining existential needs using thirst as a general, negative stimulus unrelated prior beliefs/experiences. The initial results indicated that the effect reported by Kahan et al. (2017) could not be repro- duced. This suggests that account A. — the bias being a general cognitive fallacy - does not fit. However, the average age of the sample was quite young, as compared to Kahan’s, and left the the possibility for account B. — the bias being rooted in long-term stable, habituated response patterns associated with beliefs, which did not have the time to form within the undergraduates. viii Testing account B required disentangling long-term beliefs from group needs. For this, sleep/wake patterns seemed to be a good fit, as they are not strongly related to affiliation. I conducted a study analogous to our first, where the data and background story in the statistical task either affirmed or contradicted the subject’s sleep patterns. To control for confounds, I asked whether these are shared by their family, peers, and/or any other larger group they belong to. Further, the subjects sleep/wake cycles must not be determined to a large degree by external factors. As no direct manipulation of independent variables was required for this study, I relegated subject recruitment and data collection was relegated to the online platform Prolific (Palan & Schitter, 2018). As here, too, Kahan et al.’s (2017) was not observed, it The contributions of this work are the integration of the expert bias phenomenon into the PSI architecture by: 1. Conducting behavioral studies to explore the underpinning dynamics responsible for the production of this effect. 2. Interpreting the findings and transcribing them onto the logic of the architecture. 3. Drafting the consequent steps necessary to initialize a revision of PSI to fit the ex- periments
Communities in DSpace@RPI
Select a community to browse its collections.