Three essays on ethical issues in natural language use for the design and implementation of artificial intelligence (ai) systems

Abstract

This dissertation consists of three essays on ethical issues in natural language use for the design and implementation of artificial intelligence (AI) systems. Several ethical concerns regarding the use of AI to support human decision-making have emerged. The three studies highlight concerns related to confirmation bias and algorithmic fairness in both human and AI systems. Algorithmic fairness refers to efforts to ensure that AI systems are designed and deployed in a manner that does not discriminate against a select group of people, specifically, underrepresented group members. Confirmation bias is characterized by making decisions based on existing beliefs. The dissertation examines these themes in the context of hiring and news veracity recommendations. In the first two essays of this dissertation, solutions to address algorithmic discrimination are proposed. In the third essay, strategies are proposed to impact fairness judgments about algorithms in the context of news veracity assessments to improve the acceptance of algorithmic recommendations. Essay 1. Human resources (HR) platforms using advanced information technology (IT) and artificial intelligence (AI) tools to assist with HR tasks are rapidly being adopted. Considering these platforms have the potential to remove human bias (e.g., confirmation bias) from personnel selection by standardizing the evaluation process, as well as the potential to codify discrimination in seemingly objective algorithms, there is a need to better understand their impact on diversity, equity, and inclusion (DEI). To this end, we use a real-world dataset of 2,506 applicants’ interview responses on an HR platform to examine how applicants’ job-irrelevant dialect use (i.e., African American English; AAE) affects selection decisions. We find that (1) greater use of AAE reduces an applicant’s chances of getting hired, (2) the negative impact of AAE use is stronger in unstructured interview questions, and (3) that this negative effect is strongest among Black applicants. In addition, we find machine learning models are more discriminatory against Black applicants when predicting selection outcomes with unstructured questions, and removing AAE words from machine learning models can enhance their fairness by making them less discriminatory against Black applicants. These findings have important implications for personnel selection processes in organizations seeking to improve social justice and algorithmic fairness. Essay 2. Machine learning (ML) and artificial intelligence (AI) are increasingly playing a role in personnel assessment and selection. However, the use of ML and AI to support human resources (HR) tasks in organizations has a short history rife with significant challenges. For example, algorithms may reinforce existing inequalities in human processes - which may be a result of confirmation bias - when the data used to train them reflects such inequalities. To contribute insights to address this challenge, we propose a loose coupling algorithmic fairness framework that utilizes multiple sources of ground truth labels (i.e., decentralization) thereby decoupling the relationship between predictors and target outcomes (i.e., reducing directness) in ML pipelines. We use a real-world dataset of 2,506 applicants' interview responses to predict candidate selection on a hiring platform to investigate this approach. Models based on our framework estimate the similarity of candidate responses to interview questions to human-validated exemplar answers collected from HR websites, and compared to directly predicting historical hiring decisions, the proposed approach (1) leads to fairer outcomes for underrepresented group members, and (2) is less able to predict candidate race. These results highlight how to enhance the procedural and distributive fairness of ML and AI systems in organizations through human and algorithmic collaboration. Essay 3. Advancements in information and internet technologies have contributed to the pervasiveness of information sharing on platforms such as social media, and blogs, among others. Unfortunately, this has led to the effortless dissemination of false information (or Fake news) on these platforms. While several fact-checking tools and resources have been developed to prevent the spread of Fake News, existing studies have found that these tools are not always effective due to cognitive biases such as confirmation bias. That is, people do not accept the recommendation of these fact-checking tools if the recommendations do not align with their beliefs. To address this challenge, the third study proposes a conceptual framework explaining how the similarity between AI tools and users, users’ perceived fairness of AI tools, and the autonomy AI tools provide users during interaction, are key factors that can improve the acceptance of algorithmic advice in the presence of confirmation bias. The study espouses that similarity will lead to perceived fairness, while perceived fairness will improve the acceptance of algorithmic advice. Additionally, the relationship between the perceived fairness and the decision to accept or reject an algorithm’s recommendation will be moderated by the extent of autonomy provided to the user by the AI tool during the recommendation task. Future work will focus on gathering data to test the arguments advanced in the study.