Understanding the limitations of AI: When Algorithms Fail - Databricks

Understanding the limitations of AI: When Algorithms Fail

“Automated decision making tools are currently used in high stakes scenarios. From natural language processing tools used to automatically determine one’s suitability for a job, to health diagnostic systems trained to determine a patient’s outcome, machine learning models are used to make decisions that can have serious consequences on people’s lives. In spite of the consequential nature of these use cases, vendors of such models are not required to perform specific tests showing the suitability of their models for a given task. Nor are they required to provide documentation describing the characteristics of their models, or disclose the results of algorithmic audits to ensure that certain groups are not unfairly treated. I will show some examples to examine the dire consequences of basing decisions entirely on machine learning based systems, and discuss recent work on auditing and exposing the gender and skin tone bias found in commercial gender classification systems. I will end with the concept of an AI datasheet to standardize information for datasets and pre-trained models, in order to push the field as a whole towards transparency and accountability.”



« back
About Timnit Gebru

Timnit recently finished her postdoc at Microsoft Research in New York City, in the FATE (Fairness Transparency Accountability and Ethics in AI) group, where she studied algorithmic bias and the ethical implications underlying data mining projects (see this New York Times article for an example of her work). Timnit received her PhD from the Stanford Artificial Intelligence Laboratory, studying computer vision under Fei-Fei Li. Her thesis pertained to data mining large-scale, publicly available images to gain sociological insights, and computer vision problems that arise as a result. The Economist, The New York Times, and others have covered this work. Some of the computer vision areas she is interested in include fine-grained image recognition, scalable annotation of images, and domain adaptation. Prior to joining Fei-Fei’s lab, Timnit worked at Apple, designing circuits and signal-processing algorithms for various products, including the first iPad. Her research was supported by the NSF foundation GRFP fellowship and the Stanford DARE fellowship.