Algorithm audits are powerful tools for studying black-box systems
without direct knowledge of those systems’ inner workings. While they
have been effectively deployed to identify harms and biases in
algorithmic content, algorithm audits’ narrow focus on technical
components stop short of considering users themselves as integral and
dynamic parts of the system, to be audited alongside its algorithmic
components.
After an overview of the state of the art in algorithm auditing, this talk will introduce sociotechnical auditing:
evaluating algorithmic systems at the sociotechnical level, focusing on
the interplay between algorithms and users as each impacts the other
over time. Metaxa will also present Intervenr, a platform developed to
conduct browser-based, longitudinal sociotechnical audits, and a case
study in which we deployed Intervenr to investigate the central claim of
online targeted advertising systems: that targeted ads perform better
with users. Finally, the talk will conclude by discussing some of the
group’s current work, expanding the auditing method to novices and
youth, and developing post-auditing tools for collective action.
Bio:
Danaë Metaxa is the Raj and Neera Singh Term Assistant Professor of
Computer and Information Science at the University of Pennsylvania, with
a secondary appointment in the Annenberg School for Communication.
Metaxa specializes in issues of bias and representation in algorithmic
systems and content, across high-stakes social settings including
politics, employment, and education. Most recently, their work has
focused on online ad targeting, including its effectiveness and its
impact on marginalized populations. Their work has been published and
awarded in top-tier publication venues including the Association for
Computing Machinery’s (ACM) Conference on Human Factors in Computing
Systems (CHI), and the ACM Conference on Computer-Supported Cooperative
Work (CSCW). They have also worked in an advisory role with legal,
industry, and policy groups including the NYC Commission on Human
Rights, Halt AI, and PolicyLink. Prior to Penn, Metaxa received their
Ph.D. from Stanford University in computer science.