Schedule

Suggested Readings:

• Lessons from measurement [only need to read measurement section]
• When You Hear the Margin of Error Is Plus or Minus 3 Percent, Think 7 Instead

Suggested Readings:

• Pew - comparing weighting techniques
• High-frequency polling with non-representative data
• We Gave Four Good Pollsters the Same Raw Data. They Had Four Different Results.
• How One 19-Year-Old Illinois Man Is Distorting National Polling Averages

Optional Readings:

• Designing Informative Rating Systems: Evidence from an Online Labor Market
• Reputation Inflation

Suggested Readings:

• Mining Massive Datasets, Ch 9, Recommendations
• Stitchfix Algorithms tour

Suggested Readings:

• The Engagement-Diversity Connection: Evidence from a Field Experiment on Spotify
• Algorithmic Effects on the Diversity of Consumption on Spotify

Title: High-stakes decisions with low-quality data: Learning and planning under uncertainty for conservation

Abstract: Wildlife poaching pushes countless species to the brink of extinction, with animal population sizes declining by an average of 70% since 1970. To aid rangers in preventing poaching in protected areas around the world, we have developed the Protection Assistant for Wildlife Security (PAWS). We present technical advances in multi-armed bandits, robust reinforcement learning, and causal inference, guided by research questions that emerged from on-the-ground challenges in deploying PAWS. We also discuss bridging the gap between research and practice, from field tests in Uganda and Cambodia to large-scale deployment through integration with SMART, the leading software system for protected area management used by over 1,000 wildlife parks worldwide.

Bio: Lily Xu is a computer science PhD student at Harvard developing AI techniques to address environmental planning challenges. She focuses on advancing methods in machine learning, sequential decision-making, and game theory for biodiversity conservation through preventing wildlife poaching. Her work building the PAWS system to predict poaching hotspots has been deployed across Africa, southeast Asia, and Latin America and is being scaled globally through integration with SMART conservation software. Lily co-organizes the Mechanism Design for Social Good (MD4SG) research initiative and serves as AI Lead for the SMART Partnership. Her research has been recognized with best paper runner-up at AAAI, the INFORMS Doing Good with Good OR award, and a Google PhD Fellowship.

Suggested Readings:

• The Algorithm behind Plane Ticket Prices and How to Get the Best Deal
• Book chapter on Dynamic Programming

Notes:

• Handwritten notes backward induction 1
• Handwritten notes backward induction 2

Suggested Readings:

• Fairness as a Constraint on Profit Seeking: Entitlements in the Market

Suggested Readings:

• Controlled experiments on the web: survey and practical guide
• A/B testing: A step-by-step guide in Python
• The Power of Experiments: Decision Making in a Data-Driven World

VIRTUAL GUEST LECTURE

Talk title: Algorithmic allocation of humanitarian aid with machine learning and digital data

Abstract: The vast majority of humanitarian aid and social protection programs globally are targeted, providing assistance to individuals or communities identified to be poorest or most in need. In low and middle-income countries, the targeting of aid programs is often limited by low-quality, out-of-date, or missing data on poverty and vulnerability. Novel “big” digital data sources, such as those captured by satellites, mobile phones, and financial services providers – when combined with advances in machine learning – can improve the accuracy of aid program targeting. In this talk, I will cover empirical results on the accuracy of these new data-driven and algorithmic approaches to aid allocation, as well as the potential and limitations of additional use cases in impact evaluation and poverty measurement.

Speaker bio: Emily is a PhD candidate at the UC Berkeley School of Information, where she studies the application of novel algorithms and digital data sources for social protection programs. Her work has been published in venues including Nature and Science Advances, and she is a recipient of a Microsoft Research PhD fellowship. Prior to Berkeley, Emily received her bachelor’s degree in computer science from Harvard.

Suggested Readings:

• Lyft – experimentation in marketplaces (Part 1), (Part 2), (Part 3)
• Toward Causal Inference With Interference

Suggested Readings:

• Switchback Tests and Randomized Experimentation Under Network Effects at DoorDash
• A/B testing at marketplaces

VIRTUAL GUEST LECTURE

Title: Equitable Decision-Making in Public Resource Allocation

Abstract: Algorithmically guided decisions are becoming increasingly prevalent and, if left unchecked, can amplify pre-existing societal biases. In this talk, I audit the equity of decision-making in two public resource allocation settings. First, I present a methodological framework for online advertisers to determine a demographically equitable allocation of individuals being shown ads for SNAP (food stamp) benefits – specifically, considering budget-constrained trade-offs between ad conversions for English-speaking and Spanish-speaking SNAP applicants. Second, I discuss sensitivity analyses on public funding allocation algorithms such as CalEnviroScreen, an algorithm used to promote environmental justice by aiding disadvantaged census tracts – which we find to encode bias against tracts with high immigrant populations. In both case studies, we will discuss methods to mitigate allocative harm and to foster equitable outcomes using accountability mechanisms.

Paper links: SNAP, CalEnviroScreen

Bio: Allison Koenecke is an Assistant Professor of Information Science at Cornell University. Her research applies computational methods, such as machine learning and causal inference, to study societal inequities in domains from online services to public health. Koenecke is regularly quoted as an expert on disparities in automated speech-to-text systems. She previously held a postdoctoral researcher role at Microsoft Research and received her PhD from Stanford’s Institute for Computational and Mathematical Engineering.

Optional Readings:

• Quantifying Spatial Under-reporting Disparities in Resident Crowdsourcing
• A Bayesian Spatial Model to Correct Under-Reporting in Urban Crowdsourcing

Title: Data Science for Social Equality

Abstract: Our society remains profoundly unequal. This talk discusses how data science and machine learning can be used to facilitate more equitable decision-making by presenting case studies around policing and COVID-19.

Bio: Emma Pierson is an assistant professor of computer science at the Jacobs Technion-Cornell Institute at Cornell Tech and the Technion, and a computer science field member at Cornell University. She holds a secondary joint appointment as an Assistant Professor of Population Health Sciences at Weill Cornell Medical College. She develops data science and machine learning methods to study inequality and healthcare. Her work has been recognized by best paper, poster, and talk awards, an NSF CAREER award, a Rhodes Scholarship, Hertz Fellowship, Rising Star in EECS, MIT Technology Review 35 Innovators Under 35, and Forbes 30 Under 30 in Science. Her research has been published at venues including ICML, KDD, WWW, Nature, and Nature Medicine, and she has also written for The New York Times, FiveThirtyEight, Wired, and various other publications.

Suggested Readings:

• Algorithmic Foundations of Differential Privacy
• How to perform these attacks Part 2
• Simple Demographics Often Identify People Uniquely
• Differential privacy: A survey of results

Suggested Readings:

• Limits to Prediction class, Princeton, (Pre-class summary reading)
• Measuring the predictability of life outcomes with a scientific mass collaboration
• Manipulation of Social Program Eligibility