Lectures

You can download the lectures here. We will try to upload lectures prior to their corresponding classes.

• 

  Lecture 1 - Course Introduction
  tl;dr: Course introduction.
  [slides]
  
• 

  Lecture 2 - Data challenges
  tl;dr: Common challenges in data collection.
  [slides]
  

  Suggested Readings:

  • Lessons from measurement [only need to read measurement section]
  • When You Hear the Margin of Error Is Plus or Minus 3 Percent, Think 7 Instead
• 

  Lecture 3 - Survey weighting
  tl;dr: Weighting techniques to counter selection biases.
  [slides]
  

  Suggested Readings:

  • Pew - comparing weighting techniques
  • High-frequency polling with non-representative data
  • We Gave Four Good Pollsters the Same Raw Data. They Had Four Different Results.
  • How One 19-Year-Old Illinois Man Is Distorting National Polling Averages
• 

  Lecture 4 - Weighting 2
  tl;dr: More on weighting, Uncertainty quantification with selection on unknown covariates
  [slides]
  
• 

  Lecture 5 - Other aspects of data collection
  tl;dr: Other topics in data collection
  [slides]
  

  Optional Readings:

  • Designing Informative Rating Systems: Evidence from an Online Labor Market
  • Reputation Inflation
• 

  Lecture 6 - Recommendations introduction
  tl;dr: Introduction to Recommendations, including data challenges and collaborative filtering
  [slides]
  

  Suggested Readings:

  • Mining Massive Datasets, Ch 9, Recommendations
  • Stitchfix Algorithms tour
• 

  Lecture 7 - Recommendations, from predictions to decisions
  tl;dr: From predicting ratings to making decisions: capacity constraints and multi-sided recommendations
  [slides]
  

  Suggested Readings:

  • The Engagement-Diversity Connection: Evidence from a Field Experiment on Spotify
  • Algorithmic Effects on the Diversity of Consumption on Spotify
• 

  Lecture 8 - Algorithmic Pricing basics
  tl;dr: Introduction to algorithmic Pricing
  [slides]
  
• 

  Guest - Lily Xu Guest Lecture
  tl;dr: Lily Xu (Harvard)
  
  

  Title: High-stakes decisions with low-quality data: Learning and planning under uncertainty for conservation

  Abstract: Wildlife poaching pushes countless species to the brink of extinction, with animal population sizes declining by an average of 70% since 1970. To aid rangers in preventing poaching in protected areas around the world, we have developed the Protection Assistant for Wildlife Security (PAWS). We present technical advances in multi-armed bandits, robust reinforcement learning, and causal inference, guided by research questions that emerged from on-the-ground challenges in deploying PAWS. We also discuss bridging the gap between research and practice, from field tests in Uganda and Cambodia to large-scale deployment through integration with SMART, the leading software system for protected area management used by over 1,000 wildlife parks worldwide.

  Bio: Lily Xu is a computer science PhD student at Harvard developing AI techniques to address environmental planning challenges. She focuses on advancing methods in machine learning, sequential decision-making, and game theory for biodiversity conservation through preventing wildlife poaching. Her work building the PAWS system to predict poaching hotspots has been deployed across Africa, southeast Asia, and Latin America and is being scaled globally through integration with SMART conservation software. Lily co-organizes the Mechanism Design for Social Good (MD4SG) research initiative and serves as AI Lead for the SMART Partnership. Her research has been recognized with best paper runner-up at AAAI, the INFORMS Doing Good with Good OR award, and a Google PhD Fellowship.

• 

  Lecture 9 - Algorithmic Pricing complications
  tl;dr: Algorithmic pricing: capacity, price differentiation, and competition
  [slides]
  

  Suggested Readings:

  • The Algorithm behind Plane Ticket Prices and How to Get the Best Deal
  • Book chapter on Dynamic Programming

  Notes:

  • Handwritten notes backward induction 1
  • Handwritten notes backward induction 2
• 

  Lecture 10 - Algorithmic Pricing complications 2
  tl;dr: Algorithmic pricing: capacity, price differentiation, and competition
  [slides]
  
• 

  Lecture 11 - Algorithmic Pricing practice -- ride-hailing
  tl;dr: Algorithmic pricing: ride-hailing case study
  [slides]
  
• 

  In class discussion - Ethics in (Algorithmic) Pricing
  tl;dr: What makes a pricing decision unfair?
  [Pre-class Homework]
  

  Suggested Readings:

  • Fairness as a Constraint on Profit Seeking: Entitlements in the Market
• NO CLASS -- FALL BREAK
  tl;dr:
  
• 

  Lecture 12 - Experimentation -- Introduction
  tl;dr: Introduction to experimentation
  [slides]
  

  Suggested Readings:

  • Controlled experiments on the web: survey and practical guide
  • A/B testing: A step-by-step guide in Python
  • The Power of Experiments: Decision Making in a Data-Driven World
• 

  Guest Lecture -- Emily Aiken
  tl;dr: Emily Aiken (UC Berkeley).
  
  

  VIRTUAL GUEST LECTURE

  Talk title: Algorithmic allocation of humanitarian aid with machine learning and digital data

  Abstract: The vast majority of humanitarian aid and social protection programs globally are targeted, providing assistance to individuals or communities identified to be poorest or most in need. In low and middle-income countries, the targeting of aid programs is often limited by low-quality, out-of-date, or missing data on poverty and vulnerability. Novel “big” digital data sources, such as those captured by satellites, mobile phones, and financial services providers – when combined with advances in machine learning – can improve the accuracy of aid program targeting. In this talk, I will cover empirical results on the accuracy of these new data-driven and algorithmic approaches to aid allocation, as well as the potential and limitations of additional use cases in impact evaluation and poverty measurement.

  Speaker bio: Emily is a PhD candidate at the UC Berkeley School of Information, where she studies the application of novel algorithms and digital data sources for social protection programs. Her work has been published in venues including Nature and Science Advances, and she is a recipient of a Microsoft Research PhD fellowship. Prior to Berkeley, Emily received her bachelor’s degree in computer science from Harvard.

• CANCELLED CLASS
  tl;dr:
  
• 

  Lecture 13 - Experimentation -- Peeking and Interference
  tl;dr: Experimentation challenges in online marketplaces: peeking and interference
  [slides]
  

  Suggested Readings:

  • Lyft – experimentation in marketplaces (Part 1), (Part 2), (Part 3)
  • Toward Causal Inference With Interference
• 

  Lecture 14 - Experimentation in marketplaces
  tl;dr: Experimentation in marketplaces: interference, spatial randomization, and switchbacks
  [slides]
  

  Suggested Readings:

  • Switchback Tests and Randomized Experimentation Under Network Effects at DoorDash
  • A/B testing at marketplaces
• 

  Guest Lecture -- Allison Koenecke
  tl;dr: Allison Koenecke (Cornell).
  
  

  VIRTUAL GUEST LECTURE

  Title: Equitable Decision-Making in Public Resource Allocation

  Abstract: Algorithmically guided decisions are becoming increasingly prevalent and, if left unchecked, can amplify pre-existing societal biases. In this talk, I audit the equity of decision-making in two public resource allocation settings. First, I present a methodological framework for online advertisers to determine a demographically equitable allocation of individuals being shown ads for SNAP (food stamp) benefits – specifically, considering budget-constrained trade-offs between ad conversions for English-speaking and Spanish-speaking SNAP applicants. Second, I discuss sensitivity analyses on public funding allocation algorithms such as CalEnviroScreen, an algorithm used to promote environmental justice by aiding disadvantaged census tracts – which we find to encode bias against tracts with high immigrant populations. In both case studies, we will discuss methods to mitigate allocative harm and to foster equitable outcomes using accountability mechanisms.

  Paper links: SNAP, CalEnviroScreen

  Bio: Allison Koenecke is an Assistant Professor of Information Science at Cornell University. Her research applies computational methods, such as machine learning and causal inference, to study societal inequities in domains from online services to public health. Koenecke is regularly quoted as an expert on disparities in automated speech-to-text systems. She previously held a postdoctoral researcher role at Microsoft Research and received her PhD from Stanford’s Institute for Computational and Mathematical Engineering.

• 

  Special Lecture (by Gabriel and Zhi) -- Heterogeneous reporting in NYC311 systems
  tl;dr: Zhi and Gabriel on research regarding heterogeneous reporting in 311 systems
  
  

  Optional Readings:

  • Quantifying Spatial Under-reporting Disparities in Resident Crowdsourcing
  • A Bayesian Spatial Model to Correct Under-Reporting in Urban Crowdsourcing
• 

  Guest - Emma Pierson Guest Lecture
  tl;dr: Emma Pierson (Cornell Tech).
  
  

  Title: Data Science for Social Equality

  Abstract: Our society remains profoundly unequal. This talk discusses how data science and machine learning can be used to facilitate more equitable decision-making by presenting case studies around policing and COVID-19.

  Bio: Emma Pierson is an assistant professor of computer science at the Jacobs Technion-Cornell Institute at Cornell Tech and the Technion, and a computer science field member at Cornell University. She holds a secondary joint appointment as an Assistant Professor of Population Health Sciences at Weill Cornell Medical College. She develops data science and machine learning methods to study inequality and healthcare. Her work has been recognized by best paper, poster, and talk awards, an NSF CAREER award, a Rhodes Scholarship, Hertz Fellowship, Rising Star in EECS, MIT Technology Review 35 Innovators Under 35, and Forbes 30 Under 30 in Science. Her research has been published at venues including ICML, KDD, WWW, Nature, and Nature Medicine, and she has also written for The New York Times, FiveThirtyEight, Wired, and various other publications.

• Guest - Smitha Milli Guest Lecture
  tl;dr: Smitha Milli (Cornell Tech).
  
  
• 

  Lecture 15 - Project description and game theory
  tl;dr: How to succeed in class project
  [slides]
  
• 

  Lecture 16 - Differential Privacy
  tl;dr: Introduction to differential privacy
  [slides]
  

  Suggested Readings:

  • Algorithmic Foundations of Differential Privacy
  • How to perform these attacks Part 2
  • Simple Demographics Often Identify People Uniquely
  • Differential privacy: A survey of results
• 

  Lecture 17 - Limits to prediction
  tl;dr: Prediction is hard, especially about the future
  [slides]
  

  Suggested Readings:

  • Limits to Prediction class, Princeton, (Pre-class summary reading)
  • Measuring the predictability of life outcomes with a scientific mass collaboration
  • Manipulation of Social Program Eligibility
• 

  Lecture 18 - Discrimination in Platforms
  tl;dr: Discrimination in ad platforms and two-sided marketplaces; some design mitigations
  [slides]
  
• Guest - Serina Chang Guest Lecture
  tl;dr: Serina Chang (Stanford)