2026 CODE@MIT: The Conference on Digital Experimentation at MIT

Please take the elevators to the 7th floor to check in and get your name badge. Enjoy breakfast and networking with other conference attendees and speakers.

Please join us as we kick off 2026 CODE@MIT with one of the event organizers from the MIT Initiative on the Digital Economy.

Dennis Zhang — Washington University

Dennis Zhang is a Professor of Operations Management and Professor of Marketing at the Olin Business School, Washington University in St. Louis. His research focuses on data-driven business decision making. He uses machine learning, causal inference and structural estimation with data to improve business decisions.

Presentation Abstract:

My research on AI and experimentation asks how we can learn more from limited experimental data and make better decisions. We pursue two directions.

First, we combine AI with economic models to estimate the effects of product changes that have not been tested together and design personalized incentives beyond the options included in an experiment. Second, we use AI-generated responses to supplement costly human surveys and experiments, calibrating those responses against real human data to correct systematic differences.

Across these studies, the goal is to improve measurement while providing reliable uncertainty estimates and statistical guarantees under explicit assumptions. Applications ranging from large-scale platform experiments to consumer preferences and retail pricing show how these methods can expand what we learn, improve decisions, and reduce the amount of human data needed.

Peter Hull — Brown University

Peter Hull is a Professor of Economics at Brown University, where he studies applied econometrics and various topics in education, healthcare, discrimination, and crime. Hull received his Ph.D. in economics from MIT in 2017 and before Brown was an Assistant Professor at the University of Chicago. He is a Research Associate at NBER, an Amazon Scholar, and a Coeditor at the American Economic Journal: Applied Economics.

Presentation Abstract:

We study the optimal design and analysis of experiments for estimating spillover effects. Assuming a known (e.g., linear) exposure mapping, we characterize the treatment-assignment distribution and regression-based estimator that minimize worst-case asymptotic variance against a broad class of distributions of unobservables. The design problem yields an intuitive solution in which the planner trades off spillover signal strength against diffusion of spillover variation. The analysis problem yields a simple recentered instrumental variable estimator to best leverage this variation.

This framework produces natural solutions in several benchmark cases—such as clustered exposure—and suggests computationally tractable approximations for general networks, including bipartite settings. We illustrate these new tools in semi-synthetic experiments based on two applications from development economics. Our approach yields large standard error reductions in both experiments, increasing effective sample sizes by 50–100% or more.

Please enjoy refreshments in the lobby of the Samberg Conference Center 7th Floor.

Session A: Learning Across Populations

Room TBD  |  Chair: TBD

“How Much Should We Measure Before We Replicate? Prospective Context Acquisition for External Validity”

Hossein Alidaee (Harvard)

“When Can Past Experiments Inform the Next One?”

Shima Nassiri (Lyft), Sadegh Shirani (MIT)

“Treatment Effect Modification by Latent User States”

Hui Lan (Netflix), Nathan Kallus (Netflix), Winston Chou (Netflix)

“Sampling for the Target Population: A Constrained Probability Design for Panel Experiments”

Daniel Yurovsky (Netflix), Lavanya Sharan (Netflix), Alec Schaefer (Netflix), Yu Chan (Netflix)

Session B: AI in Science, Leadership, and Creative Work

Room TBD  |  Chair: TBD

“Human–AI Collaboration in Science at Scale: A Global Large-scale Randomized Field Experiment”

Binglu Wang (Harvard Business School), Weixin Liang (Stanford University), Jiahui Xue (Northwestern University), Yuhui Zhang (Stanford University), Hancheng Cao (Emory University), Dashun Wang (Northwestern University), Yian Yin (Cornell University)

“Proactive AI Agent in Creative Teams”

Brett Beekley (Google), Aliza Hoffman (Google), Miaomiao Zhang (Harvard)

“How GenAI Complements or Substitutes Leadership Skills: A Field Experiment on Coaching Employees”

Xueming Luo (Temple)

“Less Shadow, More Governed: A Field Experiment on AI-assisted Peer Review”

Zhe Zhang (UC San Diego), Sarah Bana (LSE), Hemant Bhargava (UC Davis), Laura Brandimarte (U Arizona), VC Choudhary (UC Irvine), Brian Han (UIUC), J. Frank Li (UBC), Pantelis Loupos (UC Davis), Harry Wang (U Delaware), Daniel Zantedeschi (USF)

Session C: Adaptive Policy Learning and Evaluation

Room TBD  |  Chair: TBD

“Logging Policy Design for Off-Policy Evaluation”

Connor Douglas (NYU Stern), Joel Persson (Spotify), Foster Provost (NYU Stern)

“Inference on the Value of Policy Improvement in Adaptive Experiments”

Justin Whitehouse (Stanford), Emma Brunskill (Stanford), Vasilis Syrgkanis (Stanford)

“Warm-starting the Validation of Adaptive Experiments”

Hammaad Adam (Amazon), Blake Mason (Amazon)

“Policy Learning with Weak Signals”

Benedikt Koch (Harvard), Winston Chou (Netflix), Aurélien Bibaut (Netflix), Nathan Kallus (Netflix and Cornell)

Day 1 Posters

1. When Larger Samples Do Not Help: Experimental Anchors for Sequential Evaluations

Daniel Yurovsky (Netflix), David Landy (Netflix)

2. Learnings and Pitfalls from AI-Assisted A/B Experimentation at Scale

Brett Daley (Meta), Ilya Gorbachev (Meta), Anubhav Natani (Meta), Nicholas Brown (Meta), Athena Dai (Meta), Rachel Song (Meta), Marcin Pekalski (Meta), Anchal Choubey (Meta), Daniel Ting (Meta)

3. Robust Estimation of Prior Parameters in Normal-Normal Empirical Bayes

Manu Burson (Amazon), Ryan Kessler (Amazon)

4. Neutral Is Not Free: Evaluating Downside Risk in Neutral Launches

Pablo Alcain (Meta Platforms, Inc.), Abbas Zaidi (Meta Platforms, Inc.), Jason Kang (Meta Platforms, Inc.), Mia Garrard (Meta Platforms, Inc.), Marine Veits (Meta Platforms, Inc.), Houssam Nassif (Meta Platforms, Inc.)

5. Beyond the Average Treatment Effect: Detection and Attribution of Temporal Dynamics in Online Experiments

Tracy Zhang (Spotify), Mårten Schultzberg (Spotify)

6. The Boundary of Platform Power: Evidence from a Field Experiment in Bangladesh

Yulin Hao (Peking University), Yan Luo (MIT), Catherine Tucker (MIT), Yifei Wang (University of Pittsburgh)

7. Measuring the Difference in Interference Level between Two Online Experiment Randomization Strategies

Aurelius A. Zilko (Booking.com), Edoardo Guarnerio (Booking.com)

8. Experimenting with AI Agent Behavior via Counterfactual Environments

Manuel Cherep (MIT), Pattie Maes (MIT), Nikhil Singh (Dartmouth)

9. Conformal Policy Learning with Distribution-Free Safety Guarantees

Ying Jin (University of Pennsylvania), Naoki Egami (MIT)

10. Going From One Experiment to Another: Estimating Treatment Effects Across Populations

Mengjie (Magie) Cheng (UT Austin), Elie Ofek (Harvard Business School), Nathan Vierling-Claassen (Wayfair), Jerry Chen (Wayfair)

11. Sequentially Rerandomized Switchback Experiments

Zhenghao Zeng (Stanford GSB), Christopher Adjaho (Airbnb), Alonso Bucarey (Airbnb)

12. Deploying Overlap Experimentation for Direct Mail Model Comparison

Quinn Lanners (Upstart)

13. Testing for Trends in Online Experiments

Chris Haulk, Lee Richardson, Jacopo Soriano

14. Encouraging Online Content Contributions with Peer Recognition and Platform Rewards: Evidence from a Field Experiment and Structural Model

Yikun Jiang (Purdue)

15. Enabling Mid-Experiment Changes and Shared Controls

Daniel Ting (Meta)

16. Fitting and Checking a Multivariate Empirical Bayes Launch Rule

Bartek Kozakowski, Kenneth Hung, Houssam Nassif, Jingang Miao (Meta)

17. Optimal Variance Reduction in Randomized Experiments

Amir Najmi (Google), Michael D. Keselman (independent)

18. Learning About Spillovers Without Network Data: What Outcome Dynamics Reveal and What They Do Not

Albert Tan (Amazon), Sadegh Shirani (MIT), James Nordlund (Amazon), Mohsen Bayati (Stanford)

19. Designing Optimal Human-AI Collaboration Processes for Complex Decision Tasks

Xinlan Emily Hu (MIT), Zezhen Dawn He (MIT), Dean Eckles (MIT), Ali Jadbabaie (MIT), Thomas W. Malone (MIT)

20. Designing Interleaving Experiments for Online Platforms

Weitao Cheng (HKUST), Ningyuan Chen (University of Toronto), Pin Gao (CUHK-Shenzhen), Nian Si (HKUST)

21. Adaptive Experiments with Protected Learning and Agentic Orchestration

Shusei Eshima (Netflix), Simon Ejdemyr (Netflix)

22. Randomizing the Guardrail: Experimental Design and Inference for Runtime Governance of AI Agents

Himanshu Joshi (University of Texas at Austin, MIT), Shivani Shukla (University of San Francisco), Dhari Gandhi and Sunita Kumari (COHUMAIN Labs)

23. Measuring User Session Success as Search Moves Toward Generative AI Answers: Interpreting Good Abandonment

Minha Hwang (Microsoft), Shavak Agrawal (Microsoft), David Chen (Microsoft), Carto Wong (Microsoft), Xin Du (Microsoft), Tiannan Wang (Microsoft), Georg Buscher (Microsoft), Zheng Chen (Simplex AI)

24. Measuring Opportunity Cost with Stock Lifetime Value

Geoffrey Decrouez (Zalando SE), Tobias Huelden (Zalando SE), Paresh Nakhe (Zalando SE), Dominik Prugger (Zalando SE)

25. AI Sycophancy and Human Decision-Making

Raphael Raux (MIT), Rui Zuo (MIT), Harang Ju (Johns Hopkins), Sinan Aral (MIT)

Session D: Supporting Human Experiments with Simulated Users

Room TBD  |  Chair: TBD

“Mixed-Subjects Randomized Controlled Trials: Theory, Evidence, and Experimental Infrastructure”

Austin van Loon (MIT Sloan), Klint Kanopka (NYU Steinhardt)

“Designing Behavioral Profiles for Language Model Simulation of Out-of-Distribution Human Behavior”

Robin Na (MIT), Kehang Zhu (Stanford), Logan Cross (Google DeepMind), Michiel Bakker (MIT)

“Pricing Experimentation with Personified LLM Agents”

Tobias Huelden (Zalando), Hisham Gath (Zalando)

“Can AI Agents Simulate A/B Test Outcomes? A Validation Framework for Agentic Experimentation”

Lorenzo Masoero (Amazon), Stefan Hut (Amazon)

Session E: Experiments and Social Impact

Room TBD  |  Chair: TBD

“Optimal Causal Annotations: An Application to Casenotes in Social Services”

Ezinne Nwankwo (Stanford), Lauri Goldkind (Fordham University), Angela Zhou (University of Southern California)

“Can citations mitigate AI underreliance? Evidence from an online experiment on AI use in social service delivery”

Jennah Gosciak (Cornell), Eric Giannella (Georgetown), Zhaowen Guo (Georgetown), Michael Chen (Nava Labs), Allison Koenecke (Cornell)

“Increasing the impact of large-scale experiments with a more feasible type of adaptive randomisation”

Jay A. Olson (Government of Canada), Kieran Findlater (Government of Canada), Philip Astles (Government of Canada), Anna N. Rafferty (Carleton College), Joseph J. Williams (University of Toronto)

“Learning Treatment Effects during Resource Allocation via Priority-Queue Randomization”

JungHo Lee (CMU), Johnna Sundberg (CMU), Pim Welle (Allegheny County DHS), Bryan Wilder (CMU)

Session F: Interference in Platforms and Networks

Room TBD  |  Chair: TBD

“Interference through Shared Model State: A Shift-Share Design”

Ahmed Ansari (LinkedIn), Cheng Ding (LinkedIn), Min Gong (LinkedIn), Dhuri Shrivastava (LinkedIn), Jim Sorenson (LinkedIn)

“Network Experiments with Edge Treatments and Node Outcomes”

Artem Kuriksha (Meta), Kenneth Hung (Meta)

“Neyman Jackknife: Design-Based Variance Estimation for Causal Inference under Interference”

Bryan Park (Stanford), Stefan Wager (Stanford)

“Which Side Should Be Randomized? Bias and Variance in Two-Sided Marketplace Experiments”

Ramesh Johari (Stanford), Anushka Murthy (Stanford)

Please enjoy refreshments in the 7th Floor lobby.

Session G: Improving Experimentation Programs

Room TBD  |  Chair: TBD

“Incentive Alignment In Online Experimentation”

Ermis Soumalias (Meta Inc), Richard Mudd (Meta Inc), Abbas Zaidi (Meta Inc)

“Knowledge Extraction from Randomized Experimental Archives: An Application to AI Content Generation”

Tong Wang (Yale), K. Sudhir (Yale), Hengguang Zhou (Columbia)

“Ideas Worth Testing: Generative AI and the Returns to Experimentation”

Kevin Lee (University of Michigan), Benjamin Letham (Meta), Eytan Bakshy (Meta)

“From Experimental Evidence to Organizational Knowledge: A Framework for Experiment Agents”

Leon Leng (UW & Tencent), Shichao Han (Tencent), Daimin Tang (Tencent), Darwin Wang (Tencent), Shi Chen (UW)

Session H: Mechanisms, Dynamics, and Heterogeneous Treatment Effects

Room TBD  |  Chair: TBD

“Identifying Confounded Dynamic Systems with Multi-Arm Digital Experiments”

Ruohan Zhan (LBS), Carl Mela (Duke), Shichao Han (Tencent)

“The Online Ads Paradox: Incentive-Compatible Valuations in a Field Experiment Before and After Treatment Disclosure”

Alessandro Acquisti (MIT), Cristobal Cheyre (Cornell), Avi Collis (CMU), Cristiana Firullo (Cornell), Florian Schaub (UMichigan), Li Jiang (GWU)

“Who Interacts with What: Component-Level Effects from Bundle Experiments via Differential Exposure”

Yedilkhan Baigabulov (Amazon), Abhimanyu Mukerji (Amazon), Victor Quintas (Amazon), Jonathan Jensen (Amazon)

“From Viral Exposure to Engagement: A Causal Mediation Framework for Digital Platforms”

Bolun Liu (Johns Hopkins University), Vamsi Manthena (Roblox), Wally Toh (Roblox), and Wenjing Zheng (Roblox)

Session I: Human-AI Interaction

Room TBD  |  Chair: TBD

“The Value of Personalization in Large Language Model Responses”

Andrey Fradkin (Boston University Questrom School of Business), Yan (Stella) Si (Boston University Faculty of Computing & Data Sciences), Giorgos Zervas (Boston University Questrom School of Business)

“Metacognitive AI and Peer Information in Collective Decisions”

Charlie Cheng (UMD), Eaman Jahani (UMD), Siva Viswanathan (UMD)

“From Assistance to Action: Agency and Adoption of Generative AI”

Wajeeha Ahmad (Columbia University), Eric Horvitz (Microsoft)

“Mental Models and Personalization of Model Selection”

Zezhen (Dawn) He (MIT), Xinlan Emily Hu (MIT), Dean Eckles (MIT), Renée Richardson Gosline (MIT)

Please enjoy refreshments in the 7th Floor lobby.

Panelists

Amy Hartwig 

Mårten Schultzberg — Product Lead, Experimentation, Spotify

Mårten Schultzberg is Product Lead for Experimentation at Spotify and for Confidence, Spotify’s experimentation platform. He holds a PhD in statistics, and his work focuses on the statistical foundations of large-scale online experimentation, including sequential testing, multi-metric decisions, and the trade-offs between Bayesian and frequentist approaches to decision-making. Most recently, he has studied synthetic users, and when offline evaluations can serve as valid surrogates for online experiments in the development of LLM-based products.

Winston Chou — Winston Chou, Research Scientist, Netflix

Winston Chou is a Research Scientist at Netflix, where he works on developing methods and solutions for real-world problems under practical constraints, such as extracting insights from weak experimental signals. His research received the KDD 2025 Best Paper Award in Applied Data Science. Previously, he held research and data science roles at Apple and Facebook. He holds a PhD in political science from Princeton University.

Please join us for remarks from the MIT Initiative on the Digital Economy.

Refreshments will be served. There will be an opportunity to view both Day 1 and Day 2 posters throughout the 7th floor.

Enjoy breakfast and networking with other conference attendees and speakers.

Welcome remarks from CODE@MIT organizer David Holtz, Assistant Professor in the Decisions, Risk, and Operations (DRO) Division at Columbia Business School; Research Affiliate at the MIT Initiative on the Digital Economy.

Damon Jones

Vanessa Burbano — Columbia Business School

Vanessa Burbano is the Donald C. Waite III Associate Professor of Social Enterprise in the strategy area at Columbia Business School.

Burbano researches the strategic implications of socially responsible and irresponsible firm practices; in particular, how they influence employee behavior. Her body of work has been recognized by her receipt of the 2021 Emerging Scholar Award from the Strategic Management division of the Academy of Management, and the 2021 Emerging Sustainability Scholar Award from the Alliance for Research on Corporate Sustainability. She was also named to Poets and Quants’ 2019 list of Best 40 Business School Professors Under 40.

Her work has been published in Organization Science, Management Science, Strategic Management Journal, Organizational Behavior and Human Decision Processes, California Management Review, and Organization & Environment. Her publications have been recognized with awards including those from the Strategic Management Society, the Impact for Investment Research Prize, the Responsible Research in Management Award, the Berkeley Sustainable Business and Investment Forum, the International Association of Business & Society, the Industry Studies Association, and the Alliance for Research on Corporate Sustainability. Her work has also been cited in the popular press, including in the Wall Street Journal, the New York Times, Forbes, Fortune, and The Financial Times.

Burbano serves as Senior Editor at Organization Science, and on the editorial boards of Strategic Management Journal and Strategy Science. She also serves as the Corporate Engagement Director and Executive Committee Member of LEAP, an NSF Science and Technology Center (STC) dedicated to improving the accuracy of climate projections needed for optimal adaptation to extreme weather events.

She holds a Ph.D. from UCLA Anderson School of Management and an M.P.A. from Princeton University’s Woodrow Wilson School of Public and International Affairs. Prior to academia, she worked as an Associate at Goldman Sachs and as a Strategy Consultant at Monitor Group.

Presentation Abstract:

Rating systems are a common governance tool on two-sided platforms. Research suggests that, via indirect network effects, rating systems benefit participants on both the rating side and the rated side. However, participants on the rated side may not always recognize the benefits of the rating system and may respond negatively to its introduction. The rating side may anticipate the rated side’s negative reactions and react negatively as well, especially when the rated side holds more power over the rating side.

To empirically examine participants’ reactions to a proposed platform rating system in the context of a substantial cross-side power differential, we conducted two field experiments in collaboration with a Singapore-based labor platform that connects foreign domestic workers with employers (families). We found that employers, on average, were indifferent to the proposed rating system, even when the benefits to them were highlighted, with the highest-income employers exhibiting negative responses. We also found that workers reacted negatively, with those informed of the proposed rating system completely disengaging from the platform. Post hoc interviews and analyses support our interpretation that power differential underlies workers’ negative responses to the rating system.

This study contributes to the literature on platform rating systems by demonstrating that, in the presence of a substantial cross-side power imbalance, rating systems may not always be perceived positively, even by participants on the rating side, whom the system aims to empower. It also constitutes one of the first efforts in management research to study domestic workers, a highly vulnerable population of workers.

Please join us for refreshments in the 7th Floor lobby.

Session J: Education

Room TBD  |  Chair: TBD

“The Illusion of Competence: Learning with a Conversational AI Chatbot Amplifies the Dunning–Kruger Effect”

Shachar Reichman (Tel Aviv University), Yaniv Shani (Tel Aviv University), Dena Yadin (Northeastern University), Inbal Yahav (Tel Aviv University)

“The AI Platform That Enables Real, Durable Learning: A Randomized Experiment on Which AI Functions Drive Student Learning”

Ishaan Batra (Purdue University), Alfonso Macarrein (Purdue University), Cecilia Ying (Purdue University), XingWei Yang (Toronto Metropolitan University)

“The Economics of Algorithmic Personalization: Evidence from an Educational Technology Platform”

Keshav Agrawal (Stanford), Susan Athey (Stanford), Ayush Kanodia (Stanford), Shanjukta Nath (University of Georgia), Emil Palikot (Northeastern)

“Improving Precision in Digital Experiments Using Classroom Structure: The Ordered LOOP Estimator”

Yanping Pei (WPI), Adam Sales (WPI), Johann Gagnon-Bartsch (U-M), Eamon Worden (WPI), Neil Heffernan (WPI)

Session K: Experiment Design Under Treatment Abundance

Room TBD  |  Chair: TBD

“Coupling Designs for Randomized Experiments with Complex Treatments”

Max Cytrynbaum (Yale), Fredrik Savje (Uppsala)

“Two-Stage Adaptive A/B Testing Under Treatment Abundance”

Tomoya Sasaki (Netflix)

“Statistical Power for Fixed-Budget Adaptive Experiments”

Blake Mason (Amazon)

“Policy-Aware Design of Large-Scale Factorial Experiments”

Xin Wen (NYU), Xi Chen (NYU), Will Wei Sun (Purdue), Yichen Zhang (Purdue)

Session L: Designing Experiments Over Time

Room TBD  |  Chair: TBD

“When and How to Pilot: Design Rules for Two-Wave Experiments”

Juan C. Yamin (Brown)

“Estimating Treatment Effects in Finite Horizon Experiments with Rolling Enrollment”

Jaume Vives-i-Bastida (NYU), Liwei He (Airbnb), Erik Iverson (Airbnb), Oscar Zarate (Airbnb), Navin Sivanandam (Airbnb), Ramesh Johari (Stanford), Stefan Wager (Stanford)

“Baseline-Adjusted Truncated Policy Gradients for Rapid-Switching Experiments”

William McCloskey (Uber Technologies), Wenqian Xing (Stanford University), Ashwinkumar Badanidiyuru (Uber Technologies), Eric Becker (work done at Uber Technologies), Chris Harris (Fireworks AI; work done at Uber Technologies), Stefan Wager (Stanford University), Ramesh Johari (Stanford University)

“Spectral Design of Random-Duration Switchbacks”

Yuchen Hu (Columbia University)

Session M: Advertising, Personalization, and Consumer Welfare

Room TBD  |  Chair: TBD

“Where to Cut Ad Load on AVOD: Targeting the Entry Point of Serialized Content on ABEMA”

Shingo Uto (AbemaTV, Inc.), Shota Yasui (CyberAgent, Inc.)

“The Effects of Advertising and Personalization on Search Costs: Evidence from an Online Field Experiment”

Cristiana Firullo (Cornell), Cristobal Cheyre (Cornell), Alessandro Acquisti (MIT)

“What Do Privacy-Constrained Advertising Experiments Measure? Comparing Incrementality at the User and Market Levels”

Dingzhe Leng (Berkeley), Jingpeng Hong (TikTok)

“Algorithm or Creative? Disentangling Algorithmic Divergent Delivery from User Ad Preferences in Digital Platforms”

Pallavi Pal (Stevens Institute of Technology), Anjana Susarla (MSU)

Please join us for refreshments in the 7th Floor lobby.

Gordon Burtch — Gordon Burtch, Boston University, Meta

Gordon is the Allen and Kelli Questrom Professor in Information Systems at Boston University’s Questrom School of Business and a Research Scientist at Meta. His research primarily focuses on how digital technologies affect individual behavior, and the broader socioeconomic consequences of digital technologies. His work has been supported by more than $2 million in grants from various corporate, non-profit, and government entities. Gordon presently serves as an Associate Editor for Management Science. Prior to his current roles, Gordon worked as a researcher for Microsoft Research, a hardware design engineer, an IT systems auditor, and a financial services technology consultant with Accenture Canada.

Manish Raghavan — MIT

Manish Raghavan is the Drew Houston (2005) Career Development Professor at the MIT Sloan School of Management and Department of Electrical Engineering and Computer Science. His research centers on the societal impacts of AI & algorithmic decision making, spanning human-AI collaboration, algorithmic monoculture and homogenization, and algorithmic discrimination.

Neha Mittal — CEO & Co-Founder

Neha Mittal is the CEO and founder of JustAI (YC + Sequoia), the AI marketing platform behind personalization and experimentation for brands like Figma, Notion, and Coursera. Before JustAI, she ran Growth for Pinterest & Twitter (where she had the pleasure of working with Mr Musk for 9 months). She also cofounded CityStructure, and held various product & engineering roles at Goldman Sachs, Chan Zuckerberg & Sensorflow. She’s based in San Francisco and splits her roots between Singapore and India.

Day 2 Posters

26. Experimental Design for Matching

Chonghuan Wang (UT Dallas)

27. Personality Pairing Mitigates Diversity Collapse in Human-AI Collaboration

Harang Ju (JHU), Sinan Aral (MIT)

28. Muting Durably Reduces Engagement with Untrustworthy Online Sources; Media Literacy Doesn’t

Do Won Kim (University of Maryland), Brendan Nyhan (Dartmouth College), Ro’ee Levy (Tel Aviv University), Filippo Menczer (Indiana University), Maria Elizabeth (Betsi) Grabe (Boston University), Giovanni Luca Ciampaglia (University of Maryland)

29. Experiment-Guided Covariate Selection for Observational Causal Inference

Chen Wang (HKUST(GZ)), Shichao Han (Tencent), Yilin Li (Tencent), Linhao Liu (Tencent), Yong Wang (Tencent)

30. Beyond Two Arms: Power-Optimized Adaptive Experiments

Cheng Yu (Chicago Booth), Yixin Tang (Galileo AI), Ilya Izrailevsky (Doordash), Shuangning Li (Chicago Booth)

31. Local Validity and Cross-Context Portability: Evidence from an AI-driven Hiring Platform

Hossein Alidaee (Harvard), Tarun Khanna (Harvard), Rem Koning (Harvard)

32. Statistical Foundations of LLM-based A/B Testing: A Surrogacy Framework for Human Causal Inference

Joel Persson (Spotify), Mårten Schultzberg (Spotify), Sebastian Ankargren (Spotify)

33. Covariate Learning and Regression Adjustment for Switchback Experiments with Carryover

Cheng Yu (Chicago Booth), Shuangning Li (Chicago Booth)

34. Concurrency Control for Conflicting Experiments

Daniel Ting (Meta), Houssam Nassif (Meta)

35. Benefits and Costs of Adaptive Sampling

Yu-Shiou Willy Lin, Dae Woong Ham, Iavor Bojinov

36. Joint Experimentation and Budget Allocation: Evidence from Conversion Lift Studies on Meta

Mohammad Rashid (UW), Hema Yoganarasimhan (UW), Enoch Hyunwook Kang (UW)

37. Auditing AI-Generated Advertising: A Cross-Product Prompt Experiment on Skin-Tone Representation

Hugo Arribas (IMT BS), Grazia Cecere (IMT BS), Vincent Lefrere (IMT BS), Catherine Tucker (MIT)

38. Understanding the Role of Friction in AI Writing Tools on Writer Motivation

Isabel Corpus (Cornell Tech), Mor Naaman (Cornell Tech)

39. Policy Evaluation and Safe Improvement under State Expansion

Heyan Zhang (University of Wisconsin-Madison), Yongyi Guo (University of Wisconsin-Madison)

40. Longitudinal Adaptive Experimental Design for Learning Multiple Target Estimands with Semiparametric Efficient Inference

Wenxin Zhang (UC Berkeley), Mark van der Laan (UC Berkeley)

41. Sequential-Halving Switchback

Paul Missault (Amazon), Blake Mason (Amazon), Dhivya Eswaran (Amazon), Hancao Li (Amazon)

42. Information Silos on Social Media: Experimental Evidence from TikTok

Wenbo Wang (HKUST), Tianyu Han (HKUST), Zijun (June) Shi (HKUST)

43. Agentic Monoculture under Firm Competition: Evidence from AI Interviews

Brian Jabarian (Carnegie Mellon University), Eugenio Piga (UC San Diego), Mariya Pominova (Duke University), Pëllumb Reshidi (Florida State University)

44. Anytime-valid Optimal Policy Identification

Daniel Molitor (Cornell University)

45. CAQ: Clustered Asymptotically Anytime-Valid Quantile Treatment Effects for Online Experiments

Diane Lu (LaunchDarkly), Eric Oh (LaunchDarkly)

46. People Escalate Against a Competitor Labelled Human and Hold Back Against One Labelled an Optimising Machine

Vinicius Ferraz (KIT), Leon Houf (KIT)

47. Leveraging Structural Information about Treatments to Improve Adaptive Experimentation

Jennah Gosciak (Cornell University, Amazon), Blake Mason (Amazon), Hammaad Adam (Amazon), Mahnaz Islam (Amazon)

48. Do AI Companions Crowd Out Real Friends? Evidence from a Randomized Dosage Experiment with Behavioral Social-Media Outcomes

Zenan Chen (UT Dallas), Jason Chan (U. Minnesota), Teng Ye (U. Minnesota)

Chiara Farronato — Harvard University

Chiara Farronato is Glenn and Mary Jane Creamer Associate Professor of Business Administration at Harvard Business School (HBS), and co-Principal Investigator of the Platform Lab at the HBS AI Institute. A fellow at the National Bureau of Economic Research (NBER) and the Center for Economic Policy Research (CEPR), her research focuses on the growth and regulation of digital platforms.

Nicole Pashley — Rutgers University

Nicole Pashley is an Associate Professor in the Department of Statistics at Rutgers University. Her research sits at the intersection of experimental design and causal inference. Particular research interests include causal inference for multi-factor treatments, complex experiments, treatment effect heterogeneity, and instrumental variables methodology.

Please join us for refreshments in the 7th Floor lobby.

Session N: AI Search, Discovery, and Publisher Economics

Room TBD  |  Chair: TBD

“Measuring User Success in Randomized Search Experiments: Interpreting Good Abandonment in Generative AI Search”

Minha Hwang (Microsoft), Shavak Agrawal (Microsoft), David Chen (Microsoft), Carto Wong (Microsoft), Xin Du (Microsoft), Tiannan Wang (Microsoft), Georg Buscher (Microsoft), Zheng Chen (Simplex AI)

“Less Traffic, More Attention? A Field Experiment on Publisher AI Summaries”

Yichen Pan (CMU), Mohsen Foroughifar (CMU), Param Vir Singh (CMU)

“AI in Search Reduces Publisher Referrals Without Improving User Experience: Experimental Evidence”

Jeffrey Gleason (Northeastern), Stephanie Wang (UPenn), Yakov Bart (Northeastern), Christo Wilson (Northeastern), Danaé Metaxa (UPenn)

“Where Attention Goes: A Field Experiment on Generative Conversational Recommender Design, Sales, and Consumption Diversity”

Mir M. Molaie (KAIST), Wonjae Lee (KAIST)

“The Rise of AI Search: Implications for Information Markets and Human Judgment at Scale”

Sinan Aral (MIT), Haiwen Li (MIT), Rui Zuo (MIT)

Session O: Decision Rules, Shrinkage, and Safe Launches

Room TBD  |  Chair: TBD

“Let Them Peek: Group Sequential Designs for A/B Testing”

Mia Garrard (Meta Platforms, Inc.), Lorenzo Perini (Meta Platforms, Inc.), Rina Friedberg (Meta Platforms, Inc.), Pablo Alcain (Meta Platforms, Inc.), Houssam Nassif (Meta Platforms, Inc.), Abbas Zaidi (Meta Platforms, Inc.)

“Robust A/B Decisions”

Malika Korganbekova (Stanford University), Max H. Farrell (University of California Santa Barbara), Sanjog Misra (University of Chicago)

“Empirical Bayes inference with calibrated language-model priors”

Bohan Wu (Columbia University), James McInerney (Netflix), Nathan Kallus (Netflix and Cornell University), David M. Blei (Columbia University)

“Calibrated Empirical Bayes Guardrails for Non-Inferiority Testing in Online Experimentation”

Tatiana Xifara (Airbnb), Alex Deng (Meta)

“Adaptive Experimental Design Using Shrinkage Estimators”

Evan T. R. Rosenman (Claremont McKenna College), Kristen B. Hunter (University of New South Wales)

Session P: Experiment Design in Advertising

Room TBD  |  Chair: TBD

“More Revenue with Fewer Partners: A Field Experiment on Header Bidding Ad Auctions”

Mohsen Foroughifar (CMU), Joy Lu (CMU), Minkyung Kim (CMU)

“Adaptive Ad Load Design for Sponsored Search Markets: Evidence, Theory, and Deployment”

Mohammad Rashid (UW), Hema Yoganarasimhan (UW)

“AB2: Two-Layer Randomized Experimentation for Measuring Ad-Auction Interventions via Conversion Lift”

Simon Hollands (Snap Inc.), Paul Lo (Snap Inc.), Xiaoyi Min (Snap Inc.), Jing Pan (Snap Inc.), Qinghua Wang (Snap Inc.), Evelyn Wu (Snap Inc.), Jingxi Yu (Snap Inc.), Zimo Zhu (Snap Inc.)

“Optimal Budget Split Tests for Ads Experiments”

Shan Ba (LinkedIn), Chenxi Liu (LinkedIn), Shuangning Li (University of Chicago)

“Enhancing Position Auctions in Retail Media”

Siddharth Prusty (Indian School of Business), Carl Mela (Duke University), Hana Choi (University of Rochester)

Session Q: Quantifying Success in AI Products and Other Complex Systems

Room TBD  |  Chair: TBD

“A/B Testing for Multi-Turn LLM Conversations”

Hannah Li (Columbia University)

“When Can You Ship on Evals Alone? Trial-Level Surrogacy for Offline Evaluation of LLM Systems”

Mårten Schultzberg

“The Noise is the Signal – Correlated Sampling Error is Rank-Informative for Proxy Metric Selection”

Sandro Provenzano (Zalando SE)

“Optimal Sequential Annotations for Off-Policy Evaluation”

Woojin Chae (University of Southern California), Ezinne Nwankwo (Stanford University), Haitong Qin (University of Washington), Angela Zhou (University of Southern California)

“Making Experiment Results Comparable through Sitewide Impact Estimation”

Xiaochen Zhang (Roblox), Wenjing Zheng (Roblox), Xingye Kang (Roblox), Vamsi Manthena (Roblox)

Please join us for closing remarks from the MIT Initiative on the Digital Economy.

Refreshments will be served. There will be an opportunity to view both Day 1 and Day 2 posters throughout the 7th floor.

Thank you to all who attended MIT IDE’s 2026 CODE@MIT