In a thought-provoking study published in Nature, Distinguished Professor John List from The Australian National University (ANU) College of Business and Economics (CBE) explores ‘Option C thinking’, a new method that encourages social scientists to produce policy-based evidence from the early stages of a project.
In the flashy ‘80s, America vibed with the beats of hip hop while grappling with a dark undercurrent of drugs infiltrating schools. It was this ominous backdrop that led President Reagan to declare a war against substance abuse. And from his efforts emerged the Drug Abuse Resistance Education (D.A.R.E) program –a science-backed campaign determined to keep the crack epidemic at bay.
Hopes soared when early data showed D.A.R.E. was effective, prompting a multimillion-dollar investment that saw the plan reaching 75 per cent of United State (U.S.) schools. Yet, four decades later, multiple studies have revealed a harsh truth: the program did not stop kids from doing drugs.
The reason why this idea flopped is something John has dedicated a significant part of his life to study—what he calls a ‘voltage drop’.
“When promising ideas like D.A.R.E. collapse at scale, we can talk about a voltage drop – or what happens when that great electric charge of potential that drives people and organisations dissipates – leaving behind dashed hopes, squandered money, and time,” says John.
“They are shockingly common. Indeed, across disciplines ranging from software development to education and beyond, between 50 per cent and 90 per cent of programs lose voltage at scale, even when they showed great promise in early trials.”
In the social sciences, voltage drops are perniciously pronounced—particularly when compared to the physical sciences—as experiments tend to focus on efficacy rather than generalisability.
“The cornerstone of the experimental approach in the social sciences is A/B testing. Imagine a group of researchers trying to prove if an early education program works. They will gather a group of children and split them into two groups: one that does not receive the intervention (A) and one that does (B),” John explains.
“If the experiment’s results are extraordinary, policymakers may decide to adopt the program and scale it, thinking that this is prudent since the choice is grounded in empirical evidence from an efficacy test.
“But that choice is far from being policy-relevant, simply because researchers have only explored whether the program works in the petri dish, not whether it works at scale.”
In the last decades, numerous failed attempts at scaling have led policymakers to distrust social-science research in what some see as a scalability crisis. However, John’s new study offers a glimmer of hope.
In his recent paper, he urges social scientists to think beyond the traditional A/B approach by introducing a third dimension: ‘Option C thinking’.
“Option C thinking is meant to take the initial discovery process from an efficacy test to engaging in a bigger picture view, such as examining what constraints the idea will face at scale, what key factors can impact scaling, and whether the mediation paths and moderators are in place at scale,” he says.
To do so, I propose including relevant tests of scale in the original research design, which forces the scientist to naturally start by recognising the big picture and seeking the information they need to gain scaling confidence.
“There is no single quality that distinguishes ideas that have the potential to succeed at scale. But my paper introduces five vital signs that can prevent an idea from taking off. A deficiency in any one of them can render an idea unscalable.”
By leveraging Option C thinking, John believes social-science researchers can produce the type of policy-based evidence that scaling demands. And despite its potential, he is also realistic about the complexities of his approach.
“Demanding greater information from researchers in early stages might lead to an inequitable distribution of scientific discoveries, journal pages and subsequent grant funding should this approach only be possible by well-funded researchers,” he says.
“This concern represents a key resource call to funders, with critical investments needed in shared R&D infrastructure to move research toward Option C thinking.”
As scientists increasingly adopt experimental methods to tackle everyday social problems, scaling their solutions becomes critical to ensure they benefit the greatest number of people.
For John, no enterprise is immune to voltage drops—neither successful companies like Uber nor the Australian Government (or any government for that matter).
“But, if we all engage in Option C thinking from first inspiration to launch, and even after success, we stand a really good chance to change the world,” he stresses.
Option C thinking should be viewed as an important part of developing solutions to global challenges, such as climate change or extreme poverty.
“I am optimistic that if universities, funders, foundations and policymakers come to the same conclusion I have, we will be able to revolutionise the research approach in a very positive manner.”
Read more about ‘Option C thinking’ and the five vital signs on John’s article, published in Nature.
For more information on voltage drops and the science of scaling, read John’s best-selling book The Voltage Effect: How to Make Good Ideas Great and Great Ideas Scale (2022).
Featured expert
John is the Kenneth C. Griffin Distinguished Service Professor in Economics and works in the Department of Economics at the University of Chicago. His research interests include experimental economics, field experiments, education, youth violence, economics of charity, environmental economics, experiments in firms, multi-unit auctions and neuroeconomics.