Centralization and Scientific Progress

The more social, cultural, and financial capital flows into a scientific institution's coffers, the better off its stakeholders will be. Centralization helps that flow. A scientific community with strong solidarity, a single recognizable brand, and a legally fixed structure can deliver more value to its members, and along the way it can accelerate science itself. But none of that is one-sided. Solidarity can curdle into chauvinism. A unified brand can give cover to free riders. A legally fixed chain of command can quietly decide which ideas are allowed.

Past a certain point, centralization starts getting in the way of scientific progress. But at what point does it start getting in the way of a stakeholder's access to capital? That point may be much, much further down the line. Solidarity can slip into chauvinism and hurt science long before it hurts anyone's bank account enough that they'd care. So we really want to know where centralization turns sour, so we can stop ourselves slipping past it. The trouble is there are so many moving parts that no thought experiment is going to settle it.

I've used agent-based simulations to study peer review and grant allocation, so I was excited to see them pointed at this problem too. In Agent-Based Models of Dual-Use Research Restrictions, Elliot Wagner and Jonathan Herrington argue that more connectivity between labs working on the same problem can actually slow scientific progress. They model a community of labs as a network of bayesian bandits picking between two competing theories. They vary the connectivity, from a fully connected network where every lab talks to every other, down to one where labs work in isolation. The decentralized networks are better at landing on the truth, and just as fast about it. In their model, decentralization is good for science.

The mechanism they suggest is social proof. They call it the "Zollman effect," after Kevin Zollman. When a scientist confidently backs a position before her connected colleagues have had a chance to evaluate it themselves, those colleagues raise their credence in it. They then act on a position they haven't really vetted, and the bias spreads downstream. If the original position was wrong, the whole community is more likely to land on the wrong answer. Isolating some labs from the chatter cuts this off, so more of them get a chance to find the right answer on their own.

It's an interesting result, but real scientific practice is messier than any model, and I wouldn't be surprised if theirs is blind to variables that change the picture. Thankfully, we don't have to take their word for it. In Meta-Research: Centralized Scientific Communities are Less Likely to Generate Replicable Results, Valentin Danchev, Andrey Rzhetsky, and James A. Evans ran a big empirical study on how the structure of scientific networks relates to the accuracy of what those networks publish. Their findings make Zollman, Wagner, and Herington's conclusions a lot more credible.

Biomedical science is a nice place to study the sociology of science. Not just because the stakes are huge, but because the field has a strong tradition of annotating its publications. For years now, tens of thousands of biomed papers have been tagged with the specific chemical and biological interactions they report on, which makes them tractable for large-scale analysis.

One of my favorite uses of those annotations is in Tradition and Innovation in Scientists' Research Strategies by Jacob G. Foster, Andrey Rzhetsky, and James A. Evans. They use the annotations to ask a sharp question about how scientists actually pick what to work on. Do they study biochemical interactions that have been picked over already (the traditional strategy)? Or do they go after completely novel ones (the innovation strategy)?

Danchev et al. lean on another biomed resource: huge automated experiments where machines record the interactions of thousands of chemicals across different measuring instruments in parallel. They combine those results with the annotations to study how the social network behind a body of research relates to the replicability of its claims, which they use as a proxy for the accuracy of the community.

They found a few things. Papers by overlapping groups of authors were significantly more likely to agree on the existence and direction of a given drug-gene interaction (DGI). DGI claims that turned out to be replicable in the automated experiments were significantly more likely to have been endorsed by the literature than ones that didn't replicate. And, most importantly for the question of progress and centralization, they found a significant negative relationship between centralization and the replicability of DGI claims. With that volume of data behind it, this gives Zollman and Wagner et al.'s theoretical results a real empirical leg to stand on.

Yet another piece of this is in Does Science Advance One Funeral at a Time? by Pierre Azoulay, Christian Fons-Rosen, and Joshua S. Graff Zivin. They study what happens to a subfield when a star scientist dies. Using PubMed citation data, they find that after an elite scientist passes away, outsiders are significantly more likely to enter the field. Their conclusion is that these outsiders bring fresh approaches and make significant contributions, at least as measured by downstream citations. In other words, both the methods and the membership of the community get more diverse. Like Zollman and Wagner et al. theorize and Danchev et al. find empirically, decentralization seems to be good for science. I personally want the star scientists to keep on living, and I'm sure there's a nonlethal way to get the same effect, if it even needs to be gotten. Azoulay et al. point out that this kind of gatekeeping may actually help a young field find its footing.

Overall, the picture these results paint of a healthy scientific community looks less like a walled city with a few big castles in the middle, or a clubhouse where everyone's chummy. It looks more like a diverse, sparsely connected tapestry of cliques, with a bit of random noise let in past whichever filter the elite of the field have settled on.

Maybe some new evidence will overturn the impression I got from these papers. All I can do is wring my hands at these massive machines of meat and money, and hope they move science forward fast enough to flatten the risks looming ahead.