Science has a self-correcting mechanism which, when it works correctly, makes it a strong-link problem. The problem is that the self-correcting mechanism is fragile. Because it is fragile, keeping the self-correcting mechanism going is a weak-link problem.
Thus, for example, the fact that psychology never tried to replicate studies and trusted other studies meant that the self-correcting mechanism was broken. Making a bunch of psychologists try to replicate and then discover that a bunch of what they thought was science really wasn't, REALLY MATTERED.
But peer review to stop bad papers from being published is not part of the self-correcting mechanism. That intervention doesn't matter.
Fraud exists on a boundary. In the real world, fraudulent research helped keep billions flowing to investigating a bad theory about Alzheimer's. And helped the fraudulent researcher get promoted - all the way to President of Stanford University! At Stanford he has accelerated a destruction of both campus culture, and free speech standards, thinks to the same attitudes that set him along the path to being a fraudster in the first place. (Sadly, he's still there. And the cultural changes seem likely to ruin Stanford as key factor strengthening startup culture in Silicon Valley.) That's pretty strong evidence that fraud should be taken seriously, and isn't just an issue around the edges.
But most fraud doesn't come with such consequences. And if the self-correcting mechanisms of science are working well, fraud gets found and corrected without long-term damage.
So it is important to make sure that there are feedback loops to catch and eliminate fraud and fraudsters. We should actively protect the strength and value of those feedback loops. Because when they work, they police science and help make it into a strong-link problem.
Hot take, haven't thought about it enough, so more of a brainstorm comment.
The real self-correcting mechanism of science is engineering. Whatever we're capable of engineering will be tested again and again and again and again. This includes social sciences. Look at the most successful marketeers and most successful game-designers and there must be some form of science there that'll actually be relevant. Though, there is survivorship bias, not sure how to take that out yet, but even with survivorship bias: seeing certain trends happen in MMOs or seeing how discount pricing (e.g. now for $499!) exploits human biases. Those things get replicated way more.
Oh and then there's stuff like: buildings standing upright and rockets shot into space (or explode into our faces).
The premise of the article is dubious. With existing screening/filtering we already have a replication crisis. Why would fewer gatekeepers or more research be the answer?
I couldn't help to notice the parallels of weak and strong link problems to sexual evolution. Sexual dimorphism had the tendency to, at least in tournament species [1], evolve towards weak-link females which make up the risk-averse and "gatekeeping" (rejecting unfit males) reproductive basis of the population, and strong-link males with much higher variance in phenotype and mating success.
We humans don't have that much sexual dimorphism compared to many other species, but it has been suggested that human males exhibit higher variability, most prominently in cognitive abilities [2], i.e. more males with either high or low IQ. As always, as soon as something is related to human cognition, take it with a grain of salt, as our minds are very malleable. And, as always, there is a lot of controversy attached to that topic.
I can't imagine our species being able to survive and thrive like we did with risk seeking women. It takes about 15 years for a human infant to become independent and there are a lot bad things that can happen in 15 years even if you play it as safe as possible. One can argue that the roles can be shared or swapped, but that would be against specialization, hugely complicate mate selection, and it would break evolution by natural selection so no sex would excel at anything.
Now all of the above is really pretty conservative thinking, bound to ruffle some feathers and all that. So circling back to my first sentence, can anybody imagine things being different but with the same outcome?
> there are a lot bad things that can happen in 15 year
IMO, Our species has thrived because of our ability to communicate and work together, not because our gender roles somehow make us specially equipped for survival. The atomic family is a relatively recent development, and for much of history children were raised by a community. If your natural parents died or became unable to raise you, you weren't left out to starve. you were taken in by relatives or neighbors.
If our gender roles were different (and there are many cultures and places in history where humans had unique social structures) it would not "break evolution by natural selection". Plenty of species have gender roles that don't fit traditional western models and they continue to be subject to evolution.
I was talking more about the risks for the mother, not the children. As opposed to risk being taken by the father who has to go hunting, war, etc, for resources to raise the children. But even for the children, she can have multiple, of course. But each one comes with a cost of resources. Investing years into something to then just risk it... Not to mention things like women bleeding to death on birth that where very common.
Men on the other hand can die and another one can be found to take his place. And a man who plays it safe at the expense of resource gathering is useless (for perpetuating the species). So I think humans selected over time for safe women and risky men.
> These policies, like all forms of gatekeeping, are potentially terrific solutions for weak-link problems because they can stamp out the worst research. But they’re terrible solutions for strong-link problems because they can stamp out the best research, too. Reviewers are less likely to greenlight papers and grants if they’re novel, risky, or interdisciplinary. When you’re trying to solve a strong-link problem, this is like swallowing a big lump of kryptonite.
Scientist can publish whatever we like. Good research, bad research, whatever. Just open a blog like Tao https://terrytao.wordpress.com/ There are no "gatekeepers".
One problem is how a committee can evaluate that. The bad solution we reached is to just count the number of papers in serious peer review journals. But now there are a los of predatory journals where you can just publish whatever you like if you pay, and other kinds of bad journals. So there are more complicated rules to define what is "serious" and what is a "journal".
I don't understand what the author propose. If all journals publish whatever people submit, they just become a clone of WordPress, but we already have WordPress and a few alternatives.
Other problem is how non-scientist can consume that. Which groundbreaking results should be copied to newspapers. Now the problem is that journalist in the science section has little scientific formation, so it's difficult to evaluate every single post in WordPress and decide which are good. So they use papers in serious peer review journals as a good approximation, but many times they just copy the bad press release of the university.
Another problem is which results should get applied to public policies. We expect that the person in charge is an expert and have experts advisors in each area. But it will be too much work for them to be reading all day WordPress to find promising results, and evaluate all of them. Also, when people in other areas can just copy-and-paste a result to use it instead of measuring it again.
My guess is that in that case there will develop a net of relialable curators, something like "awesome-volcanoes" or "awesome-cetacean". Each one is too much work for a single person, so they may ask for help from trusted friends. They may even give advice about stile and clarity, before the blog post is included in the awesome list. And now we have reinvented the peer review journal system, and the only difference is that the papers are never printed in dead trees.
Maybe we are bound for the future not unlike the XVIII century where people who do original research have to be independently wealthy (or have a patron), whereas universities do something more akin to scholastics/theology (reiteration of existing knowledge with only occassional accumulation of new one).
We just need an up/down voting mechanism for scientific papers with voting restricted to people knowledgeable in the field. Don't know how to implement it but it should be designed with incentives to vote and guards against it being gamed.
Also, there is the homeopathy problem. Do only homeopaths vote if a homeopathic paper is good or bad, or any medical doctor can vote?
It will be very difficult to distinguish voting ring from serious research, because some communities are very small and each one cite each other paper, and have some papers together from time to time.
Now the idea is that the jornal has a magic public vote, and if the paper is bad the reputation of the journal is stained, and after enough bad papers the hivemind decides it's a bad journal.
Another form of votes are the citations. So instead of karma, there is the total number of citations, the h-index, and other form of counting them. They also can be gamed, because people can add self citations or citations to friends, specially in the introduction. And sometimes the reviewers ask for a cites, like in https://news.ycombinator.com/item?id=35738717 but most of the times it's more subtle.
There are a few new journals that claim to be "peer review", but are actually just a forum with upvotes and commets. I'm not sure if any of them are "serious" because they are not in my area, but moderations is very hard. If votes are secret, there is no incentive not to voter crap. If votes are public, you can be coerced to vote.
Note that now it's also very hard to know if a journal that is not in your area is serious or not. There are some metrics like impact index (average number of citations ¿during the first year?), but they are easy to game, different communities have a different number of average citations in each paper, and different community have different time to publish. An impact index of 1 is good en math and bad in physics.
I like the characterization and the premise but I completely disagree with the conclusion. To an academic, science may feel like a strong-link problem but from a layperson's pov it's the opposite. When bad science is translated into the real world, it turns into faulty engineering, bad medical interventions, terrible socioeconomic policies etc. The difficulty is that once bad science is out, it can take years/decades before it can be corrected. This is because science is thought to be our best way to get to an objective truth. This why bad science, when it goes out into the real world turns into "objective truth". Think about the "vaccines are bad for you" crap and the damage that one weak link that got out caused. Several of our modern problems are due to weak science making it out into the world and getting recognizing as "objective truth" because it came via scientific method.
The OP is right that in the long run science is self-correcting by its very nature. Over those timescales, half a generation or so only the good stuff stands the test of time. But over short timescales bad science making its way out into the world can wreak havok.
You hit the nail on the head. But maybe this is only true for current society who is in love with a deformed image of what science really is. If you decouple science from applied crafts by having a more conservative society who is skeptic by default of anything new we can keep engineering, medicine and policies weak-link and science hard-link.
This is an illuminating point of view. Science is weak-link on its output, because of the downstream damage it can do. But article's claim still holds on the input side of science - sorting ideas for funding and perhaps publishing early results with a disclaimer. That's a strong-link problem, where you want to consider impact times reliability rather than just reliability alone.
An interesting idea very well argued. I think the distinction between weak-link and strong-link problems is an important one. But there are two aspects where I’m not convinced:
1. Some parts of science actually are weak-link problems. For example, if you base your study on the conclusions of five published results and just one of them turns out to be fraudulent, then you’re screwed.
2. The reason that we often end up with weak-link culture and policy when strong-link ditto would be better is not a misdiagnosis of the problem, it’s that weak-link culture and policies benefit the great majority. Most people are (by definition) mediocre, and they benefit greatly from policies that obstruct the positive outliers, because they can’t compete with them without such policies. It’s sad, but it’s a part of human nature to try and set rules that benefits oneself.
Time, money, and attention are finite, yet output keeps growing. Hence the need for a lot of filtering. I wish there was an alternative, but I don't see any. Do journals reject quality work? Are geniuses passed up for promotion? Sure, but just giving some examples like Peter Higgs is hindsight bias.
e’ve got loads more scientists, and they publish way more papers. And yet science is less disruptive than ever, scientific productivity has been falling for decades, and scientists rate the discoveries of decades ago as worthier than the discoveries of today. (Reminder, if you want to blame this on ideas getting harder to find, I will fight you.
This can probably also be explained by low hanging fruit having been picked
Which is another huge essay to read and does not adequately answer the objection. There are for example four fundamental forces. Are we going to find another four? Maybe we can find a fifth, but that is pushing the frontier. if such a force exists, its obviously going to be much harder to find than the others. Same for fundamental particles. There are 16 of them, plus Higgs. Are we going to find another 17? Or 3 battery types. I think psychology is different because you can always find new permutations of experiments.
I read the essay, and my takeaway to "There are for example four fundamental forces. Are we going to find another four?" would be:
Maybe not, but we could find out that the model of fundamental forces wasn't really the final answer, and explain the universe much better with some alternative theory. In your comment you assume that science is more or less settled, and all there is to do is to tweak the present understanding a bit and fill some gaps. The same sentiment was shared by many prominent physicists already in the end of 1800s, who were obviously proved wrong by Einstein and others in the 1900s.
I tend to agree that rumors of the end of discovery have been greatly exaggerated, however in the case of theoretical physics, we have the unsatisfying problem of having several alternative theories, but severely constrained access to the energy/technology required to test them.
The low hanging fruit theory assumes there is only one tree and all fruit on it was visible from the beginning. Science has an exploration component that makes more low-hanging fruit visible and scaffolding component that makes existing fruit more accessible.
Isn’t scientific papers both a weak and strong link problem? If you have enough weak links you can now no longer trust papers at all so you need to scrutinise every paper. Which is like taking a swab and viol to your burger chain, run it in your lab, then order the wagyu special.
I think in science people are a strong link problem, papers are more likely a weak link problem. The gatekeeping of review, if the reviews are done correctly should make the papers better. A forcing function for no BS. Like a code review?
The last paper I published came back with 4 reviews comments:
review 1: This is important for the field, fix x, y and z and it should be ready to publish
review 2: This adds absolutely nothing, they are missing references from research group a, research group a, research group a.
review 3: Looks good, fix a, b, and c, and add references to research group b, research group b, research group b.
review 4: looks good, but figure x doesn't seem to show what you say it does, fix it or the caption.
From there it was actually fairly simple to see who was reviewing based on which papers they said we were missing. With research group A in particular, they were working on similar stuff so we think we may have scooped them. But that's also a bit of the problem with reviews, subject matter experts are probably also doing similar research. And if they have a big ego it can potentially hamper work by other groups getting published.
I think it is ridiculous to make such a sweeping statement about all of science when there is so much involved in each sub-field of science that brings its own problems to productivity and output.
Take for instance the fact that nobody in Europe seems to want to do a PhD anymore and nobody in America wants to do a post-doc (based on my experience speaking to colleagues looking for both things in the respective places). That means very often you have to settle for what you get and what you get is not necessarily the best people.
With regards to things being less disruptive, there are two issues here. One, the big papers that get published in i.e. Nature usually have extremely difficult techniques involved; something like low-temperature magnetic force microscopy that only a few research groups will have. This gatekeeps replication and further progression behind these few groups with these expensive piece of kit, so most people won't even care that it's happened. The other point I'd bring up is that a lot of research at the moment is not directly relevant to industry (so patents aren't quite as useful). Researchers are making absolutely insane devices to chase higher impact papers and patents that are so far ahead of where current industrial research is and wants to be that they're not going to adopt it easily at all.
All this is to say that more funding is great (always) but I don't buy what the author is saying about a strong-link problem at all. To me this just puts the idea of Nature = good science into people's heads and encourages bad research. The process of good science is incremental steps towards a shared goal, and lots of reports on the way there that tip off other people and help build a bigger picture.
>nobody in America wants to do a post-doc (based on my experience speaking to colleagues looking for both things in the respective places)
I think that is dependent on the field. Maybe this is true in CS or something. But nobody gets a faculty job in (for example) biomedical science without doing a postdoc. And often more than one.
i think this is true internally, bad papers more often than not get ignored within a scientific community. Even if fake findings cause other researchers to go down a rabbit hole pursuing a bad direction (as has happened apparently in biology and psychology multiple times, and probably every field) this doesn't matter all that much for society.
but when scientific research artifacts are used by people outside a scientific community, it is unfortunately a weak link problem. if there's one bad study that tells people what they want to hear, they will seek it out and find it.
Research funding and paper acceptance are not the same thing. We can do more risky research without accepting more low-quality papers. They are very different kinds of "bad".
This article is very insightful and applicable in many areas. For example, this paragraph explains why hiring is broken:
> Whether we realize it or not, we’re always making calls like this. Whenever we demand certificates, credentials, inspections, professionalism, standards, and regulations, we are saying: “this is a weak-link problem; we must prevent the bad!”
Small companies have an advantage when it comes to hiring since they often don't have a very formal process and can go about it strong-link style. Specially if the company is a startup pursuing product-market fit, going after people with outstanding skills can be very rewarding.
On the other hand, once the company grows there will be an HR department with a structured process. It operates in the weak-link style, which is aligned with the incentive structure, given that the HR personel will be remembered for bad hires, but probably won't get extra points for finding rough diamonds.
Looks like the observations is that "science is less disruptive than ever", and the proposed solution is to fund more random/weird research projects. But the former happens because all the low-hanging high-impact/low-cost research has been done already. You can't make a major discovery anymore using just your pulse and a couple of balls. So the historical analogies do not apply. The proposals to "ignore the worst", "don't gatekeep" will not work because the set of all possible research proposals is infinite, but money is not.
Yeah, I wish the author had spent a paragraph shoring up the flawed analogy. Here's how I did it in my head:
Normally, chains have their links in series, in which case the chain breaks when the weakest link breaks. Now imagine two objects connected directly by many links, i.e. a parallel chain. This is a "strongest link chain" where just one strong link, or a few strong links, can hold the two objects together even if most of the links break.
Yes this seems the way to think about it, but on the other hand linking two objects with many weak links will generally result in a strong link, because the force divides between them.
Maybe a tangent but somehow typical for this paper:
>Imagine if you could only upload a song to Spotify after you got a degree in musicology, or memorized all the sharps in the key of A-sharp minor, or demonstrated competence with the oboe.
Considering the state of music IPR we are possibly nearing that point.
By ignoring the economics of science, the article overlooks the point that the number of good stuff and the number of worse stuff is correlated. If we neglect quality control, we are spending more money on worse (or average) stuff and less on good stuff.
interesting to frame problems as strong/weak link, is kinda useful for thinking about certain things.
- - -
citing Nobel laureate's take on "if I'm working today I won't..." is not too convinsing
>In my day you could get a faculty job with 0 postdoc papers
can't imagine how that would work today, the world changes why science as a social activity that has costs can be immune to the change?
Too bad while bad music and bad novels only take up some shelf space and make searching a little harder, science projects costs $$$ and so the real Q is how to distribute $, current approach is not that good and gatekeeps some good stuff out, but what's the alternative?
It would like his solution is to somehow have perfect foresight as to which researchers will produce Nobel quality work. I don't see anything practical proposed.
I would say, in this author's phrasing, that knowledge generation is a strong link problem and knowledge consumption is a weak link problem.
You can find scientists engaged in addressing both, so I don't think the strong-link only approach would work. I like the author's model but I don't quite agree with the conclusions they draw from it.
Good read, though the food bits are trigger-warning worthy.
Science is about describing reality accurately, often describing an aspect of reality not easily and readily seen with the naked eye because it plays out over a long time or it's on a huge scale too large to view directly or a scale too tiny to see directly. It's about testing mental models to see what fits the facts and people want to be cautious about it because betting on an inaccurate model can have serious real world consequences.
Those consequences often cannot be undone.
I don't think it works to view it as one of these two models. I think science needs to both protect against quackery and also allow for innovation, even though those goals somewhat conflict.
Watching Sabine on YT for a time now I get the feeling that science can debunk quackery on the long run, sometimes in decades. Old guards who are gatekeepers of some of these misdirections have to die out. ~30-40 years to drop a major trend. Sooner or later the scientific method and the fallible humans operating it wins.
Disagree with how you've generally phrased it. Science is much more about provability, given enough evidence. The evidence could be invented, or not. I would agree that it is about observing reality with many given limitations.
I don't believe science cares that much about being 'accurate', mainly because we use fundamentally flawed models of the universe to justify our research, and then go on to abusing 'p' given the outputs of those models.
I am of course, being a little of a hardass here, but until we invent/discover something better than models, I will always hold this opinion.
That sounds like you are talking about academia, not science.
Mendel, a man remembered as the father of genetics, was a friar who failed to pass his exams for becoming a certified teacher, so he became an abbot. At that point, his scientific work languished because his duties were too burdensome to allow time for it.
I've made two major points, first about evidence or the invention of it, the second about models. I also inserted a smaller point on the usage of 'p'.
Evidence does not need a seal of approval from an institution to be considered 'good', just that statistically, using whatever tools we have, evidence has some semblance of validity that could represent something, potentially reality.
Models are those tools in pretty much all of science. Models are all broken, some fundamentally, others in just the nature of cost to develop one that can represent reality. But that's the crux, representing reality must be done cheaply and efficiently as you can, because accurately representing reality has way too much cost in just discovering all the variables to tweak. There will be no time for experiments if that is all you do. So you only have approximations to work with. Basically, you're never actually dealing with reality.
My smaller point of 'p' is just about how we firmly treat confidence intervals as measures of 'solid' research, which I guess can be extrapolated to academia, but I certainly don't think it's limited to academia.
Hopefully I'm not interpreting, but I think latency's point is more epistemological than a comment on "how science is being done". For example, scientists will still use Newtonian mechanics (as opposed to relativistic mechanics) to model reality even though we know it's "wrong". Part of this is pragmatic (tensor arithmetic is computationally expensive), but part of it is also because there's a beauty in the model itself and how it approximates what we observe. Taken to a greater extreme, we can also make Newtonian-like models that don't model observed reality at all, but
Ultimately though, I would say you're right overall - most scientists (in my experience) do want to understand the world/reality, and want to do it accurately.
Science has a self-correcting mechanism which, when it works correctly, makes it a strong-link problem. The problem is that the self-correcting mechanism is fragile. Because it is fragile, keeping the self-correcting mechanism going is a weak-link problem.
Thus, for example, the fact that psychology never tried to replicate studies and trusted other studies meant that the self-correcting mechanism was broken. Making a bunch of psychologists try to replicate and then discover that a bunch of what they thought was science really wasn't, REALLY MATTERED.
But peer review to stop bad papers from being published is not part of the self-correcting mechanism. That intervention doesn't matter.
Fraud exists on a boundary. In the real world, fraudulent research helped keep billions flowing to investigating a bad theory about Alzheimer's. And helped the fraudulent researcher get promoted - all the way to President of Stanford University! At Stanford he has accelerated a destruction of both campus culture, and free speech standards, thinks to the same attitudes that set him along the path to being a fraudster in the first place. (Sadly, he's still there. And the cultural changes seem likely to ruin Stanford as key factor strengthening startup culture in Silicon Valley.) That's pretty strong evidence that fraud should be taken seriously, and isn't just an issue around the edges.
But most fraud doesn't come with such consequences. And if the self-correcting mechanisms of science are working well, fraud gets found and corrected without long-term damage.
So it is important to make sure that there are feedback loops to catch and eliminate fraud and fraudsters. We should actively protect the strength and value of those feedback loops. Because when they work, they police science and help make it into a strong-link problem.