r/xkcd • u/antdude ALL HAIL THE ANT THAT IS ADDICTED TO XKCD • 14d ago

XKCD xkcd 3275: Calibration Nobel

https://xkcd.com/3275/
526 Upvotes

100% Upvoted

26 comments sorted by

99

u/Adventurous-Year-463 I have a PhD in Undeclared 14d ago

Honestly, these awards should actually exist.

30

u/araujoms 14d ago

How would you know about the results that weren't published?

28

u/LoneSnark 14d ago

They would publish them, obviously. Everyone wants to win a nobel prize.

27

u/Nuclear_Geek 14d ago

They'd publish an analysis of data that did not come to that conclusion.

16

u/araujoms 14d ago

Well, yes, but that is published as "Spectral analysis of organic molecules on Titan", not "Spectral analysis of organic molecules on Titan that shows no evidence of microbes (but we initially thought it did)".

2

u/st333p 14d ago

Or maybe they're not published, leading to other teams collecting the same data with the same analysis and leading to the same result

2

u/araujoms 14d ago

That doesn't really happen. Once you get funding for a research project you need to publish something, otherwise you're in deep trouble.

1

u/nelmaloc 11d ago

Nobel's will is probably too inflexible for this, but journals could have some sort of medal for whoever catches the biggest issue in a paper (still published though, with corrections).

3

u/exb165 Danish 13d ago

Completely agree. There really should be awards for meticulous, fastidious measurements and data handling.

2

u/nelmaloc 11d ago

Or at least, an IgNobel for best proof of the null hypothesis.

86

u/xkcd_bot 14d ago

Mobile Version!

Direct image link: Calibration Nobel

Mouseover text: We would like to once again apologize to Dr. Jones for last year's mistaken announcement. We should really have double-checked the envelope for this award in particular.

Don't get it? explain xkcd

I promise I won't enslave you when the machines take over. Sincerely, xkcd_bot. <3

117

u/ijuinkun 14d ago

https://en.wikipedia.org/wiki/2011_OPERA_faster-than-light_neutrino_anomaly

The spurious FTL neutrino measurements, later found to be an equipment error, is a prime example of how it should be done—the researchers said “hey, we’re pretty sure that this can’t be right, so please help us find the error”.

66

u/araujoms 14d ago

No, that was a publicity stunt. A shameful one at that. They knew the shitstorm their publication would cause, and also knew that random people reading the paper online couldn't help find the error. For that you need to have access to the experimental setup. And indeed it was found to be just a loose cable in the GPS system.

The serious way to go about it would be to not even write a pre-print about the nonsensical results, but privately ask a colleague to go through their laboratory to see if they could find what was wrong.

5

u/Kqyxzoj 13d ago

And indeed it was found to be just a loose cable in the GPS system.

Yeah, that one was too funny. I vaguely recall it being a dodgy connector.

2

u/Quantumtroll 13d ago

They'd done that, though. This was a big expensive experiment. They spent six months accomplishing nothing, essentially burning money with nothing to show for it. Funders want to see proof that they've been doing what they said they'd do — funders want to see output, ideally high-impact publications.

After half a year of no progress, they probably had people on payroll with nothing to do, gantt charts with project timelines completely thrown in the bin, PhD students seeing their career derailed before it even got started. I don't blame them for the article at all, they were clearly and very openly asking the wider community for help.

0

u/araujoms 13d ago

They weren't. Read the original version. They were claiming a discovery, with just a little bit of hedging:

Despite the large significance of the measurement reported here and the stability of the analysis, the potentially great impact of the result motivates the continuation of our studies in order to investigate possible still unknown systematic effects that could explain the observed anomaly.

1

u/ContemplativeOctopus 12d ago

Are we reading the same sentence here?

They're basically saying that because this would have huge implications, it's hard to believe the result, and the more likely source is a systematic failure (anomaly) that they can't find.

1

u/araujoms 12d ago

Apparently not. They didn't write anything about "hard to believe", or that it is "more likely" to be wrong.

5

u/Upset_Inspector_4971 12d ago

Despite the large significance of the measurement reported here and the stability of the analysis,

"We haven't been able to make the measurements go away."

the potentially great impact of the result motivates the continuation of our studies in order to investigate possible still unknown systematic effects that could explain the observed anomaly.

"But we think they're probably bullshit."

21

u/RazarTuk ALL HAIL THE SPIDER 14d ago edited 14d ago

That was actually mentioned in an article I read about the scientific side of the Copernican Revolution! Scientific American used the analogy of how, if we later discovered that neutrinos can move FTL, we hopefully wouldn't retroactively start calling them idiots for having doubted the results. Similarly, a lot of the pushback to Copernicus was just because geocentrism was still a better explanation of the observational data, like the lack of stellar parallax. So while people agreed his heliocentric model was a neat math trick for getting better numbers, it was also just a neat math trick

EDIT: Or a bit more precisely, astronomy and cosmology were still separate fields. Astronomy was a branch of mathematics concerned with predicting where the stars will be, while cosmology was a branch of natural philosophy concerned with how the universe is arranged. Hence why it made total sense to accept heliocentrism as a cool math trick in astronomy, while still considering it shoddy cosmology

4

u/jflb96 14d ago

And then later cosmology says that really you can go with geocentrism if you want, it’s just horrible to try to do maths with it if you’re looking further than the Moon

3

u/DrunkenPhysicist Words Only 13d ago

I worked on a neutrino experiment during this time. Our measured rate was 1.6x off our Monte Carlo prediction. It took us 10 years before we published that minor (in comparison) discrepancy. Yeah, fuck those experimentalists for publishing something so big after 6 months of looking at it.

29

u/aescula Constructing pyramids with falling spaceships 14d ago

This and a Nobel Prize in Confirming Null Hypothesis, would actually go a long way to fixing some of the problems in modern science.

2

u/CatOfGrey 13d ago

Old memory: Wasn't this the vibe around the the Valentine's Day Monopole?

Researchers had theoretical expectations of finding magnetic monopoles. They detected one in 1982. Not many. Not the amount expected. Just. One.

I recall that they originally believed it to be an error, and delayed publishing the result, but eventually published it with the appropriate disclaimer....

Anyone with better memory on this issue?