Bill Cypher on Nostr: Only research that shows statistically significant results gets published. ...
Only research that shows statistically significant results gets published. Statistically significant means a 5% chance of getting those results at random.
Here is the problem with that. Pick something dumb. Let's study if brushing teeth with your left or right hand results in fewer cavities. Clearly it doesn't, but we have a ton of money to burn studying it in our hypothetical.
We are going to do 200 studies. Here is what we would expect from 200 studies of something that has no effect. Even though it doesn't matter, of those 200 studies 10, 5% of them is the publish threshold, will show significant results and get published. 5 saying left is best and 5 saying right is best. 190 studies showing no relation get thrown out and never published.
How does society react to our experiments? Every health influencer picks a left or right camp and insists you must use the correct hand. You see they both have 5 studies to prove their point so there must be something too it but you can't figure out who is right.
Influencers get rich off your views and you get a new anxiety added to your life, but nothing possibly gets better in any way.
Here is the problem with that. Pick something dumb. Let's study if brushing teeth with your left or right hand results in fewer cavities. Clearly it doesn't, but we have a ton of money to burn studying it in our hypothetical.
We are going to do 200 studies. Here is what we would expect from 200 studies of something that has no effect. Even though it doesn't matter, of those 200 studies 10, 5% of them is the publish threshold, will show significant results and get published. 5 saying left is best and 5 saying right is best. 190 studies showing no relation get thrown out and never published.
How does society react to our experiments? Every health influencer picks a left or right camp and insists you must use the correct hand. You see they both have 5 studies to prove their point so there must be something too it but you can't figure out who is right.
Influencers get rich off your views and you get a new anxiety added to your life, but nothing possibly gets better in any way.