90:
Many factors contribute to publication bias. For instance, once a scientific finding is well established, it may become newsworthy to publish reliable papers that fail to reject the null hypothesis. Most commonly, investigators simply decline to submit results, leading to
24:
is a result without the expected content: that is, the proposed result is absent. It is an experimental outcome which does not show an otherwise expected effect. This does not imply a result of zero or nothing, simply a result that does not support the
44:, i.e., the threshold set prior to testing for rejection of the null hypothesis. The significance level varies, but common choices include 0.10, 0.05, and 0.01. However, a non-significant result does not necessarily mean that an effect is absent.
83:, papers with statistically significant results are three times more likely to be published than those with null results. This unduly motivates researchers to manipulate their practices to ensure statistically significant results, such as by
95:. Investigators may also assume they made a mistake, find that the null result fails to support a known finding, lose interest in the topic, or anticipate that others will be uninterested in the null results.
67:. The experiment did appear to measure a non-zero "drift", but the value was far too small to account for the theoretically expected results; it is generally thought to be inside the
641:
Luijendijk, HJ; Koolman, X (May 2012). "The incentive to publish negative studies: how beta-blockers and depression got stuck in the publication cycle".
694:
437:
111:
103:
173:
Giunti, C.; et al. (1999). "New ordering principle for the classical statistical analysis of
Poisson processes with background".
986:
982:
1032:
990:
978:
687:
1153:
1069:
934:
98:
There are several scientific journals dedicated to the publication of negative or null results, including the following:
904:
36:, a null result occurs when an experimental result is not significantly different from what is to be expected under the
312:"Testing What Matters (If You Must Test at All): A Context-Driven Approach to Substantive and Statistical Significance"
680:
237:
1191:
52:
998:
1213:
1036:
758:
33:
1218:
1184:
869:
118:
1163:
1127:
1099:
1028:
782:
1177:
853:
521:
972:
68:
910:
858:
738:
441:
956:
826:
804:
773:
748:
731:
1107:
944:
865:
948:
835:
743:
707:
80:
672:
438:"Role of the Michelson-Morley experiments in making determinations about competing theories"
914:
194:
8:
821:
753:
519:; Chan, S.; Chalmers, T. C.; et al. (1987). "Publication bias and clinical trials".
399:
350:
311:
148:
56:
55:
were of this type, as it did not detect the expected velocity relative to the postulated
198:
1074:
1024:
839:
768:
726:
498:
287:
254:
210:
184:
143:
64:
41:
1148:
1112:
1079:
1004:
926:
703:
658:
623:
538:
534:
490:
485:
466:
419:
380:
331:
292:
274:
233:
92:
502:
255:"Equivalence Tests: A Practical Primer for t Tests, Correlations, and Meta-Analyses"
214:
922:
843:
797:
792:
654:
650:
615:
574:
569:
557:
530:
480:
471:
411:
370:
362:
323:
282:
266:
202:
153:
1117:
940:
895:
175:
138:
37:
1143:
1064:
1016:
809:
721:
206:
126:
1207:
1122:
1089:
1084:
627:
516:
423:
384:
335:
278:
270:
84:
602:
Chopra, Felix; Haaland, Ingar; Roth, Christopher; Stegmann, Andreas (2023).
964:
848:
662:
296:
542:
494:
1012:
885:
875:
619:
590:
Publication bias in meta-analysis: prevention, assessment and adjustments
375:
59:. This experiment's famous failed detection, commonly referred to as the
465:
Easterbrook, P. J.; Berlin, J. A.; Gopalan, R.; Matthews, D. R. (1991).
960:
952:
918:
880:
787:
415:
366:
327:
189:
129:
also publishes negative results in the form of research or data notes.
26:
125:
While it is not exclusively dedicated to publishing negative results,
1020:
1008:
603:
1054:
40:; its probability (under the null hypothesis) does not exceed the
1059:
48:
17:
464:
562:
702:
601:
588:
H. Rothstein, A. J. Sutton and M. Borenstein. (2005).
351:"An Equivalence Approach to Balance and Placebo Tests"
515:
640:
1205:
558:"Preliminary testing: The devil of statistics?"
348:
688:
582:
555:
460:
458:
227:
259:Social Psychological and Personality Science
695:
681:
592:. Wiley. Chichester, England; Hoboken, NJ.
455:
349:Hartman, Erin; Hidalgo, F. Daniel (2018).
112:Journal of Pharmaceutical Negative Results
104:Journal of Negative Results in Biomedicine
573:
484:
374:
286:
188:
79:Despite similar quality of execution and
987:Preventable fraction among the unexposed
983:Attributable fraction for the population
991:Preventable fraction for the population
979:Attributable fraction among the exposed
467:"Publication bias in clinical research"
228:Casella, George; Berger, Roger (2002).
1206:
397:
252:
232:(2nd ed.). Duxbury. p. 385.
172:
676:
404:American Journal of Political Science
355:American Journal of Political Science
316:American Journal of Political Science
309:
1154:Correlation does not imply causation
1070:Animal testing on non-human primates
63:, contributed to the development of
13:
74:
14:
1230:
400:"Arguing for a Negligible Effect"
634:
595:
549:
1037:Pre- and post-test probability
759:Patient and public involvement
655:10.1016/j.jclinepi.2011.06.022
575:10.31273/reinvention.v12i2.339
556:Pearce, J; Derrick, B (2019).
509:
430:
391:
342:
303:
246:
221:
166:
34:statistical hypothesis testing
1:
159:
119:Journal of Unsolved Questions
1164:Sex as a biological variable
535:10.1016/0197-2456(87)90155-3
486:10.1016/0140-6736(91)90201-Y
7:
1128:Intention-to-treat analysis
1100:Analysis of clinical trials
1029:Specificity and sensitivity
783:Randomized controlled trial
132:
53:Michelson–Morley experiment
10:
1235:
522:Controlled Clinical Trials
207:10.1103/PhysRevD.59.053001
1172:
1137:Interpretation of results
1136:
1098:
1047:
997:
971:
933:
903:
894:
870:Nested case–control study
820:
767:
714:
604:"The Null Result Penalty"
398:Rainey, Carlisle (2014).
310:Gross, Justin H. (2015).
71:level of the experiment.
739:Academic clinical trials
271:10.1177/1948550617697177
957:Relative risk reduction
805:Adaptive clinical trial
749:Evidence-based medicine
732:Adaptive clinical trial
253:Lakens, Daniël (2017).
945:Number needed to treat
1214:Design of experiments
949:Number needed to harm
836:Cross-sectional study
788:Scientific experiment
744:Clinical study design
230:Statistical Inference
51:, the results of the
1219:Logic and statistics
915:Cumulative incidence
608:The Economic Journal
822:Observational study
754:Real world evidence
708:experimental design
199:1999PhRvD..59e3001G
149:Noise (electronics)
57:luminiferous aether
1108:Risk–benefit ratio
1075:First-in-man study
1025:Case fatality rate
866:Case–control study
840:Longitudinal study
620:10.1093/ej/uead060
416:10.1111/ajps.12102
367:10.1111/ajps.12387
328:10.1111/ajps.12149
144:Imponderable fluid
127:BMC Research Notes
65:special relativity
42:significance level
1201:
1200:
1149:Survivorship bias
1113:Systematic review
1080:Multicenter trial
1043:
1042:
1033:Likelihood-ratios
1005:Clinical endpoint
973:Population impact
927:Period prevalence
704:Clinical research
479:(8746): 867–872.
93:non-response bias
47:As an example in
1226:
1048:Trial/test types
923:Point prevalence
901:
900:
844:Ecological study
827:EBM II-2 to II-3
798:Open-label trial
793:Blind experiment
769:Controlled study
697:
690:
683:
674:
673:
667:
666:
643:J Clin Epidemiol
638:
632:
631:
614:(657): 193–219.
599:
593:
586:
580:
579:
577:
553:
547:
546:
513:
507:
506:
488:
462:
453:
452:
450:
449:
440:. Archived from
434:
428:
427:
410:(4): 1083–1091.
395:
389:
388:
378:
361:(4): 1000–1013.
346:
340:
339:
307:
301:
300:
290:
250:
244:
243:
225:
219:
218:
192:
170:
154:Publication bias
1234:
1233:
1229:
1228:
1227:
1225:
1224:
1223:
1204:
1203:
1202:
1197:
1168:
1132:
1094:
1039:
993:
967:
941:Risk difference
929:
890:
824:
816:
771:
763:
727:Trial protocols
710:
701:
671:
670:
639:
635:
600:
596:
587:
583:
554:
550:
514:
510:
463:
456:
447:
445:
436:
435:
431:
396:
392:
347:
343:
308:
304:
251:
247:
240:
226:
222:
171:
167:
162:
139:Aether theories
135:
77:
75:Publishing bias
38:null hypothesis
12:
11:
5:
1232:
1222:
1221:
1216:
1199:
1198:
1196:
1195:
1192:List of topics
1188:
1181:
1173:
1170:
1169:
1167:
1166:
1161:
1156:
1151:
1146:
1144:Selection bias
1140:
1138:
1134:
1133:
1131:
1130:
1125:
1120:
1115:
1110:
1104:
1102:
1096:
1095:
1093:
1092:
1087:
1082:
1077:
1072:
1067:
1065:Animal testing
1062:
1057:
1051:
1049:
1045:
1044:
1041:
1040:
1017:Mortality rate
1003:
1001:
995:
994:
977:
975:
969:
968:
939:
937:
931:
930:
909:
907:
898:
892:
891:
889:
888:
883:
878:
873:
863:
862:
861:
856:
846:
832:
830:
818:
817:
815:
814:
813:
812:
810:Platform trial
802:
801:
800:
795:
790:
779:
777:
765:
764:
762:
761:
756:
751:
746:
741:
736:
735:
734:
729:
722:Clinical trial
718:
716:
712:
711:
700:
699:
692:
685:
677:
669:
668:
633:
594:
581:
548:
529:(4): 343–353.
508:
454:
429:
390:
341:
322:(3): 775–788.
302:
265:(4): 355–362.
245:
238:
220:
190:hep-ph/9808240
164:
163:
161:
158:
157:
156:
151:
146:
141:
134:
131:
123:
122:
115:
108:
76:
73:
9:
6:
4:
3:
2:
1231:
1220:
1217:
1215:
1212:
1211:
1209:
1194:
1193:
1189:
1187:
1186:
1182:
1180:
1179:
1175:
1174:
1171:
1165:
1162:
1160:
1157:
1155:
1152:
1150:
1147:
1145:
1142:
1141:
1139:
1135:
1129:
1126:
1124:
1123:Meta-analysis
1121:
1119:
1116:
1114:
1111:
1109:
1106:
1105:
1103:
1101:
1097:
1091:
1090:Vaccine trial
1088:
1086:
1085:Seeding trial
1083:
1081:
1078:
1076:
1073:
1071:
1068:
1066:
1063:
1061:
1058:
1056:
1053:
1052:
1050:
1046:
1038:
1034:
1030:
1026:
1022:
1018:
1014:
1010:
1006:
1002:
1000:
996:
992:
988:
984:
980:
976:
974:
970:
966:
962:
958:
954:
950:
946:
942:
938:
936:
932:
928:
924:
920:
916:
912:
908:
906:
902:
899:
897:
893:
887:
884:
882:
879:
877:
874:
871:
867:
864:
860:
857:
855:
854:Retrospective
852:
851:
850:
847:
845:
841:
837:
834:
833:
831:
828:
823:
819:
811:
808:
807:
806:
803:
799:
796:
794:
791:
789:
786:
785:
784:
781:
780:
778:
775:
774:EBM I to II-1
770:
766:
760:
757:
755:
752:
750:
747:
745:
742:
740:
737:
733:
730:
728:
725:
724:
723:
720:
719:
717:
713:
709:
705:
698:
693:
691:
686:
684:
679:
678:
675:
664:
660:
656:
652:
649:(5): 488–92.
648:
644:
637:
629:
625:
621:
617:
613:
609:
605:
598:
591:
585:
576:
571:
567:
563:
559:
552:
544:
540:
536:
532:
528:
524:
523:
518:
517:Dickersin, K.
512:
504:
500:
496:
492:
487:
482:
478:
474:
473:
468:
461:
459:
444:on 2012-11-07
443:
439:
433:
425:
421:
417:
413:
409:
405:
401:
394:
386:
382:
377:
376:1721.1/126115
372:
368:
364:
360:
356:
352:
345:
337:
333:
329:
325:
321:
317:
313:
306:
298:
294:
289:
284:
280:
276:
272:
268:
264:
260:
256:
249:
241:
239:0-534-24312-6
235:
231:
224:
216:
212:
208:
204:
200:
196:
191:
186:
183:(5): 053001.
182:
178:
177:
169:
165:
155:
152:
150:
147:
145:
142:
140:
137:
136:
130:
128:
121:
120:
116:
114:
113:
109:
106:
105:
101:
100:
99:
96:
94:
88:
86:
85:data dredging
82:
72:
70:
66:
62:
58:
54:
50:
45:
43:
39:
35:
30:
28:
23:
19:
1190:
1183:
1176:
1158:
965:Hazard ratio
849:Cohort study
646:
642:
636:
611:
607:
597:
589:
584:
565:
561:
551:
526:
520:
511:
476:
470:
446:. Retrieved
442:the original
432:
407:
403:
393:
358:
354:
344:
319:
315:
305:
262:
258:
248:
229:
223:
180:
176:Phys. Rev. D
174:
168:
124:
117:
110:
102:
97:
89:
78:
60:
46:
31:
21:
15:
1159:Null result
1118:Replication
1013:Infectivity
935:Association
886:Case report
876:Case series
859:Prospective
61:null result
22:null result
1208:Categories
961:Odds ratio
953:Risk ratio
919:Prevalence
905:Occurrence
881:Case study
448:2003-07-17
160:References
27:hypothesis
1021:Morbidity
1009:Virulence
911:Incidence
628:0013-0133
424:0092-5853
385:0092-5853
336:0092-5853
279:1948-5506
107:(defunct)
1185:Glossary
1178:Category
1055:In vitro
896:Measures
715:Overview
663:22342262
503:36570135
297:28736600
215:14948954
133:See also
1060:In vivo
543:3442991
495:1672966
288:5502906
195:Bibcode
49:physics
18:science
661:
626:
541:
501:
493:
472:Lancet
422:
383:
334:
295:
285:
277:
236:
213:
81:design
999:Other
568:(2).
499:S2CID
211:S2CID
185:arXiv
69:noise
838:vs.
706:and
659:PMID
624:ISSN
539:PMID
491:PMID
420:ISSN
381:ISSN
332:ISSN
293:PMID
275:ISSN
234:ISBN
20:, a
651:doi
616:doi
612:134
570:doi
531:doi
481:doi
477:337
412:doi
371:hdl
363:doi
324:doi
283:PMC
267:doi
203:doi
32:In
16:In
1210::
1035:,
1031:,
1027:,
1023:,
1019:,
1015:,
1011:,
1007:,
989:,
985:,
981:,
963:,
959:,
955:,
951:,
947:,
943:,
925:,
921:,
917:,
913:,
842:,
657:.
647:65
645:.
622:.
610:.
606:.
566:12
564:.
560:.
537:.
525:.
497:.
489:.
475:.
469:.
457:^
418:.
408:58
406:.
402:.
379:.
369:.
359:62
357:.
353:.
330:.
320:59
318:.
314:.
291:.
281:.
273:.
261:.
257:.
209:.
201:.
193:.
181:59
179:.
87:.
29:.
872:)
868:(
829:)
825:(
776:)
772:(
696:e
689:t
682:v
665:.
653::
630:.
618::
578:.
572::
545:.
533::
527:8
505:.
483::
451:.
426:.
414::
387:.
373::
365::
338:.
326::
299:.
269::
263:8
242:.
217:.
205::
197::
187::
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.