{"id":184,"date":"2016-03-17T21:19:00","date_gmt":"2016-03-17T21:19:00","guid":{"rendered":"http:\/\/iscpif.fr\/chavalarias\/?p=184"},"modified":"2017-04-01T06:42:25","modified_gmt":"2017-04-01T06:42:25","slug":"evolution-of-reporting-p-values-in-the-biomedical-literature-1990-2015","status":"publish","type":"post","link":"https:\/\/iscpif.fr\/chavalarias\/?p=184","title":{"rendered":"Evolution of Reporting P Values in the Biomedical Literature, 1990-2015"},"content":{"rendered":"\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-av_heading-2d951caaf9c8f24417255373dab774d5\">\n#top .av-special-heading.av-av_heading-2d951caaf9c8f24417255373dab774d5{\npadding-bottom:10px;\n}\nbody .av-special-heading.av-av_heading-2d951caaf9c8f24417255373dab774d5 .av-special-heading-tag .heading-char{\nfont-size:25px;\n}\n.av-special-heading.av-av_heading-2d951caaf9c8f24417255373dab774d5 .av-subheading{\nfont-size:15px;\n}\n<\/style>\n<div  class='av-special-heading av-av_heading-2d951caaf9c8f24417255373dab774d5 av-special-heading-h2  avia-builder-el-0  el_before_av_one_fifth  avia-builder-el-first '><h2 class='av-special-heading-tag '  itemprop=\"headline\"  >Evolution of Reporting P Values in the Biomedical Literature, 1990-2015<\/h2><div class=\"special-heading-border\"><div class=\"special-heading-inner-border\"><\/div><\/div><\/div>\n\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-av_one_fifth-9497bbf8bff73ea4c6992cc17fa9ec36\">\n.flex_column.av-av_one_fifth-9497bbf8bff73ea4c6992cc17fa9ec36{\nborder-radius:0px 0px 0px 0px;\npadding:0px 0px 0px 0px;\n}\n<\/style>\n<div  class='flex_column av-av_one_fifth-9497bbf8bff73ea4c6992cc17fa9ec36 av_one_fifth  avia-builder-el-1  el_after_av_heading  el_before_av_four_fifth  first flex_column_div av-zero-column-padding  '     ><p>\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-av_image-caf01efe52293e40a28806c1723b5cbf\">\n.avia-image-container.av-av_image-caf01efe52293e40a28806c1723b5cbf img.avia_image{\nbox-shadow:none;\n}\n.avia-image-container.av-av_image-caf01efe52293e40a28806c1723b5cbf .av-image-caption-overlay-center{\ncolor:#ffffff;\n}\n<\/style>\n<div  class='avia-image-container av-av_image-caf01efe52293e40a28806c1723b5cbf av-styling- av-img-linked avia-align-center  avia-builder-el-2  el_before_av_icon_box  avia-builder-el-first '   itemprop=\"image\" itemscope=\"itemscope\" itemtype=\"https:\/\/schema.org\/ImageObject\" ><div class=\"avia-image-container-inner\"><div class=\"avia-image-overlay-wrap\"><a href=\"http:\/\/jama.jamanetwork.com\/article.aspx?articleid=2503172\" class='avia_image '  target=\"_blank\"  rel=\"noopener noreferrer\" aria-label='JAMA'><img decoding=\"async\" fetchpriority=\"high\" class='wp-image-185 avia-img-lazy-loading-not-185 avia_image ' src=\"https:\/\/iscpif.fr\/chavalarias\/wp-content\/uploads\/sites\/4\/2016\/03\/JAMA.png\" alt='' title='JAMA'  height=\"293\" width=\"576\"  itemprop=\"thumbnailUrl\" srcset=\"https:\/\/iscpif.fr\/chavalarias\/wp-content\/uploads\/sites\/4\/2016\/03\/JAMA.png 576w, https:\/\/iscpif.fr\/chavalarias\/wp-content\/uploads\/sites\/4\/2016\/03\/JAMA-300x153.png 300w, https:\/\/iscpif.fr\/chavalarias\/wp-content\/uploads\/sites\/4\/2016\/03\/JAMA-450x229.png 450w\" sizes=\"(max-width: 576px) 100vw, 576px\" \/><\/a><\/div><\/div><\/div><br \/>\n<article  class='iconbox iconbox_left_content av-av_icon_box-4a19060a8c7cf5a4437c4d368c3d4e20  avia-builder-el-3  el_after_av_image  el_before_av_textblock '  itemscope=\"itemscope\" itemtype=\"https:\/\/schema.org\/BlogPosting\" itemprop=\"blogPost\" ><a href='http:\/\/jama.jamanetwork.com\/article.aspx?articleid=2503172' title='Read the paper on JAMA' aria-label='Read the paper on JAMA'  target=\"_blank\"  rel=\"noopener noreferrer\" class='iconbox_icon heading-color avia-iconfont avia-font-entypo-fontello' data-av_icon='\ue84d' data-av_iconfont='entypo-fontello' title=\"Read the paper on JAMA\" desc=\"Read the paper on JAMA\" ><\/a><div class=\"iconbox_content\"><header class=\"entry-content-header\" aria-label=\"Icon: Read the paper on JAMA\"><h3 class='iconbox_content_title '  itemprop=\"headline\" ><a href='http:\/\/jama.jamanetwork.com\/article.aspx?articleid=2503172' title='Read the paper on JAMA'  target=\"_blank\"  rel=\"noopener noreferrer\">Read the paper on JAMA<\/a><\/h3><\/header><div class='iconbox_content_container '  itemprop=\"text\" ><\/div><\/div><footer class=\"entry-footer\"><\/footer><\/article><br \/>\n<section  class='av_textblock_section av-av_textblock-0366cc7376be6c9e82a3e9cc8987b64f '   itemscope=\"itemscope\" itemtype=\"https:\/\/schema.org\/BlogPosting\" itemprop=\"blogPost\" ><div class='avia_textblock'  itemprop=\"text\" ><p><strong>Cite as :\u00a0<\/strong> Chavalarias, D., Wallach, J.D., Li, A.H.T., Ioannidis, J.P.A., 2016. Evolution of Reporting P Values in the Biomedical Literature, 1990-2015. JAMA 315, 1141. doi:10.1001\/jama.2016.1952<\/p>\n<p class=\"aTitle\"><strong>See also :<\/strong> JAMA editorial\u00a0<span class=\"contentDate\"><span class=\"ppub\"><span class=\"month\">March\u00a0<\/span><span class=\"day\">15, <\/span><span class=\"year\">2016 <\/span><\/span><\/span><a href=\"http:\/\/jama.jamanetwork.com\/article.aspx?articleid=2503156\" target=\"_blank\"><span id=\"scm6MainContent_lblArticleTitle\">The Enduring Evolution of the <i>P<\/i> Value<\/span><\/a><\/p>\n<\/div><\/section><\/p><\/div>\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-av_four_fifth-9c1d454dc06f7247226bc4e8f931daae\">\n.flex_column.av-av_four_fifth-9c1d454dc06f7247226bc4e8f931daae{\nborder-radius:0px 0px 0px 0px;\npadding:0px 0px 0px 0px;\n}\n<\/style>\n<div  class='flex_column av-av_four_fifth-9c1d454dc06f7247226bc4e8f931daae av_four_fifth  avia-builder-el-5  el_after_av_one_fifth  avia-builder-el-last  flex_column_div av-zero-column-padding  '     ><p>\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-av_heading-6a95c1cba13ebccc811a94bd2df6c797\">\n#top .av-special-heading.av-av_heading-6a95c1cba13ebccc811a94bd2df6c797{\npadding-bottom:10px;\n}\nbody .av-special-heading.av-av_heading-6a95c1cba13ebccc811a94bd2df6c797 .av-special-heading-tag .heading-char{\nfont-size:25px;\n}\n.av-special-heading.av-av_heading-6a95c1cba13ebccc811a94bd2df6c797 .av-subheading{\nfont-size:15px;\n}\n<\/style>\n<div  class='av-special-heading av-av_heading-6a95c1cba13ebccc811a94bd2df6c797 av-special-heading-h3  avia-builder-el-6  el_before_av_textblock  avia-builder-el-first '><h3 class='av-special-heading-tag '  itemprop=\"headline\"  >Abstract<\/h3><div class=\"special-heading-border\"><div class=\"special-heading-inner-border\"><\/div><\/div><\/div><br \/>\n<section  class='av_textblock_section av-av_textblock-0366cc7376be6c9e82a3e9cc8987b64f '   itemscope=\"itemscope\" itemtype=\"https:\/\/schema.org\/BlogPosting\" itemprop=\"blogPost\" ><div class='avia_textblock'  itemprop=\"text\" ><div id=\"scm6MainContent_ucArticleContent_rptSections_contentBody_0\" class=\"contentBody clearfix\">\n<p><strong>Importance<\/strong>\u00a0 The use and misuse of <i>P<\/i> values has generated extensive debates.<\/p>\n<p><strong>Objective<\/strong>\u00a0 To evaluate in large scale the <i>P<\/i> values reported in the abstracts and full text of biomedical research articles over the past 25 years and determine how frequently statistical information is presented in ways other than <i>P<\/i> values.<\/p>\n<p><strong>Design<\/strong>\u00a0 Automated text-mining analysis was performed to extract data on <i>P<\/i> values reported in 12\u202f821\u202f790 MEDLINE abstracts and in 843\u202f884 abstracts and full-text articles in PubMed Central (PMC) from 1990 to 2015. Reporting of <i>P<\/i> values in 151 English-language core clinical journals and specific article types as classified by PubMed also was evaluated. A random sample of 1000 MEDLINE abstracts was manually assessed for reporting of <i>P<\/i> values and other types of statistical information; of those abstracts reporting empirical data, 100 articles were also assessed in full text.<\/p>\n<p><strong>Main Outcomes and Measures<\/strong>\u00a0 <i>P<\/i> values reported.<\/p>\n<p><strong>Results<\/strong>\u00a0 Text mining identified 4\u202f572\u202f043 <i>P<\/i> values in 1\u202f608\u202f736 MEDLINE abstracts and 3\u202f438\u202f299 <i>P<\/i> values in 385\u202f393 PMC full-text articles. Reporting of <i>P<\/i> values in abstracts increased from 7.3% in 1990 to 15.6% in 2014. In 2014, <i>P<\/i> values were reported in 33.0% of abstracts from the 151 core clinical journals (n\u2009=\u200929\u202f725 abstracts), 35.7% of meta-analyses (n\u2009=\u20095620), 38.9% of clinical trials (n\u2009=\u20094624), 54.8% of randomized controlled trials (n\u2009=\u200913\u202f544), and 2.4% of reviews (n\u2009=\u200971\u202f529). The distribution of reported <i>P<\/i> values in abstracts and in full text showed strong clustering at <i>P<\/i> values of .05 and of .001 or smaller. Over time, the \u201cbest\u201d (most statistically significant) reported <i>P<\/i> values were modestly smaller and the \u201cworst\u201d (least statistically significant) reported <i>P<\/i> values became modestly less significant. Among the MEDLINE abstracts and PMC full-text articles with <i>P<\/i> values, 96% reported at least 1 <i>P<\/i> value of .05 or lower, with the proportion remaining steady over time in PMC full-text articles. In 1000 abstracts that were manually reviewed, 796 were from articles reporting empirical data; <i>P<\/i> values were reported in 15.7% (125\/796 [95% CI, 13.2%-18.4%]) of abstracts, confidence intervals in 2.3% (18\/796 [95% CI, 1.3%-3.6%]), Bayes factors in 0% (0\/796 [95% CI, 0%-0.5%]), effect sizes in 13.9% (111\/796 [95% CI, 11.6%-16.5%]), other information that could lead to estimation of <i>P<\/i> values in 12.4% (99\/796 [95% CI, 10.2%-14.9%]), and qualitative statements about significance in 18.1% (181\/1000 [95% CI, 15.8%-20.6%]); only 1.8% (14\/796 [95% CI, 1.0%-2.9%]) of abstracts reported at least 1 effect size and at least 1 confidence interval. Among 99 manually extracted full-text articles with data, 55 reported <i>P<\/i> values, 4 presented confidence intervals for all reported effect sizes, none used Bayesian methods, 1 used false-discovery rates, 3 used sample size\/power calculations, and 5 specified the primary outcome.<\/p>\n<p><strong>Conclusions and Relevance<\/strong>\u00a0 In this analysis of <i>P<\/i> values reported in MEDLINE abstracts and in PMC articles from 1990-2015, more MEDLINE abstracts and articles reported <i>P<\/i> values over time, almost all abstracts and articles with <i>P<\/i> values reported statistically significant results, and, in a subgroup analysis, few articles included confidence intervals, Bayes factors, or effect sizes. Rather than reporting isolated <i>P<\/i> values, articles should include effect sizes and uncertainty metrics.<\/p>\n<\/div>\n<\/div><\/section><br \/>\n<article  class='iconbox iconbox_left av-av_icon_box-3d62f8dcda100db8056efc20b36beec0  avia-builder-el-8  el_after_av_textblock  avia-builder-el-last '  itemscope=\"itemscope\" itemtype=\"https:\/\/schema.org\/BlogPosting\" itemprop=\"blogPost\" ><div class=\"iconbox_content\"><header class=\"entry-content-header\" aria-label=\"Icon: Get the original dataset\"><a href=' http:\/\/dx.doi.org\/10.7910\/DVN\/6FMTT3' title='Get the original dataset' aria-label='Get the original dataset'  target=\"_blank\"  rel=\"noopener noreferrer\" class='iconbox_icon heading-color avia-iconfont avia-font-entypo-fontello' data-av_icon='\ue8d3' data-av_iconfont='entypo-fontello' title=\"Get the original dataset\" desc=\"Get the original dataset\" ><\/a><h3 class='iconbox_content_title '  itemprop=\"headline\" ><a href=' http:\/\/dx.doi.org\/10.7910\/DVN\/6FMTT3' title='Get the original dataset'  target=\"_blank\"  rel=\"noopener noreferrer\">Get the original dataset<\/a><\/h3><\/header><div class='iconbox_content_container '  itemprop=\"text\" ><p>Data are available on dataverse. http:\/\/dx.doi.org\/10.7910\/DVN\/6FMTT3<\/p>\n<\/div><\/div><footer class=\"entry-footer\"><\/footer><\/article><\/p><\/div><\/p>\n","protected":false},"excerpt":{"rendered":"","protected":false},"author":9,"featured_media":365,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[17,3],"tags":[],"class_list":["post-184","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-publication","category-toread"],"_links":{"self":[{"href":"https:\/\/iscpif.fr\/chavalarias\/index.php?rest_route=\/wp\/v2\/posts\/184","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/iscpif.fr\/chavalarias\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/iscpif.fr\/chavalarias\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/iscpif.fr\/chavalarias\/index.php?rest_route=\/wp\/v2\/users\/9"}],"replies":[{"embeddable":true,"href":"https:\/\/iscpif.fr\/chavalarias\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=184"}],"version-history":[{"count":7,"href":"https:\/\/iscpif.fr\/chavalarias\/index.php?rest_route=\/wp\/v2\/posts\/184\/revisions"}],"predecessor-version":[{"id":1274,"href":"https:\/\/iscpif.fr\/chavalarias\/index.php?rest_route=\/wp\/v2\/posts\/184\/revisions\/1274"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/iscpif.fr\/chavalarias\/index.php?rest_route=\/wp\/v2\/media\/365"}],"wp:attachment":[{"href":"https:\/\/iscpif.fr\/chavalarias\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=184"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/iscpif.fr\/chavalarias\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=184"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/iscpif.fr\/chavalarias\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=184"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}