Bang goes the detonator plot!

in Reproduction

Correspondence should be addressed to A Vail; Email: andy.vail@manchester.ac.uk

A key purpose of scientific publication is to share the results of experiments so that researchers may interpret, learn from, and improve upon each other’s work. Despite the perennial search for alternatives (Smith 2006), peer review remains the cornerstone of quality assurance in the publication process. A central tenet of peer review is that authors need to convince peer reviewers and scientific editors that the conclusions are justified by the observed experimental data, and the choice of data presentation style remains largely in the hands of the authors.

Manuscripts submitted to Reproduction frequently contain what are sometimes nicknamed ‘detonator boxes’ or ‘dynamite plots’. That is, graphs that summarise data by showing the mean of a set of values as the height of a bar or box, topped by a vertical line showing the height of the mean plus, typically, one s.e.m. Such graphs are offered by common statistical software under the general heading of ‘bar charts’. Such graphs are already deemed not acceptable in some clinical and scientific journals and were described as ‘a waste of space’ more than 20 years ago (Altman & Bland 1996). More recently, there has been a gradual shift away from use of such charts. For example the checklist for submissions to the Nature journals includes ‘Individual data points are shown when possible, and always for n ≤ 10’. Nonetheless their use persists in many journals (Weissgerber et al. 2015). In this editorial we concur with others that there are always better options for the graphical presentation of continuous data (Bland 2004. How to upset the statistical referee. Available at: http://www-users.york.ac.uk/~mb55/talks/upset.htm accessed November 2019 and Weissgerber et al. 2019).

Consider the figure 1. A typical bar chart presentation is shown in panel (Fig. 1A). What can be understood from this graph? The only summary statistic presented is the mean within each group of values. Note that the standard error of the mean (s.e.m.) is not a summary statistic as it does not describe the data. Instead its use is for inference, being descriptive of the hypothetical distribution of values for the mean that would be obtained from repeating the experiment ad infinitum.

Figure 1
Figure 1

Fabricated data with arbitrary units on y-axis presenting three hypothetical groups. Panel (A) shows a detonator box or dynamite plot. Panels (B and C) show spread-plots of the data (points) and means (horizontal bar) of two data sets that could give rise to the summary in Panel (A).

Citation: Reproduction 159, 2; 10.1530/REP-19-0547

Two immediate issues arise with such bar charts. First, the arithmetic mean is sensitive to extreme values. The same mean could describe (a) a group of points that are all close together around that value; (b) a group of points all just to one side of the mean with a single value distant on the other side; (c) two equally sized groups, far from the mean but equidistant above and below; (d) any number of other configurations! Without any measure of dispersion, reviewers and ultimately readers have nothing further to aid them. Although an improvement over no information, a single measure such as the s.d., inter-quartile range or range usually adds little in isolation. Secondly, the appearance of the box itself, from zero to the mean, is counterintuitive. The visual implication, if anything, is that values from zero to the mean occur, whereas values beyond do not. This would be an entirely erroneous conclusion. Bar charts are designed for showing counts, where the area described by the bar itself has meaning. In summary, the only aspect of the experimental data we can discern from panel (Fig. 1A) is the mean within each group.

Now consider the spread-plots shown in panels (Fig. 1B) and (Fig. 1C). Notice that the bar chart in (Fig. 1A) could equally have arrived from the data in either (Fig. 1B) or (Fig. 1C). With the spread plots it is immediately clear to the reader how many values were observed, their range, whether there is any suggestion of grouping (possibly indicative of a third important axis), and whether the mean has been hijacked from the ‘typical value’ by (possibly erroneous) outliers. Any statistical analysis or interpretation of the presented data should reflect the immediately apparent contrasts between these panels.

Note that presentation of the data via such graphs does not require any more journal space than presentation of the detonator box, yet conveys substantially more information. Note also that all the summary information contained within panel (Fig. 1A) is also available in panels (Fig. 1B) and (Fig. 1C). Additionally, removing the bogus constraint of starting the y-axis from zero, which is the default for bar charts, allows greater resolution for comparison of the means in panels (Fig. 1B) and (Fig. 1C).

Some publications contain a hybrid plot type that superimposes the spread-plot on the detonator box. We assume that the perceived advantage of such a plot is that it combines the familiarity of the detonator box approach with the data visualisation of the spread plot. However, this hybrid plot suffers from the detonator box requirement to extend the y-axis to zero. We occasionally see use of another option, the box-plot, also known as the box-and-whisker plot. These typically use a horizontal line to mark the median (rather than mean), a surrounding box to demonstrate the interquartile range of the data, and vertical lines or ‘whiskers’ from the top and bottom of the box to illustrate the range, with extreme values (outliers) highlighted separately. Such graphs have many similar advantages over the detonator box to the simple spread-plot. However, caution should be exercised with small sample sizes as box-plots can also mask clustering of points.

So then why do we continue to see detonator boxes in scientific articles? We can think of three explanations. First, the authors may simply be unaware that bar charts are intended to show counts. Secondly, there may be an unthinking willingness to follow where others have led. Software manufacturers facilitate production of such graphs because they respond to the market and, we can only presume, scientists unthinkingly create this market because they have seen others present their data in this way. Thirdly, and more seriously, bar charts conceal data anomalies and exaggerate the appearance of differences. The s.e.m. may not be descriptive of the data but, being smaller in magnitude than the s.d., gives the impression of ‘tighter’ data.

There are many other subtle statistical concerns around common practices in the design and analysis of life sciences research (Simmons et al. 2011, Masca et al. 2015, Nature Editorial 2016), but we focus here on bar charts as a commonly arising and relatively easily remedied concern. Importantly, ubiquitous software packages allow the simple presentation of spread-plots which, in our opinion, are always preferable to detonator boxes. As such, Reproduction is committed to transparency of data and promoting data reporting that may enable reproducibility, and so strongly encourages the use of spread plots in the place of detonator plots.

Declaration of interest

This paper was edited by Greg FitzHarris and Christopher Price. A V is an Associate Editor (Statistics) of Reproduction. A V was not involved in the review or editorial process for this paper, on which he is listed as an author.

Funding

This research did not receive any specific grant from any funding agency in the public, commercial or not-for-profit sector. J W declares that he is supported by a Wellcome Institutional Strategic Support Fund award (204796/Z/16/Z).

References

  • AltmanDGBlandJM 1996 Statistics notes: presentation of numerical data. BMJ 572. (https://doi.org/10.1136/bmj.312.7030.572)

  • MascaNGDHensorEMACorneliusVRBuffaFMMarriottHMEalesJMMessengerMPAndersonAEBootCBunceC 2015 RIPOSTE: a framework for improving the design and analysis of laboratory-based research. eLife e05519. (https://doi.org/10.7554/eLife.05519)

    • Search Google Scholar
    • Export Citation
  • Nature Editorial 2016 Reality check on reproducibility. Nature 437. (https://doi.org/10.1038/533437a)

  • SimmonsJPNelsonLDSimonsohnU 2011 False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science 13591366. (https://doi.org/10.1177/0956797611417632)

    • Search Google Scholar
    • Export Citation
  • SmithR 2006 Peer review: a flawed process at the heart of science and journals. Journal of the Royal Society of Medicine 178182. (https://doi.org/10.1258/jrsm.99.4.178)

    • Search Google Scholar
    • Export Citation
  • WeissgerberTLMilicNMWinhamSJGarovicVD 2015 Beyond bar and line graphs: time for a new data presentation paradigm. PLoS Biology e1002128. (https://doi.org/10.1371/journal.pbio.1002128)

    • Search Google Scholar
    • Export Citation
  • WeissgerberTLWinhamSJHeinzenEPMilin-LazovicJSGarcia-ValenciaOBukumiricZSavicMDGarovicVDMilicNM 2019 Reveal, don’t conceal: transforming data visualization to improve transparency. Circulation 15061518. (https://doi.org/10.1161/CIRCULATIONAHA.118.037777)

    • Search Google Scholar
    • Export Citation

If the inline PDF is not rendering correctly, you can download the PDF file here.

 

     An official journal of

    Society for Reproduction and Fertility

 

Sept 2018 onwards Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 693 693 59
PDF Downloads 400 400 15
  • View in gallery

    Fabricated data with arbitrary units on y-axis presenting three hypothetical groups. Panel (A) shows a detonator box or dynamite plot. Panels (B and C) show spread-plots of the data (points) and means (horizontal bar) of two data sets that could give rise to the summary in Panel (A).

  • AltmanDGBlandJM 1996 Statistics notes: presentation of numerical data. BMJ 572. (https://doi.org/10.1136/bmj.312.7030.572)

  • MascaNGDHensorEMACorneliusVRBuffaFMMarriottHMEalesJMMessengerMPAndersonAEBootCBunceC 2015 RIPOSTE: a framework for improving the design and analysis of laboratory-based research. eLife e05519. (https://doi.org/10.7554/eLife.05519)

    • Search Google Scholar
    • Export Citation
  • Nature Editorial 2016 Reality check on reproducibility. Nature 437. (https://doi.org/10.1038/533437a)

  • SimmonsJPNelsonLDSimonsohnU 2011 False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science 13591366. (https://doi.org/10.1177/0956797611417632)

    • Search Google Scholar
    • Export Citation
  • SmithR 2006 Peer review: a flawed process at the heart of science and journals. Journal of the Royal Society of Medicine 178182. (https://doi.org/10.1258/jrsm.99.4.178)

    • Search Google Scholar
    • Export Citation
  • WeissgerberTLMilicNMWinhamSJGarovicVD 2015 Beyond bar and line graphs: time for a new data presentation paradigm. PLoS Biology e1002128. (https://doi.org/10.1371/journal.pbio.1002128)

    • Search Google Scholar
    • Export Citation
  • WeissgerberTLWinhamSJHeinzenEPMilin-LazovicJSGarcia-ValenciaOBukumiricZSavicMDGarovicVDMilicNM 2019 Reveal, don’t conceal: transforming data visualization to improve transparency. Circulation 15061518. (https://doi.org/10.1161/CIRCULATIONAHA.118.037777)

    • Search Google Scholar
    • Export Citation