Want
Want
WANT TO SEE MY
REPORT, COACH?
SPORT SCIENCE REPORTING IN
THE REAL WORLD
– Written by Martin Buchheit, France
On the 9 March 2013, Sir Alex Ferguson athletes understand, accept and use sport can be useful to answer the questions
delivered in the Irish Times probably one science is highly variable and unpredictable. that are actually asked by coaches
of the most encouraging ever message for The path leading to effective sport science and players. Second, working with
sport scientists in football: “Sports science, support is a is a long and winding road, with relatively small numbers of athletes
without question, is the biggest and most frequent stops and constant redirections within a team setting as well as being
important change in my lifetime. It has required. Historically, many mistakes have unable to effectively control for many
moved the game onto another level that been made while we learned about the variables makes interpretation difficult
maybe we never dreamt of all those years veracity and usefulness of our data and the with traditional analytical approaches
ago. Sports Science has brought a whole best ways to report and implement sports such as Null Hypothesis Significance
new dimension to the game”. While science in the elite sports setting. Among Testing (NHST, which includes ‘p values’
such statements are gold for universities the different components of effective sport and ‘t-tests’ for example). Over the last
advertising sport sciences courses all over science support, the three most important decade or so, however, great strides
the world and for young students willing steps are likely the following: have been made in understanding
to embrace their carrier in elite clubs, the 1. Having an appropriate understanding and reporting the effects we have
actual value of sport science may not and analysis of the data; i.e. using the on our athletes and more valid and
always be rated as high in some elite clubs right metrics and statistics. The first relevant approaches exist which are
or federations1. Having an impact on the consideration is the choice of the best much easier to clinically interpret2. The
training programme, as a sport scientist, is variables, i.e. those can be trusted in modern practitioner working oblivious
anything but easy1. The way coaches and terms of validity and relativity and that to these useful variables and analytical
36
Table 1
P values and in turn, study conclusions, are sample- MBI can be applied to assess changes in individuals. While
size dependent (the greater the n, the lower the P), individual score changes can be assessed in various ways (e.g.
irrespective of the size of the effect. The drop-out Z-scores6, standard difference score7), MBI additionally allows us
of a few athletes, or the lucky involvement of two to assess the likelihood of these changes to be true for any given
more subjects can induce a 180° change in a study athlete, once the typical error of the test of interest and the SWC are
conclusion5 (Table 2). known8,9 (Figure 2).
MBI allows authors to be honest with their sample size and better
acknowledge trivial effects. While a P>0.05 is often interpreted
Significance doesn’t inform on magnitude of effects, as a lack of an effect/difference, it is actually impossible to be
yet magnitude is what matters the most10. With a confident that this is the right interpretation of the data analysis
large enough sample size, even very small, trivial or (sample size issue, type II error resulting from low statistical
non-practical effects can appear significant (P<0.05). power). The beauty of MBI is that it allows us to distinguish
In practice, with 200 athletes showing a 0.01% between clear (confidence limits within the SWC) and unclear
improvement in performance, NHST would suggest (confidence limits overlapping the SWC) trivial effects (Figure 1).
that a nutritional supplement works, while the effects This can’t be achieved by NHST. An unclear effect/difference is
may in fact be negligible. Coaches and athletes are not to be interpreted as lack of an effect, but suggests the need to
first interested in knowing what kind of performance increase sample size to improve precision.
benefits may be expected from the supplement (i.e. MBI improves data visualisation. MBI principles should be applied
how much, the actual magnitude), and how likely to graphical reports produced by sport scientists, where shaded
this magnitude is to be of practical importance (i.e. trivial areas and confidence limits (or typical errors for individual
likelihood of the effect to be greater than the SWC). data) are presented systematically to acknowledge the fact that
not all changes are worthwhile and that some uncertainty always
remains (Figures 1, 2 and 3).
Table 1: Reasons why academics and practitioners should abandon null-hypothesis significance testing (NHST) and embrace magnitude-
based inferences (MBI) (adapted from Buchheit, 20164). SWC=smallest worthwhile change.
approaches could be considered incom- The following sections will detail each of this information3. While validity/reliability
petent, in my opinion, whereas a these three components. studies are important in the search of the
practitioner aware of these approaches best variables, their practical usefulness
but clinging to the past borders on COLLECTING AND UNDERSTANDING THE should also not be overlooked, i.e. their
disingenuous. (RIGHT) DATA ability to be used to impact on the training
2. Offering attractive and informative The first important step to build a programme. This relates to ‘interesting
reports via improved data presentation/ successful sport science system is to choose vs important’ types of data. For example,
visualisation. Effectiveness in this step and work with the right data3. With the measurement of maximal oxygen uptake
depends likely more on artistic skills and exponential rise in (micro) technology, vs maximal aerobic speed; only the latter
a creative mind than proper scientific collecting data from athletes has never can be used for training prescription.
knowledge and this is often overlooked been so easy. For every training session it is Statistics are probably one of the most
in sport sciences programme. Day-to- relatively easy to fully characterise both the important aspects of sport science when
day trials and errors are likely key in the external (e.g. tracking systems, encoders, it comes to using data to make decisions.
search of the optimal data visualisation force plates) and the internal load (e.g. heart Unfortunately, the statistical proficiency
strategies. rate, muscle oxygenation, sweat rate) placed of most practitioners in the field is often
3. Having appropriate communication on each athlete. However, technology per se insufficient to maximise the use of their
skills and personal attitude to efficiently might not be the solution; the foundations data and in turn, impact meaningfully on
deliver these data and reports to coaches of successful sport science support are training programmes. One of the main
and athletes. This step is without doubt probably laid on the pitch first, when reasons for practitioners’ lack of ‘statistical
the most important of the process; practitioners select the type of data that efficiency’ is that statistical lectures at
there is however no training offered at may help them to answer the questions that university have, to date, exclusively sung the
universities for this. Nothing replaces coaches and athletes have actually asked, in praises of NHST, which is:
experience, high personal standards the way they collect these data, how they • Not appropriate to answer the types
and humility at this stage, which is understand the limitations of each variable of questions that arise from the field:
generally developed over time. and how they analyse, report and utilise all as detailed in Table 1, the magnitude
Table 2
Type of data Example of data Method to derive the SWC Common SWC value
Individual athlete 1/3 of the performance coefficient of ~1% (0.1 s) for 100 m sprint time
Track and field events
performance variation ~3% (5 mins) for marathon
~2.5% (1 cm) for CMJ height
1) 1/5 of between-athlete SD
~1.3% (0.2 km/h) % for MAS
2) performance clues e.g. based on
Physical performance CMJ, sprint times, empirical observations of direct
in team sports MAS performance benefits, such as a
~1% (0.03 s) for 20 m sprint time
distance of 20-50 cm that one soccer
player needs to be ahead of the
opponent to win a ball
Highly athlete-dependent
Physiological data The choice of the SD fraction/
Factions/multiples of the within-
with no direct link to Heart rate variability multiple depends on the expected
athlete SD
performance sensitivity (the greater the SWC, the
more conservative the decisions)
Physiological data The actual change in this variable
with relationship Submaximal HR that relates to the smallest important 1% for submaximal HR
with performance change in performance
Still debated
Physical activity that Distance covered 1) 0.2 x between-athlete SD until new
Likely depends on both tracking
has no direct impact during matches in evidence is shown
variables and intensity zones15
on performance team sports 2) Interpretation of the magnitude left
to the practitioners (Figure 3)
Table 2: Suggested methods to derive the smallest worthwhile change4. For an exhaustive list of SWCs for different performance measures see
the work of Hopkins9 and Buchheit16,17. Change/differences of 1x, 3x, 6x and 10x SWC can be considered as small, moderate, large and very
large, respectively4. SWC=smallest worthwhile change, CMJ=countermovement jump, MAS=maximal aerobic speed, SD=standard deviation.
of an effect is what matters the most MBI is based on two simple concepts: Recommendations to calculate the SWC
to practitioners – P values don’t inform 1. Changes/differences in any variable are are provided in Table 2.
this4. systematically compared to a typical 2. Instead of a classic ‘yes or no’ type
• Not appropriate to assess individuals, threshold representative of a smallest response (NHST), the probabilities for
which is the core of elite athlete important or meaningful change (later these changes/differences to be ‘real’
monitoring. In fact, conventional to be termed the smallest worthwhile (greater than the SWC) are reported.
statistics allow analysis of population- change, SWC12). a. More precisely: chances are reported
based responses only (Table 1)4. a. Why? Not all changes are worthwhile/ both quantitatively (e.g. 75/25/0 for
As a valid alternative to NHST, clear meaningful. It is the magnitude of the percentage chances of greater/similar/
analytical advances can be reached change/difference that matters first: smaller magnitude than the SWC) and
using magnitude-based inferences (MBI, ‘is the change larger/greater than the qualitatively (e.g. possibly, likely, very
Table 1). This ‘new’ statistical approach, SWC? If yes, how many times greater?’ likely – Figures 1 and 2, and Table 3).
driven largely by Will G. Hopkins’ efforts In this context, change/differences of b. How? These percentage chances and
over the past 15 years, has changed my life, 1x, 3x, 6x and 10x SWC can be considered associated qualitative interpretations
both as an academic and practitioner in as small, moderate, large and very large, are generally set a priori (e.g. <1%, almost
elite sport11. I personally hope that MBI is respectively4. certainly not; 1 to 5%, very unlikely; 5 to
influential with other scientists, as it has b. How? The most appropriate method to 25%, probably not; 25 to 75%, possible; 75
been to me. While the debate will likely define it is however variable-dependent, to 95%, likely; 95 to 99, very likely; >99%,
continue, MBI is today a well-established which forces researchers to adopt a almost certain)
analytical approach in sports science and conscious process when analysing their c. Practically: these percentage chances
in other fields, particularly clinical medicine data. “NHST is easy, but misleading. can be obtained with only a few copy and
where practical/clinical significance often MBI is hard, but honest” (W.G. paste manoeuvres using a specifically-
takes priority over statistical significance4. Hopkins, personal communication)4. designed spreadsheet freely available
38
Figure 1: Example of possible
decisions when interpreting
Case 5 Unclear changes using magnitude-
based inferences. Note the
clear vs unclear cases (based
Case 4 Possible increase
on confidence limits, in
relation to the shaded trivial
Case 3 Likely increase area), which is firstly, the
beauty of magnitude-based
inferences and, secondly, not
Case 2 Unclear possible via null hypothesis
significance testing. Note
also how, for clear effects,
Case 1 Clearly trivial the likelihood of changes
increases as the confidence
limits shrink. Reprinted with
permission from McCormack
-3 -2 -1 0 1 2 3 4 5 6
et al5.
Change (%)
95
online13,14. Final decisions can then be applied, the greatest challenge for sport b. Extra decimals and ‘noise’ removed for
translated into plain language when scientists is to find the most efficient type clarity (Table 3).
chatting with coaches: ‘This attacker of data visualisation and reporting to get c. All text written horizontally for
has very likely increased his sprinting their message across. Several considerations readability (Figure 3b).
speed. The magnitude of improvement to optimise tables, graphs and content d. Labels added to graphs so that exact
should be enough for him to win a few presentation are discussed below and values can be seen too (graph for
more balls during matches.’ illustrated in Table 3 and Figure 3. patterns, numbers for details, if
1. Reports should be as simple and as required) (Figure 3b).
PRESENTING THE DATA informative as possible (‘simple but e. Meaningful changes or differences
Similar to the aphorism that all roads lead powerful’): highlighted to be seen at a glance
to (and therefore from) Rome, the same data a. Limited to a few ‘important’ variables (Figure 2) – with different possible
and results set can be presented in many (those that can be used to answer the levels of data analysis. Microsoft
ways (Figure 3). Once the relevant questions questions that coaches and athletes Excel’s conditional formatting de-
have been identified, the best variables have have actually asked and can have an picting MBI is a useful example
been selected and the appropriate statistics impact on the programme). (Table 2).
Table 3
40
0 0
Player 1
Player 2
Player 3
Player 4
Player 5
Player 6
Player 7
Player 8
Player 9
Player 10
Player 11
Player 12
Player 13
Player 14
Player 15
Player 1
Player 2
Player 3
Player 4
Player 5
Player 6
Player 7
Player 8
Player 9
Player 10
Player 11
Player 12
Player 13
Player 14
Player 15
a b
12000
9994
8000 Player 11 9673
Player 11 9673
Player 10 110947 Player 10 110947
Player 9 10235 Player 9 10235
6000
Player 8 10649
Player 8 10649
Player 7 10704 Player 7 10704
Player
40006 11890 Player 6 11890
Player 5 8030 Player 5 8030
Player 4 12034
Player 4 12034
2000
Player 3 11569
Player 3 11569
Player 2 9891 Player 2 9891
Player 1 11560
Player 1 11560
0
0 2000 4000 6000 8000 10000 12000 14000
Player 1
Player 2
Player 3
Player 4
Player 5
Player 6
Player 7
Player 8
Player 9
Player 10
Player 11
Player 12
Player 13
Player 14
Player 15
0 2000 4000 6000 8000 10000 12000 14000
Player 12
10235 Player 12 9994 +11
Player 8
9994
10649 supporting
+118
Player 119673
staff and athletes often don’t
Player 11 9673
Player 7
Player 10 110947
10704 know what to expect from
Player 10 scientific support
110947
Player 96 Player 9 +1648
Player 10235
11890
at the club, it is 10235
only by sitting right next
Player 85
Player 8030
-362Player 8 10649 -362
Player 74
10649
12034 to them during training
Player 7 sessions and team
10704
Player 10704
Player 63
Player 11569
11890 debriefs, by sharing meals 11890
Player 6 +1206 and coffees,
Player 52 Player 5
Player 8030
9891
-782 being with them in the 'trenches' that-782
8030 sport
Player 41
Player 11560 Player 4 +1582 12034
12034
scientists can appreciate 11569
Player 3 +1612 what coaches
Player 3 11569
Player 2 0 2000 4000 9891
6000 8000 10000 12000 14000 and athletes
-304 Player 2 9891 may find useful and which -304
Table 4
Changes + Excel
Rounded % HRmax Changes + Excel Changes + Excel
conditional
HR + Excel + Excel conditional conditional
Date Raw HR Rounded HR formatting
conditional conditional formatting (when formatting (when
(based on
formatting formatting >SWC) > SWC+TE)
inferences)
Table 4: Example of various levels of data reporting using changes in submaximal heart rate responses to
a standardised submaximal run. The level of clarity and usefulness increases from left to right. Individual
changes in submaximal heart rate in a professional soccer player when running at 12 km/h throughout
two competitive seasons (% of maximal heart rate). Adapted from Buchheit, 20164. SWC=smallest
worthwhile change (1%)19, TE=typical error of measurement (3%)19. A change that is >SWC+TE has a
75% likelihood to be true4. The number of * indicates the likelihood for the changes to be substantial,
with ** referring to likely changes, and *** to very likely changes, using a specifically designed
spreadsheet freely available on the internet12. Data in the far right column are displayed in Figure 2.
42
CONCLUSION Null hypothesis
significance testing is
The value and importance of sport
science varies greatly between elite clubs
and federations. Among the different
components of effective sport science
support, the three most important elements easy, but misleading.
Magnitude-based
are likely the following:
1. Appropriate understanding and
analysis of the data; i.e. using the most
important and useful metrics only and
using magnitude-based inferences inferences are hard,
but honest
as statistics. In fact, traditional null
hypothesis significance testing (P
values) is neither appropriate to answer
the types of questions that arise from
the field (i.e. assess magnitude of
effects and examine small sample sizes)
nor to assess changes in individual information for program design. Strength 15. Buchheit M, Allen A, Poon TK, Modonutti
performances. Cond J 2013; 35:7-14. M, Gregson W, Di Salvo V. Integrating
7. Pettitt RW. The standard difference score: different tracking systems in football:
2. Attractive and informative reports
a new statistic for evaluating strength multiple camera semi-automatic system,
via improved data presentation/
and conditioning programs. J Strength local position measurement and GPS
visualisation (‘simple but powerful’).
Cond Res 2010; 24:287-291. technologies. J Sports Sci 2014; 32:1844-
3. Appropriate communication skills
1857.
and personality traits that help to 8. Al Haddad H, Simpson BM, Buchheit M.
deliver data and reports to coaches and Monitoring changes in jump and sprint 16. Buchheit M, Morgan W, Wallace J, Bode
athletes. Developing such an individual performance: best or average values? Int M, Poulos N. Physiological, psychometric,
profile requires time, effort and most J Sports Physiol Perform 2015; 10:931-934. and performance effects of the Christmas
break in Australian football. Int J Sports
importantly, humility. 9. Hopkins WG. How to interpret changes in Physiol Perform 2015; 10:120-123.
an athletic performance test. Sportscience
2004; 8:1-7. 17. Haugen T, Buchheit M. Sprint running
performance monitoring: methodological
10. Cohen J. Things I have learned (so far). Am and practical considerations. Sports Med
Psychol 1994; 45:1304-1312. 2016; 46:641-656.
References 11. Buchheit M. Any Comments? 18. Hopkins WG, Marshall SW, Batterham
1. Buchheit M. Chasing the 0.2. Int J Sports 2013. Available from: www. AM, Hanin J. Progressive statistics for
Physiol Perform 2016; 11:417-418. herearemycomments.wordpress.com/. studies in sports medicine and exercise
[Accessed 16 March 2016]. science. Med Sci Sports Exerc 2009; 41:3-13.
2. Batterham AM, Hopkins WG. Making
meaningful inferences about 12. Hopkins WG. Statistical vs clinical or 19. Buchheit M. Monitoring training status
magnitudes. Int J Sports Physiol Perform practical significance [Powerpoint with HR measures: do all roads lead to
2006; 1:50-57. presentation]. Sportscience 2002; 6. Rome? Front Physiol 2014; 27:73.
Available from: www.sportsci.org/
3. Buchheit M, Simpson B. Player tracking
jour/0201/Statistical_vs_clinical.ppt
technology: half-full or half-empty glass?
. Int J Sports Physiol Perform 2016 [In 13. Hopkins WG. Precision of the estimate of
press]. a subject's true value [Excel spreadsheet].
In: Internet Society for Sport Science.
4. Buchheit M. The numbers will love you
Sportscience 2000. Available from:
back in return – I promise. Int J Sports
w w w. s p o r t s c i . o r g / r e s o ur c e / s t at s /
Physiol Perform 2016; 11:551-554.
xprecisionsubject.xls2000 [Accessed
5. McCormack J, Vandermeer B, Allan November 2016].
GM. How confidence intervals become Martin Buchheit Ph.D.
14. Hopkins WG. A spreadsheet for deriving
confusion intervals. BMC Med Res
a confidence interval, mechanistic Head of Performance
Methodol 2013; 13:134.
inference and clinical inference from
Paris Saint Germain Football Club
6. McGuigan MR, Cormack SJ, Gill ND. a P value. Sportscience 2007; 11:16-20.
Strength and power profiling of athletes: Available from: www.sportsci.org/2007/ Paris, France
selecting tests and how to use the wghinf.htm [Accessed November 2016]. Contact: [email protected]