History of previous concussion and sports-specific skills in youth ice hockey players

Abstract

Concussions can lead to cognitive or neuromotor impairments which may influence skill performance. Few studies have investigated concussion and sports-specific skill performance, particularly in youth. Therefore, the purpose of this study was to examine previous concussion and components of the Hockey Canada Skills Test, a measure of ice hockey-specific skill performance, in youth ice hockey players (ages 11-17). A secondary purpose was todetermine the test-retest reliability of these components. Players completed a detailed baseline questionnaire on previous concussion history. On-ice measures included forward agility weave, forward/backward speed skate, transition agility, and a 6-repeat endurance skate (all measured in seconds). Multiple linear regression was conducted to examine history of concussion, number of previous concussions, time since most recent concussion, and severity of most recent concussion on on-ice performance. Test-retest reliability was assessed using intraclass correlation coefficients and mean differences with Bland-Altman Limits of Agreement. In total, 596 participants [525 males and 71 females, representing elite (upper 30% by division of play) and non-elite (lower 70%)] were recruited to examine the primary purpose. History of concussion (yes/no) and time since most recent concussion was not associated with any component. Players reporting 2 or more concussions were significantly faster than those with no previous concussion on forward agility weave with the puck. For every additional day to return to play post-concussion, player times were significantly faster on forward agility weave with and without the puck, transition agility without the puck, and backward speed with and without the puck. The intraclass correlation coefficients ranged from 0.50 to 0.92 and the Bland-Altman Limits of Agreement varied by component. These findings indicate players with and without history of concussion have similar on-ice scores, and that the components of the Hockey Canada Skills Test are a reliable measure of on-ice performance.

Headnote

Abstract:

Concussions can lead to cognitive or neuromotor impairments which may influence skill performance. Few studies have investigated concussion and sports-specific skill performance, particularly in youth. Therefore, the purpose of this study was to examine previous concussion and components of the Hockey Canada Skills Test, a measure of ice hockey-specific skill performance, in youth ice hockey players (ages 11-17). A secondary purpose was todetermine the test-retest reliability of these components. Players completed a detailed baseline questionnaire on previous concussion history. On-ice measures included forward agility weave, forward/backward speed skate, transition agility, and a 6-repeat endurance skate (all measured in seconds). Multiple linear regression was conducted to examine history of concussion, number of previous concussions, time since most recent concussion, and severity of most recent concussion on on-ice performance. Test-retest reliability was assessed using intraclass correlation coefficients and mean differences with Bland-Altman Limits of Agreement. In total, 596 participants [525 males and 71 females, representing elite (upper 30% by division of play) and non-elite (lower 70%)] were recruited to examine the primary purpose. History of concussion (yes/no) and time since most recent concussion was not associated with any component. Players reporting 2 or more concussions were significantly faster than those with no previous concussion on forward agility weave with the puck. For every additional day to return to play post-concussion, player times were significantly faster on forward agility weave with and without the puck, transition agility without the puck, and backward speed with and without the puck. The intraclass correlation coefficients ranged from 0.50 to 0.92 and the Bland-Altman Limits of Agreement varied by component. These findings indicate players with and without history of concussion have similar on-ice scores, and that the components of the Hockey Canada Skills Test are a reliable measure of on-ice performance.

Keywords: ice hockey; concussion; adolescent; reliability; sports-specific testing

Introduction

Ice hockey is a popular winter sport with over a million youth participating annually("International Ice Hockey Federation. Survey of Players," 2019). While participation in ice hockey is associated with many benefits, the rates of injury are amongst the highest in youth sport with up to 6 injuries/1000 game-hours and 2.79 concussions/1000 game-hours in body checking leagues(Black, Hagel, Palacios-Derflingher, Schneider, & Emery, 2017; C. Emery, Hagel, Decloe, & Carly, 2010; C. Emery, Kang, et al., 2010; C. Emery & Meeuwisse, 2006; C. A. Emery et al., 2017; Pfister, Pfister, Hagel, Ghali, & Ronksley, 2016).While the risk factors and injury prevention strategies for concussion have been investigated for youth ice hockey(Black et al., 2017; Black et al., 2016; C Emery et al., 2014; C. Emery, Hagel, et al., 2010; C. Emery, Kang, et al., 2010; C. Emery et al., 2011; C. Emery & Meeuwisse, 2006), the long-term sequelae of concussion, is less understood.

International consensus supports that most concussions typically resolve from a clinical perspective within one month, but suggests physiologic recovery may exceed clinical recovery time (Kamins et al., 2017; McCrory et al., 2017).While further research is required to understand the clinical implications of this, it does suggest that concussions may lead to subtle, lingering neurologic deficits. These deficits may in turn negatively affect the athletes playing performance upon return to sport. Indeed, the topic of performance measures following concussion is an emerging field and has been examined in various professional sports such as Australian Football (Makdissi, McCrary, Ugoni, Darby, & Brnkner, 2009), American Football(Kumar et al., 2014; Reams, Hayward, Kutcher, & Burke, 2017),Major League Soccer(Hardy, Jordan, Wolf, Johnson, & Brand, 2017),Major League Baseball(Wasserman, Abar, Shah, Wasserman, & Bazarian, 2015), and the National Hockey League(Buckley et al., 2019; Kuhn, Zuckerman, Totten, & Solomon, 2016; Navarro et al., 2018; Van Pelt et al., 2019).The results of these studies have been mixed, with some suggesting impaired performance after returning to play from concussion, while others found no detrimental effects. However, no study has yet to examine concussion and performance at the youth level.

At the 1999 Molson Open Ice Summit on Player Development, it was recommended that Hockey Canada, the national governing body of ice hockey in Canada, develop a nationally organized program to test and recognize skill acquisition and skill improvement in youth ice hockey players("Hockey Canada. Skills Testing," 2020). In response, Hockey Canada developed and launched the National Skills Standards and Testing Program (NSST) as a method to measure specific on-ice skills in youth players, with the goal of using the results to provide positive individual feedback. This feedback would allow for further development of the fundamental skills required to play ice hockey at the youth level("Hockey Canada. Skills Testing," 2020). The NSST is commonly referred to and named hereafter as the Hockey Canada Skills Test (HCST). The HCST is one of the first structured protocols designed to measure a variety of essential hockey skills and provides a unique opportunity to quantify ice hockey-specific performance. A benefit of using the HCST protocol as an indicator of on-ice performance is that it does not use traditional point-based performance statistics (e.g., goals, assists),which many of the previously cited professional studies used, as these performance measures can fluctuate regardless of the health of the player. Many additional factors contribute to a players' point-based performance such as quality of teammates, coaching style, and the tactical system used by the coach. Further, player performance in ice hockey is a combination of many factors both with and without the puck, which can be difficult to measure. Given these complexities, measuring the fundamental sports-specific skills through the HCST might be a better indicator of performance and could better represent the association between injury and skill performance, particularly at the youth level. As such, the purpose of this study was to examine the association between previous concussion history and sports-specific performance using the HCST in youth ice hockey players. Additionally, as the test-retest reliability of the HCST components in youth ice hockey players has not been previously assessed, a secondary purpose was to determine the test-retest reliability of the HCST in youth ice hockey players.

Methods

Participants

The study sample included youth ice hockey teams that were recruited by Hockey Canada and teams from a larger longitudinal cohort study. Teams were recruited from elite (upper 30% by division of play) and non-elite (lower 70%) Pee Wee (aged 11-12), Bantam (aged 13-14), and Midget (aged 15-17) age categories. Three elite Bantam ice hockey teams (ages 13-14; AA and AAA; top 20% by division of play) were also recruited to complete the test-retest reliability component of this study. Ethical approval was received and all participants provided written informed assent along with parent/guardian written informed consent

Off-ice Measurements

Players completed a validated baseline questionnaire that included demographics (e.g., age, height, weight, sex), playing history, any musculoskeletal (MSK) injury requiring medical attention or at least one day of missed participation from sport or physical activity within the past 12 months, and lifetime history of concussion(Black et al., 2016; C. Emery, Hagel, et al., 2010; C. Emery et al., 2011; C. Emery & Meeuwisse, 2006; Meeuwisse & Love, 1998). Specifically, concussion history was assessed by asking whether players "have ever had a concussion or been 'knocked out' or had their 'bell rung'", and if so, how many concussions they had sustained, when each occurred, and the time loss (in days) before full return to sport for each concussion. Since some players may have sustained a concussion from the time the baseline questionnaire was completed and HCST testing, injury data, including concussion, was collected prospectively from the time of the questionnaire completion and included in the analyses.

On-ice Measurements (Hockey Canada Skills Test)

Components of the HCST that were tested included a forward agility weave, agility transition skate, forward and backward speed skate, and a 6-repeat endurance skate, all measured in seconds. The four skill testing stations were completed during one ice session (typically one hour). Three of the testing stations, the agility weave, agility transition skate, and speed skate (both forwards and backwards) were administered simultaneously at the beginning of the ice session and were measured twice, once without a puck and once with. The final station, the 6-repeat endurance skate, was run after the initial three tests were completed and was measured only once, without the puck. A measurement from the 6-repeat endurance skate ("drop-off time") indicates players' endurance, and was measured by subtracting the time of the slowest lap from the time of the fastest lap. The dimensions of each drill as well as each drill description can be found on Hockey Canada's NSST website: https://www.hockeycanada.ca/en-ca/Hockey-Programs/Players/Skills-Testing.

Research assistants were trained to set up and measure the HCST components by Hockey Canada personnel. The training session included both off-ice descriptions and on-ice practice so research assistants were comfortable with the protocol and to ensure consistency in recording results. Each team completed testing on freshly resurfaced ice. The distances and locations of the pylons used for drills were measured and marked with spray paint on the ice to ensure the pylons could be replaced in the same location if knocked out of position during testing. To prevent ruts forming in the ice during the transition and forward agility weave, multiple sets of dots were measured and marked approximately one metre apart. This allowed each group of players a fresh patch of ice for testing. In addition, if teams were tested back-to-back on the same ice surface, the location of the testing stations was rotated to reduce the number of ruts formed. Timing personnel measured each component using handheld stopwatches accurate to one-hundredth of a second. Players, including goaltenders, wore full equipment and were divided into three groups. Each group started at one of the three beginning stations and were given a pre-drill demonstration. All players completed the drill without a puck first and then again with the puck, which allowed the players a rest time between repetitions. If a player fell, stumbled, or lost the puck during a drill, they had the option of either continuing the drill or starting over again. If a player knocked a cone out of place during a trial, his/her time was only allowed if he/she continued the drill properly. If timing personnel deemed a player did not correctly complete the drill, the player was instructed to start the drill over again. A maximum of three attempts was allowed on any one testing station. If a player was unable to finish the drill in three attempts, they received a "no time" on that station. If during the final station, the 6 repeat, a player was unable to complete a lap in the maximum allotted time of 30 seconds, or the player stopped of their own accord, they received no time for that lap or any additional lap(s) remaining.

Statistical Analyses

All statistical analyses were carried out using STATA 13 (StataCorp. 2013. Stata Statistical Software: Release 13. College Station).Baseline characteristics were stratified by level of play and history of concussion (yes/no). Times on HCST components were analysed as a continuous variable using multiple linear regression models and adjusted for relative age, sex, level of play, position, elite/non-elite, and previous MSK injury within the last year, while accounting for clustering by team. Separate models were conducted to examine history of concussion (yes/no), number of previous concussions, days since most recent concussion, and days of recovery following most recent concussion. A Bonferroni correction was used to account for multiple comparisons for each outcome separately (i.e., 0.05/9=0.0056). The number of subjects per variable in each model was sufficient to calculate accurate regression coefficients and standard errors(Austin & Steyerberg, 2015).

Descriptive statistics (median, quartiles) depict characteristics of participants who completed the testretest reliability component. Test-retest reliability for the time on each HCST component was assessed using intraclass correlation coefficients (ICCs) with 95% confidence intervals (CIs) and Bland-Altman plots with 95% Limits of Agreement (LOA). The ICCs were interpreted using previously published guidelines; <0.40 (poor), 0.40-0.75 (moderate), 0.75-0.90 (good), and >0.90 (excellent) (Shrout & Fleiss, 1979). The differences and magnitude between on-ice times were examined using the Bland-Altman methods of agreement and 95% LOA(Altman & Bland, 1983; Bland & Altman, 1986). Reliability estimates were based on measurements taken at baseline and again 7-10 days later.

Results

History of Concussion and HCST

In total, 596 players from 65 teams were recruited to participate from three levels of play (Pee Wee, Bantam, and Midget). Of these, 525 (88%) participants were male and 72 (12%) were female, with 202 players reporting a history of concussion (34%) while 394 (66%) reported no history. Due to the small numbers of players reporting 2 and 3 previous concussions, these groups were collapsed to a "2 or more" category for analysis. Participant characteristics are summarized in Table I.

The multiple linear regression analyses suggest that previous history of concussion (yes/no) was not associated with performance for any HCST outcome. Players reporting 2+ concussions were faster than those with no history on the component forward agility weave with the puck (mean difference 7.317 seconds; 95% CI 11.047-3.586). Number of previous concussions did not predict performance on any other HCST outcome. Time since a player's most recent concussion did not have a statistically significant independent association with any HCST component. Length of recovery time following the most recent reported concussion was associated with performance on the HCST components: forward agility weave without (0.083 seconds; 95% CI 0.131-0.036) and with the puck (0.107 seconds; 95% CI 0.160-0.055), transition agility without the puck (0.01 seconds; 95% CI 0.018-0.004)), and backward speed without (0.05 seconds; 0.074-0.026) and with the puck (0.06 seconds; 95% CI 0.097-0.032). For each of these components, player's performance is expected to improve for every additional day of reported recovery.

Test-Retest Reliability

Twenty-three players participated in both sessions of the test-retest reliability component. Reliability participant characteristics are summarized in Table II.

Intraclass correlation coefficients ranged from 0.50-0.92. Transition agility without the puck showed the highest ICC value of 0.92 (95% CI 0.85-0.98), whereas 6-repeat drop-off time showed the lowest ICC of 0.50 (95% CI 0.19-0.80). Table II outlines the mean difference between the times taken to complete HCST component tests ranged from less than 0.01 seconds (backward speed without the puck) to -0.40 seconds (transition agility with the puck). The Bland-Altman 95% LOA suggests that the second time will fall within approximately one second of the score from the first test session for most HCST components. In addition to having the largest mean difference, the transition agility weave with the puck also had the largest 95% LOA ranging from -3.89 to 3.10 seconds (Table III).

Discussion

This is the first study to examine the association between history of concussion and HCST components, as well as the test-retest reliability of these components in youth ice hockey players. Sport performance measures after concussion has been a growing topic in the literature. Makdissi et al. (2009), Kumar et al. (2014), and Reams et al. (2017) all reported no change in performance metrics in professional male athletes after concussion in Australian and American Football. This was contrary to Wasserman et al. (2015) and Hardy et al. (2017) who found reductions in batting and soccer performance metrics in Major League Baseball and Major League Soccer players, respectively. The literature investigating professional ice hockey has been mixed, with two studies finding differences in performance metrics after concussion, while two other studies found none. Specifically, Kuhn et al. (2016) found no differences in a variety of performance metrics comparing the 5 games before the concussion occurred and the 5 games after return, and Buckley et al. (2019) found no differences using more advanced metrics comparing the pre-injury and post-return performance between 5-games, 10-games, and remainder of the season. These results are contrary to Navarro et al. (2018) who found a reduction in the number of total points in the season after sustaining a concussion, and Van Pelt et al. (2019)who found an initial reduction in performance metrics in the 1-2 weeks after return to play but an improved performance at weeks 56. The results of the current study suggest that previous history of concussion was not associated with negative HCST performance, thus agreeing with the results of Kuhn et al. (2016) and Buckley et al. (2019). In fact, players reporting 2+ concussions were found to be significantly faster than those with no history on the component forward agility weave with the puck. Potentially, this result may be related to selection bias, in that those players with a history of 2+ concussions may also be more motivated to rehabilitate to full recovery than players with no history of concussion or only one reported concussion. Number of previous concussions did not significantly predict performance on any other HCST outcome. Additionally, the time since a player's most recent concussion was not associated with any HCST component suggesting that there are no performance differences between a player who sustained a concussion more recently compared with a player who sustained a concussion less recently. Finally, time of recovery following most recent concussion significantly predicted performance on the following HCST components: forward agility weave without and with the puck, transition agility without the puck, and backward speed without and with the puck. For every day of reported recovery following a concussion a player's time is expected to improve by 0.084 seconds on forward agility without the puck, 0.107 seconds on forward agility with the puck, 0.011 seconds on transition agility without the puck, 0.05 seconds on backward speed without the puck, and 0.065 seconds on backward speed with the puck. In context, compared with a player that took 10 days to recover after sustaining their most recent concussion, a player that took 30 days to recover is expected to score approximately 1.68 seconds faster on forward agility without the puck, 2.14 seconds faster on forward agility with the puck, 0.22 seconds faster on transition agility without the puck, 1 second faster on backward speed without the puck, and 1.3 seconds faster on backward speed with the puck. It is surprising to have results where participants with more prolonged recovery had better performance; however, taking more time to adequately rehabilitate following concussion may have led to improved on-ice performance. For the test-retest reliability, the ICCs ranged from 0.50-0.92 (moderate to excellent) and the mean differences for the individual times between testing sessions were quite small, ranging from less than 0.01 seconds for backward speed without the puck (95% CI 0.72-0.73) to -0.40 seconds for transition agility weave with the puck (95% CI -3.89-3.10). Reed et al. (1979) and Watson and Sargeant (1986) previously reported high test-retest correlation coefficients on the 6-repeat drop-off time (r=0.78 and r=0.96, respectively)(Reed et al., 1979; Watson & Sargeant, 1986). Interestingly in the present study, the 6-repeat drop-off time had the lowest ICC value of all HCST components measured (0.50; 95% CI 0.19-0.80).

Despite studies suggesting the importance of measuring agility on-ice (Behm, Wahl, Button, Power, & Anderson, 2005; Bracko, 2001; Bracko & George, 2001), this skill has rarely been tested on-ice for ice hockey players. One method previously described in the literature has been the 'cornering S-skate'. Originally developed by Greer et al. (1992), it involves players skating an S-shape course at full speed(Greer, Serfass, Picconatto, & Blatherwick, 1992). Although two studies have found this method to have good test-retest reliability in experienced players (r=0.95 and r=0.96) (Farlinger, Kruisselbrink, & Fowles, 2007; Greer et al., 1992), one study found much lower test-retest reliability in female players of mixed age and ability (r=0.64)(Bracko & George, 2001). In the present study, the HCST component forward agility weave had an ICC of 0.89 (95% CI 0.81-0.98) when tested without the puck, and an ICC of 0.85 (95% CI 0.74-0.97) with the puck. The cornering Sskate has been criticized in the literature as being too complex for less experienced players to maintain balance at full speed, particularly during cornering (Farlinger et al., 2007). Further, Farlinger et al. (2007) found that the cornering S-test did not differ enough from a straight sprint skate to measure agility and, therefore, argues it is not a true measure of on-ice agility. Due to the relatively simple design of the HCST forward agility weave component, it may be a better indicator of agility than the cornering S-test.

While ICCs are commonly calculated and reported as a measure of reliability(Atkinson & Nevill, 1998), there are limitations with them as they are influenced by the variance among the participants' scores(Bland & Altman, 1996), do not account for independence between measurement variability and the magnitude of measurement, and are without units of measurement(Bland & Altman, 1999). For these reasons, interpreting relative reliability through ICC's alone is difficult, though were calculated in the present study to provide reference to previous reliability studies specific to on-ice performance testing. A more ideal way to measure absolute test-retest reliability is using the Bland-Altman method which gives more context than just an ICC value. The 95% LOA varied for each HCST component in the present study, with transition agility weave with the puck having the widest range at approximately ± 3 seconds (for reference, this test took approximately 17 seconds for Bantam players to complete).The LOA for all other components were narrower and ranged approximately ± 1 second. The LOA suggests that 95% of individuals would need a change of approximately 1-3 seconds in either direction to be indicative of a change greater than what would be expected over 7-10 days in youth ice hockey players. Transition agility weave with the puck may be arguably the most technically difficult skill component of the HCST and may be why it showed the largest LOA. This test requires many skills for its successful completion such as: forward and backward skating, stopping, and pivoting from forward skating to backward skating and vice versa, all while maintaining control of the puck. The mean difference in time between the two sessions on transition agility weave without the puck (Figure I, supplemental content) was smaller in players who were slower, suggesting slower players' times were more consistent than faster players. It is not clear why this trend exists. The distribution of the differences of scores seemed to be unrelated to the magnitude of the scores for all other outcomes (Figures II-IX, supplemental content).

Limitations

The chief limitation of using a cross-sectional study design is that it does not allow for the establishment of temporality. As a result, causality between concussion and performance could not be evaluated in the present study. There is the potential for survival bias, as players that completed testing were still actively involved in sport. This study was also subject to potential measurement bias given the baseline questionnaire relied on self-report. Self-report bias and non-differential error from missing data may have influenced the results of this study. Finally, as teams were recruited throughout the playing season, some teams completed the testing protocol earlier in the season than others. It is likely that teams will have had a different number of practices, games, and/or overall training sessions before they completed the HCST, which was not controlled for in this study. While the current study has identified the HCST to be reliable measure of on-ice performance, only the 6-repeat drop-off time has been assessed as a valid measure of a players anaerobic capacity(Reed et al., 1979). It is also important to understand that the HCST only measures one aspect of players' total performance. A more global understanding of players' performance should include both in vitro (standardized settings) and in vivo (game play) approaches, with both quantitative and qualitative ratings(Nadeau, Richard, & Godbout, 2008), and the consideration of tactical skills. The use of these more global performance measurements and concurrent validation remains a future direction for studies using the HCST. Despite the attempt to standardize the testing protocol and testing procedures between test sessions for the reliability component (i.e., players were assessed on the same ice surface and at the same time of day), some players may not have followed the same order of drills at both sessions. This was due to the different number of players attending the sessions and the need to divide the players into three equally sized groups for testing to finish on time. The endurance skate, however, was always completed after the other three components. Further, there may have been slight variations in the arena's temperature, lighting, and ambient noise between the testing sessions, which may have affected the times of the players. Physical activity prior to HCST was unaccounted for, which, while unlikely to have affected the initial three HCST components, may have affected the more strenuous 6-repeat endurance skate. However, physical activity on the day of the initial or repeat testing was likely minimal as both testing sessions were completed during the morning or early afternoon on school days. Inter-rater reliability of the HCST components was not assessed as it was assumed the raters did not influence the measurement system. Even though we are assuming high consistency among the timing personnel, basic human reaction time using the stopwatches may have influenced the times of the players. This may be especially true when measuring quick duration tests such as the forward/backward speed skate where the times are approximately 6-7 seconds. Although hand stopwatches are not the ideal way to measure time on these components, they are easy to use and practical for use on-ice.

Conclusions

History of concussion and time since most recent concussion was not associated with any HCST components. Greater reported time loss following concussion and a history of multiple (i.e., 2+) concussions was associated with better performance. This finding requires further investigation, but better performance in these on-ice skills may suggest more complete rehabilitation before return to play for players with longer reported recovery time and multiple concussions. The mean time differences between the test-retest sessions were small, and the LOA suggests that 95% of individuals would need a change of approximately 1-3 seconds in either direction to be indicative of a change greater than what would be expected over 7-10 days. This suggests the HCST is a reliable method to measure on-ice sports-specific skills in youth ice hockey players.

Acknowledgments

The Sport Injury Prevention Research Centre is one of the International Research Centres for Prevention of Injury and Protection of Athlete Health supported by the International Olympic Committee. We acknowledge the funding from Canadian Institutes of Health Research, Alberta Innovates Health Solutions, Hotchkiss Brain Institute, and Alberta Children's Hospital Foundation.

The authors would like to acknowledge Hockey Canada and particularly Paul Carson for his help with this project. Additionally, we would like to acknowledge all research students as well as Hockey Calgary, Edge School for Athletes, and all parents, coaches, and players for their time and support.

Disclosure statement The authors declare no conflicts of interest.

Sidebar

Published online: July 31, 2020

(Accepted for publication: July 22, 2020)

References

References

Altman, D. G, & Bland, J. M. (1983). Measurement in medicine: the analysis of method comparison studies. The statistician, 307-317.

Atkinson, G, & Nevill, A. M. (1998). Statistical methods for assessing measurement error (reliability) in variables relevant to sports medicine. Sports Med, 26(4), 217-238.

Austin, P. C., & Steyerberg, E. W. (2015). The number of subjects per variable required in linear regression analyses. Journal of clinical epidemiology, 68(6), 627-636.

Behm, D. G, Wahl, M. J., Button, D. C., Power, K. E., & Anderson, K. G (2005). Relationship between hockey skating speed and selected performance measures. The Journal of Strength & Conditioning Research, 19(2), 326-331.

Black, A. M., Hagel, B. E., Palacios-Derflingher, L., Schneider, K. J., & Emery, C. A. (2017). The risk of injury associated with body checking among Pee Wee ice hockey players: an evaluation of Hockey Canada's national body checking policy change. Br J Sports Med. doi:10.1136/bjsports-2016-097392

Black, A. M., Macpherson, A. K., Hagel, B. E., Romiti, M. A., Palacios-Derflingher, L., Kang, J., . . . Emery, C. A. (2016). Policy change eliminating body checking in non-elite ice hockey leads to a threefold reduction in injury and concussion risk in 11- and 12-year-old players. Br J Sports Med, 50(1), 55-61. doi:http://dx.doi.org/10.1136/bjsports-2015-095103

Bland, J. M., & Altman, D. (1986). Statistical methods for assessing agreement between two methods of clinical measurement. The lancet, 327(8476), 307-310.

Bland, J. M., & Altman, D. G. (1996). Measurement error and correlation coefficients. BMJ: British Medical Journal, 313(7048), 41.

Bland, J. M., & Altman, D. G. (1999). Measuring agreement in method comparison studies. Statistical methods in medical research, 8(2), 135-160.

Bracko, M. R. (2001). On-ice performance characteristics of elite and non-elite women's ice hockey players. The Journal of Strength & Conditioning Research, 15(1), 42-47.

Bracko, M. R., & George, J. D. (2001). Prediction of Ice Skating Performance With OffIce Testing in Women's Ice Hockey Players. The Journal of Strength & Conditioning Research, 15(1), 116-122.

Buckley, T. A., Bryk, K. N., Van Pelt, K. L., Broglio, S. P., East, S. A., Zuckerman, S. L., & Kuhn, A. W. (2019). Concussion and National Hockey League player performance: an advanced hockey metrics analysis. J Athl Train, 54(5), 527-533.

Emery, C., Black, A., Macpherson, A., Kang, J., Hagel, B., Romiti, M., & Meeuwisse, W. (2014). Evaluation of a body checking policy change as an injury prevention strategy for non-elite youth ice hockey players. Br J Sports Med, 48(7), 591-591.

Emery, C., Hagel, B., Decloe, M., & Carly, M. (2010). Risk factors for injury and severe injury in youth ice hockey: a systematic review of the literature. Inj Prev, 16(2), 113-118. doi:10.1136/ip.2009.022764

Emery, C., Kang, J., Shrier, I., Goulet, C., Hagel, B., Benson, B., . . . Meeuwisse, W. (2010). Risk of injury associated with body checking among youth ice hockey players. Jama, 303(22), 2265-2272. doi:10.1001/jama.2010.755

Emery, C., Kang, J., Shrier, I., Goulet, C., Hagel, B., Benson, B., . . . Meeuwisse, W. (2011). Risk of injury associated with bodychecking experience among youth hockey players. Cmaj, 183(11), 1249-1256. doi:10.1503/cmaj.101540

Emery, C., & Meeuwisse, W. (2006). Injury rates, risk factors, and mechanisms of injury in minor hockey. Am J Sports Med, 34(12), 1960-1969. doi:10.1177/0363546506290061

Emery, C. A., Black, A. M., Kolstad, A., Martinez, G., Nettel-Aguirre, A., Engebretsen, L., . . . Tator, C. (2017). What strategies can be used to effectively reduce the risk of concussion in sport? A systematic review. Br J Sports Med, 51(12), 978-984.

Farlinger, C. M., Kruisselbrink, L. D., & Fowles, J. R. (2007). Relationships to skating performance in competitive hockey players. The Journal of Strength & Conditioning Research, 21(3), 915-922.

Greer, N., Serfass, R., Picconatto, W., & Blatherwick, J. (1992). The effects of a hockey-specific training program on performance of Bantam players. Canadian journal of sport sciences= Journal canadien des sciences du sport, 17(1), 65-69.

Hardy, R., Jordan, T., Wolf, A., Johnson, M., & Brand, J. (2017). Concussions and their effects on performance measures of Major League Soccer players: a teaching tool for physical education teachers. The Physical Educator, 74(1).

Hockey Canada. Skills Testing. (2020). Retrieved from https://www.hockeycanada.ca/en-ca/HockeyPrograms/Players/Skills-Testing

International Ice Hockey Federation. Survey of Players. (2019). Retrieved from https://www.iihf.com/en/static/5324/survey-of-players

Kamins, J., Bigler, E., Covassin, T., Henry, L., Kemp, S., Leddy, J. J., . . . Giza, C. C. (2017). What is the physiological time to recovery after concussion? A systematic review. Br J Sports Med, 51(12), 935-940. doi:10.1136/bjsports-2016-097464

Kuhn, A. W., Zuckerman, S. L., Totten, D., & Solomon, G. S. (2016). Performance and style of play after returning from concussion in the National Hockey League. Am J Sports Med, 44(8), 2152-2157.

Kumar, N. S., Chin, M., O'Neill, C., Jakoi, A. M., Tabb, L., & Wolf, M. (2014). On-field performance of national football league players after return from concussion. Am J Sports Med, 42(9), 2050-2055. doi:10.1177/0363546514539449

Makdissi, M., McCrory, P., Ugoni, A., Darby, D., & Brukner, P. (2009). A prospective study of postconcussive outcomes after return to play in Australian football. Am J Sports Med, 37(5), 877-883. doi:10.1177/0363546508328118

McCrory, P., Meeuwisse, W., Dvorak, J., Aubry, M., Bailes, J., Broglio, S., . . . Castellani, R. J. (2017). Consensus statement on concussion in sport-the 5th international conference on concussion in sport held in Berlin, October 2016. Br J Sports Med, bjsports-2017-097699.

Meeuwisse, W. H., & Love, E. J. (1998). Development, implementation, and validation of the Canadian Intercollegiate Sport Injury Registry. Clinical Journal of Sport Medicine, 8(3), 164-177.

Nadeau, L., Richard, J.-F., & Godbout, P. (2008). The validity and reliability of a performance assessment procedure in ice hockey. Physical Education and Sport Pedagogy, 13(1), 65-83.

Navarro, S. M., Pettit, R. W., Haeberle, H. S., Frangiamore, S. J., Rahman, N. M., Farrow, L. D., . . . Ramkumar, P. N. (2018). Short-term impact of concussion in the NHL: an analysis of player longevity, performance, and financial loss. Journal of neurotrauma, 35(20), 2391-2399.

Pfister, T., Pfister, K., Hagel, B., Ghali, W. A., & Ronksley, P. E. (2016). The incidence of concussion in youth sports: a systematic review and meta-analysis. Br J Sports Med, 50(5), 292-297. doi:http://dx.doi.org/10.1136/bjsports-2015-094978

Reams, N., Hayward, R. A., Kutcher, J. S., & Burke, J. F. (2017). Effect of Concussion on Performance of National Football League Players. Int J Sports Physiol Perform, 1-17. doi:10.1123/ijspp.2016-0508

Reed, A., Hansen, H., Cotton, C., Gauthier, R., Jette, M., Thoden, J., & Wenger, H. (1979). Development and validation of an on-ice hockey fitness test. Canadian Journal of Applied Sport Science, 4(4), 245.

Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: uses in assessing rater reliability. Psychol Bull, 86(2),

StataCorp. 2013. Stata Statistical Software: Release 13. College Station, T. S. L. StataCorp. 2013. Stata Statistical Software: Release 13. College Station, TX: StataCorp LP.

Van Pelt, K. L., Lapointe, A. P., Galdys, M. C., Dougherty, L. A., Buckley, T. A., & Broglio, S. P. (2019). Evaluating Performance of National Ice Hockey League Players After a Concussion Versus Lower Body Injury. JAthl Train.

Wasserman, E. B., Abar, B., Shah, M. N., Wasserman, D., & Bazarian, J. J. (2015). Concussions are associated with decreased batting performance among Major League Baseball players. Am J Sports Med, 43(5),

Watson, R. C., & Sargeant, T. L. (1986). Laboratory and on-ice test comparisons of anaerobic power of ice hockey players. Can JAppl Sport Sci, 11(4), 218-224.