The Inter-rater Variability of Clinical Assessment in Post-anoxic Myoclonus

Abstract

Background: Acute post-anoxic myoclonus (PAM) can be divided into an unfavorable (generalized/subcortical) and more favorable ((multi)focal/cortical) outcome group that could support prognostication in post-anoxic encephalopathy; however, the inter-rater variability of clinically assessing these PAM subtypes is unknown.

Methods: We prospectively examined PAM patients using a standardized video protocol. Videos were rated by three neurologists who classified PAM phenotype (generalized/(multi)focal), stimulus sensitivity, localization (proximal/distal/both), and severity (Clinical Global Impression-Severity Scale (CGI-S) and Unified Myoclonus Rating Scale (UMRS)).

Results: Poor inter-rater agreement was found for phenotype and stimulus sensitivity (κ=–0.05), moderate agreement for localization (κ=0.46). Substantial agreement was obtained for the CGI-S (intraclass correlation coefficient (ICC)=0.64) and almost perfect agreement for the UMRS (ICC=0.82).

Discussion: Clinical assessment of PAM is not reproducible between physicians, and should therefore not be used for prognostication. PAM severity measured by the UMRS appears to be reliable; however, the relation between PAM severity and outcome is unknown.

Introduction

The presence of myoclonus in the first few days after anoxic brain injury has traditionally been associated with an unfavorable outcome in patients with post-anoxic encephalopathy (PAE).1,2 This acute post-anoxic myoclonus (PAM) differs in clinical characteristics, prognosis and treatment from the Lance–Adams syndrome, in which myoclonus usually occurs a few weeks after the anoxic event.2

Nowadays, in the era of target temperature management, 9–14% of these PAM patients survive.3–5 The current incidence of PAM in PAE is estimated to be 20%.3,4 In the literature, a PAM-subtype-specific prognosis has been reported: (multi)focal PAM had a better chance of good functional outcome (17–18%) than PAE patients without PAM. In contrast, patients with generalized PAM only had a favorable outcome in 3–6%.1,4–6 This might implicate that the PAM subtype can be supportive in PAE prognostication.7–9

(Multi)focal PAM consists of subtle asynchronous jerks involving the distal musculature and is considered to have a cortical origin.4,10 Stimulus sensitivity is more often present in the cortical than the subcortical myoclonus.10 Generalized PAM (also referred to as status myoclonus, myoclonus status epilepticus, or reticular reflex myoclonus) is characterized by generalized synchronous jerks with preferential involvement of proximal musculature, and is considered to have a subcortical origin.4,6,10 Despite these well-defined differences, the reliability of clinically differentiating between these PAM subtypes is unknown. We investigated this in the current study.

Methods

Patients

We prospectively included 10 PAM patients from the intensive care units of the University Medical Center Groningen, and the Lausanne University Hospital. The institutional review board of both hospitals approved the study. For all patients informed consent was obtained from a legal representative. Exclusion criteria were age <18 years, PAM onset >72 hours, GCS (Glasgow Coma Scale) >8, history of myoclonus, traumatic brain injury, epilepsy, possible explanation myoclonus other than PAE, and medication overdose.

Study procedures

A standardized high-definition video protocol for examination of PAM was conducted after first appearance of PAM. The video protocol consisted of recording 60 seconds of rest, 30 seconds of close-up recording of the face, and recording 32 lateralized and non-lateralized stimuli: pain stimulus of the orbit, trapezius muscle and nail bed; pin prick of the cheek, forearm, and foot; pupillary light response; corneal and glabellar reflex, biceps and knee tendon reflexes; light touch of the hand and foot; flick of the finger and toe; visual and auditory threat; jaw and nose tapping. The video protocol is available as supplementary material (Supplement A), and an example examination is provided in Video 1.

[Image Omitted: See PDF]

Videos were rated by three experienced (intensive-care) neurologists (B.M.J., J.N., J.H.) blinded for outcome. Raters classified PAM phenotype (generalized or (multi)focal), localization (proximal, distal, or both), stimulus sensitivity (present or absent), severity (Clinical Global Impression of Severity Scale (CGI-S); range 1–7), and the “myoclonus at rest” part of the Unified Myoclonus Rating Scale (UMRS) score (range 0–128).11,12 Additionally, all stimuli were scored separately regarding stimulus sensitivity. If raters considered an item as not evaluable it was excluded from further analysis. The evaluations of PAM patients (total 3×10=30 evaluations) were ordered based on the total number of stimulus-sensitive stimuli (0, 1–2, 3–4, 5–6, 7–8, 9–10). These categories were corrected for group size, and correlated with how often a rater concluded “stimulus-sensitive PAM”. Finally, painful stimuli (pain stimulus of orbit, trapezius muscle, nail bed, and pin prick) were compared with non-painful stimuli in evoking PAM.

Statistical analysis

Inter-rater variability was calculated by the intraclass correlation coefficient (ICC) and kappa (Fleiss) statistics.13 The two-way mixed model and single measurement coefficients of ICCs were used. According to Landis and Koch,14 we interpreted kappa and ICC outcomes as follows: <0.00=poor; 0.00–0.20=slight; 0.21–0.40=fair; 0.41–0.60=moderate; 0.61–0.80=substantial; >0.81=almost perfect. The correlation between the amount of stimulus-sensitive stimuli and the conclusion “stimulus sensitive PAM” was performed using Spearman’s correlation. Stimulus sensitivity of PAM after painful and non-painful stimuli was compared using a two-sided Fisher’s exact test. p-Values of <0.05 were considered as statistically significant.

Results

Patients had a mean age of 59 (SD±16) years, and seven were male (Detailed patient characteristics are provided in Table 1). All patients were treated with target temperature management. One patient had a good recovery with only mild cognitive deficits after 6 months (Case 2 in Table 2, Video 1). All other patients died in the acute setting. Patients were examined a mean of 9 hours (range 2–24 hours) after the first appearance of PAM. In seven patients no symptomatic treatment was given at the moment of examination.

Age, years, mean (SD)	59	(16)
Male, n	7
Initial rhythm, n
Shockable (VF/VT)	3
Non-shockable (bradycardia/asystole/PEA)	7
Primary cause of CPR, n (%)
Cardiac	3
Hypoxic	6
Unknown	1
Location of arrest, n
OHCA	8
IHCA	2
Time to ROSC (minutes), median (IQR)	18	(10–23)
Time to occurrence of PAM (hours), median (IQR)	14	(10–36)
Initial treatment of PAM, n
Propofol	8
Clonazepam	3
Sodium valproate	3
Levetiracetam	2
Other benzodiazepine	2
>1 drug	4
SEP N20, n
Present	5
Bilaterally absent	5
EEG result, n
Normal/mild encephalopathic	0
Diffuse slowing	1
Status epilepticus	4
Burst suppression	4
Low voltage/isoelectric	0
No EEG	1
Outcome, n
Recovery with mild cerebral disability	1
Death	9
Treatment withdrawal, n	9
Time to treatment withdrawal (hours), median (IQR)	48	(38–130)
Reason for treatment withdrawal, n
Neurological examination	4
SEP	5
EEG	5
Combination	3

[Image Omitted: See PDF]

	PAM Phenotype			Stimulus Sensitivity			Localization			CGI–S			UMRS
Case	R1	R2	R3	R1	R2	R3	R1	R2	R3	R1	R2	R3	R1	R2	R3
1	gz	mf	mf	−	−	−	p+d	p+d	p+d	6	5	4	29	35	18
2	mf	gz	mf	+	−	+	p	p	p	4	4	2	4	8	2
3	gz	gz	gz	+	−	−	p+d	p+d	p+d	7	7	5	87	52	74
4	gz	mf	mf	−	−	−	p+d	p+d	d	4	4	3	22	8	32
5	mf	mf	mf	+	−	−	d	d	d	2	3	2	1	4	2
6	mf	mf	mf	+	−	−	p	p+d	p	2	2	2	4	6	5
7	gz	mf	mf	+	−	+	p+d	p+d	p+d	5	4	2	14	19	18
8	gz	mf	mf	+	−	+	p+d	p+d	p+d	6	6	4	64	41	76
9	gz	mf	mf	−	−	−	d	p+d	p+d	6	4	4	33	23	27
10	gz	mf	mf	+	−	−	p	p+d	d	5	5	4	34	22	59

[Image Omitted: See PDF]

Inter-rater variability

Poor agreement was found among raters in assessing PAM phenotype (generalized or (multi)focal) (κ=–0.05, 95% confidence interval (CI) –0.40 to 0.31). Agreement for the localization of PAM (proximal, distal, or both) was moderate (κ=0.46, 95% CI 0.20–0.73). In the rating scales used to assess PAM severity, substantial agreement was obtained for the CGI-S (ICC 0.64, 95% CI 0.14–0.89) and almost perfect agreement for the UMRS (ICC 0.82, 95% CI 0.57–0.95) (Figure 1). The specific scores of the raters are provided in Table 2.

[Image Omitted: See PDF]

Stimulus sensitivity

In total 294 of 320 (92%) stimuli were examined by the raters. Poor agreement was found for confirming the presence or absence of “stimulus-sensitive PAM” in a patient (κ=–0.05, 95% CI –0.40 to 0.30) (Figure 1, Table 2). Likewise, when raters examined all 294 stimuli regarding the effect on PAM, fair agreement was found (κ=0.276, 95% CI 0.21–0.34). A positive correlation was present between the conclusion “stimulus-sensitive PAM” and the amount of stimuli rated positive (r=0.75, p<0.001). Finally, painful stimuli were significantly more likely (p<0.001) to evoke PAM than non-painful stimuli (respectively 21% and 7%).

Discussion

Our results suggest that the clinical assessment of myoclonus in the first few days after anoxic brain injury is not consistent between different raters. Since phenotype, localization, and stimulus sensitivity findings of PAM varied considerably between raters the potential to accurately determine the etiology, i.e. cortical versus subcortical, was poor. We therefore conclude that clinical assessment of PAM should not be used for PAE prognostication or guiding treatment decisions. This is in line with the current European recommendations, which do not include PAM as a single outcome predictor for PAE.8

The clinical phenotype of PAM appears to be a continuum ranging from severe generalized PAM on one side of the spectrum and subtle focal PAM on the other side. Most likely, the PAM patients on one end of the spectrum can be assessed reliably, but as most cases show jerks in the middle of the spectrum clinical assessment is complicated and not interpreted consistently between raters.4,5 Other factors that are known to complicate the clinical evaluation of PAM are the dynamic character and cyclicality, sporadic nature, stimulus dependency, use of sedatives or neuromuscular blockers, and the presence of other involuntary movements.8,15 In our study, these factors could have caused the differences in interpretation, but we were unable to investigate this because of the small sample size. Future studies towards the etiology of PAM should use electroencephalography–electromyography to determine the origin of the myoclonus. Electrophysiology might prove a better tool to discriminate between cortical and subcortical myoclonus and a possible tool to use for prognosis.10

The CGI-S scale and especially the UMRS appear to represent reliable assessments of PAM severity. The relation between PAM severity and outcome is however unknown. The CGI-S and UMRS are recognized as reliable scales, but have not previously been used for PAM in particular.11,12 Moreover, in research evaluating the treatment of PAM, the UMRS could be reliably used to assess intervention effects.

An additional finding of our study was that painful stimuli induced significantly more PAM than non-painful stimuli. Since painful stimuli activate the ascending arousal system (AAS),16 subcortical structures could be triggered. Alternatively, since AAS promotes cortical excitability,16 (cortical) PAM might occur.

A potential limitation of our study was that assessments were based on video recordings leading to an observation that is less vivid. Contrarily, video recordings ensure that raters have the same information and the use of video is well established in movement disorders. Furthermore, the use of video enables the raters to assess myoclonus multiple times and to look at different parts of the video at different moments. Another limitation was the relatively small number of patients included.

In conclusion, our study shows that the clinical assessment of acute PAM is not consistent between physicians, and therefore the use of PAM for prognostication or guiding treatment decisions in PAE should be restrained. PAM severity measured by the UMRS appears to be reliable; however, the relation between PAM severity and outcome has not yet been established.

Supplementary Material

Supplement A referenced in this article is available here: https://doi.org/10.7916/D85T3XQK.

Notes

1.

¹ Funding: Dr. van Zijl received the MD/PhD scholarship from the University of Groningen for research purposes. Dr. Beudel received funding from the Dutch Brain Foundation. Dr. Rossetti is supported by The Swiss National Science Foundation (CR3213_143780), and received unrestricted research support from SAGE Therapeutics and UCB Pharma.

2.

² Financial Disclosures: None.

3.

³ Conflict of Interests: The authors report no conflict of interest.

4.

⁴ Ethics Statement: This study was performed in accordance with the ethical standards detailed in the Declaration of Helsinki. The authors’ institutional ethics committee has approved this study. All patients that appear on video have provided written informed consent; authorization for the videotaping and for publication of the videotape was provided.

References

01. Wijdicks, EF, Parisi, JE and Sharbrough, FW (1994). Prognostic value of myoclonus status in comatose survivors of cardiac arrest Ann Neurol 35: 239–243. doi: 10.1002/ana.410350219. [PubMed]

02. Gupta, HV and Caviness, JN (2016). Post-hypoxic myoclonus: current concepts, neurophysiology, and treatment Tremor Other Hyperkinet Mov 6: 409. doi: 10.7916/D89C6XM4.

03. Seder, DB Sunde, K Rubertsson, S Mooney, M Stammet, P Riker, RR et al. (2015). International Cardiac Arrest Registry: neurologic outcomes and postresuscitation care of patients with myoclonus following cardiac arrest Crit Care Med 43: 965–972. doi: 10.1097/CCM.0000000000000880. [PubMed]

04. Bouwes, A van Poppelen, D Koelman, JH Kuiper, MA Zandstra, DF Weinstein, HC et al. (2012). Acute posthypoxic myoclonus after cardiopulmonary resuscitation BMC Neurol 12: 63. doi: 10.1186/1471-2377-12-63. [PubMed]

05. van Zijl, JC, Beudel, M, vd Hoeven, HJ, Lange, F, Tijssen, MAJ and Elting, JWJ (2016). Electroencephalographic findings in posthypoxic myoclonus J Intensive Care Med 31: 270–275. doi: 10.1177/0885066615571533. [PubMed]

06. Young, GB, Gilbert, JJ and Zochodne, DW (1990). The significance of myoclonic status epilepticus in postanoxic coma Neurology 40: 1843–1848. doi: 10.1212/WNL.40.12.1843. [PubMed]

07. Wijdicks, EF, Hijdra, A, Young, GB, Bassetti, CL and Wiebe, S (2006). Quality Standards Subcommittee of the American Academy of Neurology. Practice parameter: prediction of outcome in comatose survivors after cardiopulmonary resuscitation (an evidence-based review): report of the Quality Standards Subcommittee of the American Academy of Neurology Neurology 67: 203–210. doi: 10.1212/01.wnl.0000227183.21314.cd. [PubMed]

08. Sandroni, C Cariou, A Cavallaro, F Cronberg, T Friberg, H Hoedemaekers, C et al. (2014). Prognostication in comatose survivors of cardiac arrest: an advisory statement from the European Resuscitation Council and the European Society of Intensive Care Medicine Resuscitation 85: 1779–1789. doi: 10.1016/j.resuscitation.2014.08.011. [PubMed]

09. Rossetti, AO, Rabinstein, AA and Oddo, M (2016). Neurological prognostication of outcome in patients in coma after cardiac arrest Lancet Neurol 15: 597–609. doi: 10.1016/S1474-4422(16)00015-6. [PubMed]

10. Cassim, F and Houdayer, E (2006). Neurophysiology of myoclonus Neurophysiol Clin 36: 281–291. doi: 10.1016/j.neucli.2006.10.001. [PubMed]

11. Busner, J and Targum, SD (2007). The Clinical Global Impressions Scale: applying a research tool in clinical practice Psychiatry (Edgmont) 4: 28–37.

12. Frucht, SJ, Leurgans, SE, Hallett, M and Fahn, S (2002). The Unified Myoclonus Rating Scale Adv Neurol 89: 361–376. [PubMed]

13. Kottner, J Audigé, L Brorson, S Donner, A Gajewski, BJ Hróbjartsson, A et al. (2011). Guidelines for reporting reliability and agreement studies (GRRAS) were proposed J Clin Epidemiol 64: 96–106. doi: 10.1016/j.jclinepi.2010.03.002. [PubMed]

14. Landis, JR and Koch, GG (1977). The measurement of observer agreement for categorical data Biometrics 33: 159. doi: 10.2307/2529310. [PubMed]

15. Samaniego, EA, Mlynash, M, Caulfield, AF, Eyngorn, I and Wijman, CAC (2011). Sedation confounds outcome prediction in cardiac arrest survivors treated with hypothermia Neurocrit Care 15: 113–119. doi: 10.1007/s12028-010-9412-8. [PubMed]

16. Schwartz, JR and Roth, T (2008). Neurophysiology of sleep and wakefulness: basic science and clinical implications Curr Neuropharmacol 6: 367–378. doi: 10.2174/157015908787386050. [PubMed]