The validity of recognition of stroke in the emergency room (ROSIER) scale in the diagnosis of Iranian patients with acute ischemic stroke in the emergency department

Abstract

OBJECTIVES: In this study, we aimed to investigate the accuracy of recognition of stroke in the Emergency Room (ROSIER) Scale in the diagnosis of patients with acute ischemic stroke (AIS) transferred to the emergency department (ED).

METHODS: The present study was a multicenter study. Records from patients suspected of stroke, who referred to the ED were reviewed. Demographic, clinical, and diagnostic data were extracted and then entered in checklists. ROSIER Scale was used to evaluate the possible diagnosis in this study. The definitive diagnosis of a stroke was made based on neurologist's assessment and clinical and neuroimaging findings, mainly brain magnetic resonance imaging (MRI). Receiver operating characteristic (ROC) curve analysis was conducted for assessing the accuracy of ROSIER in discrimination of stroke.

RESULTS: The data of 356 suspected stroke patients were analyzed. Of all, 186 patients (52.2%) were male, and the mean age was 65.2 (standard deviation = 14.0) years ranging from 26 to 95 years. One hundred and fifty-one patients (42.4%) had AIS based on the final diagnosis. The area under the ROC curve was 0.85. The best cutoff point for ROSIER scale was ≥1 with a sensitivity of 85.4% (95% confidence interval [CI]: 78.8, 90.6%) and specificity of 65.8% (95% CI: 58.9, 72.3%).

CONCLUSION: Based on the findings, although the best cutoff point was the same as the original (derivation) study, its sensitivity (85.4% vs. 92%) and specificity (65.8% vs. 86%) were considerably lower.

Introduction

Acute ischemic stroke (AIS), as a medical emergency, is not only one of the leading causes of death but also leads to loss of useful years of life. Hence, it imposes a significant additional burden on the individual and society in short as well as the long term.[1],[2] Effective treatments for AIS, both at the pharmacological and radiologic level, are available today, but the crucial issue is the timely use of these therapies in the golden time.[3],[4] Since patients with a possible diagnosis of AIS are first brought to the emergency department (ED), emergency physicians are the first line to make a prompt and accurate diagnosis.[5] Unfortunately, only 1%–8% of AIS patients benefit from the treatment they receive due to delayed referral. According to reported statistics, incorrect diagnosis and also misdiagnosis of AIS was made in 2–26 and 30%–43% of the cases, respectively.[6] Due to time-dependent management, various checklists and criteria have been developed for rapid, accurate, and early diagnosis in the pre-hospital and in-hospital settings. In 2005, Nor et al. introduced a hospital scale in Newcastle, called recognition of stroke in the emergency room (ROSIER); in which, for scores >0, sensitivity and specificity were reported as 92% and 86%, and positive and negative predictive values were 88% and 91%, respectively.[7] This scale has been further evaluated in other countries such as UK, Germany, China, and Hong Kong, with sensitivity and specificity ranges of 83%–97% and 18%–93%, respectively.[8],[9],[10],[11],[12],[13],[14] In this study, we aimed to evaluate the validity of ROSIER scale for the diagnosis of AIS in the Iranian population based on the findings of brain magnetic resonance imaging (MRI) as the gold standard for the final diagnosis.

Material and Methods

Study design

This was a retrospective and multicenter study (Test Validity Assessment), performed throughout 2019 on patients referring to the ED of Sina and Shohaday-e-Tajrish Hospitals, Tehran, Iran. Given the retrospective nature of this study, no change was made to the diagnostic and therapeutic process of the patients, and therefore, it did not impose an additional cost on the patients or the health-care system. Patients' identity was confidential, and no information was reported individually. The study complied with the Ministry of Health's ethical guidelines and was performed after obtaining necessary approval from the Ethics Committee of Tehran University of Medical Sciences (IR.TUMS.MEDICINE.REC.1397.24).

Study population

Adult patients (over 18 years old) suspected of having AIS, who had been transferred to the ED, and had undergone a brain MRI were eligible. We excluded patients who had a previous history of trauma or known neurological disease or those who had undergone prior neurosurgery. Sampling was conducted using census method and continued until the sample size was reached. According to the study conducted by Nor et al.,[7] by assuming a specificity of 83%, an error of 7% in estimation and accuracy of 95%, as well as an AIS prevalence of 46% in the Iranian population, the minimum sample size for estimating the sensitivity and specificity in this study was calculated as 207 people.

.. .

Data gathering

Patients' files were reviewed, and information, including basic characteristics, demographics, patients' signs and symptoms, and vital signs, were extracted and entered into the pre-prepared checklist. In addition, the ROSIER scale was used to evaluate the possible diagnosis in this study. [Table 1] shows the details of the ROSIER scale calculation. The scale includes two items with a negative score and five items with a positive score. The definitive diagnosis of AIS was made based on neurologist's assessment and according to the symptoms and brain MRI findings.

Table 1. The details of recognition of stroke in the emergency room scale calculation

Statistical analysis

Basic features of the data and variables in the study are presented using descriptive statistics, such as the frequency with percentage, mean with standard deviation (SD), and median with interquartile range (IQR), as appropriate. The normality assumption was checked with graphical approaches and the Shapiro–Wilk test. We used the Chi-square test, independent t-test, and Mann–Whitney U-test, as appropriate. The logistic regression model was used to investigate ROSIER criteria for predicting AIS. Simple (univariable) logistic regression was initially performed on all ROSIER criteria and then the variables (ROSIER criteria) that had a P value lower than 0.2 were put into the multivariable logistic regression model. Value of P < 0.05 was considered as statistically significant. Receiver operating characteristic (ROC) curve analysis was conducted for the assessment of ROSIER accuracy in discrimination of AIS. We used the area under the ROC curve (AUC) for comparing ROSIER between two diagnostic groups (with and without AIS). We used the Youden index to choose the best cutoffs to diagnose AIS. We calculated sensitivity and specificity with 95% confidence interval (CI). Furthermore, positive and negative predictive values of cutoff points were estimated based on final clinical diagnoses. SPSS Statistics for Windows, Version 24.0. Armonk, NY: IBM Corp) and Stata v. 14 (StataCorp. 2015. Stata Statistical Software: Release 14. College Station, TX: StataCorp LP) were used for the statistical analyses.

Results

In total, data of 356 suspected AIS patients were analyzed. Of all, 186 patients (52.2%) were male and the mean age was 65.2 (SD = 14.0) years ranging from 26 to 95 years. Demographic characteristics and history of disease and risk factors in patients with and without AIS diagnosis are reported in [Table 2]. One hundred and fifty-one patients (42.4%) had AIS based on the final diagnosis. The prevalence of AIS in male patients was significantly higher than female patients (48.4% vs. 35.9%;P= 0.017). Moreover, AIS in smokers was significantly higher than non-smoker patients.

Table 2. Demographic characteristics and history of disease and risk factors in patients with and without acute ischemic stroke diagnosis

Median (IQR) of ROSIER in AIS patients was significantly higher than those without AIS (2.0 [1.0–3.0] vs. 0.0 [0.0–1.0];P< 0.001). [Figure 1] shows the distribution of ROSIER scores in AIS and non-AIS groups. There were 129 (85.4%), and 70 (34.1%) cases with a score >0, in AIS and non-AIS groups, respectively. Prevalent positive ROSIER criteria in the AIS group were asymmetric arm weakness (56.3%), speech disturbance (56.3%), and asymmetric leg weakness (54.3%). In addition, speech disturbance was prevalent and was present in the non-AIS group more than other criteria. Based on multivariate analysis of logistic regression, speech disturbance (odds ratio [OR] = 3.58), asymmetric arm weakness (OR = 2.86), and asymmetric facial weakness (OR = 2.02) were significant factors predicting AIS, among the seven ROSIER criteria [Table 3].

Figure 1. Distribution of recognition of stroke in the emergency room score in acute ischemic stroke and non‑acute ischemic stroke groups

Table 3. Distribution of positive recognition of stroke in the emergency room criteria by stroke diagnosis and logistic regression analysis of the items of recognition of stroke in the emergency room scale

The ROC analysis showed an AUC of 0.795 (95% CI; 0.75, 0.84). The discriminatory capacity for different cutoff values of ROSIER score is illustrated in [Table 4]. Best cutoff point for AIS diagnosis was ROSIER score ≥1 with the sensitivity of 85.4% (95% CI: 78.8, 90.6%), specificity of 65.8% (95% CI: 58.9, 72.3%), and 74.2% of patients correctly classified (129 [36.2%] true positive and 135 [37.9%] true negative cases). Positive and negative predictive values at the best cutoff point were 64.8% and 86.0%, respectively.

Table 4. Best cutoff point of Recognition of stroke in the emergency room and accuracy indices for stroke diagnosis

Discussion

In this study, we found that the ROSIER scale has the highest sensitivity (99.3%) in its proposed score (score >0), but also has a very poor specificity (4.9%). However, at its best-calculated cutoff point (score ≥1), it has a sensitivity of 85.4% and the specificity of 65.8% for the diagnosis of AIS in the ED. As it was mentioned, the ROSIER scale was introduced by Nor et al. for the first time in a hospital in the UK in 2005. In its derivation-validation study, it was reported that the threshold of more than zero had a sensitivity of 92%, specificity of 86%, positive predictive value of 88%, and negative predictive value of 91%.[7] Here, in our study, although we confirmed its high sensitivity at the cut-off point of zero, we also found that it has a very poor specificity compared to the Nor et al. study (86% vs. 5%) at this threshold.

In the Jiang study in 2014, performed on 715 patients, the sensitivity was 87%, specificity 41%, positive predictive value 62%, and negative predictive value 75%.[11] In this study, the OR of parameters such as facial weakness, arm weakness, and leg weakness were lower, but closer to the present study, compared to the original study of ROSIER. Fothergill studied 312 patients with a ROSIER scale exceeding zero, finding sensitivity of 97%, specificity of 18%, positive predictive value of 64%, and negative predictive value of 78%.[15] In the Purrucker study on 689 patients, the sensitivity and specificity of ROSIER were 80% and 79%, respectively.[12] Mingfeng et al. evaluated 540 patients with a score of more than zero on the ROSIER scale and reported a sensitivity of 89.97%, specificity of 83.23%, positive predictive value of 92.66%, and negative predictive value of 77.91%.[13] Reviewing the results of previously conducted studies on the validity of the ROSIER scale reveals that most authors agreed on its proper sensitivity, but controversies exist on its specificity. We believe that the controversies may have been raised due to different tests being used for final confirmation of AIS diagnosis.

The older the population of the world, the more people are at risk for AIS. In addition, as a result of improvement in emergency medical service training, particularly about possible AIS symptoms, we expect higher referral of AIS suspected patients to designated hospitals at the appropriate time. Nowadays, it has become vitally important to rapidly differentiate stroke from stroke mimics to initiate thrombolytic therapy correctly. Therefore, correct and rapid diagnostic evaluation of suspected AIS patients is the first step in AIS management, and clinical rules are used in this regard. As a brief critique, it seems that amongst clinical stroke diagnostic tools, ROSIER is a simple one with appropriate sensitivity. It does not assess the patient's age and bedside blood sugar level and also considers a negative score for seizure, while seizure may be a symptom of stroke, and stroke may occur in younger people as well. On the other hand, items such as hemianopia and lower extremity paresis, which are assessed in ROSIER, are not included in other tests such as LAPSS, MASS, CPSS, or FAST.[7],[16[,[17],[18],[19],[20]

Limitations

The present study did not differentiate between ischemic stroke and cerebral hemorrhage. Even though the present multicenter study was performed in more than one center, the prevalence of ischemic stroke in this study may not be generalized as the prevalence of stroke in Iran due to prevalence variation based on geography and race. In the present retrospective study, we were unable to extract some data such as a history of illness and clinical symptoms accurately and completely during the file review. Moreover, patients included in this study had various underlying diseases that could influence the evaluation of a diagnostic tool. ROSIER is not able to differentiate 100% of strokes from stroke mimics (it does not have a sensitivity of 100%) and that is why negative ROSIER is defined as a low probability of stroke and unable to rule out the stroke. A history of stroke complicates the evaluation of patients with ROSIER criteria; hence, it results in higher ROSIER scores.

Conclusion

Based on the findings, although the best cut-off point was the same as the original derivation study, its sensitivity (85.4% vs. 92%) and specificity (65.8% vs. 86%) were considerably lower. The main influencing factor may be the different gold standard considered in the current study (brain MRI) in comparison with the original derivation study (neurologist decision).

How to cite this article: Zangi M, Karimi S, Mirbaha S, Sotoodehnia M, Rasooli F, Baratloo A. The validity of recognition of stroke in the emergency room (ROSIER) scale in the diagnosis of Iranian patients with acute ischemic stroke in the emergency department. Turk J Emerg Med 2021;21:1-5

Tehran University of Medical Sciences, No: IR.TUMS.MEDICINE.REC.1397.245, Date: 21.07.2018.

The conception and design of the work by AB, SK and FR; Data acquisition by SK, SM, and MS; Analysis and interpretation of data by MZ, FR and AB; Drafting the work by MS, MZ, and SK; Revising it critically for important intellectual content by AB, SM, and FR; All the authors approved the final version to be published; And agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work.

None Declared.

The study was supported with a grant received from the Vice Chancellor for Research, TUMS (97-03-30-38950).

References

1. Forouzanfar MH, Sepanlou SG, Shahraz S, Dicker D, Naghavi P, Pourmalek F, et al. Evaluating causes of death and morbidity in Iran, global burden of diseases, injuries, and risk factors study 2010. Arch Iran Med 2014;17:304-20.
2. Longstreth WT Jr., Gasca NC, Gottesman RF, Pearce JB, Sacco RL. Adjudication of transient ischemic attack and stroke in the multi-ethnic study of atherosclerosis. Neuroepidemiology 2018;50:23-8.
3. Massahikhaleghi P, Tehrani-Banihashemi A, Saeedzai SA, Hossaini SM, Hamedi SA, Moradi-Lakeh M, et al. Burden of diseases and injuries in Afghanistan, 1990-2016: Findings from the global burden of disease 2016 study. Arch Iran Med 2018;21:324-34.
4. Baratloo A, Rahimpour L, Abushouk AI, Safari S, Lee CW, Abdalvand A. Effects of telestroke on thrombolysis times and outcomes: A meta-analysis. Prehosp Emerg Care 2018;22:472-84.
5. Baratloo A, Forouzanfar MM, Hashemi B, Safari S, Kasmaei HD, Rouhipour A, et al. Tissue plasminogen activator: A literature review. Arch Neurosci 2016;3:e30452.
6. Liberman AL, Prabhakaran S. Stroke chameleons and stroke mimics in the emergency department. Curr Neurol Neurosci Rep 2017;17:15.
7. Nor AM, Davis J, Sen B, Shipsey D, Louw SJ, Dyker AG, et al. The recognition of stroke in the emergency room (ROSIER) scale: Development and validation of a stroke recognition instrument. Lancet Neurol 2005;4:727-34.
8. Rudd M, Buck D, Ford GA, Price CI. A systematic review of stroke recognition instruments in hospital and prehospital settings. Emerg Med J 2016;33:818-22.
9. Wang T, Li D, Pan M, Du G, Zhang G. Diagnostic accuracy of recognition of stroke in the emergency room scale: A Meta-analysis. Zhonghua Wei Zhong Bing Ji Jiu Yi Xue 2019;31:607-13.
10. Kwiatkowski T. The Recognition of Stroke in the Emergency Room (ROSIER) scale: A useful diagnostic tool? Nat Clin Pract Neurol 2006;2:244-6.
11. Jiang HL, Chan CP, Leung YK, Li YM, Graham CA, Rainer TH. Evaluation of the Recognition of Stroke in the Emergency Room (ROSIER) scale in Chinese patients in Hong Kong. PLoS One 2014;9:e109762.
12. Purrucker JC, Hametner C, Engelbrecht A, Bruckner T, Popp E, Poli S. Comparison of stroke recognition and stroke severity scores for stroke detection in a single cohort. J Neurol Neurosurg Psychiatry 2015;86:1021-8.
13. Mingfeng H, Zhixin W, Qihong G, Lianda L, Yanbin Y, Jinfang F. Validation of the use of the ROSIER scale in prehospital assessment of stroke. Ann Indian Acad Neurol 2012;15:191-5.
14. De Luca A, Mariani M, Riccardi MT, Damiani G. The role of the Cincinnati Prehospital Stroke Scale in the emergency department: Evidence from a systematic review and meta-analysis. Open Access Emerg Med 2019;11:147-59.
15. Fothergill RT, Williams J, Edwards MJ, Russell IT, Gompertz P. Does use of the recognition of stroke in the emergency room stroke assessment tool enhance stroke recognition by ambulance clinicians? Stroke 2013;44:3007-12.
16. Bray JE, Martin J, Cooper G, Barger B, Bernard S, Bladin C. Paramedic identification of stroke: Community validation of the melbourne ambulance stroke screen. Cerebrovasc Dis 2005;20:28-33.
17. Kidwell CS, Starkman S, Eckstein M, Weems K, Saver JL. Identifying stroke in the field. Prospective validation of the Los Angeles prehospital stroke screen (LAPSS). Stroke 2000;31:71-6.
18. Studnek JR, Asimos A, Dodds J, Swanson D. Assessing the validity of the Cincinnati prehospital stroke scale and the medic prehospital assessment for code stroke in an urban emergency medical services agency. Prehosp Emerg Care 2013;17:348-53.
19. Saberian P, Tavakoli N, Hasani-Sharamin P, Aghili M, Baratloo A. Accuracy of stroke diagnosis using FAST (Face, Arm, Speech, Time) tool by emergency medical service dispatchers and technicians and its impact on transport time. Arch Neurosci 2019;7:e98691.
20. Karimi S, Motamed H, Aliniagerdroudbari E, Babaniamansour S, Jami A, Baratloo A. The Prehospital Ambulance Stroke Test vs. the Cincinnati Prehospital Stroke Scale: a diagnostic accuracy study. Australasian Journal of Paramedicine [Internet] 2020;17. Available from: https://ajp.paramedics.org/index.php/ajp/article/view/784. [Last cited on 2020Aug 26].

This study was conducted with the support of the Prehospital Emergency Research Center, Tehran University of Medical Sciences, Tehran, Iran. The present article was extracted from the thesis by Dr. Mahdi Zangi to achieve his specialist degree in emergency medicine from the Tehran University of Medical Sciences.