Sample Size. The primary outcome for the interrater agreement analysis was the intraclass correlation coefficient (ICC) of the AHI. Given nine sleep scorers (one at each site), the 15 PSGs had a power of 83% to detect an ICC of at least 0.90, assuming a null hypothesis of ICC = 0.70. The interrater reliability measures used to examine the agreement among the nine different scorers were the ICC for continuous variables (respiratory indices, duration of sleep stages, and arousal index) and the kappa (κ) statistic for mul- tiple raters for the categorical variables (sleep stages). The levels of agreement using the ICCs of respiratory indices, dura- tion of sleep stages, and arousal index were classified as fol- lows: 0.00-0.25 = little, 0.26-0.49 = low, 0.50-0.69 = moderate, AHI (events/h) 25.8 ± 19.4 22.7 ± 18.4 25.7 ± 19.0 25.9 ± 19.8 24.3 ± 17.2 22.7 ± 17.4 23.0 ± 15.4 25.9 ± 21.3 23.6 ± 18.1 0.95 0.91-0.98 Total Apneas 54.1 ± 71.9 31.6 ± 68.2 45.3 ± 66.5 46.3 ± 63.9 42.3 ± 60.3 20.3 ± 41.8 58.1 ± 78.3 104.6 ± 127.7 51.1 ± 72.4 0.73 0.55-0.88 Obstructive 44.1 ± 61.3 28.7 ± 68.2 39.5 ± 67.1 35.9 ± 55.2 37.5 ± 57.1 17.7 ± 36.7 56.4 ± 76.6 100.7 ± 127.7 42.2 ± 68.9 0.70 0.51-0.86 Central 6.7 ± 9.1 2.9 ± 7.1 4.4 ± 7.8 7.3 ± 10.2 1.0 ± 2.0 0.9 ± 1.4 0.9 ± 1.9 3.3 ± 6.2 6.7 ± 7.8 0.46 0.27-0.70 Mixed 3.3 ± 9.5 0.1 ± 0.3 1.5 ± 3.0 3.1 ± 7.5 4.0 ± 7.5 1.7 ± 5.4 0.8 ± 2.3 0.6 ± 1.2 2.2 ± 5.6 0.42 0.24-0.67 Hypopneas 90.1 ± 76.6 100.7 ± 85.7 109.1 ± 85.4 94.8 ± 73.8 108.2 ± 79.4 109.2 ± 87.4 81.1 ± 58.1 51.0 ± 36.9 86.5 ± 75.1 0.80 0.65-0.91 ODI (events/h) 22.4 ± 17.6 22.0 ± 17.5 24.6 ± 18.3 21.6 ± 17.0 25.3 ± 17.5 22.4 ± 18.0 19.7 ± 14.2 22.9 ± 17.4 20.9 ± 16.6 0.97 0.93-0.99 AHI, apnea-hypopnea index; CI, confidence interval; ICC, intraclass correlation coefficient; ODI, oxygen desaturation index. Downloaded from ▇▇▇▇▇://▇▇▇▇▇▇▇▇.▇▇▇.▇▇▇/sleep/article/36/4/591/2595972 by guest on 02 October 2020 κ 0.78 0.77 0.78 0.78 95% CI 0.77-0.79 0.76-0.78 0.77-0.79 0.77-0.78 CI, confidence interval; NREM, nonrapid eye movement; REM, rapid eye movement. κ 0.78 0.31 0.60 0.67 0.78 0.63 95% CI 0.77-0.79 0.30-0.32 0.59-0.61 0.65-0.69 0.77-0.79 0.62-0.63 CI, confidence interval; NREM, nonrapid eye movement. 0.70-0.89 = strong, 0.90-1.00 = very strong.6-8 For the κ sta- tistic for multiple raters, the levels of agreement were classi- fied as follows: < 0 = no agreement, 0-0.20 = slight agreement, 0.21-0.40 = fair agreement, 0.41-0.60 = moderate agreement, 0.61-0.80 = substantial agreement, 0.81-1.0 = almost perfect agreement.9 Data analyses were performed using Stata software version 12 (StataCorp, LP, College Station, TX). The scored AHI ranged from 0 to 70.9 events/h of sleep. The interscorer agreement of AHI scoring (Table 1) among the SAGIC centers was very strong, with an ICC of 0.95 (95% con- fidence interval [CI] 0.91-0.98). There was strong agreement in the scoring of the total number of apneas as well as the total number of hypopneas. Table 1 displays the means and the ▇▇▇▇- dard deviations (SD) of the scoring of respiratory events in all 15 PSGs. One site (sleep center number six) tended to score the respiratory events as hypopneas, whereas another (sleep center number eight) tended to score the respiratory events as apneas (Table 1). The agreement of ODI scoring was also very strong with an ICC of 0.97 (95% CI 0.93-0.99). Figure 1 displays all 15 PSG respiratory event scoring results at the different sites. As expected from the ICC values, there were more variations in the scoring of the total apneas and hy- popneas compared with the AHI. The AHI scores were highly correlated with the ODI scores (r = 0.96, P < 0.0001). Using the ▇▇▇▇▇-▇▇▇▇▇▇ method, the mean difference between AHI and ODI scores was +2.1 events/h (limits of agreement: +12.3 and -8.1 events/h). Table 2 shows the epoch-by-epoch (n = 12,712) agreement in scoring Stages W, NREM, and REM sleep among the nine scor- ers. There was substantial agreement in the scoring of the major sleep-wake states with κ = 0.78 (95% CI 0.77-0.78). In Table 3, the κ values are shown with NREM sleep separated into differ- ent stages. The scoring of Stages N1 and N2 showed only fair (κ = 0.31) and moderate (κ = 0.60) agreements respectively, although there was still substantial agreement in the scoring of all sleep stages with κ = 0.63 (95% CI 0.62-0.63). Table 4 shows the agreement of the sleep stage duration (min) and arousal index scoring. There was strong agreement in the scoring of the duration of TST, NREM, and REM sleep, but little to low agreement in the scoring of the duration of Stage N1 and N2. There was moderate agreement in the scoring of the arousal index with an ICC of 0.68 (95% CI: 0.49-0.85).
Appears in 1 contract
Sources: International Agreement in Sleep and Respiratory Scoring
Sample Size. The primary outcome for the interrater agreement analysis was the intraclass correlation coefficient (ICC) of the AHI. Given nine sleep scorers (one at each site), the 15 PSGs had a power of 83% to detect an ICC of at least 0.90, assuming a null hypothesis of ICC = 0.70. The interrater reliability measures used to examine the agreement among the nine different scorers were the ICC for continuous variables (respiratory indices, duration of sleep stages, and arousal index) and the kappa (κ) statistic for mul- tiple raters for the categorical variables (sleep stages). The levels of agreement using the ICCs of respiratory indices, dura- tion of sleep stages, and arousal index were classified as fol- lows: 0.00-0.25 = little, 0.26-0.49 = low, 0.50-0.69 = moderate, AHI (events/h) 25.8 ± 19.4 22.7 ± 18.4 25.7 ± 19.0 25.9 ± 19.8 24.3 ± 17.2 22.7 ± 17.4 23.0 ± 15.4 25.9 ± 21.3 23.6 ± 18.1 0.95 0.91-0.98 Total Apneas 54.1 ± 71.9 31.6 ± 68.2 45.3 ± 66.5 46.3 ± 63.9 42.3 ± 60.3 20.3 ± 41.8 58.1 ± 78.3 104.6 ± 127.7 51.1 ± 72.4 0.73 0.55-0.88 Obstructive 44.1 ± 61.3 28.7 ± 68.2 39.5 ± 67.1 35.9 ± 55.2 37.5 ± 57.1 17.7 ± 36.7 56.4 ± 76.6 100.7 ± 127.7 42.2 ± 68.9 0.70 0.51-0.86 Central 6.7 ± 9.1 2.9 ± 7.1 4.4 ± 7.8 7.3 ± 10.2 1.0 ± 2.0 0.9 ± 1.4 0.9 ± 1.9 3.3 ± 6.2 6.7 ± 7.8 0.46 0.27-0.70 Mixed 3.3 ± 9.5 0.1 ± 0.3 1.5 ± 3.0 3.1 ± 7.5 4.0 ± 7.5 1.7 ± 5.4 0.8 ± 2.3 0.6 ± 1.2 2.2 ± 5.6 0.42 0.24-0.67 Hypopneas 90.1 ± 76.6 100.7 ± 85.7 109.1 ± 85.4 94.8 ± 73.8 108.2 ± 79.4 109.2 ± 87.4 81.1 ± 58.1 51.0 ± 36.9 86.5 ± 75.1 0.80 0.65-0.91 ODI (events/h) 22.4 ± 17.6 22.0 ± 17.5 24.6 ± 18.3 21.6 ± 17.0 25.3 ± 17.5 22.4 ± 18.0 19.7 ± 14.2 22.9 ± 17.4 20.9 ± 16.6 0.97 0.93-0.99 AHI, apnea-hypopnea index; CI, confidence interval; ICC, intraclass correlation coefficient; ODI, oxygen desaturation index. Downloaded from ▇▇▇▇▇://▇▇▇▇▇▇▇▇.▇▇▇.▇▇▇/sleep/article/36/4/591/2595972 by guest on 02 October 2020 κ 0.78 0.77 0.78 0.78 95% CI 0.77-0.79 0.76-0.78 0.77-0.79 0.77-0.78 CI, confidence interval; NREM, nonrapid eye movement; REM, rapid eye movement. κ 0.78 0.31 0.60 0.67 0.78 0.63 95% CI 0.77-0.79 0.30-0.32 0.59-0.61 0.65-0.69 0.77-0.79 0.62-0.63 CI, confidence interval; NREM, nonrapid eye movement. 0.70-0.89 = strong, 0.90-1.00 = very strong.6-8 For the κ sta- tistic for multiple raters, the levels of agreement were classi- fied as follows: < 0 = no agreement, 0-0.20 = slight agreement, 0.21-0.40 = fair agreement, 0.41-0.60 = moderate agreement, 0.61-0.80 = substantial agreement, 0.81-1.0 = almost perfect agreement.9 Data analyses were performed using Stata software version 12 (StataCorp, LP, College Station, TX). The scored AHI ranged from 0 to 70.9 events/h of sleep. The interscorer agreement of AHI scoring (Table 1) among the SAGIC centers was very strong, with an ICC of 0.95 (95% con- fidence interval [CI] 0.91-0.98). There was strong agreement in the scoring of the total number of apneas as well as the total number of hypopneas. Table 1 displays the means and the ▇▇▇▇- dard deviations (SD) of the scoring of respiratory events in all 15 PSGs. One site (sleep center number six) tended to score the respiratory events as hypopneas, whereas another (sleep center number eight) tended to score the respiratory events as apneas (Table 1). The agreement of ODI scoring was also very strong with an ICC of 0.97 (95% CI 0.93-0.99). Figure 1 displays all 15 PSG respiratory event scoring results at the different sites. As expected from the ICC values, there were more variations in the scoring of the total apneas and hy- popneas compared with the AHI. The AHI scores were highly correlated with the ODI scores (r = 0.96, P < 0.0001). Using the ▇▇▇▇▇-▇▇▇▇▇▇ method, the mean difference between AHI and ODI scores was +2.1 events/h (limits of agreement: +12.3 and -8.1 events/h). Table 2 shows the epoch-by-epoch (n = 12,712) agreement in scoring Stages W, NREM, and REM sleep among the nine scor- ers. There was substantial agreement in the scoring of the major sleep-wake states with κ = 0.78 (95% CI 0.77-0.78). In Table 3, the κ values are shown with NREM sleep separated into differ- ent stages. The scoring of Stages N1 and N2 showed only fair (κ = 0.31) and moderate (κ = 0.60) agreements respectively, although there was still substantial agreement in the scoring of all sleep stages with κ = 0.63 (95% CI 0.62-0.63). Table 4 shows the agreement of the sleep stage duration (min) and arousal index scoring. There was strong agreement in the scoring of the duration of TST, NREM, and REM sleep, but little to low agreement in the scoring of the duration of Stage N1 and N2. There was moderate agreement in the scoring of the arousal index with an ICC of 0.68 (95% CI: 0.49-0.85).
Appears in 1 contract
Sources: International Agreement in Sleep and Respiratory Scoring