Agreement between pairs of lay reviewers ()
As with the entire study population, a majority of subjects in the sample had partial-onset seizures, and less than one-fourth of subjects had generalized-onset seizures. As a result, the number of patients with generalized nonconvulsive seizures (absence, myoclonic, or atonic seizures) was small.
For all three pairs of lay reviewers, agreement was moderate-almost perfect (κ = 0.46 to 1.00) for the broad seizure categories, generalized onset and partial onset (). Within partial-onset seizures, agreement for all three pairs was substantial for secondarily generalized seizures (κ = 0.66 to 0.69) and substantial-almost perfect for complex partial seizures (κ = 0.68 to 0.87). For simple partial seizures, however, agreement was moderate for two pairs of reviewers and poor for the remaining pair.
Within generalized-onset seizures, agreement was moderate-substantial for generalized tonic-clonic (GTC) (κ = 0.60 to 0.73), absence (κ = 0.55 to 0.79), and atonic (κ = 0.59, 0.66) seizures, and almost perfect (κ > 0.87) for myoclonic seizures.
In general, weighted kappas were higher than unweighted kappas, since partial credit was given for partial agreement; however, weights did not change the results substantially.
Weighted and unweighted kappa for seizure classification by lay reviewers
Agreement between the neurologist and lay reviewers ()
Because of the oversampling of subjects with presumed generalized-onset seizures for the comparisons between the neurologist and lay reviewers, 55% (neurologist versus reviewer 1) and 59% (neurologist versus reviewer 2) of subjects in the two comparisons had generalized-onset seizures.
shows agreement between the neurologist and lay reviewers. For each comparison, four kappas are shown: unweighted (for partial agreement-unadjusted (for sampling probabilities), weighted-unadjusted, unweighted-adjusted, and weighted-adjusted. Kappas for comparisons between the neurologist and lay reviewers were comparable to those for comparisons between the lay reviewers. For the broad seizure classifications, generalized onset and partial onset, the unweighted-unadjusted kappas indicated that agreement was substantial (0.73 and 0.77) for the reviewer 1 comparison and moderate (0.53) and substantial (0.75) for the reviewer 2 comparison. Within partial-onset seizures, agreement was substantial for complex partial seizures (unweighted-unadjusted κ= 0.67, 0.79) but poor for simple partial seizures.
Within generalized-onset seizures, the unweighted-unadjusted kappas indicated that agreement was moderate-substantial for GTC, substantial-almost perfect for absence, moderate for myoclonic, and fair for atonic seizures.
Adjustment for sampling probabilities influenced the kappa values differently for different seizure types. For the reviewer 1 comparisons, the adjusted kappas were generally lower than the unadjusted kappas, with the exception of absence seizures. For the reviewer 2 comparisons, the relations between adjusted and unadjusted kappas fluctuated to a greater degree. Adjustment influenced the kappa values to a greater extent for rare than for common seizure types. Thus, kappas for myoclonic and atonic seizures decreased substantially as a result of the adjustment, while only small changes were observed in the broad categories of generalized- and partial-onset seizures and in primary or secondarily generalized tonicclonic (SGTC) seizures.
Based on the adjusted kappas, agreement for the broad categories of generalized- and partial-onset seizures was moderate for the reviewer 1 and moderate-almost perfect for the reviewer 2 comparisons. For absence and complex partial seizures, agreements were substantial-almost perfect for both pairwise comparisons (0.69 to 0.94), while for myoclonic and atonic seizures, agreements ranged from fair to poor (0.13 to 0.22). Adjusted kappas for simple partial seizures indicated poor agreement.
As with the comparisons between the lay reviewers, the weighted kappas (both adjusted and unadjusted) were higher than the unweighted kappas because they allowed for partial agreement; however, applying weights did not change the results substantially.
Kappa for comparisons of seizure diagnoses between neurologist and lay reviewers