Measuring Expertise in Identifying Interictal Epileptiform Discharges

**Objective.** Interictal epileptiform discharges (IEDs) in EEGs are integral
to diagnosing epilepsy. However, EEGs are interpreted by readers with and
without specialty training, and there is no accepted method to assess skill in
interpretation. We aimed to develop a test to quantify IED recognition skills.

**Methods.** 13,262 candidate IEDs were selected from EEGs and scored by eight
fellowship-trained reviewers to establish a gold standard. An online test was
developed to assess how well readers with different training levels could
distinguish candidate waveforms. Sensitivity, false positive rate, and
calibration were calculated for each reader. A simple mathematical model was
developed to estimate each reader's skill and threshold in identifying an IED,
and to develop receiver operating curves for each reader. We investigated the
number of IEDs needed to measure skill level with acceptable precision.

**Results.** 29 raters completed the test; 9 experts, 7 experienced readers,
and 13 novices. Median calibration errors for experts, experienced raters, and
novices were −0.056, 0.012, and 0.046; median sensitivities were 0.800,
0.811, and 0.715; and median false positive rates were 0.177, 0.272, and
0.396, respectively. The number of test questions needed to measure those
scores was 549. Our analysis identified novices as having higher
noise/uncertainty compared to experienced and expert readers. Using calculated
noise and threshold levels, receiver operating curves were created, showing
increasing median area under the curve from novices (0.735), to experienced
(0.852), to experts (0.891).

**Significance.** Expert and non-expert readers can be distinguished based on
ability to identify IEDs. This type of assessment could also be used to
identify and correct differences in thresholds in identifying IEDs.