Supplementary Material for: Percent Consonant Correct as an Outcome Measure for Cleft Speech in an Intervention Study

<b><i>Background/Aims:</i></b> Percent consonant correct (PCC) was originally described by Shriberg and Kwiatkowski [J Speech Hear Disord. 1982 Aug;47(3):256–70] as a severity metric for phonological speech disorders, and has been adapted and used in many studies on speech sound disorders. It is well-recognized that cleft speech is complex, consisting of several interacting parameters assessed simultaneously, with error sounds not in the listener’s own language. In speech outcome studies, narrow phonetic transcription and the reporting of intra- and inter-rater reliability are acknowledged as the gold standard. However, cleft speech brings special challenges to this task, as complex speech disorders are known to be associated with low transcriber agreement. Recent studies informed the decision to use PCC as the primary outcome measure in a cleft speech intervention study, given its common usage and familiarity. The aim was to specifically evaluate the intra- and inter-rater reliability of PCC in an intervention study, in contrast to other types of speech outcome studies. <b><i>Methods:</i></b> Two trained and experienced listeners analyzed 119 recordings, randomly selected from five data points before, during, and following intervention. The PCC score was separately calculated for words and sentences/phrases. <b><i>Results:</i></b> Using intraclass correlations (ICCs), Phase 1 results showed poor reliability for targets elicited for words (ICC = 0.07) and sentences/phrases (ICC = 0.42). Differences in classification of errors as glottal stops and consonant deletion accounted for this. Following further training, a second reliability study was undertaken showing improvement in the number of targets elicited in words (ICC = 0.85) and sentences/phrases (ICC = 0.94). There was very good inter-rater reliability for the PCC score on the word dataset (ICC = 0.9) and the sentence dataset (ICC = 0.88). Very good intra-rater reliability (ICC = 1.0) was found for the PCC score in both words and sentences/phrases for each listener. One listener consistently gave higher modified PCC scores. <b><i>Conclusions:</i></b> In cleft speech intervention studies, reliability of the number of targets elicited should be reported. Listeners need to distinguish between glottal articulation and consonant deletion, in order that the PCC score is meaningful. Attention should be paid to where listeners are reliable, but their pattern of scores consistently differs but in a consistent way. More research is needed on measuring the resolution of articulation difficulties in cleft intervention studies.

<b><i>背景与目的：</i></b> 辅音正确率（Percent consonant correct, PCC）最初由Shriberg与Kwiatkowski在[《言语与听觉障碍杂志》（J Speech Hear Disord）1982年8月;47(3):256–270]一文中提出，作为语音障碍的严重程度量化指标，现已被诸多语音障碍相关研究采纳并广泛应用。学界公认腭裂语音（cleft speech）表现复杂，需同时评估多个相互关联的参数，且异常语音可能不属于评估者的母语范畴。在语音结局研究中，窄式语音转录以及组内与组间信度（intra- and inter-rater reliability）的报告被视为金标准。然而，腭裂语音为该研究流程带来了特殊挑战，因为复杂的语音障碍往往与转录者间一致性较低相关。鉴于PCC的使用普及度与认知度较高，近期研究支持在腭裂语音干预研究中采用PCC作为主要结局指标的决策。本研究旨在针对干预研究场景，评估PCC的组内与组间信度，这与其他类型的语音结局研究存在显著差异。

<b><i>方法：</i></b> 两名经过系统培训且经验丰富的评估者对119段录音进行了分析，这些录音随机选自干预前、干预中及干预后的5个时间点采集的数据。研究分别针对单词与句子/短语单独计算了PCC评分。

<b><i>结果：</i></b> 采用组内相关系数（intraclass correlations, ICCs）进行分析，第一阶段结果显示，单词诱发测试目标的信度较差（ICC=0.07），句子/短语诱发测试目标的信度亦不佳（ICC=0.42）。将错误归类为声门塞音（glottal stops）与辅音省略（consonant deletion）时出现的差异，正是导致该结果的原因。经过进一步培训后，研究团队开展了第二阶段信度评估，结果显示单词诱发测试目标的评分一致性显著提升（ICC=0.85），句子/短语诱发测试目标的评分一致性亦得到改善（ICC=0.94）。单词数据集的PCC评分与句子数据集的PCC评分均展现出极佳的组间信度，前者ICC值为0.9，后者为0.88。两名评估者针对单词与句子/短语的PCC评分均展现出极佳的组内信度，ICC值均为1.0。其中一名评估者始终给出更高的修正版PCC评分。

<b><i>结论：</i></b> 在腭裂语音干预研究中，应当报告诱发测试目标数量的信度。评估者需明确区分声门构音与辅音省略，方能保证PCC评分具备临床参考价值。需关注评估者信度良好但评分模式始终存在系统性差异的情况。未来仍需开展更多相关研究，以精准评估腭裂语音干预研究中构音障碍的改善效果。