Supplementary Material for: Percent Consonant Correct as an Outcome Measure for Cleft Speech in an Intervention Study

Background/Aims: Percent consonant correct (PCC) was originally described by Shriberg and Kwiatkowski [J Speech Hear Disord. 1982 Aug;47(3):256–70] as a severity metric for phonological speech disorders, and has been adapted and used in many studies on speech sound disorders. It is well-recognized that cleft speech is complex, consisting of several interacting parameters assessed simultaneously, with error sounds not in the listener’s own language. In speech outcome studies, narrow phonetic transcription and the reporting of intra- and inter-rater reliability are acknowledged as the gold standard. However, cleft speech brings special challenges to this task, as complex speech disorders are known to be associated with low transcriber agreement. Recent studies informed the decision to use PCC as the primary outcome measure in a cleft speech intervention study, given its common usage and familiarity. The aim was to specifically evaluate the intra- and inter-rater reliability of PCC in an intervention study, in contrast to other types of speech outcome studies. Methods: Two trained and experienced listeners analyzed 119 recordings, randomly selected from five data points before, during, and following intervention. The PCC score was separately calculated for words and sentences/phrases. Results: Using intraclass correlations (ICCs), Phase 1 results showed poor reliability for targets elicited for words (ICC = 0.07) and sentences/phrases (ICC = 0.42). Differences in classification of errors as glottal stops and consonant deletion accounted for this. Following further training, a second reliability study was undertaken showing improvement in the number of targets elicited in words (ICC = 0.85) and sentences/phrases (ICC = 0.94). There was very good inter-rater reliability for the PCC score on the word dataset (ICC = 0.9) and the sentence dataset (ICC = 0.88). Very good intra-rater reliability (ICC = 1.0) was found for the PCC score in both words and sentences/phrases for each listener. One listener consistently gave higher modified PCC scores. Conclusions: In cleft speech intervention studies, reliability of the number of targets elicited should be reported. Listeners need to distinguish between glottal articulation and consonant deletion, in order that the PCC score is meaningful. Attention should be paid to where listeners are reliable, but their pattern of scores consistently differs but in a consistent way. More research is needed on measuring the resolution of articulation difficulties in cleft intervention studies.

背景与目的：辅音正确百分比（Percent consonant correct, PCC）最初由Shriberg与Kwiatkowski在[J Speech Hear Disord. 1982 Aug;47(3):256–70]一文中将其定义为音系言语障碍的严重程度评估指标，此后被广泛适配并应用于诸多语音障碍相关研究中。学界已达成共识：腭裂语音（cleft speech）结构复杂，需同时评估多个相互关联的参数，且其错误发音并非听者母语中的音素。在语音结局研究中，窄式音素转录（narrow phonetic transcription）以及评分者内部一致性信度（intra-rater reliability）、评分者间一致性信度（inter-rater reliability）的报告，被公认为金标准。但腭裂语音为此类任务带来了特殊挑战，因为复杂的语音障碍往往与转录者一致性偏低相关。鉴于PCC的使用场景广泛且研究者较为熟悉，近期相关研究为本次腭裂语音干预研究选用PCC作为主要结局指标提供了依据。与其他类型的语音结局研究不同，本研究的目的为专门评估腭裂语音干预研究中PCC的评分者内部一致性信度与评分者间一致性信度。

方法：招募2名受过专业训练且经验丰富的评分者，对从干预前、干预中及干预后五个数据节点中随机抽取的119段语音录音进行分析。分别针对单词与句子/短语计算PCC得分。

结果：采用组内相关系数（intraclass correlations, ICCs）进行分析，第一阶段结果显示：单词测试项的评分一致性较差（ICC=0.07），句子/短语测试项的评分一致性一般（ICC=0.42）。将错误发音归类为声门塞音（glottal stops）与辅音省略（consonant deletion）时出现的判断差异，是导致该结果的原因。经过进一步培训后，开展了第二阶段信度研究，结果显示：单词测试项的评分一致性得到显著提升（ICC=0.85），句子/短语测试项的评分一致性亦大幅改善（ICC=0.94）。单词数据集的PCC得分具有极佳的评分者间一致性（ICC=0.9），句子数据集的PCC得分同样具有极佳的评分者间一致性（ICC=0.88）。两名评分者各自的单词与句子/短语PCC得分均展现出极佳的评分者内部一致性信度（ICC=1.0）。其中一名评分者给出的修正版PCC（modified PCC）得分始终偏高。

结论：在腭裂语音干预研究中，需报告测试项数量的评分一致性信度。评分者需明确区分声门构音与辅音省略，方能保证PCC得分具备临床意义。需关注评分者一致性较高但得分模式始终存在系统性差异的情况。未来仍需开展更多研究，以评估腭裂干预研究中构音困难的改善效果。