Confidence and Prediction in Linear Mixed Models: Don’t Concatenate the Random Effects. Application in an Assay Qualification Study

In the pharmaceutical industry, all analytical methods must be shown to deliver unbiased and precise results. In an assay qualification or validation study, the trueness, accuracy and intermediate precision are usually assessed by comparing the measured concentrations to their nominal levels. Trueness is assessed by using Confidence Intervals of mean measured concentration, accuracy by Prediction Intervals for a future measured concentration, and the intermediate precision by the Total Variance. ICH and USP guidelines alike request that all relevant sources of variability must be studied, e.g. the effect of different technicians, the day-to-day variability or the use of multiple reagent lots. Those different random effects must be modeled as crossed, nested or a combination of both; while concatenating them to simplify the model is often taken place. This paper compares this simplified approach to a mixed model with the actual design. Our simulation study shows an under-estimation of the intermediate precision and, therefore, a substantial reduction of the confidence and prediction intervals. The power for accuracy or trueness is consequently over-estimated when designing a new study. Two real datasets from assay validation study during vaccine development are used to illustrate the impact of such concatenation of random variables.

在制药行业中，所有分析方法均需被证明可提供无偏且精准的检测结果。在分析方法确认或验证研究中，通常通过将实测浓度与其标称浓度进行对比，以评估正确度（trueness）、准确度（accuracy）与中间精密度（intermediate precision）。其中，正确度通过实测浓度均值的置信区间（Confidence Intervals）进行评估，准确度通过未来实测浓度的预测区间（Prediction Intervals）进行评估，而中间精密度则通过总方差（Total Variance）进行评估。ICH（人用药品注册技术要求国际协调会）与USP（美国药典，United States Pharmacopeia）的指导原则均要求，需对所有相关变异来源开展研究，例如不同操作人员的影响、日间变异，或是多批次试剂的使用情况。上述各类随机效应需被建模为交叉随机效应、嵌套随机效应，或是二者的组合；而实践中常通过合并这些随机效应来简化模型。本文将这种简化建模方法与基于实际试验设计的混合效应模型（mixed model）进行对比。我们的模拟研究结果显示，该简化方法会低估中间精密度，进而导致置信区间与预测区间大幅收窄。在设计新研究时，这种简化操作会造成对准确度或正确度的检验效能（power）出现高估。本文采用两项来自疫苗研发过程中分析方法验证研究的真实数据集（real datasets），用以说明这种随机变量合并操作所带来的实际影响。