Experimental Null Method to Guide the Development of Technical Procedures and to Control False-Positive Discovery in Quantitative Proteomics

Comprehensive and accurate evaluation
of data quality and false-positive
biomarker discovery is critical to direct the method development/optimization
for quantitative proteomics, which nonetheless remains challenging
largely due to the high complexity and unique features of proteomic
data. Here we describe an experimental null (EN) method to address
this need. Because the method experimentally measures the null distribution
(either technical or biological replicates) using the same proteomic
samples, the same procedures and the same batch as the case-vs-contol
experiment, it correctly reflects the collective effects of technical
variability (e.g., variation/bias in sample preparation, LC–MS
analysis, and data processing) and project-specific features (e.g.,
characteristics of the proteome and biological variation) on the performances
of quantitative analysis. To show a proof of concept, we employed
the EN method to assess the quantitative accuracy and precision and
the ability to quantify subtle ratio changes between groups using
different experimental and data-processing approaches and in various
cellular and tissue proteomes. It was found that choices of quantitative
features, sample size, experimental design, data-processing strategies,
and quality of chromatographic separation can profoundly affect quantitative
precision and accuracy of label-free quantification. The EN method
was also demonstrated as a practical tool to determine the optimal
experimental parameters and rational ratio cutoff for reliable protein
quantification in specific proteomic experiments, for example, to
identify the necessary number of technical/biological replicates per
group that affords sufficient power for discovery. Furthermore, we
assessed the ability of EN method to estimate levels of false-positives
in the discovery of altered proteins, using two concocted sample sets
mimicking proteomic profiling using technical and biological replicates,
respectively, where the true-positives/negatives are known and span
a wide concentration range. It was observed that the EN method correctly
reflects the null distribution in a proteomic system and accurately
measures false altered proteins discovery rate (FADR). In summary,
the EN method provides a straightforward, practical, and accurate
alternative to statistics-based approaches for the development and
evaluation of proteomic experiments and can be universally adapted
to various types of quantitative techniques.