Quantitative Evaluation of Ion Chromatogram Extraction Algorithms

Extracted ion chromatograms (XIC)
are the fundamental signal unit in mass spectrometry. There are many
algorithms for analyzing raw mass spectrometry data tasked with distinguishing
real isotopic signals from noise. While one or more of the available
algorithms are typically chained together for end-to-end mass spectrometry
analysis, analysis of each algorithm in isolation provides a specific
measurement of the strengths and weaknesses of each approach. Though
qualitative opinions on extraction algorithm performance abound, quantitative
performance has never been publicly ascertained. Quantitative evaluation
has not occurred partly due to the lack of an available quantitative
ground truth MS1 data set. Using a recently published, manually extracted
XICs as ground truth data, we evaluate the quality of popular XIC
algorithms, including MaxQuant, MZMine2, and several methods from
XCMS. The manually curated data set comprises 48 human proteins stratified
over 6 abundance orders of magnitude. Signals in the sample were manually
curated into XIC using a commercial tool for visually identifying
XIC and isotopic envelopes. XIC algorithms were applied to the manually
extracted data using a grid search of possible parameters. Performance
varied greatly between different parameter settings, though nearly
all algorithms with parameter settings optimized with respect to the
number of true positives recovered over 10 000 XICs.