Intercomparison of remote sensing retrievals: an examination of prior-induced biases in averaging kernel corrections

Abstract: In remote sensing applications, Optimal Estimation (OE) retrievals are sometimes compared 2 to independent OE retrievals of the same process. This intercomparison is often done in instrument 3 validation, where retrievals are compared with data from a separate validation instrument, and it 4 is sometimes done in data assimilation, where data from multiple instruments need to be adjusted 5 to the ‘same footing.’ In these cases, the two different retrievals are compared using an adjustment 6 that is colloquially known as the averaging kernel correction. A general misconception in the 7 existing literature is that this averaging kernel correction removes any bias introduced by prior 8 misspecification by either (or both) of the two comparative OE retrievals. In this paper, we will 9 analytically show that this is not the case, and the averaging kernel correction process is implicitly 10 ‘shifting’ both OE retrievals to a common comparison prior. We will also show that there is generally 11 a non-zero bias that is proportional to the difference between this comparison prior mean and 12 the true (but unobserved) mean state, which has large implications for retrieval validation and 13 data assimilation in remote sensing. Finally, to better characterize OE retrievals and retrieval 14 intercomparisons, we will make some recommendations for mitigating this prior-induced bias 15 in intercomparison of OE retrievals.