Does GST underestimate genetic differentiation from marker data?

The widely applied genetic differentiation statistics FST and GST have recently been criticized for underestimating differentiation when applied to highly polymorphic markers such as microsatellites. New statistics claimed to be unaffected by marker polymorphisms have been proposed and advocated to replace the traditional FST and GST. This study shows that GST gives accurate estimates and underestimates of differentiation when demographic factors are more and less important than mutations, respectively. In the former case, all markers, regardless of diversity (HS), have the same GST value in expectation and thus give replicated estimates of differentiation. In the latter case, markers of higher HS have lower GST values, resulting in a negative, roughly linear correlation between GST and HS across loci. I propose that the correlation coefficient between GST and HS across loci, rGH, can be used to distinguish the two cases and to detect mutational effects on GST. A highly negative and sig...