Comparison of the sensitivity of Self-Attention layer and FFN in BERT.

The result indicates that Self-Attention (SA) layer is more sensitive than FFN.实验结果表明，自注意力（Self-Attention，SA）层相较于前馈网络（FFN）更为敏感。