BLEU scores according to different matrix-wise pairs in the encoder.

Matrix-wise pairs from improve the performance of Transformer the most.

NIAID Data Ecosystem10

BLEU scores according to different matrix-wise pairs in the decoder.

Matrix-wise pairs perform differently in the decoder, unlike in the encoder. Specifically, matrix-wise pairs of the encoder-decoder-attention in the decoder degrade the BLEU scores significantly.

Figshare2023-07-06 更新00

BLEU scores according to different layer-wise pairs.

No pair denotes a four-layer vanilla Transformer model without layer-wise and matrix-wise pairs. Layer-wise parameter-shared model with pairs from {(L1, L3), (L2, L4)} shows the best accuracy with hal

NIAID Data Ecosystem00