Supplementary Material - Who Should Test the Requirement? A Comparative Study on Requirements Classification for Assigning Test Teams using the Pre-Trained Models

The supplementary material is related to the statistical tests performed to validate the results from the paper "Who Should Test the Requirement? A Comparative Study on Requirements Classification for Assigning Test Teams using the Pre-Trained Models" The goal of the statistical test was to investigate whether there is a significant difference between the BERT, XLNet, and ELECTRA models with regard to the probability assigned to the correct class of each requirement. For the statistical test, we used a set of probability distribution to the model classify correctly each class: Model Probability BERT 0,956809 BERT 0,942727 BERT 0,788117 BERT 0,922548 BERT 0,005487 BERT 0,942294 BERT 0,922247 BERT 0,115416 BERT 0,945645 BERT 0,945485 BERT 0,919123 BERT 0,946622 BERT 0,940579 BERT 0,954029 BERT 0,853265 BERT 0,846922 BERT 0,942807 BERT 0,140116 BERT 0,922081 BERT 0,917714 BERT 0,95244 BERT 0,919942 BERT 0,918669 BERT 0,963556 BERT 0,300082 BERT 0,998968 BERT 0,164602 BERT 0,998052 BERT 0,998711 BERT 0,004985 BERT 0,011434 BERT 0,002421 BERT 0,992682 BERT 0,004472 BERT 0,528385 BERT 0,021226 BERT 0,042576 BERT 0,998852 BERT 0,996028 BERT 0,13071 BERT 0,998036 BERT 0,903156 BERT 0,998949 BERT 0,036362 BERT 0,998974 BERT 0,06081 BERT 0,999034 BERT 0,99895 BERT 0,999011 BERT 0,998955 BERT 0,995763 BERT 0,042633 BERT 0,980891 BERT 0,99498 BERT 0,000471 BERT 0,995075 BERT 0,005023 BERT 0,001325 BERT 0,00048 BERT 0,000377 BERT 0,994233 BERT 0,000334 BERT 0,002129 BERT 0,955263 BERT 0,974376 BERT 0,992562 BERT 0,030702 BERT 0,004845 BERT 0,997385 BERT 0,002129 BERT 0,950778 BERT 0,99656 BERT 0,002736 BERT 0,870145 BERT 0,997288 BERT 0,99595 BERT 0,23246 BERT 0,779047 BERT 0,995905 BERT 0,995811 BERT 0,992693 BERT 0,995951 BERT 0,98359 BERT 0,995895 BERT 0,995951 BERT 0,993184 BERT 0,000285 BERT 0,992741 BERT 0,995833 BERT 0,042702 BERT 0,994797 BERT 0,994508 BERT 0,989475 BERT 0,989047 BERT 0,389256 BERT 0,995951 BERT 0,990609 BERT 0,995895 BERT 0,321135 BERT 0,993384 XLNet 0,990439 XLNet 0,986381 XLNet 0,987713 XLNet 0,992904 XLNet 0,001316 XLNet 0,986608 XLNet 0,992945 XLNet 0,628709 XLNet 0,988232 XLNet 0,99103 XLNet 0,98592 XLNet 0,991984 XLNet 0,992257 XLNet 0,989767 XLNet 0,996295 XLNet 0,917428 XLNet 0,986833 XLNet 0,996749 XLNet 0,989208 XLNet 0,988726 XLNet 0,989245 XLNet 0,985766 XLNet 0,98696 XLNet 0,536825 XLNet 0,928015 XLNet 0,999848 XLNet 0,427277 XLNet 0,99978 XLNet 0,999419 XLNet 0,00289 XLNet 0,993831 XLNet 0,013385 XLNet 0,999719 XLNet 0,78129 XLNet 0,999712 XLNet 0,980641 XLNet 0,972637 XLNet 0,999845 XLNet 0,999757 XLNet 0,979846 XLNet 0,999863 XLNet 0,999824 XLNet 0,999835 XLNet 0,000634 XLNet 0,999836 XLNet 0,99985 XLNet 0,999814 XLNet 0,999827 XLNet 0,999776 XLNet 0,999837 XLNet 0,998838 XLNet 0,000242 XLNet 0,990867 XLNet 0,998966 XLNet 0,581768 XLNet 0,997184 XLNet 9,16E-05 XLNet 0,983396 XLNet 0,916334 XLNet 0,977303 XLNet 0,969508 XLNet 0,913709 XLNet 0,8947 XLNet 9,87E-05 XLNet 0,988572 XLNet 0,997477 XLNet 0,984364 XLNet 0,000605 XLNet 0,99788 XLNet 0,8947 XLNet 0,975651 XLNet 0,997627 XLNet 0,000492 XLNet 0,035856 XLNet 0,998531 XLNet 0,996914 XLNet 0,673415 XLNet 0,992078 XLNet 0,992422 XLNet 0,997631 XLNet 0,100326 XLNet 0,997638 XLNet 0,998233 XLNet 0,997233 XLNet 0,997638 XLNet 0,998255 XLNet 0,0001 XLNet 0,997868 XLNet 0,997306 XLNet 0,006061 XLNet 0,997934 XLNet 0,997859 XLNet 0,998071 XLNet 0,997701 XLNet 0,002232 XLNet 0,997638 XLNet 0,998183 XLNet 0,997233 XLNet 0,00363 XLNet 0,99795 ELECTRA 0,994836 ELECTRA 0,909889 ELECTRA 0,735999 ELECTRA 0,98587 ELECTRA 0,001246 ELECTRA 0,995318 ELECTRA 0,786375 ELECTRA 0,013252 ELECTRA 0,993442 ELECTRA 0,984104 ELECTRA 0,987827 ELECTRA 0,992197 ELECTRA 0,991061 ELECTRA 0,994944 ELECTRA 0,991898 ELECTRA 0,988246 ELECTRA 0,991382 ELECTRA 0,243261 ELECTRA 0,790035 ELECTRA 0,989 ELECTRA 0,939608 ELECTRA 0,986161 ELECTRA 0,983756 ELECTRA 0,992828 ELECTRA 0,834312 ELECTRA 0,999558 ELECTRA 0,001553 ELECTRA 0,949012 ELECTRA 0,99927 ELECTRA 0,026367 ELECTRA 0,001708 ELECTRA 0,002642 ELECTRA 0,147262 ELECTRA 0,002407 ELECTRA 0,049383 ELECTRA 0,044357 ELECTRA 0,001551 ELECTRA 0,999395 ELECTRA 0,999389 ELECTRA 0,02797 ELECTRA 0,423016 ELECTRA 0,046879 ELECTRA 0,999569 ELECTRA 0,379341 ELECTRA 0,999565 ELECTRA 0,873975 ELECTRA 0,99949 ELECTRA 0,999564 ELECTRA 0,999223 ELECTRA 0,999566 ELECTRA 0,996647 ELECTRA 0,683473 ELECTRA 0,995532 ELECTRA 0,090389 ELECTRA 0,000536 ELECTRA 0,998183 ELECTRA 0,040802 ELECTRA 0,000281 ELECTRA 0,000366 ELECTRA 0,000945 ELECTRA 0,065098 ELECTRA 0,001892 ELECTRA 0,001633 ELECTRA 0,040839 ELECTRA 0,994601 ELECTRA 0,996824 ELECTRA 0,000888 ELECTRA 0,000283 ELECTRA 0,998762 ELECTRA 0,001633 ELECTRA 0,997606 ELECTRA 0,998009 ELECTRA 0,00106 ELECTRA 0,997959 ELECTRA 0,998995 ELECTRA 0,998202 ELECTRA 0,712276 ELECTRA 0,018504 ELECTRA 0,996856 ELECTRA 0,998098 ELECTRA 0,991718 ELECTRA 0,997952 ELECTRA 0,997871 ELECTRA 0,995693 ELECTRA 0,997952 ELECTRA 0,993072 ELECTRA 0,000149 ELECTRA 0,971749 ELECTRA 0,980839 ELECTRA 0,052526 ELECTRA 0,996982 ELECTRA 0,996549 ELECTRA 0,997504 ELECTRA 0,976286 ELECTRA 0,260132 ELECTRA 0,997952 ELECTRA 0,996947 ELECTRA 0,995693 ELECTRA 0,089357 ELECTRA 0,996514