On the Implementation of OS-Specific Tests: The CPython Case

This dataset was utilized in the research paper titled "On the Implementation of OS-Specific Tests: The CPython Case" for publication in Brazilian Symposium on Systematic and Automated Software Testing (SAST). Herein, we present a concise abstract of the study. Modern software systems are frequently developed and tested across multiple platforms.Consequently, practitioners may need to handle platform-specific code in production or test code.In the software testing context, practitioners adapt the tests to run differently according to the target platform.These tests, which need to identify the platform on which they will be executed, are referred to as OS-specific tests.In this paper, we present an empirical study to evaluate how developers implement OS-specific tests in CPython. This knowledge can be used to understand OS-specific tests better and discover common issues faced by practitioners when testing across multiple platforms.We mine the canonical implementation of the Python programming language and assess their OS-specific tests quantitatively.We propose three research questions to assess the frequency, location, and issues related to OS-specific tests.Our results show that OS-specific tests are common in the CPython project, and 13% of the analyzed test files are OS-specific tests (RQ1).OS Identification APIs in tests are used more frequently in test decorators (53.46%) than in test code (46.54%), and the test decorator @unittest.skipUnless is the most used to skip tests depending on the platform (RQ2).We find 170 issues related to OS-specific tests in CPython. Windows is the most targeted platform, and 27.06% of the test decorators are related to issues (RQ3).Lastly, we discussed practical implications for practitioners and researchers.Based on our findings, we discussed the importance of testing on multiple platforms, the relationship between issues and related to OS-specific tests, the cascading effects that may occur with CPython-dependent systems, and we relate the issues to vulnerabilities.