NJR-1 Dataset

NJR is a Normalized Java Resource. The <em>NJR-1</em> dataset consists of 293 Java programs. <strong>TOOLS THAT RUN ON NJR-1</strong> Each program runs successfully with the following 13 Java static analysis tools: SpotBugs (https://spotbugs.github.io) Wala (https://wala.github.io) Doop (https://bitbucket.org/yanniss/doop) Soot (https://github.com/soot-oss/soot) Petablox (https://github.com/petablox/petablox) Infer (https://fbinfer.com) Error-Prone (http://errorprone.info) Checker-Framework (https://checkerframework.org) Opium (Opal-framework) (https://www.opal-project.de) Spoon (https://spoon.gforge.inria.fr) PMD (https://pmd.github.io) CheckStyle (https://checkstyle.org) Codeguru* (https://aws.amazon.com/codeguru) In addition to these static analysis tools, the NJR dataset has also been tested with 6 other tools that operate on Java bytecode. Jacoco (https://www.jacoco.org): Dynamic analysis tool Wiretap (https://github.com/ucla-pls/wiretap): Dynamic analysis tool JReduce (https://github.com/ucla-pls/jreduce): Bytecode reduction tool Procyon (https://github.com/ststeiger/procyon): Decompiler CFR (https://www.benf.org/other/cfr/): Decompiler Fernflower (https://github.com/fesh0r/fernflower): Decompiler <strong>BENCHMARK PROGRAMS</strong> The NJR programs are repositories picked from a set of Java-8 projects on Github that compile and run successfully. Each of these programs come with a jar file, the compiled bytecode files, compiled library files and the Java source code. The availability of the files in both jar-file form, as well as source code form (with the compiled library classes) is a major reason the dataset works with so many tools, without requiring any extra effort. Internally, each benchmark program has the following structure: <em>src:</em> directory with source files. <em>classes:</em> directory with class files. <em>lib:</em> compiled third party library classes (source files not available, since libraries are distributed as class-files). <em>jarfile:</em> jar file containing the compiled application classes and third-party library classes. <em>info:</em> directory with information about the program. It includes the following files. <em>classes:</em> list of application classes (excludes third-party library classes). <em>mainclasses:</em> list of main classes that can be run. <em>sources:</em> list of source file names. <em>declarations:</em> list of method declarations categorized by source file name. The benchmarks already come with a compiled JAR file, but some tools need to compile and run the benchmarks. The following simple commands can be used for compilation and running (replace &lt;jarfilename&gt; with the file in the <em>jarfile</em> directory. replace &lt;mainclassname&gt; with any of the classes from info/mainclasses): <em>javac -d compiled_classes -cp lib @info/sources</em> <em>java -cp jarfile/&lt;jarfilename&gt; &lt;mainclassname&gt;</em> <strong>FILES AVAILABLE FOR DOWNLOAD</strong> There are 3 files available for download: <em>njr-1_dataset.zip, scripts.zip, benchmark_stats.csv.</em> <em>njr-1_dataset.zip</em> has the actual dataset programs. <em>scripts.zip</em> contains Python3 scripts for each tool, to run it on the entire dataset. It also contains a Readme detailing the version number, download link and setup instructions for each tool. The <em>benchmark_stats.csv</em> file lists, for each benchmark, the number of nodes and edges in its dynamic application call-graph, as well as the number of edges in its static application call-graph (as computed by Wala) when using the main function listed in the <em>info/mainclassname</em> file. <strong>STATISTICS</strong> Here are some statistics about the benchmark programs Mean number of application classes: 97 Each program executes at least 100 unique application methods at runtime. Mean lines of application source code: 9911 Mean number of 3rd party library classes: 2608 Mean (estimated) lines of 3rd party library source code: 250,000 A summary of the statistics from the benchmark_stats file is listed here: <em>Statistics Dynamic-Nodes Dynamic-Edges Static-Edges</em><br> Mean 205 469 1404<br> St.Dev 199 464 2523<br> Median 149 327 610 <strong>NOTES</strong> Note: Zenodo shows 4 changes for this repository. However all the changes involve updating the scripts folder, as more tools get tested on the dataset. The programs in the dataset themselves remain unchanged. *Note 2: Codeguru Reviewer is a paid, proprietary tool by Amazon. Our experiments show that it runs successfully on all the benchmarks in this dataset. However, we don't include any scripts to replicate this run because of its paid nature. To cite the dataset, please cite the following paper:<br> Jens Palsberg and Cristina V. Lopes, NJR: a Normalized Java Resource. <br> In Proceedings of ACM SIGPLAN International Workshop on State Of the Art in Program Analysis (SOAP), 2018.