Artifact: How Profilers Can Help Navigate Type Migration

Artifact for OOPSLA'23 paper: _How Profilers Can Navigate Type Migration_ ### Contents: - `artifact.tar.gz` <br> code for reproducing our rational programmer experiment<br> - `benchmarks.tar.gz`<br> GTP Benchmarks without and with modifications<br> - `cloudlab.tar.gz`<br> for measuring performance on CloudLab<br> - `figure-data.tar.gz`<br> figures and summarized data for the paper<br> - `rational-trails.tar.gz`<br> output from the rational programmer<br> - `raw-data.tar.gz`<br> running times, boundary profile output, and statistical profile output for all benchmarks OOPSLA AEC: for artifact evaluation, start with `artifact.tar.gz`.<br> Ignore the other datasets. Follow instructions in `artifact/README.md`. <br> ### How it fits together: 1. The GTP Benchmarks are the starting point. We investigated profiler-based<br> navigation on them.<br> 2. For every configuration of most benchmarks (see paper), we used CloudLab<br> to collect raw performance data: running times &amp; profiler output.<br> 3. Our Rational Programmer used the performance data to navigate lattices. It<br> created one trail for every benchmark configuration.<br> 4. We created several figures to analyze the trail data. The artifact has code for running the rational programmer experiment,<br> generating figures, and rendering a PDF. <br> ### What's in the data? - `artifact.tar.gz`<br> * `rational-experiment/` code for running the rational programmer, sample output<br> * `render/` code for analyzing rational programmer output and making figures<br> * `tex/` output from the rendering script<br> * `main.tex` source for a PDF that rebuilds the main figures from the paper<br> * `Makefile` build script for `main.tex`<br> * `all.pdf` example output based on all benchmarks<br> * `fsm.pdf` example output based on the `fsm` benchmark alone - `benchmarks.tar.gz`<br> * `gtp-bench/` GTP Benchmarks v7.0<br> * `bnd-bench/` a few benchmarks that we modified to help the boundary profiler<br> deal with adaptor modules (see paper, appendix A) - `cloudlab.tar.gz`<br> * `profile.py` a CloudLab script that makes an experiment profile (see CloudLab docs)<br> * `rds-cloudlab.tar.gz` initial code for the CloudLab profile, which contains source<br> code for the Racket programming language (the `profile.py` script should<br> download a copy of this archive from Brown University)<br> * `install_boundary.sh` run this to enable boundary profiling<br> * `install_statistical.sh` run this to enable statistical profiling<br> * `navigation-extra.tar.gz` support code for the install scripts<br> + contains a script `bmg.rkt` for running profilers<br> * `output.tar.gz` a blank gtp-measure output directory - `figure-data.tar.gz`<br> * output from our analysis of the Rational Programmer trails<br> * copy to `artifact/render/figure-data/`, then run `artifact/render/tex.rkt`,<br> and finally run `artifact/Makefile` to rebuild the main figures from the<br> paper using our figure data - `rational-trails.tar.gz`<br> * output from the Rational Programmer<br> * copy to `artifact/rational-experiment/out/`, then run `artifact/render/mkdata.rkt`<br> to regenerate figure data from our rational programmer output (then run `tex.rkt`<br> and `Makefile` to rebuild figures) - `raw-data.tar.gz` (beware, 5GB unpacked)<br> * `runtime/` running times collected by gtp-measure<br> * `profile/` statistical profiler output<br> * `boundary/` boundary profiler output <br> ### Cloudlab Instructions Instantiating the CloudLab profile is the first step toward reproducing our<br> experiments. To collect performance data, upload the benchmarks, unzip the `output.tar.gz` archive,<br> delete every empty file `output/**/*.out`, and run gtp-measure: ```<br> cd rds-cloudlab<br> sh run-benchmarks.sh<br> ``` To collect (boundary / statistical) profiler data, run the matching install<br> script. Then upload the benchmarks you wish to measure and find the matching<br> file `output/**/*.in`. Upload to your home directory. Use the `bmg.rkt` script to collect. Example: ```<br> cd rds-cloudlab ./racket-8.6.0.2/bin/racket bmg.rkt -b ../fsm ../fsm.*.in<br> # b = boundary profile ./racket-8.6.0.2/bin/racket bmg.rkt -p ../fsm ../fsm.*.in<br> # p = statistical profile<br> ```