Understanding API Usage at Scale: An Empricial Study (Artifact)

LibProbe Artifact

For the sake of the evaluation we preprocessed the CCScanner data to identify all clients of the libraries used in our evaluation and included those in our MongoDB database. This artifact will start by importing this pre-processed dependency information into a docker image which is then used for the evaluation.

Running the docker image

First load the docker image by running

gunzip -c libprobe_v1.1.tar.gz | sudo docker import - libprobe:latest

this will load the image in your local docker images.

Run a container from the image: docker run -it --device /dev/snd --privileged libprobe:latest /bin/bash This will run a docker container which maps the pulseaudio and alsa configurations from your local host to the docker image. This is necessary to get some clients for some target libraries to build correctly. You will need an ubuntu host machine which has pulseaudio and alsa installed.

Validating analysis results

The docker image provided does not contain any clients due to size limitations on sharing. The Mongo database contains all the results of running this evaluation.

To get the results it's possible to run python3 libprobe.py analyse all -n from /tmp/libprobe. This will overwrite the JSON files in the json_files directory and overwrite the graphs in the graphs directory.

Running the evaluation for one library (vorbis)

Download clients: Go to \tmp\libprobe and run python3 libprobe.py download vorbis

Process the library to get the APIs and the coverage information : python3 libprobe.py processlib vorbis

Prepare clients for excluding sub directories that might contain vorbis library code: python3 libprobe.py prepclients vorbis

Get client usages: python3 libprobe.py fetchusages vorbis

Analyse: python3 libprobe.py analyse vorbis -n

(optional) Measure differential coverage for improved coverage libs: python3 libprobe.py coverage vorbis

Running the evaluation for all libraries (this requires at least 300GB of disk space)

Download clients: Go to \tmp\libprobe and run python3 libprobe.py download all

Process the libraries: python3 libprobe.py processlib all

Prepare clients: python3 libprobe.py prepclients all

Get usages: python3 libprobe.py fetchusages all

Analyse: python3 libprobe.py analyse all -n

(optional) Measure differential coverage for improved coverage libs: python3 libprobe.py coverage <library>

Getting baseline coverage for libraries

All libraries are cloned in /tmp/data/libraries and clients are cloned in /tmp/data/clients.

MBedtls: Copy the script coverage.sh from /tmp/libprobe/extra into the the build directory of Mbedtls and run ./coverage.sh baseline this will calculate the baseline coverage for mbedtls.

FFTW: Copy the script coverage.sh from /tmp/libprobe/extra into the the root dir of FFTW and run ./coverage.sh baseline this will calculate the baseline coverage for fftw

HDF5: Copy the script coverage_hdf.sh from /tmp/libprobe/extra into the the root dir of HDF and run ./coverage_hdf.sh baseline this will calculate the baseline coverage for HDF.

LMDB: Copy the script coverage_lmdb.sh from /tmp/libprobe/extra into the /tmp/data/libraries/LMDB@@lmdb/libraries/liblmdb and run ./coverage_lmdb.sh baseline this will calculate the baseline coverage for LMDB.

Zip: Copy the script coverage_zip.sh from /tmp/libprobe/extra into /tmp/data/libraries/kuba--@@zip/build/CMakeFiles/zip.dir/src and run ./coverage_zip.sh baseline this will calculate the baseline coverage for zip.

Vorbis: Copy the script cal_cov.py from /tmp/libprobe/extra to /tmp/data/libraries/xiph@@vorbis/lib/.libs and then copy all source files in the .libs folder by running cp ../*.c . from the .libs folder. Finally run python3 cal_cov.py ..

XXhash: Copy the script coverage.sh from /tmp/libprobe/extra into /tmp/data/libraries/Cyan4973@@xxHash and run ./coverage.sh baseline this will calculate the baseline coverage for xxhash

Reproducing increased coverage using clients

LMDB: The client we will use is Knot DNS.

Change directory to /tmp/data/clients/CZ-NIC@@knot and run autogen.sh.

Run ./configure --with-lmdb=/usr/local.

Then make && make check.

Now go back to the LMDB directory and run

Run ./coverage_lmdb.sh after_knot.

Now go /tmp/libprobe and run python3 libprobe.py coverage lmdb

VORBIS: The client we will use in SFML.

Go to the vorbis library dir /tmp/data/libraries/xiph@@vorbis and run make clean.

Run make && make check && make install.

Go to the .libs folder and copy all c files there by doing cp ../*.c ..

Copy /tmp/libprobe/extra/cal_cov.py into the .libs folder and run python3 cal_cov.py .. This will show the baseline coverage.

Go to the SFML directory /tmp/data/clients/SFML@@SFML.

Create build directory mkdir build && cd build.

Run cmake -DSFML_BUILD_TEST_SUITE=TRUE -GNinja ...

Run ninja.

Run ctest. You will see some failing tests. Thats normal as we are only interested in the Audio tests for vorbis. All Audio tests should pass.

Go back to the .libs folder in vorbis and re-run the cal_cov.py script.

Now go /tmp/libprobe and run python3 libprobe.py coverage vorbis