Multi-Agent Root-Cause Diagnosis for Kernel Crashes

# Artifact for Multi-Agent Root-Cause Diagnosis for Kernel Crashes ## Artifact Structure - `MA-RCA`: Main code and dataset. - `kernel-diag-docker.tar`: docker environment with all code, data, and dependencies. - `sweagent-py311-kernel.tar`: python 3.11 environment for swe-agent, compiled with a latest linux repo. - `SWE-ReX`: a modified version of swe-rex, to support custom parameters. - `my-SWE-agent`: a modified version of swe-agent, enable multi-agent root cause diagnosis. - `explicit.txt`: conda environment for `MA-RCA` ## Project Introduction Main code and dataset are as follows: ``` |-- RootCauseEvidenceGraph # Main code dir | |-- eval_cal.py # calculate evaluation metrics | `-- pipeline.py # main pipeline for root cause diagnosis |-- data # Main dataset | `-- logRepro | |-- bug_id_range # input bug ids | |-- multi_agent-empty # all pre-pocessed data without results ` `-- multi_agent-with-result # all pre-pocessed data with results ``` ## Docker based setup We highly recommend using Docker to run the project. We provide a docker-in-docker environment, together with all necessary dependencies (python, swe-agent, docker, etc.). 1. Import pre-compiled Docker image. ``` docker load -i kernel-diag-docker.tar ``` 2. Run the container. ``` # create an empty dir to store docker files in. Important! mkdir -p /path/to/docker-files # mount your current dir, e.g., kernel-diag (Where project is located), to /data. Don't change this name! # export 22 if you want to access the container via ssh. docker run -it \ --privileged \ -p PORT:22 \ --name kernel-diag \ --shm-size=16g \ -v /path/to/docker-files:/var/lib/docker \ -v /path/to/kernel-diag:/data \ kernel-diag:latest \ /bin/zsh ``` All the code and dataset will be located inside the docker container. Check `/data` dir. prepare docker in docker. ``` # start up a tmux session, and run docker commands inside the session. tmux dockerd # then go back to main session ctrl+b d # load pre-compiled docker image for swe-agent docker load -i sweagent-py311-kernel.tar # check if the image is loaded successfully docker images ``` ## Running the project 1. Setup LLM api key. We recommend using DEEPSEEK api key. ``` vi /data/my-SWE-agent/.env # then add you LLM api key ``` 2. Run root cause diagnosis for preprocessed dataset. First set up conda env. ``` conda activate sweagent cd /data/MA-RCA ``` Then prepare the preprocessed dataset. ``` mv data/logRepro/multi_agent-empty data/logRepro/multi_agent # run main pipeline # Note this will run for a very long time. And spent lots of llm api tokens. # if you just want to evaluate on a small subset of dataset, change kgym_repro_all_279.txt to kgym_repro_demo_1.txt # this will only run for one data. You can also edit the input at data/logRepro/bug_id_range/ python RootCauseEvidenceGraph/pipeline.py --input kgym_repro_all_279.txt --workers 10 # obtain results in terms of evaluation metrics python RootCauseEvidenceGraph/eval_cal.py --input kgym_repro_all_279.txt ``` We also provide all the results and we recommend using them directly. ``` mv data/logRepro/multi_agent-with-result data/logRepro/multi_agent python RootCauseEvidenceGraph/eval_cal.py --input kgym_repro_all_279.txt ``` This will show the results presented in RQ1 in our paper. To reproduce the results of RQ2, please use pipeline\_xxx.py. The details are as follows: ``` |-- pipeline_for_flat_text.py # no evidence graph |-- pipeline_for_no_log_mapping.py # no log maping |-- pipeline_single_agent.py # single agent |-- pipeline_wo_log.py # w/o log |-- pipeline_wo_report.py # w/o report |-- pipeline_wo_syscall.py # w/o syscall ``` The usage is same as `pipeline.py`. ## Human evaluation & LLM-assisted evaluation The results of RQ3 are located at `data/logRepro/human_evaluation/`