ARcode: HPC Application Recognition Through Image-encoded Monitoring Data

This tar file contains the docker image for building the ARcode model and baseline models for application recognition for the SC22 paper with the same title. The files/folders in this image contains: notebooks: The notebooks for models and experiment results.-- ARcode.ipynb: The interactive Jupyter Notebook for the ARcode model.-- ARcode_unknown.ipynb: The interactive Jupyter Notebook for the ARcode model for detecting unknown applications.-- ARcode_partial.ipynb: The interactive Jupyter Notebook for the ARcode model on partial job signatures.-- ARcode_channel.ipynb: The interactive Jupyter Notebook for the ARcode model on one channel of job signatures.-- baselines.ipynb: The interactive Jupyter Notebook for the baseline models. These models are Random Forest, LinearSVC and SVC; all of them are implemented through Taxonomist(https://doi.org/10.6084/m9.figshare.6384248.v1).-- baselines_unknown.ipynb: The interactive Jupyter Notebook for the baseline models for detecting unknown applications. dataset: The dataset for training the models mentioned above.-- ARcode_labels.npy: A numpy array of the signatures' labels.-- ARcode_signatures.npy: A numpy array of the generated signatures.-- baseline_labels.npy: A numpy array of the labels for the baseline dataset.-- baseline_features.npy: A numpy array of the statistic features generated from the raw monitoring data.-- knl_app_code.json: Mapping of IDs to application names. This mapping is used when creating the dataset. models: The saved models.-- arcode.h5: An HDF5 file containing the serialized weights for the ARcode model.-- arcode.json: A JSON file describing the ARcode model. results: The saved experiment results. Following these steps to start Jupyter Notebook in the image: 1. Load the image into Docker on your local machine:docker load < archive-arcode.tar 2. Start the Jupyter notebook in the docker image:docker run --init --user root -p 8888:8888 artlands/arcode 3. Copy the URL shown in your terminal and paste in a brower: http://127.0.0.1:8888/?token=your_token Acknowledgement: This research used resources of the National Energy Research Scientific Computing Center (NERSC), a U.S. Department of Energy Office of Science User Facility located at Lawrence Berkeley National Laboratory, operated under Contract No. DE-AC02-05CH11231本tar压缩包包含用于构建ARcode模型与基线模型的Docker镜像，对应SC22同名论文中的应用识别任务。镜像内的文件与文件夹说明如下： notebooks目录：存放模型相关脚本与实验结果 - ARcode.ipynb：用于运行ARcode模型的交互式Jupyter Notebook - ARcode_unknown.ipynb：用于ARcode模型检测未知应用的交互式Jupyter Notebook - ARcode_partial.ipynb：针对部分作业特征运行ARcode模型的交互式Jupyter Notebook - ARcode_channel.ipynb：针对单通道作业特征运行ARcode模型的交互式Jupyter Notebook - baselines.ipynb：基线模型的交互式Jupyter Notebook，所涉模型包括随机森林（Random Forest）、线性支持向量机（LinearSVC）与支持向量机（SVC），所有模型均通过Taxonomist实现（https://doi.org/10.6084/m9.figshare.6384248.v1） - baselines_unknown.ipynb：用于基线模型检测未知应用的交互式Jupyter Notebook dataset目录：用于训练上述模型的数据集 - ARcode_labels.npy：存储作业特征标签的NumPy数组 - ARcode_signatures.npy：存储生成的作业特征的NumPy数组 - baseline_labels.npy：存储基线数据集标签的NumPy数组 - baseline_features.npy：从原始监控数据中提取的统计特征的NumPy数组 - knl_app_code.json：应用ID与名称的映射表，用于数据集构建环节 models目录：已保存的模型文件 - arcode.h5：存储ARcode模型序列化权重的HDF5文件 - arcode.json：描述ARcode模型结构的JSON文件 results目录：已保存的实验结果 镜像内Jupyter Notebook的启动步骤如下： 1. 在本地Docker环境加载该镜像：docker load < archive-arcode.tar 2. 在Docker镜像中启动Jupyter Notebook：docker run --init --user root -p 8888:8888 artlands/arcode 3. 复制终端中显示的URL并粘贴至浏览器访问：http://127.0.0.1:8888/?token=your_token 致谢：本研究使用了美国能源部科学用户设施国家能源研究科学计算中心（National Energy Research Scientific Computing Center, NERSC）的计算资源，该中心位于劳伦斯伯克利国家实验室，根据合同号DE-AC02-05CH11231运营。