Spark application traces of the run of 9 differents instance of BigDataBench applications

Spark application logs of the run of 9 different instances of BigDataBench applications. Each run was done with an input size in GB of 32, 64, 128 and with 4, 8, 16 executor respectively. There were 2 executors per node; so, it runs on 2, 4, and 8 nodes + a master node for Hadoop services. The input data were generated with the Data generator provided by BigDataBench using this procedure: https://gitlab.inria.fr/mmercier/bebida/blob/master/experiments/generate_dataset/journal.md List of the applications and their parameters: Grep: parameter: "word" WordCount no parameters Kmean: parameter: "4 3" input size in GB: "32, 64, 128" BigDataBench implementation can be downloaded here: http://prof.ict.ac.cn/download.html It was run on Debian 8, with Spark 2.1.0, on top of Hadoop 2.7.1 with Yarn and HDFS, using openjdk-7-jre-headless. All the details of the environment can be found here: https://gitlab.inria.fr/mmercier/bebida/blob/master/environments/bebida-slave.yaml The experiment in itself is described here: https://gitlab.inria.fr/mmercier/bebida/tree/master/experiments/run_big_data_workload All nodes Hardware description of the nodes: https://public-api.grid5000.fr/stable/sites/nancy/clusters/graphene/nodes.json?pretty=1 They can be visualized using the Spark History server.本数据集包含9种不同BigDataBench应用实例的运行Spark日志。每项运行的输入规模分别为32GB、64GB、128GB，且分别配置4、8、16个执行器（Executor）。每个节点部署2个执行器，因此集群分别运行于2、4、8个工作节点外加1个承载Hadoop服务的主节点。 输入数据通过BigDataBench提供的数据生成器，按照下述流程生成： https://gitlab.inria.fr/mmercier/bebida/blob/master/experiments/generate_dataset/journal.md 应用列表及其参数如下： - Grep：参数为"word" - WordCount：无参数 - Kmean：参数为"4 3"，支持输入规模为32GB、64GB、128GB BigDataBench的实现代码可从以下地址获取：http://prof.ict.ac.cn/download.html 本次实验运行于Debian 8操作系统，搭载Spark 2.1.0框架，基于集成YARN与HDFS的Hadoop 2.7.1集群，并使用openjdk-7-jre-headless运行环境。 实验环境的详细配置可参见以下链接： https://gitlab.inria.fr/mmercier/bebida/blob/master/environments/bebida-slave.yaml 实验本身的详细流程说明可参考以下链接： https://gitlab.inria.fr/mmercier/bebida/tree/master/experiments/run_big_data_workload 所有节点的硬件详情可通过以下链接查看： https://public-api.grid5000.fr/stable/sites/nancy/clusters/graphene/nodes.json?pretty=1 实验结果可通过Spark历史服务器（Spark History Server）进行可视化查看。