CRAWDAD sapienza/probe-requests

Wireless probe requests collected in Rome between February and May 2013.Mobile devices try to automatically switch to WiFi connectivity whenever possible. To facilitate this automatic process, they store the list of the names (SSID) of the networks the user typically connects to and, periodically, these SSIDs are sent in broadcast in the form of Probe Request to search for available networks. The following questions then rise naturally: "What do your smartphone probes say about you?"; "Is it possible to infer meaningful relationships among a group of people just using their smartphones' probes?". To answer all these questions, we organized a campaign of probe collection in Rome (Italy): We targeted a university campus as well as city-wide, national and international events. Our campaign lasted three months, and we managed to collect, using commodity hardware only, ~11 million probes sent by ~160 thousand different devices. The release contains anonymized traces in .pcap format.date/time of measurement start: 2013-02-05date/time of measurement end: 2013-05-03collection environment: In our data collection campaign we targeted the following scenarios: 1) events of national audience (political meetings); 2) events of international audience (Pope's Angelus at St. Peter's Square, Vatican); 3) a Mall; 4) a train station; 5) a university campus; 6) other. The first four were one-shot events lasted from 40 minutes to 6 hours each, and whose data collection was physically carried out by a team of 5 researchers equipped with their laptops. The fifth dataset was obtained through a fixed hardware positioned in a university campus over a continuous period of 6 weeks. The last dataset includes probes collected by the research group in other occasions (e.g. commuting from home to the office).data collection methodology: The data was collected by 5 researchers (equipped with their laptops) using tcpdump and applying the following filter to select only the probe-requests: "tcpdump -i $interface type mgt subtype probe-req -w $outfile". Each file in the tracesets has the following name scheme: "probes-yyyy-mm-dd.pcapX", and traces collected in the same day are numbered progressively, starting from 0 (e.g., probes-2013-02-24.pcap0, probes-2013-02-24.pcap1,...)sanitization: We use scapy to dissect and anonymize all MAC addresses and SSIDs in the dataset. In particular, we first build two sets of all SSIDs, respectively all MACs in the dataset. Both, SSIDs and MACs, are then hashed and sorted. Finally, we replace each SSID and MAC in the traces as follows:- we substitute each SSID with the string SSID_i, where i is the corresponding index in the ordered list of hashes (e.g., SSID_000001, SSID_000002, ..., SSID_140152, for a total of 140152 unique SSIDs).- for each MAC address (xx:xx:xx:yy:yy:yy), we keep the first 3 bytes that identify the OUI vendor code (xx:xx:xx), and we substitute the last 3 bytes with the corresponding index in the ordered list of hashes (e.g., xx:xx:xx:00:00:01, xx:xx:xx:00:00:02, ..., xx:xx:xx:16:43:84 , for a total of 164384 unique MAC addresses).Tracesetsapienza/probe-requests/Politics1Wireless probe requests collected at a political meeting in Rome, Italy.file: Politics1.tar.gz description: This traceset contains 12 pcap traces collected during a political meeting organized on February 22nd in San Giovanni Square, by the M5S (Movimento Cinque Stelle) party.measurement purpose: Social Network Analysis, Human Behavior Modelingsapienza/probe-requests/Politics1 TracePolitics1format: This trace contains 12 pcap files compressed in an unique tar.gz archive.sapienza/probe-requests/Politics2Probe requests collected at a political meeting in Rome, Italy.file: Politics2.tar.gzdescription: This traceset contains 7 pcap traces collected at a meeting organized on March 23rd, by the conservative party PDL (Popolo della Liberta). The meeting was held "Piazza del Popolo", a famous square in the city center.measurement purpose: Social Network Analysis, Human Behavior Modelingsapienza/probe-requests/Politics2 Trace Politics2format: This trace contains 7 pcap files compressed in a unique tar.gz archive. sapienza/probe-requests/Vatican1Probe requests collected in St. Peter's Square during Pope Benedict's Angelus and general audience.file: Vatican1.tar.gzdescription: On February 11th, 2013, Pope Benedict announced his resignation, after 8 years of service as the head of the Catholic Church. This traceset contains 11 pcap traces, collected at his farewell Angelus, on February 24th, and during his last general audience, on February 27th.measurement purpose: Social Network Analysis, Human Behavior Modelingsapienza/probe-requests/Vatican1 TraceVatican1format: This trace contains 11 pcap files compressed in an unique tar.gz archive.sapienza/probe-requests/Vatican2Probe requests collected in St. Peter's Square during Pope Francis' Angelus and general audience.file: Vatican2.tar.gzdescription: This traceset contains 7 pcap traces, collected on March 17th, when the newly elected Pope (Pope Francis), delivered his first Angelus and on March 19th, during his first general audience.measurement purpose: Social Network Analysis, Human Behavior Modelingsapienza/probe-requests/Vatican2 TraceVatican2format: This trace contains 7 pcap files compressed in an unique tar.gz archive.sapienza/probe-requests/UniversityProbe requests collected in a university campus for 15 days.file: University.tar.gzdescription: This traceset contains 418 pcap traces (of ~1M each), collected by deploying an antenna in a fixed vantage point located at one of the main entries of the Sapienza University campus. Differently from the other datasets, in this case we collected probes continuouslyfor 6 weeks days (28th of March, 2013 - 6ht of May, 2013), monitoring students as they entered and left the campus.measurement purpose: Social Network Analysis, Human Behavior Modelingsapienza/probe-requests/University TraceUniversityformat: This trace contains 418 pcap files compressed in an unique tar.gz archive.sapienza/probe-requests/TheMallProbe requests collected at one of the biggest malls in Rome.file: TheMall.tar.gzdescription: This traceset contains 7 pcap traces collected at a mall in Rome. For this case study we aimed at collecting data from local Rome residents. So, we targeted a location known for hosting many families and groups of ffriends all at once: One of the biggest malls in Rome ("Porta di Roma"). To be sure to get a relevant affluence of people, we chose to collect the data on a special afternoon, that right before Easter (March 30th). The collection lasted 3 hours and a half.measurement purpose: Social Network Analysis, Human Behavior Modelingsapienza/probe-requests/TheMall TraceTheMallformat: This trace contains 7 pcap files compressed in an unique tar.gz archive.sapienza/probe-requests/TrainStationProbe requests collected at Roma Termini central station.file: TrainStation.tar.gzdescription: This traceset contains 17 pcap traces collected at Roma Termini central station. In this case study we targeted Rome central station and collected the probes for a total of 7 hours split in a time range of four days. The collection was carried out by positioning ourselves at advantage points in the station so as to maximize the area covered.measurement purpose: Social Network Analysis, Human Behavior Modelingsapienza/probe-requests/TrainStation TraceTrainStationformat: This trace contains 17 pcap files compressed in an unique tar.gz archive.sapienza/probe-requests/OthersProbe requests collected by 3 students around Rome.file: Others.tar.gzdescription: This traceset contains 91 pcap traces, collected by 3 students during several tours across the city performed in a time span of 4 weeks while e.g., commuting from home to the university or shopping.measurement purpose: Social Network Analysis, Human Behavior Modelingsapienza/probe-requests/Others TraceOthersformat: This trace contains 91 pcap files compressed in an unique tar.gz archive.本数据集收录了2013年2月至5月间在罗马采集的无线探测请求（Wireless Probe Request）。移动设备会尽可能自动切换至WiFi网络，为实现该自动流程，设备会存储用户常连接的网络名称（即服务集标识符SSID, Service Set Identifier），并定期以广播形式发送这些SSID的探测请求，以搜寻可用网络。由此自然衍生出如下研究问题：“你的智能手机发送的探测请求暴露了哪些个人信息？”“仅通过一群人的智能手机探测请求，能否推断出他们之间有意义的社交关联？”为解答上述问题，我们在意大利罗马开展了探测请求采集活动：采集范围覆盖大学校园以及全市性、全国性和国际性活动。本次采集活动持续三个月，仅使用商用硬件便收集到约1100万条探测请求，来源为约16万个不同设备。本数据集发布的是经过匿名化处理的.pcap格式追踪数据。 采集起始时间：2013-02-05 采集结束时间：2013-05-03 采集场景： 本次数据采集活动覆盖如下场景：1）面向全国受众的活动（政治集会）；2）面向国际受众的活动（梵蒂冈圣彼得广场教皇周日三钟经活动与公开接见活动）；3）大型购物中心；4）火车站；5）大学校园；6）其他场景。前四类均为单次活动，持续时长40分钟至6小时不等，由5名研究人员携带笔记本电脑现场完成数据采集。第五类数据集通过部署在大学校园内的固定硬件采集，持续6周。最后一类数据集则由研究团队在其他场景下采集（例如通勤往返住宅与办公场所）。 数据采集方法： 数据由5名研究人员携带笔记本电脑，使用tcpdump工具并应用如下过滤规则仅采集探测请求：tcpdump -i $interface type mgt subtype probe-req -w $outfile。追踪集内的每个文件遵循如下命名规则：probes-yyyy-mm-dd.pcapX，同日采集的追踪文件会从0开始依次编号（例如probes-2013-02-24.pcap0、probes-2013-02-24.pcap1等）。 数据匿名化处理： 我们使用scapy工具解析并匿名化数据集中的所有媒体访问控制地址（MAC地址, Media Access Control Address）与SSID。具体流程为：首先分别构建数据集中所有SSID与所有MAC地址的集合，随后对二者分别进行哈希运算并排序。最后将追踪数据中的SSID与MAC地址替换如下： - 将每个SSID替换为字符串SSID_i，其中i为哈希排序后对应的索引（例如共有140152个唯一SSID，对应SSID_000001、SSID_000002……SSID_140152）。 - 对于每个MAC地址（格式为xx:xx:xx:yy:yy:yy），保留前3字节的组织唯一标识符（OUI, Organizationally Unique Identifier）厂商代码（xx:xx:xx），将后3字节替换为哈希排序后对应的索引（例如共有164384个唯一MAC地址，对应xx:xx:xx:00:00:01、xx:xx:xx:00:00:02……xx:xx:xx:16:43:84）。 各追踪数据集详情： 1. sapienza/probe-requests/Politics1 本追踪集采集自意大利罗马一场政治集会的无线探测请求。 文件名：Politics1.tar.gz 数据集说明：本追踪集包含2013年2月22日在圣乔瓦尼广场由五星运动党（Movimento Cinque Stelle, M5S）举办的一场政治集会期间采集的12个.pcap格式追踪文件。 采集用途：社交网络分析、人类行为建模 详情说明：本数据集为单个.tar.gz压缩归档文件，内含12个.pcap文件。 2. sapienza/probe-requests/Politics2 本追踪集采集自意大利罗马一场政治集会的无线探测请求。 文件名：Politics2.tar.gz 数据集说明：本追踪集包含2013年3月23日由保守派政党自由人民党（Popolo della Liberta, PDL）在罗马市中心著名广场人民广场举办的一场集会期间采集的7个.pcap格式追踪文件。 采集用途：社交网络分析、人类行为建模 详情说明：本数据集为单个.tar.gz压缩归档文件，内含7个.pcap文件。 3. sapienza/probe-requests/Vatican1 本追踪集采集自梵蒂冈圣彼得广场，对应教皇本笃十六世的周日三钟经活动与公开接见活动的无线探测请求。 文件名：Vatican1.tar.gz 数据集说明：2013年2月11日，教皇本笃十六世宣布卸任，结束其担任天主教会领袖8年的任期。本追踪集包含2月24日其离任前的周日三钟经活动，以及2月27日其最后一场公开接见活动期间采集的11个.pcap格式追踪文件。 采集用途：社交网络分析、人类行为建模 详情说明：本数据集为单个.tar.gz压缩归档文件，内含11个.pcap文件。 4. sapienza/probe-requests/Vatican2 本追踪集采集自梵蒂冈圣彼得广场，对应教皇方济各的周日三钟经活动与公开接见活动的无线探测请求。 文件名：Vatican2.tar.gz 数据集说明：本追踪集包含2013年3月17日新当选教皇方济各发表首次周日三钟经，以及3月19日其首场公开接见活动期间采集的7个.pcap格式追踪文件。 采集用途：社交网络分析、人类行为建模 详情说明：本数据集为单个.tar.gz压缩归档文件，内含7个.pcap文件。 5. sapienza/probe-requests/University 本追踪集为在大学校园内采集的无线探测请求，原标注采集时长为15天，实际采集周期为2013年3月28日至2013年5月6日，共6周。 文件名：University.tar.gz 数据集说明：本追踪集包含418个.pcap格式追踪文件（每个文件大小约1MB），采集方式为在萨皮恩扎大学（Sapienza University）主入口之一的固定高点部署天线，持续监测进出校园的学生设备。与其他数据集不同，本次采集为连续6周的不间断采集。 采集用途：社交网络分析、人类行为建模 详情说明：本数据集为单个.tar.gz压缩归档文件，内含418个.pcap文件。 6. sapienza/probe-requests/TheMall 本追踪集采集自罗马最大型购物中心之一的无线探测请求。 文件名：TheMall.tar.gz 数据集说明：本追踪集包含2013年3月30日复活节前一个下午在罗马“罗马之门”（Porta di Roma）购物中心采集的7个.pcap格式追踪文件。本次采集旨在获取罗马本地居民的探测数据，因此选择了这个以聚集大量家庭与好友群体闻名的场所，采集时长共计3.5小时。 采集用途：社交网络分析、人类行为建模 详情说明：本数据集为单个.tar.gz压缩归档文件，内含7个.pcap文件。 7. sapienza/probe-requests/TrainStation 本追踪集采集自罗马特米尼（Roma Termini）中央车站的无线探测请求。 文件名：TrainStation.tar.gz 数据集说明：本追踪集包含在罗马特米尼中央车站采集的17个.pcap格式追踪文件。本次采集覆盖四天内的共计7小时采集时长，研究人员在车站内的多个有利位置部署设备以最大化覆盖范围。 采集用途：社交网络分析、人类行为建模 详情说明：本数据集为单个.tar.gz压缩归档文件，内含17个.pcap文件。 8. sapienza/probe-requests/Others 本追踪集由3名学生在罗马各地采集的无线探测请求。 文件名：Others.tar.gz 数据集说明：本追踪集包含91个.pcap格式追踪文件，由3名学生在为期4周的多次出行中采集，例如往返住宅与校园的通勤、购物出行等场景。 采集用途：社交网络分析、人类行为建模 详情说明：本数据集为单个.tar.gz压缩归档文件，内含91个.pcap文件。