CRAWDAD dartmouth/campus

This dataset includes syslog, SNMP, and tcpdump data for 5 years or more, for over 450 access points and several thousand users at Dartmouth College.last modified :2008-09-12release date :2007-02-08date/time of measurement start :2001-04-11date/time of measurement end :2006-10-04collection environment :The Dartmouth College campus has over 190 buildings on 200 acres. Dartmouth College has about 5500 students and 1200 faculty, and during our study there were approximately 3200-3300 undergraduates on campus. Students are required to own a computer, and most purchase a computer through the campus computer store. Wireless laptops increasingly dominate those purchases, making up 45% of the total in 2000, 70% in 2001, 88% in 2002, and 97% in 2003. Assuming that students obtaining computers elsewhere choose laptops in the same proportion, we estimate that over 75% of the undergraduates owned laptops at the time of our study.network configuration :476 Cisco 802.11b access points (APs) were installed in 2001 to cover most of the campus. Since then, APs have been added to increase coverage and to cover new construction, and there are currently 566 APs. The compact nature of the campus means that the signal range of interior APs extends to cover most of the campus' outdoor areas. All APs share the same SSID, allowing wireless clients to roam seamlessly between APs. On the other hand, a building's APs are connected to the building's existing subnet. The 188 buildings with wireless coverage span 115 subnets, so clients roaming between buildings may be forced to obtain new IP addresses. Clients obtain IP addresses using DHCP (lease times were 6 or 12 hours at different points in the trace).data collection methodology :We used three techniques: syslog events, SNMP polls, and network sniffers (tcpdump). We also derived movement history data from the syslog data.sanitization :All data has been sanitized to protect the privacy of our users. We have resanitized the data (2004-11-08) so that there is a consistent MAC address mapping and a consistent IP address mapping across all of the traces. In other words, the sanitized IP address 111.222.333.444 will correspond to the same raw IP address in all of our traces. Similarly, the sanitized MAC address aa:bb:cc:dd:ee:ff will always correspond to the same raw MAC address in all of our traces. Note that these may not represent the same physical device, due to DHCP, MAC spoofing, etc.Tracesetdartmouth/campus/syslogTimestamped, sanitized syslog records from Dartmouth wireless network.files: syslog-v3.3.tar.gz, syslog-2005-2006-cisco.tar.gz, syslog-2005-2006-aruba.tar.gz, syslog-2005-2006-merged.tar.gzdescription: The traceset of nearly continuous recording of the syslog records produced by the access points, from 2001-04-11 to 2004-06-30, and from 2005-09-01 to 2006-10-04. UNIX timestamps have been added to each log record, and MAC addresses and AP names sanitized.measurement purpose: Usage Characterization, User Mobility Characterizationmethodology: We configured the access points to transmit a syslog message every time a client card associated or disassociated with the access point (We configured the Cisco APs and the Aruba APs differently. Please see the traces dartmouth/campus/syslog/01_04 and dartmouth/campus/syslog/05_06 for more details). Dartmouth currently has no authentication to associate with the network, so we do not know the identity of users, and the IP address given to a user varies from time to time and building to building.last modified: 2007-02-08dataname: dartmouth/campus/syslogversion: 20070208change: A new syslog trace (collected from 2005-09-02 to 2006-10-04) has been added.release date: 2007-02-08date/time of measurement start: 2001-04-11date/time of measurement end: 2006-10-04hole: We have not released syslog trace collected from 2004-07-01 to 2005-08-31.dartmouth/campus/syslog Traces01_04: Timestamped, sanitized syslog records from Dartmouth wireless network.format: timestamp, AP name, the MAC address of the card, and type of messagedescription: The trace of nearly continuous recording of the syslog records produced by the access points, from 2001-04-11 to 2004-06-30. UNIX timestamps have been added to each log record, and MAC addresses and AP names sanitized.last modified: 2007-01-31dataname: dartmouth/campus/syslog/01_04version: 20041218change: Syslog trace is newly created.release date: 2004-12-18date/time of measurement start: 2001-04-11date/time of measurement end: 2004-06-30url: /download/dartmouth/campus/syslog/syslog-v3.3.tar.gzhole: We only have a list of these holes in fall 2001. We had ``spatial holes'' because many APs did not send syslogs. [Configuration mistake.] And temporal holes, because our syslog recording server(s) failed. You may refer to note for details. It also appears that the engineering school's APs, building name ''cummings'' did not send any messages after they installed a firewall in early 2002 until I noticed the problem and asked them to open a hole in the firewall in late 2002. We do not release the syslog trace collected from 2004-07-01 to 2005-08-31.limitation: Since syslog messages are sent from the APs to a relaying server (ns1), and from ns1 to our syslog recording servers, as UDP messages, it is possible for them to be lost or reordered along the way. The timestamps are applied by the syslog daemon on our host, so the timestamps are monotonically increasing. But, the events may have been recorded out of order, and some may be missing. We believe this effect is small enough to be negligible. We have two syslog recording servers, and we do not see the same event with different timestamps in the two servers. From 10/19/2003 this no longer applies.sanitization: Every MAC address has been sanitized, and the IP address or host name of client machines has been removed. To sanitize the MAC address, we randomized the bottom six hex digits. We collected every MAC address from all of our syslog, SNMP, an tcpdump traces, and built a huge table mapping real MACs to randomized MACs, ensuring that all mappings are unique. Each access point name has been blinded in the form: AcadBldg10AP3 where this indicates the third AP in the tenth building of type 'Academic.' The building types are Adm (Admin), Ath (Athletic), Lib (Library), Oth (Other - mainly sysadmin test APs), Res (Residential) and Soc (Social). Refer to note for details.05_06: Timestamped, sanitized syslog records from Dartmouth wireless network for the period 2005 - 2006.configuration: [Cisco APs] We configured the Cisco access points to transmit a syslog message every time a client card authenticated, associated, reassociated, disassociated, or deauthenticated with the access point. Each message contains the AP name, the MAC address of the card, and the type of message. [Aruba APs] On our campus, we deployed Aruba wireless networks with an Aruba 5000 switch as a master switch, which controls the wireless network in centralized manner. The configuration has a three-level hierarchy (master-switches-APs) such that a number of switches are attached to the master switch, and likewise a number of APs are attached to each switch. We have three models of Aruba APs: 52, 61, and 72. The wireless network is virtually divided into several zones (subnets), each of which has a controller that controls a set of APs. Aruba syslog messages come from either the master switch (the central controller) or each zone controller. Since different zone covers different set of APs, separate syslog messages come from each zone controller. The Aruba system is able to configure multiple ESSIDs on each AP. At the time of collecting these syslog data, we used four ESSIDs - "Kiewit Wireless", "Kiewit Video", "Kiewit Voice", and "Hanover Inn". Among those ESSIDs, Kiewit Video/Voice are NATed and use private (RFC1918) addresses. We do not have L2 assoc/disassoc/auth/deauth messages in the Aruba syslog trace because the Aruba switch does not give us those. What we do have are "station up" and "station down" messages which indicate when the switch sees a client connect with a BSSID. Though we do not know what these station up|down messages equate to in the 802.11 FSM, for most analyses it should be possible to more or less correlate these messages with the assoc|disassoc messages in the Cisco syslog.format: 1. Directory and files This trace consists of three tarballs (directories): 'syslog-2005-2006-cisco', 'syslog-2005-2006-aruba', and 'syslog-2005-2006-merged'. 'syslog-2005-2006-cisco' and 'syslog-2005-2006-aruba' directories contain syslog records from cisco APs and Aruba APs, respectively. 'syslog-2005-2006-merged' directory contains syslog records merged from 'cisco' and 'aruba' syslog records, sorted by timestamps. Each directory contains syslog trace files for each day during the measurement period. File name follows the format YYMMDD.HHMMSS.{syslog or aruba or merged}. The time used for file name is in UTC. For Aruba syslog trace, we represent each access point name as location id (with the format [building_id.floor_id.ap_id] like 15.1.1). Under the trace root directory, we provide a file called 'aruba_locid_table.csv' containing a mapping between location id and sanitized AP name (e.g., a pair of 15.1.1 and AcadBldg10AP3). 2. Format of Cisco syslog records is as follows: unix_timestamp timestamp1 AP_name1 timestamp2 counter AP_name2 timestamp2 syslog_message - unix_timestamp : UNIX timestamp in UTC - timestamp1 : the time that the syslog message was received - AP_name1 : the (sanitized) hostname of the host that sent the syslog - counter : internal counter - AP_name2 : the (sanitized) hostname that the AP thinks it has. Sometimes AP_name1 and AP_name2 don't match. - timestamp2 : the AP's clock. Sometimes timestamp1 and timestamp2 don't match. - syslog_message : syslog message content The following is a sample line in Cisco IOS syslog trace. 1157014202 Aug 31 04:50:02 AcadBldg33AP1 14375: AcadBldg33AP1 Aug 31 08:50:01: %DOT11-6-ASSOC: Interface Dot11Radio0, Station 0016cbf7ca65 Associated KEY_MGMT[NONE] 3. Format of Aruba syslog records is as follows: unix_timestamp timestamp ip_address1 year [ip_address2] syslog_message - unix_timestamp : UNIX timestamp in UTC - timestamp, year : the time that the syslog message was received - ip_address1 : the (sanitized) IP address of the controller - ip_address2 : the (sanitized) IP address of another controller that sent this message. Sometimes the messages don't come directly from the controller, they come from a controller further down the tree. - syslog_message : syslog message content The following is a sample line in Aruba syslog trace. 1159954944 Oct 4 05:42:24 50.32.208.194 2006 [50.32.208.195] authmgr[510]: station down bssid 00:0b:86:d3:ab:80, essid Kiewit Voice, vlan 2242, ingress 0x1168 (tunnel 264), u_encr 1, m_encr 1, loc 15.1.1 slotport 0xfc7 This message means that a user connected to an AP is removed from the user table. Previously it was connected to VLAN 2242 at location 15.1.1. The reason for the disconnection might be one of the following: - User moved out of the network - The user is logged out of the network.description: The trace of nearly continuous recording of the syslog records produced by the access points, from 2005-09-01 to 2006-10-04. UNIX timestamps have been added to each log record, and MAC addresses and AP names sanitized.last modified: 2007-02-08dataname: dartmouth/campus/syslog/05_06version: 20070208change: This 2005-2006 syslog trace is newly created.release date: 2007-02-08date/time of measurement start: 2005-09-01date/time of measurement end: 2006-10-04sanitization: Every MAC address has been sanitized by randomizing the bottom six hex digits, and we mapped all IP addresses using a prefix-preserving sanitizer. [Cisco syslog] Each access point name has been blinded in the form: AcadBldg10AP3 where this indicates the third AP in the tenth building of type 'Academic.' The building types are Adm (Admin), Ath (Athletic), Lib (Library), Oth (Other - mainly sysadmin test APs), Res (Residential) and Soc (Social). [Aruba syslog] Each access point name is represented as location id ([building_id.floor_id.ap_id] like 15.1.1). We also provide a file called 'aruba_locid_table.csv' containing a mapping between location id and sanitized AP name (e.g., a pair of 15.1.1 and AcadBldg10AP3).dartmouth/campus/snmpRecords of SNMP polling at Dartmouth College.files: fall01.tar.gz, spring02.tar.gz, fall03.tar.gzdescription: The traceset of polling every AP every five minutes, during Fall term 2001, Spring term 2002, Fall term 2003 and Winter term 2004. The Fall 2001 data was used for [MobiCom 2002 paper]. The 2003/4 data was used for [MobiCom 2004 paper]. We recommend only using the 2003/4 data. See this important note about problems with the 2001/2002 SNMP dataset. Any questions about the 2001/2 data will go into our LBE (Less than Best Effort) queue, i.e., they may not be answered... please use the 2003/4 data instead.measurement purpose: Usage Characterizationmethodology: We used the Simple Network Management Protocol (SNMP) to poll each AP every five minutes, querying AP and client-specific counters. AP-specific variables included inbound/outbound bytes, packets and errors, and the clients associated with a given AP. Client-specific variables included MAC and IP addresses, signal strength and quality.sanitization: To sanitize the data, we randomly (but consistently) mapped the MAC address field into a randomly chosen MAC of the same vendor, and we mapped all IP addresses using a prefix-preserving sanitizer.last modified: 2008-09-12dataname: dartmouth/campus/snmpversion: 20041109change: SNMP Traceset is resanitized.release date: 2004-11-09date/time of measurement start: 2001-09-14date/time of measurement end: 2004-02-28limitation: We have no way to distinguish periods with no clients from periods when the AP was off or unreachable. We also have no way to detect an AP reboot or reset, which reset all of the per-client counters reported here. Thus it is critical to take care when interpreting the per-client byte counts... a counter can be reset or roll over between two polls, and the delta can thus appear to be negative (or, in unsigned math, a very large number).hole: There were unfortunate gaps in the data collection, generally caused by power failures.error: In 2001/2 data, the perl scripts that performed the SNMP queries suffered from some problems, in that they queried inappropriate SNMP values, or misunderstood the meaning of other values. This data was also used in a subsequent analysis. The same scripts were used to collect data for a subsequent study of another wireless network. See http://www.cs.dartmouth.edu/reports/abstracts/TR2003-480/dartmouth/campus/snmp Tracesfall01: Records of SNMP polling during Fall term 2001.configuration: The job of polling all access points was divided among two machines: agentnews and klebb. Each of those hosts has a directory here. In each directory there is a number of subdirectories, one for each day, and in each of those subdirectories there are a number of SNMP log files.format: There are two types of file. The first type of file has one file for each access point. Each file gives information about each interface for each poll. Most of the interfaces are irrelevant; you only care about the wireless interface (the one with ifSpeed=11000000). Here is an example: V1.0ap timestamp,ap,ifIndex,ifInOctets,ifOutOctets,ifSpeed,ifInErrors,ifOutErrors Notice the periodic restatment of the file format version and the description of the MIB variable names used in the lines that follow. The second type of file occurs once for each AP and each date. The file name conveys the blinded name of the AP, e.g. AcadBldg10AP3 where this indicates the third AP in the tenth building of type 'Academic.' The building types are Adm (Admin), Ath (Athletic), Lib (Library), Oth (Other - mainly sysadmin test APs), Res (Residential) and Soc (Social). Each data file contains all of the data for that access point on that date. Below is a sample of the top of one file. The top two lines are documentation; the first indicates the file format version (always V1.0). The second identifies the column headers for each of the data lines. After the timestamp (standard Unix time in seconds), the remaining fields are MIB variables from the AWC MIB (Aironet Wireless Communications is the name of the company that developed our access points; Aironet was bought by Cisco who then branded and sold the APs under their name). V1.0 timestamp,awcTpFdbAddress,awcTpFdbClassID,awcTpFdbSrcOctetsImmed,awcTpFdbDestOctetsImmed,awcTpFdbIPv4Addr,awcTpFdbDdpProdDevID,awcTpFdbDdpRadioDevID,awcDot11TpFdbAID,awcDot11TpFdbTxShortRetries,awcDot11TpFdbLatestRxSignalQuality,awcDot11TpFdbCapabilities 1001908847,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,-15,unknown,state2,73,73 1001909056,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,generic80211Client,unknown,state2,73,73 1001909266,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,-15,unknown,state2,73,73 1001909476,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,broadcast,-16,state2,73,73 1001909683,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,broadcast,-16,state2,73,73 1001909892,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,generic80211Client,unknown,state2,73,73 1001910102,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,generic80211Client,unknown,state2,73,73 1001910311,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,ethernetAP,34,state2,73,73description: The trace of SNMP polling every AP during Fall term 2001.last modified: 2008-09-12dataname: dartmouth/campus/snmp/fall01version: 20041109change: SNMP trace is resanitized.release date: 2004-11-09date/time of measurement start: 2001-09-14date/time of measurement end: 2002-01-10url: /download/dartmouth/campus/snmp/fall01.tar.gzerror: During the fall there were some access points with old, buggy firmware that sometimes filled our SNMP data files with garbage entries. Any code you write to parse the data must be very robust. Unfortunately the code that I used was hacked on by two or three different students and is not currently in presentable form. See http://www.cs.dartmouth.edu/reports/abstracts/TR2003-480/.spring02: Records of SNMP polling during Spring term 2002.configuration: The job of polling all access points was divided among three machines: agentnews, klebb, and molari. I used molari only briefly until I was able to accomplish the same thing on one of the other two (access points cummings-* were missed for a week, because they are behind a firewall, and eventually I set up molari inside the firewall, and eventually I was able to add a hole in the firewall so that agentnews and klebb could poll through the firewall). Each polling machine has a directory. Each such directory has a subdirectory for each date. In each date, there is a file for each access point polled by that machine on that date. In the file are several types of lines. Below is a sample from the top of one file. Notice that it uses a comma-separated format suitable for import into spreadsheets, or easy parsing with AWK or perlformat: The first five lines are comments. The first gives basic information: #V2.1: file format version 2.1, timestamp of file creation, AP name, and date code YYMMDD All timestamps are standard Unix timestamps (seconds since 1970). The other four comment lines describe the format of lines that occur later in the file. Other than the timestamp and AP name, the rest of these fields are MIB variable names. After the five comment lines comes a series of polls. Each poll consists of one ''sys'' line, one ''if'' line describing stats of the the wireless interface, and zero or more pairs of ''c1'' and ''c2'' lines, each pair describing a currently connected client. The c1 and c2 lines are a collection of MIB variables from the AWC MIB (Aironet Wireless Communications is the name of the company that developed our access points; Aironet was bought by Cisco who then branded and sold the APs under their name). #V2.1,1018929767,AdmBldg27AP2,020416 #sys,timestamp,AP,sysUpTime #if,timestamp,AP,ifIndex,ifType,ifSpeed,ifInOctets,ifInUcastPkts,ifInErrors,ifInDiscards,ifOutOctets,ifOutUcastPkts,ifOutErrors,ifOutDiscards #c1,timestamp,AP,awcDot11TpFdbAddress,awcDot11TpFdbClientState,awcDot11TpFdbLatestRxSignalStrength,awcDot11TpFdbLatestRxSignalQuality #c2,timestamp,AP,awcTpFdbAddress,awcTpFdbClassID,awcTpFdbSrcOctetsImmed,awcTpFdbDestOctetsImmed,awcTpFdbIPv4Addrdescription: The trace of SNMP polling every AP during Spring term 2002.last modified: 2008-09-12dataname: dartmouth/campus/snmp/spring02version: 20041109change: SNMP trace is resanitized.release date: 2004-11-09date/time of measurement start: 2002-03-25date/time of measurement end: 2002-06-09url: /download/dartmouth/campus/snmp/spring02.tar.gzerror: PLEASE NOTE THAT THERE ARE ERRORS in dartmouth/campus/snmp/fall01 and dartmouth/campus/snmp/spring02 data sets. See http://www.cs.dartmouth.edu/reports/abstracts/TR2003-480/ .fall0304: Records of SNMP polling during Fall term 2003 and Winter term 2004.configuration: We only used one machine for SNMP polls in this period. The SNMP poller was rewritten to be more robust and more efficient, and so we only needed one machine to poll all ~ 560 APs on campus. We queried more counters in this trace. The variables are listed at the beginning of each file. For more details see the MIBS: ftp://ftp-sj.cisco.com/pub/mibs/v2/AWCVX-MIB.my ftp://ftp-sj.cisco.com/pub/mibs/v2/IEEE802dot11-MIB.my At the time of this data collection, Dartmouth mainly used Cisco 340 and 350 APs. These used to run the VxWorks operating system. During December 2003 to May 2004, our 350 APs migrated from running VxWorks to the Cisco IOS (the APs didn't originally run IOS as they were made by Aironet, a company that was later bought by Cisco). IOS uses completely different SNMP MIBs to VxWorks, and so the variable names and their order are slightly different. When the upgrades started taking place, we incremented the log version number to "V3.1" (the first line of each log) to indicate the new variables being queried. We also folded both the "c1" and "c2" client-specific lines into one "cl" line (this made the parser code easier to maintain).format: We folded both the ''c1'' and ''c2'' client-specific lines into one ''cl'' line (this made the parser code easier to maintain). To identify whether a given SNMP log is IOS or VxWorks (see the configuration), look at the ''sys'' line in a V3.1 log. The fifth field of this line is a formatted ''sysDescr'' indicating the OS version of the AP. An example VxWorks file #V3.1,1073710881,ResBldg48AP1,040110 #sys,timestamp,AP,sysUpTime,sysDescr #if,timestamp,AP,ifIndex,ifDescr,ifType,ifSpeed,ifInOctets,ifInUcastPkts,ifInErrors,ifInDiscards,ifOutOctets,ifOutUcastPkts,ifOutErrors,ifOutDiscards,awcDot11AssociatedStationCount,awcDot11AuthenticatedStationCount,awcDot11ReassociatedStationCount,awcDot11RoamedStationCount,awcDot11DeauthenticateCount,awcDot11DisassociateCount,awcFtClientSTASelf,awcFtBridgeSelf,awcFtRepeaterSelf #cl,timestamp,AP,awcDot11TpFdbAddress,awcDot11TpFdbAID,awcDot11TpFdbClientState,awcDot11TpFdbLatestRxSignalStrength,awcDot11TpFdbLatestRxSignalQuality,awcTpFdbClassID,awcTpFdbSrcOctetsImmed,awcTpFdbDestOctetsImmed,awcTpFdbIPv4Addr,awcTpFdbSrcPktsImmed,awcTpFdbDestPktsImmed,awcTpFdbSrcErrorPktsImmed,awcTpFdbDestErrorPktsImmed An example IOS file: #V3.1,1075784465,ResBldg47AP1,040203 #sys,timestamp,AP,sysUpTime,sysDescr #if,timestamp,AP,ifIndex,ifDescr,ifType,ifSpeed,ifInOctets,ifInUcastPkts,ifInErrors,ifInDiscards,ifOutOctets,ifOutUcastPkts,ifOutErrors,ifOutDiscards,cDot11AssStatsAssociated,cDot11AssStatsAuthenticated,cDot11AssStatsRoamedIn,cDot11AssStatsRoamedAway,cDot11AssStatsDeauthenticated,cDot11AssStatsDisassociated,cDot11ActiveWirelessClients,cDot11ActiveBridges,cDot11ActiveRepeaters #cl,timestamp,AP,cDot11ClientAddress,cDot11ClientRoleClassType,cDot11ClientPowerSaveMode,cDot11ClientAid,cDot11ClientAssociationState,cDot11ClientIpAddress,cDot11ClientUpTime,cDot11ClientSignalStrength,cDot11ClientSigQuality,cDot11ClientBytesSent,cDot11ClientBytesReceived,cDot11ClientPacketsSent,cDot11ClientPacketsReceived,cDot11ClientDuplicates,cDot11ClientMsduRetries,cDot11ClientMsduFailsdescription: The trace of SNMP polling every AP during Fall term 2003 and Winter term 2004.last modified: 2006-11-14dataname: dartmouth/campus/snmp/fall0304version: 20041109change: SNMP trace is resanitized.release date: 2004-11-09date/time of measurement start: 2003-11-01date/time of measurement end: 2004-02-28url: /download/dartmouth/campus/snmp/fall03.tar.gzdartmouth/campus/tcpdumpPacket headers from every wireless packet sniffed in 27 buildings on Dartmouth College campus.files: fall01, spring02, fall03, README.voipdescription: The packet headers from every wireless packet sniffed in four (Fall01), five (Spring02), or 18 (Fall03) buildings on campus. The Fall 2001 data was used for [MobiCom 2002 paper]. The Fall 2001 and Spring 2002 data was used for [WiNet 2005 paper]. The 2003/4 data was used for [MobiCom 2004 paper]. This fall03 data also contains a list of device types, as determined using the OS fingerprinting tool p0f. Note that the MAC addresses in this list are only devices that we saw associate with an AP (i.e., that appeared in the syslog or SNMP data). Thus it does not include non-wireless client MAC addresses, such as routers or spoofed MACs that do not appear in syslog. The total compressed datasets are over 200 GB, so they are too large to post as tarballs. The best option is to use an http tool like curl or wget to download the whole Fall01, Spring02, or Fall03 directory from the web site. Or you can arrange to send us a USB or firewire drive (>250GB) and we can ship it back to you with all of our data on it. You can get the README, some of my analysis software, and the output of my own analysis programs listing the amount of traffic seen at each sniffer for each port number for each day, that is available as a small (660 KB) tgz file. NB: this is for the 2001/2 data.measurement purpose: Usage Characterizationmethodology: We used network ''sniffers'' to obtain detailed network-level traces. Due to the volume of traffic on the wireless network, it was impractical to capture all the traffic. Moreover, the structure of our WLAN, with several subnets, meant that there was no convenient central point for capturing wireless traffic. Instead, we installed 18 sniffers in 14 different buildings; in some large buildings, we needed multiple sniffers to monitor all of the building's APs. The buildings were among the most popular wireless locations in 2001, and included libraries, dormitories, academic departments and social areas. In total, our 18 sniffers covered 121 APs. Each sniffer was a Linux box with two Ethernet interfaces. One interface was used for remote access, to maintain the sniffer and to obtain the data for analysis. The other interface was used for collecting (''sniffing'') data. In each of the 18 switchrooms we attached the APs to a switch, and set another port on the switch to ''mirror'' mode, so that all the traffic on that switch would be sent to this port. The sniffer's second interface was attached to this mirrored port. We used tcpdump to capture any wireless traffic that came through these APs and their wired interfaces..sanitization: To sanitize the MAC address, we randomized the bottom six hex digits. We collected every MAC address from all of our syslog, SNMP, an tcpdump traces, and built a huge table mapping real MACs to randomized MACs, ensuring that all mappings are unique. [We did not change either the MAC address 000000000000 or FFFFFFFFFFFF, they remain as they were.] We applied this mapping consistently across all data files of all types, so if you see a MAC address in the tcpdump files, and see it again in the SNMP trace, you can be sure it's the same client. We used a prefix-preserving IP address sanitizer, see Xu, J., Fan, J. Ammar, M., and Moon, S. ``On the Design and Performance of Prefix-Preserving IP Traffic Trace Anonymization'', Proc. of 10th IEEE International Conference on Network Protocols (ICNP 2002), Paris, France, November 2002. What this means is that you can compare the prefixes of the sanitized IP addresses, i.e. if two IP addresses share the same k-bit prefix, the sanitized addresses will also share the same k-bit prefix.last modified: 2006-11-14dataname: dartmouth/campus/tcpdumpversion: 20041109change: TCPDUMP traceset is resanitized.release date: 2004-11-09date/time of measurement start: 2001-09-25date/time of measurement end: 2004-02-28limitation: In both Fall01 and Spring02 datasets we lose a little data each day. We restarted tcpdump once a day, to cause it to begin a new log file. We killed the tcpdump process, then started a new one; as a result, some tcpdump data files end with partial packets, and no doubt we lost a few packets in the transition. We missed any traffic between two clients associated with the same AP, as this would not be sent via the AP's wired interface, but we believe this occurred rarely.hole: There were unfortunate gaps in the data collection, generally caused by power failures.error: The Fall01 data in particular suffers from a lot of corruption. It appears that older versions of tcpdump have a serious bug that causes them to record the MAC address of many frames incorrectly. In the Spring we used a newer tcpdump that did not have this problem.dartmouth/campus/tcpdump Tracesfall01: Packet headers from every wireless packet sniffed on Dartmouth College campus during Fall 2001 term.configuration: We collected data from four sniffers around the Dartmouth campus. Each sniffer was connected to a hub, along with the building's access points. The four buildings are representative: 1. Sudikoff (AcadBldg16): the Department of Computer Science (6 APs). 2. Brown (ResBldg100): a dormitory with many first-year students (2 APs). 3. Berry (LibBldg2): the main campus library. Due to the size of the building and the switched nature of its network, we were only able to sniff 5 of the 13 APs. 4. Collis/Thayer (SocBldg1): two buildings, the student center and dining hall, containing five cafes, several lounge areas, several meeting rooms, and some offices (total 9 APs).format: tcpdump formatdescription: The packet headers from every wireless packet sniffed in four (Fall01) buildings on campus. The Fall 2001 data was used for [MobiCom 2002 paper] and [WiNet 2005 paper].last modified: 2006-11-14dataname: dartmouth/campus/tcpdump/fall01version: 20041109change: TCPDUMP trace is resanitized.release date: 2004-11-09date/time of measurement start: 2001-09-25date/time of measurement end: 2001-12-10url: /download/dartmouth/campus/tcpdump/fall01hole: % Sudi 3 GAPS of about 21h:17m:46s % Brown 15 GAPS of about 213h:10m:33s % Berry 7 GAPS of about 139h:17m:29s % Collis 8 GAPS of about 337h:1m:59slimitation: We lose a little data each day. To keep file sizes small, and to be able to backup and manipulate the data more easily, we restarted tcpdump once a day, to cause it to begin a new log file. We killed the tcpdump process, then started a new one; as a result, some tcpdump data files end with partial packets, and no doubt we lost a few packets in the transition. [We note that newer versions of tcpdump have a roll-over mechanism built in; too bad it didn't exist for our tracing.] In the fall, we restarted tcpdump at midnight;error: Data in particular suffers from a lot of corruption. It appears that older versions of tcpdump have a serious bug that causes them to record the MAC address of many frames incorrectly. These bogus MAC addresses, specifically 0:0:0:0:0:0, 0:0:0:0:0:1, 1:0:0:0:0:0, or 1:0:1:0:1:0, occurred in about 78% of all frames in the Fall data. For frames containing IP packets, we examined the source and destination IP address; if the IP address was associated with a valid, wireless MAC address in a recent IP packet, then we assumed this packet used the same MAC, and treated it as a wireless packet. We fixed about a third of bad MACs this way.spring02: Packet headers from every wireless packet sniffed on Dartmouth College campus.configuration: We collected data in the same way, in the same four locations, but added another sniffer on the 8 APs in Whittemore, which is a dormitory at the Tuck School of Business. This data is much cleaner than the fall data.format: tcpdump formatdescription: The packet headers from every wireless packet sniffed in five buildings on campus. The Spring 2002 data was used for [WiNet 2005 paper].last modified: 2006-11-14dataname: dartmouth/campus/tcpdump/spring02version: 20041109change: TCPDUMP trace is resanitized.release date: 2004-11-09date/time of measurement start: 2002-03-25date/time of measurement end: 2002-06-09url: /download/dartmouth/campus/tcpdump/spring02hole: Berry (LibBldg2): 2 GAPS of about 0h:6m:34s Brown (ResBldg100): 0 GAPS of about 0h:0m:0s Collis (SocBldg1): 5 GAPS of about 27h:24m:22s Sudikoff (AcadBldg16): 0 GAPS of about 0h:0m:0s Whittemore (ResBldg83): 1 GAPS of about 12h:28m:15slimitation: We lose a little data each day. To keep file sizes small, and to be able to backup and manipulate the data more easily, we restarted tcpdump once a day, to cause it to begin a new log file. We killed the tcpdump process, then started a new one; as a result, some tcpdump data files end with partial packets, and no doubt we lost a few packets in the transition. [We note that newer versions of tcpdump have a roll-over mechanism built in; too bad it didn't exist for our tracing.] We restarted at about 4AM when traffic was lightest.fall03: Packet headers from every wireless packet sniffed on Dartmouth College campus during Fall 2003 term.configuration: We increased the number of sniffers again, this time to 18, spread among the same buildings as before, but also with more dorms and social buildingsformat: tcpdump formatdescription: The packet headers from every wireless packet sniffed in 18 buildings on campus. The 2003/4 data was used for [MobiCom 2004 paper]. This fall03 data also contains a list of device types, as determined using the OS fingerprinting tool p0f. a list of device types, as determined using the OS fingerprinting tool p0f (note that the MAC addresses in this list are only devices that we saw associate with an AP (i.e., that appeared in the syslog or SNMP data). Thus it does not include non-wireless client MAC addresses, such as routers or spoofed MACs that do not appear in syslog.). There is also a brief note about the VoIP data included in this trace.last modified: 2006-11-14dataname: dartmouth/campus/tcpdump/fall03version: 20041109change: TCPDUMP trace is resanitized.release date: 2004-11-09date/time of measurement start: 2003-11-02date/time of measurement end: 2004-02-28dartmouth/campus/movementTwo-year records showing the location (AP association) of each wireless card seen on campus.files: movement.tar.gz, movement-v1.3.tar.gz, APlocations.csvdescription: Over three years of nearly continuous records showing the location (access-point association) of each wireless card seen on campus. We used this data for our study of location predictors, published in [INFOCOM'04 paper] and a subsequent, expanded [technical report]. This data is derived from the syslog data. The trace used for this paper is gzipped tar file [51MB]. An updated movement history dataset (up to 2004-06-30) is gzipped tar file [166MB].measurement purpose: User Mobility Characterizationmethodology: We extracted user traces from dartmouth/campus/syslog. Each user's trace is a series of locations, that is, access-point names. We introduced the special location 'OFF' to represent the user's departure from the network (which occurs when the user turns off their computer or their wireless card, or moves out of range of all access points). The traces varied widely in length (the number of locations in the sequence). Users with longer traces were either more active (using their card more), more mobile (thus changing access points more often), or used the network for a longer period (some users have been on the network since April 2001, and some others have only recently arrived on campus).sanitization: same as dartmouth/campus/sysloglast modified: 2007-01-31dataname: dartmouth/campus/movementversion: 20050308change: An updated movement history trace (up to 2004-06-30) is added.release date: 2005-03-08limitation: same as dartmouth/campus/sysloghole: same as dartmouth/campus/syslogdartmouth/campus/movement Tracesinfocom04: Two-year records showing the location (AP association) of each wireless card seen on campus.configuration: This trace is derived from the trace dartmouth/campus/syslog/01_04.format: timestamp, associated APdescription: Over two years of nearly continuous records showing the location (access-point association) of each wireless card seen on campus. We used this data for our study of location predictors, published in [INFOCOM'04 paper] and a subsequent, expanded [technical report]. This data is derived from the syslog data. The trace used for this paper is gzipped tar file [51MB]. .last modified: 2006-11-14dataname: dartmouth/campus/movement/infocom04version: 20040805change: Infocom04 movement trace is newly createdrelease date: 2004-08-05url: /download/dartmouth/campus/movement/movement.tar.gz01_04: Three-year records showing the location (AP association) of each wireless card seen on campus.configuration: This trace is derived from the trace dartmouth/campus/syslog/01_04.format: timestamp, associated APdescription: Over three years of nearly continuous records showing the location (access-point association) of each wireless card seen on campus. This updated movement history dataset (up to 2004-06-30) is gzipped tar file [166MB].last modified: 2007-01-31dataname: dartmouth/campus/movement/01_04version: 20050308change: This trace is newly added.release date: 2005-03-08date/time of measurement start: 2001-04-01date/time of measurement end: 2004-06-30url: /download/dartmouth/campus/movement/movement-v1.3.tar.gzaplocations: A comma-separated list of most of the APs on campus and their location.configuration: APlocations.csv: This file contains the locations of the APs. We used building blueprints and a campus map in AutoCAD format, and then painstakingly plotted most of the APs on the map. The rough correlation is approximately 1.7 coordinate units per foot. The Z coordinate is the floor. 99 means an unknown floor. -1 X/Y coordinates mean that we don't know where this AP is located. Tristan Henderson, November 2004format: AP, x coordinate (-1 = unknown), y coordinate (-1 = unknown), z coordinate (floor, 99 = unknown)description: A comma-separated list of most of the APs on campus and their location, as defined in coordinates on an AutoCAD map of the campus.last modified: 2006-11-14dataname: dartmouth/campus/movement/aplocationsversion: 20041109change: AP locations are addedrelease date: 2004-11-09url: /download/dartmouth/campus/movement/APlocations.csv本数据集包含来自达特茅斯学院的系统日志（syslog）、SNMP和tcpdump数据，时间跨度达5年以上，覆盖超过450个接入点（AP）和数千名用户。 最后修改：2008-09-12 发布日期：2007-02-08 测量起始时间：2001-04-11 测量结束时间：2006-10-04 采集环境： 达特茅斯学院校园占地200英亩，拥有190多栋建筑。学院约有5500名学生和1200名教职员工，在研究期间，校园内约有3200-3300名本科生。学生必须拥有个人电脑，大多数学生通过校园电脑商店购买。笔记本电脑在购买中占比逐年上升：2000年为45%，2001年为70%，2002年为88%，2003年达97%。假设在其他地方购买电脑的学生按同样比例选择笔记本电脑，我们估计在研究期间，超过75%的本科生拥有笔记本电脑。 网络配置： 2001年，校园内安装了476个Cisco 802.11b接入点（AP），覆盖了大部分校园。此后，为扩大覆盖范围和新建筑增加了AP，目前共有566个AP。校园布局紧凑，室内AP的信号范围可覆盖大部分室外区域。所有AP共享相同的SSID，允许无线客户端在AP之间无缝漫游。但每栋建筑的AP连接到该建筑现有的子网。覆盖无线的188栋建筑跨越115个子网，因此在不同建筑间漫游的客户端可能需要获取新的IP地址。客户端通过DHCP获取IP地址（在轨迹不同时段，租约时间为6或12小时）。 数据采集方法： 我们使用了三种技术：syslog事件、SNMP轮询和网络嗅探器（tcpdump）。我们还从syslog数据中推导出移动历史数据。 脱敏处理： 所有数据均已脱敏，以保护用户隐私。我们于2004-11-08重新进行了脱敏，使得在所有轨迹中，MAC地址映射和IP地址映射保持一致。换言之，脱敏后的IP地址111.222.333.444在所有轨迹中对应相同的原始IP地址；同样，脱敏后的MAC地址aa:bb:cc:dd:ee:ff始终对应相同的原始MAC地址。请注意，由于DHCP、MAC欺骗等原因，这些可能不代表同一物理设备。 轨迹集 dartmouth/campus/syslog 带时间戳的、已脱敏的达特茅斯无线网络syslog记录。 文件：syslog-v3.3.tar.gz, syslog-2005-2006-cisco.tar.gz, syslog-2005-2006-aruba.tar.gz, syslog-2005-2006-merged.tar.gz 描述：该轨迹集几乎连续记录了从2001-04-11至2004-06-30以及从2005-09-01至2006-10-04期间由接入点产生的syslog记录。每条日志记录添加了UNIX时间戳，并对MAC地址和AP名称进行了脱敏。 测量目的：使用特征刻画、用户移动性特征刻画 方法：我们将接入点配置为在客户端网卡关联或解关联时发送syslog消息（Cisco AP和Aruba AP的配置不同，详见轨迹dartmouth/campus/syslog/01_04和dartmouth/campus/syslog/05_06）。达特茅斯目前关联网络无需认证，因此我们不知道用户身份，且用户获得的IP地址在不同时间和不同建筑中会变化。 最后修改：2007-02-08 数据名称：dartmouth/campus/syslog 版本：20070208 变更：添加了新的syslog轨迹（收集自2005-09-02至2006-10-04）。 发布日期：2007-02-08 测量起始时间：2001-04-11 测量结束时间：2006-10-04 缺失时段：我们未发布2004-07-01至2005-08-31期间的syslog轨迹。 dartmouth/campus/syslog 子轨迹 01_04 带时间戳的、已脱敏的达特茅斯无线网络syslog记录。 格式：时间戳、AP名称、网卡MAC地址、消息类型 描述：从2001-04-11至2004-06-30几乎连续记录的AP产生的syslog记录。每条记录添加了UNIX时间戳，MAC地址和AP名称已脱敏。 最后修改：2007-01-31 数据名称：dartmouth/campus/syslog/01_04 版本：20041218 变更：新创建的syslog轨迹。 发布日期：2004-12-18 测量起始时间：2001-04-11 测量结束时间：2004-06-30 URL：/download/dartmouth/campus/syslog/syslog-v3.3.tar.gz 缺失：2001年秋季存在一些缺失。由于许多AP未发送syslog（配置错误），存在“空间空洞”；以及由于syslog记录服务器故障导致的“时间空洞”。详情请参见说明。另外，工程学院所在建筑“cummings”的AP在2002年初安装防火墙后未发送任何消息，直到我在2002年末注意到问题并请求他们在防火墙上打开一个缺口。我们不发布2004-07-01至2005-08-31的syslog轨迹。 局限性：由于syslog消息从AP以UDP方式发送到中继服务器（ns1），再从中继服务器到我们的syslog记录服务器，沿途可能丢失或乱序。时间戳由我们主机上的syslog守护进程添加，因此时间戳单调递增，但事件可能记录顺序错乱，部分可能丢失。我们认为这一影响足够小，可忽略不计。我们有两台syslog记录服务器，未发现同一事件在两台服务器上有不同时间戳。自2003-10-19起此问题不再适用。 脱敏：每个MAC地址已被脱敏，并移除了客户端的IP地址或主机名。脱敏MAC地址时，我们将后六位十六进制数字随机化。我们收集了所有syslog、SNMP和tcpdump轨迹中的每个MAC地址，并建立了一个将真实MAC映射到随机MAC的大型表，确保所有映射唯一。每个接入点名称以“AcadBldg10AP3”等形式匿名化，其中表示第10栋“学术”类建筑中的第3个AP。建筑类型包括：Adm（行政）、Ath（体育）、Lib（图书馆）、Oth（其他——主要是系统管理员测试AP）、Res（住宅）和Soc（社交）。详情请参见说明。 dartmouth/campus/syslog 子轨迹 05_06 2005-2006年期间带时间戳的、已脱敏的达特茅斯无线网络syslog记录。 配置：[Cisco AP] 我们将Cisco接入点配置为在客户端认证、关联、重新关联、解关联或取消认证时发送syslog消息。每条消息包含AP名称、网卡MAC地址和消息类型。[Aruba AP] 在我们的校园中，我们部署了Aruba无线网络，使用Aruba 5000交换机作为主交换机，集中控制无线网络。配置具有三级层次结构（主交换机-交换机-AP），多个交换机连接到主交换机，每个交换机下连接多个AP。我们有三种Aruba AP型号：52、61和72。无线网络被虚拟划分为多个区域（子网），每个区域有一个控制器控制一组AP。Aruba syslog消息来自主交换机（中央控制器）或每个区域控制器。由于不同区域覆盖不同AP，每个区域控制器分别发送syslog消息。Aruba系统能够在每个AP上配置多个ESSID。在收集这些syslog数据时，我们使用了四个ESSID——“Kiewit Wireless”、“Kiewit Video”、“Kiewit Voice”和“Hanover Inn”。其中，Kiewit Video/Voice使用NAT和私有（RFC1918）地址。Aruba syslog轨迹中没有L2关联/解关联/认证/取消认证消息，因为Aruba交换机不提供这些。我们拥有的是“station up”和“station down”消息，指示交换机何时看到客户端通过BSSID连接。虽然我们不知道这些station up/down消息在802.11 FSM中对应什么，但对于大多数分析，应该可以或多或少地将这些消息与Cisco syslog中的关联/解关联消息关联起来。 格式：1. 目录和文件 此轨迹包含三个压缩包（目录）：'syslog-2005-2006-cisco'、'syslog-2005-2006-aruba'和'syslog-2005-2006-merged'。'syslog-2005-2006-cisco'和'syslog-2005-2006-aruba'目录分别包含Cisco AP和Aruba AP的syslog记录。'syslog-2005-2006-merged'目录包含从'cisco'和'aruba' syslog记录合并并按时间戳排序的记录。每个目录包含测量期间每天的系统日志轨迹文件。文件名遵循YYMMDD.HHMMSS.{syslog或aruba或merged}格式，文件名中的时间为UTC。对于Aruba syslog轨迹，我们将每个接入点名称表示为位置ID（格式为[building_id.floor_id.ap_id]，如15.1.1）。在轨迹根目录下，我们提供了一个名为'aruba_locid_table.csv'的文件，包含位置ID与脱敏AP名称之间的映射（例如，15.1.1和AcadBldg10AP3的对应关系）。2. Cisco syslog记录格式如下：unix_timestamp timestamp1 AP_name1 timestamp2 counter AP_name2 timestamp2 syslog_message unix_timestamp：UTC UNIX时间戳 timestamp1：接收syslog消息的时间 AP_name1：发送syslog的主机（脱敏）主机名 counter：内部计数器 AP_name2：AP认为自身的主机名（脱敏）。有时AP_name1和AP_name2不匹配。 timestamp2：AP的时钟。有时timestamp1和timestamp2不匹配。 syslog_message：syslog消息内容 以下是Cisco IOS syslog轨迹中的示例行： 1157014202 Aug 31 04:50:02 AcadBldg33AP1 14375: AcadBldg33AP1 Aug 31 08:50:01: %DOT11-6-ASSOC: Interface Dot11Radio0, Station 0016cbf7ca65 Associated KEY_MGMT[NONE] Aruba syslog记录格式如下：unix_timestamp timestamp ip_address1 year [ip_address2] syslog_message unix_timestamp：UTC UNIX时间戳 timestamp, year：接收syslog消息的时间 ip_address1：控制器（脱敏）IP地址 ip_address2：发送此消息的另一控制器（脱敏）IP地址。有时消息并非直接来自控制器，而是来自树下层控制器。 syslog_message：syslog消息内容 以下是Aruba syslog轨迹中的示例行： 1159954944 Oct 4 05:42:24 50.32.208.194 2006 [50.32.208.195] authmgr[510]: station down bssid 00:0b:86:d3:ab:80, essid Kiewit Voice, vlan 2242, ingress 0x1168 (tunnel 264), u_encr 1, m_encr 1, loc 15.1.1 slotport 0xfc7 此消息表示连接到AP的用户已从用户表中移除。之前该用户连接到了位置15.1.1的VLAN 2242。断开原因可能如下：用户移出网络，或用户注销。 描述：从2005-09-01至2006-10-04几乎连续记录的AP产生的syslog记录。每条记录添加了UNIX时间戳，MAC地址和AP名称已脱敏。 最后修改：2007-02-08 数据名称：dartmouth/campus/syslog/05_06 版本：20070208 变更：此2005-2006 syslog轨迹为新创建。 发布日期：2007-02-08 测量起始时间：2005-09-01 测量结束时间：2006-10-04 脱敏：每个MAC地址通过随机化后六位十六进制数字进行脱敏，并使用保留前缀的IP地址脱敏器映射所有IP地址。[Cisco syslog] 每个接入点名称以“AcadBldg10AP3”等形式匿名化，建筑类型同上。[Aruba syslog] 每个接入点名称表示为位置ID（[building_id.floor_id.ap_id]，如15.1.1），并提供'aruba_locid_table.csv'文件映射。 轨迹集 dartmouth/campus/snmp 达特茅斯学院SNMP轮询记录。 文件：fall01.tar.gz, spring02.tar.gz, fall03.tar.gz 描述：该轨迹集包含2001年秋季学期、2002年春季学期、2003年秋季学期和2004年冬季学期期间每5分钟轮询一次所有AP的记录。2001年秋季数据用于[MobiCom 2002论文]。2003/2004年数据用于[MobiCom 2004论文]。我们建议仅使用2003/2004年数据。请参阅关于2001/2002 SNMP数据集问题的重要说明。关于2001/2002数据的任何问题将进入我们的LBE（Less than Best Effort）队列，即可能得不到答复……请改用2003/2004数据。 测量目的：使用特征刻画 方法：我们使用简单网络管理协议（SNMP）每5分钟轮询每个AP，查询AP和客户端特定计数器。AP特定变量包括入/出字节、数据包和错误，以及与给定AP关联的客户端。客户端特定变量包括MAC和IP地址、信号强度和信噪比。 脱敏：为脱敏数据，我们随机（但一致地）将MAC地址字段映射为同一供应商的随机MAC，并使用保留前缀的IP脱敏器映射所有IP地址。 最后修改：2008-09-12 数据名称：dartmouth/campus/snmp 版本：20041109 变更：SNMP轨迹集重新脱敏。 发布日期：2004-11-09 测量起始时间：2001-09-14 测量结束时间：2004-02-28 局限性：我们无法区分无客户端时段与AP关闭或不可达时段。我们也没有办法检测AP重启或重置，这会重置此处报告的所有每客户端计数器。因此在解释每客户端字节计数时必须格外小心……计数器可能在两次轮询之间重置或溢出，差值可能为负（或出现无符号数学中的大数）。 缺失：数据采集中存在一些不幸的缺口，通常由断电引起。 错误：在2001/2002年数据中，执行SNMP查询的Perl脚本存在一些问题，例如查询了不合适的SNMP值或误解了其他值的含义。这些数据也用于后续分析。相同的脚本用于后续对另一个无线网络的研究。参见 http://www.cs.dartmouth.edu/reports/abstracts/TR2003-480/ dartmouth/campus/snmp 子轨迹 fall01 2001年秋季学期SNMP轮询记录。 配置：轮询所有AP的工作由两台机器分担：agentnews和klebb。每个主机在此都有一个目录。每个目录下有许多子目录，每个日期一个，每个子目录中有多个SNMP日志文件。 格式：有两种文件类型。第一种类型每个AP一个文件，每个文件给出每个轮询每个接口的信息。大多数接口无关紧要；你只需关注无线接口（ifSpeed=11000000的那个）。示例如下： V1.0ap timestamp,ap,ifIndex,ifInOctets,ifOutOctets,ifSpeed,ifInErrors,ifOutErrors 注意文件格式版本和后续行中使用的MIB变量名称的定期重复声明。第二种类型文件每个AP每个日期出现一次，文件名包含AP的匿名名称，如AcadBldg10AP3，其中建筑类型同上。每个数据文件包含该AP在该日期的所有数据。以下是文件顶部示例。前两行为文档，第一行指示文件格式版本（始终为V1.0），第二行标识数据行的列标题。时间戳（标准Unix秒）之后，其余字段来自AWC MIB（Aironet Wireless Communications是开发我们AP的公司名称；Aironet被Cisco收购后以Cisco品牌销售）。 V1.0 timestamp,awcTpFdbAddress,awcTpFdbClassID,awcTpFdbSrcOctetsImmed,awcTpFdbDestOctetsImmed,awcTpFdbIPv4Addr,awcTpFdbDdpProdDevID,awcTpFdbDdpRadioDevID,awcDot11TpFdbAID,awcDot11TpFdbTxShortRetries,awcDot11TpFdbLatestRxSignalQuality,awcDot11TpFdbCapabilities 1001908847,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,-15,unknown,state2,73,73 ...（后续行） 描述：2001年秋季学期每AP SNMP轮询轨迹。 最后修改：2008-09-12 数据名称：dartmouth/campus/snmp/fall01 版本：20041109 变更：SNMP轨迹重新脱敏。 发布日期：2004-11-09 测量起始时间：2001-09-14 测量结束时间：2002-01-10 URL：/download/dartmouth/campus/snmp/fall01.tar.gz 错误：秋季期间，一些AP使用旧的有缺陷固件，有时会用垃圾条目填充我们的SNMP数据文件。编写解析代码必须非常健壮。不幸的是，我所使用的代码经过两三个学生的修改，目前无法以可展示的形式提供。参见 http://www.cs.dartmouth.edu/reports/abstracts/TR2003-480/。 dartmouth/campus/snmp 子轨迹 spring02 2002年春季学期SNMP轮询记录。 配置：轮询所有AP的工作由三台机器分担：agentnews、klebb和molari。我短暂使用molari，直到我能够在另外两台之一上完成同样的工作（cummings-* AP被遗漏了一周，因为它们位于防火墙后，最终我在防火墙内设置了molari，最终在防火墙上添加了缺口，使agentnews和klebb可以穿过防火墙轮询）。每台轮询机器有一个目录，每个目录下每个日期有一个子目录，每个日期下包含该机器当天轮询的每个AP的文件。文件中有几种类型的行。以下是从一个文件顶部摘录的示例。注意它使用逗号分隔格式，适合导入电子表格或使用AWK/perl轻松解析。 格式：前五行是注释。第一行给出基本信息：#V2.1: 文件格式版本2.1，文件创建时间戳，AP名称，日期代码YYMMDD。所有时间戳为标准Unix时间戳（自1970年以来的秒数）。其他四行注释描述文件后续行的格式。除时间戳和AP名称外，其余字段为MIB变量名。注释行之后是一系列轮询。每个轮询包含一条“sys”行、一条描述无线接口统计的“if”行，以及零条或多条“c1”和“c2”行对，每一对描述当前连接的客户端。c1和c2行是AWC MIB中的MIB变量集合。 #V2.1,1018929767,AdmBldg27AP2,020416 #sys,timestamp,AP,sysUpTime #if,timestamp,AP,ifIndex,ifType,ifSpeed,ifInOctets,ifInUcastPkts,ifInErrors,ifInDiscards,ifOutOctets,ifOutUcastPkts,ifOutErrors,ifOutDiscards #c1,timestamp,AP,awcDot11TpFdbAddress,awcDot11TpFdbClientState,awcDot11TpFdbLatestRxSignalStrength,awcDot11TpFdbLatestRxSignalQuality #c2,timestamp,AP,awcTpFdbAddress,awcTpFdbClassID,awcTpFdbSrcOctetsImmed,awcTpFdbDestOctetsImmed,awcTpFdbIPv4Addr 描述：2002年春季学期每AP SNMP轮询轨迹。 最后修改：2008-09-12 数据名称：dartmouth/campus/snmp/spring02 版本：20041109 变更：SNMP轨迹重新脱敏。 发布日期：2004-11-09 测量起始时间：2002-03-25 测量结束时间：2002-06-09 URL：/download/dartmouth/campus/snmp/spring02.tar.gz 错误：请注意，dartmouth/campus/snmp/fall01和dartmouth/campus/snmp/spring02数据集存在错误。参见 http://www.cs.dartmouth.edu/reports/abstracts/TR2003-480/。 dartmouth/campus/snmp 子轨迹 fall0304 2003年秋季学期和2004年冬季学期SNMP轮询记录。 配置：在此期间我们只使用了一台机器进行SNMP轮询。SNMP轮询器被重写，更加健壮和高效，因此我们只需一台机器即可轮询校园内约560个AP。此轨迹中我们查询了更多计数器。变量在每个文件开头列出。更多详情请参见MIB：ftp://ftp-sj.cisco.com/pub/mibs/v2/AWCVX-MIB.my 和 ftp://ftp-sj.cisco.com/pub/mibs/v2/IEEE802dot11-MIB.my。在数据收集时，达特茅斯主要使用Cisco 340和350 AP。它们曾运行VxWorks操作系统。在2003年12月至2004年5月期间，我们的350 AP从VxWorks迁移到Cisco IOS（这些AP最初并非运行IOS，因为它们由Aironet制造，后来被Cisco收购）。IOS使用与VxWorks完全不同的SNMP MIB，因此变量名及其顺序略有不同。升级开始后，我们将日志版本号提升为“V3.1”（每个日志的第一行），以指示查询的新变量。我们还将“c1”和“c2”客户端特定行合并为一条“cl”行（这使得解析器代码更易于维护）。 格式：我们将“c1”和“c2”客户端特定行合并为一条“cl”行。要确定给定的SNMP日志是IOS还是VxWorks（参见配置），请查看V3.1日志中的“sys”行。该行的第五个字段是格式化的“sysDescr”，指示AP的操作系统版本。 VxWorks文件示例： #V3.1,1073710881,ResBldg48AP1,040110 #sys,timestamp,AP,sysUpTime,sysDescr #if,timestamp,AP,ifIndex,ifDescr,ifType,ifSpeed,ifInOctets,ifInUcastPkts,ifInErrors,ifInDiscards,ifOutOctets,ifOutUcastPkts,ifOutErrors,ifOutDiscards,awcDot11AssociatedStationCount,awcDot11AuthenticatedStationCount,awcDot11ReassociatedStationCount,awcDot11RoamedStationCount,awcDot11DeauthenticateCount,awcDot11DisassociateCount,awcFtClientSTASelf,awcFtBridgeSelf,awcFtRepeaterSelf #cl,timestamp,AP,awcDot11TpFdbAddress,awcDot11TpFdbAID,awcDot11TpFdbClientState,awcDot11TpFdbLatestRxSignalStrength,awcDot11TpFdbLatestRxSignalQuality,awcTpFdbClassID,awcTpFdbSrcOctetsImmed,awcTpFdbDestOctetsImmed,awcTpFdbIPv4Addr,awcTpFdbSrcPktsImmed,awcTpFdbDestPktsImmed,awcTpFdbSrcErrorPktsImmed,awcTpFdbDestErrorPktsImmed IOS文件示例： #V3.1,1075784465,ResBldg47AP1,040203 #sys,timestamp,AP,sysUpTime,sysDescr #if,timestamp,AP,ifIndex,ifDescr,ifType,ifSpeed,ifInOctets,ifInUcastPkts,ifInErrors,ifInDiscards,ifOutOctets,ifOutUcastPkts,ifOutErrors,ifOutDiscards,cDot11AssStatsAssociated,cDot11AssStatsAuthenticated,cDot11AssStatsRoamedIn,cDot11AssStatsRoamedAway,cDot11AssStatsDeauthenticated,cDot11AssStatsDisassociated,cDot11ActiveWirelessClients,cDot11ActiveBridges,cDot11ActiveRepeaters #cl,timestamp,AP,cDot11ClientAddress,cDot11ClientRoleClassType,cDot11ClientPowerSaveMode,cDot11ClientAid,cDot11ClientAssociationState,cDot11ClientIpAddress,cDot11ClientUpTime,cDot11ClientSignalStrength,cDot11ClientSigQuality,cDot11ClientBytesSent,cDot11ClientBytesReceived,cDot11ClientPacketsSent,cDot11ClientPacketsReceived,cDot11ClientDuplicates,cDot11ClientMsduRetries,cDot11ClientMsduFails 描述：2003年秋季学期和2004年冬季学期每AP SNMP轮询轨迹。 最后修改：2006-11-14 数据名称：dartmouth/campus/snmp/fall0304 版本：20041109 变更：SNMP轨迹重新脱敏。 发布日期：2004-11-09 测量起始时间：2003-11-01 测量结束时间：2004-02-28 URL：/download/dartmouth/campus/snmp/fall03.tar.gz 轨迹集 dartmouth/campus/tcpdump 在达特茅斯学院校园27栋建筑中嗅探到的每个无线数据包的数据包头。 文件：fall01, spring02, fall03, README.voip 描述：在校园内4栋（秋季01）、5栋（春季02）或18栋（秋季03）建筑中嗅探到的每个无线数据包的数据包头。2001年秋季数据用于[MobiCom 2002论文]。2001年秋季和2002年春季数据用于[WiNet 2005论文]。2003/2004年数据用于[MobiCom 2004论文]。fall03数据还包含使用操作系统指纹工具p0f确定的设备类型列表。注意，此列表中的MAC地址仅为我们看到与AP关联的设备（即出现在syslog或SNMP数据中的设备），因此不包括非无线客户端MAC地址，如路由器或未出现在syslog中的欺骗MAC。压缩后的总数据集超过200 GB，因此无法作为压缩包发布。最佳选择是使用curl或wget等HTTP工具从网站下载整个Fall01、Spring02或Fall03目录。或者您可以安排向我们寄送USB或FireWire硬盘（>250GB），我们可以将全部数据复制后寄回。您可以获取README、部分分析软件以及我自己的分析程序输出（列出每个嗅探器每天每个端口号看到的流量量），该文件较小（660 KB）为tgz格式。注意：这仅针对2001/2002年数据。 测量目的：使用特征刻画 方法：我们使用网络“嗅探器”获取详细的网络层轨迹。由于无线网络流量巨大，捕获所有流量不现实。此外，我们的WLAN结构包含多个子网，没有方便的中央点来捕获无线流量。因此，我们在14栋不同建筑中安装了18个嗅探器；在一些大型建筑中，我们需要多个嗅探器来监控该建筑的所有AP。这些建筑是2001年最受欢迎的无线热点，包括图书馆、宿舍、学术院系和社交区域。总共18个嗅探器覆盖了121个AP。每个嗅探器是一台具有两个以太网接口的Linux机器。一个接口用于远程访问，以维护嗅探器和获取分析数据；另一个接口用于收集（“嗅探”）数据。在18个交换机房中，我们将AP连接到交换机，并将另一个端口设置为“镜像”模式，以便该交换机上的所有流量都发送到此端口。嗅探器的第二个接口连接到此镜像端口。我们使用tcpdump捕获通过这些AP及其有线接口的任何无线流量。 脱敏：为脱敏MAC地址，我们将后六位十六进制数字随机化。我们收集了所有syslog、SNMP和tcpdump轨迹中的每个MAC地址，并建立了一个将真实MAC映射到随机MAC的大型表，确保所有映射唯一。[我们没有更改MAC地址000000000000或FFFFFFFFFFFF，它们保持不变。] 我们一致地将此映射应用于所有类型的数据文件，因此如果您在tcpdump文件中看到一个MAC地址，并在SNMP轨迹中再次看到它，可以确定是同一客户端。我们使用了保留前缀的IP地址脱敏器，参见Xu, J., Fan, J. Ammar, M., and Moon, S. “On the Design and Performance of Prefix-Preserving IP Traffic Trace Anonymization”, Proc. of 10th IEEE International Conference on Network Protocols (ICNP 2002), Paris, France, November 2002。这意味着您可以比较脱敏IP地址的前缀，即如果两个IP地址共享相同的k位前缀，则脱敏后的地址也将共享相同的k位前缀。 最后修改：2006-11-14 数据名称：dartmouth/campus/tcpdump 版本：20041109 变更：TCPDUMP轨迹集重新脱敏。 发布日期：2004-11-09 测量起始时间：2001-09-25 测量结束时间：2004-02-28 局限性：在Fall01和Spring02数据集中，我们每天会丢失少量数据。我们每天重新启动tcpdump一次，以开始新的日志文件。我们终止tcpdump进程，然后启动一个新进程；因此，一些tcpdump数据文件以不完整数据包结尾，并且切换过程中无疑丢失了几个数据包。我们错过了同一AP关联的两个客户端之间的流量，因为这不会通过AP的有线接口发送，但我们认为这种情况很少发生。 缺失：数据采集中存在一些缺口，通常由断电引起。 错误：特别是Fall01数据存在大量损坏。旧版本的tcpdump似乎有一个严重错误，导致错误记录许多帧的MAC地址。春季我们使用了较新版本的tcpdump，没有此问题。 dartmouth/campus/tcpdump 子轨迹 fall01 2001年秋季学期达特茅斯学院校园嗅探到的每个无线数据包的数据包头。 配置：我们在达特茅斯校园周围收集了四个嗅探器的数据。每个嗅探器与建筑内的接入点一起连接到集线器。四栋建筑具有代表性：1. Sudikoff（AcadBldg16）：计算机科学系（6个AP）。2. Brown（ResBldg100）：有许多一年级学生的宿舍（2个AP）。3. Berry（LibBldg2）：主校区图书馆。由于建筑规模和网络交换性质，我们只能嗅探13个AP中的5个。4. Collis/Thayer（SocBldg1）：两栋建筑，学生中心和餐厅，包含五个咖啡馆、几个休息区、几个会议室和一些办公室（共9个AP）。 格式：tcpdump格式 描述：校园内四栋（Fall01）建筑中嗅探到的每个无线数据包的数据包头。2001年秋季数据用于[MobiCom 2002论文]和[WiNet 2005论文]。 最后修改：2006-11-14 数据名称：dartmouth/campus/tcpdump/fall01 版本：20041109 变更：TCPDUMP轨迹重新脱敏。 发布日期：2004-11-09 测量起始时间：2001-09-25 测量结束时间：2001-12-10 URL：/download/dartmouth/campus/tcpdump/fall01 缺失：% Sudi 3处缺口，总计约21h:17m:46s；% Brown 15处缺口，总计约213h:10m:33s；% Berry 7处缺口，总计约139h:17m:29s；% Collis 8处缺口，总计约337h:1m:59s 局限性：我们每天丢失少量数据。为保持文件较小并便于备份和操作，我们每天重新启动tcpdump一次，以开始新日志文件。我们终止tcpdump进程，然后启动新进程；因此，一些tcpdump数据文件以不完整数据包结尾，切换过程中丢失了几个数据包。[注意，新版本tcpdump有内置翻转机制，可惜我们追踪时没有。] 秋季我们在午夜重启tcpdump。 错误：数据尤其受到大量损坏。旧版本tcpdump似乎有一个严重错误，导致错误记录许多帧的MAC地址。这些虚假MAC地址，特别是0:0:0:0:0:0、0:0:0:0:0:1、1:0:0:0:0:0或1:0:1:0:1:0，出现在秋季数据约78%的帧中。对于包含IP数据包的帧，我们检查源和目标IP地址；如果该IP地址与最近IP数据包中的有效无线MAC地址相关联，则我们假定此数据包使用相同MAC，并将其视为无线数据包。我们通过此方式修复了约三分之一的错误MAC。 dartmouth/campus/tcpdump 子轨迹 spring02 达特茅斯学院校园嗅探到的每个无线数据包的数据包头（2002年春季）。 配置：我们以相同方式在相同四个位置收集数据，但增加了在Whittemore（塔克商学院的宿舍）8个AP上的嗅探器。此数据比秋季数据干净得多。 格式：tcpdump格式 描述：校园内五栋建筑中嗅探到的每个无线数据包的数据包头。2002年春季数据用于[WiNet 2005论文]。 最后修改：2006-11-14 数据名称：dartmouth/campus/tcpdump/spring02 版本：20041109 变更：TCPDUMP轨迹重新脱敏。 发布日期：2004-11-09 测量起始时间：2002-03-25 测量结束时间：2002-06-09 URL：/download/dartmouth/campus/tcpdump/spring02 缺失：Berry（LibBldg2）：2处缺口，总计约0h:6m:34s；Brown（ResBldg100）：0处缺口；Collis（SocBldg1）：5处缺口，总计约27h:24m:22s；Sudikoff（AcadBldg16）：0处缺口；Whittemore（ResBldg83）：1处缺口，约12h:28m:15s 局限性：我们每天丢失少量数据。为保持文件较小，每天重启tcpdump，在流量最轻的大约凌晨4点重启。 dartmouth/campus/tcpdump 子轨迹 fall03 2003年秋季学期达特茅斯学院校园嗅探到的每个无线数据包的数据包头。 配置：我们再次增加嗅探器数量，这次达到18个，分布在相同建筑以及更多宿舍和社交建筑中。 格式：tcpdump格式 描述：校园内18栋建筑中嗅探到的每个无线数据包的数据包头。2003/2004年数据用于[MobiCom 2004论文]。fall03数据还包含使用操作系统指纹工具p0f确定的设备类型列表（注意，此列表中的MAC地址仅为我们看到与AP关联的设备，即出现在syslog或SNMP数据中的设备，因此不包括非无线客户端MAC地址）。还有关于此轨迹中包含的VoIP数据的简短说明。 最后修改：2006-11-14 数据名称：dartmouth/campus/tcpdump/fall03 版本：20041109 变更：TCPDUMP轨迹重新脱敏。 发布日期：2004-11-09 测量起始时间：2003-11-02 测量结束时间：2004-02-28 轨迹集 dartmouth/campus/movement 两年记录，显示校园内每个无线网卡的位置（AP关联）。 文件：movement.tar.gz, movement-v1.3.tar.gz, APlocations.csv 描述：超过三年的几乎连续记录，显示校园内每个无线网卡的位置（接入点关联）。我们使用这些数据进行位置预测器研究，发表于[INFOCOM'04论文]及后续扩展[技术报告]。此数据源自syslog数据。该论文所用轨迹为gzip压缩tar文件[51MB]。更新后的移动历史数据集（截至2004-06-30）为gzip压缩tar文件[166MB]。 测量目的：用户移动性特征刻画 方法：我们从dartmouth/campus/syslog中提取用户轨迹。每个用户的轨迹是一系列位置，即接入点名称。我们引入特殊位置“OFF”表示用户离开网络（当用户关闭电脑或无线网卡，或移出所有AP范围时发生）。轨迹长度（序列中的位置数）差异很大。轨迹较长的用户要么更活跃（更常使用网卡），要么移动性更强（更频繁更换AP），要么使用网络时间更长（有些用户自2001年4月起就在网上，有些则刚到校园）。 脱敏：与dartmouth/campus/syslog相同 最后修改：2007-01-31 数据名称：dartmouth/campus/movement 版本：20050308 变更：添加了更新后的移动历史轨迹（截至2004-06-30）。 发布日期：2005-03-08 局限性：与dartmouth/campus/syslog相同 缺失：与dartmouth/campus/syslog相同 dartmouth/campus/movement 子轨迹 infocom04 两年记录，显示校园内每个无线网卡的位置（AP关联）。 配置：此轨迹源自dartmouth/campus/syslog/01_04轨迹。 格式：时间戳，关联AP 描述：超过两年的几乎连续记录，显示校园内每个无线网卡的位置。我们使用这些数据进行位置预测器研究，发表于[INFOCOM'04论文]及后续扩展[技术报告]。此数据源自syslog数据。该论文所用轨迹为gzip压缩tar文件[51MB]。 最后修改：2006-11-14 数据名称：dartmouth/campus/movement/infocom04 版本：20040805 变更：新创建Infocom04移动轨迹。 发布日期：2004-08-05 URL：/download/dartmouth/campus/movement/movement.tar.gz dartmouth/campus/movement 子轨迹 01_04 三年记录，显示校园内每个无线网卡的位置（AP关联）。 配置：此轨迹源自dartmouth/campus/syslog/01_04轨迹。 格式：时间戳，关联AP 描述：超过三年的几乎连续记录，显示校园内每个无线网卡的位置。此更新后的移动历史数据集（截至2004-06-30）为gzip压缩tar文件[166MB]。 最后修改：2007-01-31 数据名称：dartmouth/campus/movement/01_04 版本：20050308 变更：新添加此轨迹。 发布日期：2005-03-08 测量起始时间：2001-04-01 测量结束时间：2004-06-30 URL：/download/dartmouth/campus/movement/movement-v1.3.tar.gz aplocations 校园内大多数AP及其位置的逗号分隔列表。 配置：APlocations.csv：此文件包含AP的位置。我们使用建筑蓝图和AutoCAD格式的校园地图，然后费力地将大多数AP绘制在地图上。粗略相关性约为每英尺1.7个坐标单位。Z坐标为楼层，99表示未知楼层。X/Y坐标为-1表示我们不知道此AP的位置。Tristan Henderson，2004年11月 格式：AP, x坐标（-1表示未知）, y坐标（-1表示未知）, z坐标（楼层，99表示未知） 描述：校园内大多数AP及其位置的逗号分隔列表，坐标定义在校园AutoCAD地图上。 最后修改：2006-11-14 数据名称：dartmouth/campus/movement/aplocations 版本：20041109 变更：添加AP位置。 发布日期：2004-11-09 URL：/download/dartmouth/campus/movement/APlocations.csv