RSDoS - STARDUST


RSDoS

Overview

Observing Denial-of-service Attacks through Network Telescopes

image

Network telescopes can be used to detect and observe the randomly-spoofed distributed denial-of-service (RSDoS) attacks happening worldwide. To make it difficult for the attack victim (and the victim’s ISPs) to block an incoming attack, the attacker may use a fake source IP address (similar to a fake return address in postal mail) in each packet sent to the victim. Because the DoS attack victim can’t distinguish between incoming requests from an attacker and legitimate inbound requests, the victim tries to respond to every received request. When the attacker spoofs a source address in the network telescope, we observe a response destined for a computer that doesn’t exist (and therefore never sent the initial query). By detecting these unsolicited responses, researchers can identify DoS attack victims and infer information about the volume of the attack, the bandwidth of the victim, the location of the victim, and the types of services the attacker targets.

Definition

This dataset contains meta-data of randomly spoofed DoS attacks from the backscatter packets collected by the UCSD Network Telescope. The data is aggregated from the raw telescope data and is updated every day, containing data starting from October 1, 2008. The data is generated by processing 5-minute intervals of raw telescope data and extracting the response packets sent by victims of RSDoS attacks.

The criteria used to aggregrate the data is outlined in this paper: Inferring Internet Denial-of-Service Activity (2006) by Moore et al.

DoS Data

The RSDoS-Corsaro3 plugin attempts to port the data collection method from Corsaro 2 over to the new Corsaro 3 framework. However, due to increasing traffic volumes (and therefore increasing processing requirements), we have had to make some minor changes to the structure of the data that we save.

Definition

Properties

RSDoS-Corsaro3 writes its output as files using the Apache Avro format. Each record in an Avro files describes an individual attack that was observed within a particular 5 minute time interval. Attacks that span multiple time intervals will write a record for each interval that they occur in.

Record Format
Property Data Type Description
bin_timestamp long The timestamp for the interval that this attack was observed in.
initial_packet_len int The size of the first packet observed as part of this attack.
target_ip long The IP address of the address that was the target of the DoS attack (i.e. the source address of the observed packets). Encoded as a 32 bit integer.
target_protocol int The transport protocol used for the attack (1 = ICMP, 6 = TCP, 17 = UDP).
attacker_slash16_cnt long The number of distinct /16 subnets in our monitored network that received packets from the victim.
attack_port_cnt long The number of unique source ports used by the attacker (i.e. the number of unique destination ports seen on received packets attributed to this attack).
target_port_cnt long The number of unique ports that were targeted on the victim (i.e. the number of unique source ports seen on received packets attributed to this attack).
packet_cnt long The number of packets that were attributed to this attack.
icmp_mismatches long The number of ICMP packets attributed to this attack where the source IP address in the body of the ICMP packet (e.g. the original datagram reflected in a Destination Unreachable message) does not match the IP address that the ICMP packet was sent to.
byte_cnt long The number of bytes that have been sent to our network due to this attack (based on IP length).
max_ppm_interval long The peak observed packet rate observed for this attack.
start_time_sec long The seconds portion of the Unix timestamp of the first packet attributed to this attack.
start_time_usec int The microseconds portion of the Unix timestamp of the first packet attributed to this attack.
latest_time_sec long The seconds portion of the Unix timestamp of the last packet attributed to this attack.
latest_time_usec int The microseconds portion of the Unix timestamp of the last packet attributed to this attack.
first_attack_port int The source port that was used by the first packet that was attributed to this attack.
first_target_port int The destination port that was used by the first packet that was attributed to this attack.
maxmind_continent string The continent where the target IP address is located, according to Maxmind geo-location data.
maxmind_country string The country where the target IP address is located, according to Maxmind geo-location data.
initial_packet bytes A binary blob containing the entire contents of the first packet attributed to this attack, including link layer headers.

User Guide

RSDoS data collected using corsaro3 is stored in Openstack Swift using the Apache Avro data format. RSDoS data collected for the UCSD network telescope can be found in Swift within the telescope-ucsdnt-avro-rsdos container.

Users can read the Avro files directly using existing Avro-compatible tools or libraries, such as the official Apache Avro tools .jar file or the avrocat tool that is available in PyPi.

For writing your own analysis tools, a better option would be to use the PyAvro-STARDUST module to read RSDoS data within Python scripts, which will be faster and simpler to use than using fastavro or other more generic Python libraries. An example script for processing RSDoS data can be found here.

Example using avrocat

limbo@username:~$ wandiocat swift://telescope-ucsdnt-avro-rsdos/datasource=ucsd-nt/year=2020/month=07/day=14/hour=23/ucsd-nt.1594770900.rsdos.avro | avrocat | head

{"bin_timestamp":1623796800,"initial_packet_len":86,"target_ip":2412812948,"target_protocol":6,"attacker_slash16_cnt":171,"attack_port_cnt":570,"target_port_cnt":565,"packet_cnt":570,"icmp_mismatches":0,"byte_cnt":39948,"max_ppm_interval":90,"start_time_sec":1623796672,"start_time_usec":584850,"latest_time_sec":1623797099,"latest_time_usec":667525,"first_attack_port":35598,"first_target_port":50474,"maxmind_continent":"SA","maxmind_country":"BR","initial_packet":"<ýþ\u0019Ø\u0000\u0000Þûº\u0006Ç\b\u0000EÈ\u0000D|e\u0000\u00002\u0001=Ö\u0000x\u0000,.\u000B\u0000æ4\u0000\u0000\u0000\u0000E\b\u0000(\rì@\u0000\u0001\u0006zÂ,.Ð\u000EÅ*\u0014§(è|P\u0010vmG\u0000\u0000¸ùBl"}
{"bin_timestamp":1623796800,"initial_packet_len":86,"target_ip":2868824378,"target_protocol":6,"attacker_slash16_cnt":171,"attack_port_cnt":593,"target_port_cnt":597,"packet_cnt":599,"icmp_mismatches":0,"byte_cnt":41776,"max_ppm_interval":110,"start_time_sec":1623796672,"start_time_usec":658198,"latest_time_sec":1623797098,"latest_time_usec":681133,"first_attack_port":64110,"first_target_port":29995,"maxmind_continent":"SA","maxmind_country":"BR","initial_packet":"<ýþ\u0019Ø\u0000\u0000Þûº\u0006Ç\b\u0000EÈ\u0000Dÿ¢\u0000\u00001\u0001\u0002ѳ0ðA,¦¶e\u000B\u0000L_\u0000\u0000\u0000\u0000E\b\u0000(V@\u0000\u0001\u0006Ëå,¦¶eªþÉ:únu+'ü\u000EÎI¸ïÖP\u0011ÿÿx\u0000\u0000½2cC"}

Deprecated DoS Data formats

These formats are documented here for users that need to access RSDoS data that was collected using the older corsaro2 software, which has not yet been converted to the latest Avro format.

Corsaro 2 (.csv format)

The data are stored in the Swift container named data-telescope-meta-rsdos-daily.

To list the data files:

limbo@username:~$ swift list data-telescope-meta-rsdos-daily | head

Example

limbo@username:~$ swift list data-telescope-meta-rsdos-daily -d /
year=2008/
year=2009/
year=2010/
year=2011/
year=2012/
year=2013/
year=2014/
year=2015/
year=2016/
year=2017/
year=2018/
year=2019/
year=2020/

To view data:

limbo@username:~$ wandiocat swift://data-telescope-meta-rsdos-daily/year=2016/month=01/day=01/ucsd-nt.rsdos-daily-attacks.2016-01-01.ts=1451606400.csv.gz | head
target_ip,nr_attacker_ips,nr_attacker_ports,nr_target_ports,nr_packets,nr_bytes,max_ppm,start_posix_time,end_posix_time,asn,country-code,continent-code
0.33.69.0,34,1,1,373,22380,34,1451667609,1451670895,,??,??
1.1.1.1,1,12,1,7,280,99,1451636930,1451637056,15169,AU,OC
1.170.208.200,759,754,8,5196,1309558,50,1451606406,1451621421,3462,TW,AS

The example output:

target_ip nr_attacker_ips nr_attacker_ports nr_target_ports nr_packets nr_bytes max_ppm start_posix_time end_posix_time asn country-code continent-code
0.33.69.0 34 1 1 373 22380 34 1451667609 1451670895 ?? ??
1.1.1.1 1 12 1 7 280 99 1451636930 1451637056 15169 AU OC
1.170.208.200 759 754 8 5196 1309558 50 1451606406 1451621421 3462 TW AS

Data Properties

For each day the dataset is a single compressed CSV file of attack vector. Each attack vector is uniquely identified by the target IP address and the attack start timestamp and contains the following fields:

Name of Column Definition
target_ip The IP address of the attack victim (target_ip)
nr_attacker_ips The number of distinct attacker IPs in the attack
nr_attacker_ports The number of distinct attacker ports
nr_target_ports The number of distinct target ports
nr_packets The cumulative total number of packets observed in the attack
nr_bytes The cumulative total number of bytes seen for the attack
max_ppm The maximum packet rate (of backscatter packets) seen in the attack, as a moving average per minute
start_posix_time The timestamp of the first observed packet of the attack
end_posix_time The timestamp of the last observed packet of the attack
asn The autonomous system number of target_ip at the time of the attack
country-code Country geolocation of target_ip, at the time of the attack
continent-code Continent geolocation of target_ip, at the time of the attack

Corsaro 2 (subset of .csv format)

This dataset is a subset of the RSDoS Attack Metadata. It contains a single compressed CSV files of the attack vectors from March 1, 2015 to February 28, 2017. This dataset was published in Millions of Targets Under Attack: a Macroscopic Characterization of the DoS Ecosystem (2017) by Jonker et al. and is available to non-STARDUST users as well as STARDUST users.

The dataset is not on swift but can be accessed by academic researchers and US government agencies once this form is filled out and submitted.

Note that it may take between two and five business days to process your request.

More information

Published