Added advanced Heavy Hitter Detection example (#136)

* Added advanced Heavy Hitter Detection example

* Changed directory location

* Restored skeleton version

* Added files for common run infra with the other tutorials

* Updated readme

* Autogenerate setup rules

* Commends in simple_router.p4

* Fix typos

* Removed commended out lines
This commit is contained in:
Georgios Nikolaidis
2018-04-25 00:56:09 -07:00
committed by Antonin Bas
parent 494706bd60
commit e7e6899d5c
23 changed files with 2595 additions and 0 deletions

View File

@@ -0,0 +1 @@
simple_router.config

View File

@@ -0,0 +1,126 @@
# Instructions
## Introduction
In this tutorial, you will implement a heavy hitter detection filter.
Network flows typically have a fairly wide distribution in terms of the
data they transmit, with most of the flows sending little data and few
flows sending a lot. The latter flows are called heavy hitters, and they
often have a detrimental effect to network performance. This is
because they cause congestion, leading to significantly increased completion
times for small, short-lived flows. Detecting heavy hitters allows us to treat them
differently, e.g. we can put their packets in low priority queues, allowing
packets of other flows to face little or no congestion.
In this example, you will implement a heavy hitter detection filter within
a router. You can find a skeleton of the program in simple_router.p4. In that
file, you have to fill in the parts that are marked with TODO.
This example is based on [count-min sketch](http://theory.stanford.edu/~tim/s15/l/l2.pdf).
In fact, we use two count-min sketches which are reset with an offset
equal to their half-life. With every new packet coming in, we update
the values of both sketches but we use only the ones of the least
recently reset one to decide whether a packet belongs to a heavy hitter
flow or not.
> **Spoiler alert:** There is a reference solution in the `solution`
> sub-directory. Feel free to compare your implementation to the
> reference.
## Step 1: Run the (incomplete) starter code
The directory with this README also contains a skeleton P4 program,
`simple_router.p4`, which implements a simple router. Your job will be to
extend this skeleton program to properly implement a heavy hitter
detection filter.
Before that, let's compile the incomplete `simple_router.p4` and bring
up a switch in Mininet to test its behavior.
1. In your shell, run:
```bash
./run.sh
```
This will:
* create a p4app application,
* compile `simple_switch.p4`,
* generate control plane code,
* start a Mininet instance with one switch (`s1`) conected to
two hosts (`h1` and `h2`).
* install the control plane code to your switch,
* The hosts are assigned IPs of `10.0.0.10` and `10.0.1.10`.
2. You should now see a Mininet command prompt. Run ping between
`h1` and `h2` to make sure that everything runs correctly:
```bash
mininet> h1 ping h2
```
You should see all packets going through.
3. Type `exit` to leave each Mininet command line.
### A note about the control plane
A P4 program defines a packet-processing pipeline, but the rules
within each table are inserted by the control plane. When a rule
matches a packet, its action is invoked with parameters supplied by
the control plane as part of the rule.
In this exercise, we have already implemented the control plane
logic for you. As part of invoking `run.sh`, a set of rules is generated
by `setup.py` and when bringing up the Mininet instance, these
packet-processing rules are installed in the tables of
the switch. These are defined in the `simple_router.config` file.
## Step 2: Implement the heavy hitter detection filter
The `simple_router.p4` file contains a skeleton P4 program with key pieces of
logic replaced by `TODO` comments. Your implementation should follow
the structure given in this file, just replace each `TODO` with logic
implementing the missing piece.
More specifically, you need to implement the main actions used within
the heavy hitter detection block. In this example, when our filter
classifies a packet as belonging to a heavy hitter flow, it marks
it as such and then the switch drops it before reaching the
egress control.
## Step 3: Run your solution
Our heavy hitter filter requires periodic reset of the registers of the
count-min sketches. Running:
```bash
bash filter_reset.sh
```
in a terminal window does that periodic reset for you.
The filter currently allows 1000 bytes/sec (you can change that value
in `setup.py`).
In another terminal window, run:
```bash
./run.sh
```
In the minigraph window, you can try:
```
h1 ping -s 80 -i 0.1 h2
```
With this command h1, sends a packet with a total IP length
of 100 bytes every 100 ms. When you run this command, you
shouldn't see any drops. If on the other hand you run:
```
h1 ping -s 80 -i 0.05 h2
```
h1 sends a packet every 50 ms, which puts the flow above
the filter limit. In this case you will observe that about
half of the packets send by h1 are being dropped at the switch.
### Next steps
Check out the code in `setup.py` and `filter_reset.sh`. By changing
the constants in those, you can experiment with different
heavy hitter threshold levels, count-min sketch sizes and the accuracy
of the throughput approximation.

View File

@@ -0,0 +1,26 @@
#!/bin/sh
CONTAINER_ID=`docker ps | tail -n 1 | cut -d ' ' -f 1`
ACTIVE_FILTER='A'
while true; do
CUR_TIME=`echo "get_time_elapsed" | docker exec -i $CONTAINER_ID simple_switch_CLI | grep Runtime | head -n 1 | cut -d ':' -f 2`
CUR_TIME=${CUR_TIME}000
echo $CUR_TIME
echo "register_write last_reset_time 0 $CUR_TIME" | docker exec -i $CONTAINER_ID simple_switch_CLI
if [ $ACTIVE_FILTER == 'A' ] ; then
echo "register_write is_a_active 0 1"
echo "register_reset hashtable_b0" | docker exec -i $CONTAINER_ID simple_switch_CLI
echo "register_reset hashtable_b1" | docker exec -i $CONTAINER_ID simple_switch_CLI
echo "register_reset hashtable_b2" | docker exec -i $CONTAINER_ID simple_switch_CLI
echo "register_reset hashtable_b3" | docker exec -i $CONTAINER_ID simple_switch_CLI
ACTIVE_FILTER='B'
else
echo "register_write is_a_active 0 0"
echo "register_reset hashtable_a0" | docker exec -i $CONTAINER_ID simple_switch_CLI
echo "register_reset hashtable_a1" | docker exec -i $CONTAINER_ID simple_switch_CLI
echo "register_reset hashtable_a2" | docker exec -i $CONTAINER_ID simple_switch_CLI
echo "register_reset hashtable_a3" | docker exec -i $CONTAINER_ID simple_switch_CLI
ACTIVE_FILTER='A'
fi
sleep 4
done

View File

@@ -0,0 +1,83 @@
#ifndef __HEADER_P4__
#define __HEADER_P4__ 1
struct ingress_metadata_t {
bit<32> nhop_ipv4;
}
header ethernet_t {
bit<48> dstAddr;
bit<48> srcAddr;
bit<16> etherType;
}
header ipv4_t {
bit<4> version;
bit<4> ihl;
bit<8> diffserv;
bit<16> totalLen;
bit<16> identification;
bit<3> flags;
bit<13> fragOffset;
bit<8> ttl;
bit<8> protocol;
bit<16> hdrChecksum;
bit<32> srcAddr;
bit<32> dstAddr;
}
header tcp_t {
bit<16> srcPort;
bit<16> dstPort;
bit<32> seqNo;
bit<32> ackNo;
bit<4> dataOffset;
bit<4> res;
bit<8> flags;
bit<16> window;
bit<16> checksum;
bit<16> urgentPtr;
}
header udp_t {
bit<16> srcPort;
bit<16> dstPort;
bit<16> hdrLength;
bit<16> checksum;
}
struct hhd_t {
@name("filter_age")
bit<48> filter_age;
bit<32> value_a0;
bit<32> value_a1;
bit<32> value_a2;
bit<32> value_a3;
bit<32> value_b0;
bit<32> value_b1;
bit<32> value_b2;
bit<32> value_b3;
bit<32> threshold;
bit<1> is_a_active;
bit<1> is_heavy_hitter;
}
struct metadata {
@name("ingress_metadata")
ingress_metadata_t ingress_metadata;
@name("hhd")
hhd_t hhd;
}
struct headers {
@name("ethernet")
ethernet_t ethernet;
@name("ipv4")
ipv4_t ipv4;
@name("tcp")
tcp_t tcp;
@name("udp")
udp_t udp;
}
#endif // __HEADER_P4__

View File

@@ -0,0 +1,10 @@
{
"program": "simple_router.p4",
"language": "p4-16",
"targets": {
"mininet": {
"num-hosts": 2,
"switch-config": "simple_router.config"
}
}
}

View File

@@ -0,0 +1,51 @@
parser ParserImpl(packet_in packet, out headers hdr, inout metadata meta, inout standard_metadata_t standard_metadata) {
state start {
transition parse_ethernet;
}
state parse_ethernet {
packet.extract(hdr.ethernet);
transition select(hdr.ethernet.etherType) {
16w0x800: parse_ipv4;
default: accept;
}
}
state parse_ipv4 {
packet.extract(hdr.ipv4);
transition select(hdr.ipv4.protocol) {
8w0x6: parse_tcp;
default: accept;
}
}
state parse_tcp {
packet.extract(hdr.tcp);
transition accept;
}
}
control DeparserImpl(packet_out packet, in headers hdr) {
apply {
packet.emit(hdr.ethernet);
packet.emit(hdr.ipv4);
packet.emit(hdr.tcp);
}
}
control verifyChecksum(inout headers hdr, inout metadata meta) {
apply { }
}
control computeChecksum(inout headers hdr, inout metadata meta) {
apply {
update_checksum(
hdr.ipv4.isValid(),
{ hdr.ipv4.version, hdr.ipv4.ihl, hdr.ipv4.diffserv,
hdr.ipv4.totalLen, hdr.ipv4.identification,
hdr.ipv4.flags, hdr.ipv4.fragOffset, hdr.ipv4.ttl,
hdr.ipv4.protocol, hdr.ipv4.srcAddr, hdr.ipv4.dstAddr },
hdr.ipv4.hdrChecksum,
HashAlgorithm.csum16);
}
}

View File

@@ -0,0 +1,6 @@
P4APPRUNNER=../utils/p4apprunner.py
python setup.py
mkdir -p build
tar -czf build/p4app.tgz * --exclude='build'
#cd build
sudo python $P4APPRUNNER p4app.tgz --build-dir ./build

View File

@@ -0,0 +1,21 @@
import os
from shutil import copyfile
unit_duration = 20 # log_2 of unit duration (so 2**unit_duration)
total_time_bits = 48
log_units = 3 # log_2 of number of units
units = 2**log_units
threshold = 8*1000.0 # in bytes
copyfile('simple_router.config.template', 'simple_router.config')
with open('simple_router.config', 'a') as fd:
time_mask = (2**(unit_duration+log_units)-1) - (2**unit_duration -1)
for unit in range(units):
time_value = unit*2**unit_duration
if unit < units/2:
unit_threshold = int((unit+1) * threshold / units + threshold/2 )
else:
unit_threshold = int((unit+1) * threshold / units)
fd.write('table_add threshold_table set_threshold %d&&&%d => %d 0\n' % (time_value, time_mask, unit_threshold))

View File

@@ -0,0 +1,12 @@
set_crc16_parameters calc_2 0x1021 0xffff 0x0000 false false
set_crc32_parameters calc_0 0x4c11db7 0xffffffff 0x00000000 false false
table_set_default send_frame egress_drop
table_set_default forward ingress_drop
table_set_default ipv4_lpm ingress_drop
table_add send_frame rewrite_mac 1 => 00:aa:bb:00:00:00
table_add send_frame rewrite_mac 2 => 00:aa:bb:00:00:01
table_add forward set_dmac 10.0.0.10 => 00:04:00:00:00:00
table_add forward set_dmac 10.0.1.10 => 00:04:00:00:00:01
table_add ipv4_lpm set_nhop 10.0.0.10/32 => 10.0.0.10 1
table_add ipv4_lpm set_nhop 10.0.1.10/32 => 10.0.1.10 2
table_add drop_heavy_hitter heavy_hitter_drop 1 0

View File

@@ -0,0 +1,210 @@
#include <core.p4>
#include <v1model.p4>
#include "header.p4"
#include "parser.p4"
const bit<16> MAX_ADDRESS = 0x1F;
const bit<16> THRESHOLD_COUNT = 8;
register<bit<48>>(32w1) last_reset_time;
register<bit<32>>(32w32) hashtable_a0;
register<bit<32>>(32w32) hashtable_a1;
register<bit<32>>(32w32) hashtable_a2;
register<bit<32>>(32w32) hashtable_a3;
register<bit<32>>(32w32) hashtable_b0;
register<bit<32>>(32w32) hashtable_b1;
register<bit<32>>(32w32) hashtable_b2;
register<bit<32>>(32w32) hashtable_b3;
register<bit<1>>(32w1) is_a_active;
control egress(inout headers hdr, inout metadata meta, inout standard_metadata_t standard_metadata) {
action rewrite_mac(bit<48> smac) {
hdr.ethernet.srcAddr = smac;
}
action egress_drop() {
mark_to_drop();
}
table send_frame {
actions = {
rewrite_mac;
egress_drop;
NoAction;
}
key = {
standard_metadata.egress_port: exact;
}
size = 256;
default_action = NoAction();
}
apply {
if (hdr.ipv4.isValid()) {
send_frame.apply();
}
}
}
control HashtableUpdate(in register<bit<32>> hashtable,
in HashAlgorithm algo,
in headers hdr,
inout bit<32> bytecount) {
action update_hashtable() {
/* TODO
Use a hashfunction and calculate the corresponding address
of the count-min sketch based on its five-tuple (hdr.ipv4.srcAddr,
hdr.ipv4.dstAddr, hdr.ipv4.protocol, hdr.tcp.srcPort, hdr.tcp.dstPort)
Read the previous contents of that address, add the packet length to
the previous bytecount, update the register address and keep a
copy of the value in the metadata.
*/
}
apply {
if (hdr.ipv4.isValid()) {
update_hashtable();
}
}
}
control HHD(inout headers hdr,
inout metadata meta,
inout standard_metadata_t standard_metadata) {
HashtableUpdate() update_hashtable_a0;
HashtableUpdate() update_hashtable_a1;
HashtableUpdate() update_hashtable_a2;
HashtableUpdate() update_hashtable_a3;
HashtableUpdate() update_hashtable_b0;
HashtableUpdate() update_hashtable_b1;
HashtableUpdate() update_hashtable_b2;
HashtableUpdate() update_hashtable_b3;
action calculate_age() {
/* TODO
Read the last_reset_time register and calculate
how long has it been since last reset of sketch A based
on standard_metadata.ingress_global_timestamp.
Save the result in meta.hhd.filter_age.
*/
}
action set_threshold(bit<32> threshold) {
/* TODO
Copy the threshlod to metamhhd.threshold
*/
}
action set_filter() {
/* TODO
Check whether count-min sketch A is active
and set meta.hhd.is_a_active flag appropriately
*/
}
action heavy_hitter_drop() {
mark_to_drop();
}
action decide_heavy_hitter() {
/* TODO
Based on whether A is active and the appropriate
meta.hhd.value_xx values, decide, whether
the packet belongs to a heavy hitter flow or not
and set meta.hhd.is_heavy_hitter flag.
*/
}
table threshold_table {
key = {
meta.hhd.filter_age : ternary;
}
actions = {
set_threshold;
}
size = THRESHOLD_COUNT;
}
table drop_heavy_hitter {
key = {
meta.hhd.is_heavy_hitter : exact;
}
actions = {
heavy_hitter_drop;
NoAction;
}
size = 2;
default_action = NoAction();
}
apply {
calculate_age();
set_filter();
threshold_table.apply();
update_hashtable_a0.apply(hashtable_a0, HashAlgorithm.crc32, hdr, meta.hhd.value_a0);
update_hashtable_a1.apply(hashtable_a1, HashAlgorithm.crc32_custom, hdr, meta.hhd.value_a1);
update_hashtable_a2.apply(hashtable_a2, HashAlgorithm.crc16, hdr, meta.hhd.value_a2);
update_hashtable_a3.apply(hashtable_a3, HashAlgorithm.crc16_custom, hdr, meta.hhd.value_a3);
update_hashtable_b0.apply(hashtable_b0, HashAlgorithm.crc32, hdr, meta.hhd.value_b0);
update_hashtable_b1.apply(hashtable_b1, HashAlgorithm.crc32_custom, hdr, meta.hhd.value_b1);
update_hashtable_b2.apply(hashtable_b2, HashAlgorithm.crc16, hdr, meta.hhd.value_b2);
update_hashtable_b3.apply(hashtable_b3, HashAlgorithm.crc16_custom, hdr, meta.hhd.value_b3);
decide_heavy_hitter();
drop_heavy_hitter.apply();
}
}
control ingress(inout headers hdr, inout metadata meta, inout standard_metadata_t standard_metadata) {
action ingress_drop() {
mark_to_drop();
}
action set_nhop(bit<32> nhop_ipv4, bit<9> port) {
meta.ingress_metadata.nhop_ipv4 = nhop_ipv4;
standard_metadata.egress_spec = port;
hdr.ipv4.ttl = hdr.ipv4.ttl + 8w255;
}
action set_dmac(bit<48> dmac) {
hdr.ethernet.dstAddr = dmac;
}
table ipv4_lpm {
actions = {
ingress_drop;
set_nhop;
NoAction;
}
key = {
hdr.ipv4.dstAddr: lpm;
}
size = 1024;
default_action = NoAction();
}
table forward {
actions = {
set_dmac;
ingress_drop;
NoAction;
}
key = {
meta.ingress_metadata.nhop_ipv4: exact;
}
size = 512;
default_action = NoAction();
}
HHD() hhd;
apply {
if (hdr.ipv4.isValid()) {
ipv4_lpm.apply();
forward.apply();
hhd.apply(hdr, meta, standard_metadata);
}
}
}
V1Switch(ParserImpl(), verifyChecksum(), ingress(), egress(), computeChecksum(), DeparserImpl()) main;

View File

@@ -0,0 +1,222 @@
#include <core.p4>
#include <v1model.p4>
#include "header.p4"
#include "parser.p4"
const bit<16> MAX_ADDRESS = 0x1F;
const bit<16> THRESHOLD_COUNT = 8;
register<bit<48>>(32w1) last_reset_time;
register<bit<32>>(32w32) hashtable_a0;
register<bit<32>>(32w32) hashtable_a1;
register<bit<32>>(32w32) hashtable_a2;
register<bit<32>>(32w32) hashtable_a3;
register<bit<32>>(32w32) hashtable_b0;
register<bit<32>>(32w32) hashtable_b1;
register<bit<32>>(32w32) hashtable_b2;
register<bit<32>>(32w32) hashtable_b3;
register<bit<1>>(32w1) is_a_active;
control egress(inout headers hdr, inout metadata meta, inout standard_metadata_t standard_metadata) {
action rewrite_mac(bit<48> smac) {
hdr.ethernet.srcAddr = smac;
}
action egress_drop() {
mark_to_drop();
}
table send_frame {
actions = {
rewrite_mac;
egress_drop;
NoAction;
}
key = {
standard_metadata.egress_port: exact;
}
size = 256;
default_action = NoAction();
}
apply {
if (hdr.ipv4.isValid()) {
send_frame.apply();
}
}
}
control HashtableUpdate(in register<bit<32>> hashtable,
in HashAlgorithm algo,
in headers hdr,
inout bit<32> bytecount) {
action update_hashtable() {
bit<32> hashtable_address;
hash(hashtable_address,
algo,
32w0,
{ hdr.ipv4.srcAddr,
hdr.ipv4.dstAddr,
hdr.ipv4.protocol,
hdr.tcp.srcPort,
hdr.tcp.dstPort },
MAX_ADDRESS);
hashtable.read(bytecount, hashtable_address);
bytecount = bytecount + (bit<32>)hdr.ipv4.totalLen;
hashtable.write(hashtable_address, bytecount);
}
apply {
if (hdr.ipv4.isValid()) {
update_hashtable();
}
}
}
control HHD(inout headers hdr,
inout metadata meta,
inout standard_metadata_t standard_metadata) {
HashtableUpdate() update_hashtable_a0;
HashtableUpdate() update_hashtable_a1;
HashtableUpdate() update_hashtable_a2;
HashtableUpdate() update_hashtable_a3;
HashtableUpdate() update_hashtable_b0;
HashtableUpdate() update_hashtable_b1;
HashtableUpdate() update_hashtable_b2;
HashtableUpdate() update_hashtable_b3;
action calculate_age() {
last_reset_time.read(meta.hhd.filter_age, 32w0);
meta.hhd.filter_age = standard_metadata.ingress_global_timestamp - meta.hhd.filter_age;
}
action set_threshold(bit<32> threshold) {
meta.hhd.threshold = threshold;
}
action set_filter() {
is_a_active.read(meta.hhd.is_a_active, 32w0);
}
action heavy_hitter_drop() {
mark_to_drop();
}
action decide_heavy_hitter() {
if (meta.hhd.is_a_active == 1w1) {
if (meta.hhd.value_a0 > meta.hhd.threshold &&
meta.hhd.value_a1 > meta.hhd.threshold &&
meta.hhd.value_a2 > meta.hhd.threshold &&
meta.hhd.value_a3 > meta.hhd.threshold) {
meta.hhd.is_heavy_hitter = 1w1;
} else {
meta.hhd.is_heavy_hitter = 1w0;
}
} else {
if (meta.hhd.value_b0 > meta.hhd.threshold &&
meta.hhd.value_b1 > meta.hhd.threshold &&
meta.hhd.value_b2 > meta.hhd.threshold &&
meta.hhd.value_b3 > meta.hhd.threshold) {
meta.hhd.is_heavy_hitter = 1w1;
} else {
meta.hhd.is_heavy_hitter = 1w0;
}
}
}
table threshold_table {
key = {
meta.hhd.filter_age : ternary;
}
actions = {
set_threshold;
}
size = THRESHOLD_COUNT;
}
table drop_heavy_hitter {
key = {
meta.hhd.is_heavy_hitter : exact;
}
actions = {
heavy_hitter_drop;
NoAction;
}
size = 2;
default_action = NoAction();
}
apply {
calculate_age();
set_filter();
threshold_table.apply();
update_hashtable_a0.apply(hashtable_a0, HashAlgorithm.crc32, hdr, meta.hhd.value_a0);
update_hashtable_a1.apply(hashtable_a1, HashAlgorithm.crc32_custom, hdr, meta.hhd.value_a1);
update_hashtable_a2.apply(hashtable_a2, HashAlgorithm.crc16, hdr, meta.hhd.value_a2);
update_hashtable_a3.apply(hashtable_a3, HashAlgorithm.crc16_custom, hdr, meta.hhd.value_a3);
update_hashtable_b0.apply(hashtable_b0, HashAlgorithm.crc32, hdr, meta.hhd.value_b0);
update_hashtable_b1.apply(hashtable_b1, HashAlgorithm.crc32_custom, hdr, meta.hhd.value_b1);
update_hashtable_b2.apply(hashtable_b2, HashAlgorithm.crc16, hdr, meta.hhd.value_b2);
update_hashtable_b3.apply(hashtable_b3, HashAlgorithm.crc16_custom, hdr, meta.hhd.value_b3);
decide_heavy_hitter();
drop_heavy_hitter.apply();
}
}
control ingress(inout headers hdr, inout metadata meta, inout standard_metadata_t standard_metadata) {
action ingress_drop() {
mark_to_drop();
}
action set_nhop(bit<32> nhop_ipv4, bit<9> port) {
meta.ingress_metadata.nhop_ipv4 = nhop_ipv4;
standard_metadata.egress_spec = port;
hdr.ipv4.ttl = hdr.ipv4.ttl + 8w255;
}
action set_dmac(bit<48> dmac) {
hdr.ethernet.dstAddr = dmac;
}
table ipv4_lpm {
actions = {
ingress_drop;
set_nhop;
NoAction;
}
key = {
hdr.ipv4.dstAddr: lpm;
}
size = 1024;
default_action = NoAction();
}
table forward {
actions = {
set_dmac;
ingress_drop;
NoAction;
}
key = {
meta.ingress_metadata.nhop_ipv4: exact;
}
size = 512;
default_action = NoAction();
}
HHD() hhd;
apply {
if (hdr.ipv4.isValid()) {
ipv4_lpm.apply();
forward.apply();
hhd.apply(hdr, meta, standard_metadata);
}
}
}
V1Switch(ParserImpl(), verifyChecksum(), ingress(), egress(), computeChecksum(), DeparserImpl()) main;