diff --git a/.gitignore b/.gitignore index a57dea6..7fb7c0b 100644 --- a/.gitignore +++ b/.gitignore @@ -6,8 +6,12 @@ # Compiled JSON *.json +!*p4app.json *.pcap # Extracted solutions -solution*/ \ No newline at end of file +solution*/ + +# Build folders +build*/ diff --git a/SIGCOMM_2017/P4_tutorial_labs.pdf b/SIGCOMM_2017/P4_tutorial_labs.pdf new file mode 100644 index 0000000..e599505 Binary files /dev/null and b/SIGCOMM_2017/P4_tutorial_labs.pdf differ diff --git a/SIGCOMM_2017/exercises/README.md b/SIGCOMM_2017/exercises/README.md new file mode 100644 index 0000000..256d2b7 --- /dev/null +++ b/SIGCOMM_2017/exercises/README.md @@ -0,0 +1,61 @@ +# P4 Tutorial + +## Introduction + +Welcome to the P4 Tutorial! + +We've prepared a set of exercises to help you get started with P4 +programming, organized into four modules: + +1. Introduction +* [Basic Forwarding](./basic) +* [Scrambler](./scrambler) + +2. Monitoring and Debugging +* [Explicit Congestion Notification](./ecn) +* [Multi-Hop Route Inspection](./mri) + +3. Advanced Data Structures +* [Source Routing](./source_routing) +* [Calculator](./calc) + +4. Dynamic Behavior +* [Load Balancing](./load_balance) +* [HULA](./hula) + +## Obtaining required software + +If you are starting this tutorial at SIGCOMM 2017, then we've already +provided you with a virtual machine that has all of the required +software installed. + +Otherwise, to complete the exercises, you will need to either build a +virtual machine or install several dependencies. + +To build the virtual machine: +- Install [Vagrant](https://vagrantup.com) and [VirtualBox](https://virtualbox.org) +- `cd vm` +- `vagrant up` +- Log in with username `p4` and password `p4` and issue the command `sudo shutdown -r now` +- When the machine reboots, you should have a graphical desktop machine with the required software pre-installed. + +To install dependences by hand: +- `git clone https://github.com/p4lang/behavioral-model.git` +- `git clone https://github.com/p4lang/p4c` +- `git clone https://github.com/p4lang/tutorials` +Then follow the instructions for how to build each package. Each of +these repositories come with dependencies, which can be installed +using the supplied instructions. The first repository +([behavioral-model](https://github.com/p4lang/behavioral-model)) +contains the P4 behavioral model. It is a C++ software switch that +will implement the functionality specified in your P4 program. The +second repository ([p4c](https://github.com/p4lang/p4c-bm)) is the +compiler for the behavioral model. It takes P4 program and produces a +JSON file which can be loaded by the behavioral model. The third +repository ([tutorial](https://github.com/p4lang/tutorial)) is the P4 +Tutorial itself. You will also need to install `mininet`. On Ubuntu, +it would look like this: + +``` +$ sudo apt-get install mininet +``` diff --git a/SIGCOMM_2017/exercises/basic/README.md b/SIGCOMM_2017/exercises/basic/README.md new file mode 100644 index 0000000..e5203e8 --- /dev/null +++ b/SIGCOMM_2017/exercises/basic/README.md @@ -0,0 +1,163 @@ +# Implementing Basic Forwarding + +## Introduction + +The objective of this exercise is to write a P4 program that +implements basic forwarding. To keep things simple, we will just +implement forwarding for IPv4. + +With IPv4 forwarding, the switch must perform the following actions +for every packet: (i) update the source and destination MAC addresses, +(ii) decrement the time-to-live (TTL) in the IP header, and (iii) +forward the packet out the appropriate port. + +Your switch will have a single table, which the control plane will +populate with static rules. Each rule will map an IP address to the +MAC address and output port for the next hop. We have already defined +the control plane rules, so you only need to implement the data plane +logic of your P4 program. + +> **Spoiler alert:** There is a reference solution in the `solution` +> sub-directory. Feel free to compare your implementation to the +> reference. + +## Step 1: Run the (incomplete) starter code + +The directory with this README also contains a skeleton P4 program, +`basic.p4`, which initially drops all packets. Your job will be to +extend this skeleton program to properly forward IPv4 packets. + +Before that, let's compile the incomplete `basic.p4` and bring +up a switch in Mininet to test its behavior. + +1. In your shell, run: + ```bash + ./run.sh + ``` + This will: + * compile `basic.p4`, and + * start a Mininet instance with three switches (`s1`, `s2`, `s3`) + configured in a triangle, each connected to one host (`h1`, `h2`, + and `h3`). + * The hosts are assigned IPs of `10.0.1.1`, `10.0.2.2`, etc. + +2. You should now see a Mininet command prompt. Open two terminals +for `h1` and `h2`, respectively: + ```bash + mininet> xterm h1 h2 + ``` +3. Each host includes a small Python-based messaging client and +server. In `h2`'s xterm, start the server: + ```bash + ./receive.py + ``` +4. In `h1`'s xterm, send a message to `h2`: + ```bash + ./send.py 10.0.2.2 "P4 is cool" + ``` + The message will not be received. +5. Type `exit` to leave each xterm and the Mininet command line. + +The message was not received because each switch is programmed +according to `basic.p4`, which drops all packets on arrival. +Your job is to extend this file so it forwards packets. + +### A note about the control plane + +A P4 program defines a packet-processing pipeline, but the rules +within each table are inserted by the control plane. When a rule +matches a packet, its action is invoked with parameters supplied by +the control plane as part of the rule. + +In this exercise, we have already implemented the the control plane +logic for you. As part of bringing up the Mininet instance, the +`run.sh` script will install packet-processing rules in the tables of +each switch. These are defined in the `sX-commands.txt` files, where +`X` corresponds to the switch number. + +**Important:** A P4 program also defines the interface between the +switch pipeline and control plane. The commands in the files +`sX-commands.txt` refer to specific tables, keys, and actions by name, +and any changes in the P4 program that add or rename tables, keys, or +actions will need to be reflected in these command files. + +## Step 2: Implement L3 forwarding + +The `basic.p4` file contains a skeleton P4 program with key pieces of +logic replaced by `TODO` comments. Your implementation should follow +the structure given in this file---replace each `TODO` with logic +implementing the missing piece. + +A complete `basic.p4` will contain the following components: + +1. Header type definitions for Ethernet (`ethernet_t`) and IPv4 (`ipv4_t`). +2. **TODO:** Parsers for Ethernet and IPv4 that populate `ethernet_t` and `ipv4_t` fields. +3. An action to drop a packet, using `mark_to_drop()`. +4. **TODO:** An action (called `ipv4_forward`) that: + 1. Sets the egress port for the next hop. + 2. Updates the ethernet destination address with the address of the next hop. + 3. Updates the ethernet source address with the address of the switch. + 4. Decrements the TTL. +5. **TODO:** A control that: + 1. Defines a table that will read an IPv4 destination address, and + invoke either `drop` or `ipv4_forward`. + 2. An `apply` block that applies the table. +6. **TODO:** A deparser that selects the order + in which fields inserted into the outgoing packet. +7. A `package` instantiation supplied with the parser, control, and deparser. + > In general, a package also requires instances of checksum verification + > and recomputation controls. These are not necessary for this tutorial + > and are replaced with instantiations of empty controls. + +## Step 3: Run your solution + +Follow the instructions from Step 1. This time, your message from +`h1` should be delivered to `h2`. + +### Food for thought + +The "test suite" for your solution---sending a message from `h1` to +`h2`---is not very robust. What else should you test to be confident +of your implementation? + +> Although the Python `scapy` library is outside the scope of this tutorial, +> it can be used to generate packets for testing. The `send.py` file shows how +> to use it. + +Other questions to consider: + - How would you enhance your program to support next hops? + - Is this program enough to replace a router? What's missing? + +### Troubleshooting + +There are several problems that might manifest as you develop your program: + +1. `basic.p4` might fails to compile. In this case, `run.sh` will +report the error emitted from the compiler and halt. + +2. `basic.p4` might compile but fail to support the control plane +rules in the `s1-commands.txt` through `s3-command.txt` files that +`run.sh` tries to install using the Bmv2 CLI. In this case, `run.sh` +will report these errors to `stderr`. Use these error messages to fix +your `basic.p4` implementation. + +3. `basic.p4` might compile, and the control plane rules might be +installed, but the switch might not process packets in the desired +way. The `build/logs/.log` files contain detailed logs +that describing how each switch processes each packet. The output is +detailed and can help pinpoint logic errors in your implementation. + +#### Cleaning up Mininet + +In the latter two cases above, `run.sh` may leave a Mininet instance +running in the background. Use the following command to clean up +these instances: + +```bash +mn -c +``` + +## Next Steps + +Congratulations, your implementation works! Move on to the next +exercise: implementing the [scrambler](../scrambler)! diff --git a/SIGCOMM_2017/exercises/basic/basic.p4 b/SIGCOMM_2017/exercises/basic/basic.p4 new file mode 100644 index 0000000..80f3676 --- /dev/null +++ b/SIGCOMM_2017/exercises/basic/basic.p4 @@ -0,0 +1,160 @@ +/* -*- P4_16 -*- */ +#include +#include + +const bit<16> TYPE_IPV4 = 0x800; + +/************************************************************************* +*********************** H E A D E R S *********************************** +*************************************************************************/ + +typedef bit<9> egressSpec_t; +typedef bit<48> macAddr_t; +typedef bit<32> ip4Addr_t; + +header ethernet_t { + macAddr_t dstAddr; + macAddr_t srcAddr; + bit<16> etherType; +} + +header ipv4_t { + bit<4> version; + bit<4> ihl; + bit<8> diffserv; + bit<16> totalLen; + bit<16> identification; + bit<3> flags; + bit<13> fragOffset; + bit<8> ttl; + bit<8> protocol; + bit<16> hdrChecksum; + ip4Addr_t srcAddr; + ip4Addr_t dstAddr; +} + +struct metadata { + /* empty */ +} + +struct headers { + ethernet_t ethernet; + ipv4_t ipv4; +} + +/************************************************************************* +*********************** P A R S E R *********************************** +*************************************************************************/ + +parser MyParser(packet_in packet, + out headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + + state start { + /* TODO: add parser logic */ + transition accept; + } +} + + +/************************************************************************* +************ C H E C K S U M V E R I F I C A T I O N ************* +*************************************************************************/ + +control MyVerifyChecksum(in headers hdr, inout metadata meta) { + apply { } +} + + +/************************************************************************* +************** I N G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyIngress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + action drop() { + mark_to_drop(); + } + + action ipv4_forward(macAddr_t dstAddr, egressSpec_t port) { + /* TODO: fill out code in action body */ + } + + table ipv4_lpm { + key = { + hdr.ipv4.dstAddr: lpm; + } + actions = { + ipv4_forward; + drop; + NoAction; + } + size = 1024; + default_action = NoAction(); + } + + apply { + /* TODO: fix ingress control logic */ + ipv4_lpm.apply(); + } +} + +/************************************************************************* +**************** E G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyEgress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + apply { } +} + +/************************************************************************* +************* C H E C K S U M C O M P U T A T I O N ************** +*************************************************************************/ + +control MyComputeChecksum(inout headers hdr, inout metadata meta) { + apply { + update_checksum( + hdr.ipv4.isValid(), + { hdr.ipv4.version, + hdr.ipv4.ihl, + hdr.ipv4.diffserv, + hdr.ipv4.totalLen, + hdr.ipv4.identification, + hdr.ipv4.flags, + hdr.ipv4.fragOffset, + hdr.ipv4.ttl, + hdr.ipv4.protocol, + hdr.ipv4.srcAddr, + hdr.ipv4.dstAddr }, + hdr.ipv4.hdrChecksum, + HashAlgorithm.csum16); + } +} + + +/************************************************************************* +*********************** D E P A R S E R ******************************* +*************************************************************************/ + +control MyDeparser(packet_out packet, in headers hdr) { + apply { + /* TODO: add deparser logic */ + } +} + +/************************************************************************* +*********************** S W I T C H ******************************* +*************************************************************************/ + +V1Switch( +MyParser(), +MyVerifyChecksum(), +MyIngress(), +MyEgress(), +MyComputeChecksum(), +MyDeparser() +) main; \ No newline at end of file diff --git a/SIGCOMM_2017/exercises/basic/p4app.json b/SIGCOMM_2017/exercises/basic/p4app.json new file mode 100644 index 0000000..92df556 --- /dev/null +++ b/SIGCOMM_2017/exercises/basic/p4app.json @@ -0,0 +1,32 @@ +{ + "program": "basic.p4", + "language": "p4-16", + "targets": { + "multiswitch": { + "auto-control-plane": true, + "cli": true, + "pcap_dump": true, + "bmv2_log": true, + "links": [["h1", "s1"], ["s1", "s2"], ["s1", "s3"], ["s3", "s2"], ["s2", "h2"], ["s3", "h3"]], + "hosts": { + "h1": { + }, + "h2": { + }, + "h3": { + } + }, + "switches": { + "s1": { + "entries": "s1-commands.txt" + }, + "s2": { + "entries": "s2-commands.txt" + }, + "s3": { + "entries": "s3-commands.txt" + } + } + } + } +} diff --git a/SIGCOMM_2017/exercises/basic/receive.py b/SIGCOMM_2017/exercises/basic/receive.py new file mode 100755 index 0000000..c93182f --- /dev/null +++ b/SIGCOMM_2017/exercises/basic/receive.py @@ -0,0 +1,52 @@ +#!/usr/bin/env python +import sys +import struct +import os + +from scapy.all import sniff, sendp, hexdump, get_if_list, get_if_hwaddr +from scapy.all import Packet, IPOption +from scapy.all import ShortField, IntField, LongField, BitField, FieldListField, FieldLenField +from scapy.all import IP, UDP, Raw +from scapy.layers.inet import _IPOption_HDR + +def get_if(): + ifs=get_if_list() + iface=None + for i in get_if_list(): + if "eth0" in i: + iface=i + break; + if not iface: + print "Cannot find eth0 interface" + exit(1) + return iface + +class IPOption_MRI(IPOption): + name = "MRI" + option = 31 + fields_desc = [ _IPOption_HDR, + FieldLenField("length", None, fmt="B", + length_of="swids", + adjust=lambda pkt,l:l+4), + ShortField("count", 0), + FieldListField("swids", + [], + IntField("", 0), + length_from=lambda pkt:pkt.count*4) ] +def handle_pkt(pkt): + print "got a packet" + pkt.show2() +# hexdump(pkt) + sys.stdout.flush() + + +def main(): + ifaces = filter(lambda i: 'eth' in i, os.listdir('/sys/class/net/')) + iface = ifaces[0] + print "sniffing on %s" % iface + sys.stdout.flush() + sniff(filter="tcp", iface = iface, + prn = lambda x: handle_pkt(x)) + +if __name__ == '__main__': + main() diff --git a/SIGCOMM_2017/exercises/basic/run.sh b/SIGCOMM_2017/exercises/basic/run.sh new file mode 100755 index 0000000..d5c1947 --- /dev/null +++ b/SIGCOMM_2017/exercises/basic/run.sh @@ -0,0 +1,5 @@ +P4APPRUNNER=../../utils/p4apprunner.py +mkdir -p build +tar -czf build/p4app.tgz * --exclude='build' +#cd build +sudo python $P4APPRUNNER p4app.tgz --build-dir ./build diff --git a/SIGCOMM_2017/exercises/basic/s1-commands.txt b/SIGCOMM_2017/exercises/basic/s1-commands.txt new file mode 100644 index 0000000..3bfdf61 --- /dev/null +++ b/SIGCOMM_2017/exercises/basic/s1-commands.txt @@ -0,0 +1,4 @@ +table_set_default ipv4_lpm drop +table_add ipv4_lpm ipv4_forward 10.0.1.1/32 => 00:00:00:00:01:01 1 +table_add ipv4_lpm ipv4_forward 10.0.2.2/32 => 00:00:00:02:02:00 2 +table_add ipv4_lpm ipv4_forward 10.0.3.3/32 => 00:00:00:03:03:00 3 diff --git a/SIGCOMM_2017/exercises/basic/s2-commands.txt b/SIGCOMM_2017/exercises/basic/s2-commands.txt new file mode 100644 index 0000000..35a49d7 --- /dev/null +++ b/SIGCOMM_2017/exercises/basic/s2-commands.txt @@ -0,0 +1,4 @@ +table_set_default ipv4_lpm drop +table_add ipv4_lpm ipv4_forward 10.0.1.1/32 => 00:00:00:01:02:00 2 +table_add ipv4_lpm ipv4_forward 10.0.2.2/32 => 00:00:00:00:02:02 1 +table_add ipv4_lpm ipv4_forward 10.0.3.3/32 => 00:00:00:03:03:00 3 diff --git a/SIGCOMM_2017/exercises/basic/s3-commands.txt b/SIGCOMM_2017/exercises/basic/s3-commands.txt new file mode 100644 index 0000000..c111108 --- /dev/null +++ b/SIGCOMM_2017/exercises/basic/s3-commands.txt @@ -0,0 +1,4 @@ +table_set_default ipv4_lpm drop +table_add ipv4_lpm ipv4_forward 10.0.1.1/32 => 00:00:00:01:03:00 2 +table_add ipv4_lpm ipv4_forward 10.0.2.2/32 => 00:00:00:02:03:00 3 +table_add ipv4_lpm ipv4_forward 10.0.3.3/32 => 00:00:00:00:03:03 1 diff --git a/SIGCOMM_2017/exercises/basic/send.py b/SIGCOMM_2017/exercises/basic/send.py new file mode 100755 index 0000000..00496d9 --- /dev/null +++ b/SIGCOMM_2017/exercises/basic/send.py @@ -0,0 +1,41 @@ +#!/usr/bin/env python +import argparse +import sys +import socket +import random +import struct + +from scapy.all import sendp, send, get_if_list, get_if_hwaddr +from scapy.all import Packet +from scapy.all import Ether, IP, UDP, TCP + +def get_if(): + ifs=get_if_list() + iface=None # "h1-eth0" + for i in get_if_list(): + if "eth0" in i: + iface=i + break; + if not iface: + print "Cannot find eth0 interface" + exit(1) + return iface + +def main(): + + if len(sys.argv)<3: + print 'pass 2 arguments: ""' + exit(1) + + addr = socket.gethostbyname(sys.argv[1]) + iface = get_if() + + print "sending on interface %s to %s" % (iface, str(addr)) + pkt = Ether(src=get_if_hwaddr(iface), dst='ff:ff:ff:ff:ff:ff') + pkt = pkt /IP(dst=addr) / TCP(dport=1234, sport=random.randint(49152,65535)) / sys.argv[2] + pkt.show2() + sendp(pkt, iface=iface, verbose=False) + + +if __name__ == '__main__': + main() diff --git a/SIGCOMM_2017/exercises/basic/solution/basic.p4 b/SIGCOMM_2017/exercises/basic/solution/basic.p4 new file mode 100644 index 0000000..e51b285 --- /dev/null +++ b/SIGCOMM_2017/exercises/basic/solution/basic.p4 @@ -0,0 +1,176 @@ +/* -*- P4_16 -*- */ +#include +#include + +const bit<16> TYPE_IPV4 = 0x800; + +/************************************************************************* +*********************** H E A D E R S *********************************** +*************************************************************************/ + +typedef bit<9> egressSpec_t; +typedef bit<48> macAddr_t; +typedef bit<32> ip4Addr_t; + +header ethernet_t { + macAddr_t dstAddr; + macAddr_t srcAddr; + bit<16> etherType; +} + +header ipv4_t { + bit<4> version; + bit<4> ihl; + bit<8> diffserv; + bit<16> totalLen; + bit<16> identification; + bit<3> flags; + bit<13> fragOffset; + bit<8> ttl; + bit<8> protocol; + bit<16> hdrChecksum; + ip4Addr_t srcAddr; + ip4Addr_t dstAddr; +} + +struct metadata { + /* empty */ +} + +struct headers { + ethernet_t ethernet; + ipv4_t ipv4; +} + +/************************************************************************* +*********************** P A R S E R *********************************** +*************************************************************************/ + +parser MyParser(packet_in packet, + out headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + + state start { + transition parse_ethernet; + } + + state parse_ethernet { + packet.extract(hdr.ethernet); + transition select(hdr.ethernet.etherType) { + TYPE_IPV4: parse_ipv4; + default: accept; + } + } + + state parse_ipv4 { + packet.extract(hdr.ipv4); + transition accept; + } + +} + +/************************************************************************* +************ C H E C K S U M V E R I F I C A T I O N ************* +*************************************************************************/ + +control MyVerifyChecksum(in headers hdr, inout metadata meta) { + apply { } +} + + +/************************************************************************* +************** I N G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyIngress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + action drop() { + mark_to_drop(); + } + + action ipv4_forward(macAddr_t dstAddr, egressSpec_t port) { + standard_metadata.egress_spec = port; + hdr.ethernet.srcAddr = hdr.ethernet.dstAddr; + hdr.ethernet.dstAddr = dstAddr; + hdr.ipv4.ttl = hdr.ipv4.ttl - 1; + } + + table ipv4_lpm { + key = { + hdr.ipv4.dstAddr: lpm; + } + actions = { + ipv4_forward; + drop; + NoAction; + } + size = 1024; + default_action = NoAction(); + } + + apply { + if (hdr.ipv4.isValid()) { + ipv4_lpm.apply(); + } + } +} + +/************************************************************************* +**************** E G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyEgress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + apply { } +} + +/************************************************************************* +************* C H E C K S U M C O M P U T A T I O N ************** +*************************************************************************/ + +control MyComputeChecksum(inout headers hdr, inout metadata meta) { + apply { + update_checksum( + hdr.ipv4.isValid(), + { hdr.ipv4.version, + hdr.ipv4.ihl, + hdr.ipv4.diffserv, + hdr.ipv4.totalLen, + hdr.ipv4.identification, + hdr.ipv4.flags, + hdr.ipv4.fragOffset, + hdr.ipv4.ttl, + hdr.ipv4.protocol, + hdr.ipv4.srcAddr, + hdr.ipv4.dstAddr }, + hdr.ipv4.hdrChecksum, + HashAlgorithm.csum16); + } +} + +/************************************************************************* +*********************** D E P A R S E R ******************************* +*************************************************************************/ + +control MyDeparser(packet_out packet, in headers hdr) { + apply { + packet.emit(hdr.ethernet); + packet.emit(hdr.ipv4); + } +} + +/************************************************************************* +*********************** S W I T C H ******************************* +*************************************************************************/ + +V1Switch( +MyParser(), +MyVerifyChecksum(), +MyIngress(), +MyEgress(), +MyComputeChecksum(), +MyDeparser() +) main; \ No newline at end of file diff --git a/SIGCOMM_2017/exercises/calc/README.md b/SIGCOMM_2017/exercises/calc/README.md new file mode 100644 index 0000000..1e63bbc --- /dev/null +++ b/SIGCOMM_2017/exercises/calc/README.md @@ -0,0 +1,112 @@ +# Implementing a P4 Calculator + +## Introduction + +The objective of this tutorial is to implement a basic calculator +using a custom protocol header written in P4. The header will contain +an operation to perform and two operands. When a switch receives a +calculator packet header, it will execute the operation on the +operands, and return the result to the sender. + +## Step 1: Run the (incomplete) starter code + +The directory with this README also contains a skeleton P4 program, +`calc.p4`, which initially drops all packets. Your job will be to +extend it to properly implement the calculator logic. + +As a first step, compile the incomplete `calc.p4` and bring up a +switch in Mininet to test its behavior. + +1. In your shell, run: + ```bash + ./run.sh + ``` + This will: + * compile `calc.p4`, and + + * start a Mininet instance with one switches (`s1`) connected to + two hosts (`h1`, `h2`). + * The hosts are assigned IPs of `10.0.1.1` and `10.0.2.2`. + +2. We've written a small Python-based driver program that will allow +you to test your calculator. You can run the driver program directly +from the Mininet command prompt: + +``` +mininet> h1 python calc.py +> +``` + +3. The driver program will provide a new prompt, at which you can type +basic expressions. The test harness will parse your expression, and +prepare a packet with the corresponding operator and operands. It will +then send a packet to the switch for evaluation. When the switch +returns the result of the computation, the test program will print the +result. However, because the calculator program is not implemented, +you should see an error message. + +``` +> 1+1 +Didn't receive response +> +``` + +## Step 2: Implement Calculator + +To implement the calculator, you will need to define a custom +calculator header, and implement the switch logic to parse header, +perform the requested operation, write the result in the header, and +return the packet to the sender. + +We will use the following header format: + + 0 1 2 3 + +----------------+----------------+----------------+---------------+ + | P | 4 | Version | Op | + +----------------+----------------+----------------+---------------+ + | Operand A | + +----------------+----------------+----------------+---------------+ + | Operand B | + +----------------+----------------+----------------+---------------+ + | Result | + +----------------+----------------+----------------+---------------+ + + +- P is an ASCII Letter 'P' (0x50) +- 4 is an ASCII Letter '4' (0x34) +- Version is currently 0.1 (0x01) +- Op is an operation to Perform: + - '+' (0x2b) Result = OperandA + OperandB + - '-' (0x2d) Result = OperandA - OperandB + - '&' (0x26) Result = OperandA & OperandB + - '|' (0x7c) Result = OperandA | OperandB + - '^' (0x5e) Result = OperandA ^ OperandB + + +We will assume that the calculator header is carried over Ethernet, +and we will use the Ethernet type 0x1234 to indicate the presence of +the header. + +Given what you have learned so far, your task is to implement the P4 +calculator program. There is no control plane logic, so you need only +worry about the data plane implementation. + +A working calculator implementation will parse the custom headers, +execute the mathematical operation, write the result in the result +field, and return the packet to the sender. + +## Step 3: Run your solution + +Follow the instructions from Step 1. This time, you should see the +correct result: + +``` +> 1+1 +2 +> +``` + +## Next Steps + +Congratulations, your implementation works! Move on to +[Load Balancer](../load_balance). diff --git a/SIGCOMM_2017/exercises/calc/calc.config b/SIGCOMM_2017/exercises/calc/calc.config new file mode 100644 index 0000000..e69de29 diff --git a/SIGCOMM_2017/exercises/calc/calc.p4 b/SIGCOMM_2017/exercises/calc/calc.p4 new file mode 100644 index 0000000..4b235d0 --- /dev/null +++ b/SIGCOMM_2017/exercises/calc/calc.p4 @@ -0,0 +1,250 @@ +/* -*- P4_16 -*- */ + +/* + * P4 Calculator + * + * This program implements a simple protocol. It can be carried over Ethernet + * (Ethertype 0x1234). + * + * The Protocol header looks like this: + * + * 0 1 2 3 + * +----------------+----------------+----------------+---------------+ + * | P | 4 | Version | Op | + * +----------------+----------------+----------------+---------------+ + * | Operand A | + * +----------------+----------------+----------------+---------------+ + * | Operand B | + * +----------------+----------------+----------------+---------------+ + * | Result | + * +----------------+----------------+----------------+---------------+ + * + * P is an ASCII Letter 'P' (0x50) + * 4 is an ASCII Letter '4' (0x34) + * Version is currently 0.1 (0x01) + * Op is an operation to Perform: + * '+' (0x2b) Result = OperandA + OperandB + * '-' (0x2d) Result = OperandA - OperandB + * '&' (0x26) Result = OperandA & OperandB + * '|' (0x7c) Result = OperandA | OperandB + * '^' (0x5e) Result = OperandA ^ OperandB + * + * The device receives a packet, performs the requested operation, fills in the + * result and sends the packet back out of the same port it came in on, while + * swapping the source and destination addresses. + * + * If an unknown operation is specified or the header is not valid, the packet + * is dropped + */ + +#include +#include + +/* + * Define the headers the program will recognize + */ + +/* + * Standard ethernet header + */ +header ethernet_t { + bit<48> dstAddr; + bit<48> srcAddr; + bit<16> etherType; +} + +/* + * This is a custom protocol header for the calculator. We'll use + * ethertype 0x1234 for is (see parser) + */ +const bit<16> P4CALC_ETYPE = 0x1234; +const bit<8> P4CALC_P = 0x50; // 'P' +const bit<8> P4CALC_4 = 0x34; // '4' +const bit<8> P4CALC_VER = 0x01; // v0.1 +const bit<8> P4CALC_PLUS = 0x2b; // '+' +const bit<8> P4CALC_MINUS = 0x2d; // '-' +const bit<8> P4CALC_AND = 0x26; // '&' +const bit<8> P4CALC_OR = 0x7c; // '|' +const bit<8> P4CALC_CARET = 0x5e; // '^' + +header p4calc_t { + bit<8> op; +/* TODO + * fill p4calc_t header with P, four, ver, op, operand_a, operand_b, and res + entries based on above protocol header definition. + */ +} + +/* + * All headers, used in the program needs to be assembed into a single struct. + * We only need to declare the type, but there is no need to instantiate it, + * because it is done "by the architecture", i.e. outside of P4 functions + */ +struct headers { + ethernet_t ethernet; + p4calc_t p4calc; +} + +/* + * All metadata, globally used in the program, also needs to be assembed + * into a single struct. As in the case of the headers, we only need to + * declare the type, but there is no need to instantiate it, + * because it is done "by the architecture", i.e. outside of P4 functions + */ + +struct metadata { + /* In our case it is empty */ +} + +/************************************************************************* + *********************** P A R S E R *********************************** + *************************************************************************/ +parser MyParser(packet_in packet, + out headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + state start { + packet.extract(hdr.ethernet); + transition select(hdr.ethernet.etherType) { + P4CALC_ETYPE : check_p4calc; + default : accept; + } + } + + state check_p4calc { + /* TODO: just uncomment the following parse block */ + /* + transition select(packet.lookahead().p, + packet.lookahead().four, + packet.lookahead().ver) { + (P4CALC_P, P4CALC_4, P4CALC_VER) : parse_p4calc; + default : accept; + } + */ + } + + state parse_p4calc { + packet.extract(hdr.p4calc); + transition accept; + } +} + +/************************************************************************* + ************ C H E C K S U M V E R I F I C A T I O N ************* + *************************************************************************/ +control MyVerifyChecksum(in headers hdr, + inout metadata meta) { + apply { } +} + +/************************************************************************* + ************** I N G R E S S P R O C E S S I N G ******************* + *************************************************************************/ +control MyIngress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + action send_back(bit<32> result) { + /* TODO + * - put the result back in hdr.p4calc.res + * - swap MAC addresses in hdr.ethernet.dstAddr and + * hdr.ethernet.srcAddr using a temp variable + * - Send the packet back to the port it came from + by saving standard_metadata.ingress_port into + standard_metadata.egress_spec + */ + } + + action operation_add() { + /* TODO call send_back with operand_a + operand_b */ + } + + action operation_sub() { + /* TODO call send_back with operand_a - operand_b */ + } + + action operation_and() { + /* TODO call send_back with operand_a & operand_b */ + } + + action operation_or() { + /* TODO call send_back with operand_a | operand_b */ + } + + action operation_xor() { + /* TODO call send_back with operand_a ^ operand_b */ + } + + action operation_drop() { + mark_to_drop(); + } + + table calculate { + key = { + hdr.p4calc.op : exact; + } + actions = { + operation_add; + operation_sub; + operation_and; + operation_or; + operation_xor; + operation_drop; + } + const default_action = operation_drop(); + const entries = { + P4CALC_PLUS : operation_add(); + P4CALC_MINUS: operation_sub(); + P4CALC_AND : operation_and(); + P4CALC_OR : operation_or(); + P4CALC_CARET: operation_xor(); + } + } + + apply { + if (hdr.p4calc.isValid()) { + calculate.apply(); + } else { + operation_drop(); + } + } +} + +/************************************************************************* + **************** E G R E S S P R O C E S S I N G ******************* + *************************************************************************/ +control MyEgress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + apply { } +} + +/************************************************************************* + ************* C H E C K S U M C O M P U T A T I O N ************** + *************************************************************************/ + +control MyComputeChecksum(inout headers hdr, inout metadata meta) { + apply { } +} + +/************************************************************************* + *********************** D E P A R S E R ******************************* + *************************************************************************/ +control MyDeparser(packet_out packet, in headers hdr) { + apply { + packet.emit(hdr.ethernet); + packet.emit(hdr.p4calc); + } +} + +/************************************************************************* + *********************** S W I T T C H ********************************** + *************************************************************************/ + +V1Switch( +MyParser(), +MyVerifyChecksum(), +MyIngress(), +MyEgress(), +MyComputeChecksum(), +MyDeparser() +) main; diff --git a/SIGCOMM_2017/exercises/calc/calc.py b/SIGCOMM_2017/exercises/calc/calc.py new file mode 100755 index 0000000..d2fb035 --- /dev/null +++ b/SIGCOMM_2017/exercises/calc/calc.py @@ -0,0 +1,97 @@ +#!/usr/bin/env python + +import argparse +import sys +import socket +import random +import struct +import re + +from scapy.all import sendp, send, srp1 +from scapy.all import Packet, hexdump +from scapy.all import Ether, StrFixedLenField, XByteField, IntField +from scapy.all import bind_layers +import readline + +class P4calc(Packet): + name = "P4calc" + fields_desc = [ StrFixedLenField("P", "P", length=1), + StrFixedLenField("Four", "4", length=1), + XByteField("version", 0x01), + StrFixedLenField("op", "+", length=1), + IntField("operand_a", 0), + IntField("operand_b", 0), + IntField("result", 0xDEADBABE)] + +bind_layers(Ether, P4calc, type=0x1234) + +class NumParseError(Exception): + pass + +class OpParseError(Exception): + pass + +class Token: + def __init__(self,type,value = None): + self.type = type + self.value = value + +def num_parser(s, i, ts): + pattern = "^\s*([0-9]+)\s*" + match = re.match(pattern,s[i:]) + if match: + ts.append(Token('num', match.group(1))) + return i + match.end(), ts + raise NumParseError('Expected number literal.') + + +def op_parser(s, i, ts): + pattern = "^\s*([-+&|^])\s*" + match = re.match(pattern,s[i:]) + if match: + ts.append(Token('num', match.group(1))) + return i + match.end(), ts + raise NumParseError("Expected binary operator '-', '+', '&', '|', or '^'.") + + +def make_seq(p1, p2): + def parse(s, i, ts): + i,ts2 = p1(s,i,ts) + return p2(s,i,ts2) + return parse + + +def main(): + + p = make_seq(num_parser, make_seq(op_parser,num_parser)) + s = '' + iface = 'h1-eth0' + + while True: + s = str(raw_input('> ')) + if s == "quit": + break + print s + try: + i,ts = p(s,0,[]) + pkt = Ether(dst='00:04:00:00:00:00', type=0x1234) / P4calc(op=ts[1].value, + operand_a=int(ts[0].value), + operand_b=int(ts[2].value)) + pkt = pkt/' ' + +# pkt.show() + resp = srp1(pkt, iface=iface, timeout=1, verbose=False) + if resp: + p4calc=resp[P4calc] + if p4calc: + print p4calc.result + else: + print "cannot find P4calc header in the packet" + else: + print "Didn't receive response" + except Exception as error: + print error + + +if __name__ == '__main__': + main() diff --git a/SIGCOMM_2017/exercises/calc/p4app.json b/SIGCOMM_2017/exercises/calc/p4app.json new file mode 100644 index 0000000..5c92c80 --- /dev/null +++ b/SIGCOMM_2017/exercises/calc/p4app.json @@ -0,0 +1,10 @@ +{ + "program": "calc.p4", + "language": "p4-16", + "targets": { + "mininet": { + "num-hosts": 2, + "switch-config": "calc.config" + } + } +} diff --git a/SIGCOMM_2017/exercises/calc/run.sh b/SIGCOMM_2017/exercises/calc/run.sh new file mode 100755 index 0000000..d5c1947 --- /dev/null +++ b/SIGCOMM_2017/exercises/calc/run.sh @@ -0,0 +1,5 @@ +P4APPRUNNER=../../utils/p4apprunner.py +mkdir -p build +tar -czf build/p4app.tgz * --exclude='build' +#cd build +sudo python $P4APPRUNNER p4app.tgz --build-dir ./build diff --git a/SIGCOMM_2017/exercises/calc/solution/calc.p4 b/SIGCOMM_2017/exercises/calc/solution/calc.p4 new file mode 100644 index 0000000..4cc2892 --- /dev/null +++ b/SIGCOMM_2017/exercises/calc/solution/calc.p4 @@ -0,0 +1,256 @@ +/* -*- P4_16 -*- */ + +/* + * P4 Calculator + * + * This program implements a simple protocol. It can be carried over Ethernet + * (Ethertype 0x1234). + * + * The Protocol header looks like this: + * + * 0 1 2 3 + * +----------------+----------------+----------------+---------------+ + * | P | 4 | Version | Op | + * +----------------+----------------+----------------+---------------+ + * | Operand A | + * +----------------+----------------+----------------+---------------+ + * | Operand B | + * +----------------+----------------+----------------+---------------+ + * | Result | + * +----------------+----------------+----------------+---------------+ + * + * P is an ASCII Letter 'P' (0x50) + * 4 is an ASCII Letter '4' (0x34) + * Version is currently 0.1 (0x01) + * Op is an operation to Perform: + * '+' (0x2b) Result = OperandA + OperandB + * '-' (0x2d) Result = OperandA - OperandB + * '&' (0x26) Result = OperandA & OperandB + * '|' (0x7c) Result = OperandA | OperandB + * '^' (0x5e) Result = OperandA ^ OperandB + * + * The device receives a packet, performs the requested operation, fills in the + * result and sends the packet back out of the same port it came in on, while + * swapping the source and destination addresses. + * + * If an unknown operation is specified or the header is not valid, the packet + * is dropped + */ + +#include +#include + +/* + * Define the headers the program will recognize + */ + +/* + * Standard ethernet header + */ +header ethernet_t { + bit<48> dstAddr; + bit<48> srcAddr; + bit<16> etherType; +} + +/* + * This is a custom protocol header for the calculator. We'll use + * ethertype 0x1234 for is (see parser) + */ +const bit<16> P4CALC_ETYPE = 0x1234; +const bit<8> P4CALC_P = 0x50; // 'P' +const bit<8> P4CALC_4 = 0x34; // '4' +const bit<8> P4CALC_VER = 0x01; // v0.1 +const bit<8> P4CALC_PLUS = 0x2b; // '+' +const bit<8> P4CALC_MINUS = 0x2d; // '-' +const bit<8> P4CALC_AND = 0x26; // '&' +const bit<8> P4CALC_OR = 0x7c; // '|' +const bit<8> P4CALC_CARET = 0x5e; // '^' + +header p4calc_t { + bit<8> p; + bit<8> four; + bit<8> ver; + bit<8> op; + bit<32> operand_a; + bit<32> operand_b; + bit<32> res; +} + +/* + * All headers, used in the program needs to be assembed into a single struct. + * We only need to declare the type, but there is no need to instantiate it, + * because it is done "by the architecture", i.e. outside of P4 functions + */ +struct headers { + ethernet_t ethernet; + p4calc_t p4calc; +} + +/* + * All metadata, globally used in the program, also needs to be assembed + * into a single struct. As in the case of the headers, we only need to + * declare the type, but there is no need to instantiate it, + * because it is done "by the architecture", i.e. outside of P4 functions + */ + +struct metadata { + /* In our case it is empty */ +} + +/************************************************************************* + *********************** P A R S E R *********************************** + *************************************************************************/ +parser MyParser(packet_in packet, + out headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + + state start { + packet.extract(hdr.ethernet); + transition select(hdr.ethernet.etherType) { + P4CALC_ETYPE : check_p4calc; + default : accept; + } + } + + state check_p4calc { + transition select(packet.lookahead().p, + packet.lookahead().four, + packet.lookahead().ver) { + (P4CALC_P, P4CALC_4, P4CALC_VER) : parse_p4calc; + default : accept; + } + } + + state parse_p4calc { + packet.extract(hdr.p4calc); + transition accept; + } +} + +/************************************************************************* + ************ C H E C K S U M V E R I F I C A T I O N ************* + *************************************************************************/ +control MyVerifyChecksum(in headers hdr, + inout metadata meta) { + apply { } +} + +/************************************************************************* + ************** I N G R E S S P R O C E S S I N G ******************* + *************************************************************************/ +control MyIngress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + + action send_back(bit<32> result) { + bit<48> tmp; + + /* Put the result back in */ + hdr.p4calc.res = result; + + /* Swap the MAC addresses */ + tmp = hdr.ethernet.dstAddr; + hdr.ethernet.dstAddr = hdr.ethernet.srcAddr; + hdr.ethernet.srcAddr = tmp; + + /* Send the packet back to the port it came from */ + standard_metadata.egress_spec = standard_metadata.ingress_port; + } + + action operation_add() { + send_back(hdr.p4calc.operand_a + hdr.p4calc.operand_b); + } + + action operation_sub() { + send_back(hdr.p4calc.operand_a - hdr.p4calc.operand_b); + } + + action operation_and() { + send_back(hdr.p4calc.operand_a & hdr.p4calc.operand_b); + } + + action operation_or() { + send_back(hdr.p4calc.operand_a | hdr.p4calc.operand_b); + } + + action operation_xor() { + send_back(hdr.p4calc.operand_a ^ hdr.p4calc.operand_b); + } + + action operation_drop() { + mark_to_drop(); + } + + table calculate { + key = { + hdr.p4calc.op : exact; + } + actions = { + operation_add; + operation_sub; + operation_and; + operation_or; + operation_xor; + operation_drop; + } + const default_action = operation_drop(); + const entries = { + P4CALC_PLUS : operation_add(); + P4CALC_MINUS: operation_sub(); + P4CALC_AND : operation_and(); + P4CALC_OR : operation_or(); + P4CALC_CARET: operation_xor(); + } + } + + + apply { + if (hdr.p4calc.isValid()) { + calculate.apply(); + } else { + operation_drop(); + } + } +} + +/************************************************************************* + **************** E G R E S S P R O C E S S I N G ******************* + *************************************************************************/ +control MyEgress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + apply { } +} + +/************************************************************************* + ************* C H E C K S U M C O M P U T A T I O N ************** + *************************************************************************/ + +control MyComputeChecksum(inout headers hdr, inout metadata meta) { + apply { } +} + +/************************************************************************* + *********************** D E P A R S E R ******************************* + *************************************************************************/ +control MyDeparser(packet_out packet, in headers hdr) { + apply { + packet.emit(hdr.ethernet); + packet.emit(hdr.p4calc); + } +} + +/************************************************************************* + *********************** S W I T T C H ********************************** + *************************************************************************/ + +V1Switch( +MyParser(), +MyVerifyChecksum(), +MyIngress(), +MyEgress(), +MyComputeChecksum(), +MyDeparser() +) main; diff --git a/SIGCOMM_2017/exercises/ecn/README.md b/SIGCOMM_2017/exercises/ecn/README.md new file mode 100644 index 0000000..709692c --- /dev/null +++ b/SIGCOMM_2017/exercises/ecn/README.md @@ -0,0 +1,195 @@ +# Implementing ECN + +## Introduction + +The objective of this tutorial is to extend basic L3 forwarding with +an implementation of Explict Congestion Notification (ECN). + +ECN allows end-to-end notification of network congestion without +dropping packets. If an end-host supports ECN, it puts the value of 1 +or 2 in the `ipv4.ecn` field. For such packets, each switch may +change the value to 3 if the queue size is larger than a threshold. +The receiver copies the value to sender, and the sender can lower the +rate. + +As before, we have already defined the control plane rules for +routing, so you only need to implement the data plane logic of your P4 +program. + +> **Spoiler alert:** There is a reference solution in the `solution` +> sub-directory. Feel free to compare your implementation to the reference. + +## Step 1: Run the (incomplete) starter code + +The directory with this README also contains a skeleton P4 program, +`ecn.p4`, which initially implements L3 forwarding. Your job (in the +next step) will be to extend it to properly append set the ECN bits + +Before that, let's compile the incomplete `ecn.p4` and bring up a +network in Mininet to test its behavior. + +1. In your shell, run: + ```bash + ./run.sh + ``` + This will: + * compile `ecn.p4`, and + * start a Mininet instance with three switches (`s1`, `s2`, `s3`) configured + in a triangle. There are 5 hosts. `h1` and `h11` are connected to `s1`. + `h2` and `h22` are connected to `s2` and `h3` is connected to `s3`. + * The hosts are assigned IPs of `10.0.1.1`, `10.0.2.2`, etc + (`10.0..`). + * The control plane programs the P4 tables in each switch based on + `sx-commands.txt` +2. We want to send a low rate traffic from `h1` to `h2` and a high +rate iperf traffic from `h11` to `h22`. The link between `s1` and +`s2` is common between the flows and is a bottleneck because we +reduced its bandwidth to 512kbps in p4app.json. Therefore, if we +capture packets at `h2`, we should see the right ECN value. + +3. You should now see a Mininet command prompt. Open four terminals +for `h1`, `h11`, `h2`, `h22`, respectively: + ```bash + mininet> xterm h1 h11 h2 h22 + ``` +3. In `h2`'s XTerm, start the server that captures packets: + ```bash + ./receive.py + ``` +4. in `h22`'s XTerm, start the iperf UDP server: + ```bash + iperf -s -u + ``` +5. In `h1`'s XTerm, send one packet per second to `h2` using send.py +say for 30 seconds: + ```bash + ./send.py 10.0.2.2 "P4 is cool" 30 + ``` + The message "P4 is cool" should be received in `h2`'s xterm, +6. In `h11`'s XTerm, start iperf client sending for 15 seconds + ```bash + iperf -c 10.0.2.22 -t 15 -u + ``` +7. At `h2`, the `ipv4.tos` field (DiffServ+ECN) is always 1 +8. type `exit` to close each XTerm window + +Your job is to extend the code in `ecn.p4` to implement the ECN logic +for setting the ECN flag. + +## Step 2: Implement ECN + +The `ecn.p4` file contains a skeleton P4 program with key pieces of +logic replaced by `TODO` comments. These should guide your +implementation---replace each `TODO` with logic implementing the +missing piece. + +First we have to change the ipv4_t header by splitting the TOS field +into DiffServ and ECN fields. Remember to update the checksum block +accordingly. Then, in the egress control block we must compare the +queue length with ECN_THRESHOLD. If the queue length is larger than +the threshold, the ECN flag will be set. Note that this logic should +happen only if the end-host declared supporting ECN by setting the +original ECN to 1 or 2. + +A complete `ecn.p4` will contain the following components: + +1. Header type definitions for Ethernet (`ethernet_t`) and IPv4 (`ipv4_t`). +2. Parsers for Ethernet, IPv4, +3. An action to drop a packet, using `mark_to_drop()`. +4. An action (called `ipv4_forward`), which will: + 1. Set the egress port for the next hop. + 2. Update the ethernet destination address with the address of + the next hop. + 3. Update the ethernet source address with the address of the switch. + 4. Decrement the TTL. +5. An egress control block that checks the ECN and +`standard_metadata.enq_qdepth` and sets the ipv4.ecn. +6. A deparser that selects the order in which fields inserted into the outgoing + packet. +7. A `package` instantiation supplied with the parser, control, + checksum verfiication and recomputation and deparser. + +## Step 3: Run your solution + +Follow the instructions from Step 1. This time, when your message from +`h1` is delivered to `h2`, you should see `tos` values change from 1 +to 3 as the queue builds up. `tos` may change back to 1 when iperf +finishes and the queue depletes. + +To easily track the `tos` values you may want to redirect the output +of `h2` to a file by running the following for `h2` + ```bash + ./receive.py > h2.log + ``` +and just print the `tos` values `grep tos build/h2.log` in a separate window +``` + tos = 0x1 + tos = 0x1 + tos = 0x1 + tos = 0x1 + tos = 0x1 + tos = 0x1 + tos = 0x1 + tos = 0x1 + tos = 0x1 + tos = 0x1 + tos = 0x1 + tos = 0x1 + tos = 0x1 + tos = 0x3 + tos = 0x3 + tos = 0x3 + tos = 0x3 + tos = 0x3 + tos = 0x3 + tos = 0x1 + tos = 0x1 + tos = 0x1 + tos = 0x1 + tos = 0x1 + tos = 0x1 +``` + +### Food for thought + +How can we let the user configure the threshold? + +### Troubleshooting + +There are several ways that problems might manifest: + +1. `ecn.p4` fails to compile. In this case, `run.sh` will report the + error emitted from the compiler and stop. +2. `ecn.p4` compiles but does not support the control plane rules in + the `sX-commands.txt` files that `run.sh` tries to install using + the BMv2 CLI. In this case, `run.sh` will report these errors to + `stderr`. Use these error messages to fix your `ecn.p4` + implementation. +3. `ecn.p4` compiles, and the control plane rules are installed, but + the switch does not process packets in the desired way. The + `build/logs/.log` files contain trace messages + describing how each switch processes each packet. The output is + detailed and can help pinpoint logic errors in your implementation. + The `build/-.pcap` also contains the + pcap of packets on each interface. Use `tcpdump -r -xxx` + to print the hexdump of the packets. +4. `ecn.p4` compiles and all rules are installed. Packets go through + and the logs show that the queue length was not high enough to set + the ECN bit. Then either lower the threshold in the p4 code or + reduce the link bandwidth in `p4app.json` + +#### Cleaning up Mininet + +In the latter two cases above, `run.sh` may leave a Mininet instance +running in the background. Use the following command to clean up +these instances: + +```bash +mn -c +``` + +## Next Steps + +Congratulations, your implementation works! Move on to the next +exercise: [Multi-Hop Route Inspection](../mri), which identifies which +link is the source of congestion. diff --git a/SIGCOMM_2017/exercises/ecn/ecn.p4 b/SIGCOMM_2017/exercises/ecn/ecn.p4 new file mode 100644 index 0000000..6270e8b --- /dev/null +++ b/SIGCOMM_2017/exercises/ecn/ecn.p4 @@ -0,0 +1,188 @@ +/* -*- P4_16 -*- */ +#include +#include + +const bit<8> TCP_PROTOCOL = 0x06; +const bit<16> TYPE_IPV4 = 0x800; +const bit<19> ECN_THRESHOLD = 10; + +/************************************************************************* +*********************** H E A D E R S *********************************** +*************************************************************************/ + +typedef bit<9> egressSpec_t; +typedef bit<48> macAddr_t; +typedef bit<32> ip4Addr_t; + +header ethernet_t { + macAddr_t dstAddr; + macAddr_t srcAddr; + bit<16> etherType; +} + +/* + * TODO: split tos to two fields 6 bit diffserv and 2 bit ecn + */ +header ipv4_t { + bit<4> version; + bit<4> ihl; + bit<8> tos; + bit<16> totalLen; + bit<16> identification; + bit<3> flags; + bit<13> fragOffset; + bit<8> ttl; + bit<8> protocol; + bit<16> hdrChecksum; + ip4Addr_t srcAddr; + ip4Addr_t dstAddr; +} + +struct metadata { +} + +struct headers { + ethernet_t ethernet; + ipv4_t ipv4; +} + +/************************************************************************* +*********************** P A R S E R *********************************** +*************************************************************************/ + +parser MyParser(packet_in packet, + out headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + + state start { + transition parse_ethernet; + } + + state parse_ethernet { + packet.extract(hdr.ethernet); + transition select(hdr.ethernet.etherType) { + TYPE_IPV4: parse_ipv4; + default: accept; + } + } + + state parse_ipv4 { + packet.extract(hdr.ipv4); + transition accept; + } +} + + +/************************************************************************* +************ C H E C K S U M V E R I F I C A T I O N ************* +*************************************************************************/ + +control MyVerifyChecksum(in headers hdr, inout metadata meta) { + apply { } +} + + +/************************************************************************* +************** I N G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyIngress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + action drop() { + mark_to_drop(); + } + + action ipv4_forward(macAddr_t dstAddr, egressSpec_t port) { + standard_metadata.egress_spec = port; + hdr.ethernet.srcAddr = hdr.ethernet.dstAddr; + hdr.ethernet.dstAddr = dstAddr; + hdr.ipv4.ttl = hdr.ipv4.ttl - 1; + } + + table ipv4_lpm { + key = { + hdr.ipv4.dstAddr: lpm; + } + actions = { + ipv4_forward; + drop; + NoAction; + } + size = 1024; + default_action = NoAction(); + } + + apply { + if (hdr.ipv4.isValid()) { + ipv4_lpm.apply(); + } + } +} + +/************************************************************************* +**************** E G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyEgress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + apply { + /* + * TODO: + * - if ecn is 1 or 2 + * - compare standard_metadata.enq_qdepth with threshold + * and set hdr.ipv4.ecn to 3 if larger + */ + } +} + +/************************************************************************* +************* C H E C K S U M C O M P U T A T I O N ************** +*************************************************************************/ + +control MyComputeChecksum(inout headers hdr, inout metadata meta) { + apply { + /* TODO: replace tos with diffserve and ecn */ + update_checksum( + hdr.ipv4.isValid(), + { hdr.ipv4.version, + hdr.ipv4.ihl, + hdr.ipv4.tos, + hdr.ipv4.totalLen, + hdr.ipv4.identification, + hdr.ipv4.flags, + hdr.ipv4.fragOffset, + hdr.ipv4.ttl, + hdr.ipv4.protocol, + hdr.ipv4.srcAddr, + hdr.ipv4.dstAddr }, + hdr.ipv4.hdrChecksum, + HashAlgorithm.csum16); + } +} + +/************************************************************************* +*********************** D E P A R S E R ******************************* +*************************************************************************/ + +control MyDeparser(packet_out packet, in headers hdr) { + apply { + packet.emit(hdr.ethernet); + packet.emit(hdr.ipv4); + } +} + +/************************************************************************* +*********************** S W I T C H ******************************* +*************************************************************************/ + +V1Switch( +MyParser(), +MyVerifyChecksum(), +MyIngress(), +MyEgress(), +MyComputeChecksum(), +MyDeparser() +) main; \ No newline at end of file diff --git a/SIGCOMM_2017/exercises/ecn/p4app.json b/SIGCOMM_2017/exercises/ecn/p4app.json new file mode 100644 index 0000000..e54eef2 --- /dev/null +++ b/SIGCOMM_2017/exercises/ecn/p4app.json @@ -0,0 +1,37 @@ +{ + "program": "ecn.p4", + "language": "p4-16", + "targets": { + "multiswitch": { + "auto-control-plane": true, + "cli": true, + "pcap_dump": true, + "bmv2_log": true, + "links": [["h1", "s1"], ["h11", "s1"], ["s1", "s2", "0", 0.5], ["s1", "s3"], ["s3", "s2"], ["s2", "h2"], ["s2", "h22"], ["s3", "h3"]], + "hosts": { + "h1": { + }, + "h2": { + }, + "h3": { + }, + "h11": { + }, + "h22": { + } + + }, + "switches": { + "s1": { + "entries": "s1-commands.txt" + }, + "s2": { + "entries": "s2-commands.txt" + }, + "s3": { + "entries": "s3-commands.txt" + } + } + } + } +} diff --git a/SIGCOMM_2017/exercises/ecn/receive.py b/SIGCOMM_2017/exercises/ecn/receive.py new file mode 100755 index 0000000..ca5f119 --- /dev/null +++ b/SIGCOMM_2017/exercises/ecn/receive.py @@ -0,0 +1,37 @@ +#!/usr/bin/env python +import sys +import struct + +from scapy.all import sniff, sendp, hexdump, get_if_list, get_if_hwaddr +from scapy.all import Packet +from scapy.all import IP, UDP, Raw +from scapy.layers.inet import _IPOption_HDR + +def get_if(): + ifs=get_if_list() + iface=None + for i in get_if_list(): + if "eth0" in i: + iface=i + break; + if not iface: + print "Cannot find eth0 interface" + exit(1) + return iface + +def handle_pkt(pkt): + print "got a packet" + pkt.show2() +# hexdump(pkt) + sys.stdout.flush() + + +def main(): + iface = 'h2-eth0' + print "sniffing on %s" % iface + sys.stdout.flush() + sniff(filter="udp and port 4321", iface = iface, + prn = lambda x: handle_pkt(x)) + +if __name__ == '__main__': + main() diff --git a/SIGCOMM_2017/exercises/ecn/run.sh b/SIGCOMM_2017/exercises/ecn/run.sh new file mode 100755 index 0000000..d5c1947 --- /dev/null +++ b/SIGCOMM_2017/exercises/ecn/run.sh @@ -0,0 +1,5 @@ +P4APPRUNNER=../../utils/p4apprunner.py +mkdir -p build +tar -czf build/p4app.tgz * --exclude='build' +#cd build +sudo python $P4APPRUNNER p4app.tgz --build-dir ./build diff --git a/SIGCOMM_2017/exercises/ecn/s1-commands.txt b/SIGCOMM_2017/exercises/ecn/s1-commands.txt new file mode 100644 index 0000000..67e4f74 --- /dev/null +++ b/SIGCOMM_2017/exercises/ecn/s1-commands.txt @@ -0,0 +1,5 @@ +table_set_default ipv4_lpm drop +table_add ipv4_lpm ipv4_forward 10.0.1.1/32 => 00:00:00:00:01:01 1 +table_add ipv4_lpm ipv4_forward 10.0.1.11/32 => 00:00:00:00:01:0b 2 +table_add ipv4_lpm ipv4_forward 10.0.2.0/24 => 00:00:00:02:03:00 3 +table_add ipv4_lpm ipv4_forward 10.0.3.0/24 => 00:00:00:03:02:00 4 diff --git a/SIGCOMM_2017/exercises/ecn/s2-commands.txt b/SIGCOMM_2017/exercises/ecn/s2-commands.txt new file mode 100644 index 0000000..dcbf778 --- /dev/null +++ b/SIGCOMM_2017/exercises/ecn/s2-commands.txt @@ -0,0 +1,5 @@ +table_set_default ipv4_lpm drop +table_add ipv4_lpm ipv4_forward 10.0.2.2/32 => 00:00:00:00:02:02 1 +table_add ipv4_lpm ipv4_forward 10.0.2.22/32 => 00:00:00:00:02:16 2 +table_add ipv4_lpm ipv4_forward 10.0.1.0/24 => 00:00:00:01:03:00 3 +table_add ipv4_lpm ipv4_forward 10.0.3.0/24 => 00:00:00:03:03:00 4 diff --git a/SIGCOMM_2017/exercises/ecn/s3-commands.txt b/SIGCOMM_2017/exercises/ecn/s3-commands.txt new file mode 100644 index 0000000..c7285fb --- /dev/null +++ b/SIGCOMM_2017/exercises/ecn/s3-commands.txt @@ -0,0 +1,4 @@ +table_set_default ipv4_lpm drop +table_add ipv4_lpm ipv4_forward 10.0.3.3/32 => 00:00:00:00:03:03 1 +table_add ipv4_lpm ipv4_forward 10.0.1.10/24 => 00:00:00:01:04:00 2 +table_add ipv4_lpm ipv4_forward 10.0.2.10/24 => 00:00:00:02:04:00 3 diff --git a/SIGCOMM_2017/exercises/ecn/send.py b/SIGCOMM_2017/exercises/ecn/send.py new file mode 100755 index 0000000..f279d41 --- /dev/null +++ b/SIGCOMM_2017/exercises/ecn/send.py @@ -0,0 +1,50 @@ +#!/usr/bin/env python + +import argparse +import sys +import socket +import random +import struct + +from scapy.all import sendp, send, hexdump, get_if_list, get_if_hwaddr +from scapy.all import Packet, IPOption +from scapy.all import Ether, IP, UDP +from scapy.all import IntField, FieldListField, FieldLenField, ShortField +from scapy.layers.inet import _IPOption_HDR + +from time import sleep + +def get_if(): + ifs=get_if_list() + iface=None # "h1-eth0" + for i in get_if_list(): + if "eth0" in i: + iface=i + break; + if not iface: + print "Cannot find eth0 interface" + exit(1) + return iface + +def main(): + + if len(sys.argv)<4: + print 'pass 2 arguments: "" ' + exit(1) + + addr = socket.gethostbyname(sys.argv[1]) + iface = get_if() + + pkt = Ether(src=get_if_hwaddr(iface), dst="ff:ff:ff:ff:ff:ff") / IP(dst=addr, tos=1) / UDP(dport=4321, sport=1234) / sys.argv[2] + pkt.show2() + #hexdump(pkt) + try: + for i in range(int(sys.argv[3])): + sendp(pkt, iface=iface) + sleep(1) + except KeyboardInterrupt: + raise + + +if __name__ == '__main__': + main() diff --git a/SIGCOMM_2017/exercises/ecn/solution/ecn.p4 b/SIGCOMM_2017/exercises/ecn/solution/ecn.p4 new file mode 100644 index 0000000..59b9756 --- /dev/null +++ b/SIGCOMM_2017/exercises/ecn/solution/ecn.p4 @@ -0,0 +1,188 @@ +/* -*- P4_16 -*- */ +#include +#include + +const bit<8> TCP_PROTOCOL = 0x06; +const bit<16> TYPE_IPV4 = 0x800; +const bit<19> ECN_THRESHOLD = 10; + +/************************************************************************* +*********************** H E A D E R S *********************************** +*************************************************************************/ + +typedef bit<9> egressSpec_t; +typedef bit<48> macAddr_t; +typedef bit<32> ip4Addr_t; + +header ethernet_t { + macAddr_t dstAddr; + macAddr_t srcAddr; + bit<16> etherType; +} + +header ipv4_t { + bit<4> version; + bit<4> ihl; + bit<6> diffserv; + bit<2> ecn; + bit<16> totalLen; + bit<16> identification; + bit<3> flags; + bit<13> fragOffset; + bit<8> ttl; + bit<8> protocol; + bit<16> hdrChecksum; + ip4Addr_t srcAddr; + ip4Addr_t dstAddr; +} + +struct metadata { +} + +struct headers { + ethernet_t ethernet; + ipv4_t ipv4; +} + +/************************************************************************* +*********************** P A R S E R *********************************** +*************************************************************************/ + +parser MyParser(packet_in packet, + out headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + + state start { + transition parse_ethernet; + } + + state parse_ethernet { + packet.extract(hdr.ethernet); + transition select(hdr.ethernet.etherType) { + TYPE_IPV4: parse_ipv4; + default: accept; + } + } + + state parse_ipv4 { + packet.extract(hdr.ipv4); + transition accept; + } +} + + +/************************************************************************* +************ C H E C K S U M V E R I F I C A T I O N ************* +*************************************************************************/ + +control MyVerifyChecksum(in headers hdr, inout metadata meta) { + apply { } +} + + +/************************************************************************* +************** I N G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyIngress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + action drop() { + mark_to_drop(); + } + + action ipv4_forward(macAddr_t dstAddr, egressSpec_t port) { + standard_metadata.egress_spec = port; + hdr.ethernet.srcAddr = hdr.ethernet.dstAddr; + hdr.ethernet.dstAddr = dstAddr; + hdr.ipv4.ttl = hdr.ipv4.ttl - 1; + } + + table ipv4_lpm { + key = { + hdr.ipv4.dstAddr: lpm; + } + actions = { + ipv4_forward; + drop; + NoAction; + } + size = 1024; + default_action = NoAction(); + } + + apply { + if (hdr.ipv4.isValid()) { + ipv4_lpm.apply(); + } + } +} + +/************************************************************************* +**************** E G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyEgress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + action mark_ecn() { + hdr.ipv4.ecn = 3; + } + apply { + if (hdr.ipv4.ecn == 1 || hdr.ipv4.ecn == 2){ + if (standard_metadata.enq_qdepth >= ECN_THRESHOLD){ + mark_ecn(); + } + } + } +} + +/************************************************************************* +************* C H E C K S U M C O M P U T A T I O N ************** +*************************************************************************/ + +control MyComputeChecksum(inout headers hdr, inout metadata meta) { + apply { + update_checksum( + hdr.ipv4.isValid(), + { hdr.ipv4.version, + hdr.ipv4.ihl, + hdr.ipv4.diffserv, + hdr.ipv4.ecn, + hdr.ipv4.totalLen, + hdr.ipv4.identification, + hdr.ipv4.flags, + hdr.ipv4.fragOffset, + hdr.ipv4.ttl, + hdr.ipv4.protocol, + hdr.ipv4.srcAddr, + hdr.ipv4.dstAddr }, + hdr.ipv4.hdrChecksum, + HashAlgorithm.csum16); + } +} + +/************************************************************************* +*********************** D E P A R S E R ******************************* +*************************************************************************/ + +control MyDeparser(packet_out packet, in headers hdr) { + apply { + packet.emit(hdr.ethernet); + packet.emit(hdr.ipv4); + } +} + +/************************************************************************* +*********************** S W I T C H ******************************* +*************************************************************************/ + +V1Switch( +MyParser(), +MyVerifyChecksum(), +MyIngress(), +MyEgress(), +MyComputeChecksum(), +MyDeparser() +) main; \ No newline at end of file diff --git a/SIGCOMM_2017/exercises/hula/README.md b/SIGCOMM_2017/exercises/hula/README.md new file mode 100644 index 0000000..b4e5c3e --- /dev/null +++ b/SIGCOMM_2017/exercises/hula/README.md @@ -0,0 +1,300 @@ + +# Implementing HULA + +## Introduction + +The objective of this exercise is to implement a simplified version of +[HULA](http://web.mit.edu/anirudh/www/hula-sosr16.pdf). +In contrast to ECMP, which selects the next hop randomly, HULA load balances +the flows over multiple paths to a destination ToR based on queue occupancy +of switches in each path. Thus, it can use the whole bisection bandwidth. +To keep the example simple, we implement it on top of source routing exercise. + +Here is how HULA works: +- Each ToR switch generates a HULA packet to each other ToR switch + to probe the condition of every path between the source and the destination ToR. + Each HULA packet is forwarded to the destination ToR (forward path), collects the maximum + queue length it observes while being forwarded, and finally delivers that information + to the destination ToR. Based on the congestion information collected via probes, + each destination ToR then can maintain the current best path (i.e., least congested path) + from each source ToR. To share the best path information with the source ToRs so that + the sources can use that information for new flows, the destination ToRs notify + source ToRs of the current best path by returning the HULA probe back to the source + ToR (reverse path) only if the current best path changes. The probe packets include + a HULA header and a list of ports for source routing. We describe the elements of HULA header later. +- In the forward path: + - Each hop updates the queue length field in the hula header if the local queue depth observed by + the HULA packet is larger than maximum queue depth recorded in the probe packet. Thus when + the packet reaches the destination ToR, queue length field will be the maximum observed queue length + on the forward path. + - At destination ToR, + 1. find the queue length of current best path from the source ToR. + 2. if the new path is better, update the queue length and best path and return + the HULA probe to the source path. This is done by setting the direction field + in the HULA header and returning the packet to the ingress port. + 3. if the probe came through the current best path, the destination ToR just updates + the existing value. This is needed to know if the best path got worse and hence allow + other paths to replace it later. It is inefficient to save the whole path ID + (i.e., sequence of switch IDs) and compare it in the data plane; + note, P4 doesn't have a loop construct. Instead, we keep a 32 bit digest of a + path in the HULA header. Each destination ToR only saves and compares the + digest of the best path along with its queue length. + The `hula.digest` field is set by source ToR upon creating the HULA packet + and does not change along the path. +- In the reverse path: + - Each hop will update the "routing next hop" to the destination ToR based on the port + it received the HULA packet on (as it was the best path). Then it forwards the packet + to the next hop in reverse path based on source routing. + - Source ToR also drops the packet. +- Now for each data packet, + - Each hop hashes the flow header fields and looks into a "flow table". + - If it doesn't find the next hop for the flow, looks into "routing next hop" to + find the next hop for destination ToR. We assume each ToR serves a /24 IP address. + The switch also updates the "flow table". "flow table" prevents the path of a flow to change + in order to avoid packet re-ordering and path oscilation during updating next hops. + - Otherwise, each hop just uses the next hop. + +Your switch will have multiple tables, which the control plane will +populate with static rules. We have already defined +the control plane rules, so you only need to implement the data plane +logic of your P4 program. + +> **Spoiler alert:** There is a reference solution in the `solution` +> sub-directory. Feel free to compare your implementation to the reference. + + +## Step 1: Run the (incomplete) starter code + +The directory with this README also contains a skeleton P4 program, +`hula.p4`, which initially drops all packets. Your job (in the next +step) will be to extend it to properly update HULA packets and forward data packets. + +Before that, let's compile the incomplete `hula.p4` and bring up a +switch in Mininet to test its behavior. + +1. In your shell, run: + ```bash + ./run.sh + ``` + This will: + * compile `hula.p4`, and + * start a Mininet instance with three ToR switches (`s1`, `s2`, `s3`) + and two spine switches ( `s11`, `s22`). + * The hosts (`h1`, `h2`, `h3`) are assigned IPs of `10.0.1.1`, `10.0.2.2`, and `10.0.3.3`. + +2. You should now see a Mininet command prompt. Just ping `h2` from `h1`: + ```bash + mininet> h1 ping h2 + ``` +It doesn't work as no path is set. + +3. Type `exit` to close the Mininet command line. + +The message was not received because each switch is programmed with +`hula.p4`, which drops all data packets. Your job is to extend +this file. + +### A note about the control plane + +P4 programs define a packet-processing pipeline, but the rules governing packet +processing are inserted into the pipeline by the control plane. When a rule +matches a packet, its action is invoked with parameters supplied by the control +plane as part of the rule. + +In this exercise, the control plane logic has already been implemented. As +part of bringing up the Mininet instance, the `run.sh` script will install +packet-processing rules in the tables of each switch. These are defined in the +`sX-commands.txt` files, where `X` corresponds to the switch number. + +**Important:** A P4 program also defines the interface between the switch +pipeline and control plane. The `sX-commands.txt` files contain lists of +commands for the BMv2 switch API. These commands refer to specific tables, +keys, and actions by name, and any changes in the P4 program that add or rename +tables, keys, or actions will need to be reflected in these command files. + +## Step 2: Implement Hula + +The `hula.p4` file contains a skeleton P4 program with key pieces of +logic replaced by `TODO` comments. These should guide your +implementation---replace each `TODO` with logic implementing the missing piece. + +A complete `hula.p4` will contain the following components: + +1. Header type definitions for Ethernet (`ethernet_t`), Hula (`hula_t`), + Source Routing (`srcRoute_t`), IPv4 (`ipv4_t`), UDP(`udp_t`). +2. Parsers for the above headers. +3. Registers: + - `srcindex_qdepth_reg`: At destination ToR saves queue length of the best path + from each Source ToR + - `srcindex_digest_reg`: At destination ToR saves the digest of the best path + from each Source ToR + - `dstindex_nhop_reg`: At each hop, saves the next hop to reach each destination ToR + - `flow_port_reg`: At each hop saves the next hop for each flow +4. `hula_fwd table`: looks at the destination IP of a HULA packet. If it is the destination ToR, + it runs `hula_dst` action to set `meta.index` field based on source IP (source ToR). + The index is used later to find queue depth and digest of current best path from that source ToR. + Otherwise, this table just runs `srcRoute_nhop` to perform source routing. +5. `hula_bwd` table: at revere path, updates next hop to the destination ToR using `hula_set_nhop` +action. The action updates `dstindex_nhop_reg` register. +6. `hula_src` table checks the source IP address of a HULA packet in reverse path. + if this switch is the source, this is the end of reverse path, thus drop the packet. + Otherwise use `srcRoute_nhop` action to continue source routing in the reverse path. +7. `hula_nhop` table for data packets, reads destination IP/24 to get an index. + It uses the index to read `dstindex_nhop_reg` register and get best next hop to the + destination ToR. +8. dmac table just updates ethernet destination address based on next hop. +9. An apply block that has the following logic: + * If the packet has a HULA header + * In forward path (`hdr.hula.dir==0`): + * Apply `hula_fwd` table to check if it is the destination ToR or not + * If this switch is the destination ToR (`hula_dst` action ran and + set the `meta.index` based on the source IP address): + * read `srcindex_qdepth_reg` for the queue length of + the current best path from the source ToR + * If the new queue length is better, update the entry in `srcindex_qdepth_reg` and + save the path digest in `srcindex_digest_reg`. Then return the HULA packet to the source ToR + by sending to its ingress port and setting `hula.dir=1` (reverse path) + * else, if this HULA packet came through current best path (`hula.digest` is equal to + the value in `srcindex_digest_reg`), update its queue length in `srcindex_qdepth_reg`. + In this case we don't need to send the HULA packet back, thus drop the packet. + * in reverse path (`hdr.hula.dir==1`): + * apply `hula_bwd` to update the HULA next hop to the destination ToR + * apply `hula_src` table to drop the packet if it is the source ToR of the HULA packet + * If it is a data packet + * compute the hash of flow + * **TODO** read nexthop port from `flow_port_reg` into a temporary variable, say `port`. + * **TODO** If no entry found (`port==0`), read next hop by applying `hula_nhop` table. + Then save the value into `flow_port_reg` for later packets. + * **TODO** if it is found, save `port` into `standard_metadata.egress_spec` to finish routing. + * apply `dmac` table to update `ethernet.dstAddr`. This is necessary for the links that send packets + to hosts. Otherwise their NIC will drop packets. + * udpate TTL +5. **TODO:** An egress control that for HULA packets that are in forward path (`hdr.hula.dir==0`) + compares `standard_metadata.deq_qdepth` to `hdr.hula.qdepth` + in order to save the maximum in `hdr.hula.qdepth` +7. A deparser that selects the order in which fields inserted into the outgoing + packet. +8. A `package` instantiation supplied with the parser, control, checksum verification and + recomputation and deparser. + +## Step 3: Run your solution + +1. Run Mininet same as Step 1 + +2. Open a separate terminal, go to `exercises/hula`, and run `sudo ./generatehula.py`. + This python script makes each ToR switch generate one HULA probe for each other ToR and + through each separate forward path. For example, `s1` first probes `s2` via `s11` and then via `s22`. + Then `s1` probes `s3` again first via `s11` and then via `s22`. `s2` does the same thing to probe + paths to `s1` and `s3`, and so does `s3`. + +3. Now run `h1 ping h2`. The ping should work if you have completed the ingress control block in `hula.p4`. +Note at this point, every ToR considers all paths are equal because there isn't any congestion in the network. + +Now we are going to test a more complex scenario. + +We first create two iperf sessions: one from `h1` to `h3`, and the other from `h2` to `h3`. +Since both `s1` and `s2` currently think their best paths to `s3` should go through `s11`, +the two connections will use the same spine switch (`s11`). Note we throttled the +links from the spine switches to `s3` down to 1Mbps. Hence, each of the two connections +achieves only ~512Kbps. Let's confirm this by taking the following steps. + +1. open a terminal window on `h1`, `h2` and `h3`: +```bash +xterm h1 h2 h3 +``` +2. start iperf server at `h3` +```bash +iperf -s -u -i 1 +``` +3. run iperf client at `h1` +```bash +iperf -c 10.0.3.3 -t 30 -u -b 2m +``` +4. run iperf client in `h2`. try to do step 3 and 4 simultaneously. +```bash +iperf -c 10.0.3.3 -t 30 -u -b 2m +``` +While the connections are running, watch the iperf server's output at `h3`. +Although there are two completely non-overlapping paths for `h1` and `h2` to reach `h3`, +both `h1` and `h2` end up using the same spine, and hence the aggregate +throughput of the two connections is capped to 1Mbps. +You can confirm this by watching the performance of each connection. + + +Our goal is allowing the two connections to use two different spine switches and hence achieve +1Mbps each. We can do this by first causing congestion on one of the spines. More specifically +we'll create congestion at the queue in `s11` facing the link `s11-to-s3` by running a +long-running connection (an elephant flow) from `s1` to `s3` through `s11`. +Once the queue builds up due to the elephant, then we'll let `s2` generate HULA probes +several times so that it can learn to avoid forwarding new flows destined to `s3` through `s11`. +The following steps achieve this. + +1. open a terminal window on `h1`, `h2` and `h3`. (By the way, if you have already closed mininet, +you need to re-run the mininet test and run `generatehula.py` first, to setup initial routes) +```bash +xterm h1 h2 h3 +``` +2. start iperf server at `h3` +```bash +iperf -s -u -i 1 +``` +3. create a long-running full-demand connection from `h1` to `h3` through `s11`. +you can do this by running the following at `h1` +```bash +iperf -c 10.0.3.3 -t 3000 -u -b 2m +``` +4. outside mininet (in a separate terminal), go to `exercises/hula`, and run the following several (5 to 10) times +```bash +sudo ./generatehula.py +``` +This should let `s2` know that the path through `s11` to `s3` is congested and +the best path is now through the uncongested spine, `s22`. +5. Now, run iperf client at `h2` +```bash +iperf -c 10.0.3.3 -t 30 -u -b 2m +``` +You will be able to confirm both iperf sessions achieve 1Mbps because they go through two different spines. + +### Food for thought +* how can we implement flowlet routing (as opposed to flow routing) say based on the timestamp of packets +* in the ingress control logic, the destination ToR always sends a HULA packet +back on the reverse path if the queue length is better. But this is not necessary +if it came from the best path. Can you improve the code? +* the hula packets on the congested path may get dropped or extremely delayed, +thus the destination ToR would not be aware of the worsened condition of the current best path. +A solution could be that the destination ToR uses a timeout mechanism to ignore the current best path +if it doesn't receive a hula packet through it for a long time. +How can you implement this inside dataplane? + +### Troubleshooting + +There are several ways that problems might manifest: + +1. `hula.p4` fails to compile. In this case, `run.sh` will report the +error emitted from the compiler and stop. + +2. `hula.p4` compiles but does not support the control plane rules in +the `sX-commands.txt` files that `run.sh` tries to install using the BMv2 CLI. +In this case, `run.sh` will report these errors to `stderr`. Use these error +messages to fix your `hula.p4` implementation. + +3. `hula.p4` compiles, and the control plane rules are installed, but +the switch does not process packets in the desired way. The +`build/logs/.log` files contain trace messages describing how each +switch processes each packet. The output is detailed and can help pinpoint +logic errors in your implementation. +The `build/-.pcap` also contains the pcap of packets on each +interface. Use `tcpdump -r -xxx` to print the hexdump of the packets. + +#### Cleaning up Mininet + +In the latter two cases above, `run.sh` may leave a Mininet instance running in +the background. Use the following command to clean up these instances: + +```bash +mn -c +``` + +## Next Steps + +Congratulations, your implementation works! diff --git a/SIGCOMM_2017/exercises/hula/generatehula.py b/SIGCOMM_2017/exercises/hula/generatehula.py new file mode 100755 index 0000000..0acacf5 --- /dev/null +++ b/SIGCOMM_2017/exercises/hula/generatehula.py @@ -0,0 +1,82 @@ +#!/usr/bin/env python +import argparse +import sys +import socket +import random +import struct + +from scapy.all import sendp, send, get_if_list, get_if_hwaddr, bind_layers +from scapy.all import Packet +from scapy.all import Ether, IP, UDP +from scapy.fields import * +from time import sleep +import crcmod + +class Hula(Packet): + fields_desc = [ BitField("dir", 0, 1), + BitField("qdepth", 0, 15), + XIntField("digest", None)] + def post_build(self, p, pay): + p += pay + if self.digest is None: + crc32 = crcmod.Crc(0x104c11db7, initCrc=0, xorOut=0xFFFFFFFF) + crc32.update(str(p)); + c = bytes(bytearray.fromhex("%08x" % crc32.crcValue)) + p = p[:2]+ c +p[6:] + #ck = checksum(p) + #p = p[:2]+"\x00\x00"+chr(ck>>8)+chr(ck&0xff)+p[6:] + return p + +class SourceRoute(Packet): + fields_desc = [ BitField("bos", 0, 1), + BitField("port", 0, 15)] + +bind_layers(Ether, Hula, type=0x2345) +bind_layers(Hula, SourceRoute) +bind_layers(SourceRoute, SourceRoute, bos=0) +bind_layers(SourceRoute, IP, bos=1) + +def main(): + period = 0 + if len(sys.argv) > 1: + period = int(sys.argv[1]) + + # src, dst , src routing , interface + info = [ + ("10.0.1.0", "10.0.2.0", (2, 2, 1, 1), "s1-eth1"), + ("10.0.1.0", "10.0.2.0", (3, 2, 1, 1), "s1-eth1"), + ("10.0.1.0", "10.0.3.0", (2, 3, 1, 1), "s1-eth1"), + ("10.0.1.0", "10.0.3.0", (3, 3, 1, 1), "s1-eth1"), + ("10.0.2.0", "10.0.1.0", (2, 1, 2, 1), "s2-eth1"), + ("10.0.2.0", "10.0.1.0", (3, 1, 2, 1), "s2-eth1"), + ("10.0.2.0", "10.0.3.0", (2, 3, 2, 1), "s2-eth1"), + ("10.0.2.0", "10.0.3.0", (3, 3, 2, 1), "s2-eth1"), + ("10.0.3.0", "10.0.1.0", (2, 1, 3, 1), "s3-eth1"), + ("10.0.3.0", "10.0.1.0", (3, 1, 3, 1), "s3-eth1"), + ("10.0.3.0", "10.0.2.0", (2, 2, 3, 1), "s3-eth1"), + ("10.0.3.0", "10.0.2.0", (3, 2, 3, 1), "s3-eth1")] + + + try: + while True: + for e in info: + ports = e[2] + pkt = Ether(src=get_if_hwaddr(e[3]), dst='ff:ff:ff:ff:ff:ff') + pkt = pkt / Hula(dir=0, qdepth=0) + pkt = pkt / SourceRoute(bos=0, port=ports[0]) + pkt = pkt / SourceRoute(bos=0, port=ports[1]) + pkt = pkt / SourceRoute(bos=0, port=ports[2]) + pkt = pkt / SourceRoute(bos=1, port=ports[3]) + pkt = pkt / IP(dst=e[1], src=e[0]) / UDP(dport=4321, sport=1234) + #pkt.show2() + sendp(pkt, iface=e[3], verbose=False) + if period == 0: + break; + else: + sleep(period) + except KeyboardInterrupt: + raise + + +if __name__ == '__main__': + main() diff --git a/SIGCOMM_2017/exercises/hula/hula.p4 b/SIGCOMM_2017/exercises/hula/hula.p4 new file mode 100644 index 0000000..7ac4f49 --- /dev/null +++ b/SIGCOMM_2017/exercises/hula/hula.p4 @@ -0,0 +1,433 @@ +/* -*- P4_16 -*- */ +#include +#include + +const bit<16> TYPE_IPV4 = 0x800; +const bit<16> TYPE_HULA = 0x2345; + +#define MAX_HOPS 9 +#define TOR_NUM 32 +#define TOR_NUM_1 33 + +/************************************************************************* +*********************** H E A D E R S *********************************** +*************************************************************************/ + +typedef bit<9> egressSpec_t; +typedef bit<48> macAddr_t; +typedef bit<32> ip4Addr_t; +typedef bit<15> qdepth_t; +typedef bit<32> digest_t; + +header ethernet_t { + macAddr_t dstAddr; + macAddr_t srcAddr; + bit<16> etherType; +} + +header srcRoute_t { + bit<1> bos; + bit<15> port; +} + +header hula_t { + /* 0 is forward path, 1 is the backward path */ + bit<1> dir; + /* max qdepth seen so far in the forward path */ + qdepth_t qdepth; + /* digest of the source routing list to uniquely identify each path */ + digest_t digest; +} + +header ipv4_t { + bit<4> version; + bit<4> ihl; + bit<8> diffserv; + bit<16> totalLen; + bit<16> identification; + bit<3> flags; + bit<13> fragOffset; + bit<8> ttl; + bit<8> protocol; + bit<16> hdrChecksum; + ip4Addr_t srcAddr; + ip4Addr_t dstAddr; +} + +header udp_t { + bit<16> srcPort; + bit<16> dstPort; + bit<16> length_; + bit<16> checksum; +} + +struct metadata { + /* At destination ToR, this is the index of register + that saves qdepth for the best path from each source ToR */ + bit<32> index; +} + +struct headers { + ethernet_t ethernet; + srcRoute_t[MAX_HOPS] srcRoutes; + ipv4_t ipv4; + udp_t udp; + hula_t hula; +} + +/************************************************************************* +*********************** P A R S E R *********************************** +*************************************************************************/ + +parser MyParser(packet_in packet, + out headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + + state start { + transition parse_ethernet; + } + + state parse_ethernet { + packet.extract(hdr.ethernet); + transition select(hdr.ethernet.etherType) { + TYPE_HULA : parse_hula; + TYPE_IPV4 : parse_ipv4; + default : accept; + } + } + + state parse_hula { + packet.extract(hdr.hula); + transition parse_srcRouting; + } + + state parse_srcRouting { + packet.extract(hdr.srcRoutes.next); + transition select(hdr.srcRoutes.last.bos) { + 1 : parse_ipv4; + default : parse_srcRouting; + } + } + + state parse_ipv4 { + packet.extract(hdr.ipv4); + transition select(hdr.ipv4.protocol) { + 8w17: parse_udp; + default: accept; + } + } + + state parse_udp { + packet.extract(hdr.udp); + transition accept; + } + +} + + +/************************************************************************* +************ C H E C K S U M V E R I F I C A T I O N ************* +*************************************************************************/ + +control MyVerifyChecksum(in headers hdr, inout metadata meta) { + apply { } +} + + +/************************************************************************* +************** I N G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyIngress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + + /* At destination ToR, saves the queue depth of the best path from + * each source ToR + */ + register(TOR_NUM) srcindex_qdepth_reg; + + /* At destination ToR, saves the digest of the best path from + * each source ToR + */ + register(TOR_NUM) srcindex_digest_reg; + + /* At each hop, saves the next hop to reach each destination ToR */ + register>(TOR_NUM) dstindex_nhop_reg; + + /* At each hop saves the next hop for each flow */ + register>(65536) flow_port_reg; + + /* This action will drop packets */ + action drop() { + mark_to_drop(); + } + + action nop() { + } + + action update_ttl(){ + hdr.ipv4.ttl = hdr.ipv4.ttl - 1; + } + + action set_dmac(macAddr_t dstAddr){ + hdr.ethernet.srcAddr = hdr.ethernet.dstAddr; + hdr.ethernet.dstAddr = dstAddr; + } + + /* This action just applies source routing */ + action srcRoute_nhop() { + standard_metadata.egress_spec = (bit<9>)hdr.srcRoutes[0].port; + hdr.srcRoutes.pop_front(1); + } + + /* Runs if it is the destination ToR. + * Control plane Gives the index of register for best path from a source ToR + */ + action hula_dst(bit<32> index) { + meta.index = index; + } + + /* On reverse path, update nexthop to a destination ToR to the ingress port + * where we receive hula packet + */ + action hula_set_nhop(bit<32> index) { + dstindex_nhop_reg.write(index, (bit<16>)standard_metadata.ingress_port); + } + + /* Read next hop that is saved in hula_set_nhop action for data packets */ + action hula_get_nhop(bit<32> index){ + bit<16> tmp; + dstindex_nhop_reg.read(tmp, index); + standard_metadata.egress_spec = (bit<9>)tmp; + } + + /* Record best path at destination ToR */ + action change_best_path_at_dst(){ + srcindex_qdepth_reg.write(meta.index, hdr.hula.qdepth); + srcindex_digest_reg.write(meta.index, hdr.hula.digest); + } + + /* At destination ToR, return packet to source by + * - changing its hula direction + * - send it to the port it came from + */ + action return_hula_to_src(){ + hdr.hula.dir = 1; + standard_metadata.egress_spec = standard_metadata.ingress_port; + } + + /* On forward path: + * - if destination ToR: run hula_dst to set the index based on srcAddr + * - otherwise run srcRoute_nhop to perform source routing + */ + table hula_fwd { + key = { + hdr.ipv4.dstAddr: exact; + hdr.ipv4.srcAddr: exact; + } + actions = { + hula_dst; + srcRoute_nhop; + } + default_action = srcRoute_nhop; + size = TOR_NUM_1; // TOR_NUM + 1 + } + + /* At each hop in reverse path + * update next hop to destination ToR in registers. + * index is set based on dstAddr + */ + table hula_bwd { + key = { + hdr.ipv4.dstAddr: lpm; + } + actions = { + hula_set_nhop; + } + size = TOR_NUM; + } + + /* On reverse path: + * - if source ToR (srcAddr = this switch) drop hula packet + * - otherwise, just forward in the reverse path based on source routing + */ + table hula_src { + key = { + hdr.ipv4.srcAddr: exact; + } + actions = { + drop; + srcRoute_nhop; + } + default_action = srcRoute_nhop; + size = 2; + } + + /* Get nexthop based on dstAddr using registers */ + table hula_nhop { + key = { + hdr.ipv4.dstAddr: lpm; + } + actions = { + hula_get_nhop; + drop; + } + default_action = drop; + size = TOR_NUM; + } + + /* Set right dmac for packets going to hosts */ + table dmac { + key = { + standard_metadata.egress_spec : exact; + } + actions = { + set_dmac; + nop; + } + default_action = nop; + size = 16; + } + + apply { + if (hdr.hula.isValid()){ + if (hdr.hula.dir == 0){ + switch(hula_fwd.apply().action_run){ + + /* if hula_dst action ran, this is the destination ToR */ + hula_dst: { + + /* if it is the destination ToR compare qdepth */ + qdepth_t old_qdepth; + srcindex_qdepth_reg.read(old_qdepth, meta.index); + + if (old_qdepth > hdr.hula.qdepth){ + change_best_path_at_dst(); + + /* only return hula packets that update best path */ + return_hula_to_src(); + }else{ + + /* update the best path even if it has gone worse + * so that other paths can replace it later + */ + digest_t old_digest; + srcindex_digest_reg.read(old_digest, meta.index); + if (old_digest == hdr.hula.digest){ + srcindex_qdepth_reg.write(meta.index, hdr.hula.qdepth); + } + + drop(); + } + } + } + }else { + /* update routing table in reverse path */ + hula_bwd.apply(); + + /* drop if source ToR */ + hula_src.apply(); + } + + }else if (hdr.ipv4.isValid()){ + bit<16> flow_hash; + hash( + flow_hash, + HashAlgorithm.crc16, + 16w0, + { hdr.ipv4.srcAddr, hdr.ipv4.dstAddr, hdr.udp.srcPort}, + 32w65536); + + /* TODO: + * - Remove drop(); + * - Read nexthop port from flow_port_reg for the flow + * using flow_hash into a temporary variable + * - if port==0, + * - apply hula_nhop table to get next hop for destination ToR + * - write the next hop into the flow_port_reg register indexed by flow_hash + * - else: write port into standard_metadata.egress_spec + */ + drop(); + + /* set the right dmac so that ping and iperf work */ + dmac.apply(); + }else { + drop(); + } + + if (hdr.ipv4.isValid()){ + update_ttl(); + } + } +} + +/************************************************************************* +**************** E G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyEgress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + apply { + /* TODO: + * if hula header is valid and this is forward path (hdr.hula.dir==0) + * check whether the qdepth in hula is smaller than + * (qdepth_t)standard_metadata.deq_qdepth + * if so, then update hdr.hula.qdepth + */ + } +} + +/************************************************************************* +************* C H E C K S U M C O M P U T A T I O N ************** +*************************************************************************/ + +control MyComputeChecksum(inout headers hdr, inout metadata meta) { + apply { + update_checksum( + hdr.ipv4.isValid(), + { hdr.ipv4.version, + hdr.ipv4.ihl, + hdr.ipv4.diffserv, + hdr.ipv4.totalLen, + hdr.ipv4.identification, + hdr.ipv4.flags, + hdr.ipv4.fragOffset, + hdr.ipv4.ttl, + hdr.ipv4.protocol, + hdr.ipv4.srcAddr, + hdr.ipv4.dstAddr }, + hdr.ipv4.hdrChecksum, + HashAlgorithm.csum16); + } +} + + +/************************************************************************* +*********************** D E P A R S E R ******************************* +*************************************************************************/ + +control MyDeparser(packet_out packet, in headers hdr) { + apply { + packet.emit(hdr.ethernet); + packet.emit(hdr.hula); + packet.emit(hdr.srcRoutes); + packet.emit(hdr.ipv4); + packet.emit(hdr.udp); + } +} + +/************************************************************************* +*********************** S W I T C H ******************************* +*************************************************************************/ + +V1Switch( +MyParser(), +MyVerifyChecksum(), +MyIngress(), +MyEgress(), +MyComputeChecksum(), +MyDeparser() +) main; \ No newline at end of file diff --git a/SIGCOMM_2017/exercises/hula/p4app.json b/SIGCOMM_2017/exercises/hula/p4app.json new file mode 100644 index 0000000..c5c6562 --- /dev/null +++ b/SIGCOMM_2017/exercises/hula/p4app.json @@ -0,0 +1,39 @@ +{ + "program": "hula.p4", + "language": "p4-16", + "targets": { + "multiswitch": { + "auto-control-plane": true, + "cli": true, + "pcap_dump": true, + "bmv2_log": true, + "links": [["h1", "s1"], ["h2", "s2"], ["h3", "s3"], ["s1", "s11"], ["s1", "s22"], ["s2", "s11"], ["s2", "s22"], ["s11", "s3", "0", 1], ["s22", "s3", "0", 1]], + "hosts": { + "h1": { + }, + "h2": { + }, + "h3": { + } + }, + "switches": { + "s1": { + "entries": "s1-commands.txt" + }, + "s2": { + "entries": "s2-commands.txt" + }, + "s3": { + "entries": "s3-commands.txt" + }, + "s11": { + "entries": "s11-commands.txt" + }, + "s22": { + "entries": "s22-commands.txt" + } + + } + } + } +} diff --git a/SIGCOMM_2017/exercises/hula/run.sh b/SIGCOMM_2017/exercises/hula/run.sh new file mode 100755 index 0000000..d5c1947 --- /dev/null +++ b/SIGCOMM_2017/exercises/hula/run.sh @@ -0,0 +1,5 @@ +P4APPRUNNER=../../utils/p4apprunner.py +mkdir -p build +tar -czf build/p4app.tgz * --exclude='build' +#cd build +sudo python $P4APPRUNNER p4app.tgz --build-dir ./build diff --git a/SIGCOMM_2017/exercises/hula/s1-commands.txt b/SIGCOMM_2017/exercises/hula/s1-commands.txt new file mode 100644 index 0000000..2ca3bf8 --- /dev/null +++ b/SIGCOMM_2017/exercises/hula/s1-commands.txt @@ -0,0 +1,16 @@ +table_add hula_src drop 10.0.1.0 => +register_write dstindex_nhop_reg 0 1 +table_add hula_fwd hula_dst 10.0.1.0 10.0.1.0 => 0 +table_add hula_fwd hula_dst 10.0.1.0 10.0.2.0 => 1 +table_add hula_fwd hula_dst 10.0.1.0 10.0.3.0 => 2 +table_add dmac set_dmac 1 => 00:00:00:00:01:01 + +register_write srcindex_qdepth_reg 0 256 +register_write srcindex_qdepth_reg 1 256 +register_write srcindex_qdepth_reg 2 256 +table_add hula_bwd hula_set_nhop 10.0.1.0/24 => 0 +table_add hula_bwd hula_set_nhop 10.0.2.0/24 => 1 +table_add hula_bwd hula_set_nhop 10.0.3.0/24 => 2 +table_add hula_nhop hula_get_nhop 10.0.1.0/24 => 0 +table_add hula_nhop hula_get_nhop 10.0.2.0/24 => 1 +table_add hula_nhop hula_get_nhop 10.0.3.0/24 => 2 diff --git a/SIGCOMM_2017/exercises/hula/s11-commands.txt b/SIGCOMM_2017/exercises/hula/s11-commands.txt new file mode 100644 index 0000000..f63d939 --- /dev/null +++ b/SIGCOMM_2017/exercises/hula/s11-commands.txt @@ -0,0 +1,6 @@ +table_add hula_bwd hula_set_nhop 10.0.1.0/24 => 0 +table_add hula_bwd hula_set_nhop 10.0.2.0/24 => 1 +table_add hula_bwd hula_set_nhop 10.0.3.0/24 => 2 +table_add hula_nhop hula_get_nhop 10.0.1.0/24 => 0 +table_add hula_nhop hula_get_nhop 10.0.2.0/24 => 1 +table_add hula_nhop hula_get_nhop 10.0.3.0/24 => 2 diff --git a/SIGCOMM_2017/exercises/hula/s2-commands.txt b/SIGCOMM_2017/exercises/hula/s2-commands.txt new file mode 100644 index 0000000..b5508f6 --- /dev/null +++ b/SIGCOMM_2017/exercises/hula/s2-commands.txt @@ -0,0 +1,16 @@ +table_add hula_src drop 10.0.2.0 => +register_write dstindex_nhop_reg 1 1 +table_add hula_fwd hula_dst 10.0.2.0 10.0.1.0 => 0 +table_add hula_fwd hula_dst 10.0.2.0 10.0.2.0 => 1 +table_add hula_fwd hula_dst 10.0.2.0 10.0.3.0 => 2 +table_add dmac set_dmac 1 => 00:00:00:00:02:02 + +register_write srcindex_qdepth_reg 0 256 +register_write srcindex_qdepth_reg 1 256 +register_write srcindex_qdepth_reg 2 256 +table_add hula_bwd hula_set_nhop 10.0.1.0/24 => 0 +table_add hula_bwd hula_set_nhop 10.0.2.0/24 => 1 +table_add hula_bwd hula_set_nhop 10.0.3.0/24 => 2 +table_add hula_nhop hula_get_nhop 10.0.1.0/24 => 0 +table_add hula_nhop hula_get_nhop 10.0.2.0/24 => 1 +table_add hula_nhop hula_get_nhop 10.0.3.0/24 => 2 diff --git a/SIGCOMM_2017/exercises/hula/s22-commands.txt b/SIGCOMM_2017/exercises/hula/s22-commands.txt new file mode 100644 index 0000000..f63d939 --- /dev/null +++ b/SIGCOMM_2017/exercises/hula/s22-commands.txt @@ -0,0 +1,6 @@ +table_add hula_bwd hula_set_nhop 10.0.1.0/24 => 0 +table_add hula_bwd hula_set_nhop 10.0.2.0/24 => 1 +table_add hula_bwd hula_set_nhop 10.0.3.0/24 => 2 +table_add hula_nhop hula_get_nhop 10.0.1.0/24 => 0 +table_add hula_nhop hula_get_nhop 10.0.2.0/24 => 1 +table_add hula_nhop hula_get_nhop 10.0.3.0/24 => 2 diff --git a/SIGCOMM_2017/exercises/hula/s3-commands.txt b/SIGCOMM_2017/exercises/hula/s3-commands.txt new file mode 100644 index 0000000..5600d49 --- /dev/null +++ b/SIGCOMM_2017/exercises/hula/s3-commands.txt @@ -0,0 +1,16 @@ +table_add hula_src drop 10.0.3.0 => +register_write dstindex_nhop_reg 2 1 +table_add hula_fwd hula_dst 10.0.3.0 10.0.1.0 => 0 +table_add hula_fwd hula_dst 10.0.3.0 10.0.2.0 => 1 +table_add hula_fwd hula_dst 10.0.3.0 10.0.3.0 => 2 +table_add dmac set_dmac 1 => 00:00:00:00:03:03 + +register_write srcindex_qdepth_reg 0 256 +register_write srcindex_qdepth_reg 1 256 +register_write srcindex_qdepth_reg 2 256 +table_add hula_bwd hula_set_nhop 10.0.1.0/24 => 0 +table_add hula_bwd hula_set_nhop 10.0.2.0/24 => 1 +table_add hula_bwd hula_set_nhop 10.0.3.0/24 => 2 +table_add hula_nhop hula_get_nhop 10.0.1.0/24 => 0 +table_add hula_nhop hula_get_nhop 10.0.2.0/24 => 1 +table_add hula_nhop hula_get_nhop 10.0.3.0/24 => 2 diff --git a/SIGCOMM_2017/exercises/hula/solution/hula.p4 b/SIGCOMM_2017/exercises/hula/solution/hula.p4 new file mode 100644 index 0000000..06f4ae6 --- /dev/null +++ b/SIGCOMM_2017/exercises/hula/solution/hula.p4 @@ -0,0 +1,449 @@ +/* -*- P4_16 -*- */ +#include +#include + +const bit<16> TYPE_IPV4 = 0x800; +const bit<16> TYPE_HULA = 0x2345; + +#define MAX_HOPS 9 +#define TOR_NUM 32 +#define TOR_NUM_1 33 + +/************************************************************************* +*********************** H E A D E R S *********************************** +*************************************************************************/ + +typedef bit<9> egressSpec_t; +typedef bit<48> macAddr_t; +typedef bit<32> ip4Addr_t; +typedef bit<15> qdepth_t; +typedef bit<32> digest_t; + +header ethernet_t { + macAddr_t dstAddr; + macAddr_t srcAddr; + bit<16> etherType; +} + +header srcRoute_t { + bit<1> bos; + bit<15> port; +} + +header hula_t { + /* 0 is forward path, 1 is the backward path */ + bit<1> dir; + /* max qdepth seen so far in the forward path */ + qdepth_t qdepth; + /* digest of the source routing list to uniquely identify each path */ + digest_t digest; +} + +header ipv4_t { + bit<4> version; + bit<4> ihl; + bit<8> diffserv; + bit<16> totalLen; + bit<16> identification; + bit<3> flags; + bit<13> fragOffset; + bit<8> ttl; + bit<8> protocol; + bit<16> hdrChecksum; + ip4Addr_t srcAddr; + ip4Addr_t dstAddr; +} + +header udp_t { + bit<16> srcPort; + bit<16> dstPort; + bit<16> length_; + bit<16> checksum; +} + +struct metadata { + /* At destination ToR, this is the index of register + that saves qdepth for the best path from each source ToR */ + bit<32> index; +} + +struct headers { + ethernet_t ethernet; + srcRoute_t[MAX_HOPS] srcRoutes; + ipv4_t ipv4; + udp_t udp; + hula_t hula; +} + +/************************************************************************* +*********************** P A R S E R *********************************** +*************************************************************************/ + +parser MyParser(packet_in packet, + out headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + + state start { + transition parse_ethernet; + } + + state parse_ethernet { + packet.extract(hdr.ethernet); + transition select(hdr.ethernet.etherType) { + TYPE_HULA : parse_hula; + TYPE_IPV4 : parse_ipv4; + default : accept; + } + } + + state parse_hula { + packet.extract(hdr.hula); + transition parse_srcRouting; + } + + state parse_srcRouting { + packet.extract(hdr.srcRoutes.next); + transition select(hdr.srcRoutes.last.bos) { + 1 : parse_ipv4; + default : parse_srcRouting; + } + } + + state parse_ipv4 { + packet.extract(hdr.ipv4); + transition select(hdr.ipv4.protocol) { + 8w17: parse_udp; + default: accept; + } + } + + state parse_udp { + packet.extract(hdr.udp); + transition accept; + } + +} + + +/************************************************************************* +************ C H E C K S U M V E R I F I C A T I O N ************* +*************************************************************************/ + +control MyVerifyChecksum(in headers hdr, inout metadata meta) { + apply { } +} + + +/************************************************************************* +************** I N G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyIngress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + + /* + * At destination ToR, saves the queue depth of the best path from + * each source ToR + */ + register(TOR_NUM) srcindex_qdepth_reg; + + /* + * At destination ToR, saves the digest of the best path from + * each source ToR + */ + register(TOR_NUM) srcindex_digest_reg; + + /* At each hop, saves the next hop to reach each destination ToR */ + register>(TOR_NUM) dstindex_nhop_reg; + + /* At each hop saves the next hop for each flow */ + register>(65536) flow_port_reg; + + /* This action will drop packets */ + action drop() { + mark_to_drop(); + } + + action nop() { + } + + action update_ttl(){ + hdr.ipv4.ttl = hdr.ipv4.ttl - 1; + } + + action set_dmac(macAddr_t dstAddr){ + hdr.ethernet.srcAddr = hdr.ethernet.dstAddr; + hdr.ethernet.dstAddr = dstAddr; + } + + /* This action just applies source routing */ + action srcRoute_nhop() { + standard_metadata.egress_spec = (bit<9>)hdr.srcRoutes[0].port; + hdr.srcRoutes.pop_front(1); + } + + /* + * Runs if it is the destination ToR. + * Control plane Gives the index of register for best path from source ToR + */ + action hula_dst(bit<32> index) { + meta.index = index; + } + + /* + * In reverse path, update nexthop to a destination ToR to ingress port + * where we receive hula packet + */ + action hula_set_nhop(bit<32> index) { + dstindex_nhop_reg.write(index, (bit<16>)standard_metadata.ingress_port); + } + + /* Read next hop that is saved in hula_set_nhop action for data packets */ + action hula_get_nhop(bit<32> index){ + bit<16> tmp; + dstindex_nhop_reg.read(tmp, index); + standard_metadata.egress_spec = (bit<9>)tmp; + } + + /* Record best path at destination ToR */ + action change_best_path_at_dst(){ + srcindex_qdepth_reg.write(meta.index, hdr.hula.qdepth); + srcindex_digest_reg.write(meta.index, hdr.hula.digest); + } + + /* + * At destination ToR, return packet to source by + * - changing its hula direction + * - send it to the port it came from + */ + action return_hula_to_src(){ + hdr.hula.dir = 1; + standard_metadata.egress_spec = standard_metadata.ingress_port; + } + + /* + * In forward path: + * - if destination ToR: run hula_dst to set the index based on srcAddr + * - otherwise run srcRoute_nhop to perform source routing + */ + table hula_fwd { + key = { + hdr.ipv4.dstAddr: exact; + hdr.ipv4.srcAddr: exact; + } + actions = { + hula_dst; + srcRoute_nhop; + } + default_action = srcRoute_nhop; + size = TOR_NUM_1; // TOR_NUM + 1 + } + + /* + * At each hop in reverse path + * update next hop to destination ToR in registers. + * index is set based on dstAddr + */ + table hula_bwd { + key = { + hdr.ipv4.dstAddr: lpm; + } + actions = { + hula_set_nhop; + } + size = TOR_NUM; + } + + /* + * in reverse path: + * - if source ToR (srcAddr = this switch) drop hula packet + * - otherwise, just forward in the reverse path based on source routing + */ + table hula_src { + key = { + hdr.ipv4.srcAddr: exact; + } + actions = { + drop; + srcRoute_nhop; + } + default_action = srcRoute_nhop; + size = 2; + } + + /* + * get nexthop based on dstAddr using registers + */ + table hula_nhop { + key = { + hdr.ipv4.dstAddr: lpm; + } + actions = { + hula_get_nhop; + drop; + } + default_action = drop; + size = TOR_NUM; + } + + /* + * set right dmac for packets going to hosts + */ + table dmac { + key = { + standard_metadata.egress_spec : exact; + } + actions = { + set_dmac; + nop; + } + default_action = nop; + size = 16; + } + + apply { + if (hdr.hula.isValid()){ + if (hdr.hula.dir == 0){ + switch(hula_fwd.apply().action_run){ + + /* if hula_dst action ran, this is the destination ToR */ + hula_dst: { + + /* if it is the destination ToR compare qdepth */ + qdepth_t old_qdepth; + srcindex_qdepth_reg.read(old_qdepth, meta.index); + + if (old_qdepth > hdr.hula.qdepth){ + change_best_path_at_dst(); + + /* only return hula packets that update best path */ + return_hula_to_src(); + }else{ + + /* update the best path even if it has gone worse + * so that other paths can replace it later + */ + digest_t old_digest; + srcindex_digest_reg.read(old_digest, meta.index); + if (old_digest == hdr.hula.digest){ + srcindex_qdepth_reg.write(meta.index, hdr.hula.qdepth); + } + + drop(); + } + } + } + }else { + /* update routing table in reverse path */ + hula_bwd.apply(); + + /* drop if source ToR */ + hula_src.apply(); + } + + }else if (hdr.ipv4.isValid()){ + bit<16> flow_hash; + hash( + flow_hash, + HashAlgorithm.crc16, + 16w0, + { hdr.ipv4.srcAddr, hdr.ipv4.dstAddr, hdr.udp.srcPort}, + 32w65536); + + /* look into hula tables */ + bit<16> port; + flow_port_reg.read(port, (bit<32>)flow_hash); + + if (port == 0){ + /* if it is a new flow check hula paths */ + hula_nhop.apply(); + flow_port_reg.write((bit<32>)flow_hash, (bit<16>)standard_metadata.egress_spec); + }else{ + /* old flows still use old path to avoid oscilation and packet reordering */ + standard_metadata.egress_spec = (bit<9>)port; + } + + /* set the right dmac so that ping and iperf work */ + dmac.apply(); + }else { + drop(); + } + + if (hdr.ipv4.isValid()){ + update_ttl(); + } + } +} + +/************************************************************************* +**************** E G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyEgress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + apply { + if (hdr.hula.isValid() && hdr.hula.dir == 0){ + + /* pick max qdepth in hula forward path */ + if (hdr.hula.qdepth < (qdepth_t)standard_metadata.deq_qdepth){ + + /* update queue length */ + hdr.hula.qdepth = (qdepth_t)standard_metadata.deq_qdepth; + } + } + } +} + +/************************************************************************* +************* C H E C K S U M C O M P U T A T I O N ************** +*************************************************************************/ + +control MyComputeChecksum(inout headers hdr, inout metadata meta) { + apply { + update_checksum( + hdr.ipv4.isValid(), + { hdr.ipv4.version, + hdr.ipv4.ihl, + hdr.ipv4.diffserv, + hdr.ipv4.totalLen, + hdr.ipv4.identification, + hdr.ipv4.flags, + hdr.ipv4.fragOffset, + hdr.ipv4.ttl, + hdr.ipv4.protocol, + hdr.ipv4.srcAddr, + hdr.ipv4.dstAddr }, + hdr.ipv4.hdrChecksum, + HashAlgorithm.csum16); + } +} + +/************************************************************************* +*********************** D E P A R S E R ******************************* +*************************************************************************/ + +control MyDeparser(packet_out packet, in headers hdr) { + apply { + packet.emit(hdr.ethernet); + packet.emit(hdr.hula); + packet.emit(hdr.srcRoutes); + packet.emit(hdr.ipv4); + packet.emit(hdr.udp); + } +} + +/************************************************************************* +*********************** S W I T C H ******************************* +*************************************************************************/ + +V1Switch( +MyParser(), +MyVerifyChecksum(), +MyIngress(), +MyEgress(), +MyComputeChecksum(), +MyDeparser() +) main; diff --git a/SIGCOMM_2017/exercises/load_balance/README.md b/SIGCOMM_2017/exercises/load_balance/README.md new file mode 100644 index 0000000..1ff40fb --- /dev/null +++ b/SIGCOMM_2017/exercises/load_balance/README.md @@ -0,0 +1,143 @@ +# Load Balancing + +In this exercise, you will implement a form of load balancing based on +a single version of Equal-Cost Multipath Forwarding. The switch you +will implement will use two tables to forward packets to one of two +destination hosts at random. The first table will use a hash function +(applied to a 5-tuple consisting of the source and destination +Ethernet addresses, source and destination IP addresses, and IP +protocol) to select one of two hosts. The second table will use the +computed hash value to forward the packet to the selected host. + +> **Spoiler alert:** There is a reference solution in the `solution` +> sub-directory. Feel free to compare your implementation to the +> reference. + +## Step 1: Run the (incomplete) starter code + +The directory with this README also contains a skeleton P4 program, +`load_balance.p4`, which initially drops all packets. Your job (in +the next step) will be to extend it to properly forward packets. + +Before that, let's compile the incomplete `load_balance.p4` and bring +up a switch in Mininet to test its behavior. + +1. In your shell, run: + ```bash + ./run.sh + ``` + This will: + * compile `load_balance.p4`, and + * start a Mininet instance with three switches (`s1`, `s2`, `s3`) configured + in a triangle, each connected to one host (`h1`, `h2`, `h3`). + * The hosts are assigned IPs of `10.0.1.1`, `10.0.2.2`, etc. + * We use the IP address 10.0.0.1 to indicate traffic that should be + load balanced between `h2` and `h3`. + +2. You should now see a Mininet command prompt. Open three terminals + for `h1`, `h2` and `h3`, respectively: + ```bash + mininet> xterm h1 h2 h3 + ``` +3. Each host includes a small Python-based messaging client and + server. In `h2` and `h3`'s XTerms, start the servers: + ```bash + ./receive.py + ``` +4. In `h1`'s XTerm, send a message from the client: + ```bash + ./send.py 10.0.0.1 "P4 is cool" + ``` + The message will not be received. +5. Type `exit` to leave each XTerm and the Mininet command line. + +The message was not received because each switch is programmed with +`load_balance.p4`, which drops all packets on arrival. Your job is to +extend this file. + +### A note about the control plane + +P4 programs define a packet-processing pipeline, but the rules +governing packet processing are inserted into the pipeline by the +control plane. When a rule matches a packet, its action is invoked +with parameters supplied by the control plane as part of the rule. + +In this exercise, the control plane logic has already been +implemented. As part of bringing up the Mininet instance, the +`run.sh` script will install packet-processing rules in the tables of +each switch. These are defined in the `s1-commands.txt` file. + +**Important:** A P4 program also defines the interface between the +switch pipeline and control plane. The `s1-commands.txt` file contains +a list of commands for the BMv2 switch API. These commands refer to +specific tables, keys, and actions by name, and any changes in the P4 +program that add or rename tables, keys, or actions will need to be +reflected in these command files. + +## Step 2: Implement Load Balancing + +The `load_balance.p4` file contains a skeleton P4 program with key +pieces of logic replaced by `TODO` comments. These should guide your +implementation---replace each `TODO` with logic implementing the +missing piece. + +A complete `load_balance.p4` will contain the following components: + +1. Header type definitions for Ethernet (`ethernet_t`) and IPv4 (`ipv4_t`). +2. Parsers for Ethernet and IPv4 that populate `ethernet_t` and `ipv4_t` fields. +3. An action to drop a packet, using `mark_to_drop()`. +4. **TODO:** An action (called `set_ecmp_select`), which will: + 1. Hashes the 5-tuple specified above using the `hash` extern + 2. Stores the result in the `meta.ecmp_select` field +5. **TODO:** A control that: + 1. Applies the `ecmp_group` table. + 2. Applies the `ecmp_nhop` table. +6. A deparser that selects the order in which fields inserted into the outgoing + packet. +7. A `package` instantiation supplied with the parser, control, and deparser. + > In general, a package also requires instances of checksum verification + > and recomputation controls. These are not necessary for this tutorial + > and are replaced with instantiations of empty controls. + +## Step 3: Run your solution + +Follow the instructions from Step 1. This time, your message from +`h1` should be delivered to `h2` or `h3`. If you send several +messages, some should be received by each server. + +### Food for thought + + +### Troubleshooting + +There are several ways that problems might manifest: + +1. `load_balance.p4` fails to compile. In this case, `run.sh` will +report the error emitted from the compiler and stop. + +2. `load_balance.p4` compiles but does not support the control plane +rules in the `sX-commands.txt` files that `run.sh` tries to install +using the BMv2 CLI. In this case, `run.sh` will report these errors +to `stderr`. Use these error messages to fix your `load_balance.p4` +implementation. + +3. `load_balance.p4` compiles, and the control plane rules are +installed, but the switch does not process packets in the desired way. +The `build/logs/.log` files contain trace messages +describing how each switch processes each packet. The output is +detailed and can help pinpoint logic errors in your implementation. + +#### Cleaning up Mininet + +In the latter two cases above, `run.sh` may leave a Mininet instance +running in the background. Use the following command to clean up +these instances: + +```bash +mn -c +``` + +## Next Steps + +Congratulations, your implementation works! Move on to the next +exercise: [HULA](../hula). diff --git a/SIGCOMM_2017/exercises/load_balance/load_balance.p4 b/SIGCOMM_2017/exercises/load_balance/load_balance.p4 new file mode 100644 index 0000000..5ded175 --- /dev/null +++ b/SIGCOMM_2017/exercises/load_balance/load_balance.p4 @@ -0,0 +1,218 @@ +/* -*- P4_16 -*- */ +#include +#include + +/************************************************************************* +*********************** H E A D E R S *********************************** +*************************************************************************/ + +header ethernet_t { + bit<48> dstAddr; + bit<48> srcAddr; + bit<16> etherType; +} + +header ipv4_t { + bit<4> version; + bit<4> ihl; + bit<8> diffserv; + bit<16> totalLen; + bit<16> identification; + bit<3> flags; + bit<13> fragOffset; + bit<8> ttl; + bit<8> protocol; + bit<16> hdrChecksum; + bit<32> srcAddr; + bit<32> dstAddr; +} + +header tcp_t { + bit<16> srcPort; + bit<16> dstPort; + bit<32> seqNo; + bit<32> ackNo; + bit<4> dataOffset; + bit<3> res; + bit<3> ecn; + bit<6> ctrl; + bit<16> window; + bit<16> checksum; + bit<16> urgentPtr; +} + +struct metadata { + bit<14> ecmp_select; +} + +struct headers { + ethernet_t ethernet; + ipv4_t ipv4; + tcp_t tcp; +} + +/************************************************************************* +*********************** P A R S E R *********************************** +*************************************************************************/ + +parser MyParser(packet_in packet, + out headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + state start { + transition parse_ethernet; + } + state parse_ethernet { + packet.extract(hdr.ethernet); + transition select(hdr.ethernet.etherType) { + 0x800: parse_ipv4; + default: accept; + } + } + state parse_ipv4 { + packet.extract(hdr.ipv4); + transition select(hdr.ipv4.protocol) { + 6: parse_tcp; + default: accept; + } + } + state parse_tcp { + packet.extract(hdr.tcp); + transition accept; + } +} + +/************************************************************************* +************ C H E C K S U M V E R I F I C A T I O N ************* +*************************************************************************/ + +control MyVerifyChecksum(in headers hdr, inout metadata meta) { + apply { } +} + +/************************************************************************* +************** I N G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyIngress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + + action drop() { + mark_to_drop(); + } + action set_ecmp_select(bit<16> ecmp_base, bit<32> ecmp_count) { + /* TODO: hash on 5-tuple and save the hash result in meta.ecmp_select + so that the ecmp_nhop table can use it to make a forwarding decision accordingly */ + } + action set_nhop(bit<48> nhop_dmac, bit<32> nhop_ipv4, bit<9> port) { + hdr.ethernet.dstAddr = nhop_dmac; + hdr.ipv4.dstAddr = nhop_ipv4; + standard_metadata.egress_spec = port; + hdr.ipv4.ttl = hdr.ipv4.ttl - 1; + } + table ecmp_group { + key = { + hdr.ipv4.dstAddr: lpm; + } + actions = { + drop; + set_ecmp_select; + } + size = 1024; + } + table ecmp_nhop { + key = { + meta.ecmp_select: exact; + } + actions = { + drop; + set_nhop; + } + size = 2; + } + apply { + if (hdr.ipv4.isValid() && hdr.ipv4.ttl > 0) { + ecmp_group.apply(); + ecmp_nhop.apply(); + } + } +} + +/************************************************************************* +**************** E G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyEgress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + + action rewrite_mac(bit<48> smac) { + hdr.ethernet.srcAddr = smac; + } + action drop() { + mark_to_drop(); + } + table send_frame { + key = { + standard_metadata.egress_port: exact; + } + actions = { + rewrite_mac; + drop; + } + size = 256; + } + apply { + send_frame.apply(); + } +} + +/************************************************************************* +************* C H E C K S U M C O M P U T A T I O N ************** +*************************************************************************/ + +control MyComputeChecksum(inout headers hdr, inout metadata meta) { + apply { + update_checksum( + hdr.ipv4.isValid(), + { hdr.ipv4.version, + hdr.ipv4.ihl, + hdr.ipv4.diffserv, + hdr.ipv4.totalLen, + hdr.ipv4.identification, + hdr.ipv4.flags, + hdr.ipv4.fragOffset, + hdr.ipv4.ttl, + hdr.ipv4.protocol, + hdr.ipv4.srcAddr, + hdr.ipv4.dstAddr }, + hdr.ipv4.hdrChecksum, + HashAlgorithm.csum16); + } +} + +/************************************************************************* +*********************** D E P A R S E R ******************************* +*************************************************************************/ + +control MyDeparser(packet_out packet, in headers hdr) { + apply { + packet.emit(hdr.ethernet); + packet.emit(hdr.ipv4); + packet.emit(hdr.tcp); + } +} + +/************************************************************************* +*********************** S W I T C H ******************************* +*************************************************************************/ + +V1Switch( +MyParser(), +MyVerifyChecksum(), +MyIngress(), +MyEgress(), +MyComputeChecksum(), +MyDeparser() +) main; diff --git a/SIGCOMM_2017/exercises/load_balance/p4app.json b/SIGCOMM_2017/exercises/load_balance/p4app.json new file mode 100644 index 0000000..4754fda --- /dev/null +++ b/SIGCOMM_2017/exercises/load_balance/p4app.json @@ -0,0 +1,32 @@ +{ + "program": "load_balance.p4", + "language": "p4-16", + "targets": { + "multiswitch": { + "auto-control-plane": true, + "cli": true, + "pcap_dump": true, + "bmv2_log": true, + "links": [["h1", "s1"], ["s1", "s2"], ["s1", "s3"], ["s3", "s2"], ["s2", "h2"], ["s3", "h3"]], + "hosts": { + "h1": { + }, + "h2": { + }, + "h3": { + } + }, + "switches": { + "s1": { + "entries": "s1-commands.txt" + }, + "s2": { + "entries": "s2-commands.txt" + }, + "s3": { + "entries": "s3-commands.txt" + } + } + } + } +} diff --git a/SIGCOMM_2017/exercises/load_balance/receive.py b/SIGCOMM_2017/exercises/load_balance/receive.py new file mode 100755 index 0000000..c93182f --- /dev/null +++ b/SIGCOMM_2017/exercises/load_balance/receive.py @@ -0,0 +1,52 @@ +#!/usr/bin/env python +import sys +import struct +import os + +from scapy.all import sniff, sendp, hexdump, get_if_list, get_if_hwaddr +from scapy.all import Packet, IPOption +from scapy.all import ShortField, IntField, LongField, BitField, FieldListField, FieldLenField +from scapy.all import IP, UDP, Raw +from scapy.layers.inet import _IPOption_HDR + +def get_if(): + ifs=get_if_list() + iface=None + for i in get_if_list(): + if "eth0" in i: + iface=i + break; + if not iface: + print "Cannot find eth0 interface" + exit(1) + return iface + +class IPOption_MRI(IPOption): + name = "MRI" + option = 31 + fields_desc = [ _IPOption_HDR, + FieldLenField("length", None, fmt="B", + length_of="swids", + adjust=lambda pkt,l:l+4), + ShortField("count", 0), + FieldListField("swids", + [], + IntField("", 0), + length_from=lambda pkt:pkt.count*4) ] +def handle_pkt(pkt): + print "got a packet" + pkt.show2() +# hexdump(pkt) + sys.stdout.flush() + + +def main(): + ifaces = filter(lambda i: 'eth' in i, os.listdir('/sys/class/net/')) + iface = ifaces[0] + print "sniffing on %s" % iface + sys.stdout.flush() + sniff(filter="tcp", iface = iface, + prn = lambda x: handle_pkt(x)) + +if __name__ == '__main__': + main() diff --git a/SIGCOMM_2017/exercises/load_balance/run.sh b/SIGCOMM_2017/exercises/load_balance/run.sh new file mode 100755 index 0000000..15f7868 --- /dev/null +++ b/SIGCOMM_2017/exercises/load_balance/run.sh @@ -0,0 +1,4 @@ +P4APPRUNNER=../../utils/p4apprunner.py +mkdir -p build +tar -czf build/p4app.tgz * --exclude='build' +sudo python $P4APPRUNNER p4app.tgz --build-dir ./build diff --git a/SIGCOMM_2017/exercises/load_balance/s1-commands.txt b/SIGCOMM_2017/exercises/load_balance/s1-commands.txt new file mode 100644 index 0000000..a53e07c --- /dev/null +++ b/SIGCOMM_2017/exercises/load_balance/s1-commands.txt @@ -0,0 +1,6 @@ +table_set_default ecmp_group drop +table_add ecmp_group set_ecmp_select 10.0.0.1/32 => 0 2 +table_add ecmp_nhop set_nhop 0 => 00:00:00:00:01:02 10.0.2.2 2 +table_add ecmp_nhop set_nhop 1 => 00:00:00:00:01:03 10.0.3.3 3 +table_add send_frame rewrite_mac 2 => 00:00:00:01:02:00 +table_add send_frame rewrite_mac 3 => 00:00:00:01:03:00 diff --git a/SIGCOMM_2017/exercises/load_balance/s2-commands.txt b/SIGCOMM_2017/exercises/load_balance/s2-commands.txt new file mode 100644 index 0000000..ca78c19 --- /dev/null +++ b/SIGCOMM_2017/exercises/load_balance/s2-commands.txt @@ -0,0 +1,4 @@ +table_set_default ecmp_group drop +table_add ecmp_group set_ecmp_select 10.0.2.2/32 => 0 1 +table_add ecmp_nhop set_nhop 0 => 00:00:00:00:02:02 10.0.2.2 1 +table_add send_frame rewrite_mac 1 => 00:00:00:02:01:00 diff --git a/SIGCOMM_2017/exercises/load_balance/s3-commands.txt b/SIGCOMM_2017/exercises/load_balance/s3-commands.txt new file mode 100644 index 0000000..0f9e6b3 --- /dev/null +++ b/SIGCOMM_2017/exercises/load_balance/s3-commands.txt @@ -0,0 +1,4 @@ +table_set_default ecmp_group drop +table_add ecmp_group set_ecmp_select 10.0.3.3/32 => 0 1 +table_add ecmp_nhop set_nhop 0 => 00:00:00:00:03:03 10.0.3.3 1 +table_add send_frame rewrite_mac 1 => 00:00:00:03:01:00 diff --git a/SIGCOMM_2017/exercises/load_balance/send.py b/SIGCOMM_2017/exercises/load_balance/send.py new file mode 100755 index 0000000..00496d9 --- /dev/null +++ b/SIGCOMM_2017/exercises/load_balance/send.py @@ -0,0 +1,41 @@ +#!/usr/bin/env python +import argparse +import sys +import socket +import random +import struct + +from scapy.all import sendp, send, get_if_list, get_if_hwaddr +from scapy.all import Packet +from scapy.all import Ether, IP, UDP, TCP + +def get_if(): + ifs=get_if_list() + iface=None # "h1-eth0" + for i in get_if_list(): + if "eth0" in i: + iface=i + break; + if not iface: + print "Cannot find eth0 interface" + exit(1) + return iface + +def main(): + + if len(sys.argv)<3: + print 'pass 2 arguments: ""' + exit(1) + + addr = socket.gethostbyname(sys.argv[1]) + iface = get_if() + + print "sending on interface %s to %s" % (iface, str(addr)) + pkt = Ether(src=get_if_hwaddr(iface), dst='ff:ff:ff:ff:ff:ff') + pkt = pkt /IP(dst=addr) / TCP(dport=1234, sport=random.randint(49152,65535)) / sys.argv[2] + pkt.show2() + sendp(pkt, iface=iface, verbose=False) + + +if __name__ == '__main__': + main() diff --git a/SIGCOMM_2017/exercises/load_balance/solution/load_balance.p4 b/SIGCOMM_2017/exercises/load_balance/solution/load_balance.p4 new file mode 100644 index 0000000..bfb5a07 --- /dev/null +++ b/SIGCOMM_2017/exercises/load_balance/solution/load_balance.p4 @@ -0,0 +1,225 @@ +/* -*- P4_16 -*- */ +#include +#include + +/************************************************************************* +*********************** H E A D E R S *********************************** +*************************************************************************/ + +header ethernet_t { + bit<48> dstAddr; + bit<48> srcAddr; + bit<16> etherType; +} + +header ipv4_t { + bit<4> version; + bit<4> ihl; + bit<8> diffserv; + bit<16> totalLen; + bit<16> identification; + bit<3> flags; + bit<13> fragOffset; + bit<8> ttl; + bit<8> protocol; + bit<16> hdrChecksum; + bit<32> srcAddr; + bit<32> dstAddr; +} + +header tcp_t { + bit<16> srcPort; + bit<16> dstPort; + bit<32> seqNo; + bit<32> ackNo; + bit<4> dataOffset; + bit<3> res; + bit<3> ecn; + bit<6> ctrl; + bit<16> window; + bit<16> checksum; + bit<16> urgentPtr; +} + +struct metadata { + bit<14> ecmp_select; +} + +struct headers { + ethernet_t ethernet; + ipv4_t ipv4; + tcp_t tcp; +} + +/************************************************************************* +*********************** P A R S E R *********************************** +*************************************************************************/ + +parser MyParser(packet_in packet, + out headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + + state start { + transition parse_ethernet; + } + state parse_ethernet { + packet.extract(hdr.ethernet); + transition select(hdr.ethernet.etherType) { + 0x800: parse_ipv4; + default: accept; + } + } + state parse_ipv4 { + packet.extract(hdr.ipv4); + transition select(hdr.ipv4.protocol) { + 6: parse_tcp; + default: accept; + } + } + state parse_tcp { + packet.extract(hdr.tcp); + transition accept; + } +} + +/************************************************************************* +************ C H E C K S U M V E R I F I C A T I O N ************* +*************************************************************************/ + +control MyVerifyChecksum(in headers hdr, inout metadata meta) { + apply { } +} + +/************************************************************************* +************** I N G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyIngress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + action drop() { + mark_to_drop(); + } + action set_ecmp_select(bit<16> ecmp_base, bit<32> ecmp_count) { + hash(meta.ecmp_select, + HashAlgorithm.crc16, + ecmp_base, + { hdr.ipv4.srcAddr, + hdr.ipv4.dstAddr, + hdr.ipv4.protocol, + hdr.tcp.srcPort, + hdr.tcp.dstPort }, + ecmp_count); + } + action set_nhop(bit<48> nhop_dmac, bit<32> nhop_ipv4, bit<9> port) { + hdr.ethernet.dstAddr = nhop_dmac; + hdr.ipv4.dstAddr = nhop_ipv4; + standard_metadata.egress_spec = port; + hdr.ipv4.ttl = hdr.ipv4.ttl - 1; + } + table ecmp_group { + key = { + hdr.ipv4.dstAddr: lpm; + } + actions = { + drop; + set_ecmp_select; + } + size = 1024; + } + table ecmp_nhop { + key = { + meta.ecmp_select: exact; + } + actions = { + drop; + set_nhop; + } + size = 2; + } + apply { + if (hdr.ipv4.isValid() && hdr.ipv4.ttl > 0) { + ecmp_group.apply(); + ecmp_nhop.apply(); + } + } +} + +/************************************************************************* +**************** E G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyEgress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + + action rewrite_mac(bit<48> smac) { + hdr.ethernet.srcAddr = smac; + } + action drop() { + mark_to_drop(); + } + table send_frame { + key = { + standard_metadata.egress_port: exact; + } + actions = { + rewrite_mac; + drop; + } + size = 256; + } + apply { + send_frame.apply(); + } +} + +/************************************************************************* +************* C H E C K S U M C O M P U T A T I O N ************** +*************************************************************************/ + +control MyComputeChecksum(inout headers hdr, inout metadata meta) { + apply { + update_checksum( + hdr.ipv4.isValid(), + { hdr.ipv4.version, + hdr.ipv4.ihl, + hdr.ipv4.diffserv, + hdr.ipv4.totalLen, + hdr.ipv4.identification, + hdr.ipv4.flags, + hdr.ipv4.fragOffset, + hdr.ipv4.ttl, + hdr.ipv4.protocol, + hdr.ipv4.srcAddr, + hdr.ipv4.dstAddr }, + hdr.ipv4.hdrChecksum, + HashAlgorithm.csum16); + } +} + +/************************************************************************* +*********************** D E P A R S E R ******************************* +*************************************************************************/ + +control MyDeparser(packet_out packet, in headers hdr) { + apply { + packet.emit(hdr.ethernet); + packet.emit(hdr.ipv4); + packet.emit(hdr.tcp); + } +} + +/************************************************************************* +*********************** S W I T C H ******************************* +*************************************************************************/ + +V1Switch( +MyParser(), +MyVerifyChecksum(), +MyIngress(), +MyEgress(), +MyComputeChecksum(), +MyDeparser() +) main; diff --git a/SIGCOMM_2017/exercises/mri/README.md b/SIGCOMM_2017/exercises/mri/README.md new file mode 100644 index 0000000..30d20b1 --- /dev/null +++ b/SIGCOMM_2017/exercises/mri/README.md @@ -0,0 +1,239 @@ +# Implementing MRI + +## Introduction + +The objective of this tutorial is to extend basic L3 forwarding with a +scaled-down version of In-Band Network Telemetry (INT), which we call +Multi-Hop Route Inspection (MRI). + +MRI allows users to track the path and the length of queues that every +packet travels through. To support this functionality, you will need +to write a P4 program that appends an ID and queue length to the +header stack of every packet. At the destination, the sequence of +switch IDs correspond to the path, and each ID is followed by the +queue length of the port at switch. + +As before, we have already defined the control plane rules, so you +only need to implement the data plane logic of your P4 program. + +> **Spoiler alert:** There is a reference solution in the `solution` +> sub-directory. Feel free to compare your implementation to the reference. + +## Step 1: Run the (incomplete) starter code + +The directory with this README also contains a skeleton P4 program, +`mri.p4`, which initially implements L3 forwarding. Your job (in the +next step) will be to extend it to properly prepend the MRI custom +headers. + +Before that, let's compile the incomplete `mri.p4` and bring up a +switch in Mininet to test its behavior. + +1. In your shell, run: + ```bash + ./run.sh + ``` + This will: + * compile `mri.p4`, and + * start a Mininet instance with three switches (`s1`, `s2`, `s3`) configured + in a triangle. There are 5 hosts. `h1` and `h11` are connected to `s1`. + `h2` and `h22` are connected to `s2` and `h3` is connected to `s3`. + * The hosts are assigned IPs of `10.0.1.10`, `10.0.2.10`, etc + (`10.0..`). + * The control plane programs the P4 tables in each switch based on + `sx-commands.txt` + +2. We want to send a low rate traffic from `h1` to `h2` and a high + rate iperf traffic from `h11` to `h22`. The link between `s1` and + `s2` is common between the flows and is a bottleneck because we + reduced its bandwidth to 512kbps in p4app.json. Therefore, if we + capture packets at `h2`, we should see high queue size for that + link. + +3. You should now see a Mininet command prompt. Open four terminals + for `h1`, `h11`, `h2`, `h22`, respectively: + ```bash + mininet> xterm h1 h11 h2 h22 + ``` +3. In `h2`'s xterm, start the server that captures packets: + ```bash + ./receive.py + ``` +4. in `h22`'s xterm, start the iperf UDP server: + ```bash + iperf -s -u + ``` + +5. In `h1`'s xterm, send one packet per second to `h2` using send.py + say for 30 seconds: + ```bash + ./send.py 10.0.2.2 "P4 is cool" 30 + ``` + The message "P4 is cool" should be received in `h2`'s xterm, +6. In `h11`'s xterm, start iperf client sending for 15 seconds + ```bash + h11 iperf -c 10.0.2.22 -t 15 -u + ``` +7. At `h2`, the MRI header has no hop info (`count=0`) +8. type `exit` to close each xterm window + +You should see the message received at host `h2`, but without any +information about the path the message took. Your job is to extend +the code in `mri.p4` to implement the MRI logic to record the path. + +### A note about the control plane + +P4 programs define a packet-processing pipeline, but the rules +governing packet processing are inserted into the pipeline by the +control plane. When a rule matches a packet, its action is invoked +with parameters supplied by the control plane as part of the rule. + +In this exercise, the control plane logic has already been +implemented. As part of bringing up the Mininet instance, the +`run.sh` script will install packet-processing rules in the tables of +each switch. These are defined in the `sX-commands.txt` files, where +`X` corresponds to the switch number. + +## Step 2: Implement MRI + +The `mri.p4` file contains a skeleton P4 program with key pieces of +logic replaced by `TODO` comments. These should guide your +implementation---replace each `TODO` with logic implementing the +missing piece. + +MRI will require two custom headers. The first header, `mri_t`, +contains a single field `count`, which indicates the number of switch +IDs that follow. The second header, `switch_t`, contains switch ID and +Queue depth fields of each switch hop the packet goes through. + +One of the biggest challenges in implementing MRI is handling the +recursive logic for parsing these two headers. We will use a +`parser_metadata` field, `remaining`, to keep track of how many +`switch_t` headers we need to parse. In the `parse_mri` state, this +field should be set to `hdr.mri.count`. In the `parse_swtrace` state, +this field should be decremented. The `parse_swtrace` state will +transition to itself until `remaining` is 0. + +The MRI custom headers will be carried inside an IP Options +header. The IP Options header contains a field, `option`, which +indicates the type of the option. We will use a special type 31 to +indicate the presence of the MRI headers. + +Beyond the parser logic, you will add a table in egress, `swtrace` to +store the switch ID and queue depth, and actions that increment the +`count` field, and append a `switch_t` header. + +A complete `mri.p4` will contain the following components: + +1. Header type definitions for Ethernet (`ethernet_t`), IPv4 (`ipv4_t`), + IP Options (`ipv4_option_t`), MRI (`mri_t`), and Switch (`switch_t`). +2. Parsers for Ethernet, IPv4, IP Options, MRI, and Switch that will +populate `ethernet_t`, `ipv4_t`, `ipv4_option_t`, `mri_t`, and +`switch_t`. +3. An action to drop a packet, using `mark_to_drop()`. +4. An action (called `ipv4_forward`), which will: + 1. Set the egress port for the next hop. + 2. Update the ethernet destination address with the address of + the next hop. + 3. Update the ethernet source address with the address of the switch. + 4. Decrement the TTL. +5. An ingress control that: + 1. Defines a table that will read an IPv4 destination address, and + invoke either `drop` or `ipv4_forward`. + 2. An `apply` block that applies the table. +6. At egress, an action (called `add_swtrace`) that will add the + switch ID and queue depth. +8. An egress control that applies a table (`swtrace`) to store the + switch ID and queue depth, and calls `add_swtrace`. +9. A deparser that selects the order in which fields inserted into the outgoing + packet. +10. A `package` instantiation supplied with the parser, control, + checksum verification and recomputation and deparser. + +## Step 3: Run your solution + +Follow the instructions from Step 1. This time, when your message + from `h1` is delivered to `h2`, you should see the seqeunce of + switches through which the packet traveled plus the corresponding + queue depths. The expected output will look like the following, + which shows the MRI header, with a `count` of 2, and switch ids + (`swids`) 2 and 1. The queue depth at the common link (from s1 to + s2) is high. + +``` +got a packet +###[ Ethernet ]### + dst = 00:04:00:02:00:02 + src = f2:ed:e6:df:4e:fa + type = 0x800 +###[ IP ]### + version = 4L + ihl = 10L + tos = 0x0 + len = 42 + id = 1 + flags = + frag = 0L + ttl = 62 + proto = udp + chksum = 0x60c0 + src = 10.0.1.1 + dst = 10.0.2.2 + \options \ + |###[ MRI ]### + | copy_flag = 0L + | optclass = control + | option = 31L + | length = 20 + | count = 2 + | \swtraces \ + | |###[ SwitchTrace ]### + | | swid = 2 + | | qdepth = 0 + | |###[ SwitchTrace ]### + | | swid = 1 + | | qdepth = 17 +###[ Raw ]### + load = '\x04\xd2' +###[ Padding ]### + load = '\x10\xe1\x00\x12\x1c{P4 is cool' + +``` + +### Troubleshooting + +There are several ways that problems might manifest: + +1. `mri.p4` fails to compile. In this case, `run.sh` will report the +error emitted from the compiler and stop. +2. `mri.p4` compiles but does not support the control plane rules in +the `sX-commands.txt` files that `run.sh` tries to install using the BMv2 CLI. +In this case, `run.sh` will report these errors to `stderr`. Use these error +messages to fix your `mri.p4` implementation. +3. `mri.p4` compiles, and the control plane rules are installed, but +the switch does not process packets in the desired way. The +`build/logs/.log` files contain trace messages describing +how each switch processes each packet. The output is detailed and can +help pinpoint logic errors in your implementation. The +`build/-.pcap` also contains the pcap of +packets on each interface. Use `tcpdump -r -xxx` to print +the hexdump of the packets. +4. `mri.p4` compiles and all rules are installed. Packets go through +and the logs show that the queue length is always 0. Then either +reduce the link bandwidth in `p4app.json`. + +#### Cleaning up Mininet + +In the latter two cases above, `run.sh` may leave a Mininet instance +running in the background. Use the following command to clean up +these instances: + +```bash +mn -c +``` + +## Next Steps + +Congratulations, your implementation works! Move on to [Source +Routing](../source_routing). + diff --git a/SIGCOMM_2017/exercises/mri/mri.p4 b/SIGCOMM_2017/exercises/mri/mri.p4 new file mode 100644 index 0000000..5fcf1d3 --- /dev/null +++ b/SIGCOMM_2017/exercises/mri/mri.p4 @@ -0,0 +1,280 @@ +/* -*- P4_16 -*- */ +#include +#include + +const bit<8> UDP_PROTOCOL = 0x11; +const bit<16> TYPE_IPV4 = 0x800; +const bit<5> IPV4_OPTION_MRI = 31; + +#define MAX_HOPS 9 + +/************************************************************************* +*********************** H E A D E R S *********************************** +*************************************************************************/ + +typedef bit<9> egressSpec_t; +typedef bit<48> macAddr_t; +typedef bit<32> ip4Addr_t; +typedef bit<32> switchID_t; +typedef bit<32> qdepth_t; + +header ethernet_t { + macAddr_t dstAddr; + macAddr_t srcAddr; + bit<16> etherType; +} + +header ipv4_t { + bit<4> version; + bit<4> ihl; + bit<8> diffserv; + bit<16> totalLen; + bit<16> identification; + bit<3> flags; + bit<13> fragOffset; + bit<8> ttl; + bit<8> protocol; + bit<16> hdrChecksum; + ip4Addr_t srcAddr; + ip4Addr_t dstAddr; +} + +header ipv4_option_t { + bit<1> copyFlag; + bit<2> optClass; + bit<5> option; + bit<8> optionLength; +} + +header mri_t { + bit<16> count; +} + +header switch_t { + switchID_t swid; + qdepth_t qdepth; +} + +struct ingress_metadata_t { + bit<16> count; +} + +struct parser_metadata_t { + bit<16> remaining; +} + +struct metadata { + ingress_metadata_t ingress_metadata; + parser_metadata_t parser_metadata; +} + +struct headers { + ethernet_t ethernet; + ipv4_t ipv4; + ipv4_option_t ipv4_option; + mri_t mri; + switch_t[MAX_HOPS] swtraces; +} + +error { IPHeaderTooShort } + +/************************************************************************* +*********************** P A R S E R *********************************** +*************************************************************************/ + +parser MyParser(packet_in packet, + out headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + + state start { + transition parse_ethernet; + } + + state parse_ethernet { + packet.extract(hdr.ethernet); + transition select(hdr.ethernet.etherType) { + TYPE_IPV4: parse_ipv4; + default: accept; + } + } + + state parse_ipv4 { + packet.extract(hdr.ipv4); + verify(hdr.ipv4.ihl >= 5, error.IPHeaderTooShort); + transition select(hdr.ipv4.ihl) { + 5 : accept; + default : parse_ipv4_option; + } + } + + state parse_ipv4_option { + /* + * TODO: Add logic to: + * - Extract the ipv4_option header. + * - If value is equal to IPV4_OPTION_MRI, transition to parse_mri. + * - Otherwise, accept. + */ + transition accept; + } + + state parse_mri { + /* + * TODO: Add logic to: + * - Extract hdr.mri. + * - Set meta.parser_metadata.remaining to hdr.mri.count + * - Select on the value of meta.parser_metadata.remaining + * - If the value is equal to 0, accept. + * - Otherwise, transition to parse_swid. + */ + transition accept; + } + + state parse_swtrace { + /* + * TODO: Add logic to: + * - Extract hdr.swtraces.next. + * - Decrement meta.parser_metadata.remaining by 1 + * - Select on the value of meta.parser_metadata.remaining + * - If the value is equal to 0, accept. + * - Otherwise, transition to parse_swtrace. + */ + transition accept; + } +} + + +/************************************************************************* +************ C H E C K S U M V E R I F I C A T I O N ************* +*************************************************************************/ + +control MyVerifyChecksum(in headers hdr, inout metadata meta) { + apply { } +} + + +/************************************************************************* +************** I N G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyIngress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + action drop() { + mark_to_drop(); + } + + action ipv4_forward(macAddr_t dstAddr, egressSpec_t port) { + standard_metadata.egress_spec = port; + hdr.ethernet.srcAddr = hdr.ethernet.dstAddr; + hdr.ethernet.dstAddr = dstAddr; + hdr.ipv4.ttl = hdr.ipv4.ttl - 1; + } + + table ipv4_lpm { + key = { + hdr.ipv4.dstAddr: lpm; + } + actions = { + ipv4_forward; + drop; + NoAction; + } + size = 1024; + default_action = NoAction(); + } + + apply { + if (hdr.ipv4.isValid()) { + ipv4_lpm.apply(); + } + } +} + +/************************************************************************* +**************** E G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyEgress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + action add_swtrace(switchID_t swid) { + /* + * TODO: add logic to: + - Increment hdr.mri.count by 1 + - Add a new swtrace header by calling push_front(1) on hdr.swtraces. + - Set hdr.swtraces[0].swid to the id paremeter + - Set hdr.swtraces[0].qdepth to (qdepth_t)standard_metadata.deq_qdepth + - Incremement hdr.ipv4.ihl by 2 + - Incrememtn hdr.ipv4_option.optionLength by 8 + */ + } + + table swtrace { + actions = { + /* TODO: add the correct action */ + NoAction; + } + + default_action = NoAction(); + } + + apply { + /* + * TODO: add logic to: + * - If hdr.mri is valid: + * - Apply table swtrace + */ + swtrace.apply(); + } +} + +/************************************************************************* +************* C H E C K S U M C O M P U T A T I O N ************** +*************************************************************************/ + +control MyComputeChecksum(inout headers hdr, inout metadata meta) { + apply { + update_checksum( + hdr.ipv4.isValid(), + { hdr.ipv4.version, + hdr.ipv4.ihl, + hdr.ipv4.diffserv, + hdr.ipv4.totalLen, + hdr.ipv4.identification, + hdr.ipv4.flags, + hdr.ipv4.fragOffset, + hdr.ipv4.ttl, + hdr.ipv4.protocol, + hdr.ipv4.srcAddr, + hdr.ipv4.dstAddr }, + hdr.ipv4.hdrChecksum, + HashAlgorithm.csum16); + } +} + +/************************************************************************* +*********************** D E P A R S E R ******************************* +*************************************************************************/ + +control MyDeparser(packet_out packet, in headers hdr) { + apply { + packet.emit(hdr.ethernet); + packet.emit(hdr.ipv4); + + /* TODO: emit ipv4_option, mri and swtraces headers */ + } +} + +/************************************************************************* +*********************** S W I T C H ******************************* +*************************************************************************/ + +V1Switch( +MyParser(), +MyVerifyChecksum(), +MyIngress(), +MyEgress(), +MyComputeChecksum(), +MyDeparser() +) main; \ No newline at end of file diff --git a/SIGCOMM_2017/exercises/mri/p4app.json b/SIGCOMM_2017/exercises/mri/p4app.json new file mode 100644 index 0000000..2931e53 --- /dev/null +++ b/SIGCOMM_2017/exercises/mri/p4app.json @@ -0,0 +1,37 @@ +{ + "program": "mri.p4", + "language": "p4-16", + "targets": { + "multiswitch": { + "auto-control-plane": true, + "cli": true, + "pcap_dump": true, + "bmv2_log": true, + "links": [["h1", "s1"], ["h11", "s1"], ["s1", "s2", "0", 0.5], ["s1", "s3"], ["s3", "s2"], ["s2", "h2"], ["s2", "h22"], ["s3", "h3"]], + "hosts": { + "h1": { + }, + "h2": { + }, + "h3": { + }, + "h11": { + }, + "h22": { + } + + }, + "switches": { + "s1": { + "entries": "s1-commands.txt" + }, + "s2": { + "entries": "s2-commands.txt" + }, + "s3": { + "entries": "s3-commands.txt" + } + } + } + } +} diff --git a/SIGCOMM_2017/exercises/mri/receive.py b/SIGCOMM_2017/exercises/mri/receive.py new file mode 100755 index 0000000..b3c3b35 --- /dev/null +++ b/SIGCOMM_2017/exercises/mri/receive.py @@ -0,0 +1,57 @@ +#!/usr/bin/env python +import sys +import struct + +from scapy.all import sniff, sendp, hexdump, get_if_list, get_if_hwaddr +from scapy.all import Packet, IPOption +from scapy.all import PacketListField, ShortField, IntField, LongField, BitField, FieldListField, FieldLenField +from scapy.all import IP, UDP, Raw +from scapy.layers.inet import _IPOption_HDR + +def get_if(): + ifs=get_if_list() + iface=None + for i in get_if_list(): + if "eth0" in i: + iface=i + break; + if not iface: + print "Cannot find eth0 interface" + exit(1) + return iface + +class SwitchTrace(Packet): + fields_desc = [ IntField("swid", 0), + IntField("qdepth", 0)] + def extract_padding(self, p): + return "", p + +class IPOption_MRI(IPOption): + name = "MRI" + option = 31 + fields_desc = [ _IPOption_HDR, + FieldLenField("length", None, fmt="B", + length_of="swtraces", + adjust=lambda pkt,l:l*2+4), + ShortField("count", 0), + PacketListField("swtraces", + [], + SwitchTrace, + count_from=lambda pkt:(pkt.count*1)) ] + +def handle_pkt(pkt): + print "got a packet" + pkt.show2() +# hexdump(pkt) + sys.stdout.flush() + + +def main(): + iface = 'h2-eth0' + print "sniffing on %s" % iface + sys.stdout.flush() + sniff(filter="udp and port 4321", iface = iface, + prn = lambda x: handle_pkt(x)) + +if __name__ == '__main__': + main() diff --git a/SIGCOMM_2017/exercises/mri/run.sh b/SIGCOMM_2017/exercises/mri/run.sh new file mode 100755 index 0000000..d5c1947 --- /dev/null +++ b/SIGCOMM_2017/exercises/mri/run.sh @@ -0,0 +1,5 @@ +P4APPRUNNER=../../utils/p4apprunner.py +mkdir -p build +tar -czf build/p4app.tgz * --exclude='build' +#cd build +sudo python $P4APPRUNNER p4app.tgz --build-dir ./build diff --git a/SIGCOMM_2017/exercises/mri/s1-commands.txt b/SIGCOMM_2017/exercises/mri/s1-commands.txt new file mode 100644 index 0000000..d476195 --- /dev/null +++ b/SIGCOMM_2017/exercises/mri/s1-commands.txt @@ -0,0 +1,6 @@ +table_set_default ipv4_lpm drop +table_set_default swtrace add_swtrace 1 +table_add ipv4_lpm ipv4_forward 10.0.1.1/32 => 00:00:00:00:01:01 1 +table_add ipv4_lpm ipv4_forward 10.0.1.11/32 => 00:00:00:00:01:0b 2 +table_add ipv4_lpm ipv4_forward 10.0.2.0/24 => 00:00:00:02:03:00 3 +table_add ipv4_lpm ipv4_forward 10.0.3.0/24 => 00:00:00:03:02:00 4 diff --git a/SIGCOMM_2017/exercises/mri/s2-commands.txt b/SIGCOMM_2017/exercises/mri/s2-commands.txt new file mode 100644 index 0000000..915c888 --- /dev/null +++ b/SIGCOMM_2017/exercises/mri/s2-commands.txt @@ -0,0 +1,6 @@ +table_set_default ipv4_lpm drop +table_set_default swtrace add_swtrace 2 +table_add ipv4_lpm ipv4_forward 10.0.2.2/32 => 00:00:00:00:02:02 1 +table_add ipv4_lpm ipv4_forward 10.0.2.22/32 => 00:00:00:00:02:16 2 +table_add ipv4_lpm ipv4_forward 10.0.1.0/24 => 00:00:00:01:03:00 3 +table_add ipv4_lpm ipv4_forward 10.0.3.0/24 => 00:00:00:03:03:00 4 diff --git a/SIGCOMM_2017/exercises/mri/s3-commands.txt b/SIGCOMM_2017/exercises/mri/s3-commands.txt new file mode 100644 index 0000000..d27b73c --- /dev/null +++ b/SIGCOMM_2017/exercises/mri/s3-commands.txt @@ -0,0 +1,5 @@ +table_set_default ipv4_lpm drop +table_set_default swtrace add_swtrace 3 +table_add ipv4_lpm ipv4_forward 10.0.3.3/32 => 00:00:00:00:03:01 1 +table_add ipv4_lpm ipv4_forward 10.0.1.0/24 => 00:00:00:01:04:00 2 +table_add ipv4_lpm ipv4_forward 10.0.2.0/24 => 00:00:00:02:04:00 3 diff --git a/SIGCOMM_2017/exercises/mri/send.py b/SIGCOMM_2017/exercises/mri/send.py new file mode 100755 index 0000000..7b0aca7 --- /dev/null +++ b/SIGCOMM_2017/exercises/mri/send.py @@ -0,0 +1,78 @@ +#!/usr/bin/env python + +import argparse +import sys +import socket +import random +import struct + +from scapy.all import sendp, send, hexdump, get_if_list, get_if_hwaddr +from scapy.all import Packet, IPOption +from scapy.all import Ether, IP, UDP +from scapy.all import IntField, FieldListField, FieldLenField, ShortField, PacketListField +from scapy.layers.inet import _IPOption_HDR + +from time import sleep + +def get_if(): + ifs=get_if_list() + iface=None # "h1-eth0" + for i in get_if_list(): + if "eth0" in i: + iface=i + break; + if not iface: + print "Cannot find eth0 interface" + exit(1) + return iface + +class SwitchTrace(Packet): + fields_desc = [ IntField("swid", 0), + IntField("qdepth", 0)] + def extract_padding(self, p): + return "", p + +class IPOption_MRI(IPOption): + name = "MRI" + option = 31 + fields_desc = [ _IPOption_HDR, + FieldLenField("length", None, fmt="B", + length_of="swtraces", + adjust=lambda pkt,l:l*2+4), + ShortField("count", 0), + PacketListField("swtraces", + [], + SwitchTrace, + count_from=lambda pkt:(pkt.count*1)) ] + + +def main(): + + if len(sys.argv)<3: + print 'pass 2 arguments: ""' + exit(1) + + addr = socket.gethostbyname(sys.argv[1]) + iface = get_if() + + pkt = Ether(src=get_if_hwaddr(iface), dst="ff:ff:ff:ff:ff:ff") / IP( + dst=addr, options = IPOption_MRI(count=0, + swtraces=[])) / UDP( + dport=4321, sport=1234) / sys.argv[2] + + # pkt = Ether(src=get_if_hwaddr(iface), dst="ff:ff:ff:ff:ff:ff") / IP( + # dst=addr, options = IPOption_MRI(count=2, + # swtraces=[SwitchTrace(swid=0,qdepth=0), SwitchTrace(swid=1,qdepth=0)])) / UDP( + # dport=4321, sport=1234) / sys.argv[2] + pkt.show2() + #hexdump(pkt) + try: + for i in range(int(sys.argv[3])): + sendp(pkt, iface=iface) + sleep(1) + except KeyboardInterrupt: + raise + + +if __name__ == '__main__': + main() diff --git a/SIGCOMM_2017/exercises/mri/solution/mri.p4 b/SIGCOMM_2017/exercises/mri/solution/mri.p4 new file mode 100644 index 0000000..ef369b5 --- /dev/null +++ b/SIGCOMM_2017/exercises/mri/solution/mri.p4 @@ -0,0 +1,267 @@ +/* -*- P4_16 -*- */ +#include +#include + +const bit<8> UDP_PROTOCOL = 0x11; +const bit<16> TYPE_IPV4 = 0x800; +const bit<5> IPV4_OPTION_MRI = 31; + +#define MAX_HOPS 9 + +/************************************************************************* +*********************** H E A D E R S *********************************** +*************************************************************************/ + +typedef bit<9> egressSpec_t; +typedef bit<48> macAddr_t; +typedef bit<32> ip4Addr_t; +typedef bit<32> switchID_t; +typedef bit<32> qdepth_t; + +header ethernet_t { + macAddr_t dstAddr; + macAddr_t srcAddr; + bit<16> etherType; +} + +header ipv4_t { + bit<4> version; + bit<4> ihl; + bit<8> diffserv; + bit<16> totalLen; + bit<16> identification; + bit<3> flags; + bit<13> fragOffset; + bit<8> ttl; + bit<8> protocol; + bit<16> hdrChecksum; + ip4Addr_t srcAddr; + ip4Addr_t dstAddr; +} + +header ipv4_option_t { + bit<1> copyFlag; + bit<2> optClass; + bit<5> option; + bit<8> optionLength; +} + +header mri_t { + bit<16> count; +} + +header switch_t { + switchID_t swid; + qdepth_t qdepth; +} + +struct ingress_metadata_t { + bit<16> count; +} + +struct parser_metadata_t { + bit<16> remaining; +} + +struct metadata { + ingress_metadata_t ingress_metadata; + parser_metadata_t parser_metadata; +} + +struct headers { + ethernet_t ethernet; + ipv4_t ipv4; + ipv4_option_t ipv4_option; + mri_t mri; + switch_t[MAX_HOPS] swtraces; +} + +error { IPHeaderTooShort } + +/************************************************************************* +*********************** P A R S E R *********************************** +*************************************************************************/ + +parser MyParser(packet_in packet, + out headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + + state start { + transition parse_ethernet; + } + + state parse_ethernet { + packet.extract(hdr.ethernet); + transition select(hdr.ethernet.etherType) { + TYPE_IPV4: parse_ipv4; + default: accept; + } + } + + state parse_ipv4 { + packet.extract(hdr.ipv4); + verify(hdr.ipv4.ihl >= 5, error.IPHeaderTooShort); + transition select(hdr.ipv4.ihl) { + 5 : accept; + default : parse_ipv4_option; + } + } + + state parse_ipv4_option { + packet.extract(hdr.ipv4_option); + transition select(hdr.ipv4_option.option) { + IPV4_OPTION_MRI: parse_mri; + default: accept; + } + } + + state parse_mri { + packet.extract(hdr.mri); + meta.parser_metadata.remaining = hdr.mri.count; + transition select(meta.parser_metadata.remaining) { + 0 : accept; + default: parse_swtrace; + } + } + + state parse_swtrace { + packet.extract(hdr.swtraces.next); + meta.parser_metadata.remaining = meta.parser_metadata.remaining - 1; + transition select(meta.parser_metadata.remaining) { + 0 : accept; + default: parse_swtrace; + } + } +} + + +/************************************************************************* +************ C H E C K S U M V E R I F I C A T I O N ************* +*************************************************************************/ + +control MyVerifyChecksum(in headers hdr, inout metadata meta) { + apply { } +} + + +/************************************************************************* +************** I N G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyIngress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + action drop() { + mark_to_drop(); + } + + action ipv4_forward(macAddr_t dstAddr, egressSpec_t port) { + standard_metadata.egress_spec = port; + hdr.ethernet.srcAddr = hdr.ethernet.dstAddr; + hdr.ethernet.dstAddr = dstAddr; + hdr.ipv4.ttl = hdr.ipv4.ttl - 1; + } + + table ipv4_lpm { + key = { + hdr.ipv4.dstAddr: lpm; + } + actions = { + ipv4_forward; + drop; + NoAction; + } + size = 1024; + default_action = NoAction(); + } + + apply { + if (hdr.ipv4.isValid()) { + ipv4_lpm.apply(); + } + } +} + +/************************************************************************* +**************** E G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyEgress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + action add_swtrace(switchID_t swid) { + hdr.mri.count = hdr.mri.count + 1; + hdr.swtraces.push_front(1); + hdr.swtraces[0].swid = swid; + hdr.swtraces[0].qdepth = (qdepth_t)standard_metadata.deq_qdepth; + + hdr.ipv4.ihl = hdr.ipv4.ihl + 2; + hdr.ipv4_option.optionLength = hdr.ipv4_option.optionLength + 8; + } + + table swtrace { + actions = { + add_swtrace; + NoAction; + } + default_action = NoAction(); + } + + apply { + if (hdr.mri.isValid()) { + swtrace.apply(); + } + } +} + +/************************************************************************* +************* C H E C K S U M C O M P U T A T I O N ************** +*************************************************************************/ + +control MyComputeChecksum(inout headers hdr, inout metadata meta) { + apply { + update_checksum( + hdr.ipv4.isValid(), + { hdr.ipv4.version, + hdr.ipv4.ihl, + hdr.ipv4.diffserv, + hdr.ipv4.totalLen, + hdr.ipv4.identification, + hdr.ipv4.flags, + hdr.ipv4.fragOffset, + hdr.ipv4.ttl, + hdr.ipv4.protocol, + hdr.ipv4.srcAddr, + hdr.ipv4.dstAddr }, + hdr.ipv4.hdrChecksum, + HashAlgorithm.csum16); + } +} + +/************************************************************************* +*********************** D E P A R S E R ******************************* +*************************************************************************/ + +control MyDeparser(packet_out packet, in headers hdr) { + apply { + packet.emit(hdr.ethernet); + packet.emit(hdr.ipv4); + packet.emit(hdr.ipv4_option); + packet.emit(hdr.mri); + packet.emit(hdr.swtraces); + } +} + +/************************************************************************* +*********************** S W I T C H ******************************* +*************************************************************************/ + +V1Switch( +MyParser(), +MyVerifyChecksum(), +MyIngress(), +MyEgress(), +MyComputeChecksum(), +MyDeparser() +) main; \ No newline at end of file diff --git a/SIGCOMM_2017/exercises/scrambler/README.md b/SIGCOMM_2017/exercises/scrambler/README.md new file mode 100644 index 0000000..e5ae870 --- /dev/null +++ b/SIGCOMM_2017/exercises/scrambler/README.md @@ -0,0 +1,123 @@ +# Implementing basic forwarding with scrambled addresses + +## Introduction + +In this exercise, you will extend your solution to the basic +forwarding exercise with a new twist: switches will invert the bits +representing Ethernet and IPv4 address. Hence, in our triangle +topology, the packets in the interior of the network will have +unintelligble addresses. + +> **Spoiler alert:** There is a reference solution in the `solution` +> sub-directory. Feel free to compare your implementation to the +> reference. + +## Step 1: Run the (incomplete) starter code + +The directory with this README also contains a skeleton P4 program, +`scrambler.p4`, which initially drops all packets. Your job (in the +next step) will be to extend it to properly forward IPv4 packets. + +Before that, let's compile the incomplete `scrambler.p4` and bring +up a switch in Mininet to test its behavior. + +1. In your shell, run: + ```bash + ./run.sh + ``` + This will: + * compile `scrambler.p4`, and + * start a Mininet instance with three switches (`s1`, `s2`, `s3`) configured + in a triangle, each connected to one host (`h1`, `h2`, `h3`). + * The hosts are assigned IPs of `10.0.1.1`, `10.0.2.2`, etc. + +2. You should now see a Mininet command prompt. Open two terminals +for `h1` and `h2`, respectively: + ```bash + mininet> xterm h1 h2 + ``` +3. Each host includes a small Python-based messaging client and +server. In `h2`'s xterm, start the server: + ```bash + ./receive.py + ``` +4. In `h1`'s xterm, send a message from the client: + ```bash + ./send.py 10.0.2.2 "P4 is cool" + ``` + The message will not be received. +5. Type `exit` to leave each xterm and the Mininet command line. + +The message was not received because each switch is programmed with +`scrambler.p4`, which drops all packets on arrival. Your job is to +extend this file. + +### A note about the control plane + +P4 programs define a packet-processing pipeline, but the rules +governing packet processing are inserted into the pipeline by the +control plane. When a rule matches a packet, its action is invoked +with parameters supplied by the control plane as part of the rule. + +In this exercise, the control plane logic has already been +implemented. As part of bringing up the Mininet instance, the +`run.sh` script will install packet-processing rules in the tables of +each switch. These are defined in the `sX-commands.txt` files, where +`X` corresponds to the switch number. + +**Important:** A P4 program also defines the interface between the +switch pipeline and control plane. The `sX-commands.txt` files +contain lists of commands for the BMv2 switch API. These commands +refer to specific tables, keys, and actions by name, and any changes +in the P4 program that add or rename tables, keys, or actions will +need to be reflected in these command files. + +## Step 2: Extend the basic forwarding solution to flip bits + +The `scrambler.p4` file contains a skeleton P4 program in which one of +the actions has a `TODO` comment. These should guide your +implementation---replace the `TODO` with logic implementing the +missing piece. + +A complete `scrambler.p4` will add an action `flip()` that inverts the +bits in the Ethernet and IPv4 headers. + +## Step 3: Run your solution + +Follow the instructions from Step 1. This time, your message from +`h1` should be delivered to `h2`. + +### Troubleshooting + +There are several issues that might arise when developing your +solution: + +1. `scrambler.p4` fails to compile. In this case, `run.sh` will +report the error emitted from the compiler and stop. + +2. `scrambler.p4` compiles but does not support the control plane +rules in the `sX-commands.txt` files that `run.sh` tries to install +using the BMv2 CLI. In this case, `run.sh` will report these errors +to `stderr`. Use these error messages to fix your `scrambler.p4` +implementation. + +3. `scrambler.p4` compiles, and the control plane rules are installed, +but the switch does not process packets in the desired way. The +`build/logs/.log` files contain trace messages describing +how each switch processes each packet. The output is detailed and can +help pinpoint logic errors in your implementation. + +#### Cleaning up Mininet + +In the latter two cases above, `run.sh` may leave a Mininet instance +running in the background. Use the following command to clean up +these instances: + +```bash +mn -c +``` + +## Next Steps + +Congratulations, your implementation works! Move on to the next +exercise: implementing [Explicit Congestion Notification](../ecn). diff --git a/SIGCOMM_2017/exercises/scrambler/p4app.json b/SIGCOMM_2017/exercises/scrambler/p4app.json new file mode 100644 index 0000000..5215605 --- /dev/null +++ b/SIGCOMM_2017/exercises/scrambler/p4app.json @@ -0,0 +1,33 @@ +{ + "program": "scrambler.p4", + "language": "p4-16", + "targets": { + "multiswitch": { + "auto-control-plane": true, + "cli": true, + "pcap_dump": true, + "bmv2_log": true, + "links": [["h1", "s1"], ["s1", "s2"], ["s1", "s3"], ["s3", "s2"], ["s2", "h2"], ["s3", "h3"]], + "hosts": { + "h1": { + }, + "h2": { + }, + "h3": { + } + + }, + "switches": { + "s1": { + "entries": "s1-commands.txt" + }, + "s2": { + "entries": "s2-commands.txt" + }, + "s3": { + "entries": "s3-commands.txt" + } + } + } + } +} diff --git a/SIGCOMM_2017/exercises/scrambler/receive.py b/SIGCOMM_2017/exercises/scrambler/receive.py new file mode 100755 index 0000000..c93182f --- /dev/null +++ b/SIGCOMM_2017/exercises/scrambler/receive.py @@ -0,0 +1,52 @@ +#!/usr/bin/env python +import sys +import struct +import os + +from scapy.all import sniff, sendp, hexdump, get_if_list, get_if_hwaddr +from scapy.all import Packet, IPOption +from scapy.all import ShortField, IntField, LongField, BitField, FieldListField, FieldLenField +from scapy.all import IP, UDP, Raw +from scapy.layers.inet import _IPOption_HDR + +def get_if(): + ifs=get_if_list() + iface=None + for i in get_if_list(): + if "eth0" in i: + iface=i + break; + if not iface: + print "Cannot find eth0 interface" + exit(1) + return iface + +class IPOption_MRI(IPOption): + name = "MRI" + option = 31 + fields_desc = [ _IPOption_HDR, + FieldLenField("length", None, fmt="B", + length_of="swids", + adjust=lambda pkt,l:l+4), + ShortField("count", 0), + FieldListField("swids", + [], + IntField("", 0), + length_from=lambda pkt:pkt.count*4) ] +def handle_pkt(pkt): + print "got a packet" + pkt.show2() +# hexdump(pkt) + sys.stdout.flush() + + +def main(): + ifaces = filter(lambda i: 'eth' in i, os.listdir('/sys/class/net/')) + iface = ifaces[0] + print "sniffing on %s" % iface + sys.stdout.flush() + sniff(filter="tcp", iface = iface, + prn = lambda x: handle_pkt(x)) + +if __name__ == '__main__': + main() diff --git a/SIGCOMM_2017/exercises/scrambler/run.sh b/SIGCOMM_2017/exercises/scrambler/run.sh new file mode 100755 index 0000000..d5c1947 --- /dev/null +++ b/SIGCOMM_2017/exercises/scrambler/run.sh @@ -0,0 +1,5 @@ +P4APPRUNNER=../../utils/p4apprunner.py +mkdir -p build +tar -czf build/p4app.tgz * --exclude='build' +#cd build +sudo python $P4APPRUNNER p4app.tgz --build-dir ./build diff --git a/SIGCOMM_2017/exercises/scrambler/s1-commands.txt b/SIGCOMM_2017/exercises/scrambler/s1-commands.txt new file mode 100644 index 0000000..0a63bdb --- /dev/null +++ b/SIGCOMM_2017/exercises/scrambler/s1-commands.txt @@ -0,0 +1,6 @@ +table_set_default ipv4_lpm drop +table_add ipv4_lpm ipv4_forward 10.0.2.2/32 => 00:00:00:02:02:00 2 +table_add ipv4_lpm ipv4_forward 10.0.3.3/32 => 00:00:00:03:02:00 3 +table_add ipv4_lpm ipv4_forward 245.255.254.254/32 => ff:ff:ff:ff:fe:fe 1 +table_add ipv4_lpm ipv4_forward 245.255.253.253/32 => 00:00:00:02:02:00 2 +table_add ipv4_lpm ipv4_forward 245.255.252.252/32 => 00:00:00:03:02:00 3 diff --git a/SIGCOMM_2017/exercises/scrambler/s2-commands.txt b/SIGCOMM_2017/exercises/scrambler/s2-commands.txt new file mode 100644 index 0000000..f1797a6 --- /dev/null +++ b/SIGCOMM_2017/exercises/scrambler/s2-commands.txt @@ -0,0 +1,6 @@ +table_set_default ipv4_lpm drop +table_add ipv4_lpm ipv4_forward 10.0.1.1/32 => 00:00:00:01:02:00 2 +table_add ipv4_lpm ipv4_forward 10.0.3.3/32 => 00:00:00:03:03:00 3 +table_add ipv4_lpm ipv4_forward 245.255.254.254/32 => 00:00:00:01:02:00 2 +table_add ipv4_lpm ipv4_forward 245.255.253.253/32 => ff:ff:ff:ff:fd:fd 1 +table_add ipv4_lpm ipv4_forward 245.255.252.252/32 => 00:00:00:03:03:00 3 diff --git a/SIGCOMM_2017/exercises/scrambler/s3-commands.txt b/SIGCOMM_2017/exercises/scrambler/s3-commands.txt new file mode 100644 index 0000000..a04f7ba --- /dev/null +++ b/SIGCOMM_2017/exercises/scrambler/s3-commands.txt @@ -0,0 +1,6 @@ +table_set_default ipv4_lpm drop +table_add ipv4_lpm ipv4_forward 10.0.0.1/32 => 00:00:00:01:02:00 2 +table_add ipv4_lpm ipv4_forward 10.0.0.2/32 => 00:00:00:02:03:00 3 +table_add ipv4_lpm ipv4_forward 245.255.254.254/32 => 00:00:00:01:01:00 2 +table_add ipv4_lpm ipv4_forward 245.255.253.253/32 => 00:00:00:02:03:00 3 +table_add ipv4_lpm ipv4_forward 245.255.252.252/32 => ff:ff:ff:ff:fc:fc 1 diff --git a/SIGCOMM_2017/exercises/scrambler/scrambler.p4 b/SIGCOMM_2017/exercises/scrambler/scrambler.p4 new file mode 100644 index 0000000..34ebabe --- /dev/null +++ b/SIGCOMM_2017/exercises/scrambler/scrambler.p4 @@ -0,0 +1,180 @@ +/* -*- P4_16 -*- */ +#include +#include + +const bit<16> TYPE_IPV4 = 0x800; + +/************************************************************************* +*********************** H E A D E R S *********************************** +*************************************************************************/ + +typedef bit<9> egressSpec_t; +typedef bit<48> macAddr_t; +typedef bit<32> ip4Addr_t; + +header ethernet_t { + macAddr_t dstAddr; + macAddr_t srcAddr; + bit<16> etherType; +} + +header ipv4_t { + bit<4> version; + bit<4> ihl; + bit<8> diffserv; + bit<16> totalLen; + bit<16> identification; + bit<3> flags; + bit<13> fragOffset; + bit<8> ttl; + bit<8> protocol; + bit<16> hdrChecksum; + ip4Addr_t srcAddr; + ip4Addr_t dstAddr; +} + +struct metadata { + /* empty */ +} + +struct headers { + ethernet_t ethernet; + ipv4_t ipv4; +} + +/************************************************************************* +*********************** P A R S E R *********************************** +*************************************************************************/ + +parser MyParser(packet_in packet, + out headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + + state start { + transition parse_ethernet; + } + + state parse_ethernet { + packet.extract(hdr.ethernet); + transition select(hdr.ethernet.etherType) { + TYPE_IPV4: parse_ipv4; + default: accept; + } + } + + state parse_ipv4 { + packet.extract(hdr.ipv4); + transition accept; + } +} + + +/************************************************************************* +************ C H E C K S U M V E R I F I C A T I O N ************* +*************************************************************************/ + +control MyVerifyChecksum(in headers hdr, inout metadata meta) { + apply { } +} + + +/************************************************************************* +************** I N G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyIngress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + action drop() { + mark_to_drop(); + } + + action flip() { + /* TODO: add code to flip bits in Ethernet and IPv4 addresses. */ + } + + action ipv4_forward(macAddr_t dstAddr, egressSpec_t port) { + standard_metadata.egress_spec = port; + hdr.ethernet.srcAddr = hdr.ethernet.dstAddr; + hdr.ethernet.dstAddr = dstAddr; + hdr.ipv4.ttl = hdr.ipv4.ttl - 1; + flip(); + } + + table ipv4_lpm { + key = { + hdr.ipv4.dstAddr: lpm; + } + actions = { + ipv4_forward; + drop; + NoAction; + } + size = 1024; + default_action = NoAction(); + } + + apply { + if (hdr.ipv4.isValid()) { + ipv4_lpm.apply(); + } + } +} + +/************************************************************************* +**************** E G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyEgress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + apply { } +} + +/************************************************************************* +************* C H E C K S U M C O M P U T A T I O N ************** +*************************************************************************/ +control MyComputeChecksum(inout headers hdr, inout metadata meta) { + apply { + update_checksum( + hdr.ipv4.isValid(), + { hdr.ipv4.version, + hdr.ipv4.ihl, + hdr.ipv4.diffserv, + hdr.ipv4.totalLen, + hdr.ipv4.identification, + hdr.ipv4.flags, + hdr.ipv4.fragOffset, + hdr.ipv4.ttl, + hdr.ipv4.protocol, + hdr.ipv4.srcAddr, + hdr.ipv4.dstAddr }, + hdr.ipv4.hdrChecksum, + HashAlgorithm.csum16); + } +} + +/************************************************************************* +*********************** D E P A R S E R ******************************* +*************************************************************************/ + +control MyDeparser(packet_out packet, in headers hdr) { + apply { + packet.emit(hdr.ethernet); + packet.emit(hdr.ipv4); + } +} + +/************************************************************************* +*********************** S W I T C H ******************************* +*************************************************************************/ + +V1Switch( +MyParser(), +MyVerifyChecksum(), +MyIngress(), +MyEgress(), +MyComputeChecksum(), +MyDeparser() +) main; \ No newline at end of file diff --git a/SIGCOMM_2017/exercises/scrambler/send.py b/SIGCOMM_2017/exercises/scrambler/send.py new file mode 100755 index 0000000..00496d9 --- /dev/null +++ b/SIGCOMM_2017/exercises/scrambler/send.py @@ -0,0 +1,41 @@ +#!/usr/bin/env python +import argparse +import sys +import socket +import random +import struct + +from scapy.all import sendp, send, get_if_list, get_if_hwaddr +from scapy.all import Packet +from scapy.all import Ether, IP, UDP, TCP + +def get_if(): + ifs=get_if_list() + iface=None # "h1-eth0" + for i in get_if_list(): + if "eth0" in i: + iface=i + break; + if not iface: + print "Cannot find eth0 interface" + exit(1) + return iface + +def main(): + + if len(sys.argv)<3: + print 'pass 2 arguments: ""' + exit(1) + + addr = socket.gethostbyname(sys.argv[1]) + iface = get_if() + + print "sending on interface %s to %s" % (iface, str(addr)) + pkt = Ether(src=get_if_hwaddr(iface), dst='ff:ff:ff:ff:ff:ff') + pkt = pkt /IP(dst=addr) / TCP(dport=1234, sport=random.randint(49152,65535)) / sys.argv[2] + pkt.show2() + sendp(pkt, iface=iface, verbose=False) + + +if __name__ == '__main__': + main() diff --git a/SIGCOMM_2017/exercises/scrambler/solution/scrambler.p4 b/SIGCOMM_2017/exercises/scrambler/solution/scrambler.p4 new file mode 100644 index 0000000..6c79468 --- /dev/null +++ b/SIGCOMM_2017/exercises/scrambler/solution/scrambler.p4 @@ -0,0 +1,191 @@ +/* -*- P4_16 -*- */ +#include +#include + +const bit<16> TYPE_IPV4 = 0x800; + +/************************************************************************* +*********************** H E A D E R S *********************************** +*************************************************************************/ + +typedef bit<9> egressSpec_t; +typedef bit<48> macAddr_t; +typedef bit<32> ip4Addr_t; + +header ethernet_t { + macAddr_t dstAddr; + macAddr_t srcAddr; + bit<16> etherType; +} + +header ipv4_t { + bit<4> version; + bit<4> ihl; + bit<8> diffserv; + bit<16> totalLen; + bit<16> identification; + bit<3> flags; + bit<13> fragOffset; + bit<8> ttl; + bit<8> protocol; + bit<16> hdrChecksum; + ip4Addr_t srcAddr; + ip4Addr_t dstAddr; +} + +struct metadata { + /* empty */ +} + +struct headers { + ethernet_t ethernet; + ipv4_t ipv4; +} + +/************************************************************************* +*********************** P A R S E R *********************************** +*************************************************************************/ + +parser MyParser(packet_in packet, + out headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + + state start { + transition parse_ethernet; + } + + state parse_ethernet { + packet.extract(hdr.ethernet); + transition select(hdr.ethernet.etherType) { + TYPE_IPV4: parse_ipv4; + default: accept; + } + } + + state parse_ipv4 { + packet.extract(hdr.ipv4); + transition accept; + } +} + + +/************************************************************************* +************ C H E C K S U M V E R I F I C A T I O N ************* +*************************************************************************/ + +control MyVerifyChecksum(in headers hdr, inout metadata meta) { + apply { } +} + + +/************************************************************************* +************** I N G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyIngress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + action drop() { + mark_to_drop(); + } + + action flip_ethernet() { + hdr.ethernet.srcAddr = ~hdr.ethernet.srcAddr; + hdr.ethernet.dstAddr = ~hdr.ethernet.dstAddr; + } + action flip_ipv4() { + hdr.ipv4.srcAddr = ~hdr.ipv4.srcAddr; + hdr.ipv4.dstAddr = ~hdr.ipv4.dstAddr; + } + + action flip() { + flip_ethernet(); + flip_ipv4(); + } + + action ipv4_forward(macAddr_t dstAddr, egressSpec_t port) { + standard_metadata.egress_spec = port; + hdr.ethernet.srcAddr = hdr.ethernet.dstAddr; + hdr.ethernet.dstAddr = dstAddr; + hdr.ipv4.ttl = hdr.ipv4.ttl - 1; + flip(); + } + + table ipv4_lpm { + key = { + hdr.ipv4.dstAddr: lpm; + } + actions = { + ipv4_forward; + drop; + NoAction; + } + size = 1024; + default_action = NoAction(); + } + + apply { + if (hdr.ipv4.isValid()) { + ipv4_lpm.apply(); + } + } +} + +/************************************************************************* +**************** E G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyEgress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + apply { } +} + +/************************************************************************* +************* C H E C K S U M C O M P U T A T I O N ************** +*************************************************************************/ + +control MyComputeChecksum(inout headers hdr, inout metadata meta) { + apply { + update_checksum( + hdr.ipv4.isValid(), + { hdr.ipv4.version, + hdr.ipv4.ihl, + hdr.ipv4.diffserv, + hdr.ipv4.totalLen, + hdr.ipv4.identification, + hdr.ipv4.flags, + hdr.ipv4.fragOffset, + hdr.ipv4.ttl, + hdr.ipv4.protocol, + hdr.ipv4.srcAddr, + hdr.ipv4.dstAddr }, + hdr.ipv4.hdrChecksum, + HashAlgorithm.csum16); + } +} + +/************************************************************************* +*********************** D E P A R S E R ******************************* +*************************************************************************/ + +control MyDeparser(packet_out packet, in headers hdr) { + apply { + packet.emit(hdr.ethernet); + packet.emit(hdr.ipv4); + } +} + +/************************************************************************* +*********************** S W I T C H ******************************* +*************************************************************************/ + +V1Switch( +MyParser(), +MyVerifyChecksum(), +MyIngress(), +MyEgress(), +MyComputeChecksum(), +MyDeparser() +) main; \ No newline at end of file diff --git a/SIGCOMM_2017/exercises/source_routing/README.md b/SIGCOMM_2017/exercises/source_routing/README.md new file mode 100644 index 0000000..003e7a7 --- /dev/null +++ b/SIGCOMM_2017/exercises/source_routing/README.md @@ -0,0 +1,147 @@ +# Implementing Source Routing + +## Introduction + +The objective of this exercise is to implement source routing. With +source routing, the source host guides each switch in the network to +send the packet to a specific port. The host puts a stack of output +ports in the packet. In this example, we just put the stack after +Ethernet header and select a special etherType to indicate that. Each +switch pops an item from the stack and forwards the packet according +to the specified port number. + +Your switch must parse the source routing stack. Each item has a bos +(bottom of stack) bit and a port number. The bos bit is 1 only for the +last entry of stack. Then at ingress, it should pop an entry from the +stack and set the egress port accordingly. Note that the last hop can +also revert back the etherType to `TYPE_IPV4`. + +> **Spoiler alert:** There is a reference solution in the `solution` +> sub-directory. Feel free to compare your implementation to the +> reference. + +## Step 1: Run the (incomplete) starter code + +The directory with this README also contains a skeleton P4 program, +`source_routing.p4`, which initially drops all packets. Your job (in +the next step) will be to extend it to properly to route packets. + +Before that, let's compile the incomplete `source_routing.p4` and +bring up a network in Mininet to test its behavior. + +1. In your shell, run: + ```bash + ./run.sh + ``` + This will: + * compile `source_routing.p4`, and + * start a Mininet instance with three switches (`s1`, `s2`, `s3`) configured + in a triangle, each connected to one host (`h1`, `h2`, `h3`). + Check the network topology using the `net` command in mininet. + You can also change the topology in p4app.json + * The hosts are assigned IPs of `10.0.1.1`, `10.0.2.2`, etc + (`10.0..`). + +2. You should now see a Mininet command prompt. Open two terminals for + `h1` and `h2`, respectively: + ```bash + mininet> xterm h1 h2 + ``` +3. Each host includes a small Python-based messaging client and + server. In `h2`'s xterm, start the server: + ```bash + ./receive.py + ``` +4. In `h1`'s xterm, send a message from the client: + ```bash + ./send.py 10.0.2.2 + ``` + +5. Type a list of port numbers. say `2 3 2 2 1`. This should send the + packet through `h1`, `s1`, `s2`, `s3`, `s1`, `s2`, and + `h2`. However, `h2` will not receive the message. + +6. Type `q` to exit send.py and type `exit` to leave each xterm and + the Mininet command line. + +The message was not received because each switch is programmed with +`source_routing.p4`, which drops all packets on arrival. You can +verify this by looking at `build/logs/s1.log`. Your job is to extend +the P4 code so packets are delivered to their destination. + +## Step 2: Implement source routing + +The `source_routing.p4` file contains a skeleton P4 program with key +pieces of logic replaced by `TODO` comments. These should guide your +implementation---replace each `TODO` with logic implementing the +missing piece. + +A complete `source_routing.p4` will contain the following components: + +1. Header type definitions for Ethernet (`ethernet_t`) and IPv4 + (`ipv4_t`) and Source Route (`srcRoute_t`). +2. **TODO:** Parsers for Ethernet and Source Route that populate + `ethernet` and `srcRoutes` fields. +3. An action to drop a packet, using `mark_to_drop()`. +4. **TODO:** An action (called `srcRoute_nhop`), which will: + 1. Set the egress port for the next hop. + 2. remove the first entry of srcRoutes +5. A control with an `apply` block that: + 1. checks the existance of source routes. + 2. **TODO:** if statement to change etherent.etherType if it is the last hop + 3. **TODO:** call srcRoute_nhop action +6. A deparser that selects the order in which fields inserted into the outgoing + packet. +7. A `package` instantiation supplied with the parser, control, and deparser. + > In general, a package also requires instances of checksum verification + > and recomputation controls. These are not necessary for this tutorial + > and are replaced with instantiations of empty controls. + +## Step 3: Run your solution + +Follow the instructions from Step 1. This time, your message from `h1` +should be delivered to `h2`. + +Check the `ttl` of the IP header. Each hop decrements `ttl`. The port +sequence `2 3 2 2 1`, forces the packet to have a loop, so the `ttl` +should be 59 at `h2`. Can you find the port sequence for the shortest +path? + +### Food for thought +* Can we change the program to handle both IPv4 forwarding and source +routing at the same time? +* How would you enhance your program to let the first switch add the +path, so that source routing would be transparent to end-hosts? + +### Troubleshooting + +There are several ways that problems might manifest: + +1. `source_routing.p4` fails to compile. In this case, `run.sh` will + report the error emitted from the compiler and stop. +2. `source_routing.p4` compiles but switches or mininet do not start. + Do you have another instance of mininet running? Did the previous + run of mininet crash? if yes, check "Cleaning up Mininet" bellow. +3. `source_routing.p4` compiles but the switch does not process + packets in the desired way. The `build/logs/.log` + files contain trace messages describing how each switch processes + each packet. The output is detailed and can help pinpoint logic + errors in your implementation. The + `build/-.pcap` also contains the pcap + of packets on each interface. Use `tcpdump -r -xxx` to + print the hexdump of the packets. + +#### Cleaning up Mininet + +In the cases above, `run.sh` may leave a Mininet instance running in +the background. Use the following command to clean up these +instances: + +```bash +mn -c +``` + +## Next Steps + +Congratulations, your implementation works! Move on to +[Calculator](../calc). diff --git a/SIGCOMM_2017/exercises/source_routing/p4app.json b/SIGCOMM_2017/exercises/source_routing/p4app.json new file mode 100644 index 0000000..6508868 --- /dev/null +++ b/SIGCOMM_2017/exercises/source_routing/p4app.json @@ -0,0 +1,30 @@ +{ + "program": "source_routing.p4", + "language": "p4-16", + "targets": { + "multiswitch": { + "auto-control-plane": true, + "cli": true, + "pcap_dump": true, + "bmv2_log": true, + "links": [["h1", "s1"], ["s1", "s2"], ["s1", "s3"], ["s3", "s2"], ["s2", "h2"], ["s3", "h3"]], + "hosts": { + "h1": { + }, + "h2": { + }, + "h3": { + } + + }, + "switches": { + "s1": { + }, + "s2": { + }, + "s3": { + } + } + } + } +} diff --git a/SIGCOMM_2017/exercises/source_routing/receive.py b/SIGCOMM_2017/exercises/source_routing/receive.py new file mode 100755 index 0000000..efb93d6 --- /dev/null +++ b/SIGCOMM_2017/exercises/source_routing/receive.py @@ -0,0 +1,59 @@ +#!/usr/bin/env python +import sys +import struct + +from scapy.all import sniff, sendp, hexdump, get_if_list, get_if_hwaddr, bind_layers +from scapy.all import Packet, IPOption +from scapy.all import IP, UDP, Raw, Ether +from scapy.layers.inet import _IPOption_HDR +from scapy.fields import * + +def get_if(): + ifs=get_if_list() + iface=None + for i in get_if_list(): + if "eth0" in i: + iface=i + break; + if not iface: + print "Cannot find eth0 interface" + exit(1) + return iface + +class IPOption_MRI(IPOption): + name = "MRI" + option = 31 + fields_desc = [ _IPOption_HDR, + FieldLenField("length", None, fmt="B", + length_of="swids", + adjust=lambda pkt,l:l+4), + ShortField("count", 0), + FieldListField("swids", + [], + IntField("", 0), + length_from=lambda pkt:pkt.count*4) ] +def handle_pkt(pkt): + print "got a packet" + pkt.show2() +# hexdump(pkt) + sys.stdout.flush() + +class SourceRoute(Packet): + fields_desc = [ BitField("bos", 0, 1), + BitField("port", 0, 15)] +class SourceRoutingTail(Packet): + fields_desc = [ XShortField("etherType", 0x800)] + +bind_layers(Ether, SourceRoute, type=0x1234) +bind_layers(SourceRoute, SourceRoute, bos=0) +bind_layers(SourceRoute, SourceRoutingTail, bos=1) + +def main(): + iface = 'h2-eth0' + print "sniffing on %s" % iface + sys.stdout.flush() + sniff(filter="udp and port 4321", iface = iface, + prn = lambda x: handle_pkt(x)) + +if __name__ == '__main__': + main() diff --git a/SIGCOMM_2017/exercises/source_routing/run.sh b/SIGCOMM_2017/exercises/source_routing/run.sh new file mode 100755 index 0000000..d5c1947 --- /dev/null +++ b/SIGCOMM_2017/exercises/source_routing/run.sh @@ -0,0 +1,5 @@ +P4APPRUNNER=../../utils/p4apprunner.py +mkdir -p build +tar -czf build/p4app.tgz * --exclude='build' +#cd build +sudo python $P4APPRUNNER p4app.tgz --build-dir ./build diff --git a/SIGCOMM_2017/exercises/source_routing/send.py b/SIGCOMM_2017/exercises/source_routing/send.py new file mode 100755 index 0000000..21331db --- /dev/null +++ b/SIGCOMM_2017/exercises/source_routing/send.py @@ -0,0 +1,73 @@ +#!/usr/bin/env python +import argparse +import sys +import socket +import random +import struct + +from scapy.all import sendp, send, get_if_list, get_if_hwaddr, bind_layers +from scapy.all import Packet +from scapy.all import Ether, IP, UDP +from scapy.fields import * +import readline + +def get_if(): + ifs=get_if_list() + iface=None # "h1-eth0" + for i in get_if_list(): + if "eth0" in i: + iface=i + break; + if not iface: + print "Cannot find eth0 interface" + exit(1) + return iface + +class SourceRoute(Packet): + fields_desc = [ BitField("bos", 0, 1), + BitField("port", 0, 15)] + +bind_layers(Ether, SourceRoute, type=0x1234) +bind_layers(SourceRoute, SourceRoute, bos=0) +bind_layers(SourceRoute, IP, bos=1) + +def main(): + + if len(sys.argv)<2: + print 'pass 2 arguments: ' + exit(1) + + addr = socket.gethostbyname(sys.argv[1]) + iface = get_if() + print "sending on interface %s to %s" % (iface, str(addr)) + + while True: + print + s = str(raw_input('Type space separated port nums ' + '(example: "2 3 2 2 1") or "q" to quit: ')) + if s == "q": + break; + print + + i = 0 + pkt = Ether(src=get_if_hwaddr(iface), dst='ff:ff:ff:ff:ff:ff'); + for p in s.split(" "): + try: + pkt = pkt / SourceRoute(bos=0, port=int(p)) + i = i+1 + except ValueError: + pass + if pkt.haslayer(SourceRoute): + pkt.getlayer(SourceRoute, i).bos = 1 + + pkt = pkt / IP(dst=addr) / UDP(dport=4321, sport=1234) + pkt.show2() + sendp(pkt, iface=iface, verbose=False) + + #pkt = pkt / SourceRoute(bos=0, port=2) / SourceRoute(bos=0, port=3); + #pkt = pkt / SourceRoute(bos=0, port=2) / SourceRoute(bos=0, port=2); + #pkt = pkt / SourceRoute(bos=1, port=1) + + +if __name__ == '__main__': + main() diff --git a/SIGCOMM_2017/exercises/source_routing/solution/source_routing.p4 b/SIGCOMM_2017/exercises/source_routing/solution/source_routing.p4 new file mode 100644 index 0000000..33f01e8 --- /dev/null +++ b/SIGCOMM_2017/exercises/source_routing/solution/source_routing.p4 @@ -0,0 +1,181 @@ +/* -*- P4_16 -*- */ +#include +#include + +const bit<16> TYPE_IPV4 = 0x800; +const bit<16> TYPE_SRCROUTING = 0x1234; + +#define MAX_HOPS 9 + +/************************************************************************* +*********************** H E A D E R S *********************************** +*************************************************************************/ + +typedef bit<9> egressSpec_t; +typedef bit<48> macAddr_t; +typedef bit<32> ip4Addr_t; + +header ethernet_t { + macAddr_t dstAddr; + macAddr_t srcAddr; + bit<16> etherType; +} + +header srcRoute_t { + bit<1> bos; + bit<15> port; +} + +header ipv4_t { + bit<4> version; + bit<4> ihl; + bit<8> diffserv; + bit<16> totalLen; + bit<16> identification; + bit<3> flags; + bit<13> fragOffset; + bit<8> ttl; + bit<8> protocol; + bit<16> hdrChecksum; + ip4Addr_t srcAddr; + ip4Addr_t dstAddr; +} + +struct metadata { + /* empty */ +} + +struct headers { + ethernet_t ethernet; + srcRoute_t[MAX_HOPS] srcRoutes; + ipv4_t ipv4; +} + +/************************************************************************* +*********************** P A R S E R *********************************** +*************************************************************************/ + +parser MyParser(packet_in packet, + out headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + + state start { + transition parse_ethernet; + } + + state parse_ethernet { + packet.extract(hdr.ethernet); + transition select(hdr.ethernet.etherType) { + TYPE_SRCROUTING: parse_srcRouting; + default: accept; + } + } + + state parse_srcRouting { + packet.extract(hdr.srcRoutes.next); + transition select(hdr.srcRoutes.last.bos) { + 1: parse_ipv4; + default: parse_srcRouting; + } + } + + state parse_ipv4 { + packet.extract(hdr.ipv4); + transition accept; + } + +} + + +/************************************************************************* +************ C H E C K S U M V E R I F I C A T I O N ************* +*************************************************************************/ + +control MyVerifyChecksum(in headers hdr, inout metadata meta) { + apply { } +} + + +/************************************************************************* +************** I N G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyIngress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + + action drop() { + mark_to_drop(); + } + + action srcRoute_nhop() { + standard_metadata.egress_spec = (bit<9>)hdr.srcRoutes[0].port; + hdr.srcRoutes.pop_front(1); + } + + action srcRoute_finish() { + hdr.ethernet.etherType = TYPE_IPV4; + } + + action update_ttl(){ + hdr.ipv4.ttl = hdr.ipv4.ttl - 1; + } + + apply { + if (hdr.srcRoutes[0].isValid()){ + if (hdr.srcRoutes[0].bos == 1){ + srcRoute_finish(); + } + srcRoute_nhop(); + if (hdr.ipv4.isValid()){ + update_ttl(); + } + }else{ + drop(); + } + } +} + +/************************************************************************* +**************** E G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyEgress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + apply { } +} + +/************************************************************************* +************* C H E C K S U M C O M P U T A T I O N ************** +*************************************************************************/ + +control MyComputeChecksum(inout headers hdr, inout metadata meta) { + apply { } +} + +/************************************************************************* +*********************** D E P A R S E R ******************************* +*************************************************************************/ + +control MyDeparser(packet_out packet, in headers hdr) { + apply { + packet.emit(hdr.ethernet); + packet.emit(hdr.srcRoutes); + packet.emit(hdr.ipv4); + } +} + +/************************************************************************* +*********************** S W I T C H ******************************* +*************************************************************************/ + +V1Switch( +MyParser(), +MyVerifyChecksum(), +MyIngress(), +MyEgress(), +MyComputeChecksum(), +MyDeparser() +) main; \ No newline at end of file diff --git a/SIGCOMM_2017/exercises/source_routing/source_routing.p4 b/SIGCOMM_2017/exercises/source_routing/source_routing.p4 new file mode 100644 index 0000000..6c85b99 --- /dev/null +++ b/SIGCOMM_2017/exercises/source_routing/source_routing.p4 @@ -0,0 +1,191 @@ +/* -*- P4_16 -*- */ +#include +#include + +const bit<16> TYPE_IPV4 = 0x800; +const bit<16> TYPE_SRCROUTING = 0x1234; + +#define MAX_HOPS 9 + +/************************************************************************* +*********************** H E A D E R S *********************************** +*************************************************************************/ + +typedef bit<9> egressSpec_t; +typedef bit<48> macAddr_t; +typedef bit<32> ip4Addr_t; + +header ethernet_t { + macAddr_t dstAddr; + macAddr_t srcAddr; + bit<16> etherType; +} + +header srcRoute_t { + bit<1> bos; + bit<15> port; +} + +header ipv4_t { + bit<4> version; + bit<4> ihl; + bit<8> diffserv; + bit<16> totalLen; + bit<16> identification; + bit<3> flags; + bit<13> fragOffset; + bit<8> ttl; + bit<8> protocol; + bit<16> hdrChecksum; + ip4Addr_t srcAddr; + ip4Addr_t dstAddr; +} + +struct metadata { + /* empty */ +} + +struct headers { + ethernet_t ethernet; + srcRoute_t[MAX_HOPS] srcRoutes; + ipv4_t ipv4; +} + +/************************************************************************* +*********************** P A R S E R *********************************** +*************************************************************************/ + +parser MyParser(packet_in packet, + out headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + + + state start { + transition parse_ethernet; + } + + state parse_ethernet { + packet.extract(hdr.ethernet); + /* + * TODO: Modify the next line to select on hdr.ethernet.etherType + * If the value is TYPE_SRCROUTING transition to parse_srcRouting + * otherwise transition to accept. + */ + transition accept; + } + + state parse_srcRouting { + /* + * TODO: extract the next entry of hdr.srcRoutes + * while hdr.srcRoutes.last.bos is 0 transition to this state + * otherwise parse ipv4 + */ + transition accept; + } + + state parse_ipv4 { + packet.extract(hdr.ipv4); + transition accept; + } + +} + + +/************************************************************************* +************ C H E C K S U M V E R I F I C A T I O N ************* +*************************************************************************/ + +control MyVerifyChecksum(in headers hdr, inout metadata meta) { + apply { } +} + + +/************************************************************************* +************** I N G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyIngress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + + action drop() { + mark_to_drop(); + } + + action srcRoute_nhop() { + /* + * TODO: set standard_metadata.egress_spec + * to the port in hdr.srcRoutes[0] and + * pop an entry from hdr.srcRoutes + */ + } + + action srcRoute_finish() { + hdr.ethernet.etherType = TYPE_IPV4; + } + + action update_ttl(){ + hdr.ipv4.ttl = hdr.ipv4.ttl - 1; + } + + apply { + if (hdr.srcRoutes[0].isValid()){ + /* + * TODO: add logic to: + * - If final srcRoutes (top of stack has bos==1): + * - change etherType to IP + * - choose next hop and remove top of srcRoutes stack + */ + + if (hdr.ipv4.isValid()){ + update_ttl(); + } + }else{ + drop(); + } + } +} + +/************************************************************************* +**************** E G R E S S P R O C E S S I N G ******************* +*************************************************************************/ + +control MyEgress(inout headers hdr, + inout metadata meta, + inout standard_metadata_t standard_metadata) { + apply { } +} + +/************************************************************************* +************* C H E C K S U M C O M P U T A T I O N ************** +*************************************************************************/ + +control MyComputeChecksum(inout headers hdr, inout metadata meta) { + apply { } +} + +/************************************************************************* +*********************** D E P A R S E R ******************************* +*************************************************************************/ + +control MyDeparser(packet_out packet, in headers hdr) { + apply { + packet.emit(hdr.ethernet); + packet.emit(hdr.srcRoutes); + packet.emit(hdr.ipv4); + } +} + +/************************************************************************* +*********************** S W I T C H ******************************* +*************************************************************************/ + +V1Switch( +MyParser(), +MyVerifyChecksum(), +MyIngress(), +MyEgress(), +MyComputeChecksum(), +MyDeparser() +) main; \ No newline at end of file diff --git a/SIGCOMM_2017/utils/mininet/appcontroller.py b/SIGCOMM_2017/utils/mininet/appcontroller.py new file mode 100644 index 0000000..2cf9732 --- /dev/null +++ b/SIGCOMM_2017/utils/mininet/appcontroller.py @@ -0,0 +1,104 @@ +import subprocess + +from shortest_path import ShortestPath + +class AppController: + + def __init__(self, manifest=None, target=None, topo=None, net=None, links=None): + self.manifest = manifest + self.target = target + self.conf = manifest['targets'][target] + self.topo = topo + self.net = net + self.links = links + + def read_entries(self, filename): + entries = [] + with open(filename, 'r') as f: + for line in f: + line = line.strip() + if line == '': continue + entries.append(line) + return entries + + def add_entries(self, thrift_port=9090, sw=None, entries=None): + assert entries + if sw: thrift_port = sw.thrift_port + + print '\n'.join(entries) + p = subprocess.Popen(['simple_switch_CLI', '--thrift-port', str(thrift_port)], stdin=subprocess.PIPE) + p.communicate(input='\n'.join(entries)) + + def read_register(self, register, idx, thrift_port=9090, sw=None): + if sw: thrift_port = sw.thrift_port + p = subprocess.Popen(['simple_switch_CLI', '--thrift-port', str(thrift_port)], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + stdout, stderr = p.communicate(input="register_read %s %d" % (register, idx)) + reg_val = filter(lambda l: ' %s[%d]' % (register, idx) in l, stdout.split('\n'))[0].split('= ', 1)[1] + return long(reg_val) + + def start(self): + shortestpath = ShortestPath(self.links) + entries = {} + for sw in self.topo.switches(): + entries[sw] = [] + if 'switches' in self.conf and sw in self.conf['switches'] and 'entries' in self.conf['switches'][sw]: + extra_entries = self.conf['switches'][sw]['entries'] + if type(extra_entries) == list: # array of entries + entries[sw] += extra_entries + else: # path to file that contains entries + entries[sw] += self.read_entries(extra_entries) + #entries[sw] += [ + # 'table_set_default send_frame _drop', + # 'table_set_default forward _drop', + # 'table_set_default ipv4_lpm _drop'] + + for host_name in self.topo._host_links: + h = self.net.get(host_name) + for link in self.topo._host_links[host_name].values(): + sw = link['sw'] + #entries[sw].append('table_add send_frame rewrite_mac %d => %s' % (link['sw_port'], link['sw_mac'])) + #entries[sw].append('table_add forward set_dmac %s => %s' % (link['host_ip'], link['host_mac'])) + #entries[sw].append('table_add ipv4_lpm set_nhop %s/32 => %s %d' % (link['host_ip'], link['host_ip'], link['sw_port'])) + iface = h.intfNames()[link['idx']] + # use mininet to set ip and mac to let it know the change + h.setIP(link['host_ip'], 24) + h.setMAC(link['host_mac']) + #h.cmd('ifconfig %s %s hw ether %s' % (iface, link['host_ip'], link['host_mac'])) + h.cmd('arp -i %s -s %s %s' % (iface, link['sw_ip'], link['sw_mac'])) + h.cmd('ethtool --offload %s rx off tx off' % iface) + h.cmd('ip route add %s dev %s' % (link['sw_ip'], iface)) + h.setDefaultRoute("via %s" % link['sw_ip']) + + for h in self.net.hosts: + h_link = self.topo._host_links[h.name].values()[0] + for sw in self.net.switches: + path = shortestpath.get(sw.name, h.name, exclude=lambda n: n[0]=='h') + if not path: continue + if not path[1][0] == 's': continue # next hop is a switch + sw_link = self.topo._sw_links[sw.name][path[1]] + #entries[sw.name].append('table_add send_frame rewrite_mac %d => %s' % (sw_link[0]['port'], sw_link[0]['mac'])) + #entries[sw.name].append('table_add forward set_dmac %s => %s' % (h_link['host_ip'], sw_link[1]['mac'])) + #entries[sw.name].append('table_add ipv4_lpm set_nhop %s/32 => %s %d' % (h_link['host_ip'], h_link['host_ip'], sw_link[0]['port'])) + + for h2 in self.net.hosts: + if h == h2: continue + path = shortestpath.get(h.name, h2.name, exclude=lambda n: n[0]=='h') + if not path: continue + h_link = self.topo._host_links[h.name][path[1]] + h2_link = self.topo._host_links[h2.name].values()[0] + h.cmd('ip route add %s via %s' % (h2_link['host_ip'], h_link['sw_ip'])) + + + print "**********" + print "Configuring entries in p4 tables" + for sw_name in entries: + print + print "Configuring switch... %s" % sw_name + sw = self.net.get(sw_name) + if entries[sw_name]: + self.add_entries(sw=sw, entries=entries[sw_name]) + print "Configuration complete." + print "**********" + + def stop(self): + pass diff --git a/SIGCOMM_2017/utils/mininet/apptopo.py b/SIGCOMM_2017/utils/mininet/apptopo.py new file mode 100644 index 0000000..3491a3d --- /dev/null +++ b/SIGCOMM_2017/utils/mininet/apptopo.py @@ -0,0 +1,70 @@ +from mininet.topo import Topo + +class AppTopo(Topo): + + def __init__(self, links, latencies={}, manifest=None, target=None, + log_dir="/tmp", bws={}, **opts): + Topo.__init__(self, **opts) + + nodes = sum(map(list, zip(*links)), []) + host_names = sorted(list(set(filter(lambda n: n[0] == 'h', nodes)))) + sw_names = sorted(list(set(filter(lambda n: n[0] == 's', nodes)))) + sw_ports = dict([(sw, []) for sw in sw_names]) + + self._host_links = {} + self._sw_links = dict([(sw, {}) for sw in sw_names]) + + for sw_name in sw_names: + self.addSwitch(sw_name, log_file="%s/%s.log" %(log_dir, sw_name)) + + for host_name in host_names: + host_num = int(host_name[1:]) + + self.addHost(host_name) + + self._host_links[host_name] = {} + host_links = filter(lambda l: l[0]==host_name or l[1]==host_name, links) + + sw_idx = 0 + for link in host_links: + sw = link[0] if link[0] != host_name else link[1] + sw_num = int(sw[1:]) + assert sw[0]=='s', "Hosts should be connected to switches, not " + str(sw) + host_ip = "10.0.%d.%d" % (sw_num, host_num) + host_mac = '00:00:00:00:%02x:%02x' % (sw_num, host_num) + delay_key = ''.join([host_name, sw]) + delay = latencies[delay_key] if delay_key in latencies else '0ms' + bw = bws[delay_key] if delay_key in bws else None + sw_ports[sw].append(host_name) + self._host_links[host_name][sw] = dict( + idx=sw_idx, + host_mac = host_mac, + host_ip = host_ip, + sw = sw, + sw_mac = "00:00:00:00:%02x:%02x" % (sw_num, host_num), + sw_ip = "10.0.%d.%d" % (sw_num, 254), + sw_port = sw_ports[sw].index(host_name)+1 + ) + self.addLink(host_name, sw, delay=delay, bw=bw, + addr1=host_mac, addr2=self._host_links[host_name][sw]['sw_mac']) + sw_idx += 1 + + for link in links: # only check switch-switch links + sw1, sw2 = link + if sw1[0] != 's' or sw2[0] != 's': continue + + delay_key = ''.join(sorted([sw1, sw2])) + delay = latencies[delay_key] if delay_key in latencies else '0ms' + bw = bws[delay_key] if delay_key in bws else None + + self.addLink(sw1, sw2, delay=delay, bw=bw)#, max_queue_size=10) + sw_ports[sw1].append(sw2) + sw_ports[sw2].append(sw1) + + sw1_num, sw2_num = int(sw1[1:]), int(sw2[1:]) + sw1_port = dict(mac="00:00:00:%02x:%02x:00" % (sw1_num, sw2_num), port=sw_ports[sw1].index(sw2)+1) + sw2_port = dict(mac="00:00:00:%02x:%02x:00" % (sw2_num, sw1_num), port=sw_ports[sw2].index(sw1)+1) + + self._sw_links[sw1][sw2] = [sw1_port, sw2_port] + self._sw_links[sw2][sw1] = [sw2_port, sw1_port] + diff --git a/SIGCOMM_2017/utils/mininet/multi_switch_mininet.py b/SIGCOMM_2017/utils/mininet/multi_switch_mininet.py new file mode 100755 index 0000000..0bb406f --- /dev/null +++ b/SIGCOMM_2017/utils/mininet/multi_switch_mininet.py @@ -0,0 +1,243 @@ +#!/usr/bin/env python2 + +# Copyright 2013-present Barefoot Networks, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import signal +import os +import sys +import subprocess +import argparse +import json +import importlib +import re +from time import sleep + +from mininet.net import Mininet +from mininet.topo import Topo +from mininet.link import TCLink +from mininet.log import setLogLevel, info +from mininet.cli import CLI + +from p4_mininet import P4Switch, P4Host +import apptopo +import appcontroller + +parser = argparse.ArgumentParser(description='Mininet demo') +parser.add_argument('--behavioral-exe', help='Path to behavioral executable', + type=str, action="store", required=True) +parser.add_argument('--thrift-port', help='Thrift server port for table updates', + type=int, action="store", default=9090) +parser.add_argument('--bmv2-log', help='verbose messages in log file', action="store_true") +parser.add_argument('--cli', help="start the mininet cli", action="store_true") +parser.add_argument('--auto-control-plane', help='enable automatic control plane population', action="store_true") +parser.add_argument('--json', help='Path to JSON config file', + type=str, action="store", required=True) +parser.add_argument('--pcap-dump', help='Dump packets on interfaces to pcap files', + action="store_true") +parser.add_argument('--manifest', '-m', help='Path to manifest file', + type=str, action="store", required=True) +parser.add_argument('--target', '-t', help='Target in manifest file to run', + type=str, action="store", required=True) +parser.add_argument('--log-dir', '-l', help='Location to save output to', + type=str, action="store", required=True) +parser.add_argument('--cli-message', help='Message to print before starting CLI', + type=str, action="store", required=False, default=False) + + +args = parser.parse_args() + + +next_thrift_port = args.thrift_port + +def run_command(command): + return os.WEXITSTATUS(os.system(command)) + +def configureP4Switch(**switch_args): + class ConfiguredP4Switch(P4Switch): + def __init__(self, *opts, **kwargs): + global next_thrift_port + kwargs.update(switch_args) + kwargs['thrift_port'] = next_thrift_port + next_thrift_port += 1 + P4Switch.__init__(self, *opts, **kwargs) + return ConfiguredP4Switch + + +def main(): + + with open(args.manifest, 'r') as f: + manifest = json.load(f) + + conf = manifest['targets'][args.target] + params = conf['parameters'] if 'parameters' in conf else {} + + os.environ.update(dict(map(lambda (k,v): (k, str(v)), params.iteritems()))) + + def formatParams(s): + for param in params: + s = re.sub('\$'+param+'(\W|$)', str(params[param]) + r'\1', s) + s = s.replace('${'+param+'}', str(params[param])) + return s + + AppTopo = apptopo.AppTopo + AppController = appcontroller.AppController + + if 'topo_module' in conf: + sys.path.insert(0, os.path.dirname(args.manifest)) + topo_module = importlib.import_module(conf['topo_module']) + AppTopo = topo_module.CustomAppTopo + + if 'controller_module' in conf: + sys.path.insert(0, os.path.dirname(args.manifest)) + controller_module = importlib.import_module(conf['controller_module']) + AppController = controller_module.CustomAppController + + if not os.path.isdir(args.log_dir): + if os.path.exists(args.log_dir): raise Exception('Log dir exists and is not a dir') + os.mkdir(args.log_dir) + os.environ['P4APP_LOGDIR'] = args.log_dir + + + links = [l[:2] for l in conf['links']] + latencies = dict([(''.join(sorted(l[:2])), l[2]) for l in conf['links'] if len(l)>=3]) + bws = dict([(''.join(sorted(l[:2])), l[3]) for l in conf['links'] if len(l)>=4]) + + for host_name in sorted(conf['hosts'].keys()): + host = conf['hosts'][host_name] + if 'latency' not in host: continue + for a, b in links: + if a != host_name and b != host_name: continue + other = a if a != host_name else b + latencies[host_name+other] = host['latency'] + + for l in latencies: + if isinstance(latencies[l], (str, unicode)): + latencies[l] = formatParams(latencies[l]) + else: + latencies[l] = str(latencies[l]) + "ms" + + bmv2_log = args.bmv2_log or ('bmv2_log' in conf and conf['bmv2_log']) + pcap_dump = args.pcap_dump or ('pcap_dump' in conf and conf['pcap_dump']) + + topo = AppTopo(links, latencies, manifest=manifest, target=args.target, + log_dir=args.log_dir, bws=bws) + switchClass = configureP4Switch( + sw_path=args.behavioral_exe, + json_path=args.json, + log_console=bmv2_log, + pcap_dump=pcap_dump) + net = Mininet(topo = topo, + link = TCLink, + host = P4Host, + switch = switchClass, + controller = None) + net.start() + + sleep(1) + + controller = None + if args.auto_control_plane or 'controller_module' in conf: + controller = AppController(manifest=manifest, target=args.target, + topo=topo, net=net, links=links) + controller.start() + + + for h in net.hosts: + h.describe() + + if args.cli_message is not None: + with open(args.cli_message, 'r') as message_file: + print message_file.read() + + if args.cli or ('cli' in conf and conf['cli']): + CLI(net) + + stdout_files = dict() + return_codes = [] + host_procs = [] + + + def formatCmd(cmd): + for h in net.hosts: + cmd = cmd.replace(h.name, h.defaultIntf().updateIP()) + return cmd + + def _wait_for_exit(p, host): + print p.communicate() + if p.returncode is None: + p.wait() + print p.communicate() + return_codes.append(p.returncode) + if host_name in stdout_files: + stdout_files[host_name].flush() + stdout_files[host_name].close() + + print '\n'.join(map(lambda (k,v): "%s: %s"%(k,v), params.iteritems())) + '\n' + + for host_name in sorted(conf['hosts'].keys()): + host = conf['hosts'][host_name] + if 'cmd' not in host: continue + + h = net.get(host_name) + stdout_filename = os.path.join(args.log_dir, h.name + '.stdout') + stdout_files[h.name] = open(stdout_filename, 'w') + cmd = formatCmd(host['cmd']) + print h.name, cmd + p = h.popen(cmd, stdout=stdout_files[h.name], shell=True, preexec_fn=os.setpgrp) + if 'startup_sleep' in host: sleep(host['startup_sleep']) + + if 'wait' in host and host['wait']: + _wait_for_exit(p, host_name) + else: + host_procs.append((p, host_name)) + + for p, host_name in host_procs: + if 'wait' in conf['hosts'][host_name] and conf['hosts'][host_name]['wait']: + _wait_for_exit(p, host_name) + + + for p, host_name in host_procs: + if 'wait' in conf['hosts'][host_name] and conf['hosts'][host_name]['wait']: + continue + if p.returncode is None: + run_command('pkill -INT -P %d' % p.pid) + sleep(0.2) + rc = run_command('pkill -0 -P %d' % p.pid) # check if it's still running + if rc == 0: # the process group is still running, send TERM + sleep(1) # give it a little more time to exit gracefully + run_command('pkill -TERM -P %d' % p.pid) + _wait_for_exit(p, host_name) + + if 'after' in conf and 'cmd' in conf['after']: + cmds = conf['after']['cmd'] if type(conf['after']['cmd']) == list else [conf['after']['cmd']] + for cmd in cmds: + os.system(cmd) + + if controller: controller.stop() + + net.stop() + +# if bmv2_log: +# os.system('bash -c "cp /tmp/p4s.s*.log \'%s\'"' % args.log_dir) +# if pcap_dump: +# os.system('bash -c "cp *.pcap \'%s\'"' % args.log_dir) + + bad_codes = [rc for rc in return_codes if rc != 0] + if len(bad_codes): sys.exit(1) + +if __name__ == '__main__': + setLogLevel( 'info' ) + main() diff --git a/SIGCOMM_2017/utils/mininet/p4_mininet.py b/SIGCOMM_2017/utils/mininet/p4_mininet.py new file mode 100644 index 0000000..8abe79f --- /dev/null +++ b/SIGCOMM_2017/utils/mininet/p4_mininet.py @@ -0,0 +1,161 @@ +# Copyright 2013-present Barefoot Networks, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +from mininet.net import Mininet +from mininet.node import Switch, Host +from mininet.log import setLogLevel, info, error, debug +from mininet.moduledeps import pathCheck +from sys import exit +from time import sleep +import os +import tempfile +import socket + +class P4Host(Host): + def config(self, **params): + r = super(P4Host, self).config(**params) + + for off in ["rx", "tx", "sg"]: + cmd = "/sbin/ethtool --offload %s %s off" % (self.defaultIntf().name, off) + self.cmd(cmd) + + # disable IPv6 + self.cmd("sysctl -w net.ipv6.conf.all.disable_ipv6=1") + self.cmd("sysctl -w net.ipv6.conf.default.disable_ipv6=1") + self.cmd("sysctl -w net.ipv6.conf.lo.disable_ipv6=1") + + return r + + def describe(self, sw_addr=None, sw_mac=None): + print "**********" + print "Network configuration for: %s" % self.name + print "Default interface: %s\t%s\t%s" %( + self.defaultIntf().name, + self.defaultIntf().IP(), + self.defaultIntf().MAC() + ) + if sw_addr is not None or sw_mac is not None: + print "Default route to switch: %s (%s)" % (sw_addr, sw_mac) + print "**********" + +class P4Switch(Switch): + """P4 virtual switch""" + device_id = 0 + + def __init__(self, name, sw_path = None, json_path = None, + log_file = None, + thrift_port = None, + pcap_dump = False, + log_console = False, + verbose = False, + device_id = None, + enable_debugger = False, + **kwargs): + Switch.__init__(self, name, **kwargs) + assert(sw_path) + assert(json_path) + # make sure that the provided sw_path is valid + pathCheck(sw_path) + # make sure that the provided JSON file exists + if not os.path.isfile(json_path): + error("Invalid JSON file.\n") + exit(1) + self.sw_path = sw_path + self.json_path = json_path + self.verbose = verbose + self.log_file = log_file + if self.log_file is None: + self.log_file = "/tmp/p4s.{}.log".format(self.name) + self.output = open(self.log_file, 'w') + self.thrift_port = thrift_port + self.pcap_dump = pcap_dump + self.enable_debugger = enable_debugger + self.log_console = log_console + if device_id is not None: + self.device_id = device_id + P4Switch.device_id = max(P4Switch.device_id, device_id) + else: + self.device_id = P4Switch.device_id + P4Switch.device_id += 1 + self.nanomsg = "ipc:///tmp/bm-{}-log.ipc".format(self.device_id) + + @classmethod + def setup(cls): + pass + + def check_switch_started(self, pid): + """While the process is running (pid exists), we check if the Thrift + server has been started. If the Thrift server is ready, we assume that + the switch was started successfully. This is only reliable if the Thrift + server is started at the end of the init process""" + while True: + if not os.path.exists(os.path.join("/proc", str(pid))): + return False + sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + sock.settimeout(0.5) + result = sock.connect_ex(("localhost", self.thrift_port)) + if result == 0: + return True + + def start(self, controllers): + "Start up a new P4 switch" + info("Starting P4 switch {}.\n".format(self.name)) + args = [self.sw_path] + for port, intf in self.intfs.items(): + if not intf.IP(): + args.extend(['-i', str(port) + "@" + intf.name]) + if self.pcap_dump: + args.append("--pcap") + # args.append("--useFiles") + if self.thrift_port: + args.extend(['--thrift-port', str(self.thrift_port)]) + if self.nanomsg: + args.extend(['--nanolog', self.nanomsg]) + args.extend(['--device-id', str(self.device_id)]) + P4Switch.device_id += 1 + args.append(self.json_path) + if self.enable_debugger: + args.append("--debugger") + if self.log_console: + args.append("--log-console") + info(' '.join(args) + "\n") + + pid = None + with tempfile.NamedTemporaryFile() as f: + # self.cmd(' '.join(args) + ' > /dev/null 2>&1 &') + self.cmd(' '.join(args) + ' >' + self.log_file + ' 2>&1 & echo $! >> ' + f.name) + pid = int(f.read()) + debug("P4 switch {} PID is {}.\n".format(self.name, pid)) + sleep(1) + if not self.check_switch_started(pid): + error("P4 switch {} did not start correctly." + "Check the switch log file.\n".format(self.name)) + exit(1) + info("P4 switch {} has been started.\n".format(self.name)) + + def stop(self): + "Terminate P4 switch." + self.output.flush() + self.cmd('kill %' + self.sw_path) + self.cmd('wait') + self.deleteIntfs() + + def attach(self, intf): + "Connect a data port" + assert(0) + + def detach(self, intf): + "Disconnect a data port" + assert(0) diff --git a/SIGCOMM_2017/utils/mininet/shortest_path.py b/SIGCOMM_2017/utils/mininet/shortest_path.py new file mode 100644 index 0000000..971b1b4 --- /dev/null +++ b/SIGCOMM_2017/utils/mininet/shortest_path.py @@ -0,0 +1,78 @@ +class ShortestPath: + + def __init__(self, edges=[]): + self.neighbors = {} + for edge in edges: + self.addEdge(*edge) + + def addEdge(self, a, b): + if a not in self.neighbors: self.neighbors[a] = [] + if b not in self.neighbors[a]: self.neighbors[a].append(b) + + if b not in self.neighbors: self.neighbors[b] = [] + if a not in self.neighbors[b]: self.neighbors[b].append(a) + + def get(self, a, b, exclude=lambda node: False): + # Shortest path from a to b + return self._recPath(a, b, [], exclude) + + def _recPath(self, a, b, visited, exclude): + if a == b: return [a] + new_visited = visited + [a] + paths = [] + for neighbor in self.neighbors[a]: + if neighbor in new_visited: continue + if exclude(neighbor) and neighbor != b: continue + path = self._recPath(neighbor, b, new_visited, exclude) + if path: paths.append(path) + + paths.sort(key=len) + return [a] + paths[0] if len(paths) else None + +if __name__ == '__main__': + + edges = [ + (1, 2), + (1, 3), + (1, 5), + (2, 4), + (3, 4), + (3, 5), + (3, 6), + (4, 6), + (5, 6), + (7, 8) + + ] + sp = ShortestPath(edges) + + assert sp.get(1, 1) == [1] + assert sp.get(2, 2) == [2] + + assert sp.get(1, 2) == [1, 2] + assert sp.get(2, 1) == [2, 1] + + assert sp.get(1, 3) == [1, 3] + assert sp.get(3, 1) == [3, 1] + + assert sp.get(4, 6) == [4, 6] + assert sp.get(6, 4) == [6, 4] + + assert sp.get(2, 6) == [2, 4, 6] + assert sp.get(6, 2) == [6, 4, 2] + + assert sp.get(1, 6) in [[1, 3, 6], [1, 5, 6]] + assert sp.get(6, 1) in [[6, 3, 1], [6, 5, 1]] + + assert sp.get(2, 5) == [2, 1, 5] + assert sp.get(5, 2) == [5, 1, 2] + + assert sp.get(4, 5) in [[4, 3, 5], [4, 6, 5]] + assert sp.get(5, 4) in [[5, 3, 4], [6, 6, 4]] + + assert sp.get(7, 8) == [7, 8] + assert sp.get(8, 7) == [8, 7] + + assert sp.get(1, 7) == None + assert sp.get(7, 2) == None + diff --git a/SIGCOMM_2017/utils/mininet/single_switch_mininet.py b/SIGCOMM_2017/utils/mininet/single_switch_mininet.py new file mode 100755 index 0000000..e2e7636 --- /dev/null +++ b/SIGCOMM_2017/utils/mininet/single_switch_mininet.py @@ -0,0 +1,133 @@ +#!/usr/bin/env python2 + +# Copyright 2013-present Barefoot Networks, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +from mininet.net import Mininet +from mininet.topo import Topo +from mininet.log import setLogLevel, info +from mininet.cli import CLI + +from p4_mininet import P4Switch, P4Host + +import argparse +from subprocess import PIPE, Popen +from time import sleep + +parser = argparse.ArgumentParser(description='Mininet demo') +parser.add_argument('--behavioral-exe', help='Path to behavioral executable', + type=str, action="store", required=True) +parser.add_argument('--thrift-port', help='Thrift server port for table updates', + type=int, action="store", default=9090) +parser.add_argument('--num-hosts', help='Number of hosts to connect to switch', + type=int, action="store", default=2) +parser.add_argument('--mode', choices=['l2', 'l3'], type=str, default='l3') +parser.add_argument('--json', help='Path to JSON config file', + type=str, action="store", required=True) +parser.add_argument('--log-file', help='Path to write the switch log file', + type=str, action="store", required=False) +parser.add_argument('--pcap-dump', help='Dump packets on interfaces to pcap files', + type=str, action="store", required=False, default=False) +parser.add_argument('--switch-config', help='simple_switch_CLI script to configure switch', + type=str, action="store", required=False, default=False) +parser.add_argument('--cli-message', help='Message to print before starting CLI', + type=str, action="store", required=False, default=False) + +args = parser.parse_args() + + +class SingleSwitchTopo(Topo): + "Single switch connected to n (< 256) hosts." + def __init__(self, sw_path, json_path, log_file, + thrift_port, pcap_dump, n, **opts): + # Initialize topology and default options + Topo.__init__(self, **opts) + + switch = self.addSwitch('s1', + sw_path = sw_path, + json_path = json_path, + log_console = True, + log_file = log_file, + thrift_port = thrift_port, + enable_debugger = False, + pcap_dump = pcap_dump) + + for h in xrange(n): + host = self.addHost('h%d' % (h + 1), + ip = "10.0.%d.10/24" % h, + mac = '00:04:00:00:00:%02x' %h) + print "Adding host", str(host) + self.addLink(host, switch) + +def main(): + num_hosts = args.num_hosts + mode = args.mode + + topo = SingleSwitchTopo(args.behavioral_exe, + args.json, + args.log_file, + args.thrift_port, + args.pcap_dump, + num_hosts) + net = Mininet(topo = topo, + host = P4Host, + switch = P4Switch, + controller = None) + net.start() + + + sw_mac = ["00:aa:bb:00:00:%02x" % n for n in xrange(num_hosts)] + + sw_addr = ["10.0.%d.1" % n for n in xrange(num_hosts)] + + for n in xrange(num_hosts): + h = net.get('h%d' % (n + 1)) + if mode == "l2": + h.setDefaultRoute("dev %s" % h.defaultIntf().name) + else: + h.setARP(sw_addr[n], sw_mac[n]) + h.setDefaultRoute("dev %s via %s" % (h.defaultIntf().name, sw_addr[n])) + + for n in xrange(num_hosts): + h = net.get('h%d' % (n + 1)) + h.describe(sw_addr[n], sw_mac[n]) + + sleep(1) + + if args.switch_config is not None: + print + print "Reading switch configuration script:", args.switch_config + with open(args.switch_config, 'r') as config_file: + switch_config = config_file.read() + + print "Configuring switch..." + proc = Popen(["simple_switch_CLI"], stdin=PIPE) + proc.communicate(input=switch_config) + + print "Configuration complete." + print + + print "Ready !" + + if args.cli_message is not None: + with open(args.cli_message, 'r') as message_file: + print message_file.read() + + CLI( net ) + net.stop() + +if __name__ == '__main__': + setLogLevel( 'info' ) + main() diff --git a/SIGCOMM_2017/utils/p4apprunner.py b/SIGCOMM_2017/utils/p4apprunner.py new file mode 100755 index 0000000..36b9eea --- /dev/null +++ b/SIGCOMM_2017/utils/p4apprunner.py @@ -0,0 +1,320 @@ +#!/usr/bin/env python2 +# Copyright 2013-present Barefoot Networks, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from __future__ import print_function + +import argparse +from collections import OrderedDict +import json +import os +import sys +import tarfile + +parser = argparse.ArgumentParser(description='p4apprunner') +parser.add_argument('--build-dir', help='Directory to build in.', + type=str, action='store', required=False, default='/tmp') +parser.add_argument('--quiet', help='Suppress log messages.', + action='store_true', required=False, default=False) +parser.add_argument('--manifest', help='Path to manifest file.', + type=str, action='store', required=False, default='./p4app.json') +parser.add_argument('app', help='.p4app package to run.', type=str) +parser.add_argument('target', help=('Target to run. Defaults to the first target ' + 'in the package.'), + nargs='?', type=str) + +args = parser.parse_args() + +def log(*items): + if args.quiet != True: + print(*items) + +def log_error(*items): + print(*items, file=sys.stderr) + +def run_command(command): + log('>', command) + return os.WEXITSTATUS(os.system(command)) + +class Manifest: + def __init__(self, program_file, language, target, target_config): + self.program_file = program_file + self.language = language + self.target = target + self.target_config = target_config + +def read_manifest(manifest_file): + manifest = json.load(manifest_file, object_pairs_hook=OrderedDict) + + if 'program' not in manifest: + log_error('No program defined in manifest.') + sys.exit(1) + program_file = manifest['program'] + + if 'language' not in manifest: + log_error('No language defined in manifest.') + sys.exit(1) + language = manifest['language'] + + if 'targets' not in manifest or len(manifest['targets']) < 1: + log_error('No targets defined in manifest.') + sys.exit(1) + + if args.target is not None: + chosen_target = args.target + elif 'default-target' in manifest: + chosen_target = manifest['default-target'] + else: + chosen_target = manifest['targets'].keys()[0] + + if chosen_target not in manifest['targets']: + log_error('Target not found in manifest:', chosen_target) + sys.exit(1) + + return Manifest(program_file, language, chosen_target, manifest['targets'][chosen_target]) + + +def run_compile_bmv2(manifest): + if 'run-before-compile' in manifest.target_config: + commands = manifest.target_config['run-before-compile'] + if not isinstance(commands, list): + log_error('run-before-compile should be a list:', commands) + sys.exit(1) + for command in commands: + run_command(command) + + compiler_args = [] + + if manifest.language == 'p4-14': + compiler_args.append('--p4v 14') + elif manifest.language == 'p4-16': + compiler_args.append('--p4v 16') + else: + log_error('Unknown language:', manifest.language) + sys.exit(1) + + if 'compiler-flags' in manifest.target_config: + flags = manifest.target_config['compiler-flags'] + if not isinstance(flags, list): + log_error('compiler-flags should be a list:', flags) + sys.exit(1) + compiler_args.extend(flags) + + # Compile the program. + output_file = manifest.program_file + '.json' + compiler_args.append('"%s"' % manifest.program_file) + compiler_args.append('-o "%s"' % output_file) + rv = run_command('p4c-bm2-ss %s' % ' '.join(compiler_args)) + + if 'run-after-compile' in manifest.target_config: + commands = manifest.target_config['run-after-compile'] + if not isinstance(commands, list): + log_error('run-after-compile should be a list:', commands) + sys.exit(1) + for command in commands: + run_command(command) + + if rv != 0: + log_error('Compile failed.') + sys.exit(1) + + return output_file + +def run_mininet(manifest): + output_file = run_compile_bmv2(manifest) + + # Run the program using the BMV2 Mininet simple switch. + switch_args = [] + + # We'll place the switch's log file in current (build) folder. + cwd = os.getcwd() + log_file = os.path.join(cwd, manifest.program_file + '.log') + print ("*** Log file %s" % log_file) + switch_args.append('--log-file "%s"' % log_file) + + pcap_dir = os.path.join(cwd) + print ("*** Pcap folder %s" % pcap_dir) + switch_args.append('--pcap-dump "%s" '% pcap_dir) + + # Generate a message that will be printed by the Mininet CLI to make + # interacting with the simple switch a little easier. + message_file = 'mininet_message.txt' + with open(message_file, 'w') as message: + + print(file=message) + print('======================================================================', + file=message) + print('Welcome to the BMV2 Mininet CLI!', file=message) + print('======================================================================', + file=message) + print('Your P4 program is installed into the BMV2 software switch', file=message) + print('and your initial configuration is loaded. You can interact', file=message) + print('with the network using the mininet CLI below.', file=message) + print(file=message) + print('To inspect or change the switch configuration, connect to', file=message) + print('its CLI from your host operating system using this command:', file=message) + print(' simple_switch_CLI', file=message) + print(file=message) + print('To view the switch log, run this command from your host OS:', file=message) + print(' tail -f %s' % log_file, file=message) + print(file=message) + print('To view the switch output pcap, check the pcap files in %s:' % pcap_dir, file=message) + print(' for example run: sudo tcpdump -xxx -r s1-eth1.pcap', file=message) + print(file=message) +# print('To run the switch debugger, run this command from your host OS:', file=message) +# print(' bm_p4dbg' , file=message) +# print(file=message) + + switch_args.append('--cli-message "%s"' % message_file) + + if 'num-hosts' in manifest.target_config: + switch_args.append('--num-hosts %s' % manifest.target_config['num-hosts']) + + if 'switch-config' in manifest.target_config: + switch_args.append('--switch-config "%s"' % manifest.target_config['switch-config']) + + switch_args.append('--behavioral-exe "%s"' % 'simple_switch') + switch_args.append('--json "%s"' % output_file) + + program = '"%s/mininet/single_switch_mininet.py"' % sys.path[0] + return run_command('python2 %s %s' % (program, ' '.join(switch_args))) + +def run_multiswitch(manifest): + output_file = run_compile_bmv2(manifest) + + script_args = [] + cwd = os.getcwd() + log_dir = os.path.join(cwd, cwd + '/logs') + print ("*** Log directory %s" % log_dir) + script_args.append('--log-dir "%s"' % log_dir) + pcap_dir = os.path.join(cwd) + print ("*** Pcap directory %s" % cwd) + script_args.append('--manifest "%s"' % args.manifest) + script_args.append('--target "%s"' % manifest.target) + if 'auto-control-plane' in manifest.target_config and manifest.target_config['auto-control-plane']: + script_args.append('--auto-control-plane' ) + script_args.append('--behavioral-exe "%s"' % 'simple_switch') + script_args.append('--json "%s"' % output_file) + #script_args.append('--cli') + + # Generate a message that will be printed by the Mininet CLI to make + # interacting with the simple switch a little easier. + message_file = 'mininet_message.txt' + with open(message_file, 'w') as message: + + print(file=message) + print('======================================================================', + file=message) + print('Welcome to the BMV2 Mininet CLI!', file=message) + print('======================================================================', + file=message) + print('Your P4 program is installed into the BMV2 software switch', file=message) + print('and your initial configuration is loaded. You can interact', file=message) + print('with the network using the mininet CLI below.', file=message) + print(file=message) + print('To inspect or change the switch configuration, connect to', file=message) + print('its CLI from your host operating system using this command:', file=message) + print(' simple_switch_CLI --thrift-port ', file=message) + print(file=message) + print('To view a switch log, run this command from your host OS:', file=message) + print(' tail -f %s/.log' % log_dir, file=message) + print(file=message) + print('To view the switch output pcap, check the pcap files in %s:' % pcap_dir, file=message) + print(' for example run: sudo tcpdump -xxx -r s1-eth1.pcap', file=message) + print(file=message) +# print('To run the switch debugger, run this command from your host OS:', file=message) +# print(' bm_p4dbg' , file=message) +# print(file=message) + + script_args.append('--cli-message "%s"' % message_file) + + program = '"%s/mininet/multi_switch_mininet.py"' % sys.path[0] + return run_command('python2 %s %s' % (program, ' '.join(script_args))) + +def run_stf(manifest): + output_file = run_compile_bmv2(manifest) + + if not 'test' in manifest.target_config: + log_error('No STF test file provided.') + sys.exit(1) + stf_file = manifest.target_config['test'] + + # Run the program using the BMV2 STF interpreter. + stf_args = [] + stf_args.append('-v') + stf_args.append(os.path.join(args.build_dir, output_file)) + stf_args.append(os.path.join(args.build_dir, stf_file)) + + program = '"%s/stf/bmv2stf.py"' % sys.path[0] + rv = run_command('python2 %s %s' % (program, ' '.join(stf_args))) + if rv != 0: + sys.exit(1) + return rv + +def run_custom(manifest): + output_file = run_compile_bmv2(manifest) + python_path = 'PYTHONPATH=$PYTHONPATH:/scripts/mininet/' + script_args = [] + script_args.append('--behavioral-exe "%s"' % 'simple_switch') + script_args.append('--json "%s"' % output_file) + script_args.append('--cli "%s"' % 'simple_switch_CLI') + if not 'program' in manifest.target_config: + log_error('No mininet program file provided.') + sys.exit(1) + program = manifest.target_config['program'] + rv = run_command('%s python2 %s %s' % (python_path, program, ' '.join(script_args))) + + if rv != 0: + sys.exit(1) + return rv + +def main(): + log('Entering build directory.') + os.chdir(args.build_dir) + + # A '.p4app' package is really just a '.tar.gz' archive. Extract it so we + # can process its contents. + log('Extracting package.') + tar = tarfile.open(args.app) + tar.extractall() + tar.close() + + log('Reading package manifest.') + with open(args.manifest, 'r') as manifest_file: + manifest = read_manifest(manifest_file) + + # Dispatch to the backend implementation for this target. + backend = manifest.target + if 'use' in manifest.target_config: + backend = manifest.target_config['use'] + + if backend == 'mininet': + rc = run_mininet(manifest) + elif backend == 'multiswitch': + rc = run_multiswitch(manifest) + elif backend == 'stf': + rc = run_stf(manifest) + elif backend == 'custom': + rc = run_custom(manifest) + elif backend == 'compile-bmv2': + run_compile_bmv2(manifest) + rc = 0 + else: + log_error('Target specifies unknown backend:', backend) + sys.exit(1) + + sys.exit(rc) + +if __name__ == '__main__': + main() diff --git a/SIGCOMM_2017/vm/Vagrantfile b/SIGCOMM_2017/vm/Vagrantfile new file mode 100644 index 0000000..5498b02 --- /dev/null +++ b/SIGCOMM_2017/vm/Vagrantfile @@ -0,0 +1,17 @@ +# -*- mode: ruby -*- +# vi: set ft=ruby : + +Vagrant.configure(2) do |config| + config.vm.box = "bento/ubuntu-16.04" + config.vm.provider "virtualbox" do |vb| + vb.gui = true + vb.memory = "2048" + vb.customize ["modifyvm", :id, "--cableconnected1", "on"] + end + config.vm.synced_folder '.', '/vagrant', disabled: true + config.vm.hostname = "p4" + config.vm.provision "file", source: "p4-logo.png", destination: "/home/vagrant/p4-logo.png" + config.vm.provision "file", source: "p4_16-mode.el", destination: "/home/vagrant/p4_16-mode.el" + config.vm.provision "shell", path: "root-bootstrap.sh" + config.vm.provision "shell", privileged: false, path: "user-bootstrap.sh" +end diff --git a/SIGCOMM_2017/vm/p4-logo.png b/SIGCOMM_2017/vm/p4-logo.png new file mode 100644 index 0000000..cc36615 Binary files /dev/null and b/SIGCOMM_2017/vm/p4-logo.png differ diff --git a/SIGCOMM_2017/vm/p4_16-mode.el b/SIGCOMM_2017/vm/p4_16-mode.el new file mode 100644 index 0000000..7625d90 --- /dev/null +++ b/SIGCOMM_2017/vm/p4_16-mode.el @@ -0,0 +1,222 @@ +;;; p4_16-mode.el --- Support for the P4_16 programming language + +;; Copyright (C) 2016- Barefoot Networks +;; Author: Vladimir Gurevich +;; Maintainer: Vladimir Gurevich +;; Created: 15 April 2017 +;; Version: 0.2 +;; Keywords: languages p4_16 +;; Homepage: http://p4.org + +;; This file is not part of GNU Emacs. + +;; This file is free software… + +;; This mode has preliminary support for P4_16. It covers the core language, +;; but it is not clear yet, how we can highlight the indentifiers, defined +;; for a particular architecture. Core library definitions are included + +;; Placeholder for user customization code +(defvar p4_16-mode-hook nil) + +;; Define the keymap (for now it is pretty much default) +(defvar p4_16-mode-map + (let ((map (make-keymap))) + (define-key map "\C-j" 'newline-and-indent) + map) + "Keymap for P4_16 major mode") + +;; Syntactic HighLighting + +;; Main keywors (declarations and operators) +(setq p4_16-keywords + '("action" "apply" + "control" + "default" + "else" "enum" "extern" "exit" + "header" "header_union" + "if" + "match_kind" + "package" "parser" + "return" + "select" "state" "struct" "switch" + "table" "transition" "tuple" "typedef" + "verify" + )) + +(setq p4_16-annotations + '("@name" "@metadata" "@alias" + )) + +(setq p4_16-attributes + '("const" "in" "inout" "out" + ;; Tables + "key" "actions" "default_action" "entries" "implementation" + "counters" "meters" + )) + +(setq p4_16-variables + '("packet_in" "packet_out" + )) + +(setq p4_16-operations + '("&&&" ".." "++" "?" ":")) + +(setq p4_16-constants + '( + ;;; Don't care + "_" + ;;; bool + "false" "true" + ;;; error + "NoError" "PacketTooShort" "NoMatch" "StackOutOfBounds" + "OverwritingHeader" "HeaderTooShort" "ParserTiimeout" + ;;; match_kind + "exact" "ternary" "lpm" "range" + ;;; We can add constants for supported architectures here + )) + +(setq p4_16-types + '("bit" "bool" "int" "varbit" "void" "error" + )) + +(setq p4_16-primitives + '( + ;;; Header methods + "isValid" "setValid" "setInvalid" + ;;; Table Methods + "hit" "action_run" + ;;; packet_in methods + "extract" "lookahead" "advance" "length" + ;;; packet_out methods + "emit" + ;;; Known parser states + "accept" "reject" + ;;; misc + "NoAction" + )) + +(setq p4_16-cpp + '("#include" + "#define" "#undef" + "#if" "#ifdef" "#ifndef" + "#elif" "#else" + "#endif" + "defined" + "#line" "#file")) + +(setq p4_16-cppwarn + '("#error" "#warning")) + +;; Optimize the strings +(setq p4_16-keywords-regexp (regexp-opt p4_16-keywords 'words)) +(setq p4_16-annotations-regexp (regexp-opt p4_16-annotations 1)) +(setq p4_16-attributes-regexp (regexp-opt p4_16-attributes 'words)) +(setq p4_16-variables-regexp (regexp-opt p4_16-variables 'words)) +(setq p4_16-operations-regexp (regexp-opt p4_16-operations 'words)) +(setq p4_16-constants-regexp (regexp-opt p4_16-constants 'words)) +(setq p4_16-types-regexp (regexp-opt p4_16-types 'words)) +(setq p4_16-primitives-regexp (regexp-opt p4_16-primitives 'words)) +(setq p4_16-cpp-regexp (regexp-opt p4_16-cpp 1)) +(setq p4_16-cppwarn-regexp (regexp-opt p4_16-cppwarn 1)) + + +;; create the list for font-lock. +;; each category of keyword is given a particular face +(defconst p4_16-font-lock-keywords + (list + (cons p4_16-cpp-regexp font-lock-preprocessor-face) + (cons p4_16-cppwarn-regexp font-lock-warning-face) + (cons p4_16-types-regexp font-lock-type-face) + (cons p4_16-constants-regexp font-lock-constant-face) + (cons p4_16-attributes-regexp font-lock-builtin-face) + (cons p4_16-variables-regexp font-lock-variable-name-face) + ;;; This is a special case to distinguish the method from the keyword + (cons "\\.apply" font-lock-function-name-face) + (cons p4_16-primitives-regexp font-lock-function-name-face) + (cons p4_16-operations-regexp font-lock-builtin-face) + (cons p4_16-keywords-regexp font-lock-keyword-face) + (cons p4_16-annotations-regexp font-lock-keyword-face) + (cons "\\(\\w*_t +\\)" font-lock-type-face) + (cons "[^A-Z_][A-Z] " font-lock-type-face) ;; Total hack for templates + (cons "<[A-Z, ]*>" font-lock-type-face) + (cons "\\(<[^>]+>\\)" font-lock-string-face) + (cons "\\([^_A-Za-z]\\([0-9]+w\\)?0x[0-9A-Fa-f]+\\)" font-lock-constant-face) + (cons "\\([^_A-Za-z]\\([0-9]+w\\)?0b[01]+\\)" font-lock-constant-face) + (cons "\\([^_A-Za-z][+-]?\\([0-9]+w\\)?[0-9]+\\)" font-lock-constant-face) + ;;(cons "\\(\\w*\\)" font-lock-variable-name-face) + ) + "Default Highlighting Expressions for P4_16") + +(defvar p4_16-mode-syntax-table + (let ((st (make-syntax-table))) + (modify-syntax-entry ?_ "w" st) + (modify-syntax-entry ?/ ". 124b" st) + (modify-syntax-entry ?* ". 23" st) + (modify-syntax-entry ?\n "> b" st) + st) + "Syntax table for p4_16-mode") + +;;; Indentation +(defvar p4_16-indent-offset 4 + "Indentation offset for `p4_16-mode'.") + +(defun p4_16-indent-line () + "Indent current line for any balanced-paren-mode'." + (interactive) + (let ((indent-col 0) + (indentation-increasers "[{(]") + (indentation-decreasers "[})]") + ) + (save-excursion + (beginning-of-line) + (condition-case nil + (while t + (backward-up-list 1) + (when (looking-at indentation-increasers) + (setq indent-col (+ indent-col p4_16-indent-offset)))) + (error nil))) + (save-excursion + (back-to-indentation) + (when (and (looking-at indentation-decreasers) + (>= indent-col p4_16-indent-offset)) + (setq indent-col (- indent-col p4_16-indent-offset)))) + (indent-line-to indent-col))) + +;;; Imenu support +(require 'imenu) +(setq p4_16-imenu-generic-expression + '( + ("Controls" "^ *control +\\([A-Za-z0-9_]*\\)" 1) + ("Externs" "^ *extern +\\([A-Za-z0-9_]*\\) *\\([A-Za-z0-9_]*\\)" 2) + ("Tables" "^ *table +\\([A-Za-z0-9_]*\\)" 1) + ("Actions" "^ *action +\\([A-Za-z0-9_]*\\)" 1) + ("Parsers" "^ *parser +\\([A-Za-z0-9_]*\\)" 1) + ("Parser States" "^ *state +\\([A-Za-z0-9_]*\\)" 1) + ("Headers" "^ *header +\\([A-Za-z0-9_]*\\)" 1) + ("Header Unions" "^ *header_union +\\([A-Za-z0-9_]*\\)" 1) + ("Structs" "^ *struct +\\([A-Za-z0-9_]*\\)" 1) + )) + +;;; Cscope Support +(require 'xcscope) + +;; Put everything together +(defun p4_16-mode () + "Major mode for editing P4_16 programs" + (interactive) + (kill-all-local-variables) + (set-syntax-table p4_16-mode-syntax-table) + (use-local-map p4_16-mode-map) + (set (make-local-variable 'font-lock-defaults) '(p4_16-font-lock-keywords)) + (set (make-local-variable 'indent-line-function) 'p4_16-indent-line) + (setq major-mode 'p4_16-mode) + (setq mode-name "P4_16") + (setq imenu-generic-expression p4_16-imenu-generic-expression) + (imenu-add-to-menubar "P4_16") + (cscope-minor-mode) + (run-hooks 'p4_16-mode-hook) +) + +;; The most important line +(provide 'p4_16-mode) diff --git a/SIGCOMM_2017/vm/root-bootstrap.sh b/SIGCOMM_2017/vm/root-bootstrap.sh new file mode 100755 index 0000000..c460aa1 --- /dev/null +++ b/SIGCOMM_2017/vm/root-bootstrap.sh @@ -0,0 +1,54 @@ +#!/bin/bash + +set -x + +sudo add-apt-repository ppa:webupd8team/sublime-text-3 +sudo add-apt-repository ppa:webupd8team/atom + +apt-get update + +DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" upgrade + +apt-get install -y \ + lubuntu-desktop \ + git \ + vim \ + emacs24 \ + xcscope-el \ + sublime-text-installer \ + atom \ + xterm \ + mininet \ + autoconf \ + automake \ + libtool \ + curl \ + make \ + g++ \ + unzip \ + libgc-dev \ + bison \ + flex \ + libfl-dev \ + libgmp-dev \ + libboost-dev \ + libboost-iostreams-dev \ + pkg-config \ + python \ + python-scapy \ + python-ipaddr \ + tcpdump \ + cmake + +useradd -m -d /home/p4 -s /bin/bash p4 +echo "p4:p4" | chpasswd +echo "p4 ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/99_p4 +chmod 440 /etc/sudoers.d/99_p4 + +cd /usr/share/lubuntu/wallpapers/ +cp /home/vagrant/p4-logo.png . +rm lubuntu-default-wallpaper.png +ln -s p4-logo.png lubuntu-default-wallpaper.png +rm /home/vagrant/p4-logo.png +cd /home/vagrant +sed -i s@#background=@background=/usr/share/lubuntu/wallpapers/1604-lubuntu-default-wallpaper.png@ /etc/lightdm/lightdm-gtk-greeter.conf diff --git a/SIGCOMM_2017/vm/user-bootstrap.sh b/SIGCOMM_2017/vm/user-bootstrap.sh new file mode 100644 index 0000000..63e4875 --- /dev/null +++ b/SIGCOMM_2017/vm/user-bootstrap.sh @@ -0,0 +1,50 @@ +#!/bin/bash + +set -x + +# Bmv2 +git clone https://github.com/p4lang/behavioral-model +cd behavioral-model +./install_deps.sh +./autogen.sh +./configure +make +sudo make install +cd .. + +# Protobuf +git clone https://github.com/google/protobuf.git +cd protobuf +git checkout v3.0.2 +./autogen.sh +./configure +make +sudo make install +sudo ldconfig +cd .. + +# P4C +git clone --recursive https://github.com/p4lang/p4c +cd p4c +mkdir build +cd build +cmake .. +make -j4 +sudo make install +cd .. +cd .. + +# Tutorials +pip install crcmod +git clone https://github.com/p4lang/tutorials +cd tutorials +git checkout sigcomm_17 +cd .. +sudo mv tutorials /home/p4 +sudo chown -R p4:p4 /home/p4/tutorials + +# Emacs +sudo cp p4_16-mode.el /usr/share/emacs/site-lisp/ +sudo echo "(add-to-list 'auto-mode-alist '(\"\\.p4\\'\" . p4_16-mode))" >> /home/p4/.emacs +sudo chown p4:p4 /home/p4/.emacs +