P4 Developer Day 2018 Spring (#159)

* Repository reorganization for 2018 Spring P4 Developer Day.

* Port tutorial exercises to P4Runtime with static controller (#156)

* Switch VM to a minimal Ubuntu 16.04 desktop image

* Add commands to install Protobuf Python bindings to user_bootstrap.sh

* Implement P4Runtime static controller for use in exercises

From the exercise perspective, the main difference is that control plane
rules are now specified using JSON files instead of CLI commands. Such
JSON files define rules that use the same name for tables, keys, etc. as
in the P4Info file.

All P4Runtime requests generated as part of the make run process are
logged in the exercise's “logs” directory, making it easier for students
to see the actual P4Runtime messages sent to the switch.

Only the "basic" exercise has been ported to use P4Runtime.
The "p4runtime" exercise has been updated to work with P4Runtime
protocol changes.

Known issues:
- make run hangs in case of errors when running the P4Runtime controller
    (probably due to gRPC stream channel threads not terminated properly)
- missing support for inserting table entries with default action
    (can specify in P4 program as a workaround)

* Force install protobuf python module

* Fixing Ctrl-C hang by shutdown switches

* Moving gRPC error print to function for readability

Unforuntately, if this gets moved out of the file, the process hangs.
We'll need to figure out how why later.

* Renaming ShutdownAllSwitches -> ShutdownAllSwitchConnections

* Reverting counter index change

* Porting the ECN exercise to use P4 Runtime Static Controller

* updating the README in the ecn exercise to reflect the change in rule files

* Allow set table default action in P4Runtime static controller

* Fixed undefined match string when printing P4Runtime table entry

* Updated basic_tunnel exercise to use P4Runtime controller.

* Changed default action in the basic exercise's ipv4_lpm table to drop

* Porting the MRI exercise to use P4runtime with static controller

* Updating readme to reflect the change of controller for mri

* Update calc exercise for P4Runtime static controller

* Port source_routing to P4 Runtime static controller (#157)

* Port Load Balance to P4 Runtime Static Controller (#158)
This commit is contained in:
Nate Foster
2018-06-01 02:54:33 -04:00
committed by GitHub
parent e7e6899d5c
commit dc08948a34
503 changed files with 1432 additions and 30666 deletions

29
vm/Vagrantfile vendored Normal file
View File

@@ -0,0 +1,29 @@
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure(2) do |config|
config.vm.box = "fso/xenial64-desktop"
config.vm.define "p4-tutorial" do |tutorial|
end
config.vm.provider "virtualbox" do |vb|
vb.name = "P4 Tutorial" + Time.now.strftime(" %Y-%m-%d")
vb.gui = true
vb.memory = 2048
vb.cpus = 2
vb.customize ["modifyvm", :id, "--cableconnected1", "on"]
vb.customize ["storageattach", :id,
"--storagectl", "IDE Controller",
"--port", "0", "--device", "0",
"--type", "dvddrive",
"--medium", "emptydrive"]
vb.customize ["modifyvm", :id, "--vram", "32"]
end
config.vm.synced_folder '.', '/vagrant', disabled: true
config.vm.hostname = "p4"
config.vm.provision "file", source: "p4-logo.png", destination: "/home/vagrant/p4-logo.png"
config.vm.provision "file", source: "p4_16-mode.el", destination: "/home/vagrant/p4_16-mode.el"
config.vm.provision "file", source: "p4.vim", destination: "/home/vagrant/p4.vim"
config.vm.provision "shell", path: "root-bootstrap.sh"
config.vm.provision "shell", privileged: false, path: "user-bootstrap.sh"
end

BIN
vm/p4-logo.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.2 KiB

133
vm/p4.vim Normal file
View File

@@ -0,0 +1,133 @@
" Vim syntax file
" Language: P4_16
" Maintainer: Antonin Bas, Barefoot Networks Inc
" Latest Revision: 5 August 2014
" Updated By: Gyanesh Patra, Unicamp University
" Latest Revision: 12 April 2016
" Updated Again By: Robert MacDavid, Princeton University
" Latest Revision: 12 June 2017
if version < 600
syntax clear
elseif exists("b:current_syntax")
finish
endif
" Use case sensitive matching of keywords
syn case match
syn keyword p4ObjectKeyword action apply control default
syn keyword p4ObjectKeyword enum extern exit
syn keyword p4ObjectKeyword header header_union
syn keyword p4ObjectKeyword match_kind
syn keyword p4ObjectKeyword package parser
syn keyword p4ObjectKeyword state struct switch size
syn keyword p4ObjectKeyword table transition tuple typedef
syn keyword p4ObjectKeyword verify
" Tables
syn keyword p4ObjectAttributeKeyword key actions default_action entries
syn keyword p4ObjectAttributeKeyword implementation
" Counters and meters
syn keyword p4ObjectAttributeKeyword counters meters
" Var Attributes
syn keyword p4ObjectKeyword const in out inout
syn keyword p4Annotation @name @tableonly @defaultonly
syn keyword p4Annotation @globalname @atomic @hidden
syn keyword p4MatchTypeKeyword exact ternary lpm range
syn keyword p4TODO contained FIXME TODO
syn match p4Comment '\/\/.*' contains=p4TODO
syn region p4BlockComment start='\/\*' end='\*\/' contains=p4TODO keepend
syn match p4Preprocessor '#(include|define|undef|if|ifdef) .*$'
syn match p4Preprocessor '#(if|ifdef|ifndef|elif|else) .*$'
syn match p4Preprocessor '#(endif|defined|line|file) .*$'
syn match p4Preprocessor '#(error|warning) .*$'
syn keyword p4Type bit bool int varbit void error
" Integer Literals
syn match p4Int '[0-9][0-9_]*'
syn match p4Indentifier '[A-Za-z_][A-Za-z0-9_]*'
syn match p4HexadecimalInt '0[Xx][0-9a-fA-F]\+'
syn match p4DecimalInt '0[dD][0-9_]\+'
syn match p4OctalInt '0[oO][0-7_]\+'
syn match p4BinaryInt '0[bB][01_]\+'
syn region p4SizedType start='(bit|int|varbit)\<' end='\>'
syn match p4UserType '[A-Za-z_][A-Za-z0-9_]*[_][t]\W'
syn keyword p4Operators and or not &&& mask
" Header Methods
syn keyword p4Primitive isValid setValid setInvalid
" Table Methods
syn keyword p4Primitive hit action_run
" Packet_in methods
syn keyword p4Primitive extract lookahead advance length
" Packet_out methods
syn keyword p4Primitive emit
" Known parser states
syn keyword p4Primitive accept reject
" Misc
syn keyword p4Primitive NoAction
syn keyword p4Conditional if else select
syn keyword p4Statement return
" Don't Care
syn keyword p4Constant _
" Error
syn keyword p4Constant NoError PacketTooShort NoMatch StackOutOfBounds
syn keyword p4Constant OverwritingHeader HeaderTooShort ParserTiimeout
" Boolean
syn keyword p4Boolean false true
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
" Apply highlight groups to syntax groups defined above
" For version 5.7 and earlier: only when not done already
" For version 5.8 and later: only when an item doesn't have highlighting yet
if version >= 508 || !exists("did_p4_syntax_inits")
if version <= 508
let did_p4_syntax_inits = 1
command -nargs=+ HiLink hi link <args>
else
command -nargs=+ HiLink hi def link <args>
endif
HiLink p4ObjectKeyword Repeat
HiLink p4UserType Type
HiLink p4ObjectAttributeKeyword Keyword
HiLink p4TypeAttribute StorageClass
HiLink p4Annotation Special
HiLink p4MatchTypeKeyword Keyword
HiLink p4TODO Todo
HiLink p4Comment Comment
HiLink p4BlockComment Comment
HiLink p4Preprocessor PreProc
HiLink p4SizedType Type
HiLink p4Type Type
HiLink p4DecimalInt Number
HiLink p4HexadecimalInt Number
HiLink p4OctalInt Number
HiLink p4BinaryInt Number
HiLink p4Int Number
HiLink p4Operators Operator
HiLink p4Primitive Function
HiLink p4Conditional Conditional
HiLink p4Statement Statement
HiLink p4Constant Constant
HiLink p4Boolean Boolean
delcommand HiLink
endif
let b:current_syntax = "p4"

222
vm/p4_16-mode.el Normal file
View File

@@ -0,0 +1,222 @@
;;; p4_16-mode.el --- Support for the P4_16 programming language
;; Copyright (C) 2016- Barefoot Networks
;; Author: Vladimir Gurevich <vladimir.gurevich@barefootnetworks.com>
;; Maintainer: Vladimir Gurevich <vladimir.gurevich@barefootnetworks.com>
;; Created: 15 April 2017
;; Version: 0.2
;; Keywords: languages p4_16
;; Homepage: http://p4.org
;; This file is not part of GNU Emacs.
;; This file is free software…
;; This mode has preliminary support for P4_16. It covers the core language,
;; but it is not clear yet, how we can highlight the indentifiers, defined
;; for a particular architecture. Core library definitions are included
;; Placeholder for user customization code
(defvar p4_16-mode-hook nil)
;; Define the keymap (for now it is pretty much default)
(defvar p4_16-mode-map
(let ((map (make-keymap)))
(define-key map "\C-j" 'newline-and-indent)
map)
"Keymap for P4_16 major mode")
;; Syntactic HighLighting
;; Main keywors (declarations and operators)
(setq p4_16-keywords
'("action" "apply"
"control"
"default"
"else" "enum" "extern" "exit"
"header" "header_union"
"if"
"match_kind"
"package" "parser"
"return"
"select" "state" "struct" "switch"
"table" "transition" "tuple" "typedef"
"verify"
))
(setq p4_16-annotations
'("@name" "@metadata" "@alias"
))
(setq p4_16-attributes
'("const" "in" "inout" "out"
;; Tables
"key" "actions" "default_action" "entries" "implementation"
"counters" "meters"
))
(setq p4_16-variables
'("packet_in" "packet_out"
))
(setq p4_16-operations
'("&&&" ".." "++" "?" ":"))
(setq p4_16-constants
'(
;;; Don't care
"_"
;;; bool
"false" "true"
;;; error
"NoError" "PacketTooShort" "NoMatch" "StackOutOfBounds"
"OverwritingHeader" "HeaderTooShort" "ParserTiimeout"
;;; match_kind
"exact" "ternary" "lpm" "range"
;;; We can add constants for supported architectures here
))
(setq p4_16-types
'("bit" "bool" "int" "varbit" "void" "error"
))
(setq p4_16-primitives
'(
;;; Header methods
"isValid" "setValid" "setInvalid"
;;; Table Methods
"hit" "action_run"
;;; packet_in methods
"extract" "lookahead" "advance" "length"
;;; packet_out methods
"emit"
;;; Known parser states
"accept" "reject"
;;; misc
"NoAction"
))
(setq p4_16-cpp
'("#include"
"#define" "#undef"
"#if" "#ifdef" "#ifndef"
"#elif" "#else"
"#endif"
"defined"
"#line" "#file"))
(setq p4_16-cppwarn
'("#error" "#warning"))
;; Optimize the strings
(setq p4_16-keywords-regexp (regexp-opt p4_16-keywords 'words))
(setq p4_16-annotations-regexp (regexp-opt p4_16-annotations 1))
(setq p4_16-attributes-regexp (regexp-opt p4_16-attributes 'words))
(setq p4_16-variables-regexp (regexp-opt p4_16-variables 'words))
(setq p4_16-operations-regexp (regexp-opt p4_16-operations 'words))
(setq p4_16-constants-regexp (regexp-opt p4_16-constants 'words))
(setq p4_16-types-regexp (regexp-opt p4_16-types 'words))
(setq p4_16-primitives-regexp (regexp-opt p4_16-primitives 'words))
(setq p4_16-cpp-regexp (regexp-opt p4_16-cpp 1))
(setq p4_16-cppwarn-regexp (regexp-opt p4_16-cppwarn 1))
;; create the list for font-lock.
;; each category of keyword is given a particular face
(defconst p4_16-font-lock-keywords
(list
(cons p4_16-cpp-regexp font-lock-preprocessor-face)
(cons p4_16-cppwarn-regexp font-lock-warning-face)
(cons p4_16-types-regexp font-lock-type-face)
(cons p4_16-constants-regexp font-lock-constant-face)
(cons p4_16-attributes-regexp font-lock-builtin-face)
(cons p4_16-variables-regexp font-lock-variable-name-face)
;;; This is a special case to distinguish the method from the keyword
(cons "\\.apply" font-lock-function-name-face)
(cons p4_16-primitives-regexp font-lock-function-name-face)
(cons p4_16-operations-regexp font-lock-builtin-face)
(cons p4_16-keywords-regexp font-lock-keyword-face)
(cons p4_16-annotations-regexp font-lock-keyword-face)
(cons "\\(\\w*_t +\\)" font-lock-type-face)
(cons "[^A-Z_][A-Z] " font-lock-type-face) ;; Total hack for templates
(cons "<[A-Z, ]*>" font-lock-type-face)
(cons "\\(<[^>]+>\\)" font-lock-string-face)
(cons "\\([^_A-Za-z]\\([0-9]+w\\)?0x[0-9A-Fa-f]+\\)" font-lock-constant-face)
(cons "\\([^_A-Za-z]\\([0-9]+w\\)?0b[01]+\\)" font-lock-constant-face)
(cons "\\([^_A-Za-z][+-]?\\([0-9]+w\\)?[0-9]+\\)" font-lock-constant-face)
;;(cons "\\(\\w*\\)" font-lock-variable-name-face)
)
"Default Highlighting Expressions for P4_16")
(defvar p4_16-mode-syntax-table
(let ((st (make-syntax-table)))
(modify-syntax-entry ?_ "w" st)
(modify-syntax-entry ?/ ". 124b" st)
(modify-syntax-entry ?* ". 23" st)
(modify-syntax-entry ?\n "> b" st)
st)
"Syntax table for p4_16-mode")
;;; Indentation
(defvar p4_16-indent-offset 4
"Indentation offset for `p4_16-mode'.")
(defun p4_16-indent-line ()
"Indent current line for any balanced-paren-mode'."
(interactive)
(let ((indent-col 0)
(indentation-increasers "[{(]")
(indentation-decreasers "[})]")
)
(save-excursion
(beginning-of-line)
(condition-case nil
(while t
(backward-up-list 1)
(when (looking-at indentation-increasers)
(setq indent-col (+ indent-col p4_16-indent-offset))))
(error nil)))
(save-excursion
(back-to-indentation)
(when (and (looking-at indentation-decreasers)
(>= indent-col p4_16-indent-offset))
(setq indent-col (- indent-col p4_16-indent-offset))))
(indent-line-to indent-col)))
;;; Imenu support
(require 'imenu)
(setq p4_16-imenu-generic-expression
'(
("Controls" "^ *control +\\([A-Za-z0-9_]*\\)" 1)
("Externs" "^ *extern +\\([A-Za-z0-9_]*\\) *\\([A-Za-z0-9_]*\\)" 2)
("Tables" "^ *table +\\([A-Za-z0-9_]*\\)" 1)
("Actions" "^ *action +\\([A-Za-z0-9_]*\\)" 1)
("Parsers" "^ *parser +\\([A-Za-z0-9_]*\\)" 1)
("Parser States" "^ *state +\\([A-Za-z0-9_]*\\)" 1)
("Headers" "^ *header +\\([A-Za-z0-9_]*\\)" 1)
("Header Unions" "^ *header_union +\\([A-Za-z0-9_]*\\)" 1)
("Structs" "^ *struct +\\([A-Za-z0-9_]*\\)" 1)
))
;;; Cscope Support
(require 'xcscope)
;; Put everything together
(defun p4_16-mode ()
"Major mode for editing P4_16 programs"
(interactive)
(kill-all-local-variables)
(set-syntax-table p4_16-mode-syntax-table)
(use-local-map p4_16-mode-map)
(set (make-local-variable 'font-lock-defaults) '(p4_16-font-lock-keywords))
(set (make-local-variable 'indent-line-function) 'p4_16-indent-line)
(setq major-mode 'p4_16-mode)
(setq mode-name "P4_16")
(setq imenu-generic-expression p4_16-imenu-generic-expression)
(imenu-add-to-menubar "P4_16")
(cscope-minor-mode)
(run-hooks 'p4_16-mode-hook)
)
;; The most important line
(provide 'p4_16-mode)

91
vm/root-bootstrap.sh Executable file
View File

@@ -0,0 +1,91 @@
#!/bin/bash
# Print commands and exit on errors
set -xe
sudo add-apt-repository ppa:webupd8team/sublime-text-3
sudo add-apt-repository ppa:webupd8team/atom
apt-get update
KERNEL=$(uname -r)
DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" upgrade
apt-get install -y --no-install-recommends \
atom \
autoconf \
automake \
bison \
build-essential \
ca-certificates \
cmake \
cpp \
curl \
emacs24 \
flex \
git \
libboost-dev \
libboost-filesystem-dev \
libboost-iostreams1.58-dev \
libboost-program-options-dev \
libboost-system-dev \
libboost-test-dev \
libboost-thread-dev \
libc6-dev \
libevent-dev \
libffi-dev \
libfl-dev \
libgc-dev \
libgc1c2 \
libgflags-dev \
libgmp-dev \
libgmp10 \
libgmpxx4ldbl \
libjudy-dev \
libpcap-dev \
libreadline6 \
libreadline6-dev \
libssl-dev \
libtool \
linux-headers-$KERNEL\
lubuntu-desktop \
make \
mktemp \
pkg-config \
python \
python-dev \
python-ipaddr \
python-pip \
python-scapy \
python-setuptools \
sublime-text-installer \
tcpdump \
unzip \
vim \
wget \
xcscope-el \
xterm
useradd -m -d /home/p4 -s /bin/bash p4
echo "p4:p4" | chpasswd
echo "p4 ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/99_p4
chmod 440 /etc/sudoers.d/99_p4
usermod -aG vboxsf p4
cd /usr/share/lubuntu/wallpapers/
cp /home/vagrant/p4-logo.png .
rm lubuntu-default-wallpaper.png
ln -s p4-logo.png lubuntu-default-wallpaper.png
rm /home/vagrant/p4-logo.png
cd /home/vagrant
sed -i s@#background=@background=/usr/share/lubuntu/wallpapers/1604-lubuntu-default-wallpaper.png@ /etc/lightdm/lightdm-gtk-greeter.conf
# Disable screensaver
apt-get -y remove light-locker
# Automatically log into the P4 user
cat << EOF | tee -a /etc/lightdm/lightdm.conf.d/10-lightdm.conf
[SeatDefaults]
autologin-user=p4
autologin-user-timeout=0
user-session=Lubuntu
EOF

187
vm/user-bootstrap.sh Normal file
View File

@@ -0,0 +1,187 @@
#!/bin/bash
# Print script commands.
set -x
# Exit on errors.
set -e
BMV2_COMMIT="7e25eeb19d01eee1a8e982dc7ee90ee438c10a05"
PI_COMMIT="219b3d67299ec09b49f433d7341049256ab5f512"
P4C_COMMIT="48a57a6ae4f96961b74bd13f6bdeac5add7bb815"
PROTOBUF_COMMIT="v3.2.0"
GRPC_COMMIT="v1.3.2"
NUM_CORES=`grep -c ^processor /proc/cpuinfo`
# Mininet
git clone git://github.com/mininet/mininet mininet
cd mininet
sudo ./util/install.sh -nwv
cd ..
# Protobuf
git clone https://github.com/google/protobuf.git
cd protobuf
git checkout ${PROTOBUF_COMMIT}
export CFLAGS="-Os"
export CXXFLAGS="-Os"
export LDFLAGS="-Wl,-s"
./autogen.sh
./configure --prefix=/usr
make -j${NUM_CORES}
sudo make install
sudo ldconfig
unset CFLAGS CXXFLAGS LDFLAGS
# force install python module
cd python
sudo python setup.py install
cd ../..
# gRPC
git clone https://github.com/grpc/grpc.git
cd grpc
git checkout ${GRPC_COMMIT}
git submodule update --init --recursive
export LDFLAGS="-Wl,-s"
make -j${NUM_CORES}
sudo make install
sudo ldconfig
unset LDFLAGS
cd ..
# Install gRPC Python Package
sudo pip install grpcio
# BMv2 deps (needed by PI)
git clone https://github.com/p4lang/behavioral-model.git
cd behavioral-model
git checkout ${BMV2_COMMIT}
# From bmv2's install_deps.sh, we can skip apt-get install.
# Nanomsg is required by p4runtime, p4runtime is needed by BMv2...
tmpdir=`mktemp -d -p .`
cd ${tmpdir}
bash ../travis/install-thrift.sh
bash ../travis/install-nanomsg.sh
sudo ldconfig
bash ../travis/install-nnpy.sh
cd ..
sudo rm -rf $tmpdir
cd ..
# PI/P4Runtime
git clone https://github.com/p4lang/PI.git
cd PI
git checkout ${PI_COMMIT}
git submodule update --init --recursive
./autogen.sh
./configure --with-proto
make -j${NUM_CORES}
sudo make install
sudo ldconfig
cd ..
# Bmv2
cd behavioral-model
./autogen.sh
./configure --enable-debugger --with-pi
make -j${NUM_CORES}
sudo make install
sudo ldconfig
# Simple_switch_grpc target
cd targets/simple_switch_grpc
./autogen.sh
./configure --with-thrift
make -j${NUM_CORES}
sudo make install
sudo ldconfig
cd ..
cd ..
cd ..
# P4C
git clone https://github.com/p4lang/p4c
cd p4c
git checkout ${P4C_COMMIT}
git submodule update --init --recursive
mkdir -p build
cd build
cmake ..
make -j${NUM_CORES}
make -j${NUM_CORES} check
sudo make install
sudo ldconfig
cd ..
cd ..
# Tutorials
sudo pip install crcmod
git clone https://github.com/p4lang/tutorials
sudo mv tutorials /home/p4
sudo chown -R p4:p4 /home/p4/tutorials
# Emacs
sudo cp p4_16-mode.el /usr/share/emacs/site-lisp/
sudo mkdir /home/p4/.emacs.d/
echo "(autoload 'p4_16-mode' \"p4_16-mode.el\" \"P4 Syntax.\" t)" > init.el
echo "(add-to-list 'auto-mode-alist '(\"\\.p4\\'\" . p4_16-mode))" | tee -a init.el
sudo mv init.el /home/p4/.emacs.d/
sudo ln -s /usr/share/emacs/site-lisp/p4_16-mode.el /home/p4/.emacs.d/p4_16-mode.el
sudo chown -R p4:p4 /home/p4/.emacs.d/
# Vim
cd /home/vagrant
mkdir .vim
cd .vim
mkdir ftdetect
mkdir syntax
echo "au BufRead,BufNewFile *.p4 set filetype=p4" >> ftdetect/p4.vim
echo "set bg=dark" >> /home/vagrant/.vimrc
sudo mv /home/vagrant/.vimrc /home/p4/.vimrc
cp /home/vagrant/p4.vim syntax/p4.vim
cd /home/vagrant
sudo mv .vim /home/p4/.vim
sudo chown -R p4:p4 /home/p4/.vim
sudo chown p4:p4 /home/p4/.vimrc
# Adding Desktop icons
DESKTOP=/home/${USER}/Desktop
mkdir -p ${DESKTOP}
cat > ${DESKTOP}/Terminal << EOF
[Desktop Entry]
Encoding=UTF-8
Type=Application
Name=Terminal
Name[en_US]=Terminal
Icon=konsole
Exec=/usr/bin/x-terminal-emulator
Comment[en_US]=
EOF
cat > ${DESKTOP}/Wireshark << EOF
[Desktop Entry]
Encoding=UTF-8
Type=Application
Name=Wireshark
Name[en_US]=Wireshark
Icon=wireshark
Exec=/usr/bin/wireshark
Comment[en_US]=
EOF
cat > ${DESKTOP}/Sublime\ Text << EOF
[Desktop Entry]
Encoding=UTF-8
Type=Application
Name=Sublime Text
Name[en_US]=Sublime Text
Icon=sublime-text
Exec=/opt/sublime_text/sublime_text
Comment[en_US]=
EOF
sudo mkdir -p /home/p4/Desktop
sudo mv /home/${USER}/Desktop/* /home/p4/Desktop
sudo chown -R p4:p4 /home/p4/Desktop/
# Do this last!
sudo reboot