Merge pull request #65 from lowescott/ovn-docker-ansible

Add a learning environment for OVN+Docker

ovn-docker-ansible/10-exec-options.conf.j2

@@ -0,0 +1,5 @@
+[Service]
+ExecStart=
+ExecStart=/usr/bin/docker daemon -H fd:// -H tcp://0.0.0.0:2375 \
+  --cluster-store=consul://127.0.0.1:8500 \
+  --cluster-advertise={{ ansible_enp0s8.ipv4.address }}:0

ovn-docker-ansible/README.md

@@ -0,0 +1,47 @@
+# Open Virtual Network (OVN) with Docker
+
+These files were created to allow users to use Vagrant ([http://www.vagrantup.com](http://www.vagrantup.com)) to quickly and  easily spin up an environment to test Open Virtual Network (OVN), a part of the Open vSwitch project, to provide overlay networking for Docker containers.
+
+## Contents
+
+* **10-exec-options.conf.j2**: This Jinja2 template is used to create a systemd drop-in file to customize the default behavior of the Docker Engine. The file created by this template is placed by Ansible at `/etc/systemd/system/docker.service.d`.
+
+* **ansible.cfg**: This is an Ansible configuration file that instructs Ansible to use the "vagrant" remote user, and to use Vagrant's built-in Ansible inventory.
+
+* **config.json.j2**: This Jinja2 template is used to create a Consul configuration file to turn up the Consul cluster. The configuration file created from this template is placed at `/etc/consul.d/server`.
+
+* **consul-server.service**: This is a systemd unit file for the Consul cluster.
+
+* **machines.yml**: This YAML file contains a list of VM definitions and associated configuration data. It is referenced by `Vagrantfile` when Vagrant instantiates the VMs.
+
+* **provision.yml**: This Ansible playbook installs the OVS/OVN components on each Vagrant machine. It's called automatically by Vagrant.
+
+* **README.md**: This file you're currently reading.
+
+* **setup.sh.j2**: This Jinja2 template is used by Ansible to build a customized OVS/OVN configuration script for each Vagrant machine. The shell script created by this template is placed by Ansible in `/home/vagrant/setup.sh`.
+
+* **Vagrantfile**: This file is used by Vagrant to spin up the virtual machines. This file is fairly extensively commented to help explain what's happening. You should be able to use this file unchanged; all the VM configuration options are stored outside this file.
+
+## Instructions
+
+These instructions assume you've already installed Vagrant, your back-end virtualization provider (such as VMware Fusion or VirtualBox), and any necessary plugins (such as the Vagrant VMware plugin). Please refer to the documentation for those products for more information on installation or configuration.
+
+1. Use `vagrant box add` to add an Ubuntu 16.04 box to your system. The "bento/ubuntu-16.04" box is very good for both VirtualBox and VMware Fusion/Workstation. (Note that the "ubuntu/xenial64" box for VirtualBox is currently broken under Vagrant.)
+
+2. Edit the `machines.yml` file to ensure the box you downloaded in step 1 is specified on the "box:" sections of this file. Specify the name of a VMware-formatted box on the "vmw:" line; place the name of a VirtualBox-formatted box on the "vb:" line.
+
+3. Run `vagrant up` to have Vagrant instantiate the three machines configured by default in this environment, and provision them using Ansible. (Note you'll need Ansible installed locally on the system where you're running `vagrant up`.)
+
+4. Use `vagrant ssh ovn-01` to log into the first system. Run `sudo ./setup.sh` to perform final configuration steps for OVS/OVN.
+
+5. Repeat step #4 with `ovn-02` and `ovn-03`. At this point, OVN is up and running, with the OVN central components running on "ovn-01".
+
+6. To add Docker support to OVN, log into "ovn-01" and run `/usr/bin/ovn-docker-overlay-driver --detach`. This will launch the Docker driver for OVS/OVN.
+
+7. Repeat step #6 on "ovn-02" and "ovn-03". Your OVN environment now has Docker networking support.
+
+8. Create a Docker network using `docker network create -d openvswitch --subnet=<A.B.C.D/24> <name>`. You will note an OVN logical switch is created that corresponds to the Docker network.
+
+9. Launch a Docker container and attach it to the new network with `docker run -d --net=<name> <image>`. You will note OVN logical ports added to the OVN logical switch.
+
+Enjoy!

ovn-docker-ansible/Vagrantfile

@@ -0,0 +1,76 @@
+# -*- mode: ruby -*-
+# vi: set ft=ruby :
+
+# Specify minimum Vagrant version and Vagrant API version
+Vagrant.require_version '>= 1.6.0'
+VAGRANTFILE_API_VERSION = '2'
+
+# Require 'yaml' module
+require 'yaml'
+
+# Read YAML file with VM details (box, CPU, RAM, IP addresses)
+# Edit machines.yml to change VM configuration details
+machines = YAML.load_file(File.join(File.dirname(__FILE__), 'machines.yml'))
+
+# Create and configure the VMs
+Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
+
+  # Always use Vagrant's default insecure key
+  config.ssh.insert_key = false
+
+  # Iterate through entries in YAML file to create VMs
+  machines.each do |machine|
+
+    # Configure the VMs per details in machines.yml
+    config.vm.define machine['name'] do |srv|
+
+      # Don't check for box updates
+      srv.vm.box_check_update = false
+
+      # Specify the hostname of the VM
+      srv.vm.hostname = machine['name']
+
+      # Specify the Vagrant box to use (use VMware box by default)
+      srv.vm.box = machine['box']['vmw']
+
+      # Configure default synced folder (disable by default)
+      if machine['sync_disabled'] != nil
+        srv.vm.synced_folder '.', '/vagrant', disabled: machine['sync_disabled']
+      else
+        srv.vm.synced_folder '.', '/vagrant', disabled: true
+      end #if machine['sync_disabled']
+
+      # Iterate through networks as per settings in machines.yml
+      machine['nics'].each do |net|
+        if net['ip_addr'] == 'dhcp'
+          srv.vm.network net['type'], type: net['ip_addr']
+        else
+          srv.vm.network net['type'], ip: net['ip_addr']
+        end # if net['ip_addr']
+      end # machine['nics'].each
+
+      # Configure CPU & RAM per settings in machines.yml (Fusion)
+      srv.vm.provider 'vmware_fusion' do |vmw|
+        vmw.vmx['memsize'] = machine['ram']
+        vmw.vmx['numvcpus'] = machine['vcpu']
+        if machine['nested'] == true
+          vmw.vmx['vhv.enable'] = 'TRUE'
+        end #if machine['nested']
+      end # srv.vm.provider 'vmware_fusion'
+
+      # Configure CPU & RAM per settings in machines.yml (VirtualBox)
+      srv.vm.provider 'virtualbox' do |vb, override|
+        vb.memory = machine['ram']
+        vb.cpus = machine['vcpu']
+        override.vm.box = machine['box']['vb']
+        vb.customize ['modifyvm', :id, '--nictype1', 'virtio']
+        vb.customize ['modifyvm', :id, '--nictype2', 'virtio']
+      end # srv.vm.provider 'virtualbox'
+    end # config.vm.define
+  end # machines.each
+
+  # Provision the VM with Ansible
+  config.vm.provision 'ansible' do |ansible|
+    ansible.playbook = 'provision.yml'
+  end # config.vm.provision
+end # Vagrant.configure

ovn-docker-ansible/ansible.cfg

@@ -0,0 +1,5 @@
+[defaults]
+inventory = .vagrant/provisioners/ansible/inventory/vagrant_ansible_inventory
+private_key_file = ~/.vagrant.d/insecure_private_key
+remote_user = vagrant
+host_key_checking = False

ovn-docker-ansible/config.json.j2

@@ -0,0 +1,20 @@
+{% set lbracket = "[" %}
+{% set rbracket = "]" %}
+{% set quote = '"' %}
+
+{
+	"advertise_addr": "{{ hostvars[inventory_hostname]['ansible_enp0s8']['ipv4']['address'] }}",
+	"bind_addr": "{{ hostvars[inventory_hostname]['ansible_enp0s8']['ipv4']['address'] }}",
+	"bootstrap_expect": 3,
+	"client_addr": "0.0.0.0",
+	"datacenter": "dc1",
+	"data_dir": "/var/consul",
+	"enable_syslog": true,
+	"log_level": "INFO",
+	"retry_join": [ {% for host in groups['all'] %}{% if not loop.last %}
+"{{ hostvars[host]['ansible_enp0s8']['ipv4']['address'] }}", 
+      {% else %}
+"{{ hostvars[host]['ansible_enp0s8']['ipv4']['address'] }}"
+      {% endif %}{% endfor %} ],
+	"server": true
+}

ovn-docker-ansible/consul-server.service

@@ -0,0 +1,17 @@
+[Unit]
+Description=Consul Server
+Requires=network-online.target
+After=network-online.target
+
+[Service]
+User=consul
+Group=consul
+Environment="GOMAXPROCS=2"
+Restart=on-failure
+RestartSec=1
+ExecReload=/bin/kill -HUP $MAINPID
+KillSignal=SIGINT
+ExecStart=/usr/local/bin/consul agent -config-dir=/etc/consul.d/server
+
+[Install]
+WantedBy=multi-user.target

ovn-docker-ansible/machines.yml

@@ -0,0 +1,28 @@
+---
+- box:
+    vmw: "bento/ubuntu-16.04"
+    vb: "bento/ubuntu-16.04"
+  name: "docker-01"
+  nics:
+    - type: "private_network"
+      ip_addr: "192.168.100.101"
+  ram: "512"
+  vcpu: "1"
+- box:
+    vmw: "bento/ubuntu-16.04"
+    vb: "bento/ubuntu-16.04"
+  name: "docker-02"
+  nics:
+    - type: "private_network"
+      ip_addr: "192.168.100.102"
+  ram: "512"
+  vcpu: "1"
+- box:
+    vmw: "bento/ubuntu-16.04"
+    vb: "bento/ubuntu-16.04"
+  name: "docker-03"
+  nics:
+    - type: "private_network"
+      ip_addr: "192.168.100.103"
+  ram: "512"
+  vcpu: "1"

ovn-docker-ansible/provision.yml

@@ -0,0 +1,232 @@
+---
+- hosts: "all"
+  become: "yes"
+  remote_user: "vagrant"
+
+  tasks:
+  - name: Install prerequisite packages
+    apt:
+      pkg: "{{ item }}"
+      state: "latest"
+      update_cache: "yes"
+      cache_valid_time: "3600"
+    with_items:
+      - "unzip"
+      - "python-pip"
+      - "libssl-dev"
+      - "libcap-ng-dev"
+      - "git"
+      - "libtool"
+      - "autoconf"
+      - "ubuntu-cloud-keyring"
+    tags:
+      - consul
+      - packages
+      - ovn
+
+  - name: Create consul group
+    group:
+      name: "consul"
+      state: "present"
+    tags:
+      - consul
+
+  - name: Create user for Consul
+    user:
+      name: "consul"
+      group: "consul"
+      comment: "Consul daemon user"
+      shell: "/usr/sbin/nologin"
+      state: "present"
+      home: "/var/consul"
+      system: "yes"
+      createhome: "no"
+    tags:
+      - consul
+
+  - name: Create Consul data and configuration directories
+    file:
+      path: "{{ item }}"
+      state: "directory"
+      owner: "consul"
+      group: "consul"
+    with_items:
+      - "/var/consul"
+      - "/etc/consul.d/server"
+    tags:
+      - consul
+
+  - name: Download Consul package
+    get_url:
+      url: "https://releases.hashicorp.com/consul/0.7.1/consul_0.7.1_linux_amd64.zip"
+      dest: "/tmp/consul_0.7.1_linux_amd64.zip"
+    tags:
+      - consul
+
+  - name: Unzip downloaded Consul file
+    unarchive:
+      copy: "no"
+      src: "/tmp/consul_0.7.1_linux_amd64.zip"
+      dest: "/usr/local/bin/"
+      creates: "/usr/local/bin/consul"
+    tags:
+      - consul
+
+  - name: Install systemd unit for Consul
+    copy:
+      src: "consul-server.service"
+      dest: "/etc/systemd/system/consul-server.service"
+      owner: "root"
+      group: "root"
+      mode: "0644"
+    tags:
+      - consul
+
+  - name: Install Consul configuration file
+    template:
+      src: "config.json.j2"
+      dest: "/etc/consul.d/server/config.json"
+      owner: "consul"
+      group: "consul"
+      mode: "0644"
+    tags:
+      - consul
+
+  - name: Reload systemd units
+    command: "systemctl daemon-reload"
+
+  - name: Start Consul service
+    service:
+      name: "consul-server"
+      state: "started"
+    tags:
+      - consul
+
+  - name: Add Docker APT repository key
+    apt_key:
+      keyserver: "p80.pool.sks-keyservers.net"
+      id: "58118E89F3A912897C070ADBF76221572C52609D"
+      state: "present"
+    tags:
+      - docker
+      - apt-cfg
+
+  - name: Add Docker APT repository (Xenial)
+    apt_repository:
+      repo: "deb https://apt.dockerproject.org/repo ubuntu-xenial main"
+      update_cache: "yes"
+      state: "present"
+    when: ansible_distribution_release == "xenial"
+    tags:
+      - docker
+      - apt-cfg
+
+  - name: Install Docker packages
+    apt:
+      name: "{{ item }}"
+      state: "present"
+      update_cache: "yes"
+      cache_valid_time: "3600"
+    with_items:
+      - "docker-engine=1.11.2-0~xenial"
+    tags:
+      - docker
+      - packages
+
+  - name: Create systemd drop-in directory
+    file:
+      path: "/etc/systemd/system/docker.service.d"
+      state: "directory"
+      owner: "root"
+      group: "root"
+      mode: "0755"
+    tags:
+      - docker
+      - docker-cfg
+
+  - name: Customize Docker daemon configuration
+    template:
+      src: "10-exec-options.conf.j2"
+      dest: "/etc/systemd/system/docker.service.d/10-exec-options.conf"
+      owner: "root"
+      group: "root"
+      mode: "0644"
+    tags:
+      - docker
+      - docker-cfg
+
+  - name: Reload systemd units
+    command: "systemctl daemon-reload"
+
+  - name: Restart Docker daemon
+    service:
+      name: "docker"
+      state: "restarted"
+    tags:
+      - docker
+      - docker-cfg
+
+  - name: Set Vagrant user to member of Docker group
+    user:
+      name: "vagrant"
+      groups: "docker"
+      append: "yes"
+    tags:
+      - docker
+      - docker-cfg
+
+  - name: Install Python modules
+    pip:
+      name: "{{ item }}"
+    with_items:
+      - "Flask"
+      - "ovs"
+    tags:
+      - docker
+      - ovn
+
+  - name: "Add Ubuntu Cloud Archive repository"
+    apt_repository:
+      repo: "{{ item }}"
+      state: "present"
+      update_cache: "yes"
+    with_items:
+    - "deb http://ubuntu-cloud.archive.canonical.com/ubuntu xenial-updates/newton main"
+    tags:
+      - ovn
+      - apt-cfg
+
+  - name: "Install OVS/OVN 2.6.0"
+    apt:
+      state: "present"
+      update_cache: "no"
+      name: "{{ item }}"
+    with_items:
+    - "ovn-common"
+    - "ovn-host"
+    - "ovn-docker"
+    - "openvswitch-common"
+    - "openvswitch-switch"
+    tags:
+      - ovn
+      - packages
+
+  - name: "Install central OVN components"
+    apt:
+      state: "present"
+      update_cache: "no"
+      name: "ovn-central"
+    when: ansible_hostname == "docker-01"
+    tags:
+      - ovn
+      - packages
+
+  - name: "Install configuration script"
+    template:
+      src: "setup.sh.j2"
+      dest: "/home/vagrant/setup.sh"
+      owner: "vagrant"
+      group: "vagrant"
+      mode: "0755"
+    tags:
+      - ovn

ovn-docker-ansible/setup.sh.j2

@@ -0,0 +1,14 @@
+#!/bin/bash
+
+# Stop the ovn-controller daemon
+sudo /usr/share/openvswitch/scripts/ovn-ctl stop_controller
+
+# Configure OVS
+sudo ovs-vsctl set Open_vSwitch . \
+    external_ids:ovn-remote="tcp:192.168.100.101:6642" \
+    external_ids:ovn-nb="tcp:192.168.100.101:6641" \
+    external_ids:ovn-encap-ip={{ ansible_enp0s8.ipv4.address }} \
+    external_ids:ovn-encap-type="geneve"
+
+# Start the ovn-controller daemon
+sudo /usr/share/openvswitch/scripts/ovn-ctl start_controller