Merge pull request #67 from lowescott/ovn-kvm

Add learning environment for KVM+OVN

ovn-kvm/README.md

@@ -0,0 +1,59 @@
+# Open Virtual Network (OVN) with KVM/Libvirt
+
+These files were created to allow users to use Vagrant ([http://www.vagrantup.com](http://www.vagrantup.com)) to quickly and  easily spin up an environment to test Open Virtual Network (OVN), a part of the Open vSwitch project, and use it to provide connectivity to Libvirt/KVM-based virtual machines (VMs).
+
+## Contents
+
+* **ansible.cfg**: This is an Ansible configuration file that instructs Ansible to use the "vagrant" remote user, and to use Vagrant's built-in Ansible inventory.
+
+* **final-setup.sh**: This shell script is used to integrate OVS with OVN. (See the "Instructions" below for how/when it is used.)
+
+* **machines.yml**: This YAML file contains a list of VM definitions and associated configuration data. It is referenced by `Vagrantfile` when Vagrant instantiates the VMs.
+
+* **net-setup.sh**: This shell script configures Libvirt to integrate with OVS. (Refer to the "Instructions" section below for more information.)
+
+* **ovn-setup.sh**: This Jinja2 template is used by Ansible to build a customized OVS/OVN configuration script for each Vagrant machine. The shell script created by this template is placed by Ansible in `/home/vagrant/ovn-setup.sh`.
+
+* **ovs.xml**: This Libvirt network XML definition is used by `net-setup.sh` to facilitate Libvirt-OVS integration.
+
+* **provision.yml**: This Ansible playbook installs the OVS/OVN components on each Vagrant machine. It's called automatically by Vagrant.
+
+* **README.md**: This file you're currently reading.
+
+* **Vagrantfile**: This file is used by Vagrant to spin up the virtual machines. This file is fairly extensively commented to help explain what's happening. You should be able to use this file unchanged; all the VM configuration options are stored outside this file.
+
+* **vm-setup.sh**: This shell script launches a simple VM for testing OVN connectivity.
+
+## Instructions
+
+These instructions assume you've already installed Vagrant, your back-end virtualization provider (such as VMware Fusion or VirtualBox), and any necessary plugins (such as the Vagrant VMware plugin). Please refer to the documentation for those products for more information on installation or configuration. _Note that this environment relies on nested virtualization support, and therefore will **not** work with VirtualBox._
+
+1. Use `vagrant box add` to add an Ubuntu 16.04 box to your system. The "bento/ubuntu-16.04" box is very good for both VirtualBox and VMware Fusion/Workstation. (Note that the "ubuntu/xenial64" box for VirtualBox is currently broken under Vagrant.)
+
+2. Edit the `machines.yml` file to ensure the box you downloaded in step 1 is specified on the "box:" sections of this file. Specify the name of a VMware-formatted box on the "vmw:" line; place the name of a VirtualBox-formatted box on the "vb:" line. (Note that the "vb:" line will be ignored, as this environment is not supported with VirtualBox.)
+
+3. Run `vagrant up` to have Vagrant instantiate the three machines configured by default in this environment, and provision them using Ansible. (Note you'll need Ansible installed locally on the system where you're running `vagrant up`.)
+
+4. Use `vagrant ssh hv-01` to log into the first system. Run `sudo ./setup.sh` to perform final configuration steps for OVS/OVN.
+
+5. Repeat step #4 with `hv-02` and `hv-03`. OVN is now up and running, but not integrated with KVM/Libvirt.
+
+6. Log into `hv-01` and run `sudo ./net-setup` to set up the Libvirt-OVS integration (it will create a Libvirt network backed by the OVS integration bridge).
+
+7. Repeat step #6 with `hv-02` and `hv-03`.
+
+8. On `hv-01`, run `sudo ./vm-setup.sh` to launch a VM and attach it to OVS using the Libvirt integration.
+
+9. Repeat step #8 on `hv-02` and `hv-03`. You now have 3 VMs running across 3 nested KVM hypervisors.
+
+10. On `hv-01`, create an OVN logical switch named "demo" using this command:
+
+        sudo ovn-nbctl ls-add demo
+
+11. Still on `hv-01`, run `sudo ./final-sh`. This will create an OVN logical port and link it back to OVS and Libvirt.
+
+12. Repeat step #11 on `hv-02` and `hv-03`. You now have all three VMs linked together via an OVN logical switch.
+
+13. Log into the console of each VM (using whatever method you prefer) and assign an IP address. Once all three VMs are assigned an IP address from the same subnet, you should have connectivity among the VMs. This connectivity is provided by OVN.
+
+Enjoy!

ovn-kvm/Vagrantfile

@@ -0,0 +1,67 @@
+# -*- mode: ruby -*-
+# vi: set ft=ruby :
+
+# Specify minimum Vagrant version and Vagrant API version
+Vagrant.require_version '>= 1.6.0'
+VAGRANTFILE_API_VERSION = '2'
+
+# Require 'yaml' module
+require 'yaml'
+
+# Read YAML file with VM details (box, CPU, RAM, IP addresses)
+# Edit machines.yml to change VM configuration details
+machines = YAML.load_file(File.join(File.dirname(__FILE__), 'machines.yml'))
+
+# Create and configure the VMs
+Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
+
+  # Always use Vagrant's default insecure key
+  config.ssh.insert_key = false
+
+  # Iterate through entries in YAML file to create VMs
+  machines.each do |machine|
+
+    # Configure the VMs per details in machines.yml
+    config.vm.define machine['name'] do |srv|
+
+      # Don't check for box updates
+      srv.vm.box_check_update = false
+
+      # Specify the hostname of the VM
+      srv.vm.hostname = machine['name']
+
+      # Specify the Vagrant box to use (use VMware box by default)
+      srv.vm.box = machine['box']['vmw']
+
+      # Configure default synced folder (disable by default)
+      if machine['sync_disabled'] != nil
+        srv.vm.synced_folder '.', '/vagrant', disabled: machine['sync_disabled']
+      else
+        srv.vm.synced_folder '.', '/vagrant', disabled: true
+      end #if machine['sync_disabled']
+
+      # Iterate through networks as per settings in machines.yml
+      machine['nics'].each do |net|
+        if net['ip_addr'] == 'dhcp'
+          srv.vm.network net['type'], type: net['ip_addr']
+        else
+          srv.vm.network net['type'], ip: net['ip_addr']
+        end # if net['ip_addr']
+      end # machine['nics'].each
+
+      # Configure CPU & RAM per settings in machines.yml (Fusion)
+      srv.vm.provider 'vmware_fusion' do |vmw|
+        vmw.vmx['memsize'] = machine['ram']
+        vmw.vmx['numvcpus'] = machine['vcpu']
+        if machine['nested'] == true
+          vmw.vmx['vhv.enable'] = 'TRUE'
+        end #if machine['nested']
+      end # srv.vm.provider 'vmware_fusion'
+    end # config.vm.define
+  end # machines.each
+
+  # Configure the machines with Ansible
+  config.vm.provision 'ansible' do |ansible|
+    ansible.playbook = 'provision.yml'
+  end # config.vm.provision
+end # Vagrant.configure

ovn-kvm/ansible.cfg

@@ -0,0 +1,5 @@
+[defaults]
+inventory = ./.vagrant/provisioners/ansible/inventory/vagrant_ansible_inventory
+private_key_file = ~/.vagrant.d/insecure_private_key
+remote_user = vagrant
+host_key_checking = False

ovn-kvm/final-setup.sh

@@ -0,0 +1,14 @@
+#!/bin/bash
+
+# Get VM interface's iface-id (assumes VM's interface is "vnet0")
+IFACE_ID=$(sudo ovs-vsctl get interface vnet0 external_ids:iface-id | sed s/\"//g)
+
+# Get VM interface's MAC address (assumes VM's interface is "vnet0")
+MAC_ADDR=$(sudo ovs-vsctl get interface vnet0 external_ids:attached-mac | sed s/\"//g)
+
+# Create a new logical port on logical switch
+# Logical port name *must* match VM interface's iface-id value
+sudo ovn-nbctl --db=tcp:192.168.100.101:6641 lsp-add demo $IFACE_ID
+
+# Set MAC address for new logical switch port
+sudo ovn-nbctl --db=tcp:192.168.100.101:6641 lsp-set-addresses $IFACE_ID $MAC_ADDR

ovn-kvm/machines.yml

@@ -0,0 +1,31 @@
+---
+- name: "hv-01"
+  box:
+    vmw: "bento/ubuntu-16.04"
+    vb: "bento/ubuntu-16.04"
+  nested: true
+  ram: "1024"
+  vcpu: "1"
+  nics:
+    - type: "private_network"
+      ip_addr: "192.168.100.101"
+- name: "hv-02"
+  box:
+    vmw: "bento/ubuntu-16.04"
+    vb: "bento/ubuntu-16.04"
+  nested: true
+  ram: "1024"
+  vcpu: "1"
+  nics:
+    - type: "private_network"
+      ip_addr: "192.168.100.102"
+- name: "hv-03"
+  box:
+    vmw: "bento/ubuntu-16.04"
+    vb: "bento/ubuntu-16.04"
+  nested: true
+  ram: "1024"
+  vcpu: "1"
+  nics:
+    - type: "private_network"
+      ip_addr: "192.168.100.103"

ovn-kvm/net-setup.sh

@@ -0,0 +1,10 @@
+#!/bin/bash
+
+# Define Libvirt network from XML
+sudo virsh net-define /home/vagrant/ovs.xml
+sudo virsh net-autostart ovs
+sudo virsh net-start ovs
+
+# Turn off default network
+sudo virsh net-autostart --disable default
+sudo virsh net-destroy default

ovn-kvm/ovn-setup.sh.j2

@@ -0,0 +1,14 @@
+#!/bin/bash
+
+# Stop the ovn-controller daemon
+sudo /usr/share/openvswitch/scripts/ovn-ctl stop_controller
+
+# Configure OVS
+sudo ovs-vsctl set Open_vSwitch . \
+    external_ids:ovn-remote="tcp:192.168.100.101:6642" \
+    external_ids:ovn-nb="tcp:192.168.100.101:6641" \
+    external_ids:ovn-encap-ip={{ ansible_ens33.ipv4.address }} \
+    external_ids:ovn-encap-type="geneve"
+
+# Start the ovn-controller daemon
+sudo /usr/share/openvswitch/scripts/ovn-ctl start_controller

ovn-kvm/ovs.xml

@@ -0,0 +1,6 @@
+<network>
+  <name>ovs</name>
+  <forward mode='bridge'/>
+  <bridge name='br-int'/>
+  <virtualport type='openvswitch'/>
+</network>

ovn-kvm/provision.yml

@@ -0,0 +1,91 @@
+---
+- hosts: "all"
+  become: "yes"
+  remote_user: "vagrant"
+
+  tasks:
+  - name: "Install Ubuntu Cloud Keyring package"
+    apt:
+      state: "present"
+      update_cache: "yes"
+      name: "ubuntu-cloud-keyring"
+    tags:
+      - "apt-cfg"
+
+  - name: "Add Ubuntu Cloud Archive repository"
+    apt_repository:
+      repo: "{{ item }}"
+      state: "present"
+      update_cache: "yes"
+    with_items:
+      - "deb http://ubuntu-cloud.archive.canonical.com/ubuntu xenial-updates/newton main"
+    tags:
+      - "apt-cfg"
+
+  - name: "Install KVM, Libvirt, OVS/OVN 2.6.0"
+    apt:
+      state: "present"
+      update_cache: "no"
+      name: "{{ item }}"
+    with_items:
+      - "ovn-common"
+      - "ovn-host"
+      - "openvswitch-common"
+      - "openvswitch-switch"
+      - "qemu-kvm"
+      - "libvirt-bin"
+      - "virtinst"
+    tags:
+      - "packages"
+
+  - name: "Install central OVN components"
+    apt:
+      state: "present"
+      update_cache: "no"
+      name: "ovn-central"
+    when: ansible_hostname == "hv-01"
+    tags:
+      - "packages"
+
+  - name: "Install configuration script"
+    template:
+      src: "ovn-setup.sh.j2"
+      dest: "/home/vagrant/ovn-setup.sh"
+      owner: "vagrant"
+      group: "vagrant"
+      mode: "0755"
+    tags:
+      - "config"
+
+  - name: "Download Cirros image"
+    get_url:
+      url: "https://download.cirros-cloud.net/0.3.2/cirros-0.3.2-x86_64-disk.img"
+      dest: "/home/vagrant/cirros-0.3.2-x86_64-disk.img"
+      validate_certs: "no"
+
+  - name: "Set file permissions"
+    file:
+      path: "/home/vagrant/cirros-0.3.2-x86_64-disk.img"
+      owner: "vagrant"
+      group: "vagrant"
+      mode: "0644"
+
+  - name: "Copy network definition"
+    copy:
+      src: "ovs.xml"
+      dest: "/home/vagrant/ovs.xml"
+      owner: "vagrant"
+      group: "vagrant"
+      mode: "0644"
+
+  - name: "Copy network and VM setup scripts"
+    copy:
+      src: "{{ item }}"
+      dest: "/home/vagrant/"
+      owner: "vagrant"
+      group: "vagrant"
+      mode: "0755"
+    with_items:
+      - "net-setup.sh"
+      - "vm-setup.sh"
+      - "final-setup.sh"

ovn-kvm/vm-setup.sh

@@ -0,0 +1,10 @@
+#!/bin/bash
+
+# Make a copy of the CirrOS disk image
+cp cirros-0.3.2-x86_64-disk.img cirros-01.img
+
+# Launch a VM using the copy of the disk image
+sudo virt-install --name=cirros --ram=256 --vcpus=1 \
+  --disk path=./cirros-01.img,format=qcow2,bus=virtio \
+  --import --vnc --noautoconsole --hvm \
+  --network network:ovs,model=virtio