prehistoric-systems/scottslowe-learning-tools
1
0
Fork 0
mirror of https://codeberg.org/scottslowe/learning-tools.git synced 2026-03-11 09:04:37 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Update to Consul 1.2.3

Browse source 
Update to Consul 1.2.3. Replatform to Ubuntu 16.04. Update documentation. Add Libvirt support.

Signed-off-by: Scott Lowe <scott.lowe@scottlowe.org>
This commit is contained in:
Scott Lowe 2018-09-29 15:46:50 -06:00
parent ec6be675e0
commit 33ec979ae6
No known key found for this signature in database
GPG key ID: 949F43F6E6C11780
9 changed files with 112 additions and 81 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

28
consul/consul-ansible/README.md
View file

@ -1,42 +1,42 @@
# Running a Consul Cluster in Vagrant (with Ansible)
These files were created to allow users to use Vagrant ([http://www.vagrantup.com](http://www.vagrantup.com)) and Ansible ([http://www.ansible.com](http://www.ansible.com)) to quickly and relatively easily spin up a three-node Consul ([http://www.consul.io](http://www.consul.io)) cluster. The configuration was tested using Vagrant 1.8.1, VMware Fusion 8.1.0, the Vagrant VMware plugin, and Ansible 1.9.1. Other versions of these components may work, but have not been tested.
These files were created to allow users to use Vagrant ([http://www.vagrantup.com](http://www.vagrantup.com)) and Ansible ([http://www.ansible.com](http://www.ansible.com)) to quickly and relatively easily spin up a three-node Consul ([http://www.consul.io](http://www.consul.io)) cluster. The configuration was tested using Vagrant, VMware Fusion with the Vagrant VMware plugin, Libvirt, and Ansible. VirtualBox will probably work fine but hasn't been explicitly tested.
## Contents
* **ansible.cfg**: This Ansible configuration file tells Ansible which inventory file to use (a file named `hosts` in the current directory), which SSH private key to use (the default Vagrant insecure key), and which remote user to use (the default user of `vagrant` that should be common to all Vagrant boxes). No edits to this file should be necessary.
* **ansible.cfg**: This Ansible configuration file tells Ansible which inventory file to use (the default Vagrant-generated inventory), which SSH private key to use (the default Vagrant insecure key), and provides other configuration values. No edits to this file should be necessary.
* **ansible.yml**: This is the Ansible playbook that will configure the Vagrant VMs to become members of a Consul cluster. No edits to this file should be necessary.
* **config.json.j2**: This Jinja2 template is automatically filled in with appropriate values by Ansible when the Ansible playbook is applied against the Vagrant VMs. If the Vagrant box in use does not use `ethX` to refer to network interfaces, this file will need to be edit to use the correct network interface name(s). No edits to this file should be necessary.
* **config.json.j2**: This Jinja2 template is automatically filled in with appropriate values by Ansible when the Ansible playbook is applied against the Vagrant VMs. No edits to this file should be necessary.
* **consul.conf**: This Upstart script configures Consul to run as a background daemon (service) on the Ubuntu-based VMs created by Vagrant. This file is installed by Ansible when the Ansible playbook is applied against the Vagrant VMs. No edits to this file should be necessary.
* **hosts**: This Ansible inventory file is generated automatically by Vagrant once you run `vagrant status` or any other command that requires Vagrant to parse the `Vagrantfile`. Since it is automatically generated, no edits directly to this file are needed (they would be overwritten anyway).
* **consul.service.j2**: This Jinja2 template is used by Ansible to create a systemd unit for Consul. If the Vagrant box in use does not use `ethX` to refer to network interfaces, this file will need to be edit to use the correct network interface name(s). Otherwise, no edits to this file should be necessary.
* **machines.yml**: This YAML file contains a list of VM definitions. It is referenced by `Vagrantfile` when Vagrant instantiates the VMs. Generally, the only change needed to this file is to specify the correct Vagrant box you will be used (see "Instructions" below). If necessary, you may need to edit the IP addresses supplied in this file to avoid IP addressing conflicts with other networks.
* **provision.yml**: This is the Ansible playbook that will configure the Vagrant VMs to become members of a Consul cluster. No edits to this file should be necessary.
* **README.md**: This file you're currently reading.
* **setup.yml**: This simple Ansible playbook installs Python 2 via the `raw` module. No edits to this file should be necessary.
* **Vagrantfile**: This file is used by Vagrant to spin up the virtual machines. This file is fairly extensively commented to help explain what's happening. You should be able to use this file unchanged; all the VM configuration options are stored outside this file.
## Instructions
These instructions assume you've already installed VMware Fusion, Vagrant, the Vagrant VMware plugin, and Ansible. Please refer to the documentation for those products for more information on installation or configuration.
These instructions assume you've already installed Vagrant, your virtualization provider (and any associated plugins needed to use it with Vagrant), and Ansible. Please refer to the documentation for those products for more information on installation or configuration.
1. Use `vagrant box add` to install an Ubuntu 14.04 x64 box for the vmware_fusion provider. The "bento/ubuntu-14.04" box is a good option here.
1. Use `vagrant box add` to install an Ubuntu 16.04 x64 box for your virtualization provider. The `machines.yml` file contains some suggested boxes for various providers.
2. Place the files from the `consul` directory of this GitHub repository into a directory on your local system. You can clone the entire "learning-tools" repository (using `git clone`) or just download the specific files from the the `consul` folder.
2. Place the files from the `consul-ansible` directory of this GitHub repository into a directory on your local system. You can clone the entire "learning-tools" repository (using `git clone`) or just download the specific files from the the `consul-ansible` folder.
3. If you are using a Vagrant box other than my Ubuntu 14.04 base box (referred to in step #1), edit the `machines.yml` file to specify the box in use. If necessary to avoid IP address conflicts with existing networks, you may also need to edit the IP addresses specified in this file. Generally, no other changes are needed, although (if you are comfortable with the settings) you can adjust the number of virtual CPUs and/or the amount of RAM assigned to each Vagrant VM in this file as well. Note that this environment _assumes_ the presence of an `eth1` in each Vagrant VM; therefore, do not remove the "ip_addr" value from `machines.yml`.
3. If you are using a Vagrant box other than one of the ones listed in `machines.yml`, you'll need to edit `machines.yml` file to specify the box in use. If necessary to avoid IP address conflicts with existing networks, you may also need to edit the IP addresses specified in this file. Generally, no other changes are needed.
4. Once you have edited `machines.yml`, use `vagrant up` to bring up the 3 systems that will serve as your Consul cluster.
5. Once Vagrant has finished bringing up the VMs, run `ansible-playbook ansible.yml`. Ansible will use the configuration file in the current directory to pull inventory from the file named `hosts` (it was automatically generated when you ran `vagrant up`). This will provision and configure Consul on each of the Vagrant VMs.
5. Once Vagrant has finished bringing up the VMs, run `ansible-playbook ansible.yml`. Ansible will provision and configure Consul on each of the Vagrant VMs.
**NOTE**: Due to the way the Ansible provisioner in Vagrant works, it's currently not possible to provision the VMs with Ansible from within the `Vagrantfile`. As a result, step #5 (running `ansible-playbook` manually) is needed.
At this point, you have a functional Consul cluster running under Vagrant. If you are using VMware Fusion, you should have IP connectivity to the VMs, and can use the OS X `consul` binary to connect to the cluster and test it. For example, this command would work to demonstrate that Consul is working (you would need to change the IP address provided after `-rpc-addr`):
At this point, you have a functional Consul cluster running under Vagrant. You can use `vagrant ssh <vm-name>` to connect to one of the VMs and run the following command to demonstrate that Consul is working:
consul members -rpc-addr=192.168.1.101:8400

46
consul/consul-ansible/Vagrantfile vendored
View file

@ -9,16 +9,9 @@ VAGRANTFILE_API_VERSION = '2'
require 'yaml'
# Read YAML file with VM details (box, CPU, RAM, IP addresses)
# Be sure to edit machines.yml to provide correct IP addresses
machines = YAML.load_file(File.join(File.dirname(__FILE__), 'machines.yml'))
# Create custom Ansible inventory file (can't use built-in inventory)
f = File.open('hosts','w')
machines.each do |machine|
line = machine['name'] + " ansible_ssh_host=" + machine['ip_addr']
f.puts line
end # machines.each
f.close
# Create and configure the VMs
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
@ -27,11 +20,17 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
# Iterate through entries in YAML file to create VMs
machines.each do |machine|
# Configure the VMs per details in machines.yml
config.vm.define machine['name'] do |srv|
# Don't check for box updates
srv.vm.box_check_update = false
# Specify hostname and Vagrant box
srv.vm.hostname = machine['name']
# Specify the Vagrant box to use (use VMware box by default)
srv.vm.box = machine['box']['vmw']
# Configure default synced folder (disable by default)
@ -41,12 +40,16 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
srv.vm.synced_folder '.', '/vagrant', disabled: true
end #if machine['sync_disabled']
# Assign additional private network
if machine['ip_addr'] != nil
srv.vm.network 'private_network', ip: machine['ip_addr']
end # if machine['ip_addr']
# Iterate through networks as per settings in machines.yml
machine['nics'].each do |net|
if net['ip_addr'] == 'dhcp'
srv.vm.network net['type'], type: net['ip_addr']
else
srv.vm.network net['type'], ip: net['ip_addr']
end # if net['ip_addr']
end # machine['nics'].each
# Configure CPU & RAM per settings in machines.yml (Fusion)
# Configure VMs with RAM and CPUs per settings in machines.yml (Fusion)
srv.vm.provider 'vmware_fusion' do |vmw|
vmw.vmx['memsize'] = machine['ram']
vmw.vmx['numvcpus'] = machine['vcpu']
@ -55,12 +58,27 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
end #if machine['nested']
end # srv.vm.provider 'vmware_fusion'
# Configure CPU & RAM per settings in machines.yml (VirtualBox)
# Configure VMs with RAM and CPUs per settings in machines.yml (VirtualBox)
srv.vm.provider 'virtualbox' do |vb, override|
vb.memory = machine['ram']
vb.cpus = machine['vcpu']
override.vm.box = machine['box']['vb']
end # srv.vm.provider 'virtualbox'
# Configure CPU & RAM per settings in machines.yml (Libvirt)
srv.vm.provider 'libvirt' do |lv,override|
lv.memory = machine['ram']
lv.cpus = machine['vcpu']
override.vm.box = machine['box']['lv']
if machine['nested'] == true
lv.nested = true
end # if machine['nested']
end # srv.vm.provider 'libvirt'
end # config.vm.define
# Install Python 2 to support other Ansible playbooks
config.vm.provision 'ansible' do |ansible|
ansible.playbook = 'setup.yml'
end # config.vm.provision
end # machines.each
end # Vagrant.configure

5
consul/consul-ansible/ansible.cfg
View file

@ -1,5 +1,6 @@
[defaults]
inventory = ./hosts
inventory = ./.vagrant/provisioners/ansible/inventory/vagrant_ansible_inventory
private_key_file = ~/.vagrant.d/insecure_private_key
remote_user = vagrant
host_key_checking = False
host_key_checking = False
retry_files_enabled = False

21
consul/consul-ansible/consul.conf
View file

@ -1,21 +0,0 @@
description "Consul server process"
start on runlevel [2345]
stop on runlevel [!2345]
respawn
script
if [ -f "/etc/service/consul" ]; then
. /etc/service/consul
fi
export GOMAXPROCS=`nproc`
BIND_ADDR=`hostname -I | cut -f2 -d' '`
exec /usr/local/bin/consul agent \
-config-dir="/etc/consul.d/server" \
-bind $BIND_ADDR \
${CONSUL_FLAGS} \
>>/var/log/consul.log 2>&1
end script

12
consul/consul-ansible/consul.service.j2 Normal file
View file

@ -0,0 +1,12 @@
[Unit]
Description=Consul Server
Requires=network-online.target
After=network-online.target
[Service]
User=consul
ExecStart=/usr/local/bin/consul agent -config-dir="/etc/consul.d/server" -bind={{ hostvars[inventory_hostname]['ansible_eth1']['ipv4']['address'] }}
Restart=on-failure
[Install]
WantedBy=multi-user.target

3
consul/consul-ansible/hosts
View file

@ -1,3 +0,0 @@
consul-01 ansible_ssh_host=192.168.100.101
consul-02 ansible_ssh_host=192.168.100.102
consul-03 ansible_ssh_host=192.168.100.103

45
consul/consul-ansible/machines.yml
View file

@ -1,22 +1,37 @@
---
- name: "consul-01"
box:
vmw: "bento/ubuntu-14.04"
vb: "ubuntu/trusty64"
- box:
lv: "generic/ubuntu1604"
vmw: "bento/ubuntu-16.04"
vb: "ubuntu/xenial64"
name: "consul-01"
nested: false
nics:
- type: "private_network"
ip_addr: "192.168.100.101"
ram: "512"
sync_disabled: true
vcpu: "1"
ip_addr: "192.168.100.101"
- name: "consul-02"
box:
vmw: "bento/ubuntu-14.04"
vb: "ubuntu/trusty64"
- box:
lv: "generic/ubuntu1604"
vmw: "bento/ubuntu-16.04"
vb: "ubuntu/xenial64"
name: "consul-02"
nested: false
nics:
- type: "private_network"
ip_addr: "192.168.100.102"
ram: "512"
sync_disabled: true
vcpu: "1"
ip_addr: "192.168.100.102"
- name: "consul-03"
box:
vmw: "bento/ubuntu-14.04"
vb: "ubuntu/trusty64"
- box:
lv: "generic/ubuntu1604"
vmw: "bento/ubuntu-16.04"
vb: "ubuntu/xenial64"
name: "consul-03"
nested: false
nics:
- type: "private_network"
ip_addr: "192.168.100.103"
ram: "512"
sync_disabled: true
vcpu: "1"
ip_addr: "192.168.100.103"

25
consul/consul-ansible/provision.yml
View file

@ -1,11 +1,10 @@
---
- hosts: "all"
sudo: "yes"
remote_user: "vagrant"
become: "yes"
tasks:
- name: Remove i386 architecture to eliminate apt-get errors
command: "/usr/bin/dpkg --remove-architecture i386"
# - name: Remove i386 architecture to eliminate apt-get errors
# command: "/usr/bin/dpkg --remove-architecture i386"
- name: Install unzip package
apt:
@ -41,20 +40,20 @@
- name: Download Consul package
get_url:
url: "https://releases.hashicorp.com/consul/0.6.0/consul_0.6.0_linux_amd64.zip"
dest: "/tmp/consul_0.6.0_linux_amd64.zip"
url: "https://releases.hashicorp.com/consul/1.2.3/consul_1.2.3_linux_amd64.zip"
dest: "/tmp/consul_1.2.3_linux_amd64.zip"
- name: Unzip downloaded Consul file
unarchive:
copy: "no"
src: "/tmp/consul_0.6.0_linux_amd64.zip"
src: "/tmp/consul_1.2.3_linux_amd64.zip"
dest: "/usr/local/bin/"
creates: "/usr/local/bin/consul"
- name: Install Upstart script for Consul
copy:
src: "consul.conf"
dest: "/etc/init/consul.conf"
- name: Install systemd unit file for Consul
template:
src: "consul.service.j2"
dest: "/etc/systemd/system/consul.service"
owner: "root"
group: "root"
mode: "0644"
@ -68,6 +67,8 @@
mode: "0644"
- name: Start Consul service
service:
systemd:
name: "consul"
enabled: "yes"
daemon_reload: "yes"
state: "started"

8
consul/consul-ansible/setup.yml Normal file
View file

@ -0,0 +1,8 @@
---
- hosts: "all"
gather_facts: false
become: "yes"
tasks:
- name: Install Python
raw: "apt-get -y -q install python"