How can I distribute the service discovery tool consul to linux hosts? - service-discovery

Question

Consul isn't currently published in a package manager format. Whats a good way to distribute it across many systems and ensure its installed in a consistent manner?

Answer 1

I found that you can easily create a package from the consul binary using fpm:
fpm --verbose -s dir -t rpm -n consul -v 0.4 --url=http://consul.io --vendor=HashiCorp --description "A distributed service discovery tool" ./consul_/consul=/usr/local/bin/consul
That command will create an rpm file in your current working directory. You can also use 'deb' with the -t flag to create a deb package instead.
If you don't already have fpm installed, you can install it with rubygems:
gem install fpm
FPM requires the tools needed to create the package type you choose, so best to install this on a like system (Red Hat or Debain variant for RPM and DEB respectively)

Answer 2

Deliver it as a docker container.
The Dockerfile would:
wget the zip file, unzip it
Make directories for data and config
Add consul.json configuration file
Create volumes for the configuration and data
Expose consul port(s)
Define entrypoint
The Dockerfile would look approximately like this:
RUN wget 'https://dl.bintray.com/mitchellh/consul/0.3.1_linux_amd64.zip' -O consul.zip && unzip -d /usr/bin consul.zip
RUN mkdir -p /opt/consul/data /opt/consul/config
ADD consul.json /opt/consul/config/
VOLUME ["/opt/consul/data","/opt/consul/config"]
EXPOSE 8500
ENTRYPOINT ["/usr/bin/consul", "agent", "-config-dir=/opt/consul/config"]
CMD ["-server", "-bootstrap"]

Answer 3

consul is a single binary, perfectly suited for easy (re-)distribution and handling.
Packaging as .deb is just a three-liner with fpm.
Prerequisite: install fpm with gem install fpm
Full working example for consul 0.6 (current version as of January 2016):
wget -N https://releases.hashicorp.com/consul/0.6.0/consul_0.6.0_linux_amd64.zip
unzip -o consul_0.6.0_linux_amd64.zip
fpm --force --verbose -s dir -t deb -n consul -v 0.6 \
--url=http://consul.io --vendor=HashiCorp \
--description "A distributed service discovery tool" ./consul=/usr/local/bin/consul

Answer 4

There is a puppet module (https://github.com/solarkennedy/puppet-consul) which can help with this. It pulls the binary from dl.bintray.com and also helps out with configuring the system.
Install a server, joining to 172.20.20.10. we are "expecting" a 3 node cluster, so this snippet will work for all three server nodes (even the first as long as it's the "172.20.20.10" copy)
class { 'consul':
join_cluster => '172.20.20.10',
config_hash => {
'datacenter' => 'dc1',
'data_dir' => '/opt/consul',
'log_level' => 'INFO',
'node_name' => $::hostname,
'bind_addr' => $::ipaddress_eth1,
'bootstrap_expect' => 3,
'server' => true,
}
}
that snippet will also work for the client agents (just flip the "server" bit to false.) The last step is to create a service definition and register it with the local consul client agent:
consul::service { "foo",
tags => ['service'],
port => 8080,
check_script => '/opt/foo-health-checker.sh',
check_interval => '5s',
}
Here is an example Vagrantfile to build up a demo stack, complete with a 3 node consul cluster: https://github.com/benschw/consul-cluster-puppet
... and a blog post walking through how it was built: http://txt.fliglio.com/2014/10/consul-with-puppet/

Answer 5

Another option is to reuse one of the existing docker images.
e.g. progrium/consul is a great container designed to work in the Docker ecosystem

Answer 6

If you are interested in Ubuntu packages, I started maintaining a Launchpad PPA at https://launchpad.net/~bcandrea/+archive/ubuntu/consul. It currently targets LTS releases (12.04/14.04) which are the ones I need, but I might consider adding intermediate releases as well.
You can install it with the usual steps:
$ sudo apt-add-repository ppa:bcandrea/consul
$ sudo apt-get update
$ sudo apt-get install consul consul-web-ui
If you want to make a Debian/Ubuntu package for it and distribute it yourself, you might want to look at my Makefile for creating the packages: https://github.com/bcandrea/consul-deb.

Answer 7

Another alternative which only assumes that the linux target machines have ssh daemon running and that from the source machine you can ssh using keys:
Ansible can be installed on the source machine, then use a simple command line as described in http://docs.ansible.com/ansible/latest/intro_adhoc.html#file-transfer
File Transfer Here’s another use case for the /usr/bin/ansible
command line. Ansible can SCP lots of files to multiple machines in
parallel.
To transfer a file directly to many servers:
$ ansible atlanta -m copy -a "src=/etc/hosts dest=/tmp/hosts"
So assuming you already have the needed file consul, prepare a file containing the list of target machines. This is called inventory in ansible jargon - http://docs.ansible.com/ansible/latest/intro_inventory.html
then ansible my_linux_machines -m copy -a "src=consul dest=/usr/bin/consul"
you can also have ansible download the zip, untar it and then copy. search "ansible untar"

Answer 8

There is a multi-platform Ansible role that helps create a Consul cluster with clients:
https://github.com/brianshumate/ansible-consul
Another role of his can add Vault on top of Consul.