Terraforming a Nomad cluster

nomadAll the code from his blog is available on Git.

At Hashiconf, Hashicorp announced Nomad. According to the website, it’s a cluster manager and scheduler for microservices and batch workloads, that works with Docker out of the box. And it’s all in just one binary! Of course that aroused my interest and I decided to play around with it. But rather than set up Vagrant boxes, I prefer to use proper VM’s in Google’s cloud. And by coincidence, Hashicorp has a very neat tool to automate that. It’s called Terraform!

The scripts

I started with the credentials for the provider. The actual values will be read from an accompanying variables.tf file.

Then for the instance I used the default stuff for disk and network. I copy in a server.hcl containing a basic server definition, and provide the install script, for which I used the contents from the Vagrantfile provided by Hashicorp.

The network definition speaks for itself:

And I need to open up some ports in the firewall to be able to access the servers. I setup ssh access, and I’ll allow free traffic between the nodes. I also add my local public address so I can reach any port on the machines.

The last piece is the variables file.

All of this is available on Github, so you can just pull from there, adjust the variables and terraform plan & apply.

Starting the cluster

It should be possible to bootstrap the process as shown in the Atlas Examples on Github. I’m currently working on implementing that, but I ran into yakshaving issues with Systemd. I also chose to run Nomad in both agent and server mode on each node and this gives a nice Terraform variable interpolation challenge. I have no clue yet whether running agent and server combined on each node is recommended or not but I guess I’ll find out along the way.
So for now we’ll start the cluster manually. Once the terraform apply is done, open an ssh connection to each of the servers and start the Nomad agent. Be sure to use the address of the network interface, if you don’t it will bind to the loopback interface and other agents won’t be able to reach it.

Once all the agents are running, open an extra connection to one of the nodes and join the other ones from there. Specify the url of the remote server using the -address parameter. Use the local address as the argument to server-join.

Do this for every OTHER_SERVER in your cluster. Once they are all connected you can start a job, for instance by doing a nomad init which will create an example.nomad file with a Redis task.
Before you run nomad run example.nomad however, set the NOMAD_ADDR environment variable, or else you’ll connect to localhost.

I will be trying out some more of Nomad’s features, so expect more blogs on that. Also I’ll be trying to bootstrap this stuff so watch the GitHub page for development in that direction. As always, keep those comments and suggestions coming!

The following two tabs change content below.

Thijs Schnitger

Senior Engineer at Container Solutions
Thijs is a Systems Engineer specialized in all things Linux. At Container Solutions he mainly focuses on programmable infrastructure, when he’s not busy lending a hand. His years of experience with development, delivery and operations make him an allround application debugger and problem solver.

Latest posts by Thijs Schnitger (see all)

Leave a Reply

Your email address will not be published. Required fields are marked *