Securing your infrastructure with Salt
Provisioning servers on DigitalOcean like a king is one thing, but let’s be honest: default installations don’t provide a lot of security. So I’ve spent some time to manage the security on my provisioned boxes. This post (and probably the next as well) will show you how I used SaltStack to secure my infrastructure.
Given the master/minion set-up from the previous posts, I want to have a running firewall which:
- enables SSH access
- enables minion/master communication
- enables HTTP(S) access
Unfortunately, I forgot that the directory in which your
Vagrantfile resides is automatically shared over all machines.
Which means that in practice, I started out with carefully distributing my secret private keys, only to upload them to
all my hosts in the next step.
Luckily, this was easily fixed by the following line in my
Adding a Firewall
The next step was a bit more complicated: adding firewall rules. Although Salt provides an iptables module, I decided to go for ufw. Partially because I like to be contrarian, but mostly because the UFW guys know more about firewalls than me.
So, what to do when Salt does not provide a module for your needs? My first attempt looked something like this.
And, to be honest, this is not a bad first attempt. I define a base firewall which allows SSH access and enables UFW;
I then proceeded to define a specific rule for the salt master which allows TCP connections to the salt master 0MQ ports
4505 and 4506, and I applied these rules to the correct hosts in
Still, this set-up has two major problems. First of all, because the commands are stateless, they are run every time the highstate is ensured. Not a huge problem in itself, but not the most beautiful solution either.
Secondly, in order to only allow 0MQ access from my own hosts, I had to resort to a dirty trick; since the IP address
of the various hosts changes per provider, I really need to allow access based on hostname, something that is not
supported by UFW out of the box. Hence the somewhat dubious
$(getent ahosts nginx01.intranet) fragment in my
Creating Salt modules
Whenever you have a need that is not covered by the basic Salt modules, the Salt documentation suggests you create your own modules. And wow, that’s easy. (Footnote: my first attempt at provisioning was based on Puppet; customizing Puppet requires you to do some weird magic in some kind of almost-language called Ruby. It’s no fun.)
Complicated, no? By putting my execution module in the
_modules/ directory, this new execution module is automatically
synced to all minions. (If not, use
salt '*' saltutil.sync_all to force a new sync). This module is now available
like any salt module:
salt '*' ufw.is_enabled gives you a nice overview of all minions that have UFW enabled.
Note how I used the
__salt__ dict to defer the actual work to the existing
cmd.run function. Quite a nice feature.
The state module turned out to be a bit more complicated, but mostly because it has to do a lot of bookkeeping:
If you’re interested in the helpers and boilerplate, look here.
The final rules
Using these brand new modules, the new SLS files looked lot more like proper state files.
Reprovisioning the VM now behaves as expected – enabling the firewall and adding SSH is executed only once. Furthermore, the Salt master config has become much more readable:
Note how I’ve sneakily enabled pillar data as well. Deducing the configuration for
firewall/http.sls is left as
an exercise to the reader (hint).
Managing your firewall with Salt is not that hard; it requires some module magic, but that is very easy to do. Of course, having a firewall is not enough; stay tuned for the next post where I will add more security measures.
Don’t forget: my infra is a repo, so go ahead and fork it
Do the wave
Follow the path
Choosing your target
the humble beginnings