Chef Server and the risks of depending on the open source community

As a DevOps engineer I’ve played around with a number of different automation tools. Some professionally, some for a blog post, and some just because I was curious about them. When most companies are evaluating automation tools they look at some common attributes

  1. How much will it cost?
  2. Does it support the platforms we’re on?
  3. Is it a good fit for my teams skills?
  4. How long will it take to setup and see some value add?

And then there run down the list of common tools like Docker, Ansible, Chef, Puppet, Powershell DSC, and on until they find one that fits their needs will enough. But I’d argue there’s another aspect to most of these tools that’s harder to quantify, but more impactful to the long term success of a project or tool: how excited is the open source community about the tool.

Most of these tools have at least some level of dependency on open source development, whether it’s the Moby Linux vm behind running Linux containers on Windows machines, or the chef cookbooks in your Berksfile, you’re probably using something that’s open source. The effect that has on your project success can be subtle.

I’ve had two projects at work I’d like to walk you through: migrating a Django app into Docker, and adding AWS’s boto3 to a server using chef.

When I started on the Dockerfile for the Django app I googled “Django app in docker” and immediately found several good examples with notes, explanations, and a simple demo app. I could pick my Django and python versions using image tags, and I had my app starting in a container in about an hour (after that I spent a day hunting down undocumented environment variables, but that’s on us, not docker).

Overall it was a smooth process, there was rich documentation, and every question about tooling had already been answered on forums.

Compare that to trying to add a python package to a server using chef. I started off fighting with chef test kitchen. It took my an hour or so to figure out all of the variables I needed to create a server with the ec2 kitchen driver (env vars, values swapped out in my kitchen.yml file, drivers I needed to download, etc.)

Once I had an instance created in AWS I had to find an example of installing a python package. After 20 or so minutes of googling I landed on Poise python, which mostly works, but hasn’t been updated in close to a year.

The package I needed was psycopg2, and I got it added to my cookbook pretty easily, but then I tried to rerun kitchen converge, and I got an error from pretty deep inside of poise python. I’ve always believed part of the unwritten contract you agree to with open source is digging into bugs when you find them, so I dove in and eventually found the problem in here doc that creates a python script to install pip modules.

Looking at the heredoc gave me a few ideas for tweaking my chef recipe to use poise python differently and avoid the error, and after experimenting for a while I found a combination of version pinning that worked and I could converge my node.

Not long after that I a commit and PR ready to add the python module to the new node and I was done.

So what’s the difference here? The work I was doing was pretty similar in both situations, I needed to update packages for a python application, but the docker experience was smooth, there were lots of examples, and it only took a few hours total. The chef experience was painful, I spent time digging through source code, debugging my own problems, and had trouble finding examples.

To be clear I’m not promoting Docker over Chef. They’re different tools that solve similar problems. I’m trying to point out that community support has moved away from Chef and the open source Chef cookbooks over the last 3–5 years. That transition makes the tool much harder to use because there are fewer people to contribute and help solve problems.

And the scary thing is that the exact same thing could happen to Docker, leaving all of the companies containerizing their applications high and dry with less free community support than they have now. The open source world is enamored with Docker today, but they may not be tomorrow. So before you commit to an open source tool for the cost or other reasons above, I’d encourage you to ask these questions too

  1. Am I willing to contribute back to this open source project to solve my own problems?
  2. Is the open source community moving towards, or away from this project?
  3. How bad is it for my org if we start to find ourselves with less open source support for this?
  4. Am I willing to pay for enterprise support, or hire an open source community member if it comes to it?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s