Selecting Chef Servers With Environment Variables

Today I got playing around with dynamically selecting different chef servers in preparation for migrating some of our nodes away from our chef-dev server to our chef-live server (which I’m currently in the process of building and populating with data). I had been talking in the #chef IRC channel a few weeks back about making things dynamic, or at least easily switchable, when using multiple chef servers for different groups of servers in an environment.

What I want to do, is be able to set an environment variable at my console in order to switch between chef servers. Previously I had been doing this with different files in my ~/.chef/ directory and changing symlinks between the files. This method works, but is kind of annoying. So with the help of some of the folks in #chef, and with this gist of a sample file that someone is using for their hosted chef environment, I was able to build my own knife.rb and commit it to our chef.git repository.

In our chef.git repository, I created a directory .chef and placed a knife.rb file in it:

$ cd ~/src/chef-repo
$ mkdir .chef
$ touch .chef/knife.rb

I then filled knife.rb with the following contents:

current_dir = File.dirname(__FILE__)

sys_user = ENV["USER"]

log_level                :info
log_location             STDOUT
node_name                sys_user
client_key               "#{ENV["HOME"]}/.chef/#{ENV["KNIFE_ENV"]}/#{ENV["USER"]}.pem"
validation_client_name   "chef-validator"
validation_key           "#{ENV["HOME"]}/.chef/#{ENV["KNIFE_ENV"]}/validator.pem"
chef_server_url          "http://chef-#{ENV["KNIFE_ENV"]}.shifteight.org:4000"
cache_type               'BasicFile'
cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )
cookbook_path            [ "#{current_dir}/../cookbooks", "#{current_dir}/../site-cookbooks" ]

The main key is the KNIFE_ENV environment variable which I set using: export KNIFE_ENV=dev or export KNIFE_ENV=live

After setting the environment variable, which server I’m using is selected for me. Additionally, I copied my validation.pem and client.pem files into corresponding directories in my ~/.chef/ directory: $ mkdir ~/.chef/{live,dev}

With all that done, I can now easily switch between our different servers in order to start the migration of our nodes. (I might create another blog post about that in the future if I get a chance.)

“BUT HOW DO I KNOW WHICH ENVIRONMENT I’M WORKING WITH?!?!?!”, you say? Oh fancy this little PS1 and function I added to my ~/.bashrc file:

if [ "$KNIFE_ENV" == "" ]; then
 export KNIFE_ENV="dev"
fi

function which_env {
  if [ "$KNIFE_ENV" == "live" ]; then
    echo "31"
  else
    echo "32"
  fi
}

export PS1='[\u@\h \[\033[0;36m\]\W$(__git_ps1 "\[\033[0m\]\[\033[0;33m\](%s) \[\033[0;`which_env`m\]~$KNIFE_ENV~")\[\033[0m\]]\$ '

Is nice 🙂

Integration Testing Using Jenkins (Part 1)

So for the last week or so, I’ve been tasked at CoreDial with adding our own set of integration testing now that we’re moving to a more formal deployment method using chef. After much pestering of questions to thehar of Lookout Mobile Security and with help of Google, #chef and jhansche in #jenkins I’ve finally got a nice clean proof of concept that we can evaluate and likely deploy.

I’ll come back later with another article on my installation issues with jenkins and the solutions that I solved (nothing too terribly complicated), but what I wanted to blog about was the two types of tests that I’ve been focusing on and was able to finally solve.

First, I wanted to simply get a working test going in jenkins since I’d never used it before and needed a minimum viable product to look at. Based on a recommendation from thehar a couple weeks ago, I looked at foodcritic, got that working, and with their instructions, was able to get that integrated for my first automated test in jenkins.

The main problem I had was really getting an environment path variable set so that I could execute a ruby shell (#!/usr/bin/env rvm-shell 1.9.3, in the foodcritic instructions). After some searching, I came across a hint (sorry, I’ve misplaced the link) that stated I needed to add source /etc/profile to the bottom of my /etc/default/jenkins file, which worked marvellously to get the command I was trying to run to go. (Note that I installed on Ubuntu 12.04 for this test.)

(Prior to that, I installed rvm and then ran the multi-user instructions to get ruby 1.9.3 installed. I also installed foodcritic via gem install foodcritic which depends on ruby 1.9.2+.)

Having created my first job, I filled in the Git information to connect to my git server. I ran into a few issues there, and needed to create a new .ssh directory in /var/lib/jenkins/.ssh/ (/var/lib/jenkins is the $HOME directory of jenkins). I then placed the appropriate authentication keys in the directory, but was still having issues with connecting to the server. It ended up being that I needed to add a config file to the .ssh directory with the following contents:

Host coredial-git
  HostName gitserver.hostname.com
  User git
  IdentityFile /var/lib/jenkins/.ssh/id_rsa.key
  StrictHostKeyChecking no

After adding this, then I could set the repository URL to git@coredial-git:chef-repo.git and the branch specifier to something like */feature/ENG-* in order to test all our engineering testing branches. I then setup Poll SCM with polling schedule */5 * * * * (I set to */1 at first for testing, and will likely increase this further, or add a post-commit hook to git.)

The actual command I’m running in the Execute Shell section looks like this:

#!/usr/bin/env rvm-shell 1.9.3
foodcritic -f any site-cookbooks/my_awesome_cookbook

Then I saved the test, made some changes, and during the poll was able to trigger off both expected failed and expected passing tests. Very cool indeed!