Chef-solo: A fair solution to manage clusters

Author: Raimon Bosch

If you have been involved in web development you probably have heard about chef. With chef, you can automate how you build, deploy, and manage your infrastructure. Your infrastructure becomes as versionable, testable, and repeatable as application code. In this blog post I will present a guide to learn the basics about this great technology. This will save you lots of time configuring servers.

In this example we have used chef-solo instead of chef. The difference between this two systems is that chef-solo does not require the installation of any server or centralized repository. You only need to boot up a machine, install chef-solo there, and then you can start running chef scripts and perform all the installations you need. The idea of chef language is to define a series of ruby scripts to define all the packages and files that you will install in a new server. For that, you define a series of ruby scripts under the folder ./recipes. For instance, if we want to create a script to install apache2 and activate all the php configurations we would have something like this:

==========./recipes/apache2/default.rb ==========

package “apache2” do
action :install
end

execute “a2enmod rewrite” do
user “root”
end

execute “a2enmod deflate” do
user “root”
end

execute “a2enmod mime” do
user “root”
end

==========./recipes/apache2/php.rb ==========

package “php5” do
action :install
end

package “php5-curl” do
action :install
end

package “libapache2-mod-php5” do
action :install
end

execute “a2enmod php5” do
user “root”
end

package “php-pear” do
action :install
end

With the call to “package” we activate the typical apt-get call as root user if we are working on a ubuntu server. If we are working in a other kind of operative system instead of calling apt-get the system makes a different call. The call “execute” is to indicate shell commands that we want to be executed. In this script you will see how we install the “apache2” package and then we add several apache modules using a “a2enmod” call. We also install several important php libraries that usually we need to run a website.

Chef allows several syntax alternatives that you can check out at https://learn.chef.io/. You can create directories, files with templates, create variables and lots of features. I recommend you to check out pre-cooked recipes at http://cookbooks.opscode.com/. Not to use them because you don’t want to end up with unknown dependencies, but they can be useful to learn the syntax. In this article we won’t focus in the specific syntax of chef recipes, we will focus on the initial steps to install and run a chef recipe ASAP. So here you have the magic commands that you have to run just after you boot your new machine:

Step 1. Installing chef-solo

apt-get update

apt-get install ruby1.9.1 ruby1.9.1-dev build-essential

gem install chef

Step 2. Setting your ssh key to avoid passwords

sudo -i

vim ~/.ssh/authorized_keys (add one of your public keys)

mkdir -p /var/chef (this is the directory where you will put your recipes)

Step 3. Send your chef files from your local machine

rsync -avz –del –rsh=’ssh -p 22′ /var/chef/ root@HOSTNAME:/var/chef/

Step 4. Executing chef-solo recipes

chef-solo -c /var/chef/solo.rb -j /var/chef/dna.json

But wait? What do we have to define in solo.rb and dna.json? This is not very well explained in other tutorials. Here also the syntax is infinite and we will focus on a basic example:

==========./solo.rb ==========

log_level :info
log_location STDOUT
file_cache_path “/var/chef/cache”
cookbook_path [ “/var/chef/cookbooks” ]
Mixlib::Log::Formatter.show_time = true

In this file we define where do we save the cookbooks and several configuration features.

==========./dna.json ==========

{
“run_list”: [
“recipe[apache2]”,
“recipe[apache2::php]”
]
}

In dna.json we define the recipes to run. When we use as name default.rb we do not need to define a complete name such as apache2::recipe_name.rb. See more details in this link https://docs.getchef.com/chef_solo.html

How can we test a chef recipe?

We have shown the basic steps to set up and run chef recipes in a new server. But one might wonder if it is not very difficult to test a chef recipe when we need a new server all the time. The answer is no if you use an excellent tool called Vagrant. With Vagrant you can create a virtual machine on your local machine. No costs and no risks. You can set up in minutes a new server for free where you are allowed to fail and repeat.

==========./Vagrantfile ==========

VAGRANTFILE_API_VERSION = “2”

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.box = “ubuntu/trusty64”

config.vm.network “forwarded_port”, guest: 80, host: 5678
config.vm.network “forwarded_port”, guest: 8084, host: 5679

end

In Vagrantfile you specify which operative system you want and the port redirections (¡very important!). If you want to test a apache or a tomcat you need a way to access to this ports from your local machine. Vagrant uses 127.0.0.1 as host so the ports must be different if you want to see something.

Vagrant Step 0. Connect to your vagrant machine

cd /var/chef

vagrant up

vagrant ssh

Or you can access too via ssh (but with port 2222):

ssh -p 2222 vagrant@127.0.0.1

Vagrant Step 1. Installing chef-solo

#analog to the other step 1

Vagrant Step 2. Setting your ssh key to avoid passwords

#analog to the other step 2

Step 3. Send your chef files from your local machine

#similar to the other step 3, but with port 2222

rsync -avz –del –rsh=’ssh -p 2222′ /var/chef/ root@127.0.0.1:/var/chef/

Step 4. Executing chef-solo recipes

#analog to the other step 4

Conclusions

In my experience it has been useful to test several times my recipes inside a Vagrant box until I was very sure that this won’t fail in a real server. With this technique scaling your computer systems will be a smooth task that you will be able to repeat without pain. It is obvious that you always might find little issues when you want to reuse some recipes. Maybe because a version has changed or you are installing in a different OS, but most of your recipes will be reused every time you scale your system. As a result, you will save lots of time. Configuring machines one by one is definitely a thing of the past with chef-solo. Do not hesitate to contact in case you don’t understand a tutorial step or you need further information. Have fun!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s