RichyHBM

Software engineer with a focus on game development and scalable backend development

Setting up Continuous Integration with Jenkins

Recently I had to convert a manual jenkins build job into an automated job that will trigger each time a push gets made to the SCM server. For SCM I use git with the help of a git management tool called gitolite which allows you to setup and configure your git management from within a git repository, talk about meta.

There are a number of ways to setup a build trigger in jenkins, you can allow building by hitting a URL, build periodically, have jenkins poll your SCM, or if you are on Github you can have jenkins setup git hooks to build on a git push.

A simple way of doing this could be to have jenkins just poll your SCM every 5 minutes, but depending on the amount of repositories this could mean hitting your version control server quite a lot. Ideally what would be better is to have the version control tell jenkins each time a build is required, and that is what I set up.

Setting jenkins up

The first thing to do is to set jenkins up to allow remote build triggers, this allows you to trigger a build by hitting a special URL. There also needs to be a secure token set so that you can’t get anyone hitting the right URL and triggering a build on your servers.

With this enabled you should now be able to hit the URL to trigger your jenkins build.

The next step is optional, but allows you to automatically email everyone involved in the last build if the build breaks or has issues. Simply add a new build parameter, this will act as a list of users to email and will be set up by our git hook when triggering a new build.

If you have any other build parameters you may want to set sensible defaults for them in order to not have to send this data on each request.

Next up is enabling the email plugin for jenkins, or if required one of the more advanced email plugins, using our previously created parameter as the list of recipients for the email.

This plugin will email every time the build fails or goes from failure to success, amongst other things.

Finally, this is more of an FYI than anything else, I tried using the advanced email plugin and found that by default it doesn’t send emails to the set recipients but rather to whomever jenkins things are the committers of that specific commit.

Setting up git hooks

Next up is setting up the actual git hook that will call jenkins and trigger the build, in this case I am using a post-receive git hook to trigger after each git push.

The script looks like this

#!/bin/bash

while read oldrev newrev refname
do
  COMMITERS=$(git log $oldrev..$newrev --format="%ae" | sort | uniq)
  EMAIL_COMMITERS=''

  for i in $COMMITERS; do
    if [ "$EMAIL_COMMITERS" == "" ]; then
      EMAIL_COMMITERS ="$i"
    else
      EMAIL_COMMITERS ="$EMAIL_COMMITERS $i"
    fi
  done

  branch=$(git rev-parse --symbolic --abbrev-ref $refname)
  if [ "develop" == "$branch" ]
  then
    curl --data-urlencode "token=123abc" --data-urlencode "EMAIL_RECIPIENT=$EMAIL_COMMITERttp://jenkins/view/All/job/CI_BUILD/buildWithParameters
  fi
done

First we iterate over each set of inputs, generally there won’t be more than 1 set of inputs but this makes sure to catch it all. The git hook passes in the old commit, new commit and the ref name to the hook script.

Next it takes the list of committers between bot old and new commits, formatting them to just display the email and grabbing each unique email.

COMMITERS=$(git log $oldrev..$newrev --format="%ae" | sort | uniq)

Jenkins takes the list of emails as a whitespace separated list of strings, so we have to build this list from the previous list of committers.

for i in $COMMITERS; do
  if [ "$EMAIL_COMMITERS" == "" ]; then
    EMAIL_COMMITERS ="$i"
  else
    EMAIL_COMMITERS ="$EMAIL_COMMITERS $i"
  fi
done

Finally, I am using the git-flow methodology so I only really want to trigger a commit if something is pushed to develop, but this can easily be changed to work on any branch.

So it grabs the refname and extracts the branch from it, then using it in the check to ensure jenkins is only called for develop changes.

branch=$(git rev-parse --symbolic --abbrev-ref $refname)
if [ "develop" == "$branch" ]
then
  curl --data-urlencode "token=123abc" --data-urlencode "EMAIL_RECIPIENT=$EMAIL_COMMITERS" http://jenkins/view/All/job/CI_BUILD/buildWithParameters
fi

Depending on how you manage your git repositories this may be all you need to do, however as mentioned earlier I use gitolite which requires a couple of other steps to set up.

First thing is enabling gitolite configuration from within your gitolite administration repository, by default this configuration is stored on the server itself.

This is done by modifying the gitolite.rc file on your gitolite server and adding the following line

LOCAL_CODE => "$rc{GL_ADMIN_BASE}/local"

Whilst in that file, you will also need to enable the repo-specific-hooks feature, more details here

This allows you to manage git hooks within the gitolite.git/local folder, there is a specific folder structure for repo specific hooks. You will need to add your git hook script into gitolite.git/local/hooks/repo-specific

Finally you need to enable the git hook for your repository in your git config file in the gitolite repository, this is done with

repo    foo
   option hook.post-receive = ci-jenkins-build

Next steps will probably involve enabling jenkins blue ocean and using Jenkinsfiles

Share!

Follow me!

Subscribe!