VGTech is a blog where the developers and devops of Norways most visited website share code and tricks of the trade… Read more



Are you brilliant? We're hiring. Read more

Continuous Integration at VG Nett

QA

At VG Nett we use Continuous Integration (CI) for some of our projects. This article will tell you why we do it and how we do it. At the end you will hopefully want to take advantage of CI as well.

If you don’t know what CI is, head over to the article at Wikipedia first, then come back here. A short description (from the article):

In software engineering, continuous integration implements continuous processes of applying quality control — small pieces of effort, applied frequently. Continuous integration aims to improve the quality of software, and to reduce the time taken to deliver it, by replacing the traditional practice of applying quality control after completing all development.

We have several developers working on loads of different projects. We also have some projects that we (developers) regard as more important than others, for instance our internal framework that is used by many different applications. If an error sneaks undetected into the framework it would hurt us more than an error in a single application. Mostly we don’t have time to review all commits as they happen, so we’d rather have a system that informs us as soon as something is acting up.

Enter Jenkins (formerly known as Hudson), an open source CI tool written in Java. Jenkins will let you set up several projects and configure them in different ways. Each project that we add to Jenkins has its own build.xml file that Apache Ant can use to determine which tools to run, and each project is configured to invoke Ant in the build process. Some of the tools we typically include in a build are:

These are just some of the tools that can be used to further enhance the quality of the included projects. All these tools generate log files that can be used to produce charts, graphs, HTML reports and so forth. What will be done with all these log files is up to the projects’ configuration.

Here is a short example of how the build.xml file can look like:

Show code
<?xml version="1.0" encoding="UTF-8"?>
<project name="VGFramework" default="build" basedir=".">
 <target name="clean">
  <!-- Clean up -->
  <delete dir="${basedir}/build" />

  <!-- Create build directories -->
  <mkdir dir="${basedir}/build/api" />
  <mkdir dir="${basedir}/build/coverage" />
  <mkdir dir="${basedir}/build/logs" />
 </target>

 <!-- Run phpunit -->
 <target name="phpunit">
  <exec executable="phpunit" failonerror="on" />
 </target>

 <!-- Generate jdepend.xml and software metrics charts -->
 <target name="pdepend">
  <exec executable="pdepend">
   <arg line="--jdepend-xml=${basedir}/build/logs/jdepend.xml
              ${basedir}/library/VGF" />
  </exec>
 </target>

 <!-- Generate pmd.xml -->
 <target name="phpmd">
  <exec executable="phpmd">
   <arg line="${basedir}/library/VGF
              xml codesize,unusedcode
              --reportfile ${basedir}/build/logs/pmd.xml" />
  </exec>
 </target>

 <!-- Generate phploc.csv -->
 <target name="phploc">
  <exec executable="phploc">
   <arg line="--log-csv ${basedir}/build/logs/phploc.csv
              ${basedir}/library/VGF" />
  </exec>
 </target>

 <!-- Generate checkstyle.xml -->
 <target name="phpcs">
  <exec executable="phpcs">
   <arg line="--report=checkstyle
              --report-file=${basedir}/build/logs/checkstyle.xml
              --standard=VG
              ${basedir}/library/VGF" />
  </exec>
 </target>

 <!-- Generate API documentation -->
 <target name="phpdoc">
  <exec executable="phpdoc">
   <arg line="-d ${basedir}/library/VGF
              -t ${basedir}/build/api" />
  </exec>
 </target>

 <target name="build" depends="clean,phpunit,pdepend,phpmd,phpcs,phploc"/>
</project>

Apache ant will use this file and trigger the commands listed in the file with the given arguments in every build. If some of the commands in the build process fails (for instance a software change causes a unit test to fail) the whole build will be marked as failed.

Jenkins is heavily based on plugins, and there are many different plugins that consumes the reports generated by the tools mentioned above. Some of the plugins we have included in our Jenkins installation are:

Jenkins can also be configured to report build results to different channels; be it IRC or Twitter (amongst others). This can be configured differently for all projects. On our internal IRC server we have different channels for different projects, and the IRC-bot that the IRC-plugin provides notifies project build results to the respective channels.  If a build fails most will know about it instantly (along with information such as the person committing code that broke the build, commit message and so forth) so we can halt the deployment process, fix the bug, then rebuild. We will then continue this until the bug is fixed so we can continue with the deployment process.

Project builds are usually triggered by software changes (commits to the VCS). They can also be triggered manually in the Jenkins web GUI, or for instance by the IRC-bot. On most of our projects we have setup Jenkins to poll Subversion every 5 minutes to see if anything has changed since the last build. It is also possible to configure a post-commit hook in Subversion to trigger a build.

And there you have it. This is in short why and how we use Jenkins for CI. Feel free to leave a comment on things you would have done differently, or other comments regarding CI in general.

Related links:

Senior developer at VG. Coder of code, drinker/brewer of beer and listener of metal/punk/hc. @cogocogo | @BeerNorway | www.beernorway.com


3 comments

  • Eirik

    How do you solve the problems of running PHP scripts as executables on Windows, as part of an Ant build? I'm trying to use Ant to run composer.phar, and I have made a batch script "composer.bat" which in turn calls php "C:\Tools\php\pharlib\composer.phar" %1.
    When I specify , Ant will not run the bat script. If I specify "composer.bat", it works, but then build.xml is probably broken on Unix systems, so it's not cross platform. Do you have any solutions to this problem?


    • Christer Edvartsen

      I don't run Windows at all so I have no experience using ant on Windows, sorry. Perhaps some other readers have some knowledge about this they can share?

  • Eirik

    (By the way, I solved this today by modifying the source code of Apache Ant, in the method where Runtime.exec is called. It catches the IOException, appends ".bat" to the executable name and tries again. Probably not an optimal solution though.)


Leave your comment