Agile Intelligence: Continuous Integration

The philosophy of continuous integration was first brought in the process of Extreme Programming. Subsequently, it been successfully deployed in several Agile development projects. Continuous Integration is a practice where the development team members integrate their work products frequently. This is more of a learned wisdom over the years than something that got developed in a lab or meeting room. Over the years, the software projects started with big promises and landed into endless integration phases with tons of rambles around. Continuous Integration aims to manage the integration mess in a better way.

Traditional development models used to keep the integration at the end of all development work. Quite often such approach turned into the eleventh hour rush of doing everything, and in many cases redoing the development of some of the modules/functionalities. The project managers were scared to death since after all those fancy progress reporting over months they were handed over with a reality of cost and schedule overrun. No matter how well and detailed the planning was, the integration phase was never smooth tide. Rather, those unfortunate fellows (who could not leave the scene before integration) ended the project with experience which they would remember in gory details for a long period.

To tackle this problem, Continuous Integration started with a hard frequency of one day. In other words, the early proponents of continuous integration aimed to make it a daily integration. Everyday, the programmers would contribute their day’s work to the shared location. At night, or on the next morning, one volunteer would take all those codes together and build those into a single application. After that happens, it would remain open to all for viewing. This would help the individual programmer to see the behavior of his/her codes at the integrated environment. This would also help the programmer to fix some (if not all) of the broken elements into his next day’s work. As a result, the next day’s contribution of the programmer would have some new developed codes as well as some fixes on the previously developed codes. Subsequent build would be expected to have fixes for some of the broken elements, and as the days progress the software application would be getting more refined collectively.

The benefit of such a practice is manifold. It provides a tangible work product for showcasing no matter how messy that could be. The project managers can breathe easy since they now can see what’s going on, as well as a big risk of software integration failures are getting mitigated in steps. The other stakeholders who are not involved with the technical stuff would also get an opportunity to see what’s getting cooked in the kitchen. The programmers also get the benefit to self-correct their own mistakes which could have become a Herculean job if accumulated till the end. It also helps sustaining the morale of the team since a lot of eleventh hour bickering can be avoided.

Continuous Integration comes with its own limitations. Since everything in a software engineering project cannot be related to the programming only, it leaves a good part of the team out of the integration practice. Project Managers practicing agile processes should do a careful planning around them to ensure that their work products too move along harmoniously.

Also, the build process can be long and complex depending upon the size and complexity of the software application getting developed. Daily Integration would require some specialized full time staff allocation to do this job, thus raising the cost of project execution. For such cases, the frequency should be rightly defined. If you are a consultant working for a client, you may be asked to prepare daily builds since the client has read it through some magazines on Agile. If you think the software under consideration is large and complex, it may be a good idea to explain the cost impact of daily builds to the client. Agile does not address the project economics well (we will talk about it later) so it’s your responsibility to bring that under client’s radar for a better cause. For most of the software applications, a twice weekly frequency (Monday morning, and Thursday morning) should be enough. For large and complex applications, take it to a weekly frequency (Friday night, or Monday morning).

Introducing Continuous Integration could be a challenge if the integration process eats up a lot of energy and enthusiasm of the team members. The Agile practitioners use and recommend using the process improvement tools for improving the situation. Maintaining a shareable code repository would be one such step. Nowadays, having source management software like PVCS, or VSS are pretty common, so ensure that it’s in place for your project too. The other important improvement would come through automating the build process. The code compilation, build preparation, and deployment should be automated using scripts and templates to the maximum extent possible; this would provide significant incentives for increasing the frequency of integration during the course of development.