My previous post on Test Driven Development described its usefulness in a software development project. In this post, we will try to explore its utility beyond that - more specifically to projects relevant for services, and the service delivery itself. Our context of discussion will be limited to an instance, where a particular service delivery process is being setup, i.e. a project, and subsequently the service delivery. Before getting there, let's recap the principle of Test Driven Development once more: TDD is an approach that emphasizes the development to be done in a way so that the developed work product becomes just fit enough to pass through the test devised for it.
Now, let's think of setting up a process or a set of processes for a specific service delivery. Normally, we try to take a look at the existing set of processes that deliver identical or similar service(s) and then we try to replicate those processes. After replicating the processes, or establishing those with some modifications, we try to produce the work product using that setup. During this trial, we review each process block and verify whether the work products coming out it are matching up to the spec. Once we see the output of the new processes conforming to the specification, we put those outputs to tests. If the tests pass, voila! If not, let's go back to our own drawing board and try to find out the point of defect injections.
Test Driven Development approach would not challenge this method of setting up processes, though would offer a different variant. Instead of trying to match each output of each process block, it would emphasise the need to test the outputs first. In essence, the specification would be superseded by the test cases. The objective of the project manager responsible to setting up such a service delivery process would then focus on obtaining the test cases instead of the specifications.
In reality, this would make the job simpler too. Quite often, specifications go missing, even if they exist they are found to be dated, no similarity with the processes and steps running on ground. Re-writing the updated processes may become a tedious task. Moreover, the people possessing the latest knowledge may not be treating this as a priority work. They may not be willing too. Test cases, on the other hand, are easy to find in its updated form. Even if it is not updated as a documentation, involving the relevant people to perform the tests would ensure that the outputs are tested using recent updates. Moreover, an observer during testing can document all the quality checks and tests performed on the output and subsequently produce an updated test case documentation.
The problem would not end there though. This approach would bring out the test cases for the final work product, not necessarily for the interim. For the interim test cases, separate effort needs to be commissioned, and the present process owners may not be able to help here. So the approach would be to involve all the team members to understand the end work product, irrespective of the processes they are assigned, and to go through the final test cases even if their work product would not be subjected to that, so that they can apply their minds while doing their bit of the processes to strive for passing the final test. Once the test defects are identified, all the team members need to go through that defect through a common session/meeting, and do a causal analysis together. This would ensure that everybody understands the injection points for each of such defects. The collective learning would be harnessed at a later point of time to document the test cases corresponding to all interim work products.
Once the final work product passes through the final test cases, the service delivery process(s) is ready to roll. At this point of time, the team members would continue to follow the same set of steps they had done to pass the test. Simultaneously, the test cases for all the interim work products should be documented. Once that documentation is complete, the focus should shift towards passing through the interim test cases instead of trying to clear the final set of test cases. Passing through all the interim test cases would improve the probability of passing through the final test cases significantly. At the same time, the process specialist must work to establish a good degree of traceability or linkages between the test cases; between final and interim as well as between interim and interim.
As the service delivery progresses, there comes the changes. Broadly, the changes would come in two types: change in the product specification, or the change in the internal processes due to some improvement and optimization. A change in the product specification would call for a change in the final set of test cases. New test cases would get inserted, existing test cases would get deleted or altered. Using the traceability matrix, the impacted interim test cases should be identified and modified. Similarly, for the change in internal processes due to some improvement and optimization would also require change on the test cases relevant for those set of the processes. It would also require to verify and update the linked test cases as appropriate. Once done, the new set of test cases should be used for testing the final and interim work products. This change activities are to be performed by the team members only, so that maximum buy-in can be ensured, and the team members would also get an opportunity to prepare themselves to produce the new work products for passing through the new test cases.
In summary, there is room for reaping benefits by implementing a test driven development approach for a non-IT service work. However, TDD should not be taken as a replacement of classical way of setting up processes. There is no substitute of process design, setting up processes by thorough documentation of steps, determine and optimize the critical path and critical chain, encouraging the team members to perform work through the documented steps, and performing the self review through a checklist. A Test Driven Approach would help the entire service delivery to excel in quality, and to encourage the team to make the processes leaner by identifying the unwanted steps. Finally, even if it is part of Agile practices, it requires tailoring based on where it is to be applied.
Monday, May 17, 2010
Wednesday, May 12, 2010
Agile Intelligence: Leveraging Scrum for non-IT Services
My previous post on Scrum jotted down some of the key elements and thoughts for using this into the software development projects. While looking back and reflecting the same for non-IT services, the basics remain same at large. The simplicity of Scrum makes it extensible to a different area of work keeping its core same, and making necessary changes to it implementation the way new service would require. Again, I must emphasize that the service team or group considering to adopt Scrum should not be asked to change themselves nor their way of working. The implementation of Scrum needs tailoring instead, if so needed.
The core three questions of Scrum would stay same while implementing it for non-IT services as well:
1. What have you accomplished since the last scrum?
2. What are you going to accomplish till the next scrum?
3. What are the issues are you having on your way?
However, the response to these questions will vary here. As it is evident, that we are talking about the service, not really a project. So the spicy details of what's getting cooked should not be expected here. On a normal day, a team member is expected to respond something like this -
1. I have completed 140 transactions, all that were assigned to me.
2. I will complete all assigned transactions, upto 160 transactions in a given day.
3. Nothing to be mentioned.
In a room of 20, this set may be repeated multiple times, by multiple individuals, in a way that would question the relevance of the questions. However, the participation of all is a necessity here, inclusivity is the key to the excellence. So, the same answers should be allowed to get repeated. While the repeatation is underway, the scrum master should train his/her ears to pickup the relatively proactive responses, something like this one -
1. I have completed 140 transactions, all that were assigned to me. I had some free time, about an hour, and spent that in updating the review checklist.
2. I will complete all assigned transactions, upto 160 transactions in a given day, and will use my free time to complete the checklist updates.
3. The checklist updates are infrequent these days, can it be regularized?
Once such responses come out, the role of the scrum master needs to go beyond the point of asking three questions. The scrum master needs to encourage such activities as stated in #1, needs to push the work target stated in #2, if appropriate, and take cues of innovation ideas as mentioned in #3. A little more risky bet would be to ask for innovation ideas at the end of the scrum. All through, the scrum master should be patient and careful about not doling out any unrealistic plan, specifically, for #2 and #3. In a service oriented structure, the number of transactions per day cannot grow everyday, it would saturate to a level and any further upward push without planning and enablement would impair the overall service delivery structure. Similarly, all innovation ideas should not be converted to an improvement, it would not be feasible either. Each ideas need to be evaluated with a structured review, and upon qualification, should be implemented through a plan.
Finally, how much time should be devoted for scrum? For a mature process, it should not be more than 15 minutes per day. It's important to have it everyday with varying duration. For a stable situation, it should not be more than 15 minutes, but during any change implementation, more time needs to be allocated for sufficient discussion.
The core three questions of Scrum would stay same while implementing it for non-IT services as well:
1. What have you accomplished since the last scrum?
2. What are you going to accomplish till the next scrum?
3. What are the issues are you having on your way?
However, the response to these questions will vary here. As it is evident, that we are talking about the service, not really a project. So the spicy details of what's getting cooked should not be expected here. On a normal day, a team member is expected to respond something like this -
1. I have completed 140 transactions, all that were assigned to me.
2. I will complete all assigned transactions, upto 160 transactions in a given day.
3. Nothing to be mentioned.
In a room of 20, this set may be repeated multiple times, by multiple individuals, in a way that would question the relevance of the questions. However, the participation of all is a necessity here, inclusivity is the key to the excellence. So, the same answers should be allowed to get repeated. While the repeatation is underway, the scrum master should train his/her ears to pickup the relatively proactive responses, something like this one -
1. I have completed 140 transactions, all that were assigned to me. I had some free time, about an hour, and spent that in updating the review checklist.
2. I will complete all assigned transactions, upto 160 transactions in a given day, and will use my free time to complete the checklist updates.
3. The checklist updates are infrequent these days, can it be regularized?
Once such responses come out, the role of the scrum master needs to go beyond the point of asking three questions. The scrum master needs to encourage such activities as stated in #1, needs to push the work target stated in #2, if appropriate, and take cues of innovation ideas as mentioned in #3. A little more risky bet would be to ask for innovation ideas at the end of the scrum. All through, the scrum master should be patient and careful about not doling out any unrealistic plan, specifically, for #2 and #3. In a service oriented structure, the number of transactions per day cannot grow everyday, it would saturate to a level and any further upward push without planning and enablement would impair the overall service delivery structure. Similarly, all innovation ideas should not be converted to an improvement, it would not be feasible either. Each ideas need to be evaluated with a structured review, and upon qualification, should be implemented through a plan.
Finally, how much time should be devoted for scrum? For a mature process, it should not be more than 15 minutes per day. It's important to have it everyday with varying duration. For a stable situation, it should not be more than 15 minutes, but during any change implementation, more time needs to be allocated for sufficient discussion.
Tuesday, May 11, 2010
Agile Intelligence: Leveraging Agile Methods in non-IT Services
Traditionally, Agile methods were developed and applied for software development only. The foundation processes for a successful project following Agile methods were largely focusing on the interactions among programmers and managers grown out of programming environments. Nevertheless, the promise of changing specification during the course of SDLC (production period) and flexibility to demonstrate work-in-progress had its own influence, even within the community of stakeholders who were not generally related to software development work. Also, over the period of time, the IT managers started assuming the role of the BPO/KPO service delivery managers as their companies started taking such work offshore. As a result, many of the IT practices has started finding their relevance within IT Enabled Services as well.
The concept of agility was not new in the non-IT space though. In fact, Agile Manufacturing is a not-so-new terminology used to express a manufacturing process or setup that can react to the fast changing product specification in the fast changing competition landscape. Agile Manufacturing setup also allows the product marketers, product designers, and production personnel to share a common goal and a common set of information all through the product life cycle to efficiently handle the upstream and downstream activities.
In the area of IT Enabled Service delivery, Agile Programming methods are certainly not going to be applicable as-is. However, there are certain good practices, that can be adopted and applied to realize benefits. The practices that would be useful quickly and without tailoring are Scrum, Test Driven Development, Continuous Integration, and Backlog Management. As a caveat, I must submit, none of these are usable by its original names, nor by its original terms as dictated by the Agile proponents. Each of these practices are to be called with a relevant name and to be brought under relevant set of activities & steps. Moreover, it must be understood that these set of practitioners are not going to be programmers, techies, nerds, or of that sort. These practitioners will be come from a diverse set of backgrounds: by academic background, by experience, and by mindset. So, each of these four practices will require comparable level of revisions and modifications to fit the respective needs, their implementation risks are to be assessed with due care, and only after that should be brought in practice. We will look through each of these practices keeping this context in mind, in the following posts.
The concept of agility was not new in the non-IT space though. In fact, Agile Manufacturing is a not-so-new terminology used to express a manufacturing process or setup that can react to the fast changing product specification in the fast changing competition landscape. Agile Manufacturing setup also allows the product marketers, product designers, and production personnel to share a common goal and a common set of information all through the product life cycle to efficiently handle the upstream and downstream activities.
In the area of IT Enabled Service delivery, Agile Programming methods are certainly not going to be applicable as-is. However, there are certain good practices, that can be adopted and applied to realize benefits. The practices that would be useful quickly and without tailoring are Scrum, Test Driven Development, Continuous Integration, and Backlog Management. As a caveat, I must submit, none of these are usable by its original names, nor by its original terms as dictated by the Agile proponents. Each of these practices are to be called with a relevant name and to be brought under relevant set of activities & steps. Moreover, it must be understood that these set of practitioners are not going to be programmers, techies, nerds, or of that sort. These practitioners will be come from a diverse set of backgrounds: by academic background, by experience, and by mindset. So, each of these four practices will require comparable level of revisions and modifications to fit the respective needs, their implementation risks are to be assessed with due care, and only after that should be brought in practice. We will look through each of these practices keeping this context in mind, in the following posts.
Sunday, November 22, 2009
Agile Intelligence: On "Enterprise Scrum"
In his book Ken Schwaber described how an enterprise can adopt and implement scrum across all IT work. Size-wise, the 176 pages long book is divided into two parts: the general text, and appendixes. The appendixes describe some of the key concepts and nomenclature of the scrum, while the general text covers the key topics. The general text also illustrates some of the cases, where scrum implemented, and the kind of results / benefits accrued.
The good part of this book is it talks about some good theories of the classical project management and behaviors. As example, it talks about four stages for team performance: form, storm, norm, and perform. This one is nothing to do with Agile, it was a separate concept altogether. The good part of the book is that it adopted it.
The not so good part is some of the key messages. A few of those are highlighted below:
Scrum cannot be tailored for an organization, Organization needs to change itself: This illustrates the inflexibility of the case studies. If an organization needs to change itself for adopting scrum, while scrum stays unchanged, it practically makes the organization inflexible after scrum adoption. An inflexible organization is certainly going to take a downward dive on every aspect of the business.
The team needs to be self managed, self organized: This practically puts an end to the possibility of inducting junior team members. In every team there needs to be somebody who should be taking the lead role or coordinator role. The person needs to be somebody who can be looked up, and needs to be a role model for the junior developers while they are growing up. The book stays silent on this point.
Fire thy sales guy attitude: In a case study, the book found the solution in firing the sales guy who made some aggressive commitments to the client, and pushed developers harder for completion. A different case study of aggressive but successful sales guy would bring more relevance into it.
Developers should not be pressed: Throughout the book, the author emphasized not to press developers for producing more. It talks about giving the liberty of committing their own release dates, but not much have been mentioned on accountability, and aligning the dates with business reality. The book scares the readers on compromising on quality while speeding up the development, but surprisingly silent on measuring quality of the work product.
Derogating Offshore Work: The book does not support offshoring of work. Instead if asks for improving productivity at onshore. True, that productivity needs to be improved, but it’s yet to be proven that offshore work is less productive. In today’s world, when majority of the software development work is being done from offshore locations, it’s a bit surprising to see the author has missed out the success stories and data.
Overall, the book is not a recommended buy. But if you are an avid reader, go read it with open mind.
The original review comments was posted at Amazon.com.
The good part of this book is it talks about some good theories of the classical project management and behaviors. As example, it talks about four stages for team performance: form, storm, norm, and perform. This one is nothing to do with Agile, it was a separate concept altogether. The good part of the book is that it adopted it.
The not so good part is some of the key messages. A few of those are highlighted below:
Scrum cannot be tailored for an organization, Organization needs to change itself: This illustrates the inflexibility of the case studies. If an organization needs to change itself for adopting scrum, while scrum stays unchanged, it practically makes the organization inflexible after scrum adoption. An inflexible organization is certainly going to take a downward dive on every aspect of the business.
The team needs to be self managed, self organized: This practically puts an end to the possibility of inducting junior team members. In every team there needs to be somebody who should be taking the lead role or coordinator role. The person needs to be somebody who can be looked up, and needs to be a role model for the junior developers while they are growing up. The book stays silent on this point.
Fire thy sales guy attitude: In a case study, the book found the solution in firing the sales guy who made some aggressive commitments to the client, and pushed developers harder for completion. A different case study of aggressive but successful sales guy would bring more relevance into it.
Developers should not be pressed: Throughout the book, the author emphasized not to press developers for producing more. It talks about giving the liberty of committing their own release dates, but not much have been mentioned on accountability, and aligning the dates with business reality. The book scares the readers on compromising on quality while speeding up the development, but surprisingly silent on measuring quality of the work product.
Derogating Offshore Work: The book does not support offshoring of work. Instead if asks for improving productivity at onshore. True, that productivity needs to be improved, but it’s yet to be proven that offshore work is less productive. In today’s world, when majority of the software development work is being done from offshore locations, it’s a bit surprising to see the author has missed out the success stories and data.
Overall, the book is not a recommended buy. But if you are an avid reader, go read it with open mind.
The original review comments was posted at Amazon.com.
Saturday, July 11, 2009
Agile Intelligence: The Story of a Miserable Project
In one of my recent posts, I tried to tell a story of a successful project following Agile methods. Now it's time to flip the coin: let me try to tell a story of a miserable project. This was not a failed project though. The project eventually completed, with near 100% schedule variance and comparable effort variance, but with acceptable quality and conformance to all norms of closure. To give credit to myself (or to Agile), I don't have any story of failed project to tell.
It was a Sunday afternoon, when I was trying my eyes to believe what are being painted in front of me. It was a cozy meeting room, a room where I prefer to talk while standing, a room cozy enough to prevent me pacing bi-ways out of nervousness. With me, there sitting my boss, and two other developers. My boss summoned me to be there, he got a salvo or two from a senior executive of the client. The developers were trying to put forward the project situation, as-is. The first version was a little less than hunky-dory, and I learned that the team conducts a Scrum daily. An attempt to align the client's frustration with the project situation created the second version of the project situation, which was not only far away from being hunky-dory, but a lot scary as well. A critical phase in the software development life cycle got skipped: Testing. The developers were unfazed. The project called for 2 developers, so 2 developers started working. Developers never realized that they need to handover the codes on time to the testers. No body followed up with them either. As a result, the project directly moved to UAT after construction, that too when the construction was not complete. On the day one of UAT, client realized there was nothing for him to see, verify, and accept. He got furious.
Despite of having daily interactions with the client and developers, the project still went into red. It went into red, because there was a lack of appreciation for the SDLC phases. People loved Agile, tried to do some Agile stuff, without realizing that the fundamentals didn't change. Agile methodology does not cover SDLC, in certain aspects, it negates it. That prompts the team practising Agile, and bend for programming to miss out the big picture: the need of testing before releasing the product.
The story did not end here. Follow on meetings with the developers revealed none of them know how much to code and implement. There was no requirements document either. So developers were doing coding based on their understanding of the functionality which were not only inadequate, but also carried the risk of being misinterpreted. As a result, on the day two of UAT, the problem in hand was not just incompleteness of codes, but a total unawareness of the size of the remaining work.
The rest of the story was a familiar tale, an approach that have been followed from the stone age of software engineering. Some of the reputed fire fighters were deployed on the scene. They started working 20 hours a day everyday: coding, understanding requirements, testing, communicating with the client like a PM, and generating intense push-pull force between the buyer and supplier. After 2 weeks of a grueling endeavour, the project started becoming back on track. Client started seeing the work products getting delivered, client started seeing less errors in the delivered work products. The fire fighters started disbanding in search of another fire.
In short, just Agile does not work always. It requires a good understanding of the SDLC. In fact, all successful Agile practitioners today have got their foundations built through the classical SDLC process. Inherently, they practice the good stuff out of classical SDLC. An attempt to bring methods in the madness has always turned out to be effective in Agile projects, provided the teams carry an openness to accept good things from outside.
It was a Sunday afternoon, when I was trying my eyes to believe what are being painted in front of me. It was a cozy meeting room, a room where I prefer to talk while standing, a room cozy enough to prevent me pacing bi-ways out of nervousness. With me, there sitting my boss, and two other developers. My boss summoned me to be there, he got a salvo or two from a senior executive of the client. The developers were trying to put forward the project situation, as-is. The first version was a little less than hunky-dory, and I learned that the team conducts a Scrum daily. An attempt to align the client's frustration with the project situation created the second version of the project situation, which was not only far away from being hunky-dory, but a lot scary as well. A critical phase in the software development life cycle got skipped: Testing. The developers were unfazed. The project called for 2 developers, so 2 developers started working. Developers never realized that they need to handover the codes on time to the testers. No body followed up with them either. As a result, the project directly moved to UAT after construction, that too when the construction was not complete. On the day one of UAT, client realized there was nothing for him to see, verify, and accept. He got furious.
Despite of having daily interactions with the client and developers, the project still went into red. It went into red, because there was a lack of appreciation for the SDLC phases. People loved Agile, tried to do some Agile stuff, without realizing that the fundamentals didn't change. Agile methodology does not cover SDLC, in certain aspects, it negates it. That prompts the team practising Agile, and bend for programming to miss out the big picture: the need of testing before releasing the product.
The story did not end here. Follow on meetings with the developers revealed none of them know how much to code and implement. There was no requirements document either. So developers were doing coding based on their understanding of the functionality which were not only inadequate, but also carried the risk of being misinterpreted. As a result, on the day two of UAT, the problem in hand was not just incompleteness of codes, but a total unawareness of the size of the remaining work.
The rest of the story was a familiar tale, an approach that have been followed from the stone age of software engineering. Some of the reputed fire fighters were deployed on the scene. They started working 20 hours a day everyday: coding, understanding requirements, testing, communicating with the client like a PM, and generating intense push-pull force between the buyer and supplier. After 2 weeks of a grueling endeavour, the project started becoming back on track. Client started seeing the work products getting delivered, client started seeing less errors in the delivered work products. The fire fighters started disbanding in search of another fire.
In short, just Agile does not work always. It requires a good understanding of the SDLC. In fact, all successful Agile practitioners today have got their foundations built through the classical SDLC process. Inherently, they practice the good stuff out of classical SDLC. An attempt to bring methods in the madness has always turned out to be effective in Agile projects, provided the teams carry an openness to accept good things from outside.
Tuesday, June 30, 2009
Agile Intelligence: Staffing Your Team
Mobilizing the right team for executing a project successfully using Agile methods is important. More importantly, Agile method needs experienced coding hands, because of its emphasis on liberating the programmer from manager's command. A project following Agile methods does not have any project manager. The scrum master's role is limited to asking three questions only, that too far open ended. In such a situation, the developer is entrusted to compose the work performed, the work to be performed (in the guise of will be accomplished till next scrum), and estimation of effort. It's true that only the individual can specify how much time he/she would take to complete a specific task, because only the individual knows how much he/she takes to complete similar tasks. In other words, how much time he/she had taken in similar type of tasks in the past. In other words, how much experience he/she can leverage this time. More the experience, better would be the estimate, better would be the productivity.
The proponents of Agile methods counters that such a developer profile is something that everybody asks for. It's not uncommon. While such counter statements are true, the fact that Agile Programming does not have any room for junior coders is true as well. So much that the project following Agile Programming with a bunch of junior coders (and no project manager) is susceptible to failure more often than the traditional one.
The good news is, there is a way out. Tailoring the Agile methods a little bit would make a tremendous impact on achieving right mix of the team, and would help in shaping the path for individual development for the junior coders. Here is how.
Pair programming allows two developers to develop application while working on a single computer. In a project team that follows Agile methods, each pair can be comprised of a senior coder and a junior coder. At the beginning, the senior coder will act as the driver to get the work started, while the junior coder will be the observer. By junior coder, I am indicating to a person who has been trained on the necessary skills to perform the job of programming, like programming language, but not necessarily has any experience of doing similar project before. For the first week, or may be for the first two weeks, the senior coder will be the driver. Subsequently, they will start interchanging the roles. While the junior coder works like a driver, the role of the senor coder would be more like a navigator instead of an observer. This would ensure that the progress is happening in the right direction, with right quality, and adequate learning for the junior coder. This would also address the question that critiques often raise: Productivity. With two senior coders programming in pair, the productivity cannot be more than the aggregate of what they would have been accomplished in isolation. If fact, it would be far less. With a mix of senior and junior coders in the pair, the probability of accomplishing the comparable output would be higher. This is because the junior coder is always coding in presence of experienced reviewers. When it comes to the tasks like effort estimation, it would be always the senior programmer who would do the estimates. This would ensure the accuracy of the estimates stay within the tolerable limits. Also, the senior programmer needs to factor the productivity level of the junior programmer.
There is a small drawback though. The codes produced by the experienced coder will not go through proper review. For this, the team may adopt a process of peer review across pairs. Peer review, in general, is part of the traditional way of software engineering, but will become very effective in quality assurance in such a situation.
To summarize, it's possible to induct junior profiles in the project team while following Agile methods. Pair programming would be the way to achieve that. Introducing peer review across pairs would ensure that the quality of the work products are not compromised. Such a tailored process would pave the growth path for the junior developers.
The proponents of Agile methods counters that such a developer profile is something that everybody asks for. It's not uncommon. While such counter statements are true, the fact that Agile Programming does not have any room for junior coders is true as well. So much that the project following Agile Programming with a bunch of junior coders (and no project manager) is susceptible to failure more often than the traditional one.
The good news is, there is a way out. Tailoring the Agile methods a little bit would make a tremendous impact on achieving right mix of the team, and would help in shaping the path for individual development for the junior coders. Here is how.
Pair programming allows two developers to develop application while working on a single computer. In a project team that follows Agile methods, each pair can be comprised of a senior coder and a junior coder. At the beginning, the senior coder will act as the driver to get the work started, while the junior coder will be the observer. By junior coder, I am indicating to a person who has been trained on the necessary skills to perform the job of programming, like programming language, but not necessarily has any experience of doing similar project before. For the first week, or may be for the first two weeks, the senior coder will be the driver. Subsequently, they will start interchanging the roles. While the junior coder works like a driver, the role of the senor coder would be more like a navigator instead of an observer. This would ensure that the progress is happening in the right direction, with right quality, and adequate learning for the junior coder. This would also address the question that critiques often raise: Productivity. With two senior coders programming in pair, the productivity cannot be more than the aggregate of what they would have been accomplished in isolation. If fact, it would be far less. With a mix of senior and junior coders in the pair, the probability of accomplishing the comparable output would be higher. This is because the junior coder is always coding in presence of experienced reviewers. When it comes to the tasks like effort estimation, it would be always the senior programmer who would do the estimates. This would ensure the accuracy of the estimates stay within the tolerable limits. Also, the senior programmer needs to factor the productivity level of the junior programmer.
There is a small drawback though. The codes produced by the experienced coder will not go through proper review. For this, the team may adopt a process of peer review across pairs. Peer review, in general, is part of the traditional way of software engineering, but will become very effective in quality assurance in such a situation.
To summarize, it's possible to induct junior profiles in the project team while following Agile methods. Pair programming would be the way to achieve that. Introducing peer review across pairs would ensure that the quality of the work products are not compromised. Such a tailored process would pave the growth path for the junior developers.
Friday, May 15, 2009
Agile Intelligence: The Story of a Successful Project
The preamble would say that all elements of this story are fictitious and if they resemble to any real life element, that would just be a coincidence. Having said that, the story starts with a project conducive enough to practice Agile. For me, it was not the first Agile project, but my interest to run it like Agile was on the higher side. I did not do any assessment to figure out whether Agile would work here or not (like I mentioned in my previous post), but my guts were telling me that this is the project for Agile methods.
Here it starts. I visited my client (the product owner) with one of my team mates to understand what the client wants. The client told us a story. For sake of anonymity, I would keep that story out of my posts. The story triggered a brainstorming session only to result a data entry screen and two reports. After the 100 minutes of the meeting, I added two standard features into it: login, and administration. At the end of the meeting (in about 2 hours in total), we agreed on a tentative timeline (needs to be shown in 2 weeks), project commercials (some thousand dollars to be billed), and came out to write our meeting notes.
My team mate started development, mobilized servers for hosting, and at the end of the first week, we made our demo of the data entry screen. No reports were ready, but nobody complained either. The audience of the demo comprised of some of the users including my client (4 in total). Some good suggestions came in. We noted down all, on spot classified the items that would take longer time and effort to implement. In support of us, the client shoot those down immediately by flagging them as features for future. At the end of the week #2, we released the application for UAT.
The UAT was planned to be an innovative one. Instead of running a script for testing, 40 users were identified and were asked to use the system. We gave them a hour long training on the application, along with a form to log their feedback. The client has mobilized a manager to collect all feedback forms, collate those into one, analyze those to various categories and identified those with the priority to ship. This compilation indeed became a collectors' edition due to its richness of the user feedback. Some of those are truly awesome, would not be possible to anticipate or detect by any developer or tester. Many of those reveal the need of having a user's manual, it was unbelievable that we had delivered a product without user's manual just because we were following Agile.
After a week-long UAT, another week was spent on implementing the prioritized feedback. Like the first one, we did the upfront cost estimate for the changes or new features (a few thousands dollars again), and picked up all programming bugs without hesitation. We followed a principle of fixed fee charging, so all programming bugs were serviced free of cost. Subsequently, the application was laid open for all 300 users.
The story could have ended here. But the project continues. Once it goes live, the client brought out a few more things to add in the application. A few more data entry screens, a few more reports to go. Now the application is live and kicking, articulation of the new requirements were far more easy. No wonder, my next visit to the client to gather requirements ran for good three and half hours. The requirements list was long, naturally it would take longer to implement. The client was agreeable to that too. This was the time I contested his thoughts. My opinion was that the project delivered good results in 2 weeks of development (and unit testing), 1 week of UAT, and 1 week of bug fixing / changes. We should stick to this plan. The fact that the requirements has got longer this time would mean that we have to slice it into two. Or three. Or four. Whatever it takes to cut down to the 2 week schedule. We had one developer at the start, we will add another one this time. That will enable the team to take twice the amount of work. But the schedule should not change. It would not elongate, it would not be shortened either.
My client agreed. He didn't have to worry though since I was talking of frequent deliveries this time (most of the time we get our clients into the reverse kind of arguments). So we started the same schedule for quite a iterations subsequently. At this point, like a classroom teacher, I should start to map the Agile terms with the various acts in this project. But I am refraining from doing so. I assume the readers would be able to do it themselves. Those who are not sure, please read all my posts on Agile Intelligence from start.
Here it starts. I visited my client (the product owner) with one of my team mates to understand what the client wants. The client told us a story. For sake of anonymity, I would keep that story out of my posts. The story triggered a brainstorming session only to result a data entry screen and two reports. After the 100 minutes of the meeting, I added two standard features into it: login, and administration. At the end of the meeting (in about 2 hours in total), we agreed on a tentative timeline (needs to be shown in 2 weeks), project commercials (some thousand dollars to be billed), and came out to write our meeting notes.
My team mate started development, mobilized servers for hosting, and at the end of the first week, we made our demo of the data entry screen. No reports were ready, but nobody complained either. The audience of the demo comprised of some of the users including my client (4 in total). Some good suggestions came in. We noted down all, on spot classified the items that would take longer time and effort to implement. In support of us, the client shoot those down immediately by flagging them as features for future. At the end of the week #2, we released the application for UAT.
The UAT was planned to be an innovative one. Instead of running a script for testing, 40 users were identified and were asked to use the system. We gave them a hour long training on the application, along with a form to log their feedback. The client has mobilized a manager to collect all feedback forms, collate those into one, analyze those to various categories and identified those with the priority to ship. This compilation indeed became a collectors' edition due to its richness of the user feedback. Some of those are truly awesome, would not be possible to anticipate or detect by any developer or tester. Many of those reveal the need of having a user's manual, it was unbelievable that we had delivered a product without user's manual just because we were following Agile.
After a week-long UAT, another week was spent on implementing the prioritized feedback. Like the first one, we did the upfront cost estimate for the changes or new features (a few thousands dollars again), and picked up all programming bugs without hesitation. We followed a principle of fixed fee charging, so all programming bugs were serviced free of cost. Subsequently, the application was laid open for all 300 users.
The story could have ended here. But the project continues. Once it goes live, the client brought out a few more things to add in the application. A few more data entry screens, a few more reports to go. Now the application is live and kicking, articulation of the new requirements were far more easy. No wonder, my next visit to the client to gather requirements ran for good three and half hours. The requirements list was long, naturally it would take longer to implement. The client was agreeable to that too. This was the time I contested his thoughts. My opinion was that the project delivered good results in 2 weeks of development (and unit testing), 1 week of UAT, and 1 week of bug fixing / changes. We should stick to this plan. The fact that the requirements has got longer this time would mean that we have to slice it into two. Or three. Or four. Whatever it takes to cut down to the 2 week schedule. We had one developer at the start, we will add another one this time. That will enable the team to take twice the amount of work. But the schedule should not change. It would not elongate, it would not be shortened either.
My client agreed. He didn't have to worry though since I was talking of frequent deliveries this time (most of the time we get our clients into the reverse kind of arguments). So we started the same schedule for quite a iterations subsequently. At this point, like a classroom teacher, I should start to map the Agile terms with the various acts in this project. But I am refraining from doing so. I assume the readers would be able to do it themselves. Those who are not sure, please read all my posts on Agile Intelligence from start.
Tuesday, April 07, 2009
Agile Intelligence: Is Your Client Aware
So far, we have been advocating the possibility and need of executing software development projects following Agile methods and on a fixed fee agreement. The benefits of Agile programming are well understood, as well as the need of adopting other good practices. Running project in fixed fee is also a consideration to provide greater comfort and de-risk client's financial commitments. Now the question comes, is the client really aware of Agile programming?
Like any other successful projects, Agile methods too require some good deal of client commitments for successful completion. Quite often, the system implementers jump into the Agile bandwagon with a promise of making client's life simpler (read lazier). As a result, when the client contacts are asked to provide their inputs frequently, at times everyday, all hell breaks loose. Similarly, clients are promised with frequent delivery by virtue of continuous integration (read on demand delivery). Sellers, in quest of selling the work, commit all the good things that Agile can bring on the table without uttering a single word on what it takes for a client to achieve that. Once the job is sold, the programmers land on ground, and starts realizing amount of damage been done.
The big ticket question is: how to educate the client on Agile methods? Seasoned consultants know the art of educating their clients with new tools, techniques, and practices; and those still hold good here. Client education cannot be performed though a sales pitch, cannot be performed if it doesn't bring any value to the client. The system implementer needs to first assess whether Agile method is really going to bring any incremental benefits to the client for the project under consideration. If not, which is likely to be a regular instance, let's not try to push for Agile.
Secondly, the involvement need of the client contacts are also to be evaluated. Agile method requires a lot of time and effort commitment from the client contacts, all those hours are to be justified. The point to remember here, hours from the client are not free. Those too get added to the project cost. The system implementer may not need to bear that cost directly, but it's a direct cost to the client for that particular project. So, if a good part of the project progress can be achieved by minimal client involvement, let's not plan for Agile methods there. Instead, taking a monolithic project plan with built-in Agile sub-phases may work wonders here.
Also, the involvement in multiple tiers is important. The traditional project management processes include a need to define the escalation path. In the era of Agile, that is still relevant. When it comes to managing the stakeholders with a varied degree of influences, it may not make sense to bring all of them into a single scrum, that too everyday. Instead, organize multiple scrum-of-scrums. Participants of each such scrum-of-scrums are to be carefully chosen. Also, the format of the scrum-of-scrums may require tailoring. By definition, each participant of the scrum-of-scrums are expected to be represent their own scrum teams. Here, in some cases, that will not hold true. Some of the participants will not be part of any scrum team, rather they will bring their organizational insights to enrich the other participants, increase their own awareness on the project progress, and in certain instances become more aware of the practices of Agile programming.
At this point, the Agilists (and its proponents) will strongly disagree with me, I swear. Scrum cannot be tailored, tailor your organization to scrum; is their dictum. I would humbly submit that I disagree. Organizations cannot change to fit itself into something prepared externally. Organizations can only adopt, articulate, and assimilate that. During this course of adoption organization tailors the external ingredient to fit with it, and over time of adoption it changes itself too, primarily to respond to the market forces. So, tailoring of the format is inevitable, if required, to be done for each client, for each project. During the course of tailoring, involvement of the client is essential. That's the learning point for a client highest intensity.
To summarize, emphasize the benefits of Agile programming with its cost while selling a job to the client. Perform proper assessment to see whether Agile is really needed, if not, don't hesitate to discard it. If you adopt Agile programming for a client project, ensure that right kind of client awareness and commitments are there.
Like any other successful projects, Agile methods too require some good deal of client commitments for successful completion. Quite often, the system implementers jump into the Agile bandwagon with a promise of making client's life simpler (read lazier). As a result, when the client contacts are asked to provide their inputs frequently, at times everyday, all hell breaks loose. Similarly, clients are promised with frequent delivery by virtue of continuous integration (read on demand delivery). Sellers, in quest of selling the work, commit all the good things that Agile can bring on the table without uttering a single word on what it takes for a client to achieve that. Once the job is sold, the programmers land on ground, and starts realizing amount of damage been done.
The big ticket question is: how to educate the client on Agile methods? Seasoned consultants know the art of educating their clients with new tools, techniques, and practices; and those still hold good here. Client education cannot be performed though a sales pitch, cannot be performed if it doesn't bring any value to the client. The system implementer needs to first assess whether Agile method is really going to bring any incremental benefits to the client for the project under consideration. If not, which is likely to be a regular instance, let's not try to push for Agile.
Secondly, the involvement need of the client contacts are also to be evaluated. Agile method requires a lot of time and effort commitment from the client contacts, all those hours are to be justified. The point to remember here, hours from the client are not free. Those too get added to the project cost. The system implementer may not need to bear that cost directly, but it's a direct cost to the client for that particular project. So, if a good part of the project progress can be achieved by minimal client involvement, let's not plan for Agile methods there. Instead, taking a monolithic project plan with built-in Agile sub-phases may work wonders here.
Also, the involvement in multiple tiers is important. The traditional project management processes include a need to define the escalation path. In the era of Agile, that is still relevant. When it comes to managing the stakeholders with a varied degree of influences, it may not make sense to bring all of them into a single scrum, that too everyday. Instead, organize multiple scrum-of-scrums. Participants of each such scrum-of-scrums are to be carefully chosen. Also, the format of the scrum-of-scrums may require tailoring. By definition, each participant of the scrum-of-scrums are expected to be represent their own scrum teams. Here, in some cases, that will not hold true. Some of the participants will not be part of any scrum team, rather they will bring their organizational insights to enrich the other participants, increase their own awareness on the project progress, and in certain instances become more aware of the practices of Agile programming.
At this point, the Agilists (and its proponents) will strongly disagree with me, I swear. Scrum cannot be tailored, tailor your organization to scrum; is their dictum. I would humbly submit that I disagree. Organizations cannot change to fit itself into something prepared externally. Organizations can only adopt, articulate, and assimilate that. During this course of adoption organization tailors the external ingredient to fit with it, and over time of adoption it changes itself too, primarily to respond to the market forces. So, tailoring of the format is inevitable, if required, to be done for each client, for each project. During the course of tailoring, involvement of the client is essential. That's the learning point for a client highest intensity.
To summarize, emphasize the benefits of Agile programming with its cost while selling a job to the client. Perform proper assessment to see whether Agile is really needed, if not, don't hesitate to discard it. If you adopt Agile programming for a client project, ensure that right kind of client awareness and commitments are there.
Saturday, February 21, 2009
Agile Intelligence: Doing Project in Fixed Fee (contd.)
For a IT service provider, doing projects in fixed fee would mean the hours to be spent is capped. Each IT service company is having is its own benchmark hourly rate to compute fee from the estimate hours. That make the job of estimation very important. But Agile brings its own challenges: it doesn't have any end to requirements specification, not does it have any project/product sign-off criteria. Once somebody is entrusted to run a project on fixed fee, putting such boundaries are essential to keep the project afloat, financially.
Once more, we need to bring the PM back here. The PM will now evaluate the areas where boundaries are to be defined, and will do the necessary fencing to guard those. Let's talk about a bit on those common areas in general -
Scope Management: The scope of the work can be defined with various level of details. It can be a one-pager for a 5000 hours worth of work, or it can be a one-pager for a 50000 hours of work. Defining the scope of work crisply and succinctly is very important while agreeing to execute a project in a fixed fee. While drafting the scope, the local cultures are to be kept in mind too. Some regions on this planet expects the exclusions to be written down separately, others are fine with inclusions only. In some parts, it requires to be submitted in local language. All such cases are to kept in mind to ensure minimum (if not zero) room for ambiguity, second guessing, and arguments.
Requirements Management: Gathering requirements and documenting those for the software developers play a key role in keeping the costs at tab. While requirements are gathered, it's important to understand that the requirements documents will be read by the business users too. Accordingly, the clarity for each functionality / feature are to be documented. Objectivity of the requirements are also to be very clear: Is it being done for a specific sprint? Is it being done before submission of the fee quote? Is the fees are already determined? For the first and third questions, the stakeholders should be alerted if the requirements are blowing out of proportion.
The second situation is rather win-win combination. In such situations, the requirements analysis phase is taken out of the project and are performed separately. Most of the economic buyers are comfortable in paying by the hours for this phase, since the uncertainty is less. Even if it is to be capped at a fee, the duration of the work can be capped as well. Once the requirements document gets ready and accepted, the document(s) are used as the basis for quoting the implementation fee. The benefit of such situation is that it allows the IT service provider to quote an educated fee. It also allows the provider to invest, benchmark, and improve its estimation model to a higher level of maturity and certainty.
During the acceptance phase, new requirements and modified requirements are inevitable. The PM must drive those hard enough to get documented as individual CRs so that those can be priced separately. One popular approach is to bundle a set of logical CRs into one release, quote a fee (again fixed), and deliver.
Schedule Management: Once the project is underway, it's important to keep it on schedule. If the schedule is not on track, there is every possibility of leakage of hours through non-productive work and idle hours. Although, idle hours are not generally counted as the project hours, but it's still the cost to the company. Every PM should be aware of keeping it under control.
There would still be situations where the schedule cannot be kept under control. The effective way to keep the project financially viable is to cut down the team size. A slim team is capable to running long by burning less hours yet accomplishing all the expected tasks. The big assumption here is that the scope is under control.
To summarize, the project can be done in fixed fee and following some of the best practices of Agile method, as long as the team is open to borrow practices and processes from the traditionalists for ring fencing their own risks.
Once more, we need to bring the PM back here. The PM will now evaluate the areas where boundaries are to be defined, and will do the necessary fencing to guard those. Let's talk about a bit on those common areas in general -
Scope Management: The scope of the work can be defined with various level of details. It can be a one-pager for a 5000 hours worth of work, or it can be a one-pager for a 50000 hours of work. Defining the scope of work crisply and succinctly is very important while agreeing to execute a project in a fixed fee. While drafting the scope, the local cultures are to be kept in mind too. Some regions on this planet expects the exclusions to be written down separately, others are fine with inclusions only. In some parts, it requires to be submitted in local language. All such cases are to kept in mind to ensure minimum (if not zero) room for ambiguity, second guessing, and arguments.
Requirements Management: Gathering requirements and documenting those for the software developers play a key role in keeping the costs at tab. While requirements are gathered, it's important to understand that the requirements documents will be read by the business users too. Accordingly, the clarity for each functionality / feature are to be documented. Objectivity of the requirements are also to be very clear: Is it being done for a specific sprint? Is it being done before submission of the fee quote? Is the fees are already determined? For the first and third questions, the stakeholders should be alerted if the requirements are blowing out of proportion.
The second situation is rather win-win combination. In such situations, the requirements analysis phase is taken out of the project and are performed separately. Most of the economic buyers are comfortable in paying by the hours for this phase, since the uncertainty is less. Even if it is to be capped at a fee, the duration of the work can be capped as well. Once the requirements document gets ready and accepted, the document(s) are used as the basis for quoting the implementation fee. The benefit of such situation is that it allows the IT service provider to quote an educated fee. It also allows the provider to invest, benchmark, and improve its estimation model to a higher level of maturity and certainty.
During the acceptance phase, new requirements and modified requirements are inevitable. The PM must drive those hard enough to get documented as individual CRs so that those can be priced separately. One popular approach is to bundle a set of logical CRs into one release, quote a fee (again fixed), and deliver.
Schedule Management: Once the project is underway, it's important to keep it on schedule. If the schedule is not on track, there is every possibility of leakage of hours through non-productive work and idle hours. Although, idle hours are not generally counted as the project hours, but it's still the cost to the company. Every PM should be aware of keeping it under control.
There would still be situations where the schedule cannot be kept under control. The effective way to keep the project financially viable is to cut down the team size. A slim team is capable to running long by burning less hours yet accomplishing all the expected tasks. The big assumption here is that the scope is under control.
To summarize, the project can be done in fixed fee and following some of the best practices of Agile method, as long as the team is open to borrow practices and processes from the traditionalists for ring fencing their own risks.
Saturday, January 31, 2009
Agile Intelligence: Doing Project in Fixed Fee
This is in fact a tricky spot: for both Agile practitioners, and the economic buyers. I will pick up the case for Agile practitioners a bit later, let's see where do the economic buyers stand with price-to-value game. So far, these group of people were sold on the Agile methods with the fact that they would get to see what's getting cooked in the kitchen. In turn, there were no certainty on the cost involved in cooking that dish. Increasingly, the economic buyers realized that they don't really need to be in the kitchen to see how the cooking happens. It's important to get the dish right; it should be palatable, and it should be hygienic. In other words, the software application delivered for use should be effective in meeting its objective of existence, and it should be trustworthy for carrying out the business. The economic buyer, may be the CFO of the company, doesn't need to spend time in seeing that a singleton class is getting developed, tested, and deployed for other classes to invoke it.
Once that's realized, the next big question is how to ensure that the software production cost is contained with respect to the value it will eventually deliver. The easy path would be to determine the value (or IRR), and calculate back the maximum amount of money to be spent. Once the maximum amount of money to spent is confirmed, that becomes the budget for that particular application development.
So far so good. The problem is that there is still no certainty that the application will get developed within that budget. After running sprints after sprints, the budget may get burnt completely only to deliver a bunch of classes and database objects with no realizable business benefits. The poor economic buyer, being part of the entire drama of cooking codes, would now be left with the pieces of marvels, but without any clue what to do with that. Agilists delivered, though.
On the contrary, the help is actually around. The IT service suppliers are available to design, develop, test, and deploy codes with demonstrable credentials in hand. The competition among them brings the benefits of bidding fixed fee quote, thus insulating the economic buyer from any financial risk. This is a proposition too hard to resist. But can Agile methods be fitted with this fixed fee structure?
Agilists, on the other hand, had a different problem in hand. What used to be a sideline process so far, has become a mainstream. Being a mainstream software engineering process, it's difficult to avoid the monetary question: how to ensure that the business benefits are realized at right cost? To address this concern, the initial thought was to follow Agile methods with a fixed budget. The product owner should know how much money to spend. Based on that, the product owner would determine the number of sprints to be performed, with the number of team members. Subsequently, the product owner should also make a broad classification of the features and functionality to various sprints. With this, our Agile project has become a fixed sprint project. A seemingly uncompromisable deviation, but okay!!
The fun part of market forces is it not only negates inefficiency, it also makes an open mockery of it. When the competition demanded to quote fixed fee for a software development project, even if it is supposed to follow Agile, it really pushed Agilists very hard. Too hard to stay sane. This time the question of ethics came out of the pocket. Apparently, asking developers to develop a program in fixed fee is an unethical practice. With this hypothesis, the argument was structured that software development in a fixed fee fashion is unethical practice. Agilists, being always on the right side of ethics (no matter how much they abused the traditionalists before), cannot accept this. So, Agilists will do T&M work.
Without going to the details on ethical practices and all that, let's try to look into the issue with our feet on ground -
1. Agile is going to stay, some of its practices are not available anywhere
2. Fixed Fee work is inevitable, a buyer is not actually buying programmers' time, buyer is buying a piece of software
3. How to fit the best of Agile practices into a fixed fee project would be the puzzle to solve.
My next post will try to address it.
Once that's realized, the next big question is how to ensure that the software production cost is contained with respect to the value it will eventually deliver. The easy path would be to determine the value (or IRR), and calculate back the maximum amount of money to be spent. Once the maximum amount of money to spent is confirmed, that becomes the budget for that particular application development.
So far so good. The problem is that there is still no certainty that the application will get developed within that budget. After running sprints after sprints, the budget may get burnt completely only to deliver a bunch of classes and database objects with no realizable business benefits. The poor economic buyer, being part of the entire drama of cooking codes, would now be left with the pieces of marvels, but without any clue what to do with that. Agilists delivered, though.
On the contrary, the help is actually around. The IT service suppliers are available to design, develop, test, and deploy codes with demonstrable credentials in hand. The competition among them brings the benefits of bidding fixed fee quote, thus insulating the economic buyer from any financial risk. This is a proposition too hard to resist. But can Agile methods be fitted with this fixed fee structure?
Agilists, on the other hand, had a different problem in hand. What used to be a sideline process so far, has become a mainstream. Being a mainstream software engineering process, it's difficult to avoid the monetary question: how to ensure that the business benefits are realized at right cost? To address this concern, the initial thought was to follow Agile methods with a fixed budget. The product owner should know how much money to spend. Based on that, the product owner would determine the number of sprints to be performed, with the number of team members. Subsequently, the product owner should also make a broad classification of the features and functionality to various sprints. With this, our Agile project has become a fixed sprint project. A seemingly uncompromisable deviation, but okay!!
The fun part of market forces is it not only negates inefficiency, it also makes an open mockery of it. When the competition demanded to quote fixed fee for a software development project, even if it is supposed to follow Agile, it really pushed Agilists very hard. Too hard to stay sane. This time the question of ethics came out of the pocket. Apparently, asking developers to develop a program in fixed fee is an unethical practice. With this hypothesis, the argument was structured that software development in a fixed fee fashion is unethical practice. Agilists, being always on the right side of ethics (no matter how much they abused the traditionalists before), cannot accept this. So, Agilists will do T&M work.
Without going to the details on ethical practices and all that, let's try to look into the issue with our feet on ground -
1. Agile is going to stay, some of its practices are not available anywhere
2. Fixed Fee work is inevitable, a buyer is not actually buying programmers' time, buyer is buying a piece of software
3. How to fit the best of Agile practices into a fixed fee project would be the puzzle to solve.
My next post will try to address it.
Sunday, December 28, 2008
Agile Intelligence: Schedule Variance
Whenever I sit on a project review and ask the question on quantitative management, this is the first quantitative parameter that I hear: Schedule Variance. Across the projects, IT and non-IT included, this is a metric of concern. Or, may be this is the easiest metric to measure, only to discover that the results are dismal at large.
Riding on the old English anecdote that Time is Money, the importance of schedule variance is indeed paramount. However, when it comes to following the Agile method, it becomes a bit different. May be a bit interesting too. Let me explain why: Agile methods by definition, cannot have any schedule variance. All deliveries are contained within one sprint or other. Sprints do have a fixed end date. As a result, the schedule variance of all sprints are zero.
While the sprints end on time, the deliverables don't. The unfortunate fact is covered up by simply pulling the deliverable off the sprint, and pushing it to the next sprint. So, if the measurement is done sprint-by-sprint, the schedule variance is zero. If the measurement is done output-by-output (or by deliverable), the result is unlikely to be zero. Sad but so real!
At the risk of antagonizing the proponents of Agile methods, the metrication on this area needs a pragmatic approach. Sprints are fine, but their end dates are curved on stone. But the buyer get most benefit when the work products are delivered on time. The product owner may take the responsibility of listing down the macro level deliverables to define such packages. The product owner may also need to do a thorough homework in determining the realistic and beneficial date for each of these packages to become usable. For the consultants working for their clients, it would be useful to have an open meeting with the client to agree on these dates. Once the dates are agreed, measure the schedule variance of each of these deliverables, and aggregate those to the project level.
Many a times, while speaking to the buyer or sponsor about Agile methods, it was found to be drawing a good degree of enthusiasm simply because there was a promise of showing the work products early enough while in the making. The same enthusiasm get converted to frustration when the buyer gets to see a lot things frequently, but none of those make any sense to him/her. A delivery centric view covering only the deliverables that matter most would be good to sustain the enthusiasm, as well as would save some time for both developers and buyer. The same feeling can get echoed through the measurement technique described above for schedule variance. We cannot simply ignore this metric for sake of the definitions within Agile Programming method.
Riding on the old English anecdote that Time is Money, the importance of schedule variance is indeed paramount. However, when it comes to following the Agile method, it becomes a bit different. May be a bit interesting too. Let me explain why: Agile methods by definition, cannot have any schedule variance. All deliveries are contained within one sprint or other. Sprints do have a fixed end date. As a result, the schedule variance of all sprints are zero.
While the sprints end on time, the deliverables don't. The unfortunate fact is covered up by simply pulling the deliverable off the sprint, and pushing it to the next sprint. So, if the measurement is done sprint-by-sprint, the schedule variance is zero. If the measurement is done output-by-output (or by deliverable), the result is unlikely to be zero. Sad but so real!
At the risk of antagonizing the proponents of Agile methods, the metrication on this area needs a pragmatic approach. Sprints are fine, but their end dates are curved on stone. But the buyer get most benefit when the work products are delivered on time. The product owner may take the responsibility of listing down the macro level deliverables to define such packages. The product owner may also need to do a thorough homework in determining the realistic and beneficial date for each of these packages to become usable. For the consultants working for their clients, it would be useful to have an open meeting with the client to agree on these dates. Once the dates are agreed, measure the schedule variance of each of these deliverables, and aggregate those to the project level.
Many a times, while speaking to the buyer or sponsor about Agile methods, it was found to be drawing a good degree of enthusiasm simply because there was a promise of showing the work products early enough while in the making. The same enthusiasm get converted to frustration when the buyer gets to see a lot things frequently, but none of those make any sense to him/her. A delivery centric view covering only the deliverables that matter most would be good to sustain the enthusiasm, as well as would save some time for both developers and buyer. The same feeling can get echoed through the measurement technique described above for schedule variance. We cannot simply ignore this metric for sake of the definitions within Agile Programming method.
Tuesday, November 25, 2008
Agile Intelligence: Quantitative Management
The purpose of Quantitative Project Management is to manage projects by a set of defined number driven parameters to achieve the stated project objectives. Apart from having familiarity of various quantitative methods, such activity also demands planning, reporting, and analysis. Unfortunately, Agile methods don't support activities like planning and reporting in a formal way. It doesn't have any role for Project Manager. That leaves the entire space of quantitative management open for interpretation.
So, how should an Agilist address the need of Quantitative Management? The earlier proponents openly discarded the existence of such need, but no longer. As Agile methods are becoming mainstream, such questions are real, demand for demonstrating project performance through numbers are phenomenal. The only quantitative reporting available with Agile methods are burn charts. The chart shows the effort burned over a period of time, and how the budget gets consumed. The chart does not reflect anything with respect to the work product quality. The chart does not reflect anything with respect to the meeting project objectives. The chart does not reflect anything pertaining to the analysis and resolution. As a result, the seasoned Agilist is bound to look into the classical quantitative management techniques, devise his/her own plan and execute it.
This brings back to the same situation similar to my previous post: wearing a project manager's hat while still within the team. Somebody like the product owner should prepare the metrication plan, collect data during scrums/otherwise, report data through a frequency driven reporting, and perform necessary analysis. If the data collection becomes part of the scrum, then the scrum format needs to be tailored. Instead of sticking to the 3 question format, a fourth question may get introduced by asking 'Tell me your numbers'. Or, a standard form can be used to collect data in writing.
The good news is, many large projects today use some process support IT platform, even if the project follows Agile methods. Tools like Team Foundation Server, etc. becomes quite useful in storing and updating the sprint backlogs. The same tool can also provide quite a few meaningful metrics based on how much of the backlog tracking is being done through it. At a minimum, the three basic project performance criteria can be derived quite easily: schedule, cost, quality. There is a minor twist though. The definitions of these parameters within the scope of Agile Programming are different to that of what the classical project management techniques taught us. My next posts will start exploring those areas.
So, how should an Agilist address the need of Quantitative Management? The earlier proponents openly discarded the existence of such need, but no longer. As Agile methods are becoming mainstream, such questions are real, demand for demonstrating project performance through numbers are phenomenal. The only quantitative reporting available with Agile methods are burn charts. The chart shows the effort burned over a period of time, and how the budget gets consumed. The chart does not reflect anything with respect to the work product quality. The chart does not reflect anything with respect to the meeting project objectives. The chart does not reflect anything pertaining to the analysis and resolution. As a result, the seasoned Agilist is bound to look into the classical quantitative management techniques, devise his/her own plan and execute it.
This brings back to the same situation similar to my previous post: wearing a project manager's hat while still within the team. Somebody like the product owner should prepare the metrication plan, collect data during scrums/otherwise, report data through a frequency driven reporting, and perform necessary analysis. If the data collection becomes part of the scrum, then the scrum format needs to be tailored. Instead of sticking to the 3 question format, a fourth question may get introduced by asking 'Tell me your numbers'. Or, a standard form can be used to collect data in writing.
The good news is, many large projects today use some process support IT platform, even if the project follows Agile methods. Tools like Team Foundation Server, etc. becomes quite useful in storing and updating the sprint backlogs. The same tool can also provide quite a few meaningful metrics based on how much of the backlog tracking is being done through it. At a minimum, the three basic project performance criteria can be derived quite easily: schedule, cost, quality. There is a minor twist though. The definitions of these parameters within the scope of Agile Programming are different to that of what the classical project management techniques taught us. My next posts will start exploring those areas.
Monday, October 27, 2008
Agile Intelligence: Risk Management
Traditional software engineering has Risk Management as an activity by itself. It is recognized as a process area or knowledge area by the most project management standards. Interestingly, it was never been part of Agile Programming. The reason was easy: Agile Programming was conceptualized and devised by the programmers. Programmers don't like activities like risk management and many others. This worked well to the point where the team was largely formed by two (three max) seasoned developers, and they used to churn out some of the great codes every month. The problem started when large projects started following agile methods. The methods and practices started getting questions that were never thought out, nor addressed. Risk Management was one of those.
So, doesn't a project following Agile methods need any Risk Management? Certainly, it does. The project needs to take help from the established project management standards. There is a problem though: Risk Management is a PM's job. Agile projects don't keep any PM. Who would manage the risks in that case? The question has been so far addressed by the volunteers. The Product Owner or the Scrum Master put on the hat of the PM to manage the risks. That demanded the risk manager to come out of the project chores time to time and take a thirty-thousand feet view to identify and assess the risk.
The broad level steps are same for risk management in Agile: Risk Identification, Risk Assessment, Risk Planning, and Risk Monitoring. These steps are implemented through scrums, the Scrum Master facilitates the session. However, these scrums don't follow the traditional 3 question format. These are more akin to the brain storming interactive sessions to ensure that all risks are identified and their impacts are rightly analyzed and understood. These sessions also demand the Scrum Master and Product Owner to provide a good perspective on the project risks. Being part of the team and weathering the daily chores, the risk of missing the forest for trees are very high in such a situation. Many project teams try to address this by bringing in an external expert / observer into these sessions and take their inputs. Such attempts are undoubtedly good but deviates from the Agile principles to a good extent.
To summarize, IT projects following Agile methods do need risk management. To accomplish this, the project team must leverage one of the many established formal project management standards and guidelines. The project team must assimilate the role of the PM within one of them to ensure the risks are formally managed. External support, if available would do good to the entire project process of risk management.
So, doesn't a project following Agile methods need any Risk Management? Certainly, it does. The project needs to take help from the established project management standards. There is a problem though: Risk Management is a PM's job. Agile projects don't keep any PM. Who would manage the risks in that case? The question has been so far addressed by the volunteers. The Product Owner or the Scrum Master put on the hat of the PM to manage the risks. That demanded the risk manager to come out of the project chores time to time and take a thirty-thousand feet view to identify and assess the risk.
The broad level steps are same for risk management in Agile: Risk Identification, Risk Assessment, Risk Planning, and Risk Monitoring. These steps are implemented through scrums, the Scrum Master facilitates the session. However, these scrums don't follow the traditional 3 question format. These are more akin to the brain storming interactive sessions to ensure that all risks are identified and their impacts are rightly analyzed and understood. These sessions also demand the Scrum Master and Product Owner to provide a good perspective on the project risks. Being part of the team and weathering the daily chores, the risk of missing the forest for trees are very high in such a situation. Many project teams try to address this by bringing in an external expert / observer into these sessions and take their inputs. Such attempts are undoubtedly good but deviates from the Agile principles to a good extent.
To summarize, IT projects following Agile methods do need risk management. To accomplish this, the project team must leverage one of the many established formal project management standards and guidelines. The project team must assimilate the role of the PM within one of them to ensure the risks are formally managed. External support, if available would do good to the entire project process of risk management.
Saturday, September 20, 2008
Agile Intelligence: Good To Great!
This is no plagiarism. My intention was not to pickup some perfect lines from here and there, but to rationalize a thought process. A thinking that keeps me asking to myself: Are organizations becoming great by adopting Agile Programming?
There is no denying that in an environment of continuous change demand for more agility from the business support functions are at historic high. At the same time, dependence on IT to run business and deliver goods & services are increasing like never before. Combining these two, the flexibility of the IT systems and processes to respond to ever-changing demand becomes the contributor to the business success. Certainly, Agile Programming can help in removing the inflexibility of the IT systems and processes by adopting some of its proven practices.
However, the agility may bring some compromises as well. While Agile methods don't explicitly denounce any structured approach and the need for documentation, there is a subtle ignorance towards this. Programmers love Agile Programming because quite often it relieves them from doing the mundane documentation, or following some lengthy processes. This is a dangerous trap. Sooner, the organization would either become captive to those rogue programmers due to their know-it-all attitude, or it would succumb to the fact that the IT systems are no longer maintainable. Yours truly had a recent experience of the latter situation, where a great software application became miserable and painful just because it was no longer maintainable, and nobody knew the inbuilt functionality well enough to develop a new application. So, Agile Programming is not necessarily a path towards greatness.
Many companies have adopted the process standards like CMMI, and Six Sigma for their IT organization to ensure proper adherence to the processes and fostering an environment to improve. CMMI, if effectively implemented, can be helpful in having a great structure in place for IT work. Six Sigma can help in removing all the wasteful processes and steps thus making the CMMI structure quite efficient. None of these two have any conflict with Agile programming, and can very well embrace some of the proven best practices like pair programming and scrums. Such a combination may bring some amount of greatness within the IT organization.
Unfortunately, there are not enough takers of such a theory, let alone practice it. Of late, the proponents of Agile have started denouncing that there are conflicts between Agile and CMMI. They are also openly encouraging the need for documentation within an Agile process. All these are good sign. It also shows a desperation to become mainstream practice for software development. But to make an IT organization great, only Agile may not be enough.
There is no denying that in an environment of continuous change demand for more agility from the business support functions are at historic high. At the same time, dependence on IT to run business and deliver goods & services are increasing like never before. Combining these two, the flexibility of the IT systems and processes to respond to ever-changing demand becomes the contributor to the business success. Certainly, Agile Programming can help in removing the inflexibility of the IT systems and processes by adopting some of its proven practices.
However, the agility may bring some compromises as well. While Agile methods don't explicitly denounce any structured approach and the need for documentation, there is a subtle ignorance towards this. Programmers love Agile Programming because quite often it relieves them from doing the mundane documentation, or following some lengthy processes. This is a dangerous trap. Sooner, the organization would either become captive to those rogue programmers due to their know-it-all attitude, or it would succumb to the fact that the IT systems are no longer maintainable. Yours truly had a recent experience of the latter situation, where a great software application became miserable and painful just because it was no longer maintainable, and nobody knew the inbuilt functionality well enough to develop a new application. So, Agile Programming is not necessarily a path towards greatness.
Many companies have adopted the process standards like CMMI, and Six Sigma for their IT organization to ensure proper adherence to the processes and fostering an environment to improve. CMMI, if effectively implemented, can be helpful in having a great structure in place for IT work. Six Sigma can help in removing all the wasteful processes and steps thus making the CMMI structure quite efficient. None of these two have any conflict with Agile programming, and can very well embrace some of the proven best practices like pair programming and scrums. Such a combination may bring some amount of greatness within the IT organization.
Unfortunately, there are not enough takers of such a theory, let alone practice it. Of late, the proponents of Agile have started denouncing that there are conflicts between Agile and CMMI. They are also openly encouraging the need for documentation within an Agile process. All these are good sign. It also shows a desperation to become mainstream practice for software development. But to make an IT organization great, only Agile may not be enough.
Saturday, August 23, 2008
Agile Intelligence: Peer Review
There is no peer review activity in Agile Programming. It's embedded into it. When the proponents of the Agile methods started practising it in a bigger way for serious IT projects, there was serious need to doing reviews. Traditionally, the programmers following Agile were all experienced hands and hence there was a little need for doing such reviews. As the projects following Agile started becoming large and complex, two problems started bugging the Scrum Masters and Product Owners a lot:
- There is not enough experienced hands to code any more, since the team size is bigger.
- Complex codes require comprehensive look through the second pair of eyes.
Pair programming principles came very handy this time. In this setup of work, no single developer is assigned a programming task. It's always assigned to a group of two programmers. The technique instructs this pair of programmers to sit in front of a single workstation (computer) and perform the programming corresponding to the assigned task. One programmer is supposed to do the typing of the codes: to be called as the driver. The other programmer is supposed to continue reading the codes being typed and review the correctness of the codes: to be called as the observer. Some books also term the latter person as the navigator. As a result of this setup, the codes are getting reviewed on the point of production.
This technique has been adopted in many IT projects and being implemented successfully, the simple reason being its effectiveness. A traditional software development method mandates review of the program after production, but is mute on the technique. Most often, a developer writes a piece of code, compiles and re-compiles it till it becomes error free (not bug free), and then leaves the program unit for review for an elderly programmer. By the time the reviewer steps in, the program unit may reach to a size of many thousand lines, with the implementation of several requirements. The reviewer on hot seat now has a daunting task of reading through the requirements, and then reading through the programs to make some review comments. There are a many good ways to make such reviews effective, but at large the task is no more a fun affair. This is where pair programming technique clicks. It makes the programming a fun. The programmers interacts continuously between themselves about the codes and implementation and through instantaneous iterations, make the codes better. No more pain and effort for documenting review comments, no more closing the review comments through a loop of comprehensive work flow. Sample this: How many of the traditional code review comments don't start with a statement on the code indentation or comments? The answer is probably "not many". While it's very important to produce properly organized readable codes while programming, quite often the reviewer gets lost within such show-off stuff instead of getting deeper into the question whether any alternative coding would make the implementation better. Pair programming technique allows such review comments to be made right on spot by the observer so that the driver can make the necessary changes right there.
Criticism is not few though. One big question is always put up is about productivity. Deploying two programmers to program on a single workstation perceivably reduces the productivity to half. However, the equation is not that linear. The proponents of pair programming submits two points in this respect:
- Zero Review Effort: Review being embedded into the technique, no separate review activity needs to be planned and managed. This would save some good degree of time and effort. Each organization or project team maintains its own review effort benchmark, and it varies widely. So quantification may be difficult, but everybody agrees that it does not fully substitute a second programmer's time. Here comes the second point, a bit upright and direct.
- Productive Hours: Some of the proponents of this technique openly challenged to walk by the cubicles of the programmers who are nested within their own self for programming. Their revelation was that about half of the population were doing something but coding: checking emails, preparing reports, web browsing, and even blogging like yours truly. According to them, this time would not go waste if two of them staring at the workstation for programming. In a pair programming process, the pair typically gets exhausted after 6 hours of work, simply because the intensity of the focus they need to put into work. On the contrary, according to the proponents, the single programmer programming is not worth more than 4 productive hours sans review.
I admit that this is a questionable argument. Not all organizations foster similar culture, and hence the outcome would largely vary. However, given the quality of the output through pair programming, it may be useful at many times. If not for the entire set of programming tasks, but certainly for the complex and critical sets.
Pair programming also opens up an avenue that the proponents largely ignored: possibility to deploy junior resources. Agile methods don't have enough room for junior programmers, pair programming allows the formation of senior-junior tag team to perform programming and learning to do better coding on the way.
Saturday, July 12, 2008
Agile Intelligence: Project Kick Off
In traditional software development project, project kick-off is an event. Project Kick-off indicates a start of the project. It ceremonially authorizes a person as Project Manager to run the project, and recognizes the sponsors, business or otherwise. Before a kick-off happens, the project charter is ready, the stated objectives are clearly understood, and approved budget is there. The project manager knows where to start from and where to reach, with a given duration and money. Project kick-off also recognizes the key contributors and influencers, and lays down the ground rules to be followed.
In Agile Programming, project kick-off remains an event, with the exception of the roles involved. Agile Programming did not start with the project mind-set but with a programmer’s perspective. By and large it remained a programmers’ work till recently, the mainstay organizations started seriously looking into its benefits, and the skeptics started raising difficult questions. In an attempt to put a project wrapper on Agile, the project kick-off eventually started happening.
Agile Programming does not recognize the project manager. So the Product Owner and the Scrum Master jointly slip into this role. They convene the kick-off meeting. Agile does not recognize the hierarchy among programmers. So it calls all programmers into the kick-off meeting. Agile Programming does not care of project funding or sponsor’s constraints. So project charter, objectives, and budgetary considerations are not put into the kick-off agenda.
Still, there are widely recognized benefits of a project kick-off meeting in Agile Programming. The IT Services organizations have done some good amount of thinking to fit an effective kick-off meeting for projects following Agile Programming. The best practices include a soda-popcorn speech by the Product Owner, where the Product Owner is expected to outline his/her own vision about the end product. The Product Owner also hints out how he/she anticipates all the features of the products will be built out. After the Product Owner finishes his/her job, the Scrum Master steps in for the next show. The Scrum Master puts the Products Owners wish list into multiple sprints and the high level distribution of features among the sprints. As you may understand, these two persons need to do a good homework together; otherwise the meeting may turn out to a disaster through each one contradicting the other.
After that, the kick off meeting is expected to end with some questions and fixing the time for Scrums. Sprint 1 starts from the next day.
Till now, Agile does not have any standardized kick-off method for projects. Over the period of time, it is expected to mature to a certain extent. As example, the traditional projects foster multi-stage kick-off meetings based on the phase driven involvement of various roles. In many projects programmers are not called in till the requirements analysis progress to a reasonable extent, so that all the early questions of the programmers can be well answered during the programmers’ kick-off meeting. Many organizations involve external IT service providers for getting a part or whole of the software developed. Their involvement does call for a specific kick-off meeting, however, that may not be the very first kick-off meeting. Traditional project methods allow addressing such situations by sub-classifying the projects within project, whereas such a concept is yet to evolve with the Agile Programming perspective.
If you are the person assuming the role of the Scrum Master (most obvious role for an erstwhile PM), or a Product Owner; my suggestion would be to plan for an effective kick-off meeting. You may need to support this meeting with conference call and live meeting bridges since the programmers will be spread practically everywhere in the world. Lastly, don’t hesitate to borrow the project kick-off methods and techniques laid out in the traditional project management books. You will need those to make it a success.
In Agile Programming, project kick-off remains an event, with the exception of the roles involved. Agile Programming did not start with the project mind-set but with a programmer’s perspective. By and large it remained a programmers’ work till recently, the mainstay organizations started seriously looking into its benefits, and the skeptics started raising difficult questions. In an attempt to put a project wrapper on Agile, the project kick-off eventually started happening.
Agile Programming does not recognize the project manager. So the Product Owner and the Scrum Master jointly slip into this role. They convene the kick-off meeting. Agile does not recognize the hierarchy among programmers. So it calls all programmers into the kick-off meeting. Agile Programming does not care of project funding or sponsor’s constraints. So project charter, objectives, and budgetary considerations are not put into the kick-off agenda.
Still, there are widely recognized benefits of a project kick-off meeting in Agile Programming. The IT Services organizations have done some good amount of thinking to fit an effective kick-off meeting for projects following Agile Programming. The best practices include a soda-popcorn speech by the Product Owner, where the Product Owner is expected to outline his/her own vision about the end product. The Product Owner also hints out how he/she anticipates all the features of the products will be built out. After the Product Owner finishes his/her job, the Scrum Master steps in for the next show. The Scrum Master puts the Products Owners wish list into multiple sprints and the high level distribution of features among the sprints. As you may understand, these two persons need to do a good homework together; otherwise the meeting may turn out to a disaster through each one contradicting the other.
After that, the kick off meeting is expected to end with some questions and fixing the time for Scrums. Sprint 1 starts from the next day.
Till now, Agile does not have any standardized kick-off method for projects. Over the period of time, it is expected to mature to a certain extent. As example, the traditional projects foster multi-stage kick-off meetings based on the phase driven involvement of various roles. In many projects programmers are not called in till the requirements analysis progress to a reasonable extent, so that all the early questions of the programmers can be well answered during the programmers’ kick-off meeting. Many organizations involve external IT service providers for getting a part or whole of the software developed. Their involvement does call for a specific kick-off meeting, however, that may not be the very first kick-off meeting. Traditional project methods allow addressing such situations by sub-classifying the projects within project, whereas such a concept is yet to evolve with the Agile Programming perspective.
If you are the person assuming the role of the Scrum Master (most obvious role for an erstwhile PM), or a Product Owner; my suggestion would be to plan for an effective kick-off meeting. You may need to support this meeting with conference call and live meeting bridges since the programmers will be spread practically everywhere in the world. Lastly, don’t hesitate to borrow the project kick-off methods and techniques laid out in the traditional project management books. You will need those to make it a success.
Thursday, June 05, 2008
Agile Intelligence: Test Driven Development
Test Driven Development (TDD) is a widely popular practice within the Agile practitioners. As the name suggests, this approach puts extra emphasis on passing the tests. I will try to draw up the popular activity block diagram a little later, before that, let's try to understand what this approach intends to achieve.
By and large, the developers are used to do coding by looking at the software requirements specification, aka spec. Developers, or the project teams that are not aware of the mature engineering processes (software or elsewhere), tend to write lines after lines of codes by looking at the spec without realizing the purpose. They tend to forget that the codes produced by them are also intended to run at some point of time. So, they code, and they code, only to realize at the fag end that all those beautifully thought out lines are of no use, because those did not get compiled in first place. Even if those were lucky enough to get compiled, those didn't run. For those luckiest fellows, even if it ran, it did not run as desired. The bottom line is, all those effort had been meaningless. For these developers, TDD brings in a gentle reminder in its name itself: Guys, don't just code; do some testing as well.
Test harness and test bed preparation concepts have been around for a while, primarily to boost the developer's mind and encourage them to do testing. At a latter stage, the professional testers took these two terms to a more sophisticated level by incorporating theories and mature practices. Nevertheless, if we still consider the older practices, all it talked about was unit testing of the codes after completing the code writing of the smallest possible unit (say, a method within a class). While the development is on, the same test on the same unit is to be repeated many times since the new codes are getting developed and interlinked with the existing ones. Instead of putting the developers into the rigor of manual repeats of the same tests, they were encouraged to write test codes so that the testing could be done by the programs. Historically, these were the first attempts in asking the developers to eat their own cooked food.
TDD formalizes the process from a different perspective. TDD assumes that the developer is not required to develop codes that would be perfect in all counts. Rather, it initiates the idea of developing codes that would pass the fitness tests employed, doesn't necessarily would pass with flying scores. To put this into implementation, it says the developer to develop just enough codes to pass the test cases. The diktat puts the developer to write the test cases first, then do the development, and do the development in such a way that it passes through the test cases. Sometimes, the developer doesn't write the test cases still, a tester writes the test cases for him/her. The developer then uses those test cases to conduct his testing and remain compliant to TDD.
By and large, the developers are used to do coding by looking at the software requirements specification, aka spec. Developers, or the project teams that are not aware of the mature engineering processes (software or elsewhere), tend to write lines after lines of codes by looking at the spec without realizing the purpose. They tend to forget that the codes produced by them are also intended to run at some point of time. So, they code, and they code, only to realize at the fag end that all those beautifully thought out lines are of no use, because those did not get compiled in first place. Even if those were lucky enough to get compiled, those didn't run. For those luckiest fellows, even if it ran, it did not run as desired. The bottom line is, all those effort had been meaningless. For these developers, TDD brings in a gentle reminder in its name itself: Guys, don't just code; do some testing as well.
Test harness and test bed preparation concepts have been around for a while, primarily to boost the developer's mind and encourage them to do testing. At a latter stage, the professional testers took these two terms to a more sophisticated level by incorporating theories and mature practices. Nevertheless, if we still consider the older practices, all it talked about was unit testing of the codes after completing the code writing of the smallest possible unit (say, a method within a class). While the development is on, the same test on the same unit is to be repeated many times since the new codes are getting developed and interlinked with the existing ones. Instead of putting the developers into the rigor of manual repeats of the same tests, they were encouraged to write test codes so that the testing could be done by the programs. Historically, these were the first attempts in asking the developers to eat their own cooked food.
TDD formalizes the process from a different perspective. TDD assumes that the developer is not required to develop codes that would be perfect in all counts. Rather, it initiates the idea of developing codes that would pass the fitness tests employed, doesn't necessarily would pass with flying scores. To put this into implementation, it says the developer to develop just enough codes to pass the test cases. The diktat puts the developer to write the test cases first, then do the development, and do the development in such a way that it passes through the test cases. Sometimes, the developer doesn't write the test cases still, a tester writes the test cases for him/her. The developer then uses those test cases to conduct his testing and remain compliant to TDD.
I am not too sure if the image above is readable enough, so you may need to click on it too see the big picture. This is the so-famous diagram depicting a classic TDD. The process addresses the requirement preparation, followed by test cases preparation for customer and developer. By customer test cases, we can safely assume to be UAT cases or business cases. From left to right it starts from requirements, test cases, and the last block is for development. The backward direction indicates successful completion of each of these activities, i.e. after the code getting developed passes through the developer test cases successfully, and so on.
I have already talked about the benefits of the unit testing before code release, and the diagram shows how TDD implements that. There is a significant risk too. The success of the codes depend on the adequacy of the test cases and its coverage of the requirements, both explicit and implicit. With this, the developer gets tempted to pass the test cases instead of looking into the overall objective in mind. For a business consultant, this may be far detrimental than it sounds since the consultant is also expected to keep the respective business objectives in mind. Fortunately, the lack of coverage gets detected when the customer sits in for testing. The flip side is that the customer would need to sit in for testing same piece several times, and this would dilute the overall value proposition of your institutional ability to deliver a solution for business. The only emphasis I would like to make here is not to stretch the TDD to the extent where it actually doesn't; and at the same time build this practice within your developer self. Testing your own code before release would always remain a good practice.
Tuesday, May 13, 2008
Agile Intelligence: Continuous Integration
The philosophy of continuous integration was first brought in the process of Extreme Programming. Subsequently, it been successfully deployed in several Agile development projects. Continuous Integration is a practice where the development team members integrate their work products frequently. This is more of a learned wisdom over the years than something that got developed in a lab or meeting room. Over the years, the software projects started with big promises and landed into endless integration phases with tons of rambles around. Continuous Integration aims to manage the integration mess in a better way.
Traditional development models used to keep the integration at the end of all development work. Quite often such approach turned into the eleventh hour rush of doing everything, and in many cases redoing the development of some of the modules/functionalities. The project managers were scared to death since after all those fancy progress reporting over months they were handed over with a reality of cost and schedule overrun. No matter how well and detailed the planning was, the integration phase was never smooth tide. Rather, those unfortunate fellows (who could not leave the scene before integration) ended the project with experience which they would remember in gory details for a long period.
To tackle this problem, Continuous Integration started with a hard frequency of one day. In other words, the early proponents of continuous integration aimed to make it a daily integration. Everyday, the programmers would contribute their day’s work to the shared location. At night, or on the next morning, one volunteer would take all those codes together and build those into a single application. After that happens, it would remain open to all for viewing. This would help the individual programmer to see the behavior of his/her codes at the integrated environment. This would also help the programmer to fix some (if not all) of the broken elements into his next day’s work. As a result, the next day’s contribution of the programmer would have some new developed codes as well as some fixes on the previously developed codes. Subsequent build would be expected to have fixes for some of the broken elements, and as the days progress the software application would be getting more refined collectively.
The benefit of such a practice is manifold. It provides a tangible work product for showcasing no matter how messy that could be. The project managers can breathe easy since they now can see what’s going on, as well as a big risk of software integration failures are getting mitigated in steps. The other stakeholders who are not involved with the technical stuff would also get an opportunity to see what’s getting cooked in the kitchen. The programmers also get the benefit to self-correct their own mistakes which could have become a Herculean job if accumulated till the end. It also helps sustaining the morale of the team since a lot of eleventh hour bickering can be avoided.
Continuous Integration comes with its own limitations. Since everything in a software engineering project cannot be related to the programming only, it leaves a good part of the team out of the integration practice. Project Managers practicing agile processes should do a careful planning around them to ensure that their work products too move along harmoniously.
Also, the build process can be long and complex depending upon the size and complexity of the software application getting developed. Daily Integration would require some specialized full time staff allocation to do this job, thus raising the cost of project execution. For such cases, the frequency should be rightly defined. If you are a consultant working for a client, you may be asked to prepare daily builds since the client has read it through some magazines on Agile. If you think the software under consideration is large and complex, it may be a good idea to explain the cost impact of daily builds to the client. Agile does not address the project economics well (we will talk about it later) so it’s your responsibility to bring that under client’s radar for a better cause. For most of the software applications, a twice weekly frequency (Monday morning, and Thursday morning) should be enough. For large and complex applications, take it to a weekly frequency (Friday night, or Monday morning).
Introducing Continuous Integration could be a challenge if the integration process eats up a lot of energy and enthusiasm of the team members. The Agile practitioners use and recommend using the process improvement tools for improving the situation. Maintaining a shareable code repository would be one such step. Nowadays, having source management software like PVCS, or VSS are pretty common, so ensure that it’s in place for your project too. The other important improvement would come through automating the build process. The code compilation, build preparation, and deployment should be automated using scripts and templates to the maximum extent possible; this would provide significant incentives for increasing the frequency of integration during the course of development.
Monday, May 12, 2008
Agile Intelligence: Sprint
Those who have already read through the pages of Agile Alliance and Scrum Alliance should have fair familiarity of the cycle of software engineering by now. The diagram is epic, and is widely used anywhere when there is a discussion on Agile Software development. It has been hard, but somehow I resisted myself to upload the same diagram here, but the diagram is still important for everybody to understand the core purpose of being agile.
Those who are already familiar to the diagram should be aware of the term “sprint” as well. For the others, the definition goes like this: Sprint is a time period within which the team completes the development of the backlogs committed by them at the beginning.
Those who are well aware of the software development life cycle should know that the life cycle does not contain the “development” alone. To give a software application a complete shape, development should be preceded by the analysis and design; and should be followed by various flavors of testing, bug fixing, and deployment. The proponents of Agile took a conscious attempt to fuse all these activities into a continuous set of activities, and hence they used the word “development” only to describe the activities within a sprint. However, the sprint activities does not remain confined within development only, it does have analysis, design, and testing.
Those who are well experienced with the commonly used software engineering models like waterfall or iterative should know that the duration of a phase is driven by several constraints like time to market, availability of resources, etc. Sprint, on the other hand, is fixed with duration. Its duration is 30 days. Agile practitioners around the world follow the same guidelines, albeit with some minor changes to fit their purpose. In most of the cases, the sprint starts on the first working day of the calendar month, and ends on the last working day of the same calendar months. Some take it to 30 working days, thus making a 6-week schedule corresponding to every start date. Although, people stretching the duration beyond 4/5 weeks are quite rare, and those who do so, have mastered the software engineering processes before getting into Agile. Most of the Agile development teams are charged up to show something been accomplished at the end of the sprint, and sooner the better. So it’s a self imposed restriction towards stretching or postponing the deadlines, everybody around an Agile development project know that something is getting done at the end of the sprint, which in turn is not far away on the calendar.
A software development project may have many sprints. Since the sprint duration is fixed, depending upon the size and complexity of the software application, the number of sprints varies. The sprints must be sequential, i.e., end of one sprint can only trigger the start of the next sprint. While the same team is expected to continue on the following sprints as well, the Agile methodology does not put any restriction on this area. In fact, at the end of each sprint, the product owner and the sprint master should take a joint review of the resource requirements on the following sprint(s) depending upon how much of the product backlogs should be converted into the sprint backlogs in a single sprint. End of sprint review would also help the sprint master to re-balance the team with the individual team members (e.g. somebody fell sick in the middle, or expected to go on a vacation on the following month, etc.).
Those who are already familiar to the diagram should be aware of the term “sprint” as well. For the others, the definition goes like this: Sprint is a time period within which the team completes the development of the backlogs committed by them at the beginning.
Those who are well aware of the software development life cycle should know that the life cycle does not contain the “development” alone. To give a software application a complete shape, development should be preceded by the analysis and design; and should be followed by various flavors of testing, bug fixing, and deployment. The proponents of Agile took a conscious attempt to fuse all these activities into a continuous set of activities, and hence they used the word “development” only to describe the activities within a sprint. However, the sprint activities does not remain confined within development only, it does have analysis, design, and testing.
Those who are well experienced with the commonly used software engineering models like waterfall or iterative should know that the duration of a phase is driven by several constraints like time to market, availability of resources, etc. Sprint, on the other hand, is fixed with duration. Its duration is 30 days. Agile practitioners around the world follow the same guidelines, albeit with some minor changes to fit their purpose. In most of the cases, the sprint starts on the first working day of the calendar month, and ends on the last working day of the same calendar months. Some take it to 30 working days, thus making a 6-week schedule corresponding to every start date. Although, people stretching the duration beyond 4/5 weeks are quite rare, and those who do so, have mastered the software engineering processes before getting into Agile. Most of the Agile development teams are charged up to show something been accomplished at the end of the sprint, and sooner the better. So it’s a self imposed restriction towards stretching or postponing the deadlines, everybody around an Agile development project know that something is getting done at the end of the sprint, which in turn is not far away on the calendar.
A software development project may have many sprints. Since the sprint duration is fixed, depending upon the size and complexity of the software application, the number of sprints varies. The sprints must be sequential, i.e., end of one sprint can only trigger the start of the next sprint. While the same team is expected to continue on the following sprints as well, the Agile methodology does not put any restriction on this area. In fact, at the end of each sprint, the product owner and the sprint master should take a joint review of the resource requirements on the following sprint(s) depending upon how much of the product backlogs should be converted into the sprint backlogs in a single sprint. End of sprint review would also help the sprint master to re-balance the team with the individual team members (e.g. somebody fell sick in the middle, or expected to go on a vacation on the following month, etc.).
Friday, May 09, 2008
Agile Intelligence: Backlogs
The Agile development process talks about two types of backlogs: Product Backlogs, and Sprint Backlogs. We will talk about the specifics a little later; let’s try to understand the fundamental constitution of backlogs first. Backlogs, unlike the word suggests is not about falling behind. It’s about maintaining a to-do list. A backlog is a master list of all functionalities desired in a product (software application).
The proponents of Agile development are of opinion that the details specification of a software application are not known during the start of the project. So, the obvious requirements are listed down as the product backlog. As the project progresses, the requirements get more elaborated and refined, and the product backlog grows accordingly. While such a statement does not provide enough confidence at the investor level, but works pretty nicely to the people who work on the project.
The backlogs are documented in tabular format. Each backlog has a name (or short description) to it, and a priority mark (high/medium/low) with the respect to the critical success factor. The product owner takes the responsibility to maintain and update this backlog. The product owner gets the inputs from the various groups of potential users and beneficiaries of the software application, and converts those inputs into backlogs with due priority. The product backlog does not have any other project planning attributes attached to it.
The product backlog works as the input to the sprint backlog. A sprint backlog is the list of backlog items that the team commits to deliver in a particular sprint. In a way, sprint backlog is a sub-set of the product backlog. However, there may be additional backlogs into a sprint backlog which are normally the supporting work for the sprint. Sometimes, a product backlog is also broken into multiple sprint backlogs for better granularity and distribution of work. As example, the product backlog may specify that only the employees of the organization will be able to access the software application. At the sprint backlog, this may be broken down into the login pages, authentication module, and integration with the enterprise directory. Sprint backlogs are picked up by the team members, which is largely a self-assignment process. The sprint master keeps track of the assigned work through the scrums. Scrum also helps the sprint master to fill up the start and end dates of each sprint backlog.
The webmasters of the Agile Advice website has brought in the Queuing Theory to understand the backlogs better. The theory helps the mathematical analysis and optimization of the work packages flowing from starting point to the finish line. The theory denotes the entire work package completion path using the time interval of having the work packages for start, the time required to complete the work within each package, and the number of resources (in software engineering, it should be the number of team members) available for the work packages. My personal opinion is not to focus too much on the advanced techniques at the beginning. Just apply some common sense if you are a sprint master. In case you have to apply something fancy, first focus on the principles and best practices of software project planning, tracking, and supervision.
The proponents of Agile development are of opinion that the details specification of a software application are not known during the start of the project. So, the obvious requirements are listed down as the product backlog. As the project progresses, the requirements get more elaborated and refined, and the product backlog grows accordingly. While such a statement does not provide enough confidence at the investor level, but works pretty nicely to the people who work on the project.
The backlogs are documented in tabular format. Each backlog has a name (or short description) to it, and a priority mark (high/medium/low) with the respect to the critical success factor. The product owner takes the responsibility to maintain and update this backlog. The product owner gets the inputs from the various groups of potential users and beneficiaries of the software application, and converts those inputs into backlogs with due priority. The product backlog does not have any other project planning attributes attached to it.
The product backlog works as the input to the sprint backlog. A sprint backlog is the list of backlog items that the team commits to deliver in a particular sprint. In a way, sprint backlog is a sub-set of the product backlog. However, there may be additional backlogs into a sprint backlog which are normally the supporting work for the sprint. Sometimes, a product backlog is also broken into multiple sprint backlogs for better granularity and distribution of work. As example, the product backlog may specify that only the employees of the organization will be able to access the software application. At the sprint backlog, this may be broken down into the login pages, authentication module, and integration with the enterprise directory. Sprint backlogs are picked up by the team members, which is largely a self-assignment process. The sprint master keeps track of the assigned work through the scrums. Scrum also helps the sprint master to fill up the start and end dates of each sprint backlog.
The webmasters of the Agile Advice website has brought in the Queuing Theory to understand the backlogs better. The theory helps the mathematical analysis and optimization of the work packages flowing from starting point to the finish line. The theory denotes the entire work package completion path using the time interval of having the work packages for start, the time required to complete the work within each package, and the number of resources (in software engineering, it should be the number of team members) available for the work packages. My personal opinion is not to focus too much on the advanced techniques at the beginning. Just apply some common sense if you are a sprint master. In case you have to apply something fancy, first focus on the principles and best practices of software project planning, tracking, and supervision.
Subscribe to:
Posts (Atom)