Sunday, November 22, 2009

Agile Intelligence: On "Enterprise Scrum"

In his book Ken Schwaber described how an enterprise can adopt and implement scrum across all IT work. Size-wise, the 176 pages long book is divided into two parts: the general text, and appendixes. The appendixes describe some of the key concepts and nomenclature of the scrum, while the general text covers the key topics. The general text also illustrates some of the cases, where scrum implemented, and the kind of results / benefits accrued.

The good part of this book is it talks about some good theories of the classical project management and behaviors. As example, it talks about four stages for team performance: form, storm, norm, and perform. This one is nothing to do with Agile, it was a separate concept altogether. The good part of the book is that it adopted it.

The not so good part is some of the key messages. A few of those are highlighted below:

Scrum cannot be tailored for an organization, Organization needs to change itself: This illustrates the inflexibility of the case studies. If an organization needs to change itself for adopting scrum, while scrum stays unchanged, it practically makes the organization inflexible after scrum adoption. An inflexible organization is certainly going to take a downward dive on every aspect of the business.

The team needs to be self managed, self organized: This practically puts an end to the possibility of inducting junior team members. In every team there needs to be somebody who should be taking the lead role or coordinator role. The person needs to be somebody who can be looked up, and needs to be a role model for the junior developers while they are growing up. The book stays silent on this point.

Fire thy sales guy attitude: In a case study, the book found the solution in firing the sales guy who made some aggressive commitments to the client, and pushed developers harder for completion. A different case study of aggressive but successful sales guy would bring more relevance into it.

Developers should not be pressed: Throughout the book, the author emphasized not to press developers for producing more. It talks about giving the liberty of committing their own release dates, but not much have been mentioned on accountability, and aligning the dates with business reality. The book scares the readers on compromising on quality while speeding up the development, but surprisingly silent on measuring quality of the work product.

Derogating Offshore Work: The book does not support offshoring of work. Instead if asks for improving productivity at onshore. True, that productivity needs to be improved, but it’s yet to be proven that offshore work is less productive. In today’s world, when majority of the software development work is being done from offshore locations, it’s a bit surprising to see the author has missed out the success stories and data.

Overall, the book is not a recommended buy. But if you are an avid reader, go read it with open mind.

The original review comments was posted at Amazon.com.

Saturday, July 11, 2009

Agile Intelligence: The Story of a Miserable Project

In one of my recent posts, I tried to tell a story of a successful project following Agile methods. Now it's time to flip the coin: let me try to tell a story of a miserable project. This was not a failed project though. The project eventually completed, with near 100% schedule variance and comparable effort variance, but with acceptable quality and conformance to all norms of closure. To give credit to myself (or to Agile), I don't have any story of failed project to tell.

It was a Sunday afternoon, when I was trying my eyes to believe what are being painted in front of me. It was a cozy meeting room, a room where I prefer to talk while standing, a room cozy enough to prevent me pacing bi-ways out of nervousness. With me, there sitting my boss, and two other developers. My boss summoned me to be there, he got a salvo or two from a senior executive of the client. The developers were trying to put forward the project situation, as-is. The first version was a little less than hunky-dory, and I learned that the team conducts a Scrum daily. An attempt to align the client's frustration with the project situation created the second version of the project situation, which was not only far away from being hunky-dory, but a lot scary as well. A critical phase in the software development life cycle got skipped: Testing. The developers were unfazed. The project called for 2 developers, so 2 developers started working. Developers never realized that they need to handover the codes on time to the testers. No body followed up with them either. As a result, the project directly moved to UAT after construction, that too when the construction was not complete. On the day one of UAT, client realized there was nothing for him to see, verify, and accept. He got furious.

Despite of having daily interactions with the client and developers, the project still went into red. It went into red, because there was a lack of appreciation for the SDLC phases. People loved Agile, tried to do some Agile stuff, without realizing that the fundamentals didn't change. Agile methodology does not cover SDLC, in certain aspects, it negates it. That prompts the team practising Agile, and bend for programming to miss out the big picture: the need of testing before releasing the product.

The story did not end here. Follow on meetings with the developers revealed none of them know how much to code and implement. There was no requirements document either. So developers were doing coding based on their understanding of the functionality which were not only inadequate, but also carried the risk of being misinterpreted. As a result, on the day two of UAT, the problem in hand was not just incompleteness of codes, but a total unawareness of the size of the remaining work.

The rest of the story was a familiar tale, an approach that have been followed from the stone age of software engineering. Some of the reputed fire fighters were deployed on the scene. They started working 20 hours a day everyday: coding, understanding requirements, testing, communicating with the client like a PM, and generating intense push-pull force between the buyer and supplier. After 2 weeks of a grueling endeavour, the project started becoming back on track. Client started seeing the work products getting delivered, client started seeing less errors in the delivered work products. The fire fighters started disbanding in search of another fire.

In short, just Agile does not work always. It requires a good understanding of the SDLC. In fact, all successful Agile practitioners today have got their foundations built through the classical SDLC process. Inherently, they practice the good stuff out of classical SDLC. An attempt to bring methods in the madness has always turned out to be effective in Agile projects, provided the teams carry an openness to accept good things from outside.

Tuesday, June 30, 2009

Agile Intelligence: Staffing Your Team

Mobilizing the right team for executing a project successfully using Agile methods is important. More importantly, Agile method needs experienced coding hands, because of its emphasis on liberating the programmer from manager's command. A project following Agile methods does not have any project manager. The scrum master's role is limited to asking three questions only, that too far open ended. In such a situation, the developer is entrusted to compose the work performed, the work to be performed (in the guise of will be accomplished till next scrum), and estimation of effort. It's true that only the individual can specify how much time he/she would take to complete a specific task, because only the individual knows how much he/she takes to complete similar tasks. In other words, how much time he/she had taken in similar type of tasks in the past. In other words, how much experience he/she can leverage this time. More the experience, better would be the estimate, better would be the productivity.

The proponents of Agile methods counters that such a developer profile is something that everybody asks for. It's not uncommon. While such counter statements are true, the fact that Agile Programming does not have any room for junior coders is true as well. So much that the project following Agile Programming with a bunch of junior coders (and no project manager) is susceptible to failure more often than the traditional one.

The good news is, there is a way out. Tailoring the Agile methods a little bit would make a tremendous impact on achieving right mix of the team, and would help in shaping the path for individual development for the junior coders. Here is how.

Pair programming allows two developers to develop application while working on a single computer. In a project team that follows Agile methods, each pair can be comprised of a senior coder and a junior coder. At the beginning, the senior coder will act as the driver to get the work started, while the junior coder will be the observer. By junior coder, I am indicating to a person who has been trained on the necessary skills to perform the job of programming, like programming language, but not necessarily has any experience of doing similar project before. For the first week, or may be for the first two weeks, the senior coder will be the driver. Subsequently, they will start interchanging the roles. While the junior coder works like a driver, the role of the senor coder would be more like a navigator instead of an observer. This would ensure that the progress is happening in the right direction, with right quality, and adequate learning for the junior coder. This would also address the question that critiques often raise: Productivity. With two senior coders programming in pair, the productivity cannot be more than the aggregate of what they would have been accomplished in isolation. If fact, it would be far less. With a mix of senior and junior coders in the pair, the probability of accomplishing the comparable output would be higher. This is because the junior coder is always coding in presence of experienced reviewers. When it comes to the tasks like effort estimation, it would be always the senior programmer who would do the estimates. This would ensure the accuracy of the estimates stay within the tolerable limits. Also, the senior programmer needs to factor the productivity level of the junior programmer.

There is a small drawback though. The codes produced by the experienced coder will not go through proper review. For this, the team may adopt a process of peer review across pairs. Peer review, in general, is part of the traditional way of software engineering, but will become very effective in quality assurance in such a situation.

To summarize, it's possible to induct junior profiles in the project team while following Agile methods. Pair programming would be the way to achieve that. Introducing peer review across pairs would ensure that the quality of the work products are not compromised. Such a tailored process would pave the growth path for the junior developers.

Friday, May 15, 2009

Agile Intelligence: The Story of a Successful Project

The preamble would say that all elements of this story are fictitious and if they resemble to any real life element, that would just be a coincidence. Having said that, the story starts with a project conducive enough to practice Agile. For me, it was not the first Agile project, but my interest to run it like Agile was on the higher side. I did not do any assessment to figure out whether Agile would work here or not (like I mentioned in my previous post), but my guts were telling me that this is the project for Agile methods.

Here it starts. I visited my client (the product owner) with one of my team mates to understand what the client wants. The client told us a story. For sake of anonymity, I would keep that story out of my posts. The story triggered a brainstorming session only to result a data entry screen and two reports. After the 100 minutes of the meeting, I added two standard features into it: login, and administration. At the end of the meeting (in about 2 hours in total), we agreed on a tentative timeline (needs to be shown in 2 weeks), project commercials (some thousand dollars to be billed), and came out to write our meeting notes.

My team mate started development, mobilized servers for hosting, and at the end of the first week, we made our demo of the data entry screen. No reports were ready, but nobody complained either. The audience of the demo comprised of some of the users including my client (4 in total). Some good suggestions came in. We noted down all, on spot classified the items that would take longer time and effort to implement. In support of us, the client shoot those down immediately by flagging them as features for future. At the end of the week #2, we released the application for UAT.

The UAT was planned to be an innovative one. Instead of running a script for testing, 40 users were identified and were asked to use the system. We gave them a hour long training on the application, along with a form to log their feedback. The client has mobilized a manager to collect all feedback forms, collate those into one, analyze those to various categories and identified those with the priority to ship. This compilation indeed became a collectors' edition due to its richness of the user feedback. Some of those are truly awesome, would not be possible to anticipate or detect by any developer or tester. Many of those reveal the need of having a user's manual, it was unbelievable that we had delivered a product without user's manual just because we were following Agile.

After a week-long UAT, another week was spent on implementing the prioritized feedback. Like the first one, we did the upfront cost estimate for the changes or new features (a few thousands dollars again), and picked up all programming bugs without hesitation. We followed a principle of fixed fee charging, so all programming bugs were serviced free of cost. Subsequently, the application was laid open for all 300 users.

The story could have ended here. But the project continues. Once it goes live, the client brought out a few more things to add in the application. A few more data entry screens, a few more reports to go. Now the application is live and kicking, articulation of the new requirements were far more easy. No wonder, my next visit to the client to gather requirements ran for good three and half hours. The requirements list was long, naturally it would take longer to implement. The client was agreeable to that too. This was the time I contested his thoughts. My opinion was that the project delivered good results in 2 weeks of development (and unit testing), 1 week of UAT, and 1 week of bug fixing / changes. We should stick to this plan. The fact that the requirements has got longer this time would mean that we have to slice it into two. Or three. Or four. Whatever it takes to cut down to the 2 week schedule. We had one developer at the start, we will add another one this time. That will enable the team to take twice the amount of work. But the schedule should not change. It would not elongate, it would not be shortened either.

My client agreed. He didn't have to worry though since I was talking of frequent deliveries this time (most of the time we get our clients into the reverse kind of arguments). So we started the same schedule for quite a iterations subsequently. At this point, like a classroom teacher, I should start to map the Agile terms with the various acts in this project. But I am refraining from doing so. I assume the readers would be able to do it themselves. Those who are not sure, please read all my posts on Agile Intelligence from start.

Tuesday, April 07, 2009

Agile Intelligence: Is Your Client Aware

So far, we have been advocating the possibility and need of executing software development projects following Agile methods and on a fixed fee agreement. The benefits of Agile programming are well understood, as well as the need of adopting other good practices. Running project in fixed fee is also a consideration to provide greater comfort and de-risk client's financial commitments. Now the question comes, is the client really aware of Agile programming?

Like any other successful projects, Agile methods too require some good deal of client commitments for successful completion. Quite often, the system implementers jump into the Agile bandwagon with a promise of making client's life simpler (read lazier). As a result, when the client contacts are asked to provide their inputs frequently, at times everyday, all hell breaks loose. Similarly, clients are promised with frequent delivery by virtue of continuous integration (read on demand delivery). Sellers, in quest of selling the work, commit all the good things that Agile can bring on the table without uttering a single word on what it takes for a client to achieve that. Once the job is sold, the programmers land on ground, and starts realizing amount of damage been done.

The big ticket question is: how to educate the client on Agile methods? Seasoned consultants know the art of educating their clients with new tools, techniques, and practices; and those still hold good here. Client education cannot be performed though a sales pitch, cannot be performed if it doesn't bring any value to the client. The system implementer needs to first assess whether Agile method is really going to bring any incremental benefits to the client for the project under consideration. If not, which is likely to be a regular instance, let's not try to push for Agile.

Secondly, the involvement need of the client contacts are also to be evaluated. Agile method requires a lot of time and effort commitment from the client contacts, all those hours are to be justified. The point to remember here, hours from the client are not free. Those too get added to the project cost. The system implementer may not need to bear that cost directly, but it's a direct cost to the client for that particular project. So, if a good part of the project progress can be achieved by minimal client involvement, let's not plan for Agile methods there. Instead, taking a monolithic project plan with built-in Agile sub-phases may work wonders here.

Also, the involvement in multiple tiers is important. The traditional project management processes include a need to define the escalation path. In the era of Agile, that is still relevant. When it comes to managing the stakeholders with a varied degree of influences, it may not make sense to bring all of them into a single scrum, that too everyday. Instead, organize multiple scrum-of-scrums. Participants of each such scrum-of-scrums are to be carefully chosen. Also, the format of the scrum-of-scrums may require tailoring. By definition, each participant of the scrum-of-scrums are expected to be represent their own scrum teams. Here, in some cases, that will not hold true. Some of the participants will not be part of any scrum team, rather they will bring their organizational insights to enrich the other participants, increase their own awareness on the project progress, and in certain instances become more aware of the practices of Agile programming.

At this point, the Agilists (and its proponents) will strongly disagree with me, I swear. Scrum cannot be tailored, tailor your organization to scrum; is their dictum. I would humbly submit that I disagree. Organizations cannot change to fit itself into something prepared externally. Organizations can only adopt, articulate, and assimilate that. During this course of adoption organization tailors the external ingredient to fit with it, and over time of adoption it changes itself too, primarily to respond to the market forces. So, tailoring of the format is inevitable, if required, to be done for each client, for each project. During the course of tailoring, involvement of the client is essential. That's the learning point for a client highest intensity.

To summarize, emphasize the benefits of Agile programming with its cost while selling a job to the client. Perform proper assessment to see whether Agile is really needed, if not, don't hesitate to discard it. If you adopt Agile programming for a client project, ensure that right kind of client awareness and commitments are there.

Saturday, February 21, 2009

Agile Intelligence: Doing Project in Fixed Fee (contd.)

For a IT service provider, doing projects in fixed fee would mean the hours to be spent is capped. Each IT service company is having is its own benchmark hourly rate to compute fee from the estimate hours. That make the job of estimation very important. But Agile brings its own challenges: it doesn't have any end to requirements specification, not does it have any project/product sign-off criteria. Once somebody is entrusted to run a project on fixed fee, putting such boundaries are essential to keep the project afloat, financially.

Once more, we need to bring the PM back here. The PM will now evaluate the areas where boundaries are to be defined, and will do the necessary fencing to guard those. Let's talk about a bit on those common areas in general -

Scope Management: The scope of the work can be defined with various level of details. It can be a one-pager for a 5000 hours worth of work, or it can be a one-pager for a 50000 hours of work. Defining the scope of work crisply and succinctly is very important while agreeing to execute a project in a fixed fee. While drafting the scope, the local cultures are to be kept in mind too. Some regions on this planet expects the exclusions to be written down separately, others are fine with inclusions only. In some parts, it requires to be submitted in local language. All such cases are to kept in mind to ensure minimum (if not zero) room for ambiguity, second guessing, and arguments.

Requirements Management: Gathering requirements and documenting those for the software developers play a key role in keeping the costs at tab. While requirements are gathered, it's important to understand that the requirements documents will be read by the business users too. Accordingly, the clarity for each functionality / feature are to be documented. Objectivity of the requirements are also to be very clear: Is it being done for a specific sprint? Is it being done before submission of the fee quote? Is the fees are already determined? For the first and third questions, the stakeholders should be alerted if the requirements are blowing out of proportion.

The second situation is rather win-win combination. In such situations, the requirements analysis phase is taken out of the project and are performed separately. Most of the economic buyers are comfortable in paying by the hours for this phase, since the uncertainty is less. Even if it is to be capped at a fee, the duration of the work can be capped as well. Once the requirements document gets ready and accepted, the document(s) are used as the basis for quoting the implementation fee. The benefit of such situation is that it allows the IT service provider to quote an educated fee. It also allows the provider to invest, benchmark, and improve its estimation model to a higher level of maturity and certainty.

During the acceptance phase, new requirements and modified requirements are inevitable. The PM must drive those hard enough to get documented as individual CRs so that those can be priced separately. One popular approach is to bundle a set of logical CRs into one release, quote a fee (again fixed), and deliver.

Schedule Management: Once the project is underway, it's important to keep it on schedule. If the schedule is not on track, there is every possibility of leakage of hours through non-productive work and idle hours. Although, idle hours are not generally counted as the project hours, but it's still the cost to the company. Every PM should be aware of keeping it under control.

There would still be situations where the schedule cannot be kept under control. The effective way to keep the project financially viable is to cut down the team size. A slim team is capable to running long by burning less hours yet accomplishing all the expected tasks. The big assumption here is that the scope is under control.


To summarize, the project can be done in fixed fee and following some of the best practices of Agile method, as long as the team is open to borrow practices and processes from the traditionalists for ring fencing their own risks.

Saturday, January 31, 2009

Agile Intelligence: Doing Project in Fixed Fee

This is in fact a tricky spot: for both Agile practitioners, and the economic buyers. I will pick up the case for Agile practitioners a bit later, let's see where do the economic buyers stand with price-to-value game. So far, these group of people were sold on the Agile methods with the fact that they would get to see what's getting cooked in the kitchen. In turn, there were no certainty on the cost involved in cooking that dish. Increasingly, the economic buyers realized that they don't really need to be in the kitchen to see how the cooking happens. It's important to get the dish right; it should be palatable, and it should be hygienic. In other words, the software application delivered for use should be effective in meeting its objective of existence, and it should be trustworthy for carrying out the business. The economic buyer, may be the CFO of the company, doesn't need to spend time in seeing that a singleton class is getting developed, tested, and deployed for other classes to invoke it.

Once that's realized, the next big question is how to ensure that the software production cost is contained with respect to the value it will eventually deliver. The easy path would be to determine the value (or IRR), and calculate back the maximum amount of money to be spent. Once the maximum amount of money to spent is confirmed, that becomes the budget for that particular application development.

So far so good. The problem is that there is still no certainty that the application will get developed within that budget. After running sprints after sprints, the budget may get burnt completely only to deliver a bunch of classes and database objects with no realizable business benefits. The poor economic buyer, being part of the entire drama of cooking codes, would now be left with the pieces of marvels, but without any clue what to do with that. Agilists delivered, though.

On the contrary, the help is actually around. The IT service suppliers are available to design, develop, test, and deploy codes with demonstrable credentials in hand. The competition among them brings the benefits of bidding fixed fee quote, thus insulating the economic buyer from any financial risk. This is a proposition too hard to resist. But can Agile methods be fitted with this fixed fee structure?

Agilists, on the other hand, had a different problem in hand. What used to be a sideline process so far, has become a mainstream. Being a mainstream software engineering process, it's difficult to avoid the monetary question: how to ensure that the business benefits are realized at right cost? To address this concern, the initial thought was to follow Agile methods with a fixed budget. The product owner should know how much money to spend. Based on that, the product owner would determine the number of sprints to be performed, with the number of team members. Subsequently, the product owner should also make a broad classification of the features and functionality to various sprints. With this, our Agile project has become a fixed sprint project. A seemingly uncompromisable deviation, but okay!!

The fun part of market forces is it not only negates inefficiency, it also makes an open mockery of it. When the competition demanded to quote fixed fee for a software development project, even if it is supposed to follow Agile, it really pushed Agilists very hard. Too hard to stay sane. This time the question of ethics came out of the pocket. Apparently, asking developers to develop a program in fixed fee is an unethical practice. With this hypothesis, the argument was structured that software development in a fixed fee fashion is unethical practice. Agilists, being always on the right side of ethics (no matter how much they abused the traditionalists before), cannot accept this. So, Agilists will do T&M work.

Without going to the details on ethical practices and all that, let's try to look into the issue with our feet on ground -
1. Agile is going to stay, some of its practices are not available anywhere
2. Fixed Fee work is inevitable, a buyer is not actually buying programmers' time, buyer is buying a piece of software
3. How to fit the best of Agile practices into a fixed fee project would be the puzzle to solve.

My next post will try to address it.