The ideas behind agile methodologies address the very root of all the problems I’ve experienced delivering software to an unsuspecting world for over 30 years: business people and technology people are different. They use different language, think differently and worry about different concerns. In my experience, both sides of this business-technology divide are doing the best they can, but the old project management paradigm—big specification up front, along with long design, coding, testing, and delivery cycles—isn’t appropriate for software development. It probably never was, but it's what we knew how to do.

The old way caused frustration and mistrust between business and technology teams. Business, in their mind, clearly stated what they wanted. Technology heard the request and created the code, but what business said may not have been what technology heard. By the time technology delivered what was asked for, business’ needs changed. The recriminations started: business didn’t trust technology because they said everything was fine, when it apparently wasn’t, and technology didn’t trust business because they didn’t ask for what they really wanted.

At the root of this disconnect is the simple truth that the business folks did not know what they really needed as a deliverable. Business could not know what they needed because they were trying to predict the future needs and behaviors of an unknown group of users.

Agile methodologies for both business and technology shorten the specification to delivery cycle, so that the feedback cycle is short enough to expose mistakes and misunderstandings quickly, when they are cheap to fix and, thereby, trust is preserved. By delivering some functionality as early as possible, the business people can test their ideas with real users and provide a better description of what they need in the emerging product.

Extreme Programming: why it works

My team has used Extreme Programming (XP) for the last year and a half. As methodologies go, XP is very close to the bare metal of product software development. It is well-suited for projects with:

• A likelihood of change in the requirements throughout  the project

• Highly-involved business participants

• Teams of less than a dozen developers

XP addresses very hands-on tactical ways to make deliverable product iterations in a rapid manner, while maintaining quality for long-term maintainability and flattening the cost curve. XP also mitigates common problems with creating and maintaining software:

• Misunderstood and changing requirements

• Increasing complexity over time

• Poor communication

• Late feedback

• Schedule delays

• Unusable documentation

Ideally, XP releases new working software as often as every day, so the cycle of delivery and validation by business owners provides immense learning opportunities for everyone.

The engineering team learns how to interpret the business requirements. The business side learns about the speed of the team in delivering well-defined units of functionality. The backlog allows everyone to see how much work is really left, and based on the feedback of team velocity, they can predict more accurately when the next project milestone is likely to be reached. XP enables this short cycle time at the technology level, and that is very valuable.

Extreme programming works because it separates spheres of responsibility. Developers make technical decisions and business people make business decisions. The rapid feedback process provides the team a with a safe way to make small failures early on and change direction quickly. The rapid production cycle also provides a structure for the team to work together and solve problems in a supportive environment.

Finally, XP also acknowledges that technical documentation is important for the long-term maintenance of software systems. Rather than creating static design documentation that will be obsolete minutes after being published, the emphasis is on creating meaningful test cases that both describe and test the desired software behavior. With diligent adherence to creating the test cases before writing the code and running the test cases throughout coding work, the methodology ensures that coding changes will not cause regression errors and that the test suite is always up-to-date.

 XP Roles: What it looks like

The roles of XP are few, but well-defined. Each role requires people that are dedicated to the process and fully engaged.

Product Owner:

The Product Owner is co-located with the rest of the team and available for discussions and on-the-spot decision making all day, every day.  This individual represents the business and decides what features the product needs, and the order in which it needs them. The Product Owner makes business decisions, such as what to do if progress is too slow, e.g. add personnel or defer features. This role brings the distilled thinking and hard research of the organization to the implementation team. This person has the accountability to create simple, complete, and easily-measured descriptions for each desired feature. And, the Product Owner is the only person that can accept or reject a story as complete.

Engineering Team:

This is the group of developers that implements the desired features. They are the ones to estimate the implementation complexity of each feature story. They are responsible for writing comprehensive, automated unit tests for each bit of functionality in the system.

Testing Team:

This is the group of business-knowledgeable folks that verify that the iterations of the system deliver the desired functionality.

The Difficulties of Adoption: why XP isn’t the holy grail of methodologies

XP, like all agile methodologies, can be difficult to introduce into businesses with a culture of waterfall expectations:

• Crystal ball: belief in their ability to predict the future

• Silos: Rigid separation of management and individual contributors

• Tradition: Reliance on traditional project management techniques, like Gantt or Pert Charts

Acknowledging that long-held beliefs in “the way we’ve done this before” are very real and will take a lot of concentrated effort to change is the first step in making it possible for agility to be introduced successfully into these cultures. XP requires the commitment of the company and team to work.

Talk to Chuck! Ask questions @cloudcitydev