Introduction
Agile projects are sometimes perceived as a valid excuse for no documentation. But documentation is a good and valuable deliverable, especially for large-scale development projects. It keeps everyone –stakeholders and the implementation team – honest, accountable, and on the same page. And that should include Agile projects, as my team recently discovered on a web application development project where we applied Agile principles to documentation, integrating it into sprint planning and making smart choices on how to structure the document.
Documentation in Waterfall v. Agile
One of the common critiques of Waterfall is that the method generates copious documentation that has little impact on the ability to complete projects on time and budget or the quality of the resulting implementation. The argument is that a significant portion of the documentation produced before development provides little value or it quickly becomes obsolete by the time it is used. Agile methodologies, on the other hand, avoid these problems by maintaining general requirements in a backlog and organizing projects into sprints. During each sprint, a subset of requirements is properly vetted, understood, and implemented.
Further, there is a consensus about what types of documentation should be created for Waterfall projects: Requirements, Design, Use Cases, As Built, allowing for some variation to account for corporate culture, system-specific nature, and so forth. For Agile projects, however, there is little agreement around documentation practices. Common recommendations are to create as little documentation as possible and to delay documentation as much as possible within the project timeline. Oftentimes user stories captured in each sprint are considered sufficient documentation, even though this may be buried within a development tool or displayed on sticky notes on a sprint planning wall.
While Agile’s minimalist documentation strategy may be appropriate for internal or small projects, it is certainly not acceptable for major projects or for a consulting company such as Stonebridge. From that perspective, the Agile approach does not by itself negate the need for documentation. But it does require that we rethink the purpose and nature of documentation efforts when working with this methodology – which is what my team and I discovered in a recent project. While several articles and blogs agreed on the need for documentation in Agile projects, we found no concise approach that fit our needs. So, we decided to come up with our own.
Applying Agile Principles to Documentation
While we chose the Agile method for our project, we also determined that our documentation strategy must be driven by what was advantageous to our project. We knew that a key aspect of managing a successful project is to establish a common understanding or expectation for the final product between stakeholders and implementation team – which is documentation’s core value.
This step is particularly important when project work is done by an external consulting organization, as there are contractual implications for failing to meet target functionality and/or timelines. We also concluded that certain documentation related to Waterfall, such as the As Built document, would help our client with support operations and to properly plan future phases. Given our client-centric priorities, we chose to consolidate core documentation under a new document type – we called it the Specification document – and to apply Agile principles to the documentation process.
The Specification Document
The Specification document can be understood as a hybrid between traditional Waterfall Requirements and Design documents. It focuses on describing the expected behavior of the application and outlining key rules that affect application behavior, such as business process conditions, permissions, and basic validations. (I realize that from a purist perspective, there is a clear distinction between a requirement [“what is needed”] and the design [“how it is done”]. But the reality is that this line is often blurred as end-users tend to express expectations by introducing design elements into their requirements [e.g., “When the user has X rights, she should be able to double-click here and they system should …”]).
We made the Specification document an integral part of our project by managing documentation as an iterative, progressive deliverable, as we did for the application development. Additionally, instead of writing the document before starting development, we wrote a part of it in each sprint. We planned our sprints with development tasks and documentation tasks. The documentation tasks in sprint aligned with the development tasks in the subsequent sprint. The idea was to always have the Specification ready once development on a new sprint got started. By narrowly focusing on immediate requirements and development, we avoided spending effort on documenting concepts that were very likely to change.
We also kept the Specification concise by avoiding documenting the obvious and by using a layout and style that not only resonated with stakeholders, but also provided enough details to developers as they approached the implementation of specific functionality. On this specific web application project, page layouts followed a common pattern. Each presented a grid of data rows and, depending on data and user permissions, users had options to see details for a one row and/or perform actions on one or more data rows directly from the page that displayed the grid. We documented this in the Specification by having a section with each page detailing the grid (columns, available filters and grouping options); individual and group functions available for data rows; and layout for detailed view (along with actions supported in that view). Specific business rules and permissions for functions were laid out in detail when more complex business logic was required, and stated succinctly in case functionality was obvious. Different documentation styles were introduced on sections when they were appropriate to document expected behavior, but without imposing the burden of having to do this for all features. Use cases and state transition diagrams are some examples of techniques we incorporated in the Specification document.
We found this approach to be very effective in finding a middle ground to cementing the “binding contract” between the project stakeholders and the implementation team, without having to spend unreasonable time in producing lengthy documentation that – let’s face it – nobody has time to review! While there were occasional misunderstandings and on-the-fly document updates, this Agile-inspired approach to documentation paid off at the end of the project, where we found we had steadily built a comprehensive document about our application.
In hindsight, the Agile approach to documentation worked in our favor for two reasons. First, since we worked on one functional area of the application at a time, we could learn what the client liked from the implementations completed in previous sprints. That step helped ensure that our proposed specifications were closer to target as the project progressed, requiring fewer revisions. Second, we could learn the most effective way to communicate certain concepts to ensure a common vision between client and implementation team. At the end of the project, we supplemented the Specification document with a Deployment Guide and a Configuration document, which were very quick to create. In this engagement, the client took responsibility for the User Manual.
Conclusion
Agile is a valid methodology with built-in advantages for developers in terms of reducing project time. That said, Agile should never be excuse for no documentation nor should it supersede the need for stakeholders to sign off (even informally) on requirements. Agreement between the client and the development on what the final product should be is critical to a successful project. For Agile projects, this can be accomplished with the Specification document, which applies the Agile approach to documentation, integrating documentation as part of sprint planning, choosing rationally how to structure the document, and supplementing it with other concise documents at the end of implementation as needed.