After nine years of developing software packages for space mission programs, Dr Winston Royce published his “prejudices” in a presentation titled Managing the Development of Large Software Systems in 1970, more than five decades ago.
In addition to being a joy to read, Winston Royce’s paper is insightful and exceptionally lucid. It is not for those insights, however, that the work is famous; it is because it has (unwittingly) propelled Waterfall to the forefront of software project management.
Despite the influence of Royce’s paper on Waterfall’s future, the concept of a sequential process for the delivery of large software projects predates the document itself.
Waterfall was first described in a presentation at the Symposium on Advanced Programming Methods for Digital Computers on 29 June 1956 by Herbert D. Benington (Wikipedia). Finally, the term Waterfall was never used until it appeared in a 1976 paper by Bell and Thayer.
Waterfall was the uncontested software project management framework until the 1990s; by then, serious drawbacks had already been identified, and candidates like Scrum (1995) and Extreme Programming (1996) were being tested.
The final blow was dealt in 2001 by seventeen software developers meeting at a resort in Utah, US. They published the Agile Manifesto with twelve principles on how best software delivery can be achieved in the 21st century.
Despite its clear advantages, Agile had a very slow takeover, as the 15th State of Agile Report still shows; about 35% of the surveyed companies have less than two years of experience with Agile!
Waterfall lingering for over two decades after Agile’s inception is why these lines are being written. But that’s not all. We first need to understand Waterfall to understand Agile.
1.2 What Other Topics Does this Series Discuss?
In this series on Project Development, we will cover quite a few topics, which we invite the reader to go through.
2. Managing Large Software Projects in Winston Royce’s Times
2.1 The Sequential Approach of the Waterfall Methodology
Before discussing Waterfall, it is essential (and enlightening) to understand software delivery in the 1970s, which Royce describes in the same paper.
The iterative waterfall approach to managing large software development projects that Dr Winston Royce imagined.
After nine years of creating software for Lockheed Software Technology Center in Austin, Texas, Winston Royce formulated the following ideas for managing the development of large software systems:
There are only two value-generating steps in the software development lifecycle that the customer is happy to pay for: analysis and coding. Furthermore, these are the only two steps involving “genuinely creative work” where customer value is created.
In case of unforeseen difficulties, there is an “iteration with the preceding and succeeding steps but rarely with the more remote steps in the sequence”. This limitation on how far back you need to go makes the process more efficient, thus minimizing the cost of change.
The reality is that an error uncovered during testing might require at least a redesign and sometimes a rewrite of the specifications.
These points describe the reality of software development in the 1960s and 1970s. A sequential approach centred on value-generating analysis and development stages with auxiliary and supporting stages before and after.
Despite this hard reality which Royce has experienced during his assignments, he still believed that the Waterfall project management approach was “fundamentally sound”, and with five additional features, development risk could be heavily reduced.
Let’s review those enhancements relevant to our present state of technology.
2.2 Waterfall Project Management with a Hint of Agile
The five enhancements proposed in Royce’s paper are as follows:
Introducing Software Design. We insert a preliminary design phase between the Software Business Requirements and the Analysis phase.
During this phase, you “[…] allocate processing, functions, design the database, define database processing, allocate execution time, define interfaces and processing modes with the operating system, describe input and output processing, and define preliminary operating procedures”.
In this new step, Royce intends to create a software design where any conceptual issues can be uncovered before any development commences.
Create ample, clear, and adequate documentation. Royce pinpoints with surgical precision the advantages of quality documentation.
First, quality documentation provides tangible evidence of completion (avoiding the “I am 90% finished syndrome”).
Second, if documentation is poor, the design will be lacking, especially in the early phases of the project, where design, specs, and requirements documents are virtually the same.
Third, without good documentation, the software must be operated by those who built it.
Finally, good documentation permits effective redesign and updating when system improvements are required.
A Proof-of-Concept (POC) or prototype is essential if the concept of operations, technology, or idea is novel or innovative.
A POC would help uncover vital flaws or information that can be leveraged as early as possible during the implementation.
Planning and executing a test strategy. “Without question, the biggest user of project resources, whether manpower, computer time, or management judgment, is the test phase.” Therefore, it is the riskiest in terms of budget and schedule overruns. Additionally, with the testing being the last phase of the implementation, you quickly run out of alternative options. To overcome these challenges, Royce suggests:
Proper documentation like test plans and test results
Unit tests (although he does not use the name, he refers to testing every logical program path with “some kind of numerical check”)
Code review: Royce believes that visual inspection of the code can uncover small mistakes; there is no mention of style checking, clean code, or adherence to software design rules, though.
Enhancement 5:Â Involve the customer throughout the journey.
Royce focuses on customer involvement to avoid misinterpretation of requirements, but in today’s software projects, the customer is involved in most project phases.
Many sources have hinted at Dr Royce’s paper as more of a critique of Waterfall (some even say it’s harsh criticism). I have not been able to detect that in the text. On the contrary, Royce believed the Waterfall method “to be fundamentally sound”.
Other sources have claimed that Agile was hinted at in the paper’s second half, where Royce visualizes Waterfall as an iterative process. Upon closer inspection, the iterative element is not similar in spirit (at all!) to what Agile offers, especially in smaller, more frequent releases, which is entirely foreign to Waterfall and Royce’s paper.
Royce’s paper described the state of Waterfall at the time of writing (1970) and the potential enhancements to eliminate any risk of redesign and rework. The ideas were not revolutionary. On the contrary, the paper made Waterfall a star for the next decades.
3. What Is Waterfall Project Management?
This section will discuss Waterfall Project Management as implemented today. It will focus on the different stages of Waterfall methodology and its fundamental principles.
3.1 Stages of Waterfall Project Management
The Waterfall method breaks down all project tasks and activities into a pipeline of sequential stages where each stage depends on the output from its predecessor. Each stage in this pipeline specializes in one area of the software development lifecycle or SDLC.
The figure below shows the stages and documents prepared for each stage.
The present Waterfall model retained the fundamental aspects of its older versions.
Looking at the diagram in Royce’s paper and comparing it with the one above, we immediately notice that Waterfall project management has not significantly changed since Royce’s time. The same core stages, analysis and development, remain central while requirement gathering, design, testing, deployment, and operations have become integral parts of the SDLC.
In the above diagram, we can see four subloops that imply an iterative process. These subloops allow the following issues to be addressed:
Analysis –> Requirements — This loop represents the requirement definition and refinement process.
Business requirements refinement: If any inconsistency or contradiction in the business requirements is uncovered during the analysis phase, the requirements might be reassessed and modified.
Design –> Analysis — During the design phase, various design decisions will need to be made, and further input from the business stakeholders will be required.
Design constraints: Any software system is subject to design constraints from the technology or overall system architecture. The requirements must often be revised if they break the existing architecture or imply a radical change.
Coding –> Design. Developers might encounter several problems during implementation, so they must return to the drawing board.
Technical limitations: In this scenario, the system specifications laid down by architects or system engineers might require a drastic revision of the existing software architecture and design or a significant refactoring. In extreme cases, the technology used might not be suitable for implementing the new features.
Testing –> Coding — This is a scenario familiar to all software professionals. Bugs uncovered during testing will need to go back to developers for fixing.
Bugs and faulty implementations: We assume that at this stage, any bugs uncovered in the testing result from some misalignment between implementation and specifications. In this case, either the specs are incorrect or the implementation is faulty.
Operations -> Requirements Gathering — Usability issues or new features require change requests to be addressed.
Usability issues typically go unnoticed and are only uncovered in production.
A complexproduction process such as Waterfall, with its many stages, interdependencies, specialized skill requirements, and iterative subloops, can only perform well if:
Requirements and technology are well-known in advance:
The requirements can be well-known, understood, and defined in many projects before implementation. For such cases, Waterfall is ideal as it’s easy to implement and has a low risk compared to Agile and DevOps.
Examples of projects are compliance audits and hardware maintenance.
Low customer interaction:
Like-for-like software implementations where existing platforms are upgraded to newer versions or replaced entirely with new technologies without radical modifications to existing functionalities are great candidates for Waterfall.
Things can radically differ with novel ideas or technologies where the developers and the customer must continuously work together to refine the requirements and work around technical or budget limitations. Agile is better suited to coping with uncertainty and complexity in such scenarios than Waterfall.
3.3 Waterfall Project Management Stages and Documentation
The below describes the seven stages and the documents produced in each.
Project Inception
A Project Initiation Document defines the project sponsors, prerequisites, and overall scope.
A Service Level Agreement(SLA) sets the expectations between the supplier and the customer and the metrics by which the efficiency of the process is monitored and approved.
Requirement Gathering and Definition
Gap analysis determines the delta or missing features that must be implemented before the product meets the business requirements.
A High-Level Project Plan comprises the cost and schedule derived from the high-level estimates.
Statement of Work (SOW) describes the scope, deliverables, timeframes, prerequisites and milestones.
Test Strategy is a high-level plan for testing the new releases. It is more of a guideline for testing than actual test cases at this stage. Discusses test approaches, test environments, and resourcing.
Source Code artifacts are the new version of the source code. This includes code review, pull requests, and other artifacts produced in the development.
Test Cases typically consist of summaries, acceptance criteria, reproduction steps, and test parameters.
Test Report summarises the results of the test cases, highlighting those that must be retested if patches are provided or those that prevent certain features from going into production.
Deployment
A run sheet is a step-by-step plan for production deployment, typically with a duration for each step. This is especially important if downtime is involved.
Operations
Installation, Admin, and Operations Guides are the manuals required to run the system by people other than those who built it.
Part of understanding Waterfall project management requires that we know the meanings of the phrases and terms associated with this method.
Developers, managers, and project leaders use diverse and rich terminology when referring to Waterfall. At first, they all sound the same and might seem confusing. But when we examine those terms more closely, we find that some make sense while others don’t. We also find that Waterfall can be an umbrella term and, depending on the context, might refer to slightly different things.
The terms used by software professionals and which we are referring to are the following:
Terms commonly used in project management, such as framework, philosophy, methodologies, etc., have been defined here, which we invite you to read before moving on with this section, as it will help you better understand the association between these terms and Waterfall.
As you go through the different terminologies, you will see some overlap and the nuances of the definitions can become very slim. This is to be expected. You might also notice some arbitrariness in the definitions; that is also true. The definitions provided are the results of this author’s research and current views.
With enough experience and effort, business requirements can be fully understood at the beginning of the project.
Changing requirements in an ongoing project is an aberration and must be treated or accommodated separately, preferably as change requests in a well-defined change management process.
Software solutions can be engineered:
Software solutions for any business can be engineered, and the final product details can be worked out before implementation.
A perfect design, fit for purpose, can be created during the early stages of the project.
Classical team structure, hierarchies, and communication
Power and status in the team belong to the more senior and experienced members.
Hierarchies are desired and revered.
Communication is through formal channels.
6. What Is the Waterfall Framework?
The six pillars of the Waterfall framework.
The Waterfall framework is the scaffolding on which the Waterfall methods, practices, and culture are based. This scaffolding consists of the following pillars:
Similarities with the manufacturing model:
Software applications are like cars and other manufactured products, where principles such as those from the Toyota Production System (TPS) apply.
The emergence of new ideas, technologies, or client preferences is not anticipated or easily accommodated. The world of Waterfall is not a complex system; it is just complicated.
Documentation is essential:
Technical and business documents are vital to the success of every project.
Every phase of the Waterfall process has documents that must be signed off before moving to the next stage.
Detailed planning is critical for project quality and governance:
Careful and thorough planning is vital for project governance and resource and budget allocation.
When things deviate from the original plan, it is usually a sign of poor management or execution. In Waterfall, there is no room for uncertainty.
Engineering and solution design are all you need:
Engineering and solution design requires a detailed understanding of the requirements and a mastery of technology.
The evolution of customer preferences with the product and the technology is not anticipated and is challenging to control.
Business analysts, developers, testers, operations, etc., have distinct skill sets, roles, statuses, and career paths.
Homogenous teams containing members with similar profiles and skill sets are the most efficient, as members can cover for one another during absences or work overload. They can also benefit from each other’s experiences.
Value-adding vs supporting tasks
The core activities where business value is created are Analysis and Development. The rest is necessary because executing only those two stages could not achieve satisfactory outcomes.
Aside from Analysis and Development, all non-value-adding tasks must be minimized in cost and schedule to retain a competitive advantage.
Professor Edgar Schein has studied organisational culture extensively, and his definition of organisational culture revolves around the following idea. A group of people joined together by a common objective develops shared assumptions taken for granted about their identity as a group, their methods, challenges, and solutions.
Let’s say that a new software development team is set up with a team head and five developers. During the formative stages of team development, the team will experiment with different methods, a lot of which will be recommended or dictated by their manager.
When the formative period concludes, the team will have formed their view of the world. The challenges they have faced will be distilled into lessons, which subsequently are taken for granted. Questioning those lessons will be seen as a threat to their identity and will be adequately repulsed.
The challenges the groups face will typically include software development projects. If the group successfully uses Waterfall as their project management model, we expect them to have acquired a Waterfall culture.
Moving to something else, such as Agile or DevOps, requires them to go through that same formative period again (this time, it will be a transformation) while simultaneously unlearning Waterfall and replacing it with Agile — not an easy task.
The Waterfall methodology is the conceptual framework and set of principles and guidelines governing the team’s methods when delivering a specific software development project.
While a framework is a more generic and abstract set of concepts, principles, and guidelines, a methodology applies only to a specific project or project type. For example, you might choose to use one set of document templates to deliver large projects; for smaller ones, a less extensive set might be recommended.
Therefore, we can think of a Waterfall methodology as a set of principles and guidelines inspired by the Waterfall philosophy, centred on its framework, but customized to suit a specific type of software project.
9. What Are Waterfall Methods?
Methods are structured approaches to completing specific tasks. The following list specifies the most common tasks performed in Waterfall-based methodologies and the specific methods in each case.
Project Initiation
Conducting workshops is a common Waterfall method to initiate projects. Project managers and sponsors sit together with their counterparts from the client side to agree on the project’s scope, budget, schedule, change management process, and work locations.
Once an agreement is reached, the details are captured in a Project Charter and a Project Initiation Document.
Project Planning
While various tools exist to assist project managers with project planning, a Gantt chart is the most common method for tracking progress, delays, and managing resources.
In hybrid models (Waterfall methods combined with Agile practices), Epics, Stories, and Tasks are typically created and tracked in tools like Jira. These are far more convenient as progress can be monitored in real-time without developers providing input that is often shaky (I am 70% done with this task…), time-consuming, and inefficient.
The budget can be monitored via timesheets, a common Waterfall method for tracking effort spent.
A typical Waterfall resource allocation method is permanently assigning resources to a specific project. While this has specific advantages (for example, resources are guaranteed for the project’s duration), it can be inefficient as resources might spend significant time waiting for requirements or other project dependencies to materialize.
Requirement Gathering
New requirements are often communicated in workshops and subsequently documented for approval. Once the requirements have been refined, a Business Requirements Document (BRD) is created and signed off. Workshops and BRDs are common Waterfall methods for capturing requirements.
Requirements can also be gathered via gap analysis workshops, another Waterfall method designed to identify gaps between a desired and current system state.
Solution and Software Design
A high-level design (HLD) and a low-level design (LLD) are the Waterfall methods of choice for tackling software design problems. Both documents analyse the requirements from different angles and map them to specific development efforts to be made on the product.
The evolution of customer preferences with the product and the technology is not anticipated and is challenging to control.
Feature Development
Development methods in Waterfall are based on an LLD. Features are developed and delivered into higher environments in batches, often by a team dedicated to the project.
The batch-and-queue method, inherited from manufacturing, is a hallmark of Waterfall and its distinction from Agile.
A Waterfall method for software testing is to conduct functional, system integration (SIT), user acceptance (UAT), or performance tests in higher environments.
Test reports are typically used to capture test results for review and acceptance.
Manual testing also seems to be a very Waterfall-centric method. Although a Waterfall methodology can easily accommodate test automation, test automation tools were perhaps not as powerful and sophisticated as they were now when Waterfall was at its peak adoption.
Support and Maintenance
In Waterfall, support and maintenance are conducted by separate teams with different competencies, cultures, and technical proficiencies in the product.
While this might make sense regarding budgeting, resourcing, and organisational structure and management, it can also be ineffective.
10. What Is the Waterfall Process?
The three principles tenets of the Waterfall process
The Waterfall process is the systematic application of the Waterfall methods and framework. It can be understood through the following:
Planning and Execution
The Waterfall process closely follows the sequential stages of the Software Development Lifecycle (SDLC). These stages are Requirement Gathering, Analysis, Design, Development, Testing, and Deployment.
At the heart of the Waterfall work organisation process is the batch-and-queue system, where features that can be categorized similarly are developed together in a single batch. Iterations in Waterfall are not common and incur a large overhead cost.
Once the artifacts are ready, a stage is completed, and the next stage kicks off. One of the hallmarks of Waterfall is the absence of clear Definitions of Ready (DoR). I believe this crudeness is not a shortcoming of Waterfall since the methods we are familiar with today were not as mature in Waterfall’s heyday. It was the project manager’s and the team leader’s job to push for closure of the current stage and move to the next.
In Waterfall, project management quality was the project manager’s responsibility. Work quality was often compromised due to the nature of the batch-and-queue system. We discussed this crucial point in Achieving Flow in Software Teams: A Short Guide for the Busy, which we invite the reader to go through.
In Waterfall processes, isolating issues is hard, making continuous improvement to the production processes difficult. This is another flaw related to how batch-and-queue systems are prone to hiding errors until very late.
Resource Allocation
The Waterfall process requires resources to be continuously available for the project’s duration. As such, this process is not very efficient since if there are any delays in requirement readiness or development, the resource might have to idle until it becomes ready.
11. What Are Waterfall Practices?
A practice is a method, or set of methods, applied repetitively in pursuit of technical proficiency. Waterfall practices are numerous, of which we distinguish the following:
Design
Preparing High-Level Design (HLD) or Low-Level Design (LLD) documents is a refined Waterfall practice, given how much emphasis Waterfall methodologies place on getting the design right in the first place and avoiding rework next.
Technical Documentation
The meticulous preparation of documents is a common Waterfall practice, and technical documentation writing is a well-defined role. Communication between different teams mainly involves sharing design documents, specifications, test reports, and others.
Project Plans and Gantt Charts
Creating and maintaining project plans and Gantt charts with tasks, milestones, percentage completions, risks, and critical paths is another cherished Waterfall practice that has matured incredibly well over the years, as evidenced by the software tools we now have to work with them.
All three of these Waterfall practices have faded almost entirely in Agile teams.
A technique is a method perfected by repetition and which includes art and skill in its execution. The most powerful Waterfall techniques can be summarized as follows:
Waterfall Project Management Techniques
Waterfall project management and governance require accurate and precise effort estimates. These estimates are then incorporated into a project plan where the critical path is constantly monitored.
Planning based on those estimates can be quite sophisticated, with parallel-running tasks and complex contingency planning.
The Gantt chart and software estimation methods are well-known examples of Waterfall project management techniques.
Waterfall Stakeholder ManagementTechniques
Waterfall methods have been elaborated to effectively manage stakeholders by layering and filtering communication through well-defined channels and hierarchies, which can be critical for project success.
Waterfall techniques for stakeholder management include progress reports, test reports, weekly or monthly management (steering committee) meetings, project plans, and design documents.
13. Advantages of Waterfall over Agile or DevOps
Three advantages of Waterfall present themselves over Agile or DevOps.
Here are some advantages of Waterfall over other methodologies, such as Agile, which are not immediately obvious.
Accessibility
Understanding Waterfall is relatively simple with its straightforward sequential process and stages that make sense in any context. This makes Waterfall methodologies very accessible to new teams or fresh software engineers.
The Waterfall philosophy is founded on easily digestible principles, such as our ability to lay out clear requirements and fully design the perfect software right from the start. This philosophy aligns with most of our Newtonian and deterministic worldviews, making it easy to adopt with little or no friction.
Since Waterfall predated Agile by a few decades, it is well accepted within the software communities. This means adopting Waterfall project management methodologies does not face the same cultural blockers as Agile.
Suitable for large projects
Waterfall project management methods evolved to suit large and complex projects with many dependencies (suppliers, client input and feedback, resource availability) and extended timelines. It has also proved highly efficient and effective in projects with low novelty and client interactions.
Project-level accountability
Because of its hierarchical project structure, well-defined roles and responsibilities, and straightforward process, accountability in Waterfall for project success or failure is much easier to define and apply.
However, it does not work so well on lower levels, where teams of engineers share responsibility over a certain task and where a clear definition of done can be missing.
A word of caution before we conclude this section. First, Waterfall in its original form is rarely effective or efficient today, and the Hybrid model is much more powerful in projects where Waterfall works best. Second, Waterfall is a tool for delivering projects. As such, it is neither good nor bad, and its effectiveness is highly context-specific. These two points have been fairly elaborated in other articles on this website on Operational Excellence and Agile, which we highly recommend you read.
In this article, we have presented an overview and brief history of the Waterfall model. We also went to great lengths to define the Agile philosophy, methodology, framework, methods, and techniques.
The objective was not to offer Waterfall as a recommended practice in today’s world for large software project management but to:
Appreciate the conditions under which it was conceived
Understand the context where it best applies
Objectively compare and contrast it with Agile
Understand where it sits in our framework for Operational Excellence
We hope the interested reader will continue exploring by looking at our other article on Agile and Operational Excellence.
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional
Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional
Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
