Agile vs Waterfall: A Practical Evaluation of Current Software Delivery Methodologies

Georges Lteif

Georges Lteif

Software Engineer

Last Updated on May 10, 2022.
Subscribe now to stay posted!
About Us
17 min read

1. Overview

The Agile vs Waterfall discussion remains relevant today even after almost two decades of the announcement of the Agile manifesto.

We know that Agile adoption has been partial and slow, and the 15th State of Agile Report goes into quite some detail on that. If you are still undecided on which methodology is the most suitable, you have come to the right place.

Project delivery can be divided into two categories: lightweight, which includes Agile and Extreme-Programming among others, and heavyweight such as Waterfall.

waterfall vs agile
Agile or Waterfall?

Although we recommend adopting Agile practices as one of the principles of Operational Excellence in software development, we do, however, first recommend that you gain fluency in the topic.

Agile and Waterfall are built on two different sets of core principles. While Waterfall methodology is pretty intuitive, Agile may not be as much and its fundamental ideas need a bit of unpacking.

The purpose of this article is to do precisely that; give the reader enough information on both methodologies to allow for a wise decision.

We have also published a guide on choosing the best project delivery methodology for your team. We recommend you have a look once you are done with this one.

This article will compare and contrast three models: Agile, Waterfall, and a Hybrid version, and introduce a fourth one (DevOps). We will also explain some of the implementation details of Agile for those looking to migrate.


2. Table Of Contents


3. Selecting a Methodology

3.1 Why It Is Important to Make the Right Choice

It is essential to recognize that Agile is not a silver bullet nor the default choice when unsure. Make a careful choice for the below reasons.

First, choosing an unsuitable delivery method will have the opposite effect of what you desire: it might aggravate your team and introduce unnecessary delays in your projects.

Second, implementing the correct project delivery methodology will help you:

  1. Deliver faster, higher quality products
  2. Achieve Operational Excellence
  3. Provide a solid foundation on which you can design and build efficient production processes
  4. Speak a suitable, familiar language with partners, suppliers, and customers

Finally, it would be easier to improve on a working model than a broken one.

3.2 Slow Adoption of Agile

According to the 15th State of Agile Report, 94% of the participants reported using Agile practices in their organization. That’s, however, not the whole story. Have a look at the below stats from the same survey. Only 32% however reported using Agile for more than 5 years.

agile adoption 2021
State of Agile 2021

Out of the 94% using Agile, 68% have been using it for less < 5 years. We believe this fact requires some explanation, especially since Agile has been around since 2000! One of the central themes of this article is finding the root causes behind that phenomenon.


4. Waterfall vs Agile FAQ

If you would like some quick answers.


5. Waterfall Methodology

5.1 What is Waterfall

Waterfall was first described in a presentation at the Symposium on Advanced Programming Methods for Digital Computers on 29 June 1956 by Herbert D. Benington (Wikipedia).

Waterfall consists of breaking down all project tasks and activities into a pipeline of sequential stages where each stage depends on the output from its predecessor.

Later on, in 1970, a paper published by Winston Royce on Managing the Development of Large Software Systems gave Waterfall its current fame and stature. The term Waterfall was never used until it appeared in a 1976 paper by Bell and Thayer.

5.2 How Does Waterfall Work

The Waterfall model is a series of sequential stages where each stage of the pipeline specializes in one area of the software development lifecycle.

The stages of Waterfall typically are Initiation, Analysis, Design, Development, Testing, and Deployment.

waterfall stages
Waterfall Stages and Documents Produced

In a properly setup process chain, each stage in the SDLC has an owner responsible for executing a set of transformations on inputs it has received to produce artifacts, which will serve as inputs to the next stage.

The table below describes the five stages and the deliverables produced in each of them.

Waterfall Project PhaseDeliverables
Project InitiationProject Initiation Document: defines the project sponsors,
pre-requisites, overall scope.

Business Requirements Document: details the business requirements to be met.

SLA or Service Level Agreement.
AnalysisSoftware Orders of Magnitude: a list of high-level features (or Epics) and a rough order of magnitude of their efforts.

High-Level Design: represents a bird’s eye view of what the solution looks like.

Solution Design: This is a blueprint of the system to be implemented.

Detailed Project Plan: includes all the detailed tasks required for successful delivery and a detailed effort estimation

Gap Analysis: usually conducted to understand the differences between what the system offers off-the-shelf and
what the end result looks like.

Statement of Work: describes the scope, deliverables, timeframes, prerequisites and milestones.

Test Strategy: a high-level plan of how the new releases will be tested. More of a guideline to testing than actual test cases at this stage. Discusses test approaches, test environments, and resourcing.
DevelopmentSource Code: new version of the source code

Test Scripts: the set of test cases to be used for testing the changes. These also include any Test Automation scripts or unit test scripts in case Test-Driven Development is used.

User Guides and Reference Manuals: can also be produced at this stage.
Testing and QATest Cases: an actual test case with summaries, acceptance criteria, steps to produce, and test parameters.

Test Report: a report of the test runs with details on failed cases.
Deployment and SupportRollout Plan: a step-by-step plan on how production deployment will happen. Typically with a duration for each step. This is especially important if downtime is involved.

Installation, Admin, and Operations Guides: these are the manuals required to run the system in production mode.

DevOps Script: if DevOps practices are used for building and deploying applications.
Key Deliverables of Waterfall Stages

5.3 Key Principles Behind Waterfall

Waterfall project management, when compared to lightweight methodologies like Agile, presents two very distinct features.

The first feature is massive planning in the form of design and documentation, and the other is using a single sequential pipeline for delivery.

Design vs Development Efforts in Waterfall
Design vs Development Efforts in Waterfall

The most significant chunks of work in the delivery pipeline are usually design and development.

Waterfall emphasizes design activities to limit the scope and impact of any changes and therefore reduce the size of testing to the bare minimum. Most experts recommend a ratio of 70/30 for design and development.

The downside of this approach is that planning and design happen only at the beginning of the project, and revisiting the plans at any later stage is very costly.

To mitigate the risk of any rework, Waterfall tries to get it right the first time.

5.4 Advantages of Using Waterfall

Despite its core weaknesses, Waterfall has many advantages:

  1. Waterfall runs on an intuitive model that is easy to explain and understand. The sequential nature also makes it simple to manage.
  1. A fixed structure with clear milestones, checkpoints, and deliverables requires a straightforward implementation process with basic management techniques and little hand-shaking and coordination between different teams.
  1. A fixed project scope reduces the overhead associated with rescoping, redrafting documents and project plans, and acquiring approvals from senior management.
  1. Generous documentation. No one generally complains about having proper and quality documentation.

These advantages, however, are not enough to justify using Waterfall. A Hybrid model can have all the benefits and very few of the disadvantages of Waterfall. It also combines the good features of Agile to make it even more appealing. We will have a few words to say on Hybrid models later on.

5.5 When Is Waterfall Inappropriate

Waterfall project management works great if the following criteria are met: 

  1. The requirements are well known at project inception. The chances of conditions changing later on in the project are minimal.
  1. The customer can see the end result and sign it off before development starts so that the chances of developers building unwanted or unusable features are tiny.
  1. The technology used must be mature and well-understood. Any issues of a technical nature are quickly resolved. Using only mature technology is one of the principles of Operational Excellence.

Points 1 and 2 are usually challenging to satisfy. Here are some reasons why that may be true:

  1. In some cases, such as building a website, the customer may not know what they want until they see it.
  1. Using websites again as an example, you may not be able to gauge the user experience until you deploy on production. This weakness is ubiquitous for issues related to the look-and-feel of websites.
  1. For sufficiently big projects, customer priorities might change. Although not as likely as points 1 and 2, this is still possible.

The below table lists a few real-world examples of scenarios where Waterfall cannot cope.

ScenarioExample
Unknown or Unclear Requirements

In these cases, the requirements can never be fully understood at the beginning of a project.
1) Migrating legacy, poorly-documented systems are an example where requirements can never be fully known before testing.

2) Performance requirements for applications that outperform expectations.

3) Websites where users’ behaviour patterns cannot be predicted.

4) Innovative products yet to be understood.
Lengthy Projects / Changing Requirements

If the duration of the project is extended, chances are that some requirements might change.
1) Regulatory updates in financial products are one example of requirements changing during the project.

2) Change in client priorities or business strategy.
New, Poorly-Understood Technologies

The team is not fluent in the technology used.
1) Supporting new OS platforms or web framework

2) Deployment mode changes like offering the product as a service
Waterfall Shortcomings

The below caricature is the best explanation of what usually happens.

it project failures

6. Agile Methodology

6.1 What is Agile Project Management

In 2001, seventeen software developers convened at a resort in Snowbird, Utah, to discuss lightweight (in contrast to heavyweight, collectively known as Waterfall) development methods.

Together they published the Manifesto for Agile Software Development which consisted of 12 principles that embodied the essence of Agile.

The below list shows five of these principles where the deviation from Waterfall is most prominent:

Principle 1

Changing Requirements

Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

The experts who convened in Utah acknowledged changing requirements as an integral part of software development.

As such, changing requirements was something that software developers needed to work with rather than try to eliminate.

The question moved from: How do we ensure we have the final requirements to let’s adjust our processes so that changes, even those that come at the later stages of a project, are not too disruptive.


Principle 2

Frequent Delivery

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference for the shorter timescale.


Agile heavily favours shorter delivery timeframes, and for good reasons. Because Agile is designed to handle changing requirements better, the idea of releasing lighter features more frequently allows issues to surface faster.

Also, minor changes are less likely to be buggy and easier to troubleshoot. This philosophy of Agile is much more efficient from a time management perspective as it allows valuable feedback as early as possible.


Principle 3

Collaboration

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.


Agile emphasizes face-to-face interactions over lengthy and time-consuming reports of low-quality information. This suggestion does not say that investing in design documents or other helpful documentation goes against Agile principles.


Principle 4

Self-Organizing Teams

The best architectures, requirements, and designs emerge from self-organizing teams.


We have discussed self-organizing teams at length in our article on decision-making processes. Be sure to check it if interested.

For our current discussion, suffice to say that a self-organizing team leverages the synergistic relationships from group interactions. Back-and-forth discussions where valuable opinions contribute to the design and architecture of the software are highly recommended for Agile practitioners.

The natural and organic manner in which self-organizing teams evolve is highly effective in dealing with changing environments.


Principle 5

Continuous Improvement

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behaviour accordingly.


Continuous improvement is a powerful discipline that helps keep your business afloat in a changing environment.

Agile requires a constant reflection on past sprints or projects in the form of retrospectives, while Waterfall does not address any such requirement.

Continuous improvement, however, is not specific to Agile but is just something to which Agile has given some attention.

6.2 Agile Frameworks

Software development experts have created numerous practices and techniques in the spirit of Agile. Some of these ideas even predate the Agile proclamation:

Being Agile does not necessarily require the adoption and implementation of all of these techniques. Some of them, however, like User Stories and Backlogs, are so powerful and straightforward that they have become a symbol of Agile.

6.3 What Is the Key Principle of Agile

A robust and straightforward idea lies behind the Agile way of work. Understanding this idea will help you understand what Agile is meant to be and how it should be used for maximum efficiency.

Agile (or any iterative system with a feedback loop) operates by following the below steps:

  • Step 1: Determine your current position with respect to your ultimate objective.
  • Step 2: Adjust your direction so that it points to your goal.
  • Step 3: Make a small incremental step in the new direction.
  • Step 4: Repeat from Step 1

This approach to problem-solving is ubiquitous in almost every area of our daily lives.

For example, think of technology and engineering, where small progress is made through massive iterations and continuous improvement on existing models:

  • Neural Networks use a similar “algorithm” called back-propagation to learn from historical data (or, more formally, to minimize an error function).
  • Moving robots use similar methods to calculate the next thrust of their motors.
  • The Product Kata framework in Product Management is centred around feedback loops and iterative product improvement cycles.

6.4 Understanding Cost-of-Change

We cannot honestly conclude any discussion on Agile and Waterfall without explaining the concept of cost-of-change.

Because of how software development works, i.e. in sequential stages where each stage depends on the previous one, redesigning a particular feature usually incurs a high cost. Experts place that cost at about ten orders of magnitude above the original one.

The price you usually pay to redesign and redevelop a feature is called cost-of-change.

The best way to keep the cost-of-change low is to try and get it right the first time. Have a look at the below graph.

Cost-of-Change as a Function of SDLC Stage
Cost-of-Change as a Function of SDLC Stage

Experts believe that cost-of-change rises exponentially with every stage of Waterfall.

Agile and other lightweight methodologies solve this problem by breaking major changes into smaller pieces, shipping features faster into production and relying on early user feedback.

However, there are some types of projects that are high on cost-of-change by natureregardless of whether lightweight methods are used or not. Think of the construction project of a high-rise building where you cannot prototype the foundations or use iterative procedures to find the best design; you need to get it right the first time.

One example of such projects is those that require certification by regulatory authorities. Often, these certifications are lengthy and expensive, so people try to do less of them.

6.5 When Is Agile Suitable to Use

Agile is appropriate in the following scenarios:

  • The cost-of-change is low.
  • The requirements are not or cannot be fully known before implementation starts.
  • Chances are that the requirements will change during the implementation.
  • A lot of the testing has been automated.

6.6 Agile and Test Automation

In a typical Agile sprint, you need to design, develop, and test a new feature in about two weeks.

However, you cannot even commit these changes to your master branch without test automation and decent coverage. At least, not as frequently as you like. 

There needs to be a complete and manual regression cycle for every change. Lack of a decent Test Automation Framework will cripple the best Agile practices.


7. The Hybrid Model

7.1 Is There a Third Option?

Hopefully, the Agile discussions in the previous sections have convinced you that Agile is not the silver bullet we are told it is. If this is true, and indeed it is, is Waterfall still an option? Is there a different option?

The answer to this question is Yes. There is a third option that combines some of the great features of Waterfall and Agile and merges them into a Hybrid model.

The Hybrid model will be the subject of our subsequent discussions.

7.2 What Is the Hybrid Model

The Hybrid model is sometimes referred to as Wagile, Waterfall-Agile, or Wet Agile.

Hybrid-model
Hybrid Model

The Hybrid model is classic Waterfall retrofitted with powerful and non-obtrusive Agile practices such as relatively more frequent software drops, daily stand-ups, showcases, Kanban boards, story points, and automated testing.

Perhaps the most notable thing to remember is that Hybrid models are Waterfall at their core. Four of the five principles (that is all except Collaboration) that make Agile so different from Waterfall and that have been elucidated earlier remain fundamentally outside the scope of Hybrid models.

7.3 Advantages of the Hybrid Model

Following are some of the advantages of the Hybrid model with which I have personally had some first-hand experience.

One

Similarity to Waterfall

The Hybrid model, at its core, is equivalent to Waterfall

This statement means that software developers, already familiar with Waterfall since the 1980s, do not need to undergo a paradigm shift to use the Hybrid model.

Two

Smooth Migration

Moving from the classic Waterfall to the Hybrid model is not disruptive.

You can always start with a classic Waterfall Model and, at a comfortable pace, start adding Agile practices that are compatible with your process flow.

The Hybrid model can be introduced seamlessly into current development processes with little resistance. What happens is, when people experience the prowess of Agile practices, they find it extremely difficult to go back to the old ways.

Three

Resilience to Uncertainty

Although Waterfall at its core, the Hybrid Model has some resilience to changing requirements.

Agile practices such as TDDtest automation, software drops, and showcasing allow software teams to uncover problems early on and catch up with damaging delays by speeding up the testing.


8. Agile vs Waterfall Comparison Table

The below table presents a comparative analysis of the Agile vs Waterfall vs the Hybrid model we discussed earlier.

The table performs a comparison along six different dimensions: core principles, processes, implementation difficulty, objectives, and pros and cons, and necessary ingredients for success.

WaterfallAgileHybrid
Core PrinciplesDesign to development effort ratio of 70/30Continuous Improvement, Quality, Collaboration, Changing RequirementsAllows large design efforts, embraces continuous improvement and collaboration
Work ProcessSequential stagesIterative methods with quick feedback. Each cycle (or sprint) has design, development, and testing amalgamated together.Sequential stages with quick feedback
ObjectiveDeliver complex software projectsDeliver complex projects while allowing for uncertaintyAn improved Waterfall-based model with some resilience to uncertainty
ProsEasy to explain. Great for high cost-of-change projects.Improves efficiency. Allows for requirements to change.Intuitive. Combines the best of Waterfall and Agile
ConsIntroduces high cost-of-change. Is not flexible when it comes to changing requirementsRequires a lot of discipline. Not very intuitive.Is not a full replacement for Agile
Difficult to Apply?NoYesSomehow
Cost-of-ChangeHighLowMedium
Requires Test AutomationRecommendedCrucialMandatory
Agile vs Waterfall Comparison Table

9. Waterfall vs Agile vs DevOps

Before safely closing this discussion, we need to say a few words about DevOps.

DevOps is a set of practices (tools and processes) that allows you to achieve continuous delivery, a topic worthy of its own discussion.

For the moment being, suffice to say that DevOps takes Agile two steps further; one step towards fusing Development and Operations, the other towards Continuous Delivery.

DevOps requires a certain level of maturity in automation, not just test automation but also many other levels. It also requires a significant shift in mindset and a big investment in infrastructure, skills, and resources.

There is no harm, though, from looking at DevOps and what it brings to the table.

More importantly, you might want to find out whether there are any low-hanging fruits that you can benefit from immediately. There are indeed some tools and practices that you can apply directly and at a minimal cost, and the most promising of these is build and test automation.


10. Implementing Agile

If you decide that Agile is the right choice for you, remember the following items:

10.1 Topic Fluency

If you have come so far, you must have realized how complex this topic can be. Try not to take things for granted. Verify the details for yourself. These pieces of advice will help you make better decisions and avoid costly mistakes.

10.2 Appreciating the Implications

Understanding the initial setup effort, long-term implications, and the technical skills required to apply each methodology before committing to one path or another is essential.

Appreciating the potential difficulties that may arise from organizational culture resistance to change and lack of sponsorship from senior management is crucial if you want to succeed.

10.3 Publish an Operations Guide

Once you have decided on a project methodology, it’s always a great idea to internally publish an operation guide. 

This document will help people understand the big picture, including their roles and responsibilities. It will also give people a solid reference when in doubt and prevent improvisation during implementation.

Most importantly, a published document pushes everybody to comply and sets the processes for future improvements. Without standardization, improvement is impossible.


11. Further Readings

  • Great talk by Robert (Uncle Bob) Martin.
  • Extreme form of Agile in the intro to this lecture by Allen Holub.

DevOps: Finding Your Path to Successful and Continuous Improvement

1. Overview Looking at the below graph from Google Search Trends, we can see that DevOps has picked up much momentum in the last five years and is steadily increasing, making it interesting to understand more about it. The graph compares DevOps search trends with Agile and Waterfall, both software project delivery methodologies. This categorizationContinue reading “DevOps: Finding Your Path to Successful and Continuous Improvement”

Solution Design and Its Role in Successful Projects

1. Overview From the 1970s onward, IT software projects acquired a notorious amount of complexity due to the advancement of hardware (exponential rise in processing power of the personal computer) and software technologies (proliferation of programming languages and frameworks). These advancements captured the user’s interest and radically changed their preferences toward more modernization and digitalization.Continue reading “Solution Design and Its Role in Successful Projects”

Software Estimation: How to Get It Right the First Time

1. Overview Many IT professionals believe that estimating software with arbitrary precision is near impossible. And that’s, in fact, very true. Difficult as it is, estimating software development tasks is a necessary evil, and you need to get it right, at least within a small, consistent margin of error. In effect, unreliable estimation is cited as oneContinue reading “Software Estimation: How to Get It Right the First Time”

Test and Automation Strategy: A Deep-Dive Into an Essential Solution for Your Daily Agile Practices

1. Overview Software Testing and Quality Assurance is an attractive topic from a software delivery perspective for a very simple reason: it’s an area where you can make substantial gains and improvements in delivery capabilities. Why? Because of the current state of software testing, a topic we have analysed in great detail. In summary, softwareContinue reading “Test and Automation Strategy: A Deep-Dive Into an Essential Solution for Your Daily Agile Practices”

Customer Support: How to Drive Efficiency and Satisfaction

1. Overview Post-production support is the final stage in the value chain of your software delivery processes. Keeping the customer happy can be challenging but its also rewarding as its one step further towards achieving Operational Excellence. In this article, we will go over some practical thoughts on how to improve your day-to-day operations andContinue reading “Customer Support: How to Drive Efficiency and Satisfaction”

FAQ – Software Development and Delivery

Project and Delivery Management What is… Agile Project Management Waterfall Project Management Technical Debt Continuous Delivery Continuous Integration Continuous Deployment DevOps Missing in Agile and Waterfall Self-Organizing Team Cost-of-Change Hybrid Model for Project Management Should you… Start Using DevOps Keep Using Waterfall Use a Hybrid Model How does… Agile Work Waterfall Work How to… ChooseContinue reading “FAQ – Software Development and Delivery”