Agile vs Waterfall: Finding a Methodology That Works Best for You

Georges Lteif

Georges Lteif

Software Engineer

Last Updated on May 10, 2022.
Subscribe now to stay posted!
About Us
7 min read

1. Overview

Achieving Operational Excellence in your Software Project Deliveries requires that you make a decision on which delivery methodology to use. In fact, it will be the backbone of the ensuing routines and processes.

In the next section, we will see why that is absolutely critical for the success of your business.

Ideally, you have two broad paths to choose from:

  1. Lightweight: Agile, DevOps, Extreme Programming, etc.
  2. Heavyweight: Waterfall, Hybrid

Agile, Waterfall, Hybrid, or DevOps?

If you are unsure of what Agile, Waterfall, and DevOps mean and how they differ from one another, we recommend you have a look at this article first. If you do not feel like it, it’s also alright, we will include links to the relevant paragraphs as we go.

In the coming sections of this article, we will offer a few guidelines that can assist you in making that decision.


2. Table of Contents


2. Software Delivery Methodology: Why Bother

Choosing a project delivery methodology will pretty much dictate every aspect of your development, testing, and delivery processes.

Here are some of the main areas that will be heavily impacted:

ItemSummaryExample
1Technical ToolsIf you are using Agile, you need project tracking tools that support User Stories, Kanban boards, etc. Also, setting up a CI/CD pipeline requires specialized applications/tools as well.
2Day-to-day ActivitiesMeetings (stand-up or otherwise), decision-making, documentation style, and team member roles can differ significantly between the two methodologies.
3Skill SetsSpecifically when being a team lead, being fluent in the processes is quite essential to maintaining rhythm and discipline. Passing the necessary knowledge to the team can be a lot smoother.

Team members also need to be familiar with the automation or integration tools (Jenkins, git, Robot Framework, etc.) if they want to use them as well as their duties and responsibilities within that framework.
4TerminologyThe terminology used between team members and with customers can vary significantly. If you want to avoid any “translation” problems, make sure you are using the correct terms in the right context.
Aspects of project delivery changing with methodologies

But there is more at stake. In fact, the total success of your enterprise could depend on it.

Implementing the right project delivery methodology raises your chances of delivering quality projects, on time and within budget. This means:

  • An edge over competition
  • Happy customers and therefore more business
  • A growing reputation
  • Higher morale within your team

3. Agile vs Waterfall vs DevOps

Before we start, let’s take a moment to look at the three (perhaps four) contenders.

The main question is really whether Waterfall can “put up a fight” against Agile and DevOps in this day and age. If not, perhaps we just eliminate it from the list?

There is a wide acceptance within the community that Waterfall is pretty much DEAD and cannot be used in any setup.

Practitioners, however, who have faced challenges migrating to Agile, have been very pragmatic and created a Hybrid Model.

Viability of the Hybrid Model

The Hybrid Model emerged as a compromise between Waterfall and Agile. It allowed practitioners to leverage some of the very powerful techniques that Agile introduced and retrofit them into a cherished, but seriously outdated, Waterfall model.

IMPORTANT to note that in the coming sections, whenever we state that Agile (or any other lightweight methodology) is not applicable, the Hybrid Model (and not Waterfall) is expected to fill the gap.


4. Guide to Choosing a Software Delivery Methodology

4.1 Topic Fluency

Throughout many of my discussions with colleagues and professionals, I found that impressions of Agile, Waterfall, etc. were somehow different than actual facts on the ground.

The below table summarizes those impressions.

MethodologyPerception
WaterfallStable, reliable, intuitive, but also rigid, old-school, outdated
AgileModern, more hype than substance, fast, silver bullet, difficult, vague, and too complicated.
DevOpsMagic, powerful, complicated, outlandish, and more science-fiction than real.
Community Impressions of the Different Methodologies

Unfortunately, some of that is true while the other is not quite.

To avoid a “holy war” within your team, make sure you have a thorough understanding of the following:

  • The circumstances that lead to the emergence and development of each methodology
  • The problems that each tried to solve (Agile, Waterfall, DevOps), and whether that applies to your setup
  • Implications on your team from using one methodology over the other

The following sections will focus on the major players in this competition between Waterfall, Agile, and DevOps.

We will also attempt to describe the impact of using one methodology over the other. Hopefully, it should help you eliminate the incorrect choices.

4.2 Product Nature

There are two properties of a software product that are particularly interesting for us in this discussion: cost-of-change and iteration-friendliness.

Cost-of-Change

Cost-of-change is a direct measure of how easy (or difficult) it is to change your mind about the design of a certain feature in the different stages of the Software Development Lifecycle (or SDLC).

For Waterfall-based approaches, once you are past the design phase, the cost-of-change increases dramatically as you progress from development to testing to production.

To avoid that, Waterfall-based approaches focus a lot on design and planning, which is very sensible if you have a solid set of requirements. Unfortunately, in the software business, solid and stable requirements can often be a luxury.

To remedy that problem, lightweight methodologies were invented. Their basic mantra is: ship sooner, and more often. This will help you gain valuable information about the usage of the product and the success of its current implementation.

If the measured success is not adequate, perform another iteration and so on.

Iteration-Friendliness

By definition, low cost-of-change products can afford to ship smaller features, more often. This makes them perfect candidates for lightweight methodologies such as Agile and DevOps.

If the features are interdependent, whether by design or simply by virtue of dealing with legacy code, you can expect the cost-of-change to be high, and shipping those features one at a time is just too risky.

In this case, you would need to perform a lot of design work and a thorough impact analysis before any development. This makes Agile-centered approaches less than ideal.

Impact of Cost of Change and Iteration-Friendliness on Methodology

Having explored some of the rationales, we can state a few rules-of-thumb:

Rule 1

High Cost-of-Change

If your projects are inherently high on cost-of-change (such as mission-critical backend systems) and short iterations are impossible, use the Hybrid model.

Rule 2

Low Cost-of-Change

If your projects are low on cost-of-change (websites being one example) and short iterations are possible, Agile or DevOps can be used.

Rule 3

Waterfall and Iteration-Friendly Products

Using Waterfall on iteration-friendly products/projects will introduce an artificially-induced high cost-of-change.

Qualifying for Agile does not mean you can immediately apply it; at least, that’s what the 15th State of Agile Report tells us. There are other compounding factors and these will be the target of the coming sections.

4.3 Organizational Culture

Migrating to Agile or DevOps from Waterfall can be painful.

This is especially so in the case of a well-established organization that has delivered so many successful projects using Waterfall. A difficult cultural transformation would be expected if the migration was to complete successfully.

In fact, Edgar Schein, one of the most influential researchers on Organizational Culture, thoroughly discusses cultural transformation in his monumental book Organizational Culture and Leadership.

The author states that cultural transformation can only be triggered by the appearance of an existential threat combined with enough psychological safety to allow such a traumatic experience to successfully conclude.

These hurdles make two of the top 5 reasons for slow Agile adoption according to the 15th State of Agile Report:

Agile Adoption Challenges
Agile Adoption Challenges

This brings us to the next two rules.

Rule 4

Leadership Support

In case your project qualifies for Agile-driven deliveries, make sure you have adequate support from the organization’s leadership when going through the migration.

Rule 5

Cultural Clashes

A sense of urgency and a supportive culture needs to be developed for major-sized changes to succeed. If these two criteria cannot be met, a safer bet would be the Hybrid model.

4.4 Test Automation

Regardless of which methodology is ultimately used, a proper Test Automation Strategy and Framework must be present to support quicker, more reliable deliveries.

A published test strategy allows your team to understand:

  • How software testing should be conducted
  • How to use the tools and infrastructure (including any automation frameworks) available
  • The industry best-practices that are being followed
  • What artefacts should be produced at each stage
  • The roles and responsibilities of the QA staff
  • The rules of engagement with the developers and the customers
Rule 6

Test Automation

Agile requires tests to be fully automated so that design, development, and quality assurance, could be reasonably expected to complete in a typical sprint of two weeks. DevOps takes that requirement to a whole new level as the whole build, test, and deployment pipelines are required to be automated. The Hybrid model is safer if the conviction, processes, and infrastructure for automation are not available.

5. Final Words

Selecting a software delivery methodology can be daunting and requires thorough knowledge of the organization, its culture, the stimulus for change.

The people managing the change should expect an amount of resistance proportional to the strength of the organization’s culture.

Having said that, quick and immediate gains can be made by:

  1. Creating and publishing a test strategy
  2. Introducing automation where possible
  3. And moving to a Hybrid model if not already there.

Technical Risk Management and Decision Analysis — Introduction and Fundamental Principles

1. Overview I could not find a better way to start an article on Risk and Risk Management than by quoting the opening lines of Donald Lessard and Roger Miller’s 2001 paper that, briefly but lucidly, summarizes the nature of large engineering endeavours. It goes like this: This article leans heavily on three handbooks thatContinue reading “Technical Risk Management and Decision Analysis — Introduction and Fundamental Principles”

Complexity and Complex Systems From Life on Earth to the Universe: A Brief Introduction

1. Overview Dealing with complexity is an integral part of our lives, even if we do not realise it.  An organisation can be modelled as a complex system from the scale of megacorporations right down to the smallest teams. The architecture of software solutions can be equally complicated, and megaprojects and implementations are certainly involved.Continue reading “Complexity and Complex Systems From Life on Earth to the Universe: A Brief Introduction”

Book Review: Programming the Universe — A Quantum Computer Scientist Takes on the Cosmos

Synopsis Most physical theories adopt a mechanistic view when examining natural phenomena where any system can be modelled as a machine whose initial conditions and dynamics govern its future behaviour. In this book, Programming the Universe — A Computer Scientist Takes on the Cosmos, Professor Seth Lloyd proposes a radically different approach centred around aContinue reading “Book Review: Programming the Universe — A Quantum Computer Scientist Takes on the Cosmos”

From Abstract Concepts to Tangible Value: Software Architecture in Modern IT Systems

1. Overview Software design and architecture are two very elusive concepts; even Wikipedia’s entries (ref. architecture, design) are somewhat fuzzy and do not clearly distinguish between the two. The Agile manifesto’s statement on architecture and design is especially brief and raises more questions than answers. The most common definition of software architecture is as follows:Continue reading “From Abstract Concepts to Tangible Value: Software Architecture in Modern IT Systems”

Business Requirements and Stakeholder Management: An Essential Guide to Definition and Application in IT Projects

1. Overview The complexity of business requirements in IT projects has experienced exponential growth due to pressures by increasingly sophisticated client preferences, novel technologies, and fierce competition. Consider, for example, the case of financial payments. In the mid-80s, most payment transactions occurred inside bank branches, and only the biggest banks offered services on ATM orContinue reading “Business Requirements and Stakeholder Management: An Essential Guide to Definition and Application in IT Projects”

Loading…

Something went wrong. Please refresh the page and/or try again.