The Software Development Lifecycle — A Revision of Current Best Practices

Georges Lteif

Georges Lteif

Software Engineer

Last Updated on November 5, 2022.
Subscribe now to stay posted!
About Us
12 min read

1. Overview

We discussed Operational Excellence and what it means for software development in a separate article, which we recommend you first read. In the present one, we will list the practical methods that would allow us to get there.

But before we proceed, some words of caution are in order.

Note 1: Tools alone are never enough

Using the correct tools in the best possible way is essential to any activity. Unfortunately, tools alone can never be enough and must be supplemented with mature processes and a healthy organizational culture. This statement is especially true in technology.

Note 2: This is not a prescription

Operational Excellence principles, as we have described them, are context-specific, and any guidelines or best practices must not be treated as a universal and context-free prescription. The guidelines we propose here must be assessed for every situation and applied where it makes sense.

A developer’s repertoire’s enormous variety of tools, techniques, and project delivery methodologies can complicate technical decisions and be fraught with strategic risk.

Follow the link below for a comparison of delivery methodologies, and see Agile, Waterfall, and DevOps for an in-depth discussion of each topic. Selecting a suitable technology stack was also treated separately and is a recommended read.

Agile or Waterfall?

Not sure if Agile works for you or whether Waterfall is still OK to use? Have a look at this article.

2. The Software Development Lifecycle

2.1 Definition

The Software Development Lifecycle is the cornerstone of large software deliveries and is characterized by the following:

  • It comprises a sequence of stages, each providing the input to the next. These stages can be grouped into roughly three categories: preparatory (requirement gathering, architecture, and high-level design), build (development and testing), and production support (deployment, operations, and maintenance).
  • Although the steps are sequential, iterations might apply, such as when bugs must be fixed, or product features redesigned. 
  • In the very early days of software development, projects were delivered by a small number of people. Early software engineers understood the business and had a working knowledge of computer science (programming languages, coding, algorithms, and IT systems). When projects became bigger and more complex, the need for specializations arose, and distinct technical profiles crystallized: developers, testers, business analysts, support desk, DevOps engineers, project managers, etc.

The Software Development Lifecycle applies perfectly to both flavours of software projects: systems integration and product development.

2.2 Stages

Software Development Lifecycle - SDLC
Software Development Lifecycle – SDLC

In its most elaborate form, it consists of the seven following stages:

2.2.1 Planning

This stage is, historically, not part of the standard representation of the SDLC. However, we have decided to include it because, in our view, activities such as release management, and risk management, which naturally belong to this stage, directly impact the success or failure of your project.

Resource planning, time management, task prioritization, managing multiple workstreams (such as new projects and BAU), and resource allocation constitute the core activities of this stage.

2.2.2 Analysis

In this stage, architects and project managers collaborate to produce a high-level design, architectural documents, effort estimates, and project plans from locked business requirements.

A Statement of Work (SoW) is created and signed off by the relevant stakeholders and project sponsors before any implementation work commences.

2.2.3 Design

During the design stage, a low-level solution design is generated and, in some cases, signed off by the customer.

In a previous article, we have explained in great detail the importance of having a solution design. The low-level design might require additional requirement gathering and technical decision-making.

2.2.4 Development

Developers, testers, and DevOps engineers generate business value to a product by coding new features, adding functional test cases, and keeping the CI/CD infrastructures on par with the delivery requirements.

2.2.5 Verification and Validation

The Verification and Validation stage is called software testing or quality assurance.

Software testing is applied following a test strategy to ensure the product meets the customer’s expectations.

Verification ensures that the specifications are met, while validation ensures that the requirements of the concept of operations are satisfied. Ideally, you want as much automation as possible for maximum efficiency.

2.2.6 Deployment

Deployment used to be a short and rare event, but this has significantly changed with requirements for Continuous Delivery. Megacorporations can deploy thousands of small changes yearly instead of a few major releases for the more traditional and conservative organizations.

2.2.7 Maintenance and Support

Customers require a stable and working solution as their top priority. Immediately after that, they expect excellent customer support.

Exceptional time management, resource allocation, and task prioritization skills are the cornerstones of an efficient help desk and the pathway towards meeting SLAs.

2.3 Tools and Techniques

The techniques and methods employed today in software delivery result from a 70-year-old practice and a vast body of knowledge accumulated over the years and embodied in its production processes.

We can only codify a fraction of this knowledge in web articles and an even smaller percentage in one or two paragraphs.

The table below does not exhaustively describe the tools, techniques, and best practices employed at every stage of the SDLC. It will, however, list the essential ones with links to further articles that the reader is welcome to go through.

StageTechniques to Achieve the Best Results
1. PlanningWhile planning and allocating resources, delivery managers ideally aim for the below targets:

A) To create self-sufficient teams, as these can perform better if they do not need to go outside their group for approval or advice.

B) To keep the group together during future projects (as much as possible) to leverage their shared experiences and mature relationships.

C) Spread the load for optimal resource allocation so that projects can be delivered without sacrificing quality to meet demanding deadlines. Optimal resource allocation also means that resources do not idle when work is scarce.

D) Manage project risk throughout the implementation.
2. AnalysisTwo criteria must be met for this stage to be successful:

A) First, the business requirements must be clartified and documented. Remember, unclear requirements is one of the top reasons for IT project failures.

B) Second, effort estimation must be precise so that the project remains profitable and stakeholder expectations are well managed.
3. DesignAll architectural documents are produced at this stage describing the concept of operations, and main modules and functions of the systems and subsystems.

Interface design occurs at this stage as well, crucial for both development and systems integration projects.

A Solution Design is completed at different levels for different purposes. A High-Level Design (HLD) gives a bird’s eye view of the solution for major stakeholders, while a low-level solution design (LLD) is a great place to document all the implementation details.
4. DevelopmentSuccessful developers produce clean code with a unique coding style, look after software architecture, and use Test-Driven Development (TDD) as the backbone of their development processes.

Development should also investigate how it uses Unit Tests to understand its benefits and pitfalls.

Design and development are the primary produces of business value, while other stages are auxiliary and serve only to support the development effort at an extra cost.
5. TestingSoftware Testing is always a great place to start when looking for ways to optimize your delivery. Two significant factors immediately come to mind: automation and Test-Driven Development.

Test automation significantly improves your delivery capabilities and boosts the quality of your testing. Without automation, Agile and DevOps are very difficult, if not impossible, to implement. Automating repetitive testing activities also gives your QA staff more opportunities for growth.

Test-Driven Development, or TDD, is an excellent tool for streamlining your activities and allowing developers and testers to work in parallel. To realize its full potential, it must be implemented with automation, unit testing (as space as possible), and Systems Integration Testing.
6. DeploymentDeployment to production has moved to a new level with continuous delivery, automated deployment, and frequent releases.

The infrastructure that supports automated and advanced deployment methods (both tools and processes) falls under DevOps.

DevOps practices allow the fusion of development and operations in a single, streamlined process.
7. SupportTriage, time management, SLA awareness, and customer satisfaction are the hallmark of customer support.
Tools, techniques, and best practices used in the different stages of the Software Development Lifecycle

3. Managing and Improving Development Processes and Lifecycle

The below guidelines (much like everything else in this article) will work as long as no substantial shift occurs in context. They also work best in ordered systems (both simple and complicated), but their effectiveness degrades significantly in complex systems.

3.1 Process Standardization

Employees sometimes view standardization as a means for curbing creativity, stifling innovation, limiting personal initiatives, and imposing an alien organizational culture.

While that statement may not be entirely untrue, it is still essential to standardize your production processes across the organization to guarantee minimum variations in outcomes and ensure improvements are applied across the board.

What is essential in the context of standardization is that you pick methods (or better still, pick time-proven best practices), and use them consistently. You can always run continuous improvement exercises on those methods for optimal results.

If you create bespoke processes for your organization, there will be significant implications. While a customized standard of operations may work better for you in the long run, its initial cost can be pretty high.

Generally speaking, and unless you have the size and influence of Google or Tesla, it is recommended that you communicate via a language that everybody can understand.

3.2 Reducing Non-Value-Adding Work

Taiichi Ohno implemented the following technique for eliminating waste on the shop floor of the Toyota production plants to an astounding degree of perfection.

The process is as follows:

1. ObservationSpend as much time as possible observing the production processes, preferably with experienced staff.
2. Make a ListList all the repetitive steps required to complete a particular task.
3. MappingFind out which ones produce value and which ones don’t.
4. Identify Non-Value-Adding ActivitiesWork with the team members to identify non-value-adding steps such as overhead, rework, or enabling activities.
5. Eliminate Non-Value-Adding ActivitiesEnhance internal processes in collaboration with team members to eliminate all forms of waste

This five-step process should not be confused with scientific management (a.k.a. Taylorism). Scientific management, proposed by Frederick Taylor in the 1880s, was likely to be counter-productive as employees found ways to game the system.

This method is a radical deviation from the statistical analysis of data collected remotely and analysed behind a comfortable desk. Its advantages include gathering firsthand information as seen through experienced eyes. This method is known in Toyota as Genchi Genbutsu.

3.3 Team Structure and Performance

Although it would not make much sense to have a prescription for team size and profiles for its members, some rules of thumb apply in non-complex systems.

Martin Fowler, one of the 17 founders of Agile, describes the most performant team as follows:

Keep the team size small for maximum communication and efficiency. Ideally, the golden number is less than 10 people.
Martin Fowler

Another magic number seems to be seven, if we are to believe the results of a paper published in 1956 by George Miller. From experience, managing the interactions of any more than seven people overwhelms our cognitive abilities and things will start to slip.

Eric Schmidt and Jonathan Rosenberg published a book in 2014 called How Google Works. In this book, they stated that the maximum number of people a single person can efficiently manage is around seven.

Use the same team across multiple projects so that they could benefit from their shared knowledge of the application.
How Google Works

In the same book, How Google Works, Google focuses on getting teams to work in smaller spaces, without physical barriers, and encourages relationship building. They also have a specific recruitment process that ensures only the right people get on board.

At Google, they strive to hire interesting people. The criterium is as follows: if you get stuck in an airport with a colleague, would you be able to find interesting topics for discussion?
How Google Works

Let’s now turn to the Agile manifesto for more inspiration.

The best architectures, and designs emerge from self-organizing teams.

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

A not on team building exercises.

It is undeniable that kayaking with the team always positively impacts the mood. However, I am convinced that team-building exercises alone are never sufficient (or even remotely relevant) to improve the team’s performance.

Emotionally charged traumatic experiences dwarf the positive ones; unfortunately, our brains weigh experiences very differently.

Vital as it may be, such team-building exercises should only be an addendum to the real ones:

  1. Valuing employees’ contributions — However, not through competition-inducing practices such as “Employee of the month”, but through genuine appreciation of the individual’s contribution.
  2. Maintaining a safe work environment — A healthy organizational culture is the main pillar on which this would rest.
  3. Offering employees the opportunity to grow professionally and as individuals.

3.4 Tracking Progress

The ideas in of this section are probably not the most popular as they go against some of the more modern management styles that have been trending for a while.

Have a look at the below table to compare and contrast.

Traditional MethodProposed Method
Heavily dependent on lengthy reports and second-hand information to understand the situation on the ground.Go and see for yourself. Use your long experience and trained eyes to understand the processes and spot bottlenecks and inefficiencies.
Uses complex statistics and KPIs such as Six Sigma, and Taylor’s scientific management methods to monitor and improve performance.Robust, mature, time-proven, and simple heuristics focus more on outcomes than results.
Unfounded optimism and positive attitude. Ignore negative indicators.Use visual aids to signal quality problems. Stop and fix the problem before it snowballs.
Treat the symptoms as opposed to the root cause. Has a “not my problem” attitude.Ask “Why” five times until you get to the root cause of the problem. Openly discuss any issue and fix it.
Traditional vs Proposed Progress and Performance Tracking Methods

3.5 Continuous Improvment

As the organization grows and pressure for faster and better deliveries increases, people will find less time to improve what they do. In his timeless classic The 7 Habits of Highly Effective People, Stephen Covey describes this phenomenon as “sharpening the saw”.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

Even if individual initiatives were made here and there, these would continue to be local, spurious improvements and cease to suffice.

To maintain software delivery in top shape, continuous improvement methods must be ingrained in the DNA of the system. In addition to that, they also need to be amalgamated into a single, system-wide activity.

Scrum uses retrospectives after each sprint. The double activity of Kaizen and Hansei is slightly different, but the objective is the same.

Such exercises are important for morale; they carry a positive message to the staff that the company is committed to the continuous improvement of its people and processes, and it’s not just about the present but also the future.

4. Responding to Change

4.1 Technology Lifecycles

The pace at which software or any other technology-based industry changes is breathtaking. Sometimes, that leaves us with a feeling of helplessness that’s difficult to ignore.

The question then becomes, how do we, as professionals in the software industry, keep producing premium-quality products despite the ever-changing nature of technology, tools, techniques, and methods that come with it?

It might help to look at the software industry’s evolution not as a homogenous, monolithic entity but as four major sub-fields, each characterised by a predetermined lifetime.

ToolsProgramming languages, frameworks, and database engines.< 10 years
Delivery Methods, Customer PreferencesProject management techniques, project delivery methodologies (Agile, Waterfall, DevOps), business needs (eCommerce, payment methods, IoT, smartwatches, cloud technology…)10-20 years
Tech ConceptsPersonal computers, quantum computing, digitization, internet, smartphones, AI…)20-30 years
Business ManagementTaylorism, The Toyota Way, Lean Manufacturing, Six Sigma, Operational Excellence30+ years
The Lifetime of Technology Sub-fields

The lifetime in this context does not represent an expiration, but the average duration before a newer concept arises and challenges the status quo.

Like any other field in science or technology, historical progress seems to happen in jumps, followed by plateaus, rather than monotonic, incremental improvement.

4.2 Measured Responses to Disruptions

What would be an adequate response from your organization for the above four disruption levels? Below are some thoughts.

Level 1: Short-Term Fluctuations

Level 1 entries appear and disappear every day. Use reliable technology, always! The gravest mistake you can commit would be trying to catch up with their destructive pace.

Level 2: Big Changes

Level 2 entries are technological trends that change dramatically every decade or two. Consider the emergence of digital payments, for example. While people still use cash nowadays, but would also like to take advantage of the convenience of mobile phones or watches.

Despite embracing those new methods is not strictly vital, not doing so places your business in a less-than-ideal situation vis-a-vis competition.

Level 3: Major Disruptions

Looking at entries from Level 3, we immediately recognize game-changing technological revolutions that put so many giants out of business overnight.

Those past giants have failed to reinvent their value proposition and were replaced swiftly with newcomers who leveraged new technologies to offer innovative solutions.

Level 4: Paradigm Shifts

It is somehow fortunate that the more potent concepts of Level 4, those that make a business successful or otherwise, change extremely slowly. There is more than enough time for everyone to master them.

These occur as part of larger sociological, economic, or technological revolutions.

Because Level 4 fields, such as Operational Excellence and Complexity Theory, sit at the top of the hierarchy as one of the lot’s most stable concepts, ideas, and mental constructs, it makes sense to thoroughly consider them and invest time in applying and mastering it.

5. Final Words

Software development is a field where technologies, disciplines, practices, and methodologies abound, and discussions on which to adopt can quickly turn into a religious war (evidence of which can be seen in rituals, two-day Agile coaching programs, and coloured belts).

This article proposes a few principles that could help you make the right choices and firmly anchor your decisions on solid ground. We hope that, in that sense, it has achieved its objective.

I believe that doing something great is always gratifying and provides deep self-satisfaction.

Success also demands technical expertise (from practice and field experience) and deep knowledge (from reading the literature), and both are within reach if due effort is invested.

Leave a Reply