Thoughts on Six Sigma for Developing Your Software Engineering Processes

Georges Lteif

Georges Lteif

Software Engineer

Last Updated on May 10, 2022.
Subscribe now to stay posted!
About Us
11 min read

1. Overview

I have recently taken a lot of interest in the behaviour of groups trying to achieve a common objective, especially those in professional environments.

I wanted mainly to understand the differences between two arbitrary groups when it came to performance.

Specifically, I was keen on knowing whether there was something that those groups can do to dramatically increase their efficiency and lower their risks during that journey towards their common goal.

That search finally led me to two different, but closely related, business management models: The Toyota Way and Six Sigma ().

A fantastic source for understanding The Toyota Way is Jeffery Liker’s book (full review) on the subject.

The book was a great inspiration for putting together a framework under which software delivery can efficiently operate. This was mainly, in my view, due to the universality of its principles.

That framework is what we refer to in this blog as Operational Excellence.

Then, I picked up The Six Sigma Way book (full review) by By Peter S. Pande, Robert P. Neuman, Roland R. Cavanagh.

I have to admit the book itself was quite disappointing. Nevertheless, it provided a good understanding of the foundational elements of Six Sigma, as well as their limitations.

I was also able to form an opinion on whether some of the Six Sigma concepts are universal enough to be applicable in other fields such as service delivery.

This topic, the applicability of Six Sigma concepts in services will be the centrepiece of this discussion.

We will especially focus our analysis on the software development side of the services industry.

2. Table of Contents

3. What Is Six Sigma?

To fully grasp what Six Sigma is, we need to discuss four concepts:

  1. Variation of a probability distribution in production processes
  2. Closed-loop systems
  3. Continuous Improvment of production processes
  4. DMAIC

3.1 Production Variation

The term sigma (referenced by the Greek letter σ) comes from statistics and is used to denote the variance of a certain probability distribution.

A ubiquitous probability distribution is the Normal (or Gaussian) distribution which has the familiar bell-shaped curve.

The below figure shows three normal distributions, with means 0, 1.75, -1.75, and standard deviation 1.

Six Sigma and Quality Variations
Six Sigma and Quality Variations

The variation in the quality of manufactured items is assumed to follow a normal distribution and the aim of quality control measures is to keep these variations under control.

In scenarios where the probability of observing a defect follows a normal distribution, an observation that is 6 standard deviations away from the mean is of the order of 1 in a billion, a vanishingly small amount.

So why 6 sigmas and not, say 3 or 4?

The rationale is quite technical but here is the gist of it: Six standard deviations is broad enough so that even if our quality controls relax a bit, and we shift the mean by 1.5 standard deviations to the left or to the right, we have enough buffer so that the chance of observing a defect from such processes is 3.4 per million opportunities (or DPMO).

A DPMO of 3.4 can be a relative measure. For example, a 3.4 DPMO in transistors on a chip (which are in the order of billions per chip) means that every chip is defective.

On the other hand, a DPMO of 3.4 is a fantastic number in manufacturing car parts.

3.2 Closed-Loop Systems

So, in broad terms, the pan-ultimate goal of Six Sigma is to achieve a certain level of quality such that the number of defects per million is less than 3.4. Or is it?

This number is astonishingly ambitious.

I am not sure if any organization, in the manufacturing business or otherwise, was able to hit that mark on any of its processes for an extended period of time. Certainly not all of its processes.

There are however a few success stories that companies like GE, Motorola, Johnson & Johnson, and Texas Instruments have reported and publicized.

Those adopters of Six Sigma claimed to have saved hundreds of millions of dollars by following this process.

Closed Loop System with Inputs, Outputs, and a Feedback Loop
Closed Loop System with Inputs, Outputs, and a Feedback Loop

So what does Six Sigma actually mean?

I will quote the following definition, which I found very useful, from the Six Sigma Way book mentioned earlier:

Six Sigma is based in large measure on creating a closed-loop-business system that is sensitive enough to reduce the company’s “wobbling” and keep it safely on the often-twisted path to performance and success.
The Six Sigma Way

A closed-loop system is well-known in system engineering. It is basically made up of four elements:

  1. Production processes
  2. That consume inputs
  3. And produce outputs
  4. As well as a feedback loop

This feedback loop carries information on the deviation of the real output from the desired target value and helps the system keep track of its objectives.

The closed-loop system, through this design, is able to self-correct itself.

3.3 Process Improvement

We have seen that the objective of Operational Excellence is to eliminate non-value adding effort (or waste) and increase the efficiency of the production processes while promoting a healthy and safe work environment.

So what do companies want to achieve with Six Sigma?

Companies that apply Six Sigma want to reduce destructive variations in the quality of their products. And not just that, but they also want to do it for extended periods of time and in a sustainable manner.

For that to happen, they rely on a systematic approach to addressing quality problems known as DMAIC, or Define, Measure, Analyze, Improve, and Control.

Besides inputs and outputs, they need to measure all kinds of metrics from inside the production processes.

Data on duration, yield, efficiency, throughput, and other useful measures are collected and subsequently analyzed.

If the number of defects is high, or efficiency is low, the leadership (or a group of Green and Black belts) would come up with a plan for improving these processes.

The plan would then be applied and new data is now ready for collection and analysis. The cycle goes on indefinitely.

3.4 Process Improvement with DMAIC

Process improvement completed in this manner is often referred to as DMAIC which stands for:

DMAIC Process Improvement
DMAIC Process Improvement
  1. Define the system’s processes, the customer and their requirements, and what you would like to achieve in terms of long-term goals.
  2. Measure relevant metrics of the current processes and determine the system’s current capabilities. This will establish a baseline against which you measure future performance.
  3. Analyze the data collected to find the root cause(s) of the problem. Establish correlation and causality. More importantly, understand the relationship between the inputs, outputs, and internal processes and whether any patterns in the data linking those entities can be observed.
  4. Improve or update the processes if the data is sufficient for a decision to be made. You might find that you need to redesign some of the processes from scratch. Setup pilot phases before deploying to production.
  5. Control the production process to ensure that any deviation from the norm is quickly identified and the relevant data collected.

Searching for a Guide on Process Improvement?

Process management includes process improvement and redesign. Click the button for a full guide on DMAIC as well as unique techniques from The Toyota Way.

4. Challenges of Applying Six Sigma to Software Development

Do Six Sigma practices carry over to other industries such as services? Are its concepts universal enough to be transferable to software for example?

There are a number of deficiencies that make Six Sigma either inapplicable or undesirable in certain settings but there are also some useful concepts that would be interesting to look at for software businesses.

Let’s first have a look at some of those limitations before moving on to the benefits in the next section.

4.1 Limited in Scope and Utility

By comparison to The Toyota Way which, in our view, is the gold standard model of business management (as the history of Toyota share prices may indicate), the targets of Six Sigma are less ambitious by comparison.

Oftentimes, Six Sigma is marketed as an all-encompassing framework that covers the organizational culture, people, processes, as well as quality control.

This stretches the limits of the application of Six Sigma significantly.

Six Sigma vs Operational Excellence
Six Sigma vs Operational Excellence

A quick look at how Six Sigma works is sufficient to understand that it’s only designed and equipped for controlling quality under specific conditions.

In fact, Six Sigma is centred around DMAIC, a data-driven method for continuous improvement and optimization of business processes. That’s what it is, and what it’s meant to be.

Any additional efforts to stretch its usability seems, in our view, a (poor) way of marketing the consultancy industry and the services that have flourished around it.

In addition to quality control, you also need to look after your people, your organization’s culture, as well as external challenges like competition, technological disruptions, and economic mood swings.

Whether you are in the business of software development or something else, Six Sigma may bring benefits to quality control, but it will need to neatly fit in a much larger framework that deals with other types of challenges.

4.2 Inadequate For Small-Sized Businesses

Six Sigma requires a serious investment in training, effort, and tools. This might be enough to put most small to medium-sized businesses off.

In addition to that, small businesses are typically more flexible and can deal with problems of quality control in much simpler and more practical ways.

  • The Andon system is a perfect example. It’s ridiculously efficient and extremely simple to implement. It’s also easy to understand and explain to both senior management as well as the average worker on the shop floor.

In short, Six Sigma is an effort-intensive exercise that requires a complex infrastructure to support it. This raises the bar a bit too high for small businesses to enjoy its benefits.

4.3 Overreliance on Statistics

Methods that heavily rely on statistical analysis only work when the data collected is of high quality.

Data is deemed high quality if it satisfies two conditions.

  • First, the processes that generate the data are consistent and time independent. This means that observing a value of 10 today means the same thing if the same value is observed tomorrow.
  • Second, the amount of data collected is large enough to eliminate sample bias. This means that the sample we are currently looking at is an accurate and true representation of the population / processes it came from.

When data quality is high, the inferred results are statistically significant and can be relied upon to make decisions.

When you have quality data, you can establish a baseline against which any future improvements can be measured. This is crucial if DMAIC, and therefore Six Sigma, were to work.

As anybody who has ever worked in software development can tell you, repetitive tasks carried over for extended periods of time are pretty rare. It’s just not how software development works (at least most o the time, more on that later).

Each project has some unique features and a different mix of team members. Also, technology changes fast and new ideas come and go every day.

This is all the more true in small businesses where processes are not yet mature and standardized.

All of these are enough to make any sort of data collection difficult and results of statistical analysis very weak.

4.4 Six Sigma and Agile Practices

Agile was developed as a response to ever-changing requirements in software projects.

Essentially, Agile offered a way of dealing with unexpected events which, at some point, became a hallmark of software and IT projects. This fact leaves us with two thoughts on the subject.

  • Agile accepted the fluid and somehow unpredictable nature of software. Instead of trying to eliminate those uncertainties, it tried to provide innovative ways for dealing with them.
  • This fluidity in a very dynamic environment can be difficult to track and measure with enough consistency for Six Sigma processes to apply.

Does this mean that Agile and Six Sigma cannot coexist in the same environment?

Agile has brought a lot of advantages to software delivery. This makes it very hard to justify any destructive interference that might come along from Six Sigma.

5. How Can Software Development Benefit from Six Sigma

5.1 Universal Concepts of Six Sigma

So, are there any Six Sigma practices or ideas that are generic or universal enough to be carried over to software development?

The answer turns out to be YES.

The first concept that can be easily carried over is that of a closed-loop system. The fact that you can faithfully model any complex system of processes along those lines can be very helpful in understanding its internal dynamics.

Once you understand how it works, you can then proceed to analyze its performance and improve upon it where required.

The second concept has to do with measurement and data collection. Despite the fact that these concepts are difficult to apply to software activities, this is less so when some of these activities have been automated.

This is, in fact, one of the objectives that DevOps aims to achieve.

Finally, repetitive tasks that have a quantitative and measurable outcome such as software estimation are great candidates for statistical analysis as those seen in the previous paragraphs.

5.2 Six Sigma and DevOps

Automating software production tasks (building, testing, deployment) removes the human variable from the equation and makes the results more or less consistent.

Also, machine activities can be easily measured, tracked, analyzed, and improved using pre-existing tools.

This means a lot less infrastructure to invest in and a lot less impact on personal relationships with employees.

To give a concrete example, we can look at Unit Testing and its efficiency.

We have argued in a previous article that the efficiency of unit testing depends a whole lot on the size of the unit being tested. If the size is small, the number of test cases required to cover all the different scenarios can be tremendously high.

This immediately translates into a huge cost-of-ownership and an equally large running time. Both of these effects can be easily measured if Six Sigma was to be applied to enhance it.

5.2 Six Sigma and Customer Service

There are other areas that can still benefit from Six Sigma practices even without automation.

Modern ticketing systems like JIRA (which have now evolved well beyond ticket tracking tools) offer a lot of flexibility and tools for measuring and collecting data.

Despite the fact that the usefulness of the data as well as its quality can always be questioned, the fact remains that the infrastructure is ready is a magnificent advantage.

Looking at it from another perspective, customer support, for example, is not as volatile as development, and with systems like JIRA, some metrics can be easily set up and insights into customer satisfaction can be obtained.

6. Final Words

Let us, for a moment, ignore the Six Sigma hype that’s partly due to the consultancy business built around it, which aims to portray Six Sigma as a silver bullet to all your business management problems.

Let’s see if we can judge Six Sigma solely on its merits.

Our view is that Six Sigma practices are just tools like any others. They are neither good nor bad. Their utility depends on how and when they are applied.

In this sense, we believe that turning a set of valuable practices into a cult where they are ritualized and assumed to hold in all times and circumstances can do more harm than good. This statement is quite valid for Six Sigma as well.

Finally, because of the nature of those concepts and tools that involve a lot of statistical analysis and data gathering, they become less useful for small businesses, especially in the software development world.

Automation, however, can drastically change the picture.

You might uncover exciting insights if you run some primary analysis on the metrics that govern those tasks.

To help you decide on whether Six Sigma is right for your business, you need a principle ingredient: Fluency on the topic.

We hope this article has brought you closer to the subject at hand.

7. Further Reading

Technical Risk Management and Decision Analysis — Introduction and Fundamental Principles

1. Overview I could not find a better way to start an article on Risk and Risk Management than by quoting the opening lines of Donald Lessard and Roger Miller’s 2001 paper that, briefly but lucidly, summarizes the nature of large engineering endeavours. It goes like this: This article leans heavily on three handbooks thatContinue reading “Technical Risk Management and Decision Analysis — Introduction and Fundamental Principles”

Complexity and Complex Systems From Life on Earth to the Universe: A Brief Introduction

1. Overview Dealing with complexity is an integral part of our lives, even if we do not realise it.  An organisation can be modelled as a complex system from the scale of megacorporations right down to the smallest teams. The architecture of software solutions can be equally complicated, and megaprojects and implementations are certainly involved.Continue reading “Complexity and Complex Systems From Life on Earth to the Universe: A Brief Introduction”

Book Review: Programming the Universe — A Quantum Computer Scientist Takes on the Cosmos

Synopsis Most physical theories adopt a mechanistic view when examining natural phenomena where any system can be modelled as a machine whose initial conditions and dynamics govern its future behaviour. In this book, Programming the Universe — A Computer Scientist Takes on the Cosmos, Professor Seth Lloyd proposes a radically different approach centred around aContinue reading “Book Review: Programming the Universe — A Quantum Computer Scientist Takes on the Cosmos”

From Abstract Concepts to Tangible Value: Software Architecture in Modern IT Systems

1. Overview Software design and architecture are two very elusive concepts; even Wikipedia’s entries (ref. architecture, design) are somewhat fuzzy and do not clearly distinguish between the two. The Agile manifesto’s statement on architecture and design is especially brief and raises more questions than answers. The most common definition of software architecture is as follows:Continue reading “From Abstract Concepts to Tangible Value: Software Architecture in Modern IT Systems”

Business Requirements and Stakeholder Management: An Essential Guide to Definition and Application in IT Projects

1. Overview The complexity of business requirements in IT projects has experienced exponential growth due to pressures by increasingly sophisticated client preferences, novel technologies, and fierce competition. Consider, for example, the case of financial payments. In the mid-80s, most payment transactions occurred inside bank branches, and only the biggest banks offered services on ATM orContinue reading “Business Requirements and Stakeholder Management: An Essential Guide to Definition and Application in IT Projects”


Something went wrong. Please refresh the page and/or try again.

Leave a Reply