Operational Excellence and the Structure of Software Development and Delivery
1. Overview
I wrote the first article on Operational Excellence in Software Development in 2020. Since then, the original (and crude) ideas have become more lucid and crisp, and it’s time for a rewrite. In this two-part series, we’ll discuss the theory and principles of Operational Excellence in Software Development, a modern guide to understanding software development and the people involved in it.
This article (Part 1) will delve into the theories behind the principal ideas of Operational Excellence, exploring some critical junctions in the history of software development, why IT projects often fail, and the software delivery value chain. We’ll also examine the influential drivers in software delivery businesses and discuss the cultural transformations required for success. Then, we provide a comparative analysis of the manufacturing and service models and why the latter is more suited to software businesses, highlighting the benefits of a theory-informed practice. With these principles in mind, software technicians and professionals can appreciate the forces driving software delivery and software teams.
Part 2 will then elaborate on the six principles of Operational Excellence, focusing mainly on Software Development and Delivery.
This article series provides practical guidance for businesses seeking to optimize their software development and delivery processes. Organizations can drive innovation, improve quality, and deliver value to their customers by understanding the theory and principles of Operational Excellence in Software Development.
The main themes of this discussion are as follows:
The remainder of the topics in this series can be found below:
2. Software Development: A Brief History
There are extremely strong economic pressures on manufacturers, both from users and from other manufacturers. Some of these pressures, which are a major contributory cause of our problems, are quite understandable. For example, the rate of increase of air traffic in Europe is such that there is a pressing need for an automated system of control.
Understanding the current state of software development requires a (brief) inspection of the historical trajectory it has followed since its inception in the early 50s. Significant technological breakthroughs dot this trajectory. Closely following these advancements are noticeable changes in customer (or end-user) behaviour and preferences, fuelling more changes and bringing about further innovations.
These intertwining historical threads (technological advancements and changing customer preferences) will allow us to understand the current trends, practices, and mainstream thoughts in software development. More importantly, they will provide some context to the industry’s present challenges, although these should be examined under anthropological and psychological lenses to fully appreciate their significance.
2.1 A Timeline of Major Milestones in Software Development
The following is a brief history of software development with some of the most significant milestones, technological advancements, and people involved:
The above timeline is a high-level summary of the major events that formed and shaped the software industry. Countless individuals and organizations contributed to its ascent. It continues to evolve rapidly as new technologies and approaches are developed. Despite being a relatively young field, software development has brought products and applications to every aspect of our lives.
2.2 Dealing With a Changing Customer Base
The customer base for software products has undergone a significant transformation over the years, which has had a major impact on the industry. The following are some of the major historical bifurcations:
Overall, expanding the customer base for software products has profoundly impacted the industry, driving innovation and competition and creating new opportunities for companies of all sizes. However, it has also brought new challenges, such as the need to develop software that is accessible, user-friendly, and compatible with a wide range of devices and platforms. As the customer base for software products continues to evolve, the industry will need to remain agile and adaptable to stay competitive.
2.3 An Ever-Changing Landscape of Challenges
As the software industry has evolved, so too have the skills, education, expertise, organization, project management and delivery methodologies required to succeed. In the early days of software development, many programmers were self-taught or learned on the job, and there were few established best practices or standards for development.
The changing nature of the software industry has required developers to become more specialized and knowledgeable while promoting cross-functional collaboration and communication. The industry has also placed a greater emphasis on interoperability and platform proliferation, as well as adopting agile project management and delivery methodologies.
3. Troubleshooting Organisational Behaviour
3.1 Organisational Strategies and Behaviour
To respond effectively to organisational challenges, we must first comprehend organisational behaviour. Organisational psychologists have developed frameworks to aid this understanding over the past few decades. These frameworks include:
While software engineers may not see the immediate relevance of these theories when dealing with the challenges of upcoming releases, it is crucial to recognize their importance. This is especially true when considering the topic of IT project failure, which will be explored in the following sections. It will become clear why simplistic approaches to this issue will not suffice and why more sophisticated interventions are required.
3.2 Why IT Projects Fail
IT projects have become vital to modern business, enabling organizations to streamline processes, improve productivity, and enhance customer experiences. However, despite the widespread use of technology, IT project failures are still prevalent. The consequences of these failures can be disastrous due to poor planning, unrealistic expectations, or lack of communication, resulting in financial losses, damaged reputations, and missed opportunities.
What puzzles me is not the event (IT project failures) itself but why it has become so unsurprising in the industry and why professionals have become so accustomed to it almost to the point of resignation. The answer to this question lies heavily in the complexity of business management issues and the immense challenges of turning the ship around.
To illustrate this complexity, we will do the following exercise. In the next two sections, we will try to uncover the underlying issues causing IT projects to fail widely. The aim is not to pinpoint the causes per se but to expose their intricacy, depth, and interconnectedness.
3.3 A Brief (and Potentially Shallow) Analysis
If you ask Google or ChatGPT about the “Top reasons for IT failure”, you will get an answer resembling the following:
- Poor Planning, which translates into:
- Inadequate project management
- Lack of clear goals and objectives
- Inadequate resource allocation
- Poor Communication, often manifesting as:
- Inadequate communication between stakeholders
- Inadequate communication within project teams
- Communication barriers due to language and cultural differences
- Technical Issues, summarised by:
- Inadequate technical expertise
- Technology incompatibility issues
- Technical infrastructure issues
But if this diagnosis has already been made and is widely accepted, why can’t we proceed to the resolution? The answer lies in the multiplicity of interpretations and the depth at which the analysis might be made. Causes interpreted at different levels or meanings will lead us to different solutions. This proliferous analysis will induce conflict as stakeholders support whichever version of the solution they favour (or that which causes them the least damage).
To investigate these ideas further, let’s continue with this exercise. We will use the Five Whys method to investigate one problem, particularly the “Inadequate technical expertise”.
3.4 More Subtle Reasons
Using Inadequate technical expertise as an example, here are some hypotheses that can explain the lack of talent in an organisation:
Again, the idea here is not to develop a complete list of reasons why a particular project is not adequately resourced but to make two observations:
To illustrate Observation 2, let’s take Insufficient training and development opportunities and see if we can break it down further. Here is a list of candidates:
The tree we have now built has many branches, each of which can be a potential cause or symptom of a, yet again, deeper underlying issue. At this stage, we can see the following questions arise:
The (practical) answers to these questions are probably for a different article (in Part 2, we will examine the six Principles of Operational Excellence in Software Development, which will shed some light on one or two aspects). Here, we will focus on the question itself: how to understand organisational problems and what solutions can be applied.
3.5 Organisations as Complex Adaptive Systems
According to this Newtonian view of the world, humans will ultimately be able to dominate nature. […] That thinking is based on the belief that managers can, in principle, control the long-term future of a human system. […] These theories say that when a deterministic nonlinear system moves away from stable equilibrium towards a state of explosive instability, it passes through a stage of bounded instability in which it displays highly complex behaviour.
— Ralph Stacey, Strategic Management and Organisational Dynamics — The Challenge of Complexity to Ways of Thinking About Organisations
The following is a list of fundamental principles and observations that will allow us to answer “how to understand and act in an organisation” (or how to achieve Operational Excellence in software development teams) by modelling organisations as Complex Adaptive Systems. This list should hopefully dispel some old myths and replace them with new insights.
Eminent thinkers like Dave Snowden and Ralph Stacey have provided alternative tools (such as the Cynefin framework or process thinking) for managing complexity. Dave Snowden has fantastic lectures online, while Stacey’s seminal textbook Strategic Management and Organisational Dynamics — The Challenge of Complexity to Ways of Thinking About Organisations is a must-read.
4. The Software Delivery Value Chain
Value generation occurs when end users realize what technology (automation, fast computation, mechanization) can do for them. Once a solution is prototyped or deployed in the field, end users start interacting with it, spotting new opportunities they never thought of before, subsequently altering their preferences and creating new business needs. This cycle drives value generation.
4.1 Software Delivery and Value Creation
It’s impossible to understand software delivery without a solid grasp of customer and business value creation processes. The mainstream view of value creation rest on the following assumptions:
4.2 Optimising the Value Chain
The debate on the primacy of certain stages regarding value creation is fundamental to process optimisation. Its outcome will determine how you optimise your value chain for efficiency and maximum productivity by selecting the stages that need to be expanded, reduced, automated, or optimized.
The debates also cover the optimisation of primary stages or what the analysis, design, and development processes should look like. The following questions illustrate the point:
Requirement gathering and definition is also a topic for optimisation. These SDLC stages are trickier and more complex as they involve end users instead of solely technicians. For example:
The answers to these questions are not universal but context-specific. However, fundamental principles and best practices can be identified to aid in selecting the best project delivery methodology (Agile, Waterfall, DevOps), for example.
4.3 Politics, Psychology, and Optimisation
Office politics play a crucial role in driving organisational behaviour. Aside from the common motives for increasing one’s power and prestige vis-a-vis one’s peers, the incentives and rewards system is another influential moderator of behaviour. Consider the following quote from Conway’s article:
A manager must subcontract a crucial and difficult design task. He has a choice of two contractors, a small new organization that proposes an intuitively appealing approach for much less money than is budgeted and an established but conventional outfit asking a more “realistic” fee. He knows that if the bright young organization fails to produce adequate results, he will be accused of mismanagement, whereas if the established outfit fails, it will be evidence that the problem is indeed a difficult one.
— Melvin Conway, How Do Committees Invent
What it’s saying is that, more often than not, people will choose a conventional, safe, but suboptimal solution over a radically different, potentially risky, but promising solution. This can be readily observed today if a minority of developers might, for example, argue against an Agile methodology in a context where Agile can be shown to be suboptimal. Instead, job security considerations dictate that Agile is the best there is.
Power relations will also influence which teams or departments have adequate budgets, resources, and attention to effectively conduct optimisation exercises. Organisational politics do not always reflect the true requirements of the situation. Teams with no champion for their cause might not get the right headcounts, budget, or tools they need to perform their job adequately.
Most design activity requires continually making choices. Many of these choices may be more than design decisions; they may also be personal decisions the designer makes about his own future. As we shall see later, the incentives which exist in a conventional management environment can motivate choices which subvert the intent of the sponsor.
— Melvin Conway, How Do Committees Invent
Here also, the problem diagnosis (i.e. finding the source of inefficiencies) will determine which solutions will be applied. In most cases, technicians will endeavour to downplay their inefficiencies while emphasizing those of other departments.
Optimising the value chain is strategically important and must be data-driven and applied regularly and consistently (much like hygiene).
5. The Impact of Tools and Technology
5.1 Business and People First
It might seem absurd to the reader who has followed us thus far that a discussion about software development has not yet discussed programming languages, frameworks, automation, or any tech-related topic. The reasons are quite straightforward:
In the following two sections, we will discuss technology and tools, not from a technical perspective, but from a delivery one. Instead of listing and comparing two programming languages or algorithms, for example, we will focus on what aspects of technology drive business decisions and what the implications of this are.
5.2 Technology Serving the Business
The following few paragraphs are centred on the following idea: Any technology discussion without referencing time and organisation scales is futile. For example, technology is rarely influential in small initiatives with a limited scope since the tools and software products involved are not being questioned. Most of the time, it’s either an integration project, a change request, or a new feature with a predefined tech stack.
Let’s distinguish three contexts and see how technology drives decisions in each.
5.3 Tools Working For (and Not Against) You
The DevOps age ushered in an unprecedented deployment of sophisticated tools aiming to automate many aspects of repetitive tasks where manual interventions never add any value. This change has brought in immense benefits such as:
- Build automation of complex projects
- Automated unit and system integration testing
- Automated deployment and monitoring
Points 1 and 2 allow what is now called Continuous Integration (CI), while point 3 allows for Continuous Deployment (CD). Continuous Integration and Continuous Deployment combined allow businesses to achieve Continuous Delivery, a software delivery methodology whereby new features are deployed on production daily.
What about the costs associated with this software delivery method? Here is a list of the top contributors (in no specific order):
The complexity involved in maintaining an up-to-date continuous delivery pipeline is quite high and must be managed accordingly. There is no point in owning a car that breaks down every 100m; in this case, walking or riding a bicycle would be much more efficient.
6. Business and People Management: Three Schools of Thought
6.1 Overview
One More Time: How Do You Motivate Employees? is an influential article by Frederick Herzberg published in the Harvard Business Review in 1968. The title is a bit misleading as the article offers insights well beyond employee motivation and is, therefore, a highly recommended read. Surprisingly, while quite dated, its ideas remain as relevant today as they were in the 60s, and for a simple reason: the nature of the relationship between the organisation and its managers on the one hand and employees on the other changes slowly.
The following sections discuss three schools of thought on managing people: Organisational Theory, Industrial Engineering, and Behavioural Science Theory.
6.2 Organizational Theory
6.2.1 Overview
The organizational theory school of thought focuses on an organisation’s formal structures and processes. It emphasizes the importance of clear lines of authority, defined roles and responsibilities, and standardization of processes to achieve efficiency and effectiveness. In terms of people management, this school of thought strongly emphasises hierarchy and the control of individual behaviour through formal rules and procedures.
6.2.2 Merits
6.2.3 Critiques
6.3 Industrial Engineering
6.3.1 Overview
Industrial engineering focuses on optimising organisational processes and systems. It seeks to identify ways to improve efficiency and reduce waste to increase productivity. In terms of people management, this school of thought views employees as resources to be utilized to achieve maximum productivity. The focus is finding the most efficient way to get work done, often through technology and automation.
6.3.2 Merits
6.3.3 Critiques
6.4 Behavioural Science
6.4.1 Overview
The behavioural scientist’s school of thought takes a more humanistic approach to people management. It emphasizes the importance of understanding individual behaviour and motivation and the social and cultural context in which work is performed. This school of thought emphasizes empowering employees, creating a supportive work environment, and fostering positive relationships between managers and employees. The goal is to create a workplace where employees are motivated to perform at their best and can achieve their full potential.
6.4.2 Merits
6.4.3 Critiques
6.5 Choosing a Management Style
Several factors can determine which of the three schools of thought (Organizational Theory, Industrial Engineering, or Behavioral Science) is applied in an organization and at what ratio. Here are some of the critical factors:
Organisations and leadership must balance two conflicting forces: maximising profit while keeping employees engaged and motivated. Although these requirements are not always mutually exclusive and do not necessarily come at the expense of one another, poor management can undoubtedly make it so.
6.6 The Management Style Triad
Imagine a triangle with three corners representing the Organizational Theory, Industrial Engineering, and Behavioral Science management schools of thought. We can locate a specific leadership style by placing a point inside this triangle. Dave Snowden advocates using this triangle to assess and enhance leadership rather than relying on the conventional, hypothesis-based questionnaire, an exercise that people can easily game.
We label the vertices of this triangle as follows:
Ideally, we want the leadership and management style to be balanced, i.e. equidistant from the three vertices. That, however, might not be the optimal state of affairs. Consider the following situations where a tilt towards one vertex or another can provide considerable advantages.
| Context | Leadership and management styles | Why will that work |
|---|---|---|
| Crisis Management | Assertive and pragmatic leadership, able to go against basic assumptions. Takes immediate action to restore order. | People are happy to conform to restore order and predictability and reduce anxiety. |
| Strategic, Long-Term Planning | Slow and rational decision-making process | Here assertiveness will backfire, and people will rarely cooperate. |
| Business-as-Usual (BAU) | Rational, fair, and process-oriented | This would be the best time to apply incremental improvements to leverage lessons learned. |
| Disruptions and Transformations | Assertiveness, clarity, visionary, empathy | High anxiety levels typically accompany major transformations. |
Let’s review these in more detail:
6.7 Leadership and Technical Proficiency
One of the drawbacks of using the triad from the previous section is that a point in the middle of the triangle can mean several things. It tells us that, on average, all three management styles are applied, but it does not tell us whether they were applied at the right time or whether actions and decisions were random and neurotic.
To avoid this problem, technical proficiency might be a prerequisite for the triad to work. This means that, at a bare minimum, management must fully understand the technical and commercial aspects of the business. It must also seek to understand the people, the processes, and the work involved in producing the organization’s products or services.
Erratic, ad-hoc decisions or frequent backpedalling are usually signs of technical deficiency, panic, or both.
While it is not always necessary for management to be technically proficient in every aspect of their business, having a good understanding of the technical aspects of their industry can be beneficial.
Managers who possess technical skills can better understand the intricacies of their business operations and be more effective in making technology, innovation, and product development decisions. They can communicate more effectively with their technical staff and make informed decisions about investment in new technology or IT infrastructure.
7. Managing Organisational Change
7.1 We Understand the Problem, Why Is It So Difficult to Change?
The answer to this fundamental question on transformation and change can be articulated by examining a textbook theory of change, such as John P. Kotter’s model. In summary, the success of Kotter’s model relies on the following assumptions being held for all transformations:
As ancient wisdom goes, there are many ways to go down but only one way to go up, and in this case, it’s enough to invalidate one of these six assumptions before the entire edifice comes tumbling down.
In the famous article Disruptive Technologies: Catching the Wave, authors Joseph Bower and Clayton Christensen recommend that organisations investing in potentially disruptive technologies create separate organisations to deal with new products, markets, and customers and keep the old and new organisations independent. In other words, they seem to be saying that transforming an old organisation so that it simultaneously sells two potentially competing products, an old and new, is extremely challenging. Here, the best approach is to let the sister organisations compete and thrive independently, and if the old one loses its customer base, it would be allowed to retire gracefully. This sounds more reasonable and straightforward than a cumbersome and highly risky transformation.
Kotter’s model, like many others, is based on a few fundamental premises regarding the nature of reality and how organisations behave:
This view was refuted earlier in favour of an approach based on Complex Adaptive Systems. In the next section, we will use this paradigm to present a radically different theory of change by management expert Dave Snowden.
7.2 An Alternative Theory of Change
Dave Snowden’s theory of change relies on the following premises based on our current state of knowledge in philosophy and natural sciences and his rich experience in the field.
The ideas we presented above are but a few of which Snowden champions in his business management theories. We highly recommend you watch his online videos and read his blog posts.
8. The Manufacturing vs Services Model
In the design of automobiles, the knowledge that you can design the motor more or less independently of the wheels is an important insight, an important part of an automobile designer’s trade. In our field, if there are a few specific things to be produced, such as compilers, assemblers, monitors, and a few more, then it would be very important to decide what their parts are and what is the proper sequence of deciding on their parts. […] The approach suggested by Christopher Alexander in his book: Notes on the Synthesis of Form, is to make a tree structure of the decisions, so that you start by considering together those decisions that hang most closely together, and develop components that are sub-systems of your final design.
8.1 What Is Similar
Some concerns have recently been raised among experts that we are too reliant on the manufacturing model as our source of inspiration. This source, however, has its merits; there are too many similarities between producing cars and software to be ignored. Let’s list a few of those similarities:
8.2 What Is Different
It’s also true that most of what goes on in Toyota’s vehicle centres is routine product development making incremental change from one model to the next. But the beauty of the Toyota Way is that it allows Toyota to periodically break from this “conservative” mold and innovatively develop a new vehicle with a new developmental approach.
— Jeffrey Liker, The Toyota Way – 14 Management Principles From the World’s Greatest Manufacturer
Remarkable differences also exist, meaning we must be critical of what applies and what doesn’t. Let’s look at some areas where the manufacturing model and the service model diverge:
In manufacturing, many problems are engineering, where the solution is already known to experts, and different solutions can be tested under laboratory conditions to determine the best. The problems are more complex in software, as we have endeavoured to show in the previous sections, fundamentally because the human element is strongly present and software applications tend to be unique. The potential of software applications (AI, ChatGPT, social media) to radically modify customer behaviour is non-trivial. At the same time, new digital camera models have not changed how we take pictures.
8.3 Cross-Industry Learning
We conclude this section with the following thoughts. Best practices can be effectively shared between industries, and corporations like Toyota have led the way in innovative techniques for dealing with organisational challenges.
While lessons learned from Toyota have been and continue to be applied across many industries, the same cannot be said for Six Sigma, so we must be selective in our choices, with decisions made based on theory combined with practice regarding what will work versus what won’t.
9. Evolution of the Software Development Industry
9.1 Reconstructing History
It seems natural to think about aspects of our modern lives, such as technology, as static and that “things were always that way”, but that’s very far from the truth. The smartphone and all the sophisticated apps that live in it have become such an integral part of our lives that we can barely remember what those lives looked like before them.
This stretching of the past in a manner that makes old tech look ancient and not worth understanding is OK when making minor decisions. However, when it becomes crucial for mid-term or strategic planning, more important is retrospection or going back to first principles and examining the history and evolution of certain ideas (Agile is a great example of a concept where context is vital). This approach to certain problems involves critical thinking to determine how these ideas fared and whether they are still relevant today.
History, however, is a tricky business. For ordinary people, the past is a series of inevitable events, one leading the next. Historians would profoundly disagree and prefer to view historical events as a series of dots that could be connected in many different ways. Each path of connected dots is one narrative seen from a narrow perspective.
Moreover, these narratives can intermingle and dramatically influence each other’s evolution. These junctures (sometimes called bifurcation points or milestones) must be pinpointed to understand what happened truly.
9.2 A Regression of Many Intertwining Narratives
At the beginning of this article, we described two such narratives: technological advancement and changing customer preferences. They were and continue to be heavily intertwined, reinforcing each other (think of positive feedback loops) within the complex system comprising the tech organisations and their client bases.
Let us add a few more to these two narratives, which we list below. The order of the list is important. As you move down the list, the frequency of change in how things are done decreases, i.e., the top elements change faster than the ones below. Let’s see how this works.
9.2.1 Advancements in Software and IT Technology
This represents changes in programming languages, frameworks, deployment methods, business models, project management and delivery methodologies. The average lifetime of these changes is about 5-10 years, with a tiny minority becoming mainstream (C++, Java, Agile, Waterfall, client/server architecture, etc.).
9.2.2 Major technological advancements
This category includes groundbreaking leaps in technological know-how that can transform markets and client preferences and topple corporate giants. Examples are printers, personal computers, Compact Disks (CDs), DVDs, walkmans, the internet, the iPod, tablets, smartphones, social media, artificial intelligence, and quantum computing. The average lifetime of these technologies is 10-20 years, with naturally some surviving a lot longer (the lifetime probably follows a power law distribution rather than a Gaussian).
| Technology Name | Date it Became Mainstream | Date it Died (if applicable) |
|---|---|---|
| Television | 1940s | N/A |
| Microwave Oven | 1950s | N/A |
| Portable Radios | 1950s-1960s | 1990s-2000s |
| Personal Computer | 1980s | N/A |
| Walkman | 1980s-1990s | 2010s |
| CDs | 1980s-1990s | 2010s |
| Internet | 1990s | N/A |
| DVDs | 1990s-2000s | 2010s |
| Video Gaming Consoles | 1970s-1980s | N/A |
| iPod | 2000s | 2010s |
| Smartphone | 2000s | N/A |
| Social Media | 2000s | N/A |
| Tablets | 2010s | N/A |
| Streaming Video | 2010s | N/A |
| Virtual Reality | 2010s | N/A |
| Artificial Intelligence | 2010s | N/A |
9.2.3 Business Management and Organisational Theory
Now, the pace has slowed considerably, and rather than in a few decades, it can be significantly more. The table below presents some of these ideas, their key thinkers, their inception dates, and how long they survived. Notice how many of these ideas survive today, some holding wildly contradicting views of others.
| Theory/Concept | Date of Emergence | Key Thinkers | Date it Stopped Becoming Mainstream |
|---|---|---|---|
| Scientific Management | Late 19th/early 20th century | Frederick Taylor | 1930s-1940s |
| Administrative Theory | Early 20th century | Henri Fayol | N/A |
| Bureaucratic Theory | Early 20th century | Max Weber | N/A |
| Human Relations Theory | 1920s-1930s | Elton Mayo, Mary Parker Follett | 1950s-1960s |
| Systems Theory | 1940s-1950s | Ludwig von Bertalanffy | N/A |
| System Dynamics | 1950s-1960s | Jay Forrester | N/A |
| Complexity Theory | 1980s-1990s | Stuart Kauffman, John Holland, Dave Snowden, Ralph Stacey | N/A |
| Systems Thinking | 1990s-2000s | Peter Senge, Donella Meadows | N/A |
| Contingency Theory | 1960s-1970s | Joan Woodward, Paul Lawrence, Jay Lorsch | N/A |
| Total Quality Management | 1980s-1990s | W. Edwards Deming, Joseph Juran, Philip Crosby | 1990s-2000s |
| Lean Management | 1990s-2000s | Taiichi Ohno | N/A |
| Agile Management | 2000s-2010s | Jeff Sutherland, Ken Schwaber, Martin Fowler, Robert Martin | N/A |
| Design Thinking | 2000s-2010s | Tim Brown, David Kelley | N/A |
9.2.4 Organisational Culture
Our final narrative (as far as this discussion goes) is Organisational Culture. The main schools of thought have been listed below, with the majority still surviving.
| Concept | Date of Emergence | Key Thinkers | Date it Stopped Becoming Mainstream |
|---|---|---|---|
| Organizational Culture | 1950s-1960s | Edgar Schein | N/A |
| Organizational Climate | 1960s-1970s | Robert Quinn | N/A |
| Organizational Identity | 1980s-1990s | Albert and Whetten | N/A |
| Organizational Learning | 1980s-1990s | Chris Argyris, Peter Senge | N/A |
| Organizational Citizenship Behavior | 1980s-1990s | Dennis Organ | N/A |
9.3. Key Takeaway
The key takeaway from this discussion is the following heuristic:
The amount of time invested in a specific aspect of the business (technology, processes, management philosophy, and organisational culture) must be inversely proportional to its change frequency.
This implies the following:
On a slightly tangential, but not completely off-topic, subject, a heuristic for determining the lifetime of ideas might be the following: if a product, idea, or concept’s age is X years, it’s likely to live for another X years. (see Fooled by Randomness)
10. Summary
Let us now summarize the fundamental ideas of this article by order of importance from least to most important.
11. Conclusion
We hope the previous paragraphs have taken the reader on a journey of self-discovery, whose primary aim is to help them make sense of their past experiences by placing at their disposal new tools and frameworks distilled from natural sciences, organisational and human psychology, complexity theory, and business management.
While the infinite alternative future paths potentially available to us collapse and narrow with every decision we make or new person we meet, our past is also under perpetual reconstruction, redesigned and coloured by novel ideas and insights we acquire. Reinterpreting the past on a sound and solid basis prepares us for future challenges, of which we hope to have enough to make our lives interesting but not too stressful.
For software engineers whose education is primarily in mathematics, science, technology, and engineering, humans and organisations are cybernetic machines led by objective and rational leaders whose main job is to formulate long-term organisational strategies. Once those strategies are translated into action plans that they and their peers can dutifully apply, success should follow. We hope that the ideas presented in this text have convinced the reader of the mythical nature of this view and the complexity and messy nature of the real world.
If the world is wild, messy, unpredictable, and unwieldy, organisations must modify how they sense, interpret, and act in this world to survive. While the ideas we explained barely scratched the surface of their respective disciples, we hope they have intrigued the reader’s mind and spurred them into looking further. We also hope they have provided enough theoretical background to allow you to efficiently use (and potentially adapt and modify) the six Principles of Operational Excellence in Software Development articulated in Part 2 of this series.
12. References
- Thinking, Fast and Slow — Daniel Kahneman
- The Toyota Way – 14 Management Principles From the World’s Greatest Manufacturer — Jeffrey K. Liker
- Strategic Management and Organisational Dynamics — The Challenge of Complexity to Ways of Thinking About Organisations — Ralph Stacey
- Book Review: Organisational Culture and Leadership — Edgar Schein
- HBR at 100 — The Most Influential and Innovative Articles from Hard Business Review’s First Century — Harvard Business Review
- Cultures and Organisations — Software for the Mind — Geert Hofstede, Gert Hofstede