The 7 Timeless Steps to Guide You Through Complex Problem Solving

As we go through life, we inevitably encounter problems that require extensive forethought, critical thinking, and creativity. Solving complex problems is a crucial skill for success, whether it’s a business challenge, a personal dilemma, or a societal issue.

This guide will explore the fundamentals of complex problem-solving and provide practical tips and strategies for mastering this critical skill.

This article is part of a series on complex problem-solving. The list below will guide you through the different subtopics.

1. What Is a Complex Problem?

1.1 Generic Definition of Complex Problems

Complex problems

Four properties allow us to distinguish complex problems from simple ones.

  • Complex problems accept alternative solutions
  • Choices can weighed in multiple ways
  • Data supports multiple hypotheses
  • Breakdown of causal chains.

In crude terms, a complex problem presents no trivial or obvious solution. In other words, it shows the following characteristics:

  • Accepts alternative solutions:
  • Complex problems often accept multiple competing solutions. Despite showing signs of contradiction, the solutions are not inconsistent.
  • A common example is buying your first house. Do you take a loan now or later, or do you prefer to keep renting? While both solutions appear to be competitors, convincing arguments can be made for either.
  • Choices can weighed in multiple ways:
  • The solutions to complex problems can be weighed from different angles, with each angle favouring one set of criteria over the other or giving more weight to certain parameters.
  • For example, a single person might favour lifestyle over lower mortgage rates when considering a house purchase, while a family person might consider the opposite.
  • Data supports multiple hypotheses:
  • Due to insufficient data, multiple competing hypotheses may be plausible when analysing strategies that deal with complex problems, leading to different solutions.
  • Consider the case where you know where an object is heading or at what speed, but not both (Heisenberg’s Uncertainty Principle). In such a case, you could not predict where this object will be in five minutes. However, based on whether you know the speed or the location, you might make predictions regarding future direction or location.
  • Breakdown of causal chains:
  • The root-cause analysis does not yield clear-cut solutions with definitive causal links to the problem. Similarly, the outcomes of action applied today cannot be predicted.
  • For example, a hospital instituting a policy to measure surgeons’ skills based on the number of successful operations they perform can observe a radical and unintended change in their behaviour. Surgeons may start declining difficult or risky operations to keep their scores high.

Now that we have defined the general notion of a complex problem, let’s look at some specific cases related to software development, business management, and complexity theory.

1.2 Complex Problems in Software Development

A complex software development problem involves intricate interactions between numerous system components and requires a sophisticated understanding of the business problem, computing, algorithms and data structures.

Source: “Domain-Driven Design: Tackling Complexity in the Heart of Software” by Eric Evans

1.3 Complex Problems in Business Management

In business management, a complex problem is characterized by interconnected elements, uncertainty, and dynamic interactions, making it challenging to predict outcomes and devise straightforward solutions. This is most obviously seen in formulating effective organisational strategies or leading successful enterprise transformations.

Source:Strategic Management and Organisational Dynamics: The Challenge of Complexity” by Ralph D. Stacey

1.4 Complex Problems in Complexity Theory

From a complexity theory standpoint, a complex problem involves many interacting agents or components, often exhibiting emergent properties that cannot be easily deduced from the properties of individual agents.

Source:The Quark and the Jaguar: Adventures in the Simple and the Complex” by Murray Gell-Mann

Complex problems are contrasted with complicated problems. Complicated problems have clear causes and effects, can be broken down into smaller parts, and have predictable solutions. Complex problems, however, are dynamic, have interconnected parts, and exhibit emergent properties (unpredictable outcomes from the interaction of parts).

Source: “Cynefin Framework” (2007) by Dave Snowden

1.5 What are Complex Problem Solving Skills?

Complex problem-solving skills involve identifying, analysing, and solving non-routine problems requiring high cognitive effort.

These problems typically involve a large number of variables and require the application of creative and critical thinking skills to identify potential solutions. Individuals with complex problem-solving skills can work through ambiguity and uncertainty and use logical reasoning to develop effective solutions.

2. Solving Complex Problems: A Generic Approach

While developing a universal solution that works in any context would be very challenging, we will describe a generic approach consisting of seven steps that will assist you in creating a bespoke method suitable to the specific context you are working in.

At the heart of this approach is logical decomposition, or breaking down a complex problem into smaller, more manageable ones and then developing and implementing effective solutions for each. It is a key skill essential for success in many areas of life, including business, education, and personal relationships.

Logical decomposition is at the heart of scientific thought, as described in Edsger W. Dijkstra’s paper “On the Role of Scientific Thought“.

The seven steps to solving complex problems are listed below. We will go through them in great detail in the following sections.

The 7 steps to creative solutions

  • Step 1: Understand the nature of complex problems
  • Step 2: Identifying and defining the problem
  • Step 3: Gathering reliable data
  • Step 4: Logical decomposition
  • Step 5: Generating several candidate solutions
  • Step 6: Implementing the solution
  • Step 7: Evaluating the solution

3. Complex Problem-Solving Skills

3.1 Why are Complex Problem Solving Skills Essential?

In today’s rapidly changing world, individuals and organizations must possess complex problem-solving skills to succeed. These skills are essential for several reasons:

3.2 How to Develop Complex Problem-Solving Skills

While some individuals possess a natural aptitude for complex problem-solving, these skills can be developed and improved over time. Here are some tips to help you develop complex problem-solving skills:

3.2.1 Build Your Knowledge Base

Developing complex problem-solving skills requires a strong foundation of knowledge in your area of expertise. Stay updated on your field’s latest trends, research, and developments to enhance your problem-solving abilities.

3.2.2 Practice Critical Thinking

Developing critical thinking skills is essential for complex problem-solving. Practice questioning assumptions, analyzing information, and evaluating arguments to develop critical thinking skills.

3.2.3 Embrace Creativity

Complex problems require creative solutions. Embrace your creativity by exploring new ideas, brainstorming solutions, and seeking diverse perspectives.

3.2.4 Collaborate with Others

Collaborating with others can help you develop your complex problem-solving skills. Working in a team environment can expose you to new ideas and approaches, help you identify blind spots, and provide opportunities for feedback and support.

3.2.5 Seek Out Challenging Problems

Developing complex problem-solving skills requires practice. Seek out challenging problems and apply your problem-solving skills to real-world situations.

4. Step 1: Understanding the Nature of Complex vs Complicated

4.1 The Cynefin Framework

Complex and complicated problems are two distinct types of challenges that require different approaches to solve. Dave Snowden, a management consultant and researcher, developed the Cynefin framework, a conceptual model used to understand complex systems and situations. The framework identifies five domains: simple, complicated, complex, chaotic, and disordered, and guides how to approach challenges in each domain.

4.2 Complicated Problems

Complicated Problems:

  • are characterized by having many interrelated parts and require specialized knowledge and expertise to solve.
  • have a clear cause-and-effect relationship, and the solution can be discovered by systematically analysing the components.
  • are best addressed through a top-down, expert-driven approach, where the experts can identify the best solution through analysis and evaluation.

4.3 Complex Problems

Complex problems are characterized by uncertainty, ambiguity, and the involvement of multiple interconnected factors. There is no clear cause-and-effect relationship, and the solution cannot be found by simply analysing the components. Complex problems require a bottom-up, participatory approach, where multiple perspectives and ideas are considered to develop a solution. The solution may not be clear initially but involves experimentation, adaptation, and feedback.

The Cynefin framework proposes that complex problems belong to the complex domain, where emergent solutions cannot be predicted or prescribed. The complex domain should explore the problem, generate hypotheses, and test them through experimentation. The emphasis is on learning from the process, adapting to changing circumstances, and using feedback to guide the solution.

4.4 Practical Tips on Identifying an Appropriate Framework

Objective — Classify the problem as complex, complicated, or disordered. This classification will determine the approach to be used.

How it’s done — You can do that by asking the following questions.

  1. Do we have multiple, internally consistent, competing hypotheses explaining the issue?
  2. Does the available data support both theories?

In this case, the problem lies in the complex domain, and the preferred approach is to identify good solutions and conduct safe-to-fail experiments. If it’s a complicated (but not complex) problem, the following questions can be answered in the affirmative:

  1. Do we have a single view that explains the problem?
  2. Do we know the engineering part of the solution?
  3. Is the problem sufficiently familiar to be solved by an expert?

5. Step 2: Identifying and Defining the Problem

5.1 Problem Identification

The first step in problem-solving is identifying the problem. This step involves recognizing that a problem exists and understanding its nature. Some tips for identifying the problem include:

  • Observing the situation: Observe the situation and look for any signs that suggest a problem. This could be anything from an unexpected result to increased customer complaints.
  • Asking questions: Ask questions to gain a better understanding of the situation. This can help you identify the underlying issues and identify potential solutions.
  • Gathering information: Gather information about the problem by talking to people involved, researching the issue, and analyzing data.

Once you have identified the problem, the next step is to define it. This step involves breaking down the problem into smaller parts and better understanding its nature. Some tips for defining the problem include:

  • Writing it down: Write down the problem statement clearly and concisely. This will help you to focus on the specific issue and avoid confusion.
  • Breaking it down: Break the problem into smaller parts to better understand its nature. This can help you to identify the underlying causes and potential solutions. The logical decomposition of the issues is vital, and we have dedicated the next section.
  • Identifying the scope: Identify the scope of the problem and determine its impact. This can help you to prioritize the problem and allocate resources accordingly.

Reliable data and statistical analysis skills are crucial in problem-solving. Data provides information and insights necessary for understanding the root cause of the problem. Statistical analysis allows us to make sense of the data and extract meaningful information. This article will discuss the importance of reliable data and statistical analysis skills in problem identification.

5.2 Practical Tips on Identifying the Problem

Objective — Paint a full picture of the problem by laying out the details, preferably on a piece of paper, classifying it, and deciding on an approach to solving it.

How it’s done — Write down a complete description of the problem, including its scope and impact on the various stakeholders or aspects of the business. Use data as evidence to support initial hypotheses. Find out if the problem is localised and can be resolved locally or whether it might need escalation and support from higher levels of management.

6. Step 3: Gathering and Analyzing Data

6.1 Gathering Reliable Data

In today’s fast-paced business environment, reliable data is more critical than ever. It is vital to have accurate and objective information to identify problems and determine their root cause.

Reliable data is the basis of any evidence-based decision-making, without which what we have is opinions and assumptions.
Reliable data is the basis of any evidence-based decision-making, without which what we have is opinions and assumptions.

Without reliable data, it is difficult to make informed decisions that can lead to effective problem-solving. Here are some of the benefits of using reliable data in problem identification:

  • Objective information: Reliable data provides an objective perspective of the situation.
  • Evidence-based decision-making: Using reliable data ensures that decisions are based on evidence rather than assumptions or opinions.
  • Improved accuracy: Reliable data improves the accuracy of problem identification, leading to better solutions.
  • Better understanding: Reliable data provides a better understanding of the situation, leading to a more comprehensive and holistic approach to problem-solving.
  • Improved Risk Management: Reliable helps put problems into perspective by allowing analysts to calculate their occurrence probabilities and impacts. Based on impact and probability, risk can then be categorised and prioritized.

6.2 Statistical Analysis Skills

Statistical analysis skills are necessary for making sense of the data and extracting meaningful information. These skills allow us to identify patterns and trends, understand the relationships between different variables, and (sometimes) predict future outcomes.

How statistical analysis can help with complex problem solving.
How can statistical analysis help with complex problem-solving?

Some benefits of using statistical analysis skills in problem identification include the following:

  • Identifying patterns: Statistical analysis skills enable us to identify patterns and trends in the data, which can help identify the problem accurately.
  • Understanding relationships: Statistical analysis skills help us understand the relationships between different variables, which can help identify the problem’s root cause.
  • Predictive capabilities: Statistical analysis skills allow us to predict future outcomes based on the data, which can help develop effective solutions.
  • Objective analysis: Statistical analysis provides objective data analysis, which can help make evidence-based decisions.

Interpreting data, however, requires technical skills to avoid misinterpretations. The following is a common list of statistical analysis mistakes non-professionals can make.

  • Drawing conclusions based on small or biased sample sizes: Non-professionals often make the mistake of drawing conclusions based on small sample sizes, leading to inaccurate or biased results. They may also use biased samples, such as convenience samples or samples that are not representative of the population.
  • Ignoring outliers: Outliers are data points that lie far away from most data points. Ignoring outliers can lead to inaccurate results, as they may significantly impact the overall outcome.
  • Confusing correlation with causation: Non-professionals often make the mistake of assuming that correlation between two variables implies causation. However, correlation does not always imply causation, and looking for other factors contributing to the observed relationship is important.
  • Failing to consider confounding variables: A confounding variable affects both the independent and dependent variables. Non-professionals often fail to consider confounding variables, leading to inaccurate or misleading results.
  • Using inappropriate statistical tests: Non-professionals may use statistical tests that are inappropriate for the analysed data. For example, using a t-test when the data is not normally distributed can lead to inaccurate results.
  • Overfitting models: Overfitting occurs when a statistical model is too complex, leading to a poor fit and generalization to new data. Non-professionals may overfit models by including too many variables or by selecting variables based on the results of statistical tests.
  • Misrepresenting data: Non-professionals may misrepresent data, such as using inappropriate scales or selectively presenting data that supports their conclusions, leading to incorrect interpretations of results.

6.3 How Software Team Leads Can Gather Reliable Data

Software team leads need reliable data on their performance to make informed decisions and identify areas for improvement. Here are some sources where software team leads can gather reliable data on their team’s performance:

  • Project management tools: Most project management tools have built-in reporting features, allowing team leads to track performance metrics such as task completion rates, sprint velocity, and burn-down charts. This data can be used to identify areas for improvement and make data-driven decisions.
  • Team feedback: Gathering feedback from team members through one-on-one meetings or anonymous feedback forms can provide valuable insights into team performance. This data can help team leads identify areas where team members may struggle or additional training or resources may be needed. Crucially, it also provides insights into the organisational culture.
  • Code analysis tools like SonarQube or Code Climate can provide insights into code quality, maintainability, and security. This data can help team leads identify needed code improvements and prioritize technical debt reduction.
  • Customer feedback: Customer feedback, such as ratings, reviews, and support tickets, can provide insights into the usability and functionality of deployed applications. This data can help team leads identify areas for improvement and prioritize feature development.

The software team should gather data from multiple sources, use that data to inform decisions and identify areas for improvement. By using reliable data sources and monitoring team performance metrics regularly, software team leads can drive continuous improvement and ensure project success.

6.4 Practical Tips on Gathering Data to Support the Proposed Hypotheses

Objective — The availability of data can help place the problem into perspective. For example, a dollar figure of the losses due to process inefficiencies can help identify the potential solutions that management will deem feasible.

How it’s done — All modern project management and tracking tools have sophisticated built-in data capture tools that can be exported, cleaned, and analysed for insights.

For example, when evaluating a team’s productivity, you can export data from JIRA, Jenkins, or BitBucket and measure performance metrics such as team velocity, overruns, and time-to-market.

When evidence is insufficient, you can gather more data, abandon the hypothesis, or temporarily shelve it.

7 Step 4: Logical Decomposition in Problem Solving

7.1 Logical Decomposition

Logical decomposition is a problem-solving technique that breaks down complex problems into smaller, more manageable pieces. It is a structured approach that enables individuals to examine a problem from multiple angles, identify key issues and sub-problems, and develop a solution that addresses each piece of the problem.

The process of logical decomposition involves breaking down the main problem into smaller sub-problems, which are then broken down into smaller pieces. Each piece is analyzed in detail to determine its underlying cause-and-effect relationships and potential solutions. By breaking down the problem into smaller pieces, the individual can better understand the overall problem, identify potential solutions more easily, and prioritize which sub-problems to address first.

Logical decomposition is particularly useful for dealing with complex issues, as it allows individuals to break down a large, overwhelming problem into smaller, more manageable pieces. This not only makes the problem easier to understand and solve but also makes it less daunting and more approachable. Additionally, by breaking down the problem into smaller pieces, individuals can identify and focus on the underlying root causes of the problem rather than just treating the symptoms.

Logical decomposition is a vital stage of architecting large systems and solutions.

7.2 Practical Tips on Logical Decomposition

Objective — Most problems worth tackling are also overwhelming in size and complexity (or complicatedness). Luckily, a logical decomposition into specialized areas or modules will help focus the team’s efforts on a small enough subproblem or bring in the right expertise.

How it’s done — This author prefers mindmaps. A mindmap is a tree that starts with a single node and branches off into different areas, views, or perspectives of the problem. Mindmaps help analysts stay focused on a key area and ensure that all aspects of a problem are covered.

Once a mindmap has been created, potential solutions can be explored.

8. Step 5: Generating and Evaluating (Several) Potential Solutions

Generating multiple solutions to solve a problem is an effective way to increase creativity and innovation in problem-solving. By exploring different options, individuals can identify the strengths and weaknesses of each solution and determine the most effective approach to solving the problem. This section will discuss the advantages and techniques of generating multiple solutions to solve problems more effectively.

8.1 Advantages of Generating Multiple Solutions

The advantages of generating multiple solutions during problem-solving are:

  • Increases creativity and innovation: Generating multiple solutions allows individuals to explore different approaches to solving a problem, which can lead to more creative and innovative solutions.
  • Increases the likelihood of finding an effective solution: By exploring multiple solutions, individuals are more likely to find a solution that addresses all aspects of the problem.
  • Increases engagement and ownership: Individuals who generate multiple solutions feel more ownership and engagement in problem-solving.
  • Avoid being locked or overcommitted to one solution that may prove suboptimal.

8.2 Techniques for Generating Multiple Solutions

Techniques for generating multiple solutions:

  • Brainstorming involves generating as many ideas as possible without evaluating them initially. This technique encourages individuals to be creative and open-minded, which can lead to the development of unique solutions.
  • Mind mapping involves visually organizing ideas and concepts around a central theme or problem. This technique can help individuals see connections between ideas and develop new solutions.
  • Reverse brainstorming involves identifying solutions that would make the problem worse rather than better. This technique can help individuals identify the underlying causes of the problem and develop more effective solutions.
  • SCAMPER is an acronym for Substitute, Combine, Adapt, Modify, Put to another use, Eliminate, and Reverse. This technique involves using these prompts to generate new ideas and solutions by altering existing ones.
  • The Six Thinking Hats technique involves assigning different roles to team members to encourage different perspectives and generate multiple solutions. The six roles are White (facts and information), Red (emotions and feelings), Black (potential problems and criticisms), Yellow (potential benefits and opportunities), Green (creativity and new ideas), and Blue (organizational and planning).

8.3 Practical Tips on Solution Generation and Selection

Objective — The key principle of solution generation is comprehensively exploring the solution space. This exploration allows teams to avoid local minima or overcommitting to a suboptimal solution.

How it’s done — The most effective approach is to bring in several people from different areas of expertise or seniority and to offer every suggestion the opportunity to be heard and thoroughly explored.

Also, different stakeholders might favour solutions that maximise their (potentially) narrow gains. If not consulted, they might actively block the implementation of the selected solution if it adversely impacts their interests.

The technical aspect of problem-solving is relatively easy to generate and implement without budgetary or scheduling constraints. It’s only when you consider the cost and impact of a solution that complexity arises.

9. Step 6: Implementing and Assessing Solutions

Implementing solutions to complex problems requires a structured approach that considers the unique challenges and variables involved. Effective problem-solving involves implementing practical, feasible, and sustainable solutions.

This section will first discuss two approaches to implementing solutions to complex problems: small, safe-to-fail solutions and solving easy problems with large benefits.

9.1 Implementing Many Safe-to-Fail Solutions

One effective approach to implementing solutions to complex problems is small, safe-to-fail solutions. This technique involves implementing a small-scale solution that can be tested quickly and easily to gather feedback.

Exploring multiple paths allows analysts to avoid over-commitment to suboptimal solutions.
Exploring multiple paths allows analysts to avoid over-commitment to suboptimal solutions.

Starting with small-scale solutions allows individuals to gather feedback and adjust before investing significant resources in a larger solution. This approach can save time and resources while ensuring that the final solution meets the needs of stakeholders.

Small safe-to-fail experiments effectively deal with complexity where an engineering solution is unknown priori.

9.2 Prioritizing High-Yield Solutions

Another effective approach to implementing solutions to complex problems is to first solve easy problems with large benefits. This technique involves identifying and solving simple, straightforward problems that significantly impact the overall problem.

By prioritising easy problems, individuals can progress quickly and gain momentum towards solving the larger problem. This approach can also help build trust and credibility with stakeholders, as progress is visible and measurable.

9.3 A Systematic Approach to Implementing Solutions

It is important to note that both approaches should be used with a broader problem-solving methodology. Effective problem-solving requires a systematic approach that involves identifying the problem, gathering information, analyzing data, developing and evaluating potential solutions, and implementing the best solution. By implementing small, safe-to-fail solutions and solving easy problems with large benefits, individuals can enhance their problem-solving approach and increase the likelihood of success.

In conclusion, implementing solutions to complex problems requires a structured approach that considers the unique challenges and variables involved. Implementing small, safe-to-fail solutions and solving easy problems with large benefits are two effective techniques for enhancing problem-solving. These techniques should be used with a broader problem-solving methodology to ensure the final solution is practical, feasible, and sustainable.

9.4 Implementing the Solution

Objective — This stage aims to efficiently and effectively implement the (optimal) selected solution(s).

How it’s done — Three principal techniques are required for the implementation of the solution to succeed. The first is conducting safe-to-fail experiments. The second is allocating resources to conduct each experiment. The third is setting up the criteria for success or failure.

10. Step 7: Evaluating the Solution

Objective — Solutions might work well under laboratory conditions but fail spectacularly in the field. Evaluating solutions after a trial is vital to avoid continuing investment in failed solutions.

How it’s done — The best way to evaluate a solution is to monitor the Key Performance Indicators (KPIs) originally used in the problem diagnosis. When solutions are successful, noticeable and measurable improvements should be observed.

Measuring second-order effects or observing undesirable team or business dynamics changes is key to continuing or aborting initiatives.

11. FAQ

Complex problem-solving refers to the ability to solve complex, ambiguous problems that often require creative and innovative solutions. It involves identifying the root cause of a problem, analyzing different variables and factors, developing and evaluating possible solutions, and selecting the best course of action.

Complex problem-solving is essential because it allows individuals and organizations to overcome challenges and obstacles hindering their progress and success. It enables them to identify opportunities, improve processes, and innovate to stay ahead of the competition.

To develop your complex problem-solving skills, you can practice consistently, develop a systematic approach, and leverage the right tools and resources. You can also seek feedback from others, learn from your mistakes, and adopt a growth mindset that values continuous learning and improvement.

Some common obstacles to effective problem-solving include cognitive biases, lack of information, unclear objectives, and groupthink. These obstacles can hinder individuals and teams from developing effective solutions to complex problems.

Various tools and techniques for complex problem-solving include root cause analysis, fishbone diagrams, SWOT analysis, Pareto analysis, decision trees, and scenario planning. These tools can help individuals and teams to analyze complex problems, identify underlying causes, and develop effective solutions.

To improve your decision-making skills, you can develop a structured approach, gather and analyze relevant data, evaluate different options, and consider each alternative’s potential risks and benefits. You can also seek feedback from others and reflect on your past decisions to learn from your mistakes.

Complex problem-solving skills can be applied in various aspects of your personal life, such as improving your relationships, managing your finances, and achieving your goals. You can overcome obstacles and succeed personally by systematically analyzing different variables and factors and developing creative and innovative solutions.

To overcome cognitive biases in problem-solving, you can challenge your assumptions, seek diverse perspectives, and use data and evidence to inform your decisions. You can also use brainstorming and mind-mapping techniques to generate new ideas and avoid tunnel vision.

12. Final Words

In conclusion, complex problem-solving is a crucial skill that can significantly impact your professional and personal life. It allows you to navigate complex challenges, identify the root cause of a problem, and develop effective solutions.

By mastering the art of complex problem-solving, you can enhance your critical thinking, analytical skills, and decision-making abilities, which are essential for success in today’s fast-paced and dynamic business environment.

The key to mastering complex problem-solving is to practice consistently, develop a systematic approach, and leverage the right tools and resources. With patience, persistence, and a growth mindset, anyone can become a skilled problem solver and tackle even the most challenging problems.

Leave a Reply

Your email address will not be published. Required fields are marked *