Achieving Flow in Software Teams: A Short Guide for the Busy

1. Introduction

The reader of this website will quickly recognize the immensity of The Toyota Way‘s influence on our thoughts and how we understand the services industry. Despite this powerful influence, one-piece flow, a fundamental pillar of the Toyota Production System, was systematically ignored.

Somehow, I could not relate the problems that one-piece flow solved to the day-to-day activities of software developers. At first, I dismissed it as being highly specific to manufacturing. But the curiosity lingered long enough, and new ideas have recently broken through. This article will discuss the synergy between the one-piece flow method and software development.

2. The Four Dimensions of Flow

The four dimensions of flow we discuss here are individual, team, process, and one-piece flow. One might argue that One-Piece Flow is a specific implementation of Process Flow; however, its insightfulness and innovation are so unique that it merits its own category.
The four dimensions of flow we discuss here are individual, team, process, and one-piece flow. One might argue that One-Piece Flow is a specific implementation of Process Flow; however, its insightfulness and innovation are so unique that it merits its own category.

Flow is widely used in psychology and process management, referring to employees’ perceptions, experiences, behaviour, and performance while doing a certain activity. Flow has a positive connotation, typically conveying a state of smooth and orderly progression of time, rivers, activities, and projects.

The following sections will describe flow in four separate but closely related domains: employee motivation, manufacturing processes, team flow in general, and flow in software development teams. The last of those definitions, i.e. flow in software development teams, is the one we will adopt for the remainder of this discussion.

2.1 Flow, Being in the Zone, and Employee Motivation

Dr. Mihaly Csikszentmihalyi is a Hungarian-American psychologist who authored the book “Flow – The Psychology of optimal experience”. In a paper published about the book, he defines Optimal Experience as follows:

“Optimal experience” describes those occasions where we feel a sense of exhilaration, a deep sense of enjoyment, which we cherish for long and becomes a landmark in our lives. These moments are often not passive, receptive, or relaxing times. They tend to occur when a person’s body or mind is stretched to its limits in a voluntary effort to accomplish something difficult or worthwhile.

— Mihaly Cziksentmihalyi, Flow – The Psychology of Optimal Experience

Cziksentmihalyi defines enjoyment (the outcome of having an optimal experience) when:

  • The tasks we choose should have a reasonable chance of completion, allowing for a sense of achievement and progress.
  • Setting clear and attainable goals gives us a sense of direction and purpose.
  • When we can see the direct impact of our actions through immediate feedback, it enhances our motivation and engagement.
  • It’s vital to experience a deep but effortless involvement. This state of flow removes the frustrations and worries of everyday life, allowing us to focus on the task at hand fully.
  • It feels like time melts away (alteration of the concept of time), with hours passing by in what feels like mere minutes and minutes stretching out to seem like hours.
  • We momentarily cast aside our concerns for the self, focusing solely on the task.
  • Having a sense of control over our actions is paramount. This sense of agency enables us to take ownership of our tasks and fully invest ourselves in them.

When these criteria are satisfied, we are in the zone, experiencing flow, mindfulness, and enjoyment. This is the essence of Dr. Cziksentmihalyi’s idea. Since it is not restricted to any aspect or stage of life and can apply at any time, conditions that create flow can also be used to trigger employee motivation instead of the proverbial KITA (see Frederick Herzberg’s article in HBR: One More Time: How Do You Motivate Employees?).

While essential to individual performance within a team, flow, as defined by Cziksentmihalyi, is not what we are after in this, so let’s move to the next definition.

2.1 Toyota’s One-Piece Flow

One-piece flow, developed by Taiichi Ohno, Shigeo Shingo, and Eiji Toyoda, revolutionized manufacturing at Toyota by cutting down waste and shortening lead times while maintaining low costs and high quality. So how does it work?

We examined the problems plaguing the manufacturing industry in the late twentieth century in Parts 1 and 2 of our exposition on Operational Excellence in Software Development, which gave Ohno and his senior management considerable grief.

The mass production thinking and the batch-and-queue system that worked so well for Ford were too costly and inefficient for Japanese auto manufacturers like Toyota, who did not possess the same resources or market share back then. Through painstaking observation, incessant tinkering, and incremental but persistent improvements of the manufacturing processes on the workshop floor (genba), the method was subsequently called “one-piece flow”. In its idealized form, one-piece flow sought to:

  • Eliminate inventory by producing only the needed parts.
  • Through various ingenious improvements, minimize the time required to reconfigure heavy machinery.
  • Eliminate waiting times by lining up the machines required to produce a specific part.
  • Align production frequency with customer demand to avoid overburdening the machines and the people.

Despite its unequivocal success, one-piece flow remained confined to manufacturing for many years. Applying one-piece flow in other domains, such as software development, has been relatively limited. However, there has been growing interest in recent years. Software professionals and researchers, like Nigel Thurlow, an ex-coach at Toyota, have broached the topic and written books about flow and how to achieve it in software development teams.

Thurlow’s insights from his experience at Toyota offer a unique perspective on the potential benefits of adopting flow (but perhaps not one-piece flow?) in software development. While highly original and insightful, Thurlow’s discussion on flow in software development has, in my view, not covered one-piece flow but was more in line with Cziksentmihalyi’s ideas, although applied to teams rather than individuals. In this article, we are interested more in achieving process flow, an integral aspect of operational excellence. But before moving on to process flow, it’s worth presenting Thurlow’s view.

2.3 Nigel Thurlow’s Interpretation of Flow

If you haven’t seen Nigel Thurlow’s talk on Flow, I recommend you do; insightful and engaging to say the least. Thurlow has also co-authored a book on the topic and has written papers on various ideas. Thurlow starts his flow definition by first stating what it isn’t.

Flow is not a replacement for Agile. It’s also not a framework, methodology, tool, prescription, manifesto, or annual certificate. Flow is a construct, an idea or theory containing various conceptual elements, typically considered subjective and not based on empirical evidence.

— Nigel Thurlow – Have we forgotten our key denominator: FLOW? – Lean and Agile Summit 2022

Thurlow extends Cziksentmihalyi’s definition of flow, which is mostly individualistic, to cover the experiences of a team that can share the same enjoyment, albeit as a group.

Flow comes from the interaction with others when structure and practice become unnoticed, and all you are conscious of is the act of being.

— Nigel Thurlow – Have we forgotten our key denominator: FLOW? – Lean and Agile Summit 2022

This powerful statement beautifully captures the essence of group flow as a feeling that members experience when they become unaware of the hierarchy and any other structural differences between them. They also become unconscious of the formal processes expected to govern their interactions. The only subject of their attention is the act of doing.

2.4 Operational Excellence and Process Flow

So far, we have discussed individual, team, and manufacturing process flow. What is missing is an equivalent formulation for flow in software teams. While employee and team motivation is paramount for success, what is relevant to our discussion here is, in fact, not that it is the “process flow” or the ability of a software team to deliver while simultaneously navigating powerful and unpredictable currents.

Imagine you have pipes, valves, pumps and other plumbing equipment installed in an intertwined and complicated network. Water flows from one end of the network to another while a sophisticated computer system disassembles and reassembles the network’s components to adapt to changing flow requirements. While the network operates:

  • It must not break, abort, or produce bottlenecks or inefficiencies.
  • It must promptly and effectively respond to changing inputs and output requirements.

While the laws of fluid mechanics that govern the flow of water through the plumbing system will never change, the architecture of the network, the connectivities between the nodes, and the feedback loops created are constantly changing in response to many internal and external constraints.

This is what it feels like to work on complicated software projects, where developers have a restricted and localised view of the network they are embedded in while the latter constantly evolves. How can developers keep work smoothly flowing amidst this chaos?

3. Flow and Software Development

3.1 What is Flow in Software Development?

A matrix organisation that allows cross-functional teams to assemble and disassemble during projects is a great start to achieving flow but misses other fundamental pillars.
A matrix organisation that allows cross-functional teams to assemble and disassemble during projects is a great start to achieving flow but misses other fundamental pillars.

Let’s set the scene with the following questions:

  • How much of the lessons learned at Toyota can be transferred to the services industry, especially software?
  • Is there an equivalent to the one-piece flow method?
  • If yes, how would one-piece flow assist software developers in improving deliveries?

The answer to the first question was articulated in Parts 1 and 2 of our discussions on Operational Excellence in Software Development; below are our thoughts on the other two.

Flow in Software Teams

Flow in software teams can be attained by forming cross-functional teams, installing specialized and well-defined roles, and shortening user stories. Furthermore, members of such teams must have access to leadership, work within hierarchies capable of effective decision-making, and belong to dense social networks from which they can draw support. Leaders of such teams must derive their knowledge from the genba.

Let’s break down that statement into its fundamental constituents and discuss these separately, showing how their synergy creates flow.

3.2 Cross-Functional Teams

Cross-functional derive their composition from a pool of technical resources that allow the team to become self-sufficient in decision-making on how features can be designed, implemented, and tested.
Cross-functional derive their composition from a pool of technical resources that allow the team to become self-sufficient in decision-making on how features can be designed, implemented, and tested.

Melvin Conway’s law states that the architecture of an application often reflects the organizational structure responsible for its design. This means an organisation’s communication structures and hierarchies tend to be mirrored in the architecture of the produced software. This concept has significant implications for effectively structuring and organising development teams.

Unlike traditional teams formed based on functional departments (e.g., marketing, engineering, finance), a cross-functional team is a group of individuals with diverse skills, knowledge, and expertise who work on a specific project, task, or goal within an organization.

Cross-Functional Teams

Cross-functional teams are often utilized in situations that require a multidisciplinary approach, where no single department or function can fully address the complexity of the task. By bringing together individuals with different backgrounds, skills, and viewpoints, cross-functional teams create the necessary diversity for innovation and quick and effective decision-making.

The challenges of organisational silos are inherited from mass-production thinking. In mass production, an assembly line is optimized to produce a large quantity of spare parts quickly and efficiently. However, reconfiguring the machines on the line to produce a different spare part can be time-consuming. This is because the organization of the factory is centred around the division of labour and specialization of tasks.

To address this issue, one-piece flow requires arranging the machines needed to produce an entire car to minimise the movement of materials and machine reconfiguration. This means that each machine on the line can perform various tasks and can be quickly adjusted to produce a different part.

Cross-Functional Software Teams

Similarly, in software development, cross-functional teams aim to have all the necessary skills and expertise within one team to develop an entire feature from start to finish. This includes activities such as data modelling, design, test case creation, and code implementation.

While the siloed structure of an organization may have its advantages (division of labour, specialisation of skills and knowledge, planning flexibility), adopting the Agile principle of cross-functional teams offers numerous benefits in terms of flexibility, efficiency, collaboration, and knowledge sharing.

It’s important to note that a cross-functional team must have interdependent work to be effective. Otherwise, it’s no better than a regular team or an individual worker. Interdependent work means real-time collaboration, like in a football match or when delivering a complex feature where client preferences and requirements co-evolve with the feature being implemented.

3.3 Specialized and Well-Defined Roles

Specialising team members in different application areas or competencies is essential to team growth and scalability. When team members have well-defined roles and responsibilities, it creates a more efficient and productive environment.

  • By defining roles with clear and solid boundaries, there is little overlap in duties, reducing the risk of duplication or neglected tasks. This role clarity also fosters a sense of accountability, as individuals are responsible for specific outcomes and can be held responsible for their performance.
  • Having well-defined roles enables crews to assemble and disassemble based on project needs. This flexibility allows for a more agile approach to projects, as teams can be quickly formed and start working immediately without going through extensive team-building phases (forming, storming, norming, performing).
  • Additionally, well-defined roles help to streamline communication and decision-making processes. When team members have clear responsibilities, it becomes easier to identify and approach the right person for specific issues or decisions. This reduces the time and effort spent on unnecessary communication and ensures that the appropriate individuals make decisions with the relevant expertise.

3.4 Short User Stories

User stories are a powerful tool in software development, providing a concise yet informative description of a feature or functionality from the user’s perspective. However, not all user stories are created equal.

To understand the significance of short user stories, let’s consider the metaphor proposed by Dr. Liker in The Toyota Way. Imagine a body of water representing the development process, wherein quality issues act as hidden rocks beneath the surface. In large projects or user stories, these rocks may go unnoticed for an extended period, like a defective part that remains concealed within a vast inventory.

The lengthy lead time and extensive implementation effort associated with large stories often overshadow the underlying problems in the process, making problem identification in production processes harder.

Conversely, the adoption of short user stories allows for a different approach. By breaking down user requirements into smaller, more manageable units, we create a scenario where the lead time and implementation effort are typically minimal. Should any issues arise during the implementation of these short user stories, they can be swiftly identified and addressed. Consequently, if the lead time or implementation effort becomes significant unexpectedly, it is a clear signal of a potential fault or inefficiency in the overall process.

In essence, short user stories serve as a litmus test for the efficiency and effectiveness of the development process. If you want to estimate the overhead or waste of your production process, try implementing a user story with no business changes. A small technical change would do. Then, measure the lead time and the time-to-market, and there would be your overhead.

3.5 Access to Leadership

Hierarchies and what they represent in terms of power, influence, and status are here to stay. Moreover, people in the higher echelons of the hierarchy have little time to spare and tens or hundreds of decisions to make every day. The paradox, however, comes from the necessity of knowledge and being close to where the action is. The more senior a leader is, the more sparse, distorted, and filtered their view of what’s happening on the ground.

So, how can leaders stay informed without drowning in a sea of noise? How can leaders decide without complete knowledge from the genba? Here are a few thoughts:

  • Genchi Genbutsu (現地現物) is a Japanese term often used in the context of the Toyota Production System (TPS) and lean manufacturing principles. It can be translated to mean “go and see for yourself“. Genchi Genbutsu emphasizes the importance of going to the actual location of a problem or process to understand it thoroughly. It encourages people, particularly managers and decision-makers, to be directly involved and engaged with the real situation on the ground rather than relying solely on reports, data, or second-hand information. By physically going to the place where the work is being done, practitioners can gather first-hand information, observe the processes, identify issues, and make informed decisions based on a deep understanding of the situation.
  • Adopting a matrix organisation. A matrix organization is a management structure that combines elements of both functional and project-based organizational structures. In a matrix organization, employees report to a functional manager (based on their expertise or department) and a project manager (based on the specific project they are working on). This creates a dual reporting relationship and can offer several advantages. A similar structure could be set up with a flat direct report hierarchy and a deep functional/technical one. The line manager manages the staff (promotion, leaves, evaluations, recruitment, P&L), whereas the technical lead oversees functional duties. Needless to say, the technical lead must enjoy the line manager’s trust and must be able to demonstrate his or her competency for this setup to work.

In both of these approaches, the gap between managers and employers is reduced, new communication channels open up, and reliable sources of information are harnessed. This paradigm shift in management allows for a more inclusive and collaborative work environment where employees feel valued and heard and are encouraged to contribute their unique perspectives.

Additionally, the flow of information is no longer restricted to a one-way street from managers to employees. Rather, it becomes a bidirectional exchange, where employees can provide feedback and ideas, leading to better decision-making and problem-solving. This inclusive communication culture fosters creativity and innovation, as diverse perspectives are heard and valued.

Furthermore, a more decentralized management structure can be implemented, where technical managers play a pivotal role in bridging the gap between line managers and higher-level leadership. These technical managers can guide and support line managers with their specialised expertise, allowing for a flatter hierarchy while maintaining productivity and efficiency. This way, line managers can focus on empowering their teams and nurturing individual growth without being burdened by excessive administrative tasks.

3.6 Hierarchies for Effective Decision-Making

In the previous section, we explored some concepts on how leadership can be close to the genba, or where the action is while retaining the lucidity that allows efficient and effective decision-making. This section will focus on two distinct but closely related ideas that are enormously popular, especially in the Agile community. These two ideas are self-organisation in teams and how leaders decide.

One of the most revolutionary ideas of Agile was the notion of self-organisation in teams, which promised great rewards if the division of labour and the decision-making processes were allowed to emerge from internal self-organisation processes rather than being externally engineered and imposed by management.

This concept, however, falls short of its promises for the following reasons:

  • First, as Professor Jeffery Pfeffer constantly emphasises in his lectures, hierarchies are inevitable; if management doesn’t impose them, the teams will select one, not necessarily along the best possible routes. This means that self-organisation and hierarchies are not incompatible but can have a compounding effect.
  • Second, teams given a chance to self-organise will necessarily go through forming, storming, norming, and performing stages. The formative stages before performing require time and patience to complete, during which the team’s performance might be poor. This can be alleviated with assemblages of crews based on well-defined roles, as discussed previously.
  • Thirdly, the formative stages do not always happily conclude; the team might disintegrate if the internal cohesion forces do not overcome internal integration challenges and external threats. Behaviour must be moderated through incentives and rewards in complex adaptive systems to which social groups belong. This moderation guides the evolution of the team along the right path.
  • Finally, the hierarchy that emerges from self-organisation might look after the power brokers’ interests rather than the group’s collective interest. While power, status, and influence, concentrated in the hands of a minority, are inevitable with hierarchies, the latter’s formation is best guided by sound principles that place the group’s interest over that of specific individuals.

The matrix-organisation variation for small teams proposed in the previous section can avoid the issues that emerge from unguided self-organisation by placing a line manager at the top, assisted by a technical lead accountable to the line manager.

The line manager has a broad spectrum of leadership methods that range from leader-centred to subordinate-centred with all the gradations in between. In the former, leaders make decisions and announce them without consulting with their team. This method is critical during crises where decisions must be made quickly under great uncertainty. In the former case, subordinates have complete freedom in deciding their matters. This method is crucial for long-term objectives that impact how the team works.

The leader can decide when to use their authority and when to delegate it collectively to the team. In these cases and all those in between, the matrix organisation or line manager, technical lead, and crew assemblage can offer significant richness in how decisions are made without limiting the manager or the team in the methods at their disposal.

3.7 Social Networks and Support

Dave Snowden, a renowned expert on complexity and the creator of the Cynefin framework, conducted extensive studies that shed light on the dynamics of informal networks within organizations. His research revealed an interesting phenomenon: informal networks increase when formal processes and hierarchies become more rigid.

Unlike formal hierarchies defined by organizational charts and reporting lines, informal networks comprise relationships and connections outside these predefined structures. They are characterized by the bonds between individuals who share common interests, expertise, or simply a mutual desire to collaborate and exchange knowledge.

One of the primary advantages of informal networks is their ability to facilitate information sharing and problem-solving across organizational boundaries. When faced with complex issues or unfamiliar challenges, individuals can tap into these networks to seek advice, gather insights, and access expertise that might not be readily available within their immediate teams or departments. This informal knowledge exchange and experience can often lead to creative solutions and innovative approaches.

This approach to organisational structure in its informal manifestation is a radical departure from the manufacturing model. In the latter, teams perform dependent tasks where the output of Task A feeds into Task B. Tasks A and B might coevolve in software teams, and coordination between teams A and B is not sufficient; collaboration is needed.

4. Summary

  • What is flow? In this context, flow refers to the unobstructed, smooth progress of individual and team activities towards a common business goal despite a potentially turbulent environment.
  • How does flow relate to operational excellence? Operational excellence is an organisation’s ability to realize the leadership’s vision. It involves thoroughly understanding the business, its processes and people, human and organisational behaviour and psychology. Flow, as we used it in this article, is operational excellence on the team level.
  • How can teams achieve flow? Flow can be attained through a) cross-functional teams, 2) specialized and well-defined roles, 3) short user stories, 4) direct and easy access to leadership, 5) effective decision-making processes, 6) functioning hierarchies, and 7) dense social networks.
  • What is the significance of short user stories in flow? Short user stories are the equivalent of lowering the water levels so that all rocks surface. Quality issues are more readily hidden in large implementations. On the other hand, unreasonable overheads, inefficiencies, and quality issues immediately impose themselves when changes are small enough.
  • How does mindset impact flow? Mindset is an individual and subjective view of the world that constantly evolves. People behave according to their worldview; the environment’s response to their actions will reshape their worldviews. Influencing a team member’s behaviour requires changing the rules of engagement with their peers and managers.

5. References

Leave a Reply

Your email address will not be published. Required fields are marked *