Achieving Flow in Software Teams: A Short Guide for the Busy

1. Introduction

The reader of this website will quickly recognize the immensity of The Toyota Way‘s influence on our thoughts and how we understand the services industry. Despite this powerful influence, one-piece flow, a fundamental pillar of the Toyota Production System, was systematically ignored.

Somehow, I could not relate the problems that one-piece flow solved to the day-to-day activities of software developers. At first, I dismissed it as being highly specific to manufacturing. But the curiosity lingered long enough, and new ideas have recently broken through. This article will discuss the synergy between the one-piece flow method and software development.

2. The Four Dimensions of Flow

Flow is widely used in psychology and process management, referring to employees’ perceptions, experiences, behaviour, and performance while doing a certain activity. Flow has a positive connotation, typically conveying a state of smooth and orderly progression of time, rivers, activities, and projects.

The following sections will describe flow in four separate but closely related domains: employee motivation, manufacturing processes, team flow in general, and flow in software development teams. The last of those definitions, i.e. flow in software development teams, is the one we will adopt for the remainder of this discussion.

2.1 Flow, Being in the Zone, and Employee Motivation

Dr. Mihaly Csikszentmihalyi is a Hungarian-American psychologist who authored the book “Flow – The Psychology of optimal experience”. In a paper published about the book, he defines Optimal Experience as follows:

Cziksentmihalyi defines enjoyment (the outcome of having an optimal experience) when:

When these criteria are satisfied, we are in the zone, experiencing flow, mindfulness, and enjoyment. This is the essence of Dr. Cziksentmihalyi’s idea. Since it is not restricted to any aspect or stage of life and can apply at any time, conditions that create flow can also be used to trigger employee motivation instead of the proverbial KITA (see Frederick Herzberg’s article in HBR: One More Time: How Do You Motivate Employees?).

While essential to individual performance within a team, flow, as defined by Cziksentmihalyi, is not what we are after in this, so let’s move to the next definition.

2.1 Toyota’s One-Piece Flow

One-piece flow, developed by Taiichi Ohno, Shigeo Shingo, and Eiji Toyoda, revolutionized manufacturing at Toyota by cutting down waste and shortening lead times while maintaining low costs and high quality. So how does it work?

We examined the problems plaguing the manufacturing industry in the late twentieth century in Parts 1 and 2 of our exposition on Operational Excellence in Software Development, which gave Ohno and his senior management considerable grief.

The mass production thinking and the batch-and-queue system that worked so well for Ford were too costly and inefficient for Japanese auto manufacturers like Toyota, who did not possess the same resources or market share back then. Through painstaking observation, incessant tinkering, and incremental but persistent improvements of the manufacturing processes on the workshop floor (genba), the method was subsequently called “one-piece flow”. In its idealized form, one-piece flow sought to:

Despite its unequivocal success, one-piece flow remained confined to manufacturing for many years. Applying one-piece flow in other domains, such as software development, has been relatively limited. However, there has been growing interest in recent years. Software professionals and researchers, like Nigel Thurlow, an ex-coach at Toyota, have broached the topic and written books about flow and how to achieve it in software development teams.

Thurlow’s insights from his experience at Toyota offer a unique perspective on the potential benefits of adopting flow (but perhaps not one-piece flow?) in software development. While highly original and insightful, Thurlow’s discussion on flow in software development has, in my view, not covered one-piece flow but was more in line with Cziksentmihalyi’s ideas, although applied to teams rather than individuals. In this article, we are interested more in achieving process flow, an integral aspect of operational excellence. But before moving on to process flow, it’s worth presenting Thurlow’s view.

2.3 Nigel Thurlow’s Interpretation of Flow

If you haven’t seen Nigel Thurlow’s talk on Flow, I recommend you do; insightful and engaging to say the least. Thurlow has also co-authored a book on the topic and has written papers on various ideas. Thurlow starts his flow definition by first stating what it isn’t.

Thurlow extends Cziksentmihalyi’s definition of flow, which is mostly individualistic, to cover the experiences of a team that can share the same enjoyment, albeit as a group.

This powerful statement beautifully captures the essence of group flow as a feeling that members experience when they become unaware of the hierarchy and any other structural differences between them. They also become unconscious of the formal processes expected to govern their interactions. The only subject of their attention is the act of doing.

2.4 Operational Excellence and Process Flow

So far, we have discussed individual, team, and manufacturing process flow. What is missing is an equivalent formulation for flow in software teams. While employee and team motivation is paramount for success, what is relevant to our discussion here is, in fact, not that it is the “process flow” or the ability of a software team to deliver while simultaneously navigating powerful and unpredictable currents.

Imagine you have pipes, valves, pumps and other plumbing equipment installed in an intertwined and complicated network. Water flows from one end of the network to another while a sophisticated computer system disassembles and reassembles the network’s components to adapt to changing flow requirements. While the network operates:

This is what it feels like to work on complicated software projects, where developers have a restricted and localised view of the network they are embedded in while the latter constantly evolves. How can developers keep work smoothly flowing amidst this chaos?

3. Flow and Software Development

3.1 What is Flow in Software Development?

Let’s set the scene with the following questions:

The answer to the first question was articulated in Parts 1 and 2 of our discussions on Operational Excellence in Software Development; below are our thoughts on the other two.

Let’s break down that statement into its fundamental constituents and discuss these separately, showing how their synergy creates flow.

3.2 Cross-Functional Teams

Melvin Conway’s law states that the architecture of an application often reflects the organizational structure responsible for its design. This means an organisation’s communication structures and hierarchies tend to be mirrored in the architecture of the produced software. This concept has significant implications for effectively structuring and organising development teams.

Unlike traditional teams formed based on functional departments (e.g., marketing, engineering, finance), a cross-functional team is a group of individuals with diverse skills, knowledge, and expertise who work on a specific project, task, or goal within an organization.

The challenges of organisational silos are inherited from mass-production thinking. In mass production, an assembly line is optimized to produce a large quantity of spare parts quickly and efficiently. However, reconfiguring the machines on the line to produce a different spare part can be time-consuming. This is because the organization of the factory is centred around the division of labour and specialization of tasks.

To address this issue, one-piece flow requires arranging the machines needed to produce an entire car to minimise the movement of materials and machine reconfiguration. This means that each machine on the line can perform various tasks and can be quickly adjusted to produce a different part.

While the siloed structure of an organization may have its advantages (division of labour, specialisation of skills and knowledge, planning flexibility), adopting the Agile principle of cross-functional teams offers numerous benefits in terms of flexibility, efficiency, collaboration, and knowledge sharing.

It’s important to note that a cross-functional team must have interdependent work to be effective. Otherwise, it’s no better than a regular team or an individual worker. Interdependent work means real-time collaboration, like in a football match or when delivering a complex feature where client preferences and requirements co-evolve with the feature being implemented.

3.3 Specialized and Well-Defined Roles

Specialising team members in different application areas or competencies is essential to team growth and scalability. When team members have well-defined roles and responsibilities, it creates a more efficient and productive environment.

3.4 Short User Stories

User stories are a powerful tool in software development, providing a concise yet informative description of a feature or functionality from the user’s perspective. However, not all user stories are created equal.

To understand the significance of short user stories, let’s consider the metaphor proposed by Dr. Liker in The Toyota Way. Imagine a body of water representing the development process, wherein quality issues act as hidden rocks beneath the surface. In large projects or user stories, these rocks may go unnoticed for an extended period, like a defective part that remains concealed within a vast inventory.

Conversely, the adoption of short user stories allows for a different approach. By breaking down user requirements into smaller, more manageable units, we create a scenario where the lead time and implementation effort are typically minimal. Should any issues arise during the implementation of these short user stories, they can be swiftly identified and addressed. Consequently, if the lead time or implementation effort becomes significant unexpectedly, it is a clear signal of a potential fault or inefficiency in the overall process.

In essence, short user stories serve as a litmus test for the efficiency and effectiveness of the development process. If you want to estimate the overhead or waste of your production process, try implementing a user story with no business changes. A small technical change would do. Then, measure the lead time and the time-to-market, and there would be your overhead.

3.5 Access to Leadership

Hierarchies and what they represent in terms of power, influence, and status are here to stay. Moreover, people in the higher echelons of the hierarchy have little time to spare and tens or hundreds of decisions to make every day. The paradox, however, comes from the necessity of knowledge and being close to where the action is. The more senior a leader is, the more sparse, distorted, and filtered their view of what’s happening on the ground.

So, how can leaders stay informed without drowning in a sea of noise? How can leaders decide without complete knowledge from the genba? Here are a few thoughts:

In both of these approaches, the gap between managers and employers is reduced, new communication channels open up, and reliable sources of information are harnessed. This paradigm shift in management allows for a more inclusive and collaborative work environment where employees feel valued and heard and are encouraged to contribute their unique perspectives.

Additionally, the flow of information is no longer restricted to a one-way street from managers to employees. Rather, it becomes a bidirectional exchange, where employees can provide feedback and ideas, leading to better decision-making and problem-solving. This inclusive communication culture fosters creativity and innovation, as diverse perspectives are heard and valued.

Furthermore, a more decentralized management structure can be implemented, where technical managers play a pivotal role in bridging the gap between line managers and higher-level leadership. These technical managers can guide and support line managers with their specialised expertise, allowing for a flatter hierarchy while maintaining productivity and efficiency. This way, line managers can focus on empowering their teams and nurturing individual growth without being burdened by excessive administrative tasks.

3.6 Hierarchies for Effective Decision-Making

In the previous section, we explored some concepts on how leadership can be close to the genba, or where the action is while retaining the lucidity that allows efficient and effective decision-making. This section will focus on two distinct but closely related ideas that are enormously popular, especially in the Agile community. These two ideas are self-organisation in teams and how leaders decide.

This concept, however, falls short of its promises for the following reasons:

The matrix-organisation variation for small teams proposed in the previous section can avoid the issues that emerge from unguided self-organisation by placing a line manager at the top, assisted by a technical lead accountable to the line manager.

The line manager has a broad spectrum of leadership methods that range from leader-centred to subordinate-centred with all the gradations in between. In the former, leaders make decisions and announce them without consulting with their team. This method is critical during crises where decisions must be made quickly under great uncertainty. In the former case, subordinates have complete freedom in deciding their matters. This method is crucial for long-term objectives that impact how the team works.

The leader can decide when to use their authority and when to delegate it collectively to the team. In these cases and all those in between, the matrix organisation or line manager, technical lead, and crew assemblage can offer significant richness in how decisions are made without limiting the manager or the team in the methods at their disposal.

3.7 Social Networks and Support

Dave Snowden, a renowned expert on complexity and the creator of the Cynefin framework, conducted extensive studies that shed light on the dynamics of informal networks within organizations. His research revealed an interesting phenomenon: informal networks increase when formal processes and hierarchies become more rigid.

Unlike formal hierarchies defined by organizational charts and reporting lines, informal networks comprise relationships and connections outside these predefined structures. They are characterized by the bonds between individuals who share common interests, expertise, or simply a mutual desire to collaborate and exchange knowledge.

One of the primary advantages of informal networks is their ability to facilitate information sharing and problem-solving across organizational boundaries. When faced with complex issues or unfamiliar challenges, individuals can tap into these networks to seek advice, gather insights, and access expertise that might not be readily available within their immediate teams or departments. This informal knowledge exchange and experience can often lead to creative solutions and innovative approaches.

This approach to organisational structure in its informal manifestation is a radical departure from the manufacturing model. In the latter, teams perform dependent tasks where the output of Task A feeds into Task B. Tasks A and B might coevolve in software teams, and coordination between teams A and B is not sufficient; collaboration is needed.

4. Summary

5. References

• Process Consultation — Its Role in Organisation Development by Dr Edgar Schein
• Strategic Management and Organisational Dynamics — The Challenge of Complexity to Ways of Thinking About Organisations by Dr Ralph Stacey
• Core Concepts in Cultural Anthropology by Drs Robert Lavenda and Emily Schultz