Software Development and Delivery and The Story of an Engineer

1. Humble Beginnings and a Desire to Achieve

While he waited for his morning coffee, Achilles’ mind wandered far back into the past, with faint memories of his hometown on the eastern coast of the Mediterranean flashing before his eyes.

What a journey it was! He was now a proud veteran software engineer working for a multinational organisation, the size of which he could only have dreamed of even a few years ago.

The journey was arduous, tortuous, and exhausting, but not more than Achilles’ desire to achieve.

His world two decades ago was homogenous, tiny, and easy to understand. Causes and effects were readily correlated, technical proficiency and higher education were highly regarded, human relationships and networking were never a topic of relevance, and organisational culture was an alien concept. He had no clue of what lay ahead.

Soon after graduation, Achilles accepted an offer from a large corporation for a software programmer position, where he spent many years.

Although he was trained as an industrial engineer, software programming seemed like a nice extension that was within his grasp, and no more than a little effort was anticipated to fill the gap. How far from the truth those thoughts were!

Achilles walked down the avenue leading to his modern office with his hot black coffee, now sitting comfortably in both hands. The wind was chilly, and the morning was fresh.

2. The Messy State of Software Development

Achilles often wondered whether the challenges he faced as a software programmer were the product of his behaviour or attitude, whether they were unique to his situation or context, or were just manifestations of a deeper and more pervasive issue in the software industry.

In particular, the following doubts lingered for a long time.

Achilles slowly figured out the answers to these questions primarily through serendipity. Still, the latter can only work if the beneficiary is already on a quest to find the truth, armed with an open mind, psychological safety and critical thinking.

Achilles was lucky enough to acquire those three skills while his mind matured, shaped by (sometimes traumatic) experience.

An open mind means giving new ideas a fair go, an enzyme for change. With psychological safety, one can readily admit mistakes, learn from them, and move on without fear of repercussions.

Finally, critical thinking is the ability to emerge from one’s subjective existence, objectively examine one’s reality, notice fundamental cracks in the cultural, societal, or value edifice (should the need arise), seek explanations and return with an action list.

With psychological safety, critical thinking is more accessible as people can safely expose themselves to the truth.

3. The Newtonian View of the World

The engineering courses that Achilles was taught focused on hard sciences like physics, chemistry, and mathematics, with electrical, mechanical, software, and civil engineering theories built on top.

Exciting and relevant as they may have been, such studies did not explain the entire spectrum of phenomena that Achilles participated in creating during his career.

Engineers work in groups of organisations, constantly interacting with a changing and complex human environment.

The mechanistic mental models that Achilles relied on were deprived of crucial knowledge of human nature and complexity theory and could not explain creativity, faulty judgements (cognitive biases), and what appeared to be the irrational behaviour of his peers. These shortcomings caused Achilles much grief.

The discrepancy between a deterministic view of the world (where the laws of nature are immutable and causality is simple and evident) and a messy, complex one was agonising.

On the one hand, Achilles remained convinced that rational decision-making processes enabled him (and his managers) to effectively control the long-term evolution of complex social groups (like teams and organisations).

Therefore, Achilles maintained for a while that there was no need to delve into natural sciences or anything remotely connected with human nature and groups.

As Peter Drucker wrote, there was a feeling of disdain even for anything related to natural or human sciences. In Achilles’ mind, a corollary of that assumption was that relationships were subordinate and inferior to technical proficiency, and teamwork, networking, and organisational culture were irrelevant.

On the other hand, his colleagues refused to play nice, his managers’ decisions seemed erratic, and the world remained unorderly and unpredictable.

But Achilles was lucky. Occasionally, through serendipity and other times through research, he pieced together a larger puzzle, a framework with better explanatory powers of how the world functioned.

After that, Achilles’ daily interactions were smoother, and anxiety hit much lower levels.

It was a great relief for Achilles when he finally acknowledged that absolute control of every detail of his world was utterly and desperately illusory.

4. Organisational Culture and Group Integration

After the first few weeks in his new job in early 2000, Achilles had made a solid impression of what life in that organisation meant.

Senior members of his team recounted many stories of extraordinary effort, late-night calls with clients, and extra long days of diligent coding to meet critical deadlines. He was often reminded that these criteria were vital if he wished to be successful.

These assumptions on the nature of satisfactory work and achievement puzzled Achilles for a long time. Wasn’t the motto always “work smart, not hard”? Surely his peers were aware of that concept already? Does working late not reflect an endemic state of fire-fighting rather than fire prevention?

Achilles had to wait many years before he came to appreciate the full potential of corporate narratives and organisational culture.

As it turned out, late nights at the office did not always signal faulty processes or a lack of planning. In most cases, it was a way of expressing personal motivation and a genuine commitment to the team and organisation by demonstrating the will to go above and beyond. It was a ritual that heralded complete integration into the team’s culture.

On the other hand, Achilles believed that innovative, quality work would eventually pay off. He did not feel comfortable bowing to the mandates of what he saw as the arbitrary rules of organizational culture.

These actions resonated with his core values, centred around enjoying time with friends and family, personal growth outside work, and work-life balance.

Achilles made slow progress in this organisation and never felt like a legitimate team member despite his many years there. He eventually moved on to new roles where he could grow.

However, the lessons learned were immense, although making sense of them took years longer than expected. Achilles also made lifelong friends whom he called on Christmas and other Middle Eastern high holidays.

5. Context Shift and New Challenges

A fragile system, including almost all artificial objects (like tea cups or tables) and machines (like laptops and hair dryers), will buckle and break under pressure.

In contrast, Nassim Taleb’s brainchild, an antifragile system (like all living organisms, including nature), thrives under bounded stressors and slowly decays in sedentary lifestyles.

When subjected to minor, localized, but destructive environmental variations, subcomponents of a complex system will break down, fade away, and be replaced with younger, more robust components that immunize the system against future pressures of a similar magnitude.

Although he did not quite understand them in those terms, Achilles interpreted his experiences along similar lines, frequently quoting Nietzsche, “Out of life’s school of war—what doesn’t kill me, makes me stronger”.

During his early years as an immigrant, rife with anxieties, Achilles underwent a series of paradigm shifts, an inner transformation.

Transformations are always traumatic, especially since, for people with high self-awareness, changes of such fundamental nature and magnitude directly threaten one’s identity and integrity.

Achilles learned to live with multiple identities that didn’t necessarily conflict with or compromise his core beliefs and values system.

Managing and deploying those identities in the relevant context was an important learning exercise. Achilles managed, nevertheless, with the help of his medium-sized library (which looks massive relative to his apartment) and a small but supportive environment.

Archaic ideas and Newtonian-anchored perceptions made way for modern, sophisticated, and complex paradigms better suited to deal with Achilles’ world.

Achilles carries his scars with pride; he is a survivor.

6. Things Are Not as Simple as They Seem

It’s comical and tragic at the same time when people are compelled to state something as apparent and trivial as “things are not as simple as they seem”. Yet, we often unwittingly assume normality, causality, predictability, agency, and familiarity with every situation. In the (modern) world we created, a reductionist approach like this can never work.

But, we are invariably motivated to make these assumptions as they allow us to efficiently make decisions (using cognitive biases or heuristics) that (almost) always work, especially where inconsequential choices are involved. The alternative (engaging rational and slow reasoning processes) is costly and inefficient.

But if things are not always straightforward, what can they be? Natural sciences tell us that systems fall into one of four groups: ordered, random, complicated, and complex. Because we are not always sure which situation we are dealing with, we have a fifth alternative: confusion (see Cynefin framework).

Natural systems abhor randomness as the cost of maintaining it is utterly high. Soon enough, agents within the system will loosely bond and self-organise, and behavioural patterns will emerge. We now have complexity.

Causality, agency, familiarity, and predictability are difficult to obtain in complex systems, but that’s not the problem. The fundamental problem lies in working in a complex system while assuming it’s ordered. Here, we apply strategies that (always!) lead to unintended, second-order consequences.

This website and its ideas on business management and software development are a testament to this author’s journey and the milestones achieved when searching for the fundamental questions of business management, business strategy, and software development.

7. First Principles and Literature Research

The Eastern Mediterranean populations are not famous for their scepticism or questioning of traditions and legacy thoughts (although this was quite the opposite in Greek antiquity and medieval Arabia, where philosophy was born and matured).

Born and bred in the land of figs and olive trees, Achilles was destined to inherit the attributes of the local culture; luckily, it happened only partially. He attributed his sceptical approach to mainstream thinking purely to luck, as the high variance of characteristics is a common and natural phenomenon within a population.

Achilles was rarely satisfied with superficial explanations that did not get to the core of the matter. He was also not scared to stray from group thinking to avoid exclusion.

When exclusion was inevitable, he found solace through reading scientific texts and literature research, primarily by returning to the first principles.

Returning to the first research principles involves examining fundamental assumptions and building knowledge from foundational concepts. There are several benefits of this approach:

Returning to the first principles forces the researcher to examine the fundamental assumptions. This examination must be precise and technical for it to bear fruit.

When you describe a phenomenon in precise technical terms, you cast aside any emotional biases that might influence your hypothesis about it. This technique is instrumental in finding the truth.

8. The Software Engineering Toolkit

8.1 What Is Software Engineering

Software engineering involves designing, developing, and maintaining software using engineering principles and methods. In today’s world, software engineering plays a crucial role in every aspect of our lives, from mobile applications to enterprise systems. In this article, we’ll explore the different aspects of software engineering, including its processes, principles, key concepts, best practices, tools, challenges, and future outlook.

Software Engineering Processes Software engineering processes refer to the methods and frameworks used to design and develop software. Some of the popular software engineering processes include:

8.2 Software Engineering Principles

Software engineering principles refer to the fundamental concepts and practices that guide the design and development of software. Some of the key software engineering principles include:

8.3 Key Software Engineering Concepts

Software engineering concepts refer to the techniques and practices used in software development. Some of the key software engineering concepts include:

8.4 Best Practices in Software Engineering

Software engineering best practices refer to the techniques and methods to ensure high-quality and maintainable software. Some of the popular software engineering best practices include:

8.5 Challenges in Software Engineering

Despite the benefits of software engineering, there are also several challenges associated with it. Some of the popular challenges include:

8.6 Future Outlook for Software Engineering

Software engineering is dynamic, and several emerging trends are expected to shape its future. Some of the popular trends include:

9. Final Words

Our website’s series of articles tackles the daily age-old and modern challenges software engineers face. The article will cover three broad topics:

All this written knowledge aims to help software engineers achieve Operational Excellence, a concept that started with manufacturing but may as well extend to any human endeavour.

We hope you find this information helpful and derive as much joy from reading it as I did writing it.