Part 5: High-Level Solution Design Documents: What Is It and When Do You Need One

Georges Lteif

Software Engineer

Last Updated on January 19, 2023.
Subscribe now to stay posted!
About Us
14 min read

---

1. Overview

All large IT projects will require a High-Level Solution Design, also known as High-Level Design or HLD, an artifact instrumental to the Software Development Lifecycle analysis phase.

The HLD gives the stakeholders a bird’s eye view of the system once the implementation is completed. At its core is the proposed solution architecture, designed to solve a business problem and generate customer value.

Architecting a solution generates different concepts and selects an optimal candidate that satisfies budgetary and planning constraints while mitigating project risk.

This article examines all aspects of a High-Level Design for a project. It will give software practitioners an idea of when, why, and how to create a valuable HLD.

A few notes are due before we proceed:

• Development vs Integration — High-Level Designs (HLD) work best for system integration projects where a solution is assembled from mature technology and products. The components of this solution and the interfaces joining them might need to be customized for every implementation. However, no substantial development effort is expected in such undertakings, in contrast to what we see in green field assignments.

• Designing an IT solution — We recommend reading the Solution Design article for context and background. A design exercise uses an iterative approach to determine the best technical solution, ensuring that functional requirements and non-functional constraints are adequately understood, documented, and satisfied.

• High vs Low-Level Design — The HLD should not be confused with the Low-Level Design (or LLD), whose primary role is to decompose the systems defined in the HLD into subsystems, subcomponents and, most importantly, select optimal values for key design and operations parameters laid out in the architecture.

• Agile Architecture and Design — Can a design or architecture be Agile? The answer is Yes, and it has been discussed thoroughly on the two topics of solution architecture and software architecture. Depending on the context, a top-to-bottom or a bottom-up approach can be used for building large software systems. Requirement volatility compels you to use Agile methodologies, while Waterfall is perfectly appropriate for familiar and classical implementations. More on that later.

---

2. Pre-requisites

Primo —

• If this is your first time working on a solution design document, we recommend having a quick look at Solution Design — Part 1: Introduction and First Principles.

Secondo —

• We recommend reading From Abstract Concepts to Tangible Value: Solution Architecture in Modern IT Systems for advanced concepts on architecting IT solutions.

Terzo —

• Once the High-Level Design (HLD) is approved and work commences, you might need to generate a low-level design, another document we describe in Part 4: Solution Design Documents — What You Need to Know.

---

3. High-Level Design (HLD)

3.1 What Is a High-Level Design (HLD) Document

A High-Level Design (HLD) is a technical document for a (generally) non-technical audience.

A High-level Design aims to provide all relevant stakeholders with a bird’s eye view of the solution architecture and design after implementation (or integration) is completed.

It’s mainly comprised of the following items:

1. A brief description of the solution — While keeping the discussion fairly simple, the HLD describes the primary features of the system. It is best to list all the elements for clarity without going near the implementation details, as these are not for this particular audience.

1. Reference to requirements — A reference to the functional requirements covering user functionality, such as business features, and non-functional requirements revolving around reliability, performance, high availability, security, and compliance. Business requirements are documented outside the HLD in a Business Requirements Document (BRD).

1. A high-level network and architecture diagram — Architectural diagrams describe the major components of a software system, such as platforms, applications, and interfaces. Network diagrams describe servers, network devices, communication links, and specialized hardware. Make sure you use consistent system modelling language across all your diagrams.

1. A logical breakdown of the solution is necessary if it is complicated, adds new modules, or updates existing ones. This section details whether new modules, skillsets, licenses, or operational requirements are needed for this project.

1. The business processes, use cases, and user stories — Aside from pure compliance or maintenance, IT projects typically add or modify business use cases, such as user interfaces or business functionality. These must be suitably documented as they might impact operations or how users interact with the system.

1. Data flow diagrams describe the input and output data and its path in the system. Information capture, processing, storage, replication, validation, and reporting are all aspects of data flow that must be documented.

1. Other forms and representations — In addition to the popular decomposition methods presented above, an IT solution can be represented in a few different views: behavioural, temporal flow, and states and modes. Illustrating solutions from different angles helps reduce their complexity and enhance their assimilation.

3.2 Why We Need an HLD

A high-level design offers the following advantages:

• It forms the basis of the stakeholders’ acceptance of the proposed solution, a baseline from which implementation can begin.

• The HLD is generally provided with the Functional Specs to support the Statement of Work (SOW), a binding document between the software vendor and the client.

• High-level estimates can be derived from the HLD, allowing project planners to calculate the project’s cost and assess project risk.

• Like any technical artifact, the HLD is an excellent communication and collaboration tool. It’s a vehicle for bouncing off ideas on a complex and potentially cognitively overloading topic.

Professionals espousing Agile values consider extensive documentation an effort-intensive, low-value activity and overhead that can be abandoned without risk. That may be true for small projects or standard, off-the-shelf offerings.

We examined Agile extensively in other articles, and we believe that Agile is sometimes promoted as a context-free answer to a context-specific question. In other words, there are instances where it applies and others where it doesn’t.

The effort to produce an HLD can be easily justified by understanding when it’s used and why. A detailed exploration was given in Part 2 of this series.

Related —

• Waterfall, Agile, and DevOps: A Critique of Current Challenges
• Part 2: Solution Design — How to Identify the Design Characteristics of Modern IT Systems

3.3 When Is a High-Level Design Required

An HLD is usually prepared at the early stages of a software project, typically during the Analysis phase of the SDLC.

The HLD is one of the documents accompanying the SoW in the Analysis phase. It clarifies the solution’s high-level concepts so stakeholders can decide whether the design satisfies their requirements.

3.4 Why Not Go Straight to an LLD

Preparing a Low-Level Design is a multi-iteration, customized, bespoke, and expensive process involving a broad audience of technical staff (business analysts, developers, testers, and DevOps engineers).

This investment cannot be justified at the early stages of a project as the stakeholders have yet to sign off on the requirements, Statement of Work, and scope.

Stakeholders could even ask for significant design changes or have alternative options considered before deciding on a specific path.

A document highlighting the fundamental concepts of a proposed solution allowing them to make an informed decision on the budget, schedule, and project risk, is required during this stage. The HLD, Functional Specifications and SOW should provide precisely this information.

3.5 High-Level Design (HLD) Document Ownership

The Solution Architect usually owns the high-level design (HLD) document.

People might cringe when they hear the word “architect”. They imagine a person somehow detached from real-world implementation realities, but that does not have to be the case.

So what is a solution designer/architect?

A solution architect is a highly-skilled technical resource who is proficient in the following areas:

•  The industry and the business — Architects thoroughly understand the industry and the end-to-end business requirements. They are often senior people familiar with the organisational culture and the significant business and technical decisions that shape the organization’s value proposition.

• The state of the current systems — The architect understands the system’s strengths and weaknesses and can offer insights on how to address them. They also have a thorough understanding of the interfaces and dependencies between the different components and can quickly gauge the impact of any change.

• The technology stack with which the system was built — Architects have a fair knowledge of the technology’s pros and cons, challenges and strong points, and how these compare to alternatives on the market. They also offer insights and best practices on system integration issues.

• Any compliance or regulatory requirements — The long expertise in the industry exposes architects to the different regulations and compliance requirements that must be satisfied before a system can run in production.

---

4. Architecture Documents and High-Level Design

Let’s briefly examine architecture and how its documents are relevant to High-Level Design.

4.1 What Is Solution Architecture

Solution architecture assembles technological assets (applications, products, servers) in an IT solution that satisfies specific business needs (functional requirements).

The subject has been explored from all angles in the series of instalments (of which this article is the fifth and final one), and, therefore, we invite the reader to look at Parts 1 and 2 for first principles and details.

Solution architecture is relevant to this discussion, and the high-level design captures its broad features.

4.2 What Is Software Architecture

Software architecture is an entirely different topic, which is not relevant here—first, a brief definition.

People have spent much time attempting to articulate a rigorous definition of architecture in software applications.

In property development, architecture determines the user experience of a resident. This analogy, however, is of less value in software.

For now, we use Martin Fowler’s definition, which is quite lucid (see the related link below for an in-depth analysis of the topic):

---

---

So, where does that leave us as regards the HLD?

At this stage of the SDLC, the low-level, highly technical issues are not yet relevant. The SOW has not been signed, the scope is not yet locked, and developers are yet to be engaged.

Related —

• Software Architecture and Design — Order, Complexity, and Chaos

---

5. Design Documents and Agile

Detailed documentation, top-down overarching design, and extensive planning are hallmarks of Waterfall; practices entirely left behind by purist Agile practitioners.

So how does a high-level design (HLD) document fit into Agile frameworks? It doesn’t!

Agile responds to the context-specific question of dealing with unarticulated needs and requirement volatility on a small scale. Agile performs poorly under the following circumstances:

• Massive projects where budgetary and scheduling constraints are paramount to stakeholders. Organizations cannot manage their strategic planning on two-week sprints. In this sense, Agile applies poorly outside semi-isolated development teams.

• Large-scale systems integration projects typically involve several platforms, suppliers, and technical and business teams. Synchronizing their cadence on two-week sprints is impractical.

• Projects with little or no novelty where all the details have been figured out are delivered more efficiently with Waterfall. The overhead incurred with Agile cannot be quickly justified.

Therefore, and in addition to all the advantages described in this article, planning and top-down design approaches for large projects involving many stakeholders require serious investment in design efforts, hence the need for high-level design and low-level design documents.

Related —

• Agile Software Delivery — First Principles and Foundational Elements
• Part 3: Agile Design Techniques for Creating Strong and Scalable Solutions

---

6. High vs Low-Level Design

The below table provides a comparison between the two design documents.

Property	High-Level Design	Low-Level Design
Essential	Yes	Yes
Names	HLD	LLD, SD
SDLC Stage	Analysis	Design
Content	Relatively technical	Highly technical
Audience	Project sponsors and managers, senior leadership	Business analysts, developers, testers, DevOps engineers, and various technical staff
Owner	Architect / Tech Lead	Principal or senior developer, tech lead
Stakeholders	Business people, senior management	Technical staff
Aim	Support SOW, sign-off solution	Drive design decisions, development, and testing tasks
Contents	System and solution architecture, use cases, data flow, interfaces	Feature design, impact analysis, low-level design, pseudo-code, algorithms, interface specifications and design considerations

---

7. High-Level Design (HLD) Document Template

7.1 Structure and Form

Visualizations are powerful tools that help reduce the cognitive load and channel complex ideas in a more digestible manner. Architectural, process flow and network diagrams are always welcomed in architectural documents.

But graphs are not enough, and a high-level design (HLD) should follow clear technical documentation-writing guidelines. Here is a quick summary of what to expect:

1. Clear and concise language with no ambiguity or open questions — We generally want to minimize any opportunities for misunderstandings resulting from weak assumptions.

1. A structure that is easy to read and follow — Use white spaces to your advantage. Separate different ideas and concepts into paragraphs and sections where applicable.

1. Helper sections such as an Overview, Glossary, Reference, and Target Audience — Try not to leave the reader in the dark regarding acronyms, external references, and context-specific terms.

1. Comprehensiveness — Address all critical topics. If a particular issue is irrelevant to the project, mark it as such so that the reader knows that it has been discussed but was discarded for a specific reason.

These guidelines should cover the form and structure of an HLD. The following sections will cover the contents.

7.2 Contents

7.2.1 Network Diagrams

An architectural describes the following topics:

1. All major components of the IT solution
2. How they fit in the overall IT ecosystem, which we refer to as the context
3. The different connections that integrate these components together

More specifically:

1. TCP/IP addresses and ports in the communications links
2. Firewalls where rules might need to be updated
3. New servers that need to be provisioned
4. New licenses that need to be acquired or updated
5. Geographical locations in case the servers are not sitting in the same data centre

The below diagram should illustrate the idea:

7.2.2 Architectural Diagrams

If you’re familiar with the C4 model for architectural visualization, this corresponds to Level 1: System Context Diagrams. Its job is to lay out the whole ecosystem of the IT solution.

At this stage, the nitty-gritty details of the implementation are not required. What is needed is a mapping between the high-level design decisions and the solution’s essential requirements.

Once you have completed the description of all the major components, it’s time to zoom in on the application level. The next topic you must include is application-level details in the high-level design.

An architectural diagram looks like this:

Architectural diagrams and sections cover the following topics:

• Core system components provide basic IT or business functions. For example, an Enterprise Message Bus is a typical architectural pattern in many online transaction-processing systems. For the HLD, keep the non-business components at a minimum, as they may not be particularly interesting to a business audience—focus instead on the business-centric elements.

• User interfaces tell the reader how users interact with the system or a particular component. This information signals changes on the operational level. Such changes are usually followed with training, user manual updates, and user education and are essential to communicate to the customer well ahead of development.

• The database layer is where all the data resides. Although not fascinating, data storage and management are a pathway leading to discussions on data backup, disaster recovery, redundancy, and high availability, all of which are critical for uptime and business continuity.

• The business logic modules deliver the business functionality of the solution. For example, a retail business can have billing, e-commerce, shipping, warehousing, and business intelligence components. It is best to have business components depicted in a separate diagram than IT components such as databases or web servers.

• Licensed components can be interesting for a business audience since they must be budgeted and sourced and impact cost and schedule.

• Application Programming Interfaces (APIs) are essential to call out so that inter-platform dependencies are vividly illustrated. Interface design and management are vital, especially in large systems integration projects.

7.2.3 Data Flow and Use Cases

The data flow and use cases section documents all necessary interactions between the user and the system.

Software solutions can be very complex and typically involve many stakeholders. List and define the different user profiles interacting with the system, for example, online users, system admins, or application admins.

This section captures the user’s expectations of how they intend to operate the system. These expectations will drive the development of the necessary UI components, reports, and internal software functionality to make that happen.

A typical business process flow diagram can look like this, with swimlanes delineating the boundaries between different business units:

The processes must be organized for the system to work, primarily if they depend heavily on one another.

Designing performing processes is a topic in itself, and there are rules and best practices that can help you achieve that; see production processes for a complete discussion.

Ensure this section of your high-level design satisfies the following:

• List all the parties, users, and modules involved.
• Define all the process steps while establishing ownership and responsibilities (the swimlane diagram can be beneficial).
• Make sure the criteria for movement between the steps is clear as well (inputs, outputs, tasks to be completed in-between stages)
• List the outcomes of each process and when it is considered “done.“

7.3 Download Template

Use the link below to download a basic High-Level Design (HLD) template.

---

8. Systems Modeling Language

8.1 The Need for a Specialized Language

Historically, describing and documenting IT system design was done using natural language and visualizations. While both are powerful and rich, allowing the author much room for creativity and expression, they have considerable drawbacks.

These limitations are more pronounced when systems increase in complexity, with many layers of abstraction requiring multiple decomposition levels.

Below is a list of these problems:

• Natural language can be ambiguous, and the same sentence can be open to different interpretations.
•  Errors are difficult to detect.
• Different system components can be described with varying depth and rigour, allowing inconsistencies to emerge.
• Documents written in natural language can overwhelm any individual’s cognitive abilities. The result is piles of barely used documentation.

8.2 Object-Process Methodology (OPM)

Thus, the need for a design modelling language arose. The anticipated language would address all the above issues at the price of confinement to a specific set of rules.

Any universal and practical language must adhere to specific rules governing the following areas:

• The language describes the objects and how they relate, group, categorize and subdivide.

• The meaning associated with these objects, semantics, is the corresponding field of study.

• The grammar would allow the proper construction of complex “sentences.” These rules are known as syntax.

Because objects can mean different things in different contexts, a special branch of philosophy called ontology engineering has been brought into existence. Its job is to overcome the inter-domain problem of associating different meanings with different terms.

Unified Modeling Language (UML) is one implementation of the above rules. UML, however, was not good enough and had to be replaced.

A universal, powerful, and domain-agnostic system modelling language was important enough to merit its own ISO standard, the ISO/PAS 19450:2015.

This ISO standard is called Object-Process Methodology (OPM) and defines the following concepts:

1. Entities:
   1. Objects: list of entities and building blocks of the system.
   2. Object States: an object can exist in one of several states during its lifetime and transition from one state to another.
   3. Processes are transformations that apply to objects. Processes can create, destroy, or transform objects between different object states.
2. Links:
   1. Structural links define associations between objects (composition, inheritance, exhibition, instantiation)
   2. Procedural designating how objects, states, and processes affect each other.
   3. Event and condition: the Event-Condition-Action model describes an event that triggers an action if a specific condition is satisfied.

While the OPM model may seem cumbersome and non-intuitive, it is very powerful once you get used to it.

In general, whether using OPM or something else, it’s helpful to remember the following when writing technical documentation:

1. Use consistent language, symbols, and expressions when describing the system (components, relationships, and functions).
2. Consistency should apply to both text and diagrams, content and style.
3. Publish a documentation style guide that everybody can access and use.
4. Use diagrams to convey complex ideas in a condensed yet very digestible format.

---

9. Final Words

The value of documentation and design has been downplayed significantly with Agile practices, although the advent of powerful content creation tools like Confluence has started to reverse the tide.

Technical documentation remains an integral part of the delivery of large software projects. So are architecture and design, especially for large system integration projects.

Doing it right can make an enormous difference in the quality of your deliveries, reflecting on the organisation’s image and value proposition.

Finding the right balance between quantity and productivity can take time, but it will be worth it. It also requires ingenuity to focus on critical and subtle information only and present it in a way that facilitates quick reading and assimilation.

A simple trick for managing documentation activities is maintaining templates that vary with project size; small projects use small templates, and large projects use large templates.

Collaboration, clarity, and stakeholder management keep everybody happy. The team’s morale remains high, and the customer remains satisfied and willing to do more business with you.

Those objectives and the best practices we described allow Operational Excellence in Software Development to become a tangible thing; a safe and healthy environment for your people and a prosperous business at the service of the community.

---

10. Featured Articles

Click on the tags to search by topic or the title to continue reading!