Part 5: High-Level Solution Design Documents: What Is It and When Do You Need One

Georges Lteif
Georges Lteif
Software Engineer

Last Updated on January 19, 2023.
Subscribe now to stay posted!
About Us
14 min read


1. Overview

All large IT projects will require a High-Level Solution Design, also known as High-Level Design or HLD, an artifact instrumental to the Software Development Lifecycle analysis phase.

The HLD gives the stakeholders a bird’s eye view of the system once the implementation is completed. At its core is the proposed solution architecture, designed to solve a business problem and generate customer value.

Architecting a solution generates different concepts and selects an optimal candidate that satisfies budgetary and planning constraints while mitigating project risk.

This article examines all aspects of a High-Level Design for a project. It will give software practitioners an idea of whenwhy, and how to create a valuable HLD.

A few notes are due before we proceed:

  • Development vs Integration — High-Level Designs (HLD) work best for system integration projects where a solution is assembled from mature technology and products. The components of this solution and the interfaces joining them might need to be customized for every implementation. However, no substantial development effort is expected in such undertakings, in contrast to what we see in green field assignments.
  • Designing an IT solution — We recommend reading the Solution Design article for context and background. A design exercise uses an iterative approach to determine the best technical solution, ensuring that functional requirements and non-functional constraints are adequately understood, documented, and satisfied.
  • High vs Low-Level Design — The HLD should not be confused with the Low-Level Design (or LLD), whose primary role is to decompose the systems defined in the HLD into subsystems, subcomponents and, most importantly, select optimal values for key design and operations parameters laid out in the architecture.
  • Agile Architecture and Design — Can a design or architecture be Agile? The answer is Yes, and it has been discussed thoroughly on the two topics of solution architecture and software architecture. Depending on the context, a top-to-bottom or a bottom-up approach can be used for building large software systems. Requirement volatility compels you to use Agile methodologies, while Waterfall is perfectly appropriate for familiar and classical implementations. More on that later.

2. Pre-requisites

Primo

Secondo

Terzo


3. High-Level Design (HLD)

3.1 What Is a High-Level Design (HLD) Document

A High-Level Design (HLD) is a technical document for a (generally) non-technical audience.

A High-level Design aims to provide all relevant stakeholders with a bird’s eye view of the solution architecture and design after implementation (or integration) is completed.

A breakdown of the content of a High Level Design (HLD)
A breakdown of the content of a High-Level Design (HLD)

It’s mainly comprised of the following items:

  1. A brief description of the solution — While keeping the discussion fairly simple, the HLD describes the primary features of the system. It is best to list all the elements for clarity without going near the implementation details, as these are not for this particular audience.
  1. Reference to requirements — A reference to the functional requirements covering user functionality, such as business features, and non-functional requirements revolving around reliability, performance, high availability, security, and compliance. Business requirements are documented outside the HLD in a Business Requirements Document (BRD).
  1. A high-level network and architecture diagram — Architectural diagrams describe the major components of a software system, such as platforms, applications, and interfaces. Network diagrams describe servers, network devices, communication links, and specialized hardware. Make sure you use consistent system modelling language across all your diagrams.
  1. A logical breakdown of the solution is necessary if it is complicated, adds new modules, or updates existing ones. This section details whether new modules, skillsets, licenses, or operational requirements are needed for this project.
  1. The business processes, use cases, and user stories Aside from pure compliance or maintenance, IT projects typically add or modify business use cases, such as user interfaces or business functionality. These must be suitably documented as they might impact operations or how users interact with the system.
  1. Data flow diagrams describe the input and output data and its path in the system. Information capture, processing, storage, replication, validation, and reporting are all aspects of data flow that must be documented.
  1. Other forms and representations — In addition to the popular decomposition methods presented above, an IT solution can be represented in a few different views: behavioural, temporal flow, and states and modes. Illustrating solutions from different angles helps reduce their complexity and enhance their assimilation.

3.2 Why We Need an HLD

The value of a High-Level Design (HLD) includes providing stakeholders with a basis for acceptance, support for the Scope of Work, facilitating high-level estimations, and documenting the design.
The value of a High-Level Design (HLD) includes providing stakeholders with a basis for acceptance, support for the Scope of Work, facilitating high-level estimations, and documenting the design.

A high-level design offers the following advantages:

  • It forms the basis of the stakeholders’ acceptance of the proposed solution, a baseline from which implementation can begin.
  • The HLD is generally provided with the Functional Specs to support the Statement of Work (SOW), a binding document between the software vendor and the client.
  • High-level estimates can be derived from the HLD, allowing project planners to calculate the project’s cost and assess project risk.
  • Like any technical artifact, the HLD is an excellent communication and collaboration tool. It’s a vehicle for bouncing off ideas on a complex and potentially cognitively overloading topic.

Professionals espousing Agile values consider extensive documentation an effort-intensive, low-value activity and overhead that can be abandoned without risk. That may be true for small projects or standard, off-the-shelf offerings.

We examined Agile extensively in other articles, and we believe that Agile is sometimes promoted as a context-free answer to a context-specific question. In other words, there are instances where it applies and others where it doesn’t.

The effort to produce an HLD can be easily justified by understanding when it’s used and why. A detailed exploration was given in Part 2 of this series.

Related —

3.3 When Is a High-Level Design Required

An HLD is usually prepared at the early stages of a software project, typically during the Analysis phase of the SDLC.

The High-Level Design HLD is prepared during the Analysis phase of the SDLC.
The High-Level Design HLD is prepared during the Analysis phase of the SDLC.

The HLD is one of the documents accompanying the SoW in the Analysis phase. It clarifies the solution’s high-level concepts so stakeholders can decide whether the design satisfies their requirements.

3.4 Why Not Go Straight to an LLD

Preparing a Low-Level Design is a multi-iteration, customized, bespoke, and expensive process involving a broad audience of technical staff (business analysts, developers, testers, and DevOps engineers).

This investment cannot be justified at the early stages of a project as the stakeholders have yet to sign off on the requirements, Statement of Work, and scope.

Stakeholders could even ask for significant design changes or have alternative options considered before deciding on a specific path.

A document highlighting the fundamental concepts of a proposed solution allowing them to make an informed decision on the budget, schedule, and project risk, is required during this stage. The HLD, Functional Specifications and SOW should provide precisely this information.

3.5 High-Level Design (HLD) Document Ownership

The Solution Architect usually owns the high-level design (HLD) document.

People might cringe when they hear the word “architect”. They imagine a person somehow detached from real-world implementation realities, but that does not have to be the case.

So what is a solution designer/architect?

Skilful solution architects have a thorough understanding of the industry, its requirements and regulation, existing platforms on the market, and most importantly, an intimate knowledge of the business model and the platforms deployed/supported within the business unit.
Skilful solution architects have a thorough understanding of the industry, its requirements and regulation, existing platforms on the market, and most importantly, an intimate knowledge of the business model and the platforms deployed/supported within the business unit.

A solution architect is a highly-skilled technical resource who is proficient in the following areas:

  • The state of the current systems — The architect understands the system’s strengths and weaknesses and can offer insights on how to address them. They also have a thorough understanding of the interfaces and dependencies between the different components and can quickly gauge the impact of any change.
  • The technology stack with which the system was built — Architects have a fair knowledge of the technology’s pros and cons, challenges and strong points, and how these compare to alternatives on the market. They also offer insights and best practices on system integration issues.
  • Any compliance or regulatory requirements — The long expertise in the industry exposes architects to the different regulations and compliance requirements that must be satisfied before a system can run in production.

4. Architecture Documents and High-Level Design

Let’s briefly examine architecture and how its documents are relevant to High-Level Design.

4.1 What Is Solution Architecture

Solution architecture assembles technological assets (applications, products, servers) in an IT solution that satisfies specific business needs (functional requirements).

The subject has been explored from all angles in the series of instalments (of which this article is the fifth and final one), and, therefore, we invite the reader to look at Parts 1 and 2 for first principles and details.

Solution architecture is relevant to this discussion, and the high-level design captures its broad features.

4.2 What Is Software Architecture

Software architecture is an entirely different topic, which is not relevant here—first, a brief definition.

Martin Fowler's definition of software architecture encompasses what's important and hard to change. Although insightful, it's more qualitative and high-level than we would like.
Martin Fowler’s definition of software architecture encompasses what’s important and hard to change. Although insightful, it’s more qualitative and high-level than we would like.

People have spent much time attempting to articulate a rigorous definition of architecture in software applications.

In property development, architecture determines the user experience of a resident. This analogy, however, is of less value in software.

For now, we use Martin Fowler’s definition, which is quite lucid (see the related link below for an in-depth analysis of the topic):


Software architecture is those technical decisions which are both important and hard to change. This means it includes things like the choice of programming language, something architects sometimes gloss over or dismiss. Both aspects land squarely on the economics of software development.
Martin Fowler

So, where does that leave us as regards the HLD?

At this stage of the SDLC, the low-level, highly technical issues are not yet relevant. The SOW has not been signed, the scope is not yet locked, and developers are yet to be engaged.

Related —


5. Design Documents and Agile

Detailed documentation, top-down overarching design, and extensive planning are hallmarks of Waterfall; practices entirely left behind by purist Agile practitioners.

Careful planning and design vs Agile and DevOps, which is more efficient and suitable for our modern, fast-paced software industry? Turns out that we are seeking a context-free solution to a context-sensitive question, and the answer is: it depends.
Careful planning and design vs Agile and DevOps, which is more efficient and suitable for our modern, fast-paced software industry? Turns out that we are seeking a context-free solution to a context-sensitive question, and the answer is: it depends.

So how does a high-level design (HLD) document fit into Agile frameworks? It doesn’t!

Agile responds to the context-specific question of dealing with unarticulated needs and requirement volatility on a small scale. Agile performs poorly under the following circumstances:

  • Massive projects where budgetary and scheduling constraints are paramount to stakeholders. Organizations cannot manage their strategic planning on two-week sprints. In this sense, Agile applies poorly outside semi-isolated development teams.
  • Large-scale systems integration projects typically involve several platforms, suppliers, and technical and business teams. Synchronizing their cadence on two-week sprints is impractical.
  • Projects with little or no novelty where all the details have been figured out are delivered more efficiently with Waterfall. The overhead incurred with Agile cannot be quickly justified.

Therefore, and in addition to all the advantages described in this article, planning and top-down design approaches for large projects involving many stakeholders require serious investment in design efforts, hence the need for high-level design and low-level design documents.

Related —


6. High vs Low-Level Design

The below table provides a comparison between the two design documents.

PropertyHigh-Level DesignLow-Level Design
EssentialYesYes
NamesHLDLLD, SD
SDLC StageAnalysisDesign
ContentRelatively technicalHighly technical
AudienceProject sponsors and managers, senior leadershipBusiness analysts, developers,
testers, DevOps engineers, and various technical staff
OwnerArchitect / Tech LeadPrincipal or senior developer, tech lead
StakeholdersBusiness people, senior managementTechnical staff
AimSupport SOW, sign-off solutionDrive design decisions, development,
and testing tasks
ContentsSystem and solution
architecture,
use cases, data flow, interfaces
Feature design,
impact analysis, low-level design, pseudo-code, algorithms, interface specifications and
design considerations
Comparison between a high-level design and a low-level design

7. High-Level Design (HLD) Document Template

7.1 Structure and Form

Visualizations are powerful tools that help reduce the cognitive load and channel complex ideas in a more digestible manner. Architectural, process flow and network diagrams are always welcomed in architectural documents.

But graphs are not enough, and a high-level design (HLD) should follow clear technical documentation-writing guidelines. Here is a quick summary of what to expect:

  1. Clear and concise language with no ambiguity or open questions — We generally want to minimize any opportunities for misunderstandings resulting from weak assumptions.
  1. A structure that is easy to read and follow — Use white spaces to your advantage. Separate different ideas and concepts into paragraphs and sections where applicable.
  1. Helper sections such as an Overview, Glossary, Reference, and Target Audience — Try not to leave the reader in the dark regarding acronyms, external references, and context-specific terms.
  1. Comprehensiveness — Address all critical topics. If a particular issue is irrelevant to the project, mark it as such so that the reader knows that it has been discussed but was discarded for a specific reason.

These guidelines should cover the form and structure of an HLD. The following sections will cover the contents.

7.2 Contents

7.2.1 Network Diagrams

An architectural describes the following topics:

  1. All major components of the IT solution
  2. How they fit in the overall IT ecosystem, which we refer to as the context
  3. The different connections that integrate these components together

More specifically:

  1. TCP/IP addresses and ports in the communications links
  2. Firewalls where rules might need to be updated
  3. New servers that need to be provisioned
  4. New licenses that need to be acquired or updated
  5. Geographical locations in case the servers are not sitting in the same data centre

Create More Diagrams If Required

Each diagram should have only one purpose. Create two separate diagrams to describe more than one solution aspect (such as hardware, security, or geographical distribution).

Example 1: Server and Network Plan

In this case, the components in the diagram can be physical servers (or VMs) interacting via TCP/IP links.

This diagram is probably the best place to add any firewalls or network routers that might be installed or configured for the system to function.

Remember, a high-level solution design’s objective is to highlight any potential work or effort needed during the implementation.

The below diagram should illustrate the idea:

Example of a network diagram, a fundamental component of a High-Level Design (HLD) document. This diagram shows all the network components involved and what interfaces facilitate their communications.
Example of a network diagram, a fundamental component of a High-Level Design (HLD) document. This diagram shows all the network components involved and what interfaces facilitate their communications.

7.2.2 Architectural Diagrams

If you’re familiar with the C4 model for architectural visualization, this corresponds to Level 1: System Context Diagrams. Its job is to lay out the whole ecosystem of the IT solution.

At this stage, the nitty-gritty details of the implementation are not required. What is needed is a mapping between the high-level design decisions and the solution’s essential requirements.

Once you have completed the description of all the major components, it’s time to zoom in on the application level. The next topic you must include is application-level details in the high-level design.

An architectural diagram looks like this:

Logical decomposition in an architectural diagram of a computer system. Logical decompositions allow architects to define a solution or system architecture and validate its integrity at an arbitrary level.
Logical decomposition in an architectural diagram of a computer system. Logical decompositions allow architects to define a solution or system architecture and validate its integrity at an arbitrary level.

Architectural diagrams and sections cover the following topics:

  • Core system components provide basic IT or business functions. For example, an Enterprise Message Bus is a typical architectural pattern in many online transaction-processing systems. For the HLD, keep the non-business components at a minimum, as they may not be particularly interesting to a business audience—focus instead on the business-centric elements.
  • User interfaces tell the reader how users interact with the system or a particular component. This information signals changes on the operational level. Such changes are usually followed with training, user manual updates, and user education and are essential to communicate to the customer well ahead of development.
  • The database layer is where all the data resides. Although not fascinating, data storage and management are a pathway leading to discussions on data backup, disaster recovery, redundancy, and high availability, all of which are critical for uptime and business continuity.
  • The business logic modules deliver the business functionality of the solution. For example, a retail business can have billing, e-commerce, shipping, warehousing, and business intelligence components. It is best to have business components depicted in a separate diagram than IT components such as databases or web servers.
  • Licensed components can be interesting for a business audience since they must be budgeted and sourced and impact cost and schedule.
  • Application Programming Interfaces (APIs) are essential to call out so that inter-platform dependencies are vividly illustrated. Interface design and management are vital, especially in large systems integration projects.

7.2.3 Data Flow and Use Cases

The data flow and use cases section documents all necessary interactions between the user and the system.

Software solutions can be very complex and typically involve many stakeholders. List and define the different user profiles interacting with the system, for example, online users, system admins, or application admins.

This section captures the user’s expectations of how they intend to operate the system. These expectations will drive the development of the necessary UI components, reports, and internal software functionality to make that happen.

A typical business process flow diagram can look like this, with swimlanes delineating the boundaries between different business units:

User process flow in a high-level design describing a business use-case of an ATM transaction withdrawal. Use cases are perfect examples of what HLD should contain. They show the stakeholders or end users of the system and how they interact with it.
User process flow in a high-level design describing a business use-case of an ATM transaction withdrawal. Use cases are perfect examples of what HLD should contain. They show the stakeholders or end users of the system and how they interact with it.

The processes must be organized for the system to work, primarily if they depend heavily on one another.

Designing performing processes is a topic in itself, and there are rules and best practices that can help you achieve that; see production processes for a complete discussion.

Ensure this section of your high-level design satisfies the following:

  • List all the parties, users, and modules involved.
  • Define all the process steps while establishing ownership and responsibilities (the swimlane diagram can be beneficial).
  • Make sure the criteria for movement between the steps is clear as well (inputs, outputs, tasks to be completed in-between stages)
  • List the outcomes of each process and when it is considered “done.

Example: Opening a Bank Account

Let’s say you have implemented a banking system that allows customers to enter any branch and open a bank account for personal or business use.

The process usually involves multiple people within that branch. Each person is responsible for a particular step in the application’s approval process.

7.3 Download Template

Use the link below to download a basic High-Level Design (HLD) template.


8. Systems Modeling Language

8.1 The Need for a Specialized Language

Historically, describing and documenting IT system design was done using natural language and visualizations. While both are powerful and rich, allowing the author much room for creativity and expression, they have considerable drawbacks.

These limitations are more pronounced when systems increase in complexity, with many layers of abstraction requiring multiple decomposition levels.

Below is a list of these problems:

  • Natural language can be ambiguous, and the same sentence can be open to different interpretations.
  •  Errors are difficult to detect.
  • Different system components can be described with varying depth and rigour, allowing inconsistencies to emerge.
  • Documents written in natural language can overwhelm any individual’s cognitive abilities. The result is piles of barely used documentation.

8.2 Object-Process Methodology (OPM)

Thus, the need for a design modelling language arose. The anticipated language would address all the above issues at the price of confinement to a specific set of rules.

Any universal and practical language must adhere to specific rules governing the following areas:

  • The language describes the objects and how they relate, group, categorize and subdivide.
  • The meaning associated with these objects, semantics, is the corresponding field of study.
  • The grammar would allow the proper construction of complex “sentences.” These rules are known as syntax.

Because objects can mean different things in different contexts, a special branch of philosophy called ontology engineering has been brought into existence. Its job is to overcome the inter-domain problem of associating different meanings with different terms.

Object-Process Methodology (OPM) devices its constituents into Entities, Links of various types, events, Objects, States, and Processes.
Object-Process Methodology (OPM) devices its constituents into Entities, Links of various types, events, Objects, States, and Processes.

Unified Modeling Language (UML) is one implementation of the above rules. UML, however, was not good enough and had to be replaced.

A universal, powerful, and domain-agnostic system modelling language was important enough to merit its own ISO standard, the ISO/PAS 19450:2015.

This ISO standard is called Object-Process Methodology (OPM) and defines the following concepts:

  1. Entities:
    1. Objects: list of entities and building blocks of the system.
    2. Object States: an object can exist in one of several states during its lifetime and transition from one state to another.
    3. Processes are transformations that apply to objects. Processes can create, destroy, or transform objects between different object states.
  2. Links:
    1. Structural links define associations between objects (composition, inheritance, exhibition, instantiation)
    2. Procedural designating how objects, states, and processes affect each other.
    3. Event and condition: the Event-Condition-Action model describes an event that triggers an action if a specific condition is satisfied.

While the OPM model may seem cumbersome and non-intuitive, it is very powerful once you get used to it.

In general, whether using OPM or something else, it’s helpful to remember the following when writing technical documentation:

  1. Use consistent language, symbols, and expressions when describing the system (components, relationships, and functions).
  2. Consistency should apply to both text and diagrams, content and style.
  3. Publish a documentation style guide that everybody can access and use.
  4. Use diagrams to convey complex ideas in a condensed yet very digestible format.

9. Final Words

The value of documentation and design has been downplayed significantly with Agile practices, although the advent of powerful content creation tools like Confluence has started to reverse the tide.

Technical documentation remains an integral part of the delivery of large software projects. So are architecture and design, especially for large system integration projects.

Doing it right can make an enormous difference in the quality of your deliveries, reflecting on the organisation’s image and value proposition.

Finding the right balance between quantity and productivity can take time, but it will be worth it. It also requires ingenuity to focus on critical and subtle information only and present it in a way that facilitates quick reading and assimilation.

A simple trick for managing documentation activities is maintaining templates that vary with project size; small projects use small templates, and large projects use large templates.

Collaboration, clarity, and stakeholder management keep everybody happy. The team’s morale remains high, and the customer remains satisfied and willing to do more business with you.

Those objectives and the best practices we described allow Operational Excellence in Software Development to become a tangible thing; a safe and healthy environment for your people and a prosperous business at the service of the community.


Click on the tags to search by topic or the title to continue reading!

9 thoughts on “Part 5: High-Level Solution Design Documents: What Is It and When Do You Need One

    1. I have recently updated this page to include more details in Section 5, hoping to cover any info required for creating the document, is it not clear enough? Or are you looking for something more inline with the one we have in the Solution Design page?

Leave a Reply