All IT projects will require a High-Level Solution Design, also known as HLD, an artifact instrumental in the Analysis phase of the SDLC.
The High-Level Design (HLD) gives the project stakeholders a bird’s eye view of the system at the end of the project.
The core of the High-Level Design (HLD) is the solution architecture, a process we have covered in great detail in this article. In summary, architecting a solution generates different concepts and selects the optimal one that satisfies the project’s risk, budgetary, and schedule constraints.
Two notes are due before we proceed:
- To give the reader some context, background, and introduction to the fundamental concepts of solution design, we recommend that you read the article on Solution Design. In summary, A solution design uses an iterative approach to determine the best solution that ensures functional and non-functional requirements are adequately understood, documented, and satisfied.
- The HLD should not be confused with the Low-Level Solution Design (or LLD), whose primary role is to decompose the systems defined in the HLD into subsystems, subcomponents and, most importantly, select optimal values for key design and operations parameters laid out in the architecture phase.
This article is tailored for IT projects. It will explain what an HLD is and why you need it before finally providing a template of what it should contain.
2. Table of Contents
- 1. Overview
- 2. Table of Contents
- 3. High-Level Design Document (HLD)
- 4. Software Architecture Documents
- 5. High-Level Design Document (HLD) Template
- 6. Systems Modeling Language
- 7. Final Words
- 8. Featured Articles
3. High-Level Design Document (HLD)
3.1 What Is a High-Level Design Document (HLD)
A High-level Design (HLD) is a slightly technical document for a (generally) non-technical audience.
It’s mainly comprised of the following items:
- A brief description of the solution: While keeping the discussion at a high level, the HLD describes the primary features it will deliver. It is best to list all the elements for clarity without going near the implementation details, as these aspects are probably not for this particular audience. Examples of non-functional requirements are performance, high availability, security, and compliance.
- A high-level network and architecture diagram: that elucidates the significant components in the system (platforms, applications, servers, network devices, specialized hardware). Its purpose is to highlight the hardware components and third-party systems impacted by this project. Make sure you use consistent system modelling language across all your diagrams.
- A logical breakdown of the different application modules and components is sometimes necessary if the solution is complicated or adds new modules or updates existing ones. This section gives visibility on whether new modules, skillsets, licenses, or operational requirements are needed for this project to work.
- The business processes, use cases, and user stories: Solutions typically add or modify use cases. If new user interfaces and business functionality are introduced, these have to be suitably documented as they might introduce additional efforts such as updates to field operations guides and generic documentation, as well as training on the new system.
- Data flow typically describes the input and output data and its path in the system. Capture, processing, storage, replication, validation, and output are all aspects of data flow that must be documented.
- Other solution representations: In addition to the more popular decomposition methods presented above, a solution can be represented in a few different views: behavioural, temporal flow, and states and modes.
3.2 Why Do You Need an HLD
The HLD, like many other technical artifacts created by the organisation’s production processes, is an excellent communication and collaboration tool. It’s a vehicle for bouncing off ideas in a way that you can easily track, update, and collaborate on.
Most importantly, an HLD forms the basis of stakeholders’ solution acceptance before detailed design and implementation can start.
The HLD is generally provided along with the Functional Specs as supporting documents for the Statement of Work (SOW).
Agile practitioners may sometimes frown on documentation as they consider it an effort-intensive, low-value activity. That may be true if the project is tiny or the time you spend writing it is excessive.
In general, however, technical documentation in the correct quantity and quality is essential for quality deliveries; it can be the difference between risky projects and flawless execution.
Like many other tools and documents that can be used to improve the quality of your deliveries, the HLD plays a specific role in the Analysis phase of the project.
Managing stakeholders’ expectations is vital for smooth deliveries and customer satisfaction.
3.3 Why Not Go Straight to an LLD?
You cannot justify this effort at this stage of the project, mainly because the project’s scope has not yet been signed off, and the internal and external stakeholders have not yet agreed on the deliverables.
Remember, the HLD is one of the documents accompanying the SoW, so we are still in the Analysis phase.
The HLD focuses on the big picture, while the LLD focuses on technical and implementation details, which, at this stage, have yet to be confirmed.
3.4 When Is a High-Level Design Required
An HLD is usually prepared at the very beginning of the software project, typically during the Analysis phase of the SDLC.
It is accompanied by the SOW t provide clarity on the solution’s high-level concepts.
3.5 Who Owns the High-Level Design Document
The Solution Designer, also known as Solution Architect, is usually the owner of the high-level design document.
People might cringe when they hear the word “architect” in a business environment. They typically imagine a person who is somehow detached from the realities of the real-world implementation. But that does not have to be the case.
So what is a solution designer/architect?
A solution architect or solution designer is a highly-skilled technical resource who is proficient in:
- The business aspects of the project: They have a thorough understanding of the industry and the business’s end-to-end requirements. They are typically senior people in the organization. They have great familiarity with the significant business and technical decisions made throughout the applications’ history.
- The state of the current systems: The architect understands the system’s weaknesses and can offer insights on how to address them. They also have a thorough understanding of the interactions between the different components and can quickly gauge the impact of any change.
- The technology that runs these systems: Architects have a fair knowledge of the technology’s pros and cons, challenges and strong points, and how these compare to alternatives on the market.
- Any compliance or regulatory requirements: This skill ensures the system remains compliant at all times and that projects factor in any such conditions.
A solution architect can also be a technical lead, someone who gets his hands dirty every once in a while and still understands the system’s essential features.
4. Software Architecture Documents
4.1 What Is Software Architecture
Having discussed the role of an Architect, we must also say a few words about software architecture.
People have spent quite some time defining what architecture means in software applications. In property development, architecture determines the user experience when someone is using the house or building.
This definition does not map to what architecture is in the software world. So perhaps a better word might be structural engineering.
We have found Martin Fowler’s definition to be quite lucid:
Software architecture is those technical decisions which are both important and hard to change. This means it includes things like the choice of programming language, something architects sometimes gloss over or dismiss. Both aspects land squarely on the economics of software development.
A full guide on solution architecture is published here, we recommend that you give it a read for context and background.
4.2 Architecture Documents
There are essentially two types of architecture documents relevant to most projects: the High-Level Design (HLD), which we discuss in this article, and the Low-Level Solution Design (LLD).
To better understand when and where these documents converge or diverge, have a look at the table below.
|Property||High-Level Design (HLD)||Low-Level Design (LLD)|
Technical Staff, Testers, DevOps Engineers
|Owner||Architect / Team Lead||Architect / Development Lead|
|Stakeholders||Business people, management||Technical teams|
|Aim||Support SOW||Drive development |
and testing tasks
|Topics||System and solution|
use cases, data flow
|Feature design, |
4.3 Architecture Documents Relevant in Agile?
Some Agile professionals have even preached a complete abandonment of documentation in its traditional form. But that is not quite what the creators of Agile wanted. What they did decree though in the Agile manifesto was:
A) Working software over comprehensive documentation
B) Responding to change over following a plan
That is quite different from, say, no documentation or no planning!
The business value that technical documentation generates lies in two key areas: collaboration and stakeholder expectation management.
A document review exercise gives relevant stakeholders the opportunity to voice any concerns and eventually buy in.
Collaboration can be done through other means like JIRA Epics and Stories, and Tasks, but it is far more convenient to have a design in one place.
In managing stakeholder expectations, you have to know that the customer will probably never hand you a blank cheque; they will need to see the design and sign it off before any work can safely commence.
This process is especially true for critical or sizeable projects where customer requirements can be easily misunderstood.
If there is business value in documenting design work, a realistic view would be to keep them.
5. High-Level Design Document (HLD) Template
5.1 Structure and Form of an HLD
People discussing solution architecture generally like to see many diagrams as these are especially powerful in providing a visual aid to understanding the underlying concepts.
An excellent technical document, in general, is rich in diagrams that help the reader digest complex ideas. But diagrams are not enough.
The high-level design should also follow clear technical documentation-writing guidelines. Here is a quick summary of what to expect:
- Clear and concise language with no ambiguity or open questions: we generally want to minimize any opportunities for misunderstandings that may result from unjustified assumptions.
- A structure that is easy to read and follow: Use white spaces to your advantage. Separate different ideas and concepts into paragraphs and sections where applicable.
- Helper sections such as an Overview, Glossary, Reference, and Target Audience: Try not to leave the reader in the dark regarding acronyms, external references, and context-dependent terms.
- Comprehensiveness: Do not leave any critical topic out. If a particular issue is not relevant, mark it so that the reader knows that this topic has been discussed but found to be irrelevant for this project.
These guidelines should cover the form and structure of an HLD. The following sections will cover the contents.
5.2 Contents of an HLD
5.2.1 Architectural Plan
An architectural plan is the first ingredient of a high-level design document (HLD). It serves to describe:
- All the major components of the IT solution
- How they fit in the overall IT ecosystem, which we refer to as the context
- The different connections that help integrate these components together
You can also include the following information:
- IPs and ports used by the links
- Firewalls that need to be updated
- New servers that need to be provisioned
- New licenses that need to be acquired or updated
- Geographical locations in case the servers are not sitting in the same data centre
Create More Diagrams If Required
Each diagram should have only one purpose. Create two separate diagrams if you want to describe more than one aspect of the solution (such as hardware and security).
The below diagram should illustrate the idea:
If you’re familiar with the C4 model for architectural visualization, this corresponds to Level 1: System Context Diagrams. Its job is to lay out the whole ecosystem of the IT solution.
Example 1: Server and Network Plan
In this case, the components in the diagram can be physical servers (or VMs) interacting via TCP/IP links.
This diagram is probably the best place to add any firewalls or network routers that might meet to be installed or configured for the system to function.
Remember, the objective of a high-level solution design is also to highlight any potential work or effort needed during the implementation.
Example 2: Application and Services Plan
Another example could be an architectural plan that involves the different services and applications required to support the system’s business functionality.
The interactions between those components are usually APIs or generic messaging links.
Again, at this stage, the nitty-gritty details of the implementation or not required.
In fact, what is required is a mapping between the high-level design considerations (such as redundancy, load-balancing, disaster recovery…) and those components of the solution which would allow these requirements to be satisfied.
5.2.2 Application Modules
Once you have completed the description of all the major components in the solution’s ecosystem, it’s time to zoom in to the application level. Application-level details are the next topic you need to include in the high-level design (HLD).
An application-level design usually looks like this:
At this stage, you will define the different subsystems of an application. These could be:
- Core components: provide basic IT functionality and may or may not be relevant to the industry. An Enterprise Message Bus architecture, for example, is a typical architectural pattern present in almost any online transaction processing IT system. For the HLD, keep these components at a minimum as they may not be particularly interesting to a business audience.
- User interfaces and layers: can tell the reader what type of users will interact with the system or a particular component. This information signals changes on the operational level. Such changes are usually followed with training, user manual updates, and user education and are essential to communicate to the customer well ahead of development.
- The database layer: is where all the data resides. Although not fascinating in itself, data storage and management typically bring up topics such as backup, disaster recovery, redundancy, and high availability, all of which are critical elements for uptime and business continuity.
- The business logic modules: provide the business functionality of the system. For example, you can have billing, e-commerce, shipping, warehousing, and business intelligence components in a retail business.
- Licensed components: can be interesting for a business audience as these require budgeting and sourcing. It’s important to point these out as part of the solution offering.
- APIs and external connections are essential to call out as they usually come in the context of integration with third-party systems and need to be considered for development, integration, and testing. Like any other dependency, they also need to be accounted for on the project management side.
Today’s systems are vastly more complex than that and they could include a whole lot of different modules related to security, compliance, reporting, auditing, or some other peripheral functionality. Also, very few systems nowadays run in isolated environments.
5.2.3 Data Flow and Use Cases
The Transactions and User Flows section documents all necessary interactions between the user and the system. Make sure you list and define all the user types that will interact with the system: online users, system admins, or application admins.
Software today is very complex and is usually handled by many different parties. This makes this section essential as it captures the user’s expectations of how they intend to operate the system.
These expectations will drive the development of the necessary UI components, reports, and internal software functionality to make that happen.
A typical business process flow diagram can look like this, with swimlanes delineating the boundaries between different business units:
The processes need to be organized for the system to work, especially if they might be heavily dependent on one another. Designing great processes is a topic in itself, and there are rules and best practices that can help you achieve that.
Make sure this section of your high-level design (HLD) satisfies the following:
- List all the parties/users/systems involved
- Define all the steps of the process while clearly establishing ownership and responsibilities (the swimlane diagram can be very helpful)
- Make sure the criteria for movement between the steps is clear as well (inputs, outputs, tasks to be completed in-between stages)
- List the outcomes of each process and when it is considered “done”
Example: Opening a Bank Account
Let’s say you have implemented a banking system that allows customers to walk into any branch and open a bank account for personal or business use.
The process usually involves multiple people within that branch. Each of those people is responsible for a particular step in the application’s approval process.
5.3 Download Template
Use the link below to download a basic High-Level Design (HLD) template.
6. Systems Modeling Language
Historically, describing and documenting the design of an IT system was done using one of two methods: natural language or visualizations. While both are powerful and rich, allowing the author much room for creativity and expression, they showed significant drawbacks and limitations.
These are more pronounced when systems increase in complexity, having many layers of abstraction and requiring multiple levels of decomposition.
Below is a list of these problems:
- Natural language can be ambiguous, and the same sentence can be open to different interpretations.
- Errors are difficult to detect.
- Both are prone to allow variable depth and rigour in the description across the decomposition levels or system components.
- Documents written in natural language can overwhelm any individual’s cognitive abilities. The result is piles of documents that nobody would read them.
Thus, the need for a design modelling language arose. The anticipated language would address all the issues listed above, perhaps at the price of confinement to a specific set of rules.
Any valuable and universal language must adhere to specific rules. These rules would govern:
- The language describes the objects and how they relate, group, categorize and subdivide. Because objects can mean different things in different contexts, a special branch of philosophy called ontology engineering has been brought into existence. Its job is to overcome the inter-domain problem of associating different meanings with different terms.
- The meaning associated with these objects: semantics is the corresponding field of study.
- The grammar that would allow the proper construction of complex “sentences.” These rules are known as syntax.
If you have ever used the Unified Modeling Language (UML), you should have some idea of how these concepts materialize.
Having a universal, powerful, and domain-agnostic system modelling language was important enough to merit its own ISO standard, the ISO/PAS 19450:2015.
This ISO standard is titled OPM or Object-Process Methodology. OPM defines the following concepts:
- Objects: list of entities and building blocks of the system.
- Object States: an object can exist in one of several states during its lifetime and can transition from one state to another.
- Processes are transformations that apply to objects. Processes can create, destroy, or transform objects between different object states.
- Structural links define associations between objects (composition, inheritance, exhibition, instantiation)
- Procedural designating how objects, states, and processes affect each other.
- Event and condition: the Event-Condition-Action model describes an event that triggers an action if a specific condition is satisfied.
While the OPM model may seem cumbersome and non-intuitive, the rationale behind its creation and the fundamental concepts it’s built around are very relevant to technical documentation.
When writing technical documentation, it’s helpful to remember the following:
- Use consistent language, symbols, and expressions when describing the system (components and functions).
- Consistency should apply to both text and diagrams.
- Publish a documentation style guide that everybody can access and use.
- Use diagrams to convey complex ideas in a condensed yet very digestible format.
7. Final Words
Writing technical documentation is an integral part of delivering software, and doing it right can make an enormous difference in the quality of your deliveries, reflecting on the organisation’s image and its value proposition.
Finding the right balance between quantity and productivity can take time but will be worth it.
A simple trick for managing documentation activities is maintaining templates that vary between project sizes; small projects use small templates, and large projects use large templates.
Collaboration, clarity, and stakeholder management keep everybody happy. The team’s morale remains high, and the customer remains satisfied and willing to do more business with you.
Those objectives and the best practices we described allow Operational Excellence in Software Development to become a tangible thing; a safe and healthy environment for your people and a prosperous business at the service of the community.