Systems Integration — Challenges and Solutions to Assembling Large Systems
1. What Is System Integration?
Systems integration addresses the challenges of assembling a (typically large and complex) solution from components or subsystems that may be independently sourced. Their architectural patterns, designs, and technology stack can vary widely.
The target system is the set of all components interacting via software interfaces that satisfy predefined business requirements.
The complexity of an integration task arises from the following factors:
On top of that, constraints typical of large software programs apply:
These factors combine to produce high levels of project risk that must be adequately managed.
In addition to discusisng system integration, these topics are also covered by this series:
2. System Integration Methods
The following activities occur during a systems integration project:
The above is typically an iterative process run for every subsystem, although a single integration approach might work in particular cases. The sequence of activities significantly impacts the efficiency of the integration process and will be the topic of our next discussion.
2. Approaches to Systems Integration
2.1 Single-Stage Approach
In a single-stage approach, we assume the constituent parts will arrive simultaneously, and testing will be carried out once the system has been fully assembled.
This approach can be efficient if the below criteria are met:
The single-stage approach might sometimes be inevitable despite being highly inefficient. This scenario can occur under the following conditions:
2.2 Multi-Stage Approach
Bringing experts from overarching domains to integrate, test, and troubleshoot a large system in a single stage is highly inefficient.
Consider the case where three experts integrate three platforms into a single solution. One expert will probably work whenever an issue is identified, while the other three await them to complete their analysis. In this simplistic (yet highly realistic) scenario, the efficiency is only 33%.
If there were four platforms and four experts, the efficiency would drop to 25%. For five experts, it’s 20%.
In a multi-stage (or incremental) approach, we assume the below to be true:
In a multi-stage approach, subsystems are assembled and tested incrementally as soon as they are ready. A new subsystem is integrated with the rest during each stage until the target solution is complete.
Task synchronization, dependency management, and stakeholder management are crucial for the success of large integration projects.
3. What Differentiates Software Development Methodologies from System Integration?
There are apparent similarities (with subtle differences) between systems integration approaches (single vs multi-stage) and software delivery methodologies (Agile, Waterfall, and DevOps).
To better understand the context in which one approach is superior to another, we shall closely consider this duality (single-stage and Waterfall, multi-stage and Agile).
In both single-stage and Waterfall, we assume great familiarity with the software applications and the integration tasks ahead; uncertainty is relatively low, requirement volatility virtually non-existent, and project risk is minimal.
Operational Excellence is guaranteed in these contexts with expertise, standardized processes, and best practices.
Multi-stage integration processes and Agile are also closely related but with subtle nuances.
In a multi-stage approach, the overhead incurred from additional testing of isolated components or larger subsystems is still smaller than the cost of a single-stage method for the reasons detailed earlier. This cost/benefit analysis is also accurate for Agile, albeit for different reasons.
The underlying concept in Agile (where requirement volatility is high) is that smaller, faster iterations are more efficient than a single, slower cycle since you can incorporate valuable knowledge gained with every iteration while slowly spiralling down towards the target solution.
The same concept is used in multi-stage integration. This time, we are optimising our time usage by kicking off integration on smaller scales as quickly as possible and only bringing in more people (and more subsystems) when confidence in the isolated subsystems peaks.
4. Verification and Validation in System Integration Projects
Although generic software testing concepts remain valid, this section will elaborate on some of the more relevant ones for large integration projects.
4.1 Systems Integration Testing (SIT)
In the (recommended) multi-stage incremental approach to systems integration, testing works as follows:
The scope of an SIT is restricted to connectivity, accessibility, and basic interface functionality tests. However, the lines are becoming increasingly blurred with automation and the ease the latter brings in running entire suites of test cases on demand.
The systems integration test staff must be technically proficient and understand IT systems well. SITs are typically conducted in lab settings that may differ significantly from production. Test data is almost exclusively used.
During SITs, we are still unsure whether the system is integrated correctly and adheres well to the specifications. Patches and further test iterations are also expected.
End-to-End tests and User Acceptance Testing (UAT) can only start once the system is fully integrated and tested.
4.2 End-To-End Testing
Once all the pieces have been assembled, and the target system emerges, testing shifts to a distinct mode. We are no longer solely interested in ensuring the interfaces are connected and perform as expected. Instead, our primary focus is ascertaining the entire system’s stability, reliability, and resilience through end-to-end testing.
The questions we are interested in can be:
In addition to fundamental software testing skills, end-to-end testers:
- Have a thorough understanding of the platforms across the entire landscape.
- Can pinpoint the specific subsystem responsible for the error.
- Are technically proficient; they can differentiate between low-level hardware or software errors and faulty features or implementation.
- Might be specialized in one module more than others.
- Are familiar with the business requirements and the software development lifecycle of the various products.
Although highly variable between teams and organizations, the scope of end-to-end testing does not cover the entire gamut of functionality but focuses primarily on the system’s essential features, with industry-relevant system certifications and User Acceptance Testing (UAT) covering the rest.
Finally, the data used in end-to-end testing and the third-party or external interfaces are typically more similar to production and less similar to dummy test data.
4.3 The Role of Test Tools and Simulators
Developers have been crafty in creating testing tools, from simple programs that use TCP/IP connections to send dummy messages and receive dummy responses to elaborate transaction simulators that generate production-like data and perform serious validation on responses.
Simulators have played a vital role in allowing interfaces to be tested offline during the implementation stages of the software development lifecycle.
Naturally, sophisticated simulators generate higher-quality production-similar requests and API calls, but they are typically commercial products with a decent learning curve.
Multi-stage incremental integration cycles would have been highly challenging without testing tools like this.
Standardized interface protocols also helped promote the reusability of simulators and made them viable commercial products. Using standardized interfaces (even internally) has advantages by allowing message simulator reuse (rather than redevelopment).