Software that is in production periodically requires a maintenance release. As new needs emerge or defects are discovered after deployment, maintenance is performed.
When Maintenance Testing is Done?
Once delivered, Software and Systems are maintained for various changes like:
- Adding new functionality
- Deleting existing functionality
- Altering existing functionality
- Fixing any defect
- Improve or preserve Non-Functional characteristics, including compatibility, reliability, security, portability, and performance efficiency.
Maintenance testing is performed when any changes are made as a part of maintenance. Maintenance Testing ensures that changes are made successfully and there is no possible side effect (e.g., regression) on the part of the system that is not changed. Maintenance releases are sometimes scheduled in advance with defined feature sets or may also take the form of emergency fixes (hotfixes) with little planning (if any).
Scope of Maintenance Testing
Maintenance Testing is required at different Test Types and Levels based on its scope. The scope of maintenance testing depends on three factors:
- Degree of Risk of the changes (for example, the degree to which the changed area of software communicates with other components or systems.)
- Size of the Existing System
- Size of the Changes
Triggers of Maintenance Testing
For Planned and Unplanned changes, maintenance testing takes place for several reasons. The Triggers for maintenance testing can be classified into two types:
Modifications are required:
- For planned release-based enhancement
- Corrective Changes
- Emergency changes
- Operational environment changes (OS or database upgrades)
- COTS software upgrades
- Patches for defects and vulnerabilities
Migration from one platform to another require operational testing of the new environment, changed software, and data conversion.
- An application is at the retirement stage when it reaches the end of life. During retirement, testing is done for data migration or archiving if long data-retention periods are required.
- Testing restores/retrieve procedures after archiving for long retention periods may also be needed.
- Regression testing is also required to ensure functionality still in service is working.
Maintenance testing may be triggered for the Internet of Things when completely new or modified things, such as hardware devices or software services, are introduced to the overall system. For such a system, maintenance testing emphasizes integration testing at different levels, including application and network levels. Maintenance Testing also focuses on the security aspects, in particular, those relating to personal data.
Impact Analysis for Maintenance
Impact Analysis is determining how the existing system may be affected by changes. Maintenance testing is performed to test the impact of changes to an operational system or its environment. For maintenance releases, impact analysis is done to evaluate the changes to identify the intended consequence as well as expected and possible side effects. The analysis also identifies areas of the system that will be affected by the change.
Impact analysis also helps to identify the impact of the change on existing tests. The side effects and affected areas in the system need to be tested for regressions, possibly after updating any existing tests affected by the change. Impact analysis may be done before a change is made to help decide if the change should be made based on the potential consequences in other areas of the system.
Impact analysis can be difficult if:
- Specifications (e.g., business requirements, user stories, architecture) are outdated or missing.
- Test cases are not documented or are out of date.
- Bi-directional traceability between tests and the test basis has not been maintained.
- Tool support is weak or non-existent.
- The people involved do not have domain and/or system knowledge.
- Insufficient attention has been paid to the software's maintainability during development.