Automated regression testing
Regression tests ensure that code that was previously functioning correctly does not regress when changes are made. Having a comprehensive suite of unit level regression tests provides a safety net, making sure that code changes do not break existing functionality.
It is common to automate running regression tests. However, some automated regression testing solutions make it possible to automate regression testing for multiple test stages:
Automate regression test generation
Relying on system testing makes it very expensive and inefficient to modify legacy code, but equally it is not commercially viable to manually write a comprehensive set of unit tests from scratch.
Automated test tools can automatically generate a suite of passing unit tests by parsing the source code and determining all possible paths through the code. The test suite can be generated according to a specified structural code coverage target.
These auto-generated tests can then be used as a baseline safety net to identify regression bugs when code is changed. This reduces reliance on time consuming and expensive system tests as well as being much more thorough and identifying the location of errors more precisely.
Learn more about Cantata automatic test generation for safety critical C code
Run regression tests automatically
Once a set of passing regression tests have been created, re-running them can be automated using a continuous integration solution such as Jenkins® or Bamboo®
Automatically running regression tests means that tests can be run easily for every build. Frequent test runs make bugs easier to fix as regression errors are identified earlier. This means that code changes are still fresh in mind and the problem is less likely to have been compounded by further changes.
Automate regression test maintenance
Changes to source code can sometimes mean that regression tests need to be updated. Test maintenance can be expensive when significant developer effort is required to identify dependencies and update tests to ensure they are compatable with the changed code under test.
Much of this process can be automated. Cantata Code Change Analyser uses information from the last occasion tests were run to automate test updates. When there is more than one appropriate way to update tests, an automated test tool can provide developers with options for re-synchronising tests with code:
- For each code change, too
- ls can suggest suitable updates to the test
- Providing guidance on the effects of accepting the suggestion
- Automatically refactoring tests to implement the suggested update
