Test, Test, and More Test

Posted on December 3, 2019 by Allison Sluis

Test more. Test earlier. Test better. These are important goals that when put into practice produce a better product with a higher degree of confidence in its performance.

No matter what the product, from consumer electronics to manufacturing test equipment the software must be tested from several angles to evaluate accuracy and reliability.

Test More

At the unit test level, more testing can easily be accomplished using a Continuous Integration (CI) server such as Jenkins. Implementing a CI server has such advantage; it has its own post. Developing unit tests for C code is made easy through Cmock. This type of testing is typically made by the developers; however it can also be generated by another party.

At the system test level, more testing is accomplished through the use of thoughtful test automation. Automated tests take time to develop and maintain. Many tests are straight forward and are good candidates for test automation. Care should be given when considering which tests are good candidates for automation. Tests whose requirements are still unstable can cause a large ripple and rework when using test automation. Some tests will require user interaction that is costly to simulate and therefore do not have the cost benefit reward to warrant test automation.  This type of testing is typically made by a party other than the developer.

Test Earlier

Test activities should start as early as the requirements phase. Involving testers as soon as possible prevents testing from being an afterthought. Involving all the stakeholders in the entire life cycle reduces the number of issues and rework required to resolve issues found during testing. Resolutions found earlier in the life cycle reduce both time and financial costs.

Test Better

Unit test completeness can be measured with automated code coverage tools such as Bullseye. These tools report how many code paths were exercised during the test set and where coverage is missing. Code coverage is a simple but extremely valuable tool to evaluate the quality of the unit tests and by extension the software under test. Unit tests help to build a strong foundation on which to continue into system testing.

It is important in system testing to test the entire requirement and then some. Positive “happy path” testing is only a fraction of the testing that needs to happen. The bulk of tests should be made up of negative tests. These tests exercise the boundaries and limits of system interactions, inputs and outputs. Negative tests identify how the product behaves in the event of a failure.

Testing, in the end

Testing produces information that can be used to measure the status of a product. More information enables a more informed evaluation of the product status and a higher degree of confidence in the accuracy and reliability of the product.

Allison Sluis
by Allison Sluis
Chief Automation Test Engineer
Allison Sluis is Project Manager – Medical Solutions Group at DornerWorks.