Test more. Test earlier. Test better. These are important goals that when put into practice produce a better product with a higher degree of confidence in its performance.
No matter what the product, from consumer electronics to manufacturing test equipment the software must be tested from several angles to evaluate accuracy and reliability.
At the unit test level, more testing can easily be accomplished using a Continuous Integration (CI) server such as Jenkins. Implementing a CI server has such advantage; it has its own post. Developing unit tests for C code is made easy through Cmock. This type of testing is typically made by the developers; however it can also be generated by another party.
At the system test level, more testing is accomplished through the use of thoughtful test automation. Automated tests take time to develop and maintain. Many tests are straight forward and are good candidates for test automation. Care should be given when considering which tests are good candidates for automation. Tests whose requirements are still unstable can cause a large ripple and rework when using test automation. Some tests will require user interaction that is costly to simulate and therefore do not have the cost benefit reward to warrant test automation. This type of testing is typically made by a party other than the developer.
Test activities should start as early as the requirements phase. Involving testers as soon as possible prevents testing from being an afterthought. Involving all the stakeholders in the entire life cycle reduces the number of issues and rework required to resolve issues found during testing. Resolutions found earlier in the life cycle reduce both time and financial costs.
Unit test completeness can be measured with automated code coverage tools such as Bullseye. These tools report how many code paths were exercised during the test set and where coverage is missing. Code coverage is a simple but extremely valuable tool to evaluate the quality of the unit tests and by extension the software under test. Unit tests help to build a strong foundation on which to continue into system testing.
It is important in system testing to test the entire requirement and then some. Positive “happy path” testing is only a fraction of the testing that needs to happen. The bulk of tests should be made up of negative tests. These tests exercise the boundaries and limits of system interactions, inputs and outputs. Negative tests identify how the product behaves in the event of a failure.
Testing produces information that can be used to measure the status of a product. More information enables a more informed evaluation of the product status and a higher degree of confidence in the accuracy and reliability of the product.