Saturday, May 29, 2010

Testing cannot show that software is free of defects, but it can identify their existence. Thus, the purpose of testing is not to show that a program works, but to find its defects. Following are several practices for testing software.

Generate functional test cases from defined scenarios. These scenarios represent common transactions that reflect the tasks that the users will carry out. These test cases should be included in a regression test suite.

2- Use a code coverage tool. The reason for using a code coverage tool is that it provides quantitative evidence on how thorough a test effort is. Weigers illustrates the value of a code coverage tool when he said:

“Good programmers testing without benefit of a coverage tool are only achieving about 80 percent code coverage, while average programmers are getting 50 to 60 percent, and poor programmers are hitting less than 30 percent. Test strategies that emphasize only the functional behaviors of the software may cover the requirements adequately, but this is no guarantee that the internal structure of the program has been thoroughly exercised.”

3- Perform basis path testing. Basis path testing independently tests each conditional predicate of a module. That is, the outcome of each condition that can affect control flow is independently tested. Thus, basis path testing localizes a defect to a single subpath of a module because there are no interactions between decision outcomes.

A further benefit of this approach is that the size of the testing effort is predictable before testing begins, whereas other techniques are predicated on examining the ongoing progress of a test effort. In basis path testing, the required number of test cases for a module is exactly its cyclomatic complexity, which is computed by counting the number of conditional predicates of a module plus one (each separate case of a case statement counts as one). In sum, basis path testing is an effective, efficient, and quantifiable software testing technique that catches about two thirds of all defects.

Consider the following function, which computes the factorial of a number, and its associated flow graph, shown in the following figure.



The cyclomatic complexity of this function is 2, indicating that two edges exist in the flow graph: one test case is required for the condition 2 <= n and another one when 2 > n. Hence, n equals 1 and n equals 2 are two valid inputs that will transverse every edge of the control flow graph of the Fibonacci function, as shown in the previous table.


Note that many people have suggested breaking up complex modules into smaller modules, as measured by cyclomatic complexity. This is not necessarily a good thing for two reasons:

First, decomposing a module into multiple modules may not make sense because it may be difficult to decompose the larger module into smaller ones, each having specific responsibilities. Thus, decomposition may turn out to be arbitrary and not very meaningful.

Second, decomposing a module into more modules actually increases the testing effort. Consider, for example, how one might decompose the greatest common denominator function

4- Examine the boundary conditions affecting the control flow of a program. Test predicate variables in a condition by testing the two values one unit of measurement from the specified value. For example, if a predicate is x == 1, where x is an integer value, test cases should be written so that x is bound to 0 and 2 when the condition is executed. For the factorial function in the previous example checking the boundary conditions would yield the two test cases corresponding to n equals 1 and n equals 3. By choosing the input values wisely for basis path and boundary value testing, one can reduce the total testing effort. For this example, there are three unique test cases — n equals 1, 2, and 3 — that satisfy the structural testing procedures identified by Practices 3 and 4, which are shown in Table of factorial function. For the greatest common denominator function of the previous factorial algorithm, three boundary value conditions occur, which are satisfied by the wise selection of the inputs for basis path testing.

Furthermore, examine loop statements, which are a special case of boundary value testing, using the following heuristics.

- Execute each loop statement zero, one, and two times. This is more-or-less performing boundary value testing of the initial step of the iteration variable. In addition, it has been shown that executing a loop twice can detect some unique initialization problems.

- When using nested iteration statements, select a constant value for all iteration variables except the one being tested, vary it as previously described, and then repeat this process for the remaining nested iteration variables. This heuristic reduces the number of required test cases.

5- Verify data and file usage patterns of a program. That is, check that a program opens files before it operates on them and that it closes files before it terminates. Similarly, verify that a program declares and initializes data before it operates on the data, and that it reclaims all data before it terminates.

6- Verify that invalid and unexpected inputs are handled, as well as valid and expected ones. People often detect numerous defects when they use a program in proper but unexpected ways. Consequently, test cases containing unexpected and invalid inputs often find greater numbers of defects than do test cases containing valid input. In addition, an important class of inputs — command languages and data formatting protocols — should be rigorously tested.

Useful heuristics include:

(1) Executing repeated rewrite rules to find limitations in buffer sizes;

(2) Violating syntactic rules;

(3) Reversing syntactic tokens and other forms;

(4) Creating the null program;

(5) Creating null expressions (e.g., {});

(6) Examining code for missing, wrong, and too many delimiters; and

(7) Checking code for field value errors.

One might consider writing a syntax checker that mutates valid input strings in defined ways, which requires a formal definition of the syntax.

7- Verify all critical timing modes and time-out conditions. A critical timing mode occurs whenever execution timing is mission critical, consisting of:

(1) A fixed number of tasks, threads, or event-handlers running concurrently on a critical processor;

(2) A fixed allocation of functions to tasks, threads, or event-handlers; and

(3) fixed priorities and frequencies for each task, thread, or event-handler.

8- Verify that systems work in a variety of configurations as many as practical. Use the same hardware and software environments that the deployment sites will use during the integration test. The benefit of this is that an organization will detect defects unique to host environments of the field sites before it deploys software to them.

9- Verify the accuracy of the documentation. This includes verifying the correctness of the software installation procedures. Following this practice is important because documentation defects represent 10 to 25 percent of all delivered software defects.

Software Quality By :

Ahmed Abdelhamid
Software Quality Engineer
Interactive Saudi Arabia Ltd.
An Economic Offset Program Co.



Anonymous said...

very nice practices