TESTING

Testing performs a critical role for quality assurance and ensuring the reliability of software.

During testing, program to be tested is executed with set of test cases and output of program for test cases is evaluated to determine if program is performing as it is expected to.

ERROR, FAULT, FAILURE AND RELIABILITY

ERROR: Error refers to discrepancy between a computed, observed or measured value and true, specified or theoretically correct value i.e. error refers to difference between actual output of S/W and correct output.

Actual error is measure of difference between ideal and actual.

FAULT: Fault is condition that causes the software to fail to perform its required function.

Suppose take example of S/W of calculator, our calculator program does not perform multiplication function, it is a fault.

FAILURE: Failure is inability of system or component to perform a required function according to specifications. An S/W failure occurs if behavior of S/W is different from specified (not actual) behavior.

SOFTWARE RELIABILITY: Software Reliability that software will not undergo a failure for specified time for specified condition.

Actually Reliability is a level of confidence that takes place in S/W and is dependent on time. Reliability is defined as Mean Time To Failure (MTTF) which specifies average time between the occurrences of two failures.

Reliability of S/W depends on lesser number of faults. There are two methods for improving S/W reliability:

STATIC METHOD: Static methods are that where the system is not observed by executing the system example reviews, program proving and code reading. Faults are directly detected.

DYNAMIC METHOD: Dynamic Methods are that where by behavior of system is observed by executing the system.

N.B.: Static Methods are likely to be more cost effective for identifying and detecting the faults.

LEVEL OF TESTING: We know that faults occur in any phase in S/W development cycle. Verification technology is performed in the end of each pace but still some faults are undetected and reflected in next phase. So these faults are removed by testing. So there are different levels of testing.

The basic levels are

Unit testing, integration testing , system and acceptance testing.

Client needs Acceptance testing

Requirement System testing

Design Integration testing

Code unit testing

Ist level of testing is Unit Testing. In this method different modules are tested against the specification produced during design. Unit testing is essential for code produced during coding phase. Its goal is to test internal logic of modulus

Next level of testing is called Integration testing. In this many modules are combined into subsystem, which are then tested. Its goal is if modules are integrated properly, the emphasis being on testing interfaces between modules. Integration testing considers the testing of design and to see there is proper interaction between modules.

Next level is system testing to test entire s/w system. For this testing we refer the requirement document and see the system are according to specifications.

Next level is Acceptance testing to test s/w system with realistic data of client to demonstrate that s/w is properly working. Testing here focus external behavior of system.

Test Cases and Test Criteria:

The faults occur in program can not removed by testing. Due to this limitation, we select minimum no. of test cases whose goal is to detect maximum no. of faults.

An ideal test case is that which detects all faults. There are two aspects of test case selection – specifying criteria for evaluating a set of test cases and a procedure for generating a set of test cases that satisfy a given criterion.

There are many test criterions, but there less procedure for generating procedure so problem of test case selection is very challenging. If our criterion is weak, then testing is not useful. E.g. a criterion is unsatisfiable will be satisfied by empty set of test cases will be meaningless.

There are two properties for testing criteria.

1. Reliability

2. Validity

A criteria is reliable if all sets (all test cases) that satisfy criteria detect the same error. A criteria is valid if any error occurs in program, there is some set of test case which removes error.

On the other hand, generating criteria for test case is not simple, for example each statement of program should be executed. it is extremely hard to construct a set of test cases that will satisfy this criteria for a large program.

Test oracle: It is method of determining behavior of software system.

a test oracle is mechanism ,different from program itself that use to check the correctness of output of program for test cases.

In this process, test case which used during testing, given to test oracle, then compared the output obtained by testing and by test oracle. If there is no error then our result is confirmed. Software system is properly working.

Test oracle are human being, mostly result compute by hand.

e.g. Multiply 3*7 by calculator same result confirmed correctness

Or 3*7 by human

Psychology of testing:

Everything is not pure. So no method gives guarantee all errors will be detected. But main aim of testing is to remove all errors. Due to this reason, the psychology of performing test becomes important.

The aim of testing to demonstrate that program is showing no errors.

For testing, following precautions are necessary.

1) If your goal is to show that program works you find errors, one, two, three and stop. on the other hand if your goal is to show program does not work , we challenge our brain to find test cases, and we detect more errors.

2) Many organizations appoints special testing engineer for testing of software and not involve the person who develops the software. Behind this there is a psychology f actor. Because the new person tests the system with washable brain. Old person tests with the same fashion as which he developed s/w, which is harmful.

Top-down and bottom up approach:

When testing large programs, it is necessary to test parts of programs before testing the entire program.

If the test case detects an error in a large program, it will be extremely difficult to pin point the source error.

Actually system is Heap of modules. There are two methods of combining modules

1) Top to bottom

2) Bottom to top

1) Top to bottom:

We start with top of heap and add one by one when it calls, then test new system. During this approach, we require new system which is dummy routine that simulate a module. it uses when flows downwards.

2) Bottom to top:

We start with bottom of the heap which have no subordinates are tested then higher level modules are combined with bottom module. Any stage of testing before hand. to perform this approach , we require drivers which set the appropriate environment and invoke module . It is used when flows upward

N.B: if designing top to bottom, them development of program should be bottom to top.

Functional testing:

There are two basic approaches of testing

1) Functional

2) Structural

In functional testing, test cases are decided on the basis of requirements or specification of modules and internal logic of modules are not considered. Due to its nature, functional testing is called black box testing

In the structural testing, test cases are generated or basis of actual code program. Due to its nature, it is called glass box testing

N.B : Exhaustive testing :

in this testing we select all possible inputs in test cases which is impractical for small program. Functional testing is actually exhaustive testing because test cases selected on the basis of requirements of software system. so there is no precise criteria.

There is no. of heuristic which are used to select test cases that are very useful in detecting errors.

1) Equivalence class partitioning:

Since we are not interested in Exhaustive testing, so we divide domain of all inputs into set of equivalence classes. That if any test in any equivalence class succeeds then every test in that class will succeed.

But our emphasis is on select ideal equivalence class. for this we make equivalence class on the basis of specifications of module show positive integer. Then we decide inputs and arrange in the +ve integer equivalence class.

If module show behavior of -ve integer then arrange inputs in the -ve integer equivalence class. If our test is successful in one equivalence class, then further other test will also be successful and reject other equivalence classes.

2) Boundary value analysis:

There are some inputs lies on the edge or on the boundary of equivalence class, then output also an edge of equivalence class

For e.g 0<x<1 (inputs 0, 1)

Then invalid input are (-1.0, 1.1)

In the boundary value analysis, we generate class of output, then we test case that produces output lies at boundary of equivalence classes

Cause effect graphing:

One weakness with equivalence class partitioning and boundary value method is they consider each input separately. Both concentrate on condition and classes of one input.

They do not consider combination of input circumstances that form interesting situations that should be tested.

If there are n different input conditions such that any combination of input condition is valid we have 2 raised to the power n test cases.

Cause effect graphing is tech. which helps in selecting combinations in a systematic way such that no. of test cases do not become large.

This tech. starts with identifying causes and effects of system under testing.

a cause is distinct input condition

An effect is distinct output condition.'

Each condition forms a node in cause-effect graphing. Condition should be started which are true or false. Conditions are combined with Boolean operator 'and' and ‘or’, 'not'.

Cause: c1 command is credit

c2 command id debit

c3 accounts no. is valid

c4 transaction amt is valid

Effects: c1 and or c2 - e1 print " invalid command"

c1 or ~ c3 - e2 print " invalid account no."

c2, c3, ~ c4 - e3 print "debit acc. is not valid.

c2, c3, c4 - e4 debit account

c1, c3, c4 - e5 credit account

Structural testing:

The main aim of structural testing is to achieve test cases that will force the desired coverage of different structures.

Coverage criteria:

Most coverage criteria for selecting test case based on no. of statements, branches, or path in code of program. Simplest and weakest coverage criteria is statement coverage which require that each statement is executed at least once. This coverage is weak because errors are undetected. For e.g. if there is if statement without else clause, then coverage criteria satisfied by test case which evaluate condition is to be true. This is serious shortcoming.

Second coverage criteria is branch coverage which require (take) each branch in program is transverse at least once during testing. So each decision in program evaluated true or false. The trouble with branch testing if many conditions imposed on decision e.g. the data item valid if lies 0 and 100

begin

If (x>=0) and (x <=200)

Then check = true

Else check = false

End;

This module is incorrect as it check for x <= 200 instead of 100.

Path coverage:

A logical path is sequence of branches that are executed from start to end. The main difficulty is it contains loops which have indefinite paths.

So all these coverage criteria are not sufficient to select test cases.

Complexity based criteria:

It based on cyclomatic complexity, a test criteria proposed for structural testing. In this criteria, we select baseline of path, compare requirement (specification of modulus) with decision of branch of baseline of path, if decision is true, neglect false part of decision so that each statement are not executed and each branch is not transverse.

Mutual testing:

In this process, program is modified slightly to obtain mutants of original program then different mutations are tested along with original program. If we get different output by mutent or original program, then we have good test case. Naturally output will be different for two different programs.

Testing process:

A no. of activities are performed during testing a software. In general testing starts with test plan. Test plan identifies activities that are performed along with schedule and guidelines.

The plan also specifies level of testing. Based upon plan, first test cases are selected and specified. Then test unit is executes with test cases and report is produced. These reports are analyses. The testing units go on parallel and when testing of some units has been completed, the test unit combined along with some untested modules to form new test units.

A test plan is a document which contains following

1-Test unit specification

2-Feature to be tested

3-Approach to be tested

4-Test deliverables

5-Schedules

6-Personnel allocation

1- A test unit is a set of one or more modules together with data which forms program and object of testing. Thus test unit is a module or complete program. Test unit performs through different levels of testing.

2- Features to be tested: it includes all s/w feature and combinations of features that should be tested. The features of test unit are functionality, performance design constraints and attributes.

3- Approach for testing specifies overall approach followed in project. Which type of coverage criteria are used for selecting test cases?

4- Testing deliverable are list of test cases that are used, detailed result of testing, test summary, report, test log.

Test summary report records the result of testing activities and evaluate the result.

Test log provides records of relevant details about the execution of test cases

5- Schedule specifies amount tome and effort to be spent on different activities of testing.

6- Personnel allocation identifies persons responsible for performing different activities.

After test plan, select test case, discuss criteria, equivalence class, boundary value analysis etc. after selecting execute test case specified in doc. test procedure specification which set testing environment for it we write procedure starting, shutdown, restart etc.

Reliability assessment: reliability is particular pertinent metric can be assessed after testing is done. Every user wants to know the reliability of its own product. But concept of reliability of s/w system is different from h/w and mechanical systems. In mechanical or h/w system, unreliability is caused by wearing out or aging of system. But in s/w systems, reliability is discussed in terms of faults and failures. Reliability defined as the probability that faults do not cause a failure in a (MTTF) specified period of time.

1) JELINSKI AND MORANDA (JM) MODEL:--

This method is widely used assume S/W initially have N faults. The failure rate at any time is proportional to no. of faults present in S/W at that time. After 1^st failure faults N-2 and so on So failure rate is given by time t_I between i^th and (I+1)^th failure

Z(t_I) =#(n-I) # is a constant

The n is estimated by following equation:-

1/n +1/n-1 + ------------------------------------+1/n-(n-1)

Goel and Okumoto model:--

Go model based on number of failures observed in given period of time. If failure occurs, faults are removed; failure per unit time will decrease.

Let n(t) be commulative no. of failure observed by time t.

M(t) be expected no. of failure observed by time t.

The probability at given time t, n(t) is given by a equation :-

P (n(t)=y) = |m(t)| ^y e ^{_m t} y= 0, 1,2,3,4 ---------------------

M(t)=a(1-e ^–bt )

Failure rate z(t) at time t obtained by differentiating m(t)

Z(t) =a b e ^{– b t}

a = expected no. of failure.

M(t) == a when t == infinity

b= fault detection rate per fault. Given n(t) is y, since y is faults detect at time t , expected no. of remaining faults can be obtained

E{n(t)| n(t)} === a-y

If given failure are detected at t₁ ,t_{2 ,------------------------------}t_n

a and b estimated by

n/a == 1-e ^{– b t}_n

n/b == summation of t_I + t_n e ^–bt_n

Programmer productivity:

Productivity can be used for managing resources and reducing cost by increasing productivity. to measure productivity is lines of codes written per programmer month. There are problem in lines of codes because high level languages have less lines that low-level languages. So we check how much time computer utilizes by computer. But it has own shortcoming. So analyst assess the project, give the responsibility of module to programmer individually.

Monitor and control:

It is last step of testing. Our aim is to cover every unit of program. We select appropriate criteria for test case. Test summary report monitor all testing activities and test log keeps all record. Testing time is total time spent by project time.