CSE1IS Information Systems Week 12 Lecture 1 System Testing

1. Systems Testing Overview:

Effective and comprehensive systems and software testing is a critical activity in IS development. It is through testing that you demonstrate the reliability and accuracy of a system. The objective is to deliver reliable and high quality software. We will concentrate first on software testing. Obviously this is also an important topic in any software development subject, so there may be some overlap between what follows here and what you may be told in other units within your course.

When developing a test plan and running the test cases, you will be undertaking the following activities:  

1.      develop test cases/scenarios
2.      run the tests
3.      report the errors
4.      correct the errors
5.      estimate (quantify) the reliability of the software

The philosophy is to attempt to 'crash' the software!

The test cases should cover as many scenarios as possible.

Test cases and test data should be prepared for:

•      testing attributes of individual fields
•      testing entry, storing and retrieval of records
•      testing opening, closing and access of files
•      testing for inaccurate, incomplete or obsolete data
•      testing controls for data entry
•      testing the internal logic of program modules
•      testing the interface between program modules

2. White Box Versus Black Box Testing:

These terms refer to the testing of software. White box testing focuses on testing internal logic whereas black box testing focuses on testing output against input.

White Box testing

..........
DO WHILE
 ..........
 IF .... THEN

    etc.  

You would test each internal logical structure in a program module.
Equivalent test cases should reduce the number of testing cases.

Black Box Testing

input ---->   [program module]  -----> output  

Test the expected and actual output for each program module.

3. Testing Strategies:

There are normally four main strategies used in testing

• module testing
• integration testing
• systems testing
• acceptance testing

3.1 Module testing:

• This is sometimes called 'unit' testing and is an example of white-box testing.
• The use of test-cases is appropriate.
  • It involves the testing of the internal logic of the software modules that make up the system.
• Highly cohesive modules are easier to test.
  • A highly cohesive module is one that only performs a single function or task.
• When planning and conducting module testing focus on the testing of areas of greatest risk

3.2 Integration testing:

• Integration testing tests the combination of modules.
• It focuses on the data being passed between modules and on whether individual modules retain their functionality when combined with others.
• You should adopt an incremental approach (i.e. add a module and test, add another module and test )
• Module stubs can be used as a testing aid.

A top-down integration or bottom-up integration approach is possible.

• Notice that each module is uniquely numbered. The modules do not only represent a screen or report, but also may represent the software components necessary to run the screens (i.e. to validate the input data, to store data in files, to retreive data from files etc.).
• To illustrate the difference between top-down and bottom up integration testing we will consider modules 0-Main, 3.0-appointments, 3.1-create appointment,  3.2-cancel appointment and 3.3-produce appointment schedule.

The top-down approach

• Program 1, say main menu, would be developed first and tested internally [i.e we need to make sure that the menu is displayed correctly, that the user selection is processed correctly and that we exit from the module correctly].
• The next thing to be tested is that control is passed to the appropriate module when the user makes a selection. e.g when they select the option `appointments' then program 2 [say, appointments] is called. To do this we do not have to write the complete module. All we do is write a very simple module that  displays a message (to say that it has been called) and provides a mechanism to exit the module and return control to the calling module (in this case program 1-Main).
  • The same thing would be done for program 3.
  • We are only testing the interaction between 1-Main and the two modules underneath.
  • These very simple modules are sometimes called `stubs' and later they will be developed into the fully functional modules specified in the design.
• Now we would fully develop program 2 (appointments), which is another menu and make sure that it displays the correct options and that the user selection is processed properly. `Stubs' would be created for programs 4 and 5 and we would test that control is passed to these when the appropriate menu selection is made in program 2.
• In this way we develop and test the modules working from the highest level down to the lowest level in the hierarchy --- hence the term top down integration (or development or testing).

The bottom up approach

• In this case the opposite approach is taken.
• For example, program 4 (create appointment) would be treated as a stand alone module and developed accordingly.
  • It is probably concerned with accepting user input and creating a new record in the file (or table) that stores information about appointments.
  • It would be fully tested to ensure that input data is validated correctly and that the correct data is written to file.
• Program 5 would be developed in a similar fashion.
• After programs 4 and 5 are developed we are concerned about the module that calls them (i.e. program 2).
  • Firstly a 'driver' module would be developed.
  • A 'driver' module is very simple and things such as enhancements to the menu display etc are left to later.
  • The focus is on whether programs 4 and 5 are called correctly and that they still retain their functionality when they are called.
• In a similar fashion, programs 6 and 3 are developed, plus any other modules in the hierarchy
• Finally the top menu is devoloped, i.e. program 1.
• Thus the development proceeds from the lower modules, upwards through the hierarchy to the top --- hence the term bottom up integration (testing, development)

3.3 System testing:

• System testing involves testing the software in the context of the whole system (i.e. both automated and manual aspects).
• Key objectives (pp. 432):
  • final test of all programs
  • Verify that the system will handle all input data properly, both valid and invalid
  • Ensure that the IT staff has the documentation and instructions needed to operate the system properly and that backup and restart capabilities of the system are adequate
  • Demonstrate that users can interact with the system successfully
  • Verify that all system components are integrated properly and that actual processing situations will be handled correctly
  • Confirm that the information system can handle predicted volumes of data in a timely and efficient manner
• It may also include:
  • Recovery testing.  Can the system recover after a fatal error?
  • Security testing. Can system security be breached? Can such breaches be detected?
  • Stress testing. How does the system perform under abnormal condition? Can it cope with a peak processing load?

3.4 Acceptance testing:

• An acceptance test is a formal test to ensure the system meets the user requirements.
• If it 'passes' then ownership of the system passes from you as a developer to the client.
• The acceptance test criteria are agreed to between the system designers and the client (user) group.
• If conflicting views exist, management will decide whether or not to install the system after a full discussion of the options

4. Example Test Plan:

• A test plan should be developed for every data entry screen.
• Sample test data should be provided and specify where it is to be placed within the database
• Separate plans should also be done for a database with sample data and an empty database
• Example of the start of a test plan for the student screen of the Pharmacy project:

References:

• Shelly, Cashman & Rosenblatt, Systems Analysis and Design, 6th Edition,  Course Technology, 2006.

NOTE: this webpage has not been brought up to WCAG 1.0 standard nor does it contain validated HTML