7      TEST DESIGN FOR UAT

Chapter 3 introduced the test development process (TDP), the process by which the tests for UAT are created. Each of the three components of the TDP, test conditions, test cases and test procedure specifications (test scripts), represents a step in a process or hierarchy that starts with the definition of the requirements and ends with UAT.

In this chapter we will explore that hierarchy, explain in more detail how test conditions, test cases and test scripts are built, and apply this directly to our UAT exercise.

THE HIERARCHY OF TEST DESIGN

The levels in the test design hierarchy (see Figure 7.1) are as follows:

•  The business requirements represent the needs of the business.

•  The test conditions state what aspects of a given requirement should be tested.

•  The test cases specify, for each test condition, what preconditions, inputs, expected results and post-conditions apply.

•  The test procedure specification lists the steps the UA testers will carry out for each test case and identifies the test data to be used for each test.

Figure 7.1 The test design hierarchy

The test design process starts at the top of the hierarchy: the test conditions are identified from the test basis, test conditions are turned into test cases and the test cases are used to create the test scripts.

The relationship between the documentation levels is maintained by assigning a unique reference number to the requirements and referring all the following documentation back to the requirement from which it arose. This is known as traceability. Traceability is the key to ensuring that the test design covers all the requirements in the RS and no others. It will also become invaluable in managing documentation changes, so that when any document in the hierarchy is changed it will be easy to find out which related test conditions, test cases and test scripts must also be changed.

Test design does not deal solely with how to write individual examples of test conditions, test cases and test scripts; it also defines how to manage the process of writing all the test conditions, test cases and test scripts in a way that is efficient and effective. All of the test design phase, but particularly the creation of the test conditions and test cases, should be carried out with assistance from the end-users in the UAT team, bringing together the expertise of UAT with expertise of the business to create the best and most relevant test materials. At a minimum a representative of the business must sign off the requirements, the test conditions and test cases as complete and accurate, but this is a poor substitute for direct end-user involvement in test design. Tests defined by others will almost certainly lack the unique end-user perspective that makes UAT an effective tool in reducing risk.

IDENTIFYING TEST CONDITIONS

Test conditions are the first step in the creation of tests. Test conditions are logical statements that represent some aspect of a requirement. In other words, a single test condition is ‘something that can be tested.’ Each test condition represents a single component of a feature that can be assessed as either true or false and the feature is correctly implemented if and only if all conditions are true.

Depending on how business requirements are written, one requirement can give rise to many test conditions. In Chapter 3 we used a login example and noted some of the variations that ought to be tested. Each variation equals a test condition. We will use the same example and look in some more detail at the requirements for login to the system.

Example 7.1

Table 7.1 Login requirements

The test conditions must represent every variation of test that together would confirm that the requirement has been met. The three test conditions we identified in Chapter 3 were as follows:

1.  If a valid username is entered with the correct password, the user is logged in to the system.

2.  If a valid username is entered with an incorrect password, an error message appears.

3.  If a non-valid username is entered with a password, an error message appears.

These are the relevant test conditions for requirement 4.1 but there are some more conditions we can identify based on requirements 4.1.1 to 4.1.3.

4.  When a valid username is entered in all caps with the correct password, the user is logged on to the system.

5.  When a valid username is entered in all lower case with the correct password, the user is logged on to the system.

6.  When a valid username is entered in caps and lower case with the correct password, the user is logged on to the system.

7.  When a valid username is entered with a password containing fewer than six characters, an error message appears.

8.  When a valid username is entered with a password containing no numbers, an error message appears.

It is more efficient to note down all the possible test conditions that can be conceived of, even if it seems that there is an overlap between them. It is easier to combine duplicate test conditions than it is to spot ones that are missing. In any case a list of test conditions, even when based on the requirements, is not guaranteed to be an exhaustive list, but is a list of the things we can think of to test.

Getting team members or other end-users to review the test conditions should lead to useful feedback and ideas for further conditions. In our example what qualifies as an overlap depends on the error messages. If unique error messages need to appear such as: ‘Passwords are at least six characters in length’ or ‘Passwords contain at least one number’, more test conditions are needed than if the error message is generic for all login failures.

Because conditions 4, 5 and 6 all require entry of a valid username and password – one where the username is entered in caps, the second where the password is entered in lower case, and the third where the username is entered in both caps and lower case – test condition 1 is no longer required. You could also argue that test condition 6 is not required and is adequately tested by a combination of conditions 4 and 5. If, on the other hand, you tried to combine condition 4, which checks that the password entered in all caps is allowable, with condition 7, which checks that a password with fewer than six characters is not allowed, this cannot provide an answer to both conditions in one test. If we enter a password of fewer than six characters in all caps and the login succeeds, we know there is an issue and requirement 7 has not been met, but if the login fails, we cannot deduce from the test result whether the system will allow caps in the username.

Creating the test conditions is a crucial stage in the test design process. For simple functionality, such as logging on to the system, you could still logically deduce what testing might need to be done without writing the conditions down, but being able to conceive of all the possible test conditions is vital when it comes to more complex system functionality. In order to make sure that all the requirements have been covered, and covered in enough detail, it may be useful to create a matrix. Matrices are widely used in testing and help to compare the content of one document against another. For example a matrix could show the functional requirements that have been created for each business requirement.

Similarly a spreadsheet can be created that shows the requirement ID and the requirement description horizontally and each of the test conditions alongside the requirement in columns. Table 7.2 is an example of a matrix matching conditions related to the Excelsior contracts functionality to the business requirements. The requirements identify the required functionality and the conditions for the basic testing that should be carried out in order to test that the functionality worked.

Table 7.2 Test condition matrix

This type of matrix is easy for all interested parties to understand and sign off.

Most implementations are going to yield a large number of test conditions and risk analysis may need to be carried out to determine which test conditions are most important. Entering the test condition into a spreadsheet or matrix has the added advantage that the test condition can be linked to the requirement (and the requirement’s priority) and that the information needed to create the test case can be added to the same spreadsheet.

DESIGNING TEST CASES

We previously learnt that a test case specifies the preconditions, inputs, expected outputs and post-conditions for a test of whether a test condition is true:

•  The preconditions and post-conditions identify the state the system must be in before the test is executed and the state it will be in after test execution, respectively. Understanding what the start and end ‘state’ of each test will be helps to identify the sequence the testing should be carried out in for the most efficient UAT session. If a number of test cases have the precondition that the user is logged on to the system, it would make sense to run the login tests first. That way the time involved in carrying out tasks that are not part of testing is kept to a minimum.

•  The test input data state what the entries in to the system should consist of in order to test the condition. Defining the data entries for the test case is an important part of preparing the test data for UAT. In Example 7.2, unless the implementation is for a change or addition to an existing system, in which case real user accounts may be replicated, test accounts will need to be created with usernames and passwords. These accounts must support the different user roles that will be accessing the new system.

•  The expected output from a test case is the vital element that enables us to determine immediately, objectively and consistently whether a test has passed or failed when it is executed.

The test case contains all the basic information required in order to test, but without the form (the test script) that the end-user will need to identify the specific data to be used at test execution time and to record the outcomes of testing.

Example 7.2

Here is a simple test case for a login procedure.

Table 7.3 Login use case

Practical user testing

In Chapter 3 we considered different ways of constructing tests using standard test-case design techniques and more practically oriented approaches such as requirements-based, risk-based and process-based testing. Test-case design techniques will prove very helpful in generating specific test cases to support our overall approach, but we should also use our applied common sense and experience to produce tests that work for users and provide meaningful results.

We also learned that we are testing a system, not just a piece of software, and that we are interested in what the system can or cannot do for users. We are not particularly interested in whether the software performs according to its technical specification because the developers and testers will have already tested the system from that perspective. Our concern is not about compliance with specification; we are using the RS as a basis for our test plan mainly because it gives us a well-defined scope for our operations.

Focus on what matters – test by risk

The idea of risk-based testing is useful here. If we have an understanding of what aspects of the system are most important and might cause a serious problem if they were not effective, that is if they represent risk, we can prioritise tests by risk level and test the highest risk areas first. We can begin by identifying the risk level for each requirement or group of requirements and putting them in priority order. That is at least a practical starting point for developing test cases.

Test at the edges

We have already pointed out the fact that boundaries and edges are rich with problems, so it is a good idea to focus attention on the edges. To do this you can use BVA with or without EP – they are related techniques but independent. You can exploit BVA wherever you find a boundary.

Remember also that we are not only interested in boundaries within programs. Business processes have edges and boundaries and these will also be a good place to explore.

So when you are building test cases for a business process, make sure some of them explore the edges of the process.

Keep the tests simple – test by practical scenarios

If we know how the end-user expects to use the system and what business processes will interface with it, we can set up some practical scenarios that represent commonly used processes and build test cases around them. The collection of processes will have some patterns – hierarchical, time-based, customer-based or some other pattern. The patterns can be exploited to link together similar processes so that we can test a whole area of the system at one time with a relatively small set of test cases and test data.

One further thing to consider is sequencing the tests. Remember in the definition of test cases we ran into ‘execution preconditions’ and ‘execution post-conditions’. These are no more than the definition of the way the system needs to be set up to run a particular scenario (that is an execution precondition) and the state the system is in when the test scenario is complete (that is an execution post-condition). The advantage of identifying these pre- and post-conditions is that we can match them up: a test scenario with a precondition that matches the post-condition of another scenario can be run immediately after that scenario with no additional setting up.

There are other ways we can link test scenarios together to make the testing more meaningful and more convenient to run by saving us setting-up time. We can sequence tests by date and time so that each scenario is in a natural date and time sequence from its predecessor. If there are scenarios with no time gaps, we can run them without breaking to reset the system date and time. Similarly if one test updates a database and another test reads the database, we can again use the natural sequence to save time.

The key idea is to organise the tests by scenario and then look for patterns that can be exploited to make the testing easier and more natural.

The overall aim of UAT is to determine whether the system can be effectively used by its intended end-users and all of your tests should reflect that. What we need are simple tests – simple to implement and interpret – that directly exercise the things that users need to happen.

Concentrate on typical and unwanted scenarios

If it proves difficult to prioritise test scenarios by criticality, frequency or value, we can use our previous experience to identify what is a ‘typical’ scenario, meaning a scenario that will be used often and that perhaps underpins other more complex scenarios. This is testing by ‘mainstream’ activities. Even if they are not the most critical, they will be the most often used so problems with them will cause user frustration and potentially damage productivity.

We can similarly look for the scenarios that represent our worst nightmare – the things that should never happen. If we test these, we can gain confidence by discovering that worst cases actually do not happen or we can identify early a situation that could prove fatal for acceptance. This is an example of an extreme scenario; something at the edges or boundaries of system behaviour. The BVA test-case design technique can be useful here to test relatively few cases that are on or near this boundary – either inside it or outside it – and gain confidence that the system will not stray outside.

Exceptions are the things that do not fit the normal rules. They are similar to edge cases because they may generate a different way of processing data than that used for the bulk of the system. If there is anything wrong in these areas, it may only involve a relatively small area of the system but it will generate those mysterious errors that never seem to get sorted out.

Look for any exception cases in the RS, the business processes, the user interface or anywhere else you can think of. The effort will be well worthwhile.

Make the scenarios work together

It is a small step from the idea of end-to-end testing to the idea of linked scenarios. Building up a sequence of scenarios that represent a typical working day is one way of linking scenarios together. Not only will this make the testing realistic, it will also make it easier to implement and it will take advantage of the fact that, in many cases, the postcondition of one test will automatically match the precondition of another.

Take this a step further and use the working day at the beginning or end of a week, a month, a period or a year and we will end up testing some of the most important boundary cases for the business.

A little imagination goes a long way in inventing and linking scenarios to achieve multiple testing objectives. It saves time, adds realism, increases efficiency and, best of all, it means the UAT will be able to maximise the amount of testing it can cover in the time available.

Make use of business processes

Business processes should already be documented at this stage. They will almost certainly involve data entry and use of output data, as well as some interactive dialogue with the computer. We need to identify these flows so that we can build end-to-end tests of business processes. Within the end-to-end tests we have the opportunity to introduce specific test cases of interest so we need to construct scenarios with data that will be flowed through the system during the end-to-end tests. That way we get confidence in the processes, the system’s interface with the processes and the specific data handling that is part of the processes.

Test in cycles

Testing can be time-consuming and expensive to set up because we have to test many scenarios, each with potentially different starting conditions. If we can group tests together that have a common starting point or use a common test environment, we may be able to make the tests more effective, quicker and more cost-effective to execute and easier to manage at the same time.

We can also test time-related aspects of the system by running tests in cycles. Each test cycle has a different starting point in time and in system status, so the input cycle might run first, followed by an update cycle and so on. This will enable us to generate input data for each cycle from the cycle before.

Focus on the user interface

One of the principal reasons for having users test systems before they come into use is because the users have a unique perspective on whether a user interface actually

‘works’. Detailed usability tests may or may not have been done on the system but, whether they have or not, the system needs a user’s evaluation.

How ‘comfortable’ is the screen layout? How are data entered? Are fields sensibly located for ease of data entry? How fast can you carry out your most common actions? Can you achieve the kind of throughput that has been specified for the system? Users have an internal specification for a good user interface and the system needs to be tested against it. The tests will be informal and mostly embedded in other tests, such as tests of business process flow, but it is vital that concerns about the user interface are highlighted through incident reports.

This is not the same thing as usability testing, which is more technical system testing, but users will be interacting with the system. If you have been trained on how to use the system before you begin testing, you will be able to identify any user interface problems that training does not prepare you for, and your insights into what works well and what does not should be recorded. So although we will not design usability, reliability, availability or a host of other specialist tests, we will be able to raise incident reports on any aspect of the system that causes us concern about how it will behave when newly trained users are expected to use it to deliver real results.

Think about performance

Related to the user interface evaluation, we need to know how easily and quickly we can work with the business process data requirements. This is not a formal performance test (some formal performance testing may well have been done during system testing) but it is a final check that the system can deliver the throughput needed to achieve the proposed business benefits.

Remember negative testing

We explored some negative test cases when we looked at BVA, but it is useful to be aware that UAT can often focus on those tests that are positive in nature. A positive test is one that demonstrates that something that should happen does happen – a function works, for example.

Negative testing is concerned with ensuring that what should not happen does not happen. This is a much more difficult and time-consuming approach to testing because there are many more negative test cases than positive test cases. Here is a very simple example.

Example 7.3

An input field is defined as a six-character alphabetic field. A valid input will have up to six alphabetic characters. Any other input is not valid and should trigger an error message.

As a positive test we can enter a single alphabetic character, two alphabetic characters and so on up to six alphabetic characters. The system should respond according to the input data.

Negative tests must now fill in all the other possibilities: an empty field, a field containing five alphabetic characters and a single numeric character; and a field containing one or more of the non-alphanumeric characters. A complete set of negative tests would need to include every possible combination of six keyboard characters that is not in the range of one to six alphabetic characters.

This is daunting, but is it important? It is clearly important to confirm that the system will recognise input that is not valid and behave appropriately because users will inevitably make mistakes at some point in the future and we do not want the system to respond in a way that will cause problems (for example by ‘freezing’ the input screen).

We cannot exhaust all possible negative tests for each and every scenario because it would take an infinite amount of time. We equally cannot ignore negative testing because the impact of an inappropriate response will be unknown. There has to be a compromise.

The most effective compromise is to add some negative testing to every positive test, using what we already know about user behaviour such as the most common keying errors. If we give some thought to the kinds of mistakes users are most likely to make and run limited negative testing around those mistakes, we can ensure we have made the system a little more robust than it otherwise would be. Negative testing is one good way to exploit the benefits of EP and BVA.

Things that we should not test

Just to reinforce what has already been said about usability testing and performance testing, we should not test what has already been tested, nor things that we are not technically competent to test, nor things that we do not have a test environment for.

Regression

If we raise incident reports that identify defects that are subsequently fixed, we need to rerun the test that originally failed. We will also need to schedule some regression testing to check that the changes made have not affected any other part of the system.

Testing perspective

When you have considered all these alternative approaches and weighed them up, always bear in mind the single most important idea behind UAT. It is the end-user perspective, the end-users’ instincts for what will or will not work, the end-users’ experience and the end-users’ common sense that will generate the best tests for UAT. Identifying all the ways things can and do go wrong in a system they have previously used will provide a long list of relevant tests without even thinking about techniques or theory.

DESIGNING TEST SCRIPTS

Detailed or high level?

Some user tests are designed in such detail that the user merely executes the test script and reports whether the program passed or failed it. This is a good way to design tests if your goal is to provide a carefully scripted demonstration of the system, covering all the processes and inputs you need to test.

If your goal is to discover what problems a user will encounter in the real use of the system, your task is more difficult. UA tests are quite expensive to administer and the risk is that they may not yield much useful information. A good user test must allow enough room for cognitive activity by the user while providing enough structure for the user to report the results in a way that is useful to those who have to fix the problem.

The answer is to provide a mixture of high-level tests and more detailed testing in order to capture all the processes and data that need to be covered and the issues that may arise as a result of a more intuitive approach to using the system by the end-user. High-level tests can contain as little information as ‘log on and book a training course’, while detailed test scripts may go down to a level that elaborates all the steps defined in the test case.

Preparing a simple test script

Test scripts are the lowest-level detail required to run the test. Usually when we refer to test scripts we mean the formal documents that describe the test steps and expected outcomes in detail and allow the tester to note any errors against those detailed steps. Test scripts can also be informal and you may find that a lot of useful feedback comes from a test script that simply says: ‘Log on and request a day off.’ Formal testing is required for UAT so it is important to remember that the kind of feedback received from informal testing may not relate directly to whether specific requirements were met. We can easily make the informal test formal by creating a simple script.

Example 7.4

Table 7.4 Simple login test script

Test scripts versus test cases

If a test case describes a single piece of functionality that requires testing, then every test case will require a test script. A script should be created for each test case because the purpose of the test script is to provide the detailed test steps to the UA tester. Essentially, for a given test plan, you should only need to run a single test script to consider that the test case has been executed. If you need to run more than one, the test case contains multiple conditions and more test cases are needed (one for each condition).

However, test scripts are more flexible than that. A test case can be run many times with different input data with a single test script, or a number of different data entries can be tested as part of the same set of test scripts to cover a number of different test cases.

Test scripts are not usually executed in isolation but combined into a collection that represents a scenario, perhaps related to a single business process or different elements of a user interface. The scenario should represent a process or actions that otherwise logically follow one another or represent different data entries that test the same field or functionality.

Example 7.5 The Excelsior Contracts system

This example relates to part of the Excelsior system called Contracts. The Contracts functionality allows users to create, manage and store contracts that the organisation has entered into, or intends to enter into, with suppliers and third parties.

There are two user roles that interact with the Contracts functionality in specific ways:

•  The Contracts team member can create and manage the contracts but not approve them.

•  The Contracts team manager cannot create or manage contracts but can approve them.

The Contracts system allows users to see and access contracts from a number of different views that represent the status of the contract:

•  new;

•  in progress;

•  awaiting approval;

•  approved;

•  declined;

•  signed;

•  archived.

The test scripts below have been written to check that for each of the two user roles, the system displays the correct status, columns, data and links on the In Progress screen. The In Progress screen lists all the contracts that have not been finalised and have not yet been sent for approval.

4.11 Test script 17 – Check the functionality on the In Progress view

Purpose: ensures the In Progress view contains the expected columns and that data are mapped correctly, displaying documents with a status of In Progress appropriate to the user role.

4.11.1 Test criteria/procedure

1.  Check the In Progress view contains the expected columns for a user with the role of Contracts team member. Check the document details are as expected for a user with the role of Contracts team member, the content of the data on the view are mapped to the correct columns and the links on the screen function as expected.

2.  Check the In Progress view contains the expected columns for a user with the role of Contracts team manager, Check the document details are as expected for a user with the role of Contracts team manager, the content of the data on the view is mapped to the correct columns and a link to approve the document details is available to the Contracts team manager.

Test #1 – Check the In Progress view contains the expected columns for a user with the role of Contracts team member.

Test #2 – Check the In Progress view contains the expected columns for a user with the role of Contracts team manager.

Note that, as is the case in the example used above, information from the test procedure and the test case, such as the expected results and the preconditions, may also appear on the test procedure specification. This makes it easier to understand and schedule each test case without having to read the test steps.

DATA CREATION

Creating the master data that can be used for testing by the business users is a key part of the preparation for UAT. Before any test script is executed the correct test data must be available in the system and no test should fail because the correct data were not available to be selected or otherwise used by the test team.

What is test data?

When the test cases are created the input data are noted on the test case or a list of possible input values may be provided for UAT. All the input data that are going to be tested should therefore be a known quantity. Test data also extends to the data that need to be available in the system for testing to be completed successfully. Test data are any kind of input, any kind of file that is loaded by the application, or entries read from a database. This may require you to create logins and passwords and any data that testers must verify or use. A great starting point to determine what data you need is to look at the business processes and then look at the scripts. For example if you are testing a training system, upload sample training courses for testers to take during UAT.

Input data can be included in test scripts or provided separately in a list or spreadsheet by test case that references what data should be used in each script.

For any test case or test script, the data that need to be available for that test is a precondition so we have to consider all the test data at test design. The earlier this is done the better because making up user accounts and populating them with data for testing may be time-consuming and it will need support from specialists who may not be available at short notice.

Depending on the data requirements for UAT the data available in the system may be previous test data, standard production data or a copy of data from a live environment. Data can also be created during one test script that supports later test scripts. If a script requires the UA tester to create a new account for example, a later test script may need the tester to log in to an existing account for which the details of the account that was previously created may be used. Standard production data are unlikely to suffice completely unchanged and some test-specific data will probably need to be added.

If the requirements include a check of the performance of the system, you will require a large data set. If the system acquires data from or updates data to a large database, then a large data volume plays an important role while testing the system for performance. Clearly a system with very little data in it will perform these tasks more quickly, all things being equal, than a system with a lot of data. If you need real-time data that are impossible to create manually, then you could ask the project manager to make them available from the live environment. If this is not possible, some performance tests should already have been carried out as part of system testing that may provide reassurance to the business representatives about the performance of the system.

One tempting source of data is the live environment but be aware that using copies of live data or the live environment can infringe the Data Protection Act, compromise commercially sensitive information and represent a significant risk to the organisation. If use of live data is being considered it will need approval at the highest level, and even if approved the security of the data must be paramount:

•  Is encryption in place that protects the database?

•  Will you know whether test data has been compromised?

•  Who is responsible for securing test data?

•  Does the data contain sensitive staff or client information?

Even if all these questions can be answered satisfactorily remember that if real data is displayed during testing, even when the database is encrypted, it can be downloaded or copied using a simple screen grab. If it is crucial to UAT that real data is used and its use is approved, data masking can be used to retain the data’s properties yet create new, depersonalised, data. Finally, remember that the transfer of live data could overwrite any manually created test data.

What defines good test data?

The perfect test data are the smallest data set that allows UAT to identify all the errors in the system. Test data should obviously incorporate all the aspects of the system being tested but not exceed cost and time constraints for preparing test data and running tests. The exception to this rule is where it is quicker to use a copy of the live environment than to create test data from scratch or from the standard production data, in which case there may be more data than needed but the provision of this excess data has not affected project timescales or costs.

Preparing appropriate test data is a key part of UAT preparation. The test data set should be as close to perfect as possible in terms of cost and time.

What does a test script look like?

A completed test script will identify the test condition(s) and the test case(s) from which it was derived. It must also define the test precondition(s), input data, expected outcome and post-condition(s). Finally, it must incorporate the test data to be used in running the test, either embedded in the test script or in a file that is referenced from the test script.

Here is one example of what a test script might look like. There are many valid variations but a valid test script must include the details recorded in the example.

Example 7.6

An example test script

One of the requirements of Excelsior is that users can only access the parts of the system they are allowed to. Everyone in the company is able to access the HR module for instance, but only certain roles are allowed to access the Contracts module. This means the system needs to check that the person already logged on to Excelsior is on the list of those allowed access to the particular module they are trying to access. This is done using the system logon details.

1.2.1 Test scenario

1.  Check an unverified user cannot log on to Contracts.

2.  Check that a verified user can log on to Contracts.

Test scenario #1 – Check an unverified user cannot on to Contracts.

Test scenario #2 – Check a verified user can log on to Contracts.

These are very short examples of the information that should be contained in a test script and may not be the most efficient when run in isolation.

Questions:

1.  What other related testing do you think ought to be carried out?

2.  As part of what other test scenarios could these tests be run?

Our answers can be found in Appendix B.

Example 7.7

A data input sheet

You can make the test data to be entered part of the test script so that the values to be entered are given in the test description column. However, if you would like the end-users to test a number of different values on the same screen, you can also consider providing a separate data input sheet to be used in conjunction with the test script. In the BVA example we used in this chapter you will probably want testers to attempt to enter 0, 1, 6 and 7 characters as part of the same test of the single field in order to save time. These values can be listed on a separate sheet that is used in conjunction with the test script. Data input sheets are also useful where a complex set of combinations of values need to be tested, for instance where a number of different types contract need to be tested that each contains different fields and needs to be accessible by different user roles. Keeping the data input values separate from the test scripts also gives you greater control over what is tested when the number of testers is reduced for instance, and saves having to extrapolate data from the missing tester’s scripts. Note that, where possible, making end-users enter information based on real examples (but not using real data), customer or address details will add to the usefulness of testing.

Shaded areas indicate that the field does not need to be completed.

Table 7.5 A contract data input sheet

Note that a data input sheet can contain the data relevant per test script or can contain multiple entries that should be applied as part of the same test script. How the data input sheets should be used by the testers should be included in the UAT training.

What have I learned?

Test your knowledge of Chapter 7 by answering the following questions. The correct answers can be found in Appendix B.

1. Traceability is important because:

A.  It links test conditions, cases and scripts back to the requirements

B.  It is a measure of risk

C.  It makes sure that the data for testing are complete

D.  It ensures each requirement has a unique ID

2. A test condition is:

A.  A condition that must be met for testing to take place

B.  A combination of features that are tested as part of a scenario

C.  A statement about a feature that can be verified by a test case

D.  An exit criterion for UAT

3. What three things should you be most aware of when it comes to test-case design?

A.  Have the requirements been covered?

B.  How many testers will there be?

C.  Do the cases cover the processes?

D.  Are the user interfaces being sufficiently tested?

E.  Which cases represent the greatest risk?

4. Which of the following is the most important reason for applying BVA?

A.  To keep the number of negative test cases to a minimum

B.  To identify all the negative test cases

C.  To identify the negative test cases most likely to uncover problems

D.  To identify the partitions between equal sets of data

Some questions to consider (our responses are in Appendix B)

1.  The test cases have been created. When deciding what order they should be placed in, the project manager wants you to focus on risk and the project sponsor wants you to focus on processes. How do you decide what order they should be placed in?

2.  On the first day of UAT half the team are off ill with a cold. What decision would you make about whether to run testing? What tests do you think you could run?