In this chapter, we will cover:
This section intends to cover:
Test automation framework (often referred to in the testing industry as testware automation frameworks) is an integrated software solution that defines the rules for the development, maintenance, and execution of automated test assets. Typically, such a solution also defines how test results are reported, how runtime errors are handled, and how test data is managed. A test automation framework comprises function libraries, data sources (for example, Excel or XML files and DB), object models (such as stored in an object repository), and it may also include additional reusable external modules (for example, DLL files, COM objects, configuration scripts, and so on).
A well-designed test automation framework typically contributes towards lower reduce development and maintenance costs of automated test assets. This means that, ideally, a framework should use generic and agnostic design patterns to provide a solution that is relevant, effective, maintainable, efficient, manageable, portable, reliable, and diagnosable.
The preceding list is not meant to be exhaustive, but provides the main features that are expected from a robust automation framework.
Another aspect of test automation that is quite often neglected, but one that can make a great contribution to design patterns, is the command wrapper (used when implementing test procedures and functions). This will also be explained later in this section.
There are several generic types of test automation frameworks' design patterns:
As far as which framework type to select is concerned, it depends on the requirements we have to meet and the level of automation maturity required. No single pattern type is a silver bullet, and a more complex framework should not be used if the additional complexity is not expected to yield significant added value. We shall now define and explain the different patterns mentioned earlier.
A modular-driven pattern uses classes and objects to encapsulate all the entities involved in the automation project. Basically, a modular-driven pattern can be considered a particular case of the hybrid approach. This is because the utilization of classes and objects is typically also accompanied by the usage of the features of the keyword-driven pattern (in more complex implementations) and the data-driven pattern.
Using classes (as described in detail in the previous chapters) allows for a better organized code base, typically resulting in a more concise, clearer, as well as more flexible, extensible, and, maintainable code. This also enables us to use object-oriented design patterns to maximize reusability and enhance the performance of our test suites. For example, a typical modular-driven pattern has a well-defined structure (or template) to implement runnable processes using the command wrapper pattern (refer to Chapter 7, Using Classes). This, in turn, simplifies the way data is loaded, saved, and shared; the way events are reported to the log file; how the flow is controlled; and finally, the way exceptions are handled.
A data-driven pattern supports test iterations and flow branching according to external input data. UFT can, by default, offer such a framework out-of-the-box; all we have to do is define the parameters in the DataTable, retrieve the values of these in our code where appropriate, and set the test or Action iterations (datasheet rows range) we wish to execute (refer to Chapter 1, Data-driven Tests).
A keyword-driven pattern is also a data-driven pattern, but with another level of abstraction; commands are encapsulated as data entities (keywords) and mapped to actual functions implemented in code.
These keywords are listed in their planned order of execution in some kind of data source (for instance, Excel worksheet, XML file, or DB). Typically, this data source would also include the corresponding parameter values for each operation. A central mechanism, often referred to as a controller or parser, reads the sequence of keywords and invokes the procedures associated with them.
Central to a keyword-driven pattern is the desire to provide nontechnical test engineers, who do not possess coding skills, with the ability to implement automated test scenarios (also referred to as test scripts) using a high-level structured language composed with words that represent underlying coded processes of varying complexity.
This pattern is very popular, and its main advantages are:
PressOK
) or a procedure (for example, Login
). The latter is basically ABT or BPT, as the functionality offered by UFT requires connectivity with HP ALM/QC.A keyword-driven pattern also has some challenges that require our attention. For instance, it typically carries the additional cost of developing a User Interface (UI) to manage, build, and validate the data entered by the designer of the tests while defining automated test scenarios.
A hybrid pattern is one that combines some or all the features of the mentioned framework types. It is typically implemented with a strong emphasis on design patterns, to provide a solution that yields high scores in (source Hybrid (keyword/data-driven) frameworks, ANZTB Conference, 2010 by Jonathon Wright):