7 Preparing for the Exam
Not everyone who doesn’t study for the exam will fail. But everyone who does fail didn’t study enough.
A frequent warning from Rex when teaching ISTQB courses
The seventh chapter of this book is concerned with topics that you need to know in order to prepare for the ISTQB Advanced Technical Test Analyst exam. The chapter starts with a discussion of the ISTQB Advanced Technical Test Analyst syllabus 2012 learning objectives, which are the basis of the exams.
Chapter 7 of this book has two sections:
1. Learning Objectives
2. ISTQB Advanced Exams
If you are not interested in taking the ISTQB Advanced Technical Test Analyst exam, this chapter might not be pertinent for you.
Learning Objectives
National boards and exam boards develop the Advanced level exams based on learning objectives. A learning objective states what you should be able to do prior to taking an Advanced level exam. Each Advanced level exam has its own set of learning objectives and its own text. There are no shared learning objectives or text across the three separate Advanced level syllabi, so don’t bother to read or study the other two Advanced level syllabi if you’re studying for the Advanced Technical Test Analyst exam. We listed the learning objectives for the Advanced Technical Test Analyst exam at the beginning of each section in each chapter.
The learning objectives are at four levels of increasing difficulty. Question writers will structure exam questions so that you must have achieved these learning objectives to determine the correct answers for the questions. The exams will cover the more basic levels of remembrance and understanding implicitly as part of the more sophisticated levels of application and analysis. For example, to answer a question about how to create a white-box test case, you will have to remember and understand the contents of such a document. So, unlike the Foundation exam, where simple remembrance and understanding often suffice to determine the correct answer, an Advanced exam requires you to apply or analyze the facts that you remember and understand in order to determine the correct answers.
Let’s take a closer look at the four levels of learning objectives you will encounter on the Advanced exams. The tags K1, K2, K3, and K4 are used to indicate these four levels, so remember those tags as you review the Advanced Technical Test Analyst syllabus.
Level 1: Remember (K1)
At this lowest level of learning, you will be expected to recognize, remember, and recall a term or concept. Watch for keywords such as remember, recall, recognize, and know. Again, this level of learning is likely to be implicit within a higher-level question.
For example, you should be able to recognize the definition of failure as follows:
non-delivery of service to an end user or any other stakeholder, and
actual deviation of the component or system from its expected delivery, service or result.
This means that you should be able to remember the ISTQB glossary definitions of terms used in the ISTQB Advanced Technical Test Analyst syllabus. Expect this level of learning to be required for questions focused on higher levels of learning like K3 and K4. There are no K1 questions, but your ability to remember the Foundation, Advanced Overview, and Advanced Technical Test Analyst syllabi are all examinable at the K1 level, embedded within K2, K3, and K4 questions.
Level 2: Understand (K2)
At this second level of learning, you will be expected to be able to select the reasons or explanations for statements related to the topic and summarize, differentiate, classify, and give examples. This learning objective applies to facts, so you should be able to compare the meanings of terms. You should also be able to understand testing concepts. In addition, you should be able to understand test procedures, such as explaining the sequence of tasks. Watch for keywords such as summarize, classify, compare, map, contrast, exemplify, interpret, translate, represent, infer, conclude, and categorize.
For example, you should be able to explain the reason tests should be designed as early as possible:
To find defects when they are cheaper to remove
To find the most important defects first
You should also be able to explain the similarities and differences between integration and system testing:
Similarities: Testing more than one component and testing nonfunctional aspects.
Differences: Integration testing concentrates on interfaces and interactions, while system testing concentrates on whole-system aspects, such as end-to-end processing.
This means that you should be able to understand the ISTQB glossary terms used in the ISTQB Advanced Technical Test Analyst syllabus. Expect this level of learning to be required for questions focused on higher levels of learning like K3 and K4.
Level 3: Apply (K3)
At this third level of learning, you should be able to select the correct application of a concept or technique and apply it to a given context. This level is normally applicable to procedural knowledge. At K3, you don’t need to evaluate a software application or create a testing model for a given software application. If the syllabus gives a model, the coverage requirements for that model, and the procedural steps to create test cases from a model in the Advanced Technical Test Analyst syllabus, then you are dealing with a K3 learning objective. Watch for keywords such as implement, execute, use, follow a procedure, and apply a procedure.
For example, you should be able to do the following:
Design tests to achieve statement, decision, and MC/DC coverage.
Create a set of keywords for keyword-driven test automation.
This means that you should be able to apply the techniques described in the ISTQB Advanced Technical Test Analyst syllabus to specific exam questions. Expect this level of learning to include lower levels of learning like K1 and K2.
Level 4: Analyze (K4)
At this fourth level of learning, you should be able to separate information related to a procedure or technique into its constituent parts for better understanding and distinguish between facts and inferences. A typical exam question at this level will require you to analyze a document, software, or project situation and propose appropriate actions to solve a problem or complete a task. Watch for keywords such as analyze, differentiate, select, structure, focus, attribute, deconstruct, evaluate, judge, monitor, coordinate, create, synthesize, generate, hypothesize, plan, design, construct, and produce.
For example, you should be able to do the following:
Analyze product risks and propose preventive and corrective mitigation activities.
Evaluate a set of tests to determine what level of white-box coverage (if any) is achieved.
This means that you should be able to analyze the techniques and concepts described in the ISTQB Advanced Technical Test Analyst syllabus in order to answer specific exam questions. Expect this level of learning to include lower levels of learning like K1, K2, and perhaps even K3.
Where Did These Levels of Learning Objectives Come From?
If you are curious about how this taxonomy and these levels of learning objectives came to be in the Foundation and Advanced syllabi, then you’ll want to refer to Bloom’s taxonomy of learning objectives, defined in the 1950s. It’s standard educational fare, though you probably haven’t encountered it unless you’ve been involved in teaching training courses.
You might find it simpler to think about the levels this way:
K1 requires the ability to remember basic facts, techniques, and standards, though you might not understand what they mean.
K2 requires the ability to understand the facts, techniques, and standards and how they interrelate, though you might not be able to apply them to your projects.
K3 requires the ability to apply facts, techniques, and standards to your projects, though you might not be able to adapt them or select the most appropriate ones for your project.
K4 requires the ability to analyze facts, techniques, and standards as they might apply to your projects and adapt them or select the most appropriate ones for your project.
As you can see, there is an upward progression of ability that adheres to each increasing level of learning. Much of the focus at the Advanced level is on application and analysis.
ISTQB Advanced Exams
Like the Foundation exam, the Advanced exams are multiple-choice exams. Multiple-choice questions consist of three main parts. The first part is the stem, which is the body of the question. The stem may include a figure or table as well as text. The second part is the distracters, the choices that are wrong. If you don’t have a full understanding of the learning objectives that the question covers, you might find the distracters to be reasonable choices. The third part is the answer or answers, the choice or choices that are correct.
If you sailed through the Foundation exam, you might think that you’ll manage to do the same with the Advanced exams. That’s unlikely. Unlike the Foundation exam, the Advanced exams are heavily focused on questions derived from K3 and K4 level learning objectives. In other words, the ability to apply and to analyze ideas dominates the exams. K1 and K2 level learning objectives, which make up the bulk of the Foundation exam, are often covered implicitly within the higher-level questions.
In addition, unlike with the Foundation exam, the questions are weighted. K3 and K4 questions will be assigned two and three points, respectively, and in most cases K2 questions will be assigned only one point.
For example, the Foundation exam might typically include a question like this:
Which of the following is a major section of an IEEE 829–compliant test plan?
A. Test items
B. Probe effect
C. Purpose
D. Expected results
The answer is A, while B, C, and D are distracters. All that is required here is to recall the major sections of the IEEE 829 templates. Only A is found in the test plan, while C and D are in the test procedure specification and the test case specification, respectively. B is an ISTQB glossary term. As you can see, it’s all simple recall.
Recall is useful, especially when you’re first learning a subject. However, the ability to recall facts does not make you an expert, any more than the ability to recall song lyrics from the 1970s qualifies someone to work as the lead singer for the band AC/DC.
On the Advanced exam, you might find a question like this:
Consider the following excerpt from the Test Items section of a test plan.
During system test execution, the configuration management team shall deliver test releases to the test team every Monday morning by 9:00 a.m. Each test release shall include a test item transmittal report. The test item transmittal report will describe the results of the automated build and smoke test associated with the release. Upon receipt of the test release, if the smoke test was successful, the test manager will install it in the test lab. Testing will commence on Monday morning once the new weekly release is installed.
Should the test team not receive a test release, or if the smoke test results are negative, or if the release will not install, or should the release arrive without a transmittal report, the test manager shall immediately contact the configuration management team manager. If the problem is not resolved within one hour, the test manager shall notify the project manager and continue testing against the previous week’s release, if possible. If the test release fails installation, additionally the test analyst who attempted the installation shall file an incident report.
Assume that you are working as the test manager on this project. Suppose that you have received two working, installable, testable releases so far. On Monday of the third week, you do not receive the test release.
Which of the following courses of action is consistent with the test plan?
A. Filing a defect report describing the time and date at which you first noticed the missing test release
B. Creating a test that describes how to install a test release
C. Sending an SMS text message to the configuration management team manager
D. Sending an email to the project manager and the configuration management team manager
The answer is C. A, B, and D are distracters. A is wrong because it is not that the release didn’t install, it’s that it didn’t even arrive. B is wrong because, while such a test might be useful for installation testing, it has nothing to do with the escalation process described in the test plan. C is consistent with the test plan. D is not consistent with the test plan because the spirit of the one-hour delay described in the test plan excerpt is that the configuration management team manager should have a chance to resolve the problem before the project manager is engaged. In addition, when time is of the essence, email is not a good escalation technique.
As you can see, this kind of question requires analysis of a situation. Yes, it helps to know the contents of documents such as the test plan, defect report, and test item transmittal report. In fact, you’ll probably get lost in the terminology if you don’t know the standard. However, simply knowing the IEEE 829 standard for test documents will not allow you to get the right answer on this question except by chance.
Scenario-Based and Pick-N Questions
Further complicating this situation is the fact that many exam questions will actually consider a scenario. In scenario-based questions, the exam will describe a set of circumstances. It will then present you with a sequence of two, three, or even more questions based on that scenario.
For example, the questions about the scenario of the test plan excerpt and the missing test release might continue with another pair of questions:
Assume that on Monday afternoon you finally receive a test release. When your lead test analyst attempts to install it, the database configuration scripts included in the installation terminate in midstream. An error message is presented on the database server in Cyrillic script, though the chosen language is US English. At that point, the database tables are corrupted and any attempt to use the application under test results in various database connection error messages (which are at least presented in US English).
Consider the following possible actions:
I. Notifying the configuration management team manager
II. Notifying the project manager
III. Filing a defect report
IV. Attempting to repeat the installation
V. Suspending testing
VI. Continuing testing
Which of the following sequence of actions is in the correct order, is the most reasonable, and is most consistent with the intent of the test plan?
A. I, II, V
B. V, I, IV, III, I
C. VI, II, I, III, IV
D. II, I, V
The answer is B, while A, C, and D are distracters. A is wrong because there is no defect report filed, which is required by the test plan when the installation fails. C is wrong because meaningful testing cannot continue against the corrupted database because the project manager is notified before the configuration management team manager and because the defect report is filed before an attempt to reproduce the failure has occurred. D is wrong because the project manager is notified before the configuration management team manager and because no defect report is filed.
As you can see, with a scenario-based question it’s very important that you study the scenario carefully before trying to answer the questions that relate to it. If you misunderstand the scenario—perhaps due to a rushed reading of it—you can anticipate missing most if not all of the questions related to it.
In addition to scenario questions, you’ll see another new type of question on the Advanced exams, Pick-N questions. In these questions, you will pick two or three answers out of a list of five or seven options, respectively. These questions are often a harder form of a Roman-type question, in that it is more difficult to use a process of elimination to select the right answer. And, if you only get some of the right answers—that is, one out of two or two out of three—you might not get partial credit.
Let us go back to this question of learning objectives for a moment. We said that the exam covers K1 and K2 learning objectives—those requiring recall and understanding, respectively—as part of a higher-level K3 or K4 question. There’s an added complication with K1 learning objectives: They are not explicitly defined. The entire syllabus, including glossary terms used and standards referenced, is implicitly covered by K1 learning objectives. So, you’ll want to read the Advanced Technical Test Analyst syllabus carefully, a number of times.
Not only should you read the Advanced Technical Test Analyst syllabus, but you’ll need to go back and refresh yourself on the Foundation syllabus. Material that is examinable at the Foundation level is also examinable at the Advanced level, especially when material in the Advanced level builds on the Foundation level. It would be smart to take a sample Foundation exam and reread the Foundation syllabus as part of studying for the Advanced Technical Test Analyst exam.
On the Structure of the Exams
So, enough about the questions on the exam; what can you expect from the exam itself? In the Advanced Technical Test Analyst exam, you will get 45 questions. You’ll have two hours (120 minutes) to complete it. (If your native language is not the same as the language of the exam, you’ll be allowed an extra 30 minutes, for a total of 150 minutes.) Most of our customers find that the time limitation is not an issue, unlike with the Foundation exam, where a significant percentage of people need the entire hour.
Now, with the Foundation exam, you could estimate how many questions were going to be asked on each section by using the time allocated in the syllabus for that section. This trick will not work on the Advanced Technical Test Analyst exam. For the Advanced Technical Test Analyst exam, when we wrote the exam guidelines, we used a process of weighting the learning objectives for importance. So, for Chapter 1, here are the number of questions per learning objective:
TTA-1.3.1 (K2): 1
TTA-1.x.1 (K2): 1
There will be two questions on Chapter 1. Remember that an exam question might cover multiple learning objectives, so it could be that a single question counts against two (or more) learning objectives for this chapter or across multiple chapters.
For Chapter 2, here are the number of questions per learning objective:
TTA-2.2.1 (K2): 1
TTA-2.3.1 (K3): 1
TTA-2.4.1 (K3): 1
TTA-2.5.1 (K3): 1
TTA-2.6.1 (K3): 1
TTA-2.7.1 (K2): 2
TTA-2.8.1 (K4): 2
There will be nine questions on Chapter 2.
For Chapter 3, here are the number of questions per learning objective:
TTA-3.2.1 (K3): 3
TTA-3.2.2 (K3): 1
TTA-3.2.3 (K3): 1
TTA-3.2.4 (K2): 1
TTA-3.3.1 (K3): 1
For Chapter 3, you will see seven questions, as specified above.
For Chapter 4, here are the number of questions per learning objective:
TTA-4.2.1 (K4): 2
TTA-4.3.1 (K3): 1
TTA-4.4.1 (K3): 1
TTA-4.5.1 (K3): 1
TTA-4.x.1 (K2): 1
TTA-4.x.2 (K3): 2
TTA-4.x.3 (K2): 2
TTA-4.x.4 (K3): 2
For Chapter 4, you will see 12 questions, as specified above.
For Chapter 5, here are the number of questions per learning objective:
TTA-5.1.1 (K2): 1
TTA-5.2.1 (K4): 2
TTA-5.2.2 (K4): 2
You’ll see five questions on Chapter 5.
For Chapter 6, here are the number of questions per learning objectives:
TTA-6.6.1 (K2): 1
TTA-6.2.1 (K2): 1
TTA-6.2.2 (K2): 1
TTA-6.2.3 (K2): 1
TTA-6.2.4 (K3): 1
TTA-6.3.1 (K2): 1
TTA-6.3.2 (K2): 1
TTA-6.3.3 (K2): 1
TTA-6.3.4 (K2): 1
TTA-6.3.5 (K2): 1
You’ll see 10 questions for Chapter 6.
Based on the suggested point allocation—1 point for a K1 question, 2 points for a K3 question, and 3 points for a K4 questions—you’ll see 79 total points. You have to get 52 points (65 percent) to pass.
Okay, we realize that you might be panicking. Don’t panic! Remember, the exam is meant to test your achievement of the learning objectives in the Advanced Technical Test Analyst syllabus. This book contains solid features to help you do that. Ask yourself the following questions:
Did you work through all the exercises in the book? If so, then you have a solid grasp of the most difficult learning objectives, the K3 and K4 objectives. If not, then go back and do so now.
Did you work through all the sample exam questions in the book? If so, then you have tried a sample exam question for most of the learning objectives in the syllabus. If not, then go back and do so now.
Did you read the ISTQB glossary term definitions where they occurred in the chapters? If so, then you are familiar with these terms. If not, then return to the ISTQB glossary now and review those terms.
Did you read every chapter of this book and the entire ISTQB Advanced Technical Test Analyst syllabus? If so, then you know the material in the ISTQB Advanced Test Manager syllabus. If not, then review the ISTQB Advanced Technical Test Analyst syllabus and reread those sections of this book that correspond to the parts of the syllabus you find most confusing.
We can’t guarantee that you will pass the exam. However, if you have taken advantage of the learning opportunities created by this book, by the ISTQB glossary, and by the ISTQB Advanced Technical Test Analyst syllabus, you will be in good shape for the exam.
Good luck to you when you take the exam, and the best of success when you apply the ideas in the Advanced Technical Test Analyst syllabus to your next testing project.