Instructional Objectives

Much has been made of targeted outcome statements, or instructional objectives, perhaps beginning with Robert Mager's insightful and pragmatic how-to books on instructional design, such as Preparing Instructional Objectives (Mager, 1997c), Measuring Instructional Results (Mager, 1997b), and Goal Analysis (Mager, 1997a). Instructional designers are taught to prepare objectives early in the design process and to list learning objectives for learners at the beginning of each module of instruction. Few classically educated instructional designers would consider omitting the opening list of objectives.

Mager provides three primary reasons why objectives are important:

1. Objectives…are useful in providing a sound basis for
   1. Selecting or designing instructional content and procedures
   2. Evaluating or assessing the success of the instruction
   3. Organizing learners' own efforts and activities for the accomplishment of the important instructional intents. (Mager, 1997c, p. 6)

There's no doubt about the first two uses. If you don't know what abilities you are helping your learners build, how can you know if you're having them do the right things? As I've emphasized before, success depends on people doing the right thing at the right time. If no declaration of “the right thing” is established, you can neither develop effective training nor measure the effectiveness of it—except, perhaps, by sheer luck. Objectives are a studied and effective way of declaring what the “right thing” is. As Mager says, “if you know where you are going, you have a better chance of getting there” (Ibid).

It's the third point that's of interest here—using objectives to help learners organize their learning efforts. Certainly, objectives can help. When objectives are not present, learners must often guess what is important. In academic or certification contexts when objectives are not shared, learners must guess what will be included in the all-important final examination. After they have taken the final exam, learners know how they might have better organized their learning efforts. But, of course, it's too late by then.

There is a bit of gamesmanship here that some instructors use. They justify it by saying comprehensive tests would be too difficult to prepare, require too much time to take, and be too laborious to score. If students don't know what's going to be tested, they'll have to prepare themselves for testing on all topics, even if the test only covers a sampling.

There's logic to it, but definitely downsides as well. Are the topics which are not tested unimportant? If scores are to be reflective of performance mastery, what are we to assume about untested skills? Aren't some performance skills more important than others? And if so, shouldn't learners be aware of this so they can concentrate their study on the most important things? Aren't some objectives just enablers, the mastery of which has little value on its own? Do courses impart so many useful skills they can't all be tested?

For objectives to provide benefits to learners, learners have to read, understand, and think about them. Unfortunately, learners rarely spend the time to read objectives, much less use them as learning tools. Rather, they discover that the objectives screen is, happily, a screen that can be skipped over quickly. Learners think, “I'm supposed to do my best to learn whatever is here, so I might as well spend all my time learning it rather than reading about what I'll learn.” Next.

Lists of Objectives Are Not Motivating!

Let's just admit it: Listing objectives at the start of a courseware module looks professional and makes us feel we've done the right thing, but those lists probably aren't going to contribute much to the learning experience. Many designers hope objectives will not only help learners organize their study but also enhance their motivation to learn the included content. Will they? Obviously, objectives do nothing if learners don't read them as we've said. But just how readable are they? Following Mager, knowledgeable instructional writers know objectives should have three parts (see Figure 10.2).

Instructional writers have learned the importance of writing behavior-based objectives that are observable and measurable. (See Table 10.2 to get the gist of it.)

Measurable behavioral objectives are, indeed, critical components to guide the design of effective training applications. Designers need such objectives, and none of these components should be missing from their design plans. But the question here is about their use as learning motivators.

You can hardly yawn fast enough when you read a stack of statements containing “proper” objectives like:

Given a typical business letter, you will be able to identify at least 80 percent of the common errors by underlining inappropriate elements or placing an “X” where essential components are missing.

After reading a few of these objectives, will you be motivated? Sure. You'll be motivated to skip the rest and move on. Objectives are important for us as designers and attainable outcomes are valuable to learners, but listing such statements as these in bullet points at the start of a program is boring and ineffective. If there are better ways to motivate learners, what are they?

How about Better-Written Objectives?

We can and should write objectives in more interesting ways and make them more relevant to the learner. Indeed, when they're deciding just how much energy and involvement to commit, learners want to know in what way the material will be meaningful to them personally (i.e., how the instructional content relates to their personal network of motivations). Effective objectives answer the question and give motivation a little boost (see Table 10.3).

Remember the more fully learners recognize and value the advantages of learning the material at hand, the more effective the material will be. But if you agree on this point, you might wonder whether a textual listing of objectives is really the best way to sell anyone on learning. Good question. Yes, we can do better.

Don't List Objectives

If learners aren't going to read objectives, even valuable and well-written objectives, listing them at the beginning of each module of instruction isn't a very useful thing to do. In fact, it may do more harm than good since it communicates to learners from the beginning that yours will be just another typical boring e-learning experience.

In e-learning, we have techniques for drawing attention to vital information. We use interactivity, graphics, and animation—in short, all the powers of interactive multimedia—to help learners focus on beneficial content. Why, then, shouldn't we use these same powers to portray the objectives and sell the learning opportunity to the learner?

Instead of just listing the objectives, provide meaningful and memorable experiences.

Example 1: Put the Learner to Work

Perhaps the clearest statement of possible outcomes comes from setting the learner to work immediately on a stated task. If learners try and fail, at least they'll know what they are going to be able to do when they complete the learning far more clearly than they could glean from reading a list of objectives.

In this award-winning example, Expedia needed to train all their phone-based customer service representatives on a vast amount of travel knowledge for every major city in the United States. Traditional methods of informative presentations with assessments had not been effective, leading Expedia to a learner-centric design focused on the authentic on-the-job activities trainees could expect to perform. The content ranged from information about local attractions to neighborhood characteristics, hotels, and transportation choices. (See Figure 10.3.)

Let's walk through the example. Learners are presented with a series of customers who wish to book a dream vacation. The performance task requires learners to choose the best lodging, attractions, activities, and transportation based on simulated customers profiles and preference-revealing quotes. (See Figure 10.4.)

To find the best lodging choice, a comprehensive list of neighborhoods is presented with descriptions. Each neighborhood contains a number of hotels, at least one of which matches the customer needs and preferences. Other categories contain simple lists with item descriptions that provide sufficient information to match specific needs/wants. (See Figure 10.5.)

Once the entire trip has been planned, the learner receives a comment card with customer feedback. (See Figure 10.6.)

The learner must then correct all problems with the trip until the customer is completely satisfied. (See Figure 10.7.)

At that point, another customer calls in with different needs and preferences. Once learners have repeated enough exercises, they have much higher prospects of getting a trip right on the first try. A trip planned successfully on the first try is quite rewarding. This learner-centric design with multiple scenarios effectively motivates learners to familiarize themselves with each neighborhood, hotel, attraction, and so forth during the pursuit of the perfect vacation.

You might be concerned that some failures, especially early on, will frustrate or demoralize learners. It is always an appropriate concern that initial challenges may produce more negative frustration than positive motivation. Interactive features must be provided to protect learners from becoming overwhelmed and discouraged. Either the software and/or the learner must be able to adjust the level of challenge when appropriate. Please see Magic Key 2 for a discussion of risk management.

Example 2: Drama

Motivation is not an exclusively cognitive process. Motivations involve our emotions and our physiological drives, as well. Good speakers, writers, and filmmakers are able to inspire us to take action, to reevaluate currently held positions, and to stir emotions that stick with us to guide future decisions.

Imagine this scenario. Airline mechanics are to be trained in the process of changing tires on an aircraft in its current position at a gate. You can easily imagine the typical first page of the training materials to read something like this:

Now imagine this approach:

You press a key to begin your training. Your computer screen slowly fades to full black. Lightning crashes and your screen flickers bright white a few times. You see a few bolts of lightning between the tall panes of commercial windows just barely visible. Through the departure gate window, you see an airplane parked at the gate. The rain is splashing off the fuselage. A gust of wind makes a familiar threatening sound.

The scene cuts to two men in yellow rain slickers shouting at each other to be heard over the background noise of the storm and airport traffic.

“You'll have to change that tire in record speed. The pilot insists there's a problem, and that's all it takes. We have to change it. There's going to be a break in the storm and there are thirty-some flights hoping to get out before we'll probably have to close down again.”

“No problem, Bob. We're on it.” Return to the windows, where we now see people at the gate—a young man in business attire is pacing worriedly past an elderly woman seated near the windows.

“Don't worry, young man. They won't take off if the storm presents a serious danger. Even in a storm and with all the possible risks of air travel, I still think it's the safest way for me to get around.”

“Oh, thanks, ma'am. But you see, my wife is in labor with our first child. I missed an earlier flight by ten minutes, and now this is the last one out tonight. I'm so worried this flight will be delayed for hours—or worse, even cancelled due to weather. I'm not sure what my options are, but I need to get home tonight.”

Back outside, Bob runs up to three mechanics as they run toward him in the rain.

“All done, Bob. A little final paperwork inside where it's dry, and she's set to fly.”

“Your team must have set a record. Even under pleasant circumstances, I don't know any other employees who could have done such a good job so fast. The departure of this aircraft won't be delayed one minute because of that tire change. A lot of people will benefit from your work tonight.

“This is going to headline the company's newsletter!”

I hope you can see how this dramatic context can much more effectively communicate the learning objectives and motivate learners to pursue them. Who wouldn't want to be a hero in this circumstance? Who wouldn't begin thinking, “I'll bet I could learn to do that…maybe even do it better.” And, then, “I wonder what it takes. Hope I get a chance to find out in this learning program.”

An emotion-arousing motivational event like this can be done through everything from a few stick figures with audio to a full fidelity video of professional mechanics or actors. The expense can be kept small, yet the benefit to performance and learning can be invaluable.

Problems with Risk as a Learning Motivator

Consider instructor-led training for a moment. In the classroom, putting the learner at risk can be as simple as the instructor posing a question to an individual learner. The risk of embarrassment goes far beyond failing to answer correctly. Public performance can affect social status, social image, and self-confidence for better or worse. Even if we're in a class in which we know none of our classmates, being asked a question typically causes a rush of adrenaline because of what's at stake.

Competition is a risk-based device used by many classroom instructors to motivate learners. In some of my early work with PLATO, I managed an employee who considered one of PLATO's greatest strengths to be its underlying communication capabilities that were able to pit multiple learners against one another in various forms of competitive interplay. Although these capabilities were often used just for gaming, he was intrigued with the idea of using them to motivate learners.

Competitive e-learning environments can certainly be created, but, unfortunately, pitting learners against one another may create a counter-productive win/lose environment. It may be true that even the losers are gaining strengths, and they may be effectively motivated to do their best as long as the situation or their competitors don't overly intimidate them. Anonymity can help. But it's difficult to prevent nonwinners from seeing themselves as losers. As one axiom puts it:

As trainers, our goal is to make all learners winners. And to that end, we need learners focusing on skill development, not each other.

Private versus Social Learning Environments

There are advantages to being in the company of others when we learn. We are motivated to keep up with other learners and benefit from their questions. We learn from watching the mistakes and successes of others. Communication with others helps round out our understanding. In addition, successful public performance gives us confidence—a performance attribute that can be lacking when learning and practice take place only in private.

However, the risk of public humiliation causes, for many, an immediate and unpleasant rush of adrenaline. While a successful public performance can easily meet our two essential criteria of being a meaningful and memorable learning event, the event may be memorable because of the fear associated with it. And although traumatic experiences may be memorable, the emotional penalty is too high. For many individuals, practice in a private environment avoids all risk of humiliation and can bring significant learning rewards.

Interestingly, research finds people respond to their computers as if they were people (Reeves and Nass, 1999). We try to win the favor of our computers and respond to compliments extended to us by software. Quite surprisingly, solo learning activities undertaken with the computer have more characteristics of social learning environments than one would expect. Talented instructional designers build personality into their instructional software to maximize the positive social aspect of the learning environment.

Asynchronous electronic communications, such as e-mail, and synchronous learning events, such as are now possible with various implementations of remote learning, can build a sense of togetherness. Where resources are available to assist learning through such technologies, a stronger social learning environment can be offered, although in this case, just as with other forms of e-learning, design of learning events, not the mere use of technology, determines the level of success achieved.

For many organizations, though, one of the greatest advantages of e-learning comes from its constant availability. When there's a lull in their work, employees can be building skills rather than unproductively waiting for more work assignments or scheduled classes to roll around. Instant availability and the possibility of spending whatever time happens to be available whenever it is available are important benefits of e-learning. Unless you can count on the availability of appropriate colearners at all times, it is probably best to design independent learning opportunities at least for a good share of your learning events.

Don't Baby Your Learners

Some organizations are very concerned about frustrating learners, generating complaints, or simply losing learners because they were too strongly challenged. They are so concerned, in fact, that they make sure learners are unlikely to make mistakes. Learners can cruise through the training, getting frequent rewards for little accomplishment.

The organization expects to get outstanding ratings from learners on surveys at the end of riskless courses, and may indeed get them even if no significant learning occurred (Thalheimer, 2016). But even better ratings may be achieved by actually helping learners improve their skills. More importantly, both individual and organizational success might be achieved. A lot is learned from making mistakes. And people like overcoming challenges! Look at how many people go back to the golf course week after week and spend a fair amount of money for the privilege of overcoming challenges.

Go ahead, break the rules and put learners at some measure of risk. Then provide structures that avoid the potential perils of doing so. Here are some suggestions:

Avoiding Risk Negatives

The advantages of risk are great, but they don't always outweigh the negatives. Fortunately, the negatives can be avoided in almost every instance, so that we're left only with the truly precious positive benefits. Effective techniques include:

• Allowing learners to ask for the correct answer. If learners see they aren't forced into a risk and can back out at any time, learners often warm up to taking chances, especially when they see the advantages of doing so.
• Allowing learners to set the level of challenge. Low levels of challenge become uninteresting with repetition. Given the option, learners will usually choose successively greater challenges.
• Complimenting learners on their attempts. Some encouragement, even when learner attempts are unsuccessful, does a lot to keep learners trying. It is important for learners to know that a few failures here and there are helpful and expected.
• Providing easier challenges after failures. A few successes help ready learners to take on greater challenges.
• Providing multiple levels of assistance. Instead of all or nothing, learners can ask for and receive help ranging from general hints to high-fidelity demonstrations.

Using these techniques, it is possible to offer highly motivating e-learning experiences with almost none of the typical side effects that accompany learning risk in other environments.

Staying Alive

Why do kids (and adults) play video games for hours on end? Shortly after home versions of games like the classic Super Mario Bros. became available, our son and all the children on our block knew every opportunity hidden throughout hundreds of screens under a multiplicity of changing variables. They stayed up too late and would have missed meals, if allowed to, in order to find every kickable brick in walls of thousands of bricks (see Figure 10.8). They learned lightning-fast responses to jump and duck at exactly the right times in hundreds of situations. And for what? Toys? Money? Vacations? Prizes? Nope.

We play these games as skillfully as possible for the reward of being able to continue playing, to see how far we can get, to avoid dying and having to start over, to feel good about ourselves, and to enjoy the energy of a risky, but not harmful, situation.

In the process of starting over (and over, and over), we practice all the skills from the basics through the recently acquired skills, overlearning them in the process and becoming ever more proficient. We take satisfaction in confirming our abilities as our adrenaline rises in anticipation of reencountering the challenges that doomed us last time.

Imagine, in contrast, the typical schoolteacher setting out to teach students to identify which bricks hide award points. There would be charts of the many thousands of bricks in walls of various configurations. You can just imagine the homework assignments requiring students to memorize locations by counting over and down so many bricks and circling the correct ones. Students would be so bored that behavior problems would soon erupt.

The teacher would earnestly remind students that someday in the future, they would be glad they could identify the bricks to kick. Great success would come to those learners who worked diligently to pinpoint all the correct bricks. After several years of pushing groups of students through these exercises, a bored teacher might turn to e-learning for help. With a happy thought of transferring the teaching tedium to the computer, the teacher would likely create an online version of the same grueling instructional activities.

To me, this sounds similar to a lot of corporate training. Although we might not see our trainees pulling one another's hair, making crude noises, and writing on desks, we can be sure they will be checking e-mail, catching up on social media, and even ordering holiday gifts online when learning tasks are so uninteresting.

Just teaching one task within the complex behaviors of the venerable Super Mario Bros. player would be a daunting challenge for many educators; yet, millions of people have become adroit at these tasks with no instruction—at least with no typical instruction. Why aren't these obviously effective instructional techniques applied to e-learning, with all its interactive multimedia capabilities?

Good question. Why not, indeed!

Repetition and Goals

Repetition through practice is a primary way of preventing memory loss (see Figure 10.9). Although strong stimuli, such as traumatic and emotionally charged events, can stick with us without repetition, new information and skills often require rehearsing.

The good news is that e-learning is a perfect medium for practice, having infinite patience and the ability to retrieve practice exercises from a pool if not to actually generate as many exercises as may be needed.

The bad news is that practice is often boring. Think of all the things we cajole our children into practicing as they grow up: alphabet, multiplication tables, notes of the base and treble clef, spelling, etc. We often search for more ways to get kids to practice and have to get creative in our methods. And this is exactly what we must do in e-learning design as well.

This is where goals come in. By establishing performance goals, we draw attention away from repetition and toward results—the progress toward goal achievement. If learners fail over and over again, they will generally want to stop. But by setting increasingly challenging goals, learners will receive interim and practice-sustaining rewards. Visible progress keeps learners trying.

Adding risk into such a framework is rather straightforward and can increase motivation to persevere to a level of mastery. Having incremental goals, perhaps the most obvious means of harnessing risk is to limit the number of attempts permitted at each advanced level before setting the learner back a level.

Let's look at a very successful example.

Learners are put to “work” almost immediately not just to make the learning active but also as a means of determining the right level of challenge and practice needed for each learner (see Figure 10.10).

Adequate but minimal instruction is given so learners can begin showing what they know, can, and can't do almost immediately (see Figures 10.11 and 10.12). The amount of instruction and the level of challenge adapt to the learner throughout.

As learners progress, the variety and number of items needing to be served simultaneously increases. In Figure 10.13, the apprentice cook is facing “Challenge 4” which includes a total of six items, three for each of two plates. All items are cooking and this learner was smart enough to align cooking items to plates to minimize having to spend time rechecking tickets when it becomes time to plate. Customer wait time is shown and keeps the pressure on.

Because each food item requires a different amount of cooking time, all six items will be ready to plate together only if they started cooking in the proper sequence and were timed properly. The cook will then need to get items to the plates quickly enough to prevent those items still cooking from burning and those items plated from getting cold.

There are many ways to fail a challenge, and for most people a considerable amount of practice is necessary to reach and meet the final goal. Feedback (see Figure 10.14) is detailed after each challenge is completed or time runs out.

Resources are available for learners. They inform how to cook each item and how long. Learners can reference the information between challenges and even during them, but if referenced during a challenge, cooking items may burn if in progress while the learner is away. To increase the risk of burned items or unacceptably slow service, when learners return to the kitchen from looking up information, they find a delaying mess of broken eggs to pick up (Figure 10.15). “Oh, no!” Learners are effectively encouraged to internalize critical information rather than looking everything up as they try to cook.

Learners are given only a couple of chances to succeed at each level. The major risk is that multiple failures will produce the dreaded black screen (Figure 10.16) and set the learner back a level where it's necessary to work back up. It's even possible that the learner will regress more than one level and need to climb back up through continuous successes.

This use of risk is amazingly effective in getting learners to practice. Learners witness that just knowing what to do isn't nearly enough to perform successfully and that practice really is necessary. But practice can be fun and very much is in this e-learning. Whether relevant or not to them personally, we've found almost everyone who gives this lesson a try feels compelled to reach and succeed at the top level. But it's unlikely you'll get there without some intensive practice.

What's Boring?

Long presentations of information you already know, find irrelevant, or can't understand.

What's Interesting?

• Learning how your knowledge can be put to new and valuable uses
• Understanding something that has always been puzzling (such as how those magicians switch places with tigers)
• Seeing new possibilities for success
• Discovering talents and capabilities you didn't know you had
• Doing something successfully that you've always failed at before

Moral? It's not enough for learners to believe there is value in prospective instruction (so that they have sufficient initial motivation to get involved). There must also be real value to sustain the required engagement. The content must fit with what they are ready to learn and capable of learning.

Individualization

From some of the earliest work with computers in education, there has been an alluring vision: a vision that someday technology would make practical the adaptation of instruction to the different needs of individual learners. Some learners grasp concepts easily, whereas others need more time, more examples, more analogies, or more counterexamples. Some can concentrate for long periods; others need frequent breaks. Some learn quickly from verbal explanations, others from graphics, and others from hands-on manipulation. The vision saw that, while instructors must generally provide information appropriate to the needs of the average learner, technology-delivered instruction could vary the pacing and selection of content as dictated by the needs of each learner.

Not only could appropriate content be selected for each learner's goals, interests, and abilities, but instruction could also be adapted to interests, cultural backgrounds, and even personal lifestyles.

Perhaps no instructional path would ever be the same for two learners. It would be possible for some learners to take cross-discipline paths, whereas others might pursue a single topic in great depth before turning to another topic. Some learners would need to keep diversions to a minimum as they worked on basic skills; others would benefit from putting basic skills into real-life contexts almost from the beginning. Areas of potential individualization seemed almost endless and possible. And given enough learners across which to amortize the cost of development, the individualization could be quite affordable.

Many classroom instructors do their best to address individual needs, but they know there are severe limits to what they can do specifically for individual learners, especially when there are high levels of variation among learners in a class. The pragmatic issues of determining each learner's needs, constructing an individual road map, and communicating the plan to the learner are insurmountable even without considering the mainstay of the typical instructor's day: presenting information. It's difficult to handle more than a small number of learners in a highly individualized program without the aid of technology. With e-learning, we do have the technology necessary for truly exciting programs that can adapt effectively to the needs of individual learners. Through technology, the delivery and support of a highly individualized process of instruction is quite practical, and there are actually many ways instruction can be fitted to individual needs.

Fortunately, it's not necessary to get esoteric in the approach. The simplest individualized approaches contribute great advantages over one-size-fits-all and often have the greatest return on investment. They may require an expanded up-front investment, of course, as there's hardly anything cheaper than delivering one form of the curriculum in the same manner to many learners at the same time. But there are some practical tactics that result in far more individualized instruction than is often provided.

Let's look at some alternative paradigms to see how instruction has been individualized. Our emphasis here will be on just one parameter—content matching. Efforts to match preferences, psychological profiles, and other learner characteristics are technically possible. But where we haven't begun to match content to individual needs, we'll hardly be ready for the more refined subtleties of matching instructional approaches to learners.

Common Instruction

The most prevalent form of instruction does not attempt to adapt content to learners. Learners show up (Figure 10.17). The presentations and activities commence. After a while, a test is given and grades are issued. Tell, test, grade. Done. Next class, please.

This common form of instruction has been perpetrated on learners almost since the concept of organized group instruction originated. It's as simple and inexpensive as you can get, and it's credible today not because of its effectiveness but because of its ubiquity.

It's unfortunate that so many use common instruction as a model for education and training. This tell-and-test paradigm is truly content-centric. With honorable intentions and dedication, designers frequently put great effort into the organization and presentation of content and sometimes into preparation of test questions, but they do nothing to determine the readiness of learners or the viability of planned presentations for specific individuals. Consequently, although the application easily handles any number of learners and has almost no administrative or management problems, it doesn't motivate the learner.

The effectiveness of common instruction ranges widely, depending on many factors that are usually not considered, such as motivation of learners, their expectations, their listening or reading skills, their study habits, and their content-specific entry skills (Figure 10.18). It is often thought that the testing event and grading provide sufficient motivation, so the issue of motivation is moot. Besides, motivation is really the learners' problem, right? If they don't learn, they'll have to pay the consequences—and hopefully will know better next time.

Wrong thinking, at least in the business world. It's the employer and the organization as a whole who will pay the consequences—consequences that are far greater than most organizations recognize.

Selective Instruction

Selective instruction adds a front end filter to common instruction. The front end is designed to improve effectiveness of the legacy common instruction by narrowing the range of learners to be taught. Clever, huh? Change the target audience rather than offer better instruction.

Most American colleges and universities employ selective instruction at an institutional level; that is, they select and prepare to teach only those learners who meet minimum standards. Students are chosen through use of entry examinations, measures of intellectual abilities, academic accomplishments, and so on. The higher those standards can be, the better the outcomes will be.

It makes sense that learners who have been highly successful in previous common instructional activities will deal effectively again with common instruction's unfortunate limitations. Organizations of higher learning will, therefore, find their instructional tasks simplified. They need to do little in the way of motivating learners and adapting instruction to individual needs. They can just find bright, capable, energetic learners, and then do whatever they want for instruction. The learners will find a way to make the best of it and learn—at least enough to pass a test at the end of the course.

It dismays me that our most prestigious institutions boast of the high entrance examination scores achieved by their entering learners. What they know, but aren't saying, is that they will have few instructional challenges, and their learners will continue to be outstanding performers. By turning away learners with lower scores, they will have far less challenging instructional tasks and will be able, if they wish, to devote more of their time to research and publishing. They actually could, if they wished (and I don't mean to suggest that they do), provide the weakest instructional experience and still anticipate impressive learning outcomes.

We do need organizations that can take our best learners and move them to ever-higher levels of achievement. But perhaps the more significant instructional challenges lie in working with more typical learners who require greater instructional support for learning. Here, common instruction probably has been less effective than desired and should be supplanted by instruction that is based more on our knowledge of human learning than on tradition.

Remedial Instruction

Recognizing that issuing grades is not the purpose of training, some have tried modifying common instruction to help failing learners. In this remedial approach, when testing indicates that a learner has not reached an acceptable level of proficiency, alternate instructional events are provided to remedy the situation.

These added events, it is hoped, will bring failing learners to an acceptable level of performance.

Remediation is something of an advance toward purposeful instruction, because, rather than simply issuing grades, it attempts to help all learners achieve needed performance levels.

If learners fail on retesting, it is possible to continue the remediation, perhaps by providing alternate activities and support, until they achieve the required performance levels.

One has to ask, if the remediation is more likely to produce acceptable performance, why isn't it provided initially? The answer could be that it requires more instructor time or more expensive resources. If many learners can succeed without it, they can move on while special attention is given to the remedial learners.

The unfortunate thing about remedial instruction is that its individualization is based on failure. Until all the telling/testing is done, instructors may not discover that some learners are inadequately prepared for the course, or don't have requisite language skills, or aren't sufficiently motivated. Although more appropriate learning events can be arranged after testing reveals individual problems, precious time may have been wasted. The instructor and many learners (including those for whom the instruction is truly appropriate) may have been grievously frustrated, and remedial learners may be more confused and more resistant to learning in their reluctance to have more experiences of failure.

All in all, remediation represents better recognition of the true goals of instruction, but employs a less-than-optimal process of reaching them.

Individualized Instruction

The same components of telling, testing, and deciding what to do next can be re-sequenced to achieve a remarkably different instructional process. As with so many instructional designs, a seemingly small difference can create a dramatically different experience with consequential outcomes.

A basic individualized system begins with assessment. The intent of the assessment is to determine the readiness of learners for learning with the instructional support available. Learners are not ready if, on the one hand, they already possess the competencies that are the goal of the program, or, on the other hand, they do not meet the necessary entry requirements to understand and work with the instructional program. Unless assumptions can be made accurately, as is rarely the case despite a common willingness to make assumptions regarding learner competencies, it's very important to test for both entry readiness and prior mastery.

Record keeping is essential for individualized programs to work, because progress and needs are frequently reassessed to be sure the program is responding to the learner's evolving competencies. Specific results of each testing event must be kept, not just as an overall performance score but also as a measurement of progress and achievement for every instructional objective each learner is pursuing.

As progress is made and records are updated, the questions of readiness and needs are posed. If there are (or remain) areas of weakness in which the training program can assist, learning activities are selected for the individual learner. Any conceivable type of learning activity that can be offered to the learner can be selected including workbook exercises, group activities, field trips, computer-based training, videos, and so on.

The individualized learning system is greatly empowered if alternate activities are available for each objective or set of objectives. Note that it is not necessary to have separate learning activities available for each objective, although it helps to have some that are limited to single objectives wherever possible. An arrangement suggested by Table 10.5 would be advantageous and perhaps typical. However, great variations are workable; it is often possible to set up an individualized learning program based on a variety of existing support materials.

Looking at Table 10.5, you can see how an individualized learning system works. If a learner's assessment indicated needed instruction on all four objectives, what would you suggest the learner do? You'd probably select resources A and D, because we don't happen to have one resource that covers all four and because together A and D provide complete coverage of the four objectives.

Further, to the learner's benefit, these two resources cover more than one objective each. When resources cover multiple objectives, it is more likely that they will discuss how various concepts relate to one another. Such discussions provide bridges and a supportive context for deeper understanding. Some supportive context, particularly important for a novice, is likely to be there in the selection of A and D in the preceding example, whereas it would be less likely in the combination of C, B, and F.

Just as with remedial instruction, if a prescribed course of study proves unsuccessful, it's better to have an alternative available than to send the learner simply back to repeat an unsuccessful experience. If the learner assigned to A and D were reassessed and still showed insufficient mastery of Objective 3, it would probably be more effective to assign Learning Resource E at this point than to suggest repetition of D.

The closed loop of individualized instruction systems returns learners to assessment following learning activities, where readiness and needs are again determined. Once the learner has mastered all the targeted objectives, the accomplishment is documented, and the learner is ready to move on.

The framework of individualized instruction is very powerful, yet, happily, very simple in concept. Its implementation can be extraordinarily sophisticated, as is the case with some learning management systems (LMSs) or learning content management systems (LCMSs), whether custom-built or off the shelf. However, it can also be built rather simply within a single instructional application and have great success.

Note that this structure does not require all learners to pursue exactly the same set or the same order of objectives. Very different programs of study are easily managed. Learners who require advanced work in some areas or reduced immersion in others are easily accommodated (Figure 10.24).

Practical Solutions

As we have noted, there is a continuum of individualization. Some individualization is more valuable than none, even if the full possibilities of individualized instruction are beyond reach for a particular project. One of the most practical ways of achieving a valuable measure of individualization is simply to reverse the paradigm of tell and test in common instruction.

Fixed Time, Variable Learning

With the almost-never-appropriate tell-and-test approach, exemplified by prevalent instructor-led practices, learners are initially told everything they need to know through classroom presentations, textbooks, videotapes, and other available means. Learners are then tested to see what they have learned (more likely, what they can recall). After the test, if time permits (since instructor-led programs almost always work within a preset, fixed time period, such as a week, a quarter, or a semester), more telling will be followed by more testing until the time is exhausted (Figure 10.26). The program appears well planned if a final test closely follows the final content presentation, just before the time period expires.

When e-learning follows this ancient plan, the plan's archaic weaknesses are preserved.

Put the Test First

Many instructional designers embark on a path that results in tell-and-test applications. They may disguise the underlying approach rather effectively, perhaps by the clever use of techno-media, but the tell-and-test method nevertheless retains all the weaknesses of common instruction and none of the advantages of individualized instruction.

Let's compare the tell-and-test and test-and-tell methods (see Table 10.6). The advantages gained by simply putting the test first and allowing learners to ask for content assistance as they need it are amazing. As previously noted, just a slight alteration of an instructional approach often makes a dramatic difference. This is one of those cases. There is very little expense or effort difference between tell-and-test on the one hand and test-and-tell on the other, but the learning experiences are fundamentally different.

Getting novice designers to break the tendency to begin their applications with a lot of presented information (telling) is not simple. Indeed, almost everyone has the tendency to launch into content presentation as the natural, appropriate, and most essential thing to do. I have been frustrated over the years as my learners and employees, especially novices to instructional design, have found themselves drawn almost magnetically to this fundamental error. I have, however, discovered a practical remedy: After designers complete their first prototype, I simply ask them to switch it around in the next iteration. This makes content presentations available on demand and subject to learner needs.

This example, drawn from e-learning initiatives at the National Food Services Management Institute, illustrates the power of the test-and-tell technique to select the right content for each learner, and avoids instruction on principles some learners may already know. Cooking with Flair: Preparing Fruits, Salads, and Vegetables teaches a range of facts and procedural information regarding the preparation of fruits and vegetables. The target learners usually have some general but incomplete knowledge of the topics to be covered, and the gaps in their knowledge can vary greatly. A typical strategy in such a context is to tell everybody everything, regardless of whether or not they need it. This is rarely helpful, as learners tend to stop paying attention when forced to read information that they already know or, on the other end of the scale, that they can't understand.

This is exactly the type of situation in which test-and-tell is most powerful.

We'll look at a module called “Do the Dip!,” in which learners are to master how to keep freshly cut fruit from turning brown (Figure 10.28). The principle at hand: fruits that tend to darken should be dipped in a dilute solution of citrus juice, while other fruits do not need to be treated in this way. The learner is to learn how this principle affects common fruits. The setup here is very simple. Learners are given a very brief statement that sets the context. Then, they are immediately tasked with the job of preparing a fruit tray. They can place the cut fruit directly on the serving tray, or they can first dip the fruit in a dilute solution of lemon juice.

If the user correctly dips a fruit, the information regarding that choice is confirmed with a message and a Do the Dip! flag on the fruit (Figure 10.29). Whether the learner already knew the concept or guessed correctly, the “tell” part of the instruction is contained in the feedback—presented after the learner's attention has been focused by the challenge.

How about when the learner is wrong? When a fruit is not dipped when it should be, the fruit turns brown and unappealing, while instructional text specific to the error is presented on the screen (Figure 10.30).

After the exercise is complete, the learner is given a summary screen: a complete and attractive fruit platter and screen elements that further reinforce the lesson of which fruits require a citrus treatment (Figure 10.31).

This is a small and straightforward application of the test-and-tell principle, used to select what information to present to the learner and when simply to confirm the learner's present knowledge and skills. You should notice this magic key in action in more sophisticated ways in nearly every other example described in this book. It's engaging, efficient, and effective.

Nonetheless, a common response from instructional designers is, “Isn't it unfair to ask learners to do a task for which you haven't prepared them?” (Interestingly, we rarely if ever hear this complaint from actual students.) But it really isn't unfair at all. Quite the reverse: The challenge focuses learning attention to the task at hand, allows capable learners to succeed quickly (saving time and frustration), provides learning content to the right people at the right time, and motivates all learners to engage in critical thinking about a task instead of simply being passive recipients of training. When the “tell” information is presented in the context of a learner action (i.e., right after a mistake), learners are in an optimal position to assimilate the new information meaningfully into their existing understanding of the topic.

The Typing Ball Syndrome

When I did my first major project with computer-based instruction, it was with computer terminals connected to a remote IBM 360 central computer. The terminals were IBM 3270s, which had no display screen but were more like typewriters, employing an IBM Selectric typewriter ball.

Continuous sheets of fanfold paper were threaded into the machine. A ball spun and tilted almost magically to align each character to be typed. With thin, flat wires manipulating the ball's mechanism, the ball, like a mesmerized marionette, hammered the typewriter ribbon to produce crisp, perfectly aligned text.

Selectric typewriters were something of an engineering fascination.

They so quickly produced a clean, sharp character in response to a tap on the keyboard, it was hard to see the movement. And yet, since a perfect character appeared on the paper, you knew the ball had spun to precisely the correct place and had tapped against the ribbon with exactly the right force.

If the Selectric typewriter was fascinating, the IBM 3270 terminal was enthralling. Its Selectric ball typed by itself like a player piano! Looking very much like the Selectric typewriter with a floor stand, the terminal was able to receive output from a remote computer and could type faster than any human. The keys on the keyboard didn't move. The ball just did its dance while letters appeared on the paper.

The computer could type out information and learners could type their answers, to be evaluated by the computer. The technology was fascinating. There was much talk about this being the future of education.

It felt like classroom delivery of instruction was near the end of its life span, and, despite fears of lost teaching jobs, there was great excitement that private, personalized learning supported by amazingly intelligent computers would truly make learning fun and easy.

Novelty versus Reality

There were some very positive signs that technology-led instruction would indeed be a reality in the near future. Learners eagerly signed up for courses taught with the use of computer terminals. Seats filled up rapidly, requiring dawdling learners to sign up for traditionally taught classes or wait another term to see if they would be lucky enough to get into one of the computer-assisted classes.

But some disturbing realities crept in, realities that countered initial perceptions and rosy predictions. For one thing, it took a lot longer for information to be typed out than to find it in a book. While you didn't get to watch them being typed, books are pretty fast at presenting information—and they are portable, too. They don't make any noise, either—so they can be used in libraries and other quiet places that are conducive to study. They often include graphics and visual aids—things our Selectric typewriters couldn't provide. You can earmark a page and easily review. You can skim ahead to get some orientation and get a preview of what was to come.

Learners, the harsh but insightful critics that they often are, did the most unexpected thing. Instead of sitting at the terminals waiting for the typing ball to spit out text, they simply took their “borrowed” copies of printouts to a comfortable location and studied them. Sometimes they'd work together in pairs or groups: Using just the printout, one learner would read the computer's questions, and another learner would answer them. The first learner would look up and read the feedback for that particular answer, and together, they would have a pleasant, effective learning experience. They were executing, in effect, the programmed instructional logic. It was much better than sitting at a typing machine.

The typing ball was not essential to learning in any way. Learners quickly dispensed with it and created a more effective context in which they could freely discuss the content, not only in a more convenient manner, but in a more meaningful manner, as well. In fact, the typing ball context was not really an instructional context at all; it was simply a new technology for instructional delivery. This seems like an obvious confusion to us today, but this mistake has been made time and again as new technologies have arrived to save the future of education. It is happening now, with the Internet and mobile devices.

Learning Sequences and Learning Contexts

Breaking complex tasks apart and teaching components separately can be an effective instructional practice. Part-task training is effective and efficient because it allows learners to focus on component behaviors and master them before having to deal with the interrelationships among sequences of behaviors to be performed.

Don't Start at the Bottom of the Skills Hierarchy

A valued tool of instructional designers is the skills hierarchy. Desired skills are broken down into their constituent parts, and then ordered in a prerequisite sequence or hierarchy. The target skill resides at the top, while the most basic, elementary skills sit at the bottom, with intermediary skills building the hierarchical tree (Figure 10.33).

If the hierarchy is accurately constructed, tasks in the tree cannot be learned or performed before the skills below them have been learned. The hierarchy, therefore, prescribes a useful sequence for teaching. You start at the bottom, where all learners can be assumed to be able to learn the skills. Once these prerequisite skills have been mastered, you can move up a level.

Break the rules! Instead of beginning with the tedious task of explaining a flush, royal flush, pairs, and so on to teach poker, deal the cards and play the game. Maybe you have all participants organize their cards and lay them face up. An experienced poker aficionado might announce who has the best hand and ask the learners to figure out why it's so strong. Of course, the best way to put the learners at risk is to give them a pile of chips and have them begin betting. Once students are in danger of losing chips (or gaining the whole multicolored pile), they ask good questions, learn winning combinations faster, and pay greater attention to explanations. There's no better way to learn about bluffing than to lose your chips to someone who wins the hand with two jacks!

Boring!

Now, think about this:

A Great Learning Journey Starts…in the Middle?

Good e-learning designs do a lot to keep activities relevant and interesting. They don't work on the all-too-common instructor admonishment, “Trust me. Someday you'll be glad you know this.”

How do good designs avoid the problem of uninteresting, seemingly irrelevant starter content? They start somewhere in the middle of the skills hierarchy, if not at the top, and let learners participate in leading the charge to fill in the missing pieces as necessary.

As you look at a skills hierarchy (and, by the way, you should construct one just for the experience), scan up from the bottom in search of a point where a collection of prerequisites comes together in support of a skill or concept. Look at that skill or concept and see if it is a useful one. Points of convergence on a skills hierarchy often identify a capability that new learners can see as meaningful and desirable.

Novelty Can Be a Valuable Tool

We earlier debunked technological novelty as an instructional context; however, novelty does have value. Novelty is powerful for only a short time, but it can be useful when applied thoughtfully and purposefully. Although it is not a substitute for effective learning interactions, novelty does have two important uses in effective e-learning:

• Novelty draws attention and energizes curiosity.
• Novelty can help make an experience memorable.

Drawing Attention

As previously discussed, the most important thing designers must accomplish is motivating learners to learn. This is done by connecting with learners' needs and creating interest. And this means first getting their attention and helping them build positive expectations, curiosity, and a willingness to try (Figure 10.34).

Because novelty loses its value quickly, its use is well suited to attracting attention. It increases learner receptivity to messages that market the value of learning the content and skills at hand.

Memorable Experiences

An unexpected quip, a fascinating morph, an amusing sound, or a simulated explosion could provide an element of appeal or surprise that learners will remember for some time. It's important, of course, that these elements be closely tied to important learning points. We don't want to end up with the effect of a clever TV commercial that viewers can remember in great detail, except for one minor point—the product or company being promoted.

As we know, to be effective, e-learning experiences must be both meaningful and memorable. A novel context can help tremendously to attract attention, giving us the opportunity to market the value of learning the content. Through demonstrations, testimonials, or “give this a try” experiences, novelty can turn the attention it won into perceived value. But we have to get off the novelty as soon as possible. Amusement is not the goal. It is a tool and means to better learning.

Just one more step: how do we turn motivation into learning and realized value?

The power of a story to provide a compelling context is shown in this example, which immerses learners with this scenario:

Ralph Retailer quickly realizes how difficult it will be to order handbags from Artie Guy to make sure he always has enough handbags at each of the stores to support customer demand. He also needs to consider that he shouldn't have too many handbags, which would make things confusing and possibly require him to move to a larger warehouse. He asks his very intelligent son, Ronnie, to take over the job of ordering handbags. He provides him with two goals: satisfy at least 98 percent of customer demand, but maintain no more than a 10-day supply of handbags in either the warehouse or any one of the stores.

KTCA-TV and 3M Company, Inc., in association with ICONOS, Inc., created a series of classic e-learning experiences to accompany the public television program Newton's Apple. The intent was to provide highly interactive and educational experiences as companions to the popular television series. The “Why Does Glue Stick?” segment is a masterful example of how a creative context—in this case novelty, sequencing, and skills transfer—can impact motivation.

The novelty in the challenge is apparent immediately (Figure 10.41). The learner needs to build a bridge to help an elephant cross a chasm. The tricky part is that the bridge must be built by gluing together such diverse objects as a circular saw blade, a birdhouse, and an aquarium. Luckily, there are five adhesives available for use in solving this puzzle.

Again, using one of my favorite techniques, the learner is thrown into the middle of the content. You will find no boring discussions of glue characteristics here. While the context is unusual, it shines the spotlight fully on the content at hand: Which among various adhesives work with which surfaces?

To build the bridge, the learner chooses an object to glue to an existing object and an adhesive to use. If it is a successful combination, the elephant takes one step closer to its destination (Figure 10.42). If the adhesive does not hold, the learner gets a surprising and humorous result (Figure 10.43).

Now, critics might suggest this is just unfair to the learner. How is the learner to know how to combine these unusual items into a bridge?

If this were all there were to the piece, the criticism might be justified. As it turns out, the technical information necessary to make informed decisions is easily available to the learner. Clicking the magnifying glass initiates an investigation mode, through which the learner can read about the adhesives and the surfaces, even to the detail of seeing electron microscope images (Figure 10.44).

Learners can research adhesives and make informed decisions, or they can experiment with materials and adhesives to see what will work. Ultimately, a successful effort is rewarded: A whole herd of elephants charges across the learner's bridge (Figure 10.45).

It should be obvious how the novel context empowers learning in this example. But skills transfer was also addressed in the design.

How can that be? It hardly seems possible to imagine a less useful task than building a bridge out of a preposterous combination of items so that elephants can cross a gap.

Actually, it's the preposterousness of the setting that enhances the transferability of this content. Imagine if the task were something more like gluing a metal number to a door. Because the task is so reasonable, it is easy to focus too tightly on the details of the metal number and the door, which are really irrelevant to the higher-level concept. Because it is nonsensical even to attempt to build a bridge from an aquarium and a saw blade, the learner naturally generalizes the learning to the underlying, more important concept of surface types. The learner is figuring out in general how one might bond paper to metal or glass to wood. And that is really the point of the exercise.

Carefully selecting the right novel context made this great learning experience possible. Brilliant.

Authentic Tasks

Much has been made of standardized question forms, such as multiple choice, true/false, matching, and fill-in-the-blank structures. While they have their uses, they do not create engaging learning activities. Because they look at isolated points of knowledge, they frequently provide little of the critical performance context.

Learning tasks must be authentic—they must relate directly to the effective performance of tasks on the job. Authentic tasks are far more appealing than almost any rhetorical or academic task. They heighten our propensity to get involved. And that's what we are after: involvement, engagement, and learning—the kind of learning that leads to performance success.

Don't Tell Learners If They Are Right or Wrong

At least not too quickly. First, let learners see for themselves whether or not their performance works as well as it needs to. Some aspects of a simulation are needed for this, although not necessarily a complex or even visual one. For example, the familiar story problem, often used in mathematics, provides the needed improvement over bare arithmetic problems. Intrinsic feedback responds within context, such as reporting where the train stopped, whether the apples overflowed the pie pan, or if the corporation was fined for tax underpayment.

The feedback can be entirely visual to demonstrate outcomes of student performance. This is often best, but you need to be sure learners fully grasp the quality of their performance and in what way it needs to be improved. In fact, it may be wise to ask learners to rate their own performance and to state their own plans for improvement. But don't give feedback yet, even if you know their plans will fail. Let learners try their plans and see the results. Provide hints if their next plans repeat previous errors.

Intrinsic feedback in e-learning typically causes a positive domino effect. Consider these likelihoods:

• The design will probably speak to learning outcomes the learner values.
• The context will relate strongly to actual work or performance conditions.
• Tasks are likely to be multistepped.

It's actually possible that all seven of the Magic Keys to motivating e-learning will be incorporated as a result of working toward intrinsic feedback. If you specify intrinsic feedback as a design requirement, you're likely to build very effective e-learning—and it won't be boring.

EXAMPLES

Intrinsic feedback is featured in nearly all of the examples included in this book because I feel it's fundamental to good design. Intrinsic feedback couples very nicely with delayed judgment, the next Magic Key, so review all examples with an eye for intrinsic feedback.

A-ha!

When I ask instructional designers what constitutes good instructional design, I receive a wide range of answers. One frequent and fascinating response is that good instructional applications invoke a steady stream of “A-ha!” experiences. Everyone agrees. Epiphany is a wondrous thing!

Great satisfaction comes from meeting a challenge successfully, even if the competency demonstrated is relatively useless. When the competency really means something, when it actually empowers us to do things we care about, the success—the “A-ha!”—can be prodigious. It is, in fact, this quality that we seek and should perhaps expect from a technology that has so few boundaries.

Too bad we so often get “Huh?” instead of “A-ha!

Valuable U-Turns

If an assessment is given following each response, it's harder for learners to focus on the bigger context; they tend to focus only on the singular response at hand. There are occasions when what we are trying to teach is an isolated, single response, but most education and training challenges are far more complex. For these types of learning, delayed judgment is appropriate.

Suppose learners were working on the following subtraction problem.

It wouldn't be surprising if a learner began by subtracting 8 from 20, getting 2:

Buzz! You could make the e-learning application respond instantly, indicating that an error was made. You could tell the learner to reverse the numbers—to subtract 20 from 108, and then use the sign of the larger number. But what if you waited? The learner might proceed with:

And then realize that the approach wasn't working. The number 12 really doesn't describe any relationship between ±108 and 20. If the learner held the bigger concept of negative numbers and positive numbers lying on a continuum:

and therefore understood that the positive number (20) would push the negative number (108) back 20 notches toward 0, the learner might, on his or her own, decide to try an alternative approach, quite possibly constructing the correct one along the way:

Of course, there are other things learners might do as well, such as getting +88. They might also get −128 or +128. Having time to evaluate is precious, important learning time. Our software should not interrupt and intrude into it. But it can and should silently monitor learners' various attempts, respond to questions, and provide reminders when asked, otherwise staying silent until the learner concludes he or she has reached the correct answer. A wonderful opportunity exists for an “A-ha!” realization at this point, and learners will need only the slightest confirming feedback to feel rewarded.

Immediate feedback deprives learners of opportunities to make U-turns—that is, opportunities to correct themselves, to go back a few steps, or to start over to see if they are right or what might be going wrong. By evaluating a specific response within a meaningful context, learners begin to take responsibility for evaluating their behaviors. Just as important as having the basic skills is the ability to assess one's own work properly and habitually. It contributes greatly to success in real-world performance. Providing opportunities to build and rehearse these work habits, supported by e-learning software, provides a vastly more interesting, valuable, and effective learning experience. It's not boring, either.

It's important to remember that the appreciation and understanding of context may well be more important than any one skill and that developing each related skill will be much easier if the learner has a firm grasp of the big picture—of the context. Delayed judgment is an extremely helpful technique for keeping a proper orientation and avoiding the acquisition of detached skills and facts that tend to have value only for answering multiple-choice questions. Interestingly, e-learning is uniquely capable of giving both instantaneous judgment and delayed judgment. We need to use these capabilities wisely.

Why, You Ask?

We can't see what learners are thinking, and neither can e-learning software, unless learners communicate each step as they work toward an answer. Although one can consider requiring learners to declare each intermediate step, it sometimes makes more sense to ask afterward. In doing so, we might actually facilitate learning epiphanies.

The technique is reasonably simple. After learners have responded (answered a question, solved a problem, made a decision, etc.), you ask them, “Why is this the right thing to do?” or “Why is this the correct answer?” (See Figure 10.53.)

There are then four possibilities—two when learners give the correct answer and two when they don't.

Even when giving the correct answer, learners may or may not be able to justify it—such as in the case of a lucky guess.

With an incorrect answer, learners may cite valid attributes about the answer given, but these attributes would not relate sufficiently to the problem and therefore wouldn't justify the answer. Alternatively, learners may give points that would actually justify another answer, perhaps even the correct answer, but not the one given.

Delayed Judgment

Critical to the full effectiveness of this approach is the delay of judgment. Feedback is not given until learners have submitted justification for their answers (i.e., learners are not told whether they are correct immediately after they answer). This delay is crucial, because while learners are working on justifying their answers, there is a high probability they will realize they have given the wrong answers. We therefore allow learners to back up and change their answers and to do so as many times as they desire until they have both answered correctly and justified their answers.

It's very rewarding for instructional designers to observe learners working through an implementation of this paradigm. Many learners cannot help muttering out loud as they realize they cannot justify their answers and that the answers they gave were wrong. Learners may feel privileged by the ability to correct themselves. Combined with multistep tasks and opportunities to back up, delayed judgment can provide fascinating and effective learning experiences.

Many permutations of this design are possible. The approach can be articulated into sophisticated paradigms, including those effectively incorporating artificial intelligence. But the use of this design is quite practical once the basics have been implemented. The structure can be used over and over again with different content for cost-effective, rapidly implemented learning solutions.

In this solution, delayed judgment is essential since in real sales situations there may be many paths to a sale, though some are better than others. The required skill is much more than just knowing what to say or when to ask for the deal; it requires the sales person to assess the situation and react to questions, objections, and even responses to previous decisions that may have been less than optimal. Once the learner selects the “Ask for the Sale” button, he or she is presented with the outcome of his or her sales approach. The outcome could be customers walking away for good, wanting to take more time, making a counteroffer, asking more questions, or going ahead with the deal.