3. Dramatic Foundations: Part II: Orchestrating Action

What is possible in a given representational “world”? In drama—on the stage, in film, or even on television—discovering what is possible is a twofold source of pleasure for audiences. First is the stimulation of imagination and emotion that is created by carefully crafted uncertainty. Second is the satisfaction provided by closure when the action is complete, if the plot has been successfully constructed. When representational “worlds” are interactive, whether they be avant-garde theatre productions or virtual offices, how people find the edges of the universe—discovering the limits of what is possible—is a central issue in design. This chapter deals with how plots—representational actions—are constructed so that they provide emotional and intellectual satisfaction and how these dramatic principles can inform the design of human-computer interaction.

One answer is, to misquote the famous turtle, “it’s actions all the way up”—that is, several whole actions are being braided into an even larger one, which is itself a whole, with all the associated formal and structural characteristics. The upper limit of this recursion is supplied, in part, by the notion of magnitude (something of a size that can be perceived as a whole) and in part by the context(s) of activity. While working on this book, for instance, all of the actions I undertake (and all of the applications I use) during a session with the computer are typically related to the activity of authoring the book. To the extent that the operating environment supplies a consistent context (its “interface”), consistent “tools” (like cut and paste), and some transportability (e.g., the ability to bring a Photoshop image into a Microsoft Word file), the system reinforces this sense of wholeness.

Contrariwise, I may simply get up in the morning, boot up the computer, and diddle around with various tasks: e-mail correspondence, journal entries, designing party invitations, or what have you. The artificial bracketing events of turning the computer on and off are not equivalent to the beginning and end of a whole action; rather, there are several “whole actions” being pursued concurrently. The possibility of multiple “whole actions” being undertaken in a multitasking fashion is not unique to computing; the same phenomenon occurs in the typical day of any worker, artist, or homemaker, and it is quite familiar to the sort of reader who has several books going at once, reading science fiction in bed and journal articles in the bathroom. The point here is not to assert that there is necessarily a single “whole action” being constructed every time that a person uses a computer, but rather to suggest that the quality of wholeness has contextual, structural, and formal characteristics.

The multitasking “user” may not experience whole actions. This may be due to the intent of the “user”; that is, whether or not the actions being performed in various applications or environments are related in some way to a common intent. Within a particular application, especially in games, the player may not experience a whole action when there are parallel plots or levels unless connections are designed into the game. Why do we experience frustration when we watch a film or TV show with parallel plots that do not converge or at least have some relation to each other? We expect a whole action. Having two separate actions (plots) intercut does not satisfy. We seek wholeness in dramatic experience. To graduate from one “level” of a game into another with different affordances and goals and without obvious connection to the previous levels does not satisfy. Likewise, action games that can never be “won” may leave us lacking the satisfaction of a whole experience with beginning, middle, and end. In an unpublished letter to Alan Kay at Atari Labs, science fiction author Harlan Ellison observed that it is not possible to meet that goal in many games if the bad guys just keep getting better—an affliction shared by many video games. “. . . the lesson,” moans Ellison, “is the lesson of Sisyphus. You cannot win. You can only waste your life struggling and struggling, getting as good as you can be, with no hope of triumph.”

We can look to characteristics of good dramatic structure to inform us in designing the potential for whole actions in interactive media.

The action of a play consists of a series of incidents that are causally related to one another. Those incidents are specified in the script and enacted by actors in performance. In the previous chapter, we likened a computer program to the script of a play, with one important difference; whereas the action specified in a given script will not change from performance to performance,¹ a computer application can lead to actions (composed of incidents) that can vary widely from session to session, depending upon the choices made and actions performed by human agents. In other words, programs generally contain more potential for action than plays. To understand the implications of this fact, we need to explore the nature of dramatic potential and how it is formulated into action.

1. Of course, the qualities of the performances of the actors may vary, but not the action itself. There are exceptions, such as the interactive plays mentioned in the previous chapter.

Potential is defined as something that can develop or become “actual.”² Dramatic potential refers to the set of actions that might occur in the course of a play, as seen from the perspective of any given point in time (that is, a location along the axis of time, as the action of the play unfolds). At the beginning of a play, that set is very large; in fact, virtually anything can happen. From the instant that the first ray of light falls on the set, even perhaps before an actor has entered the scene or spoken a single word, the set of potential actions begins to narrow. What could happen begins to be constrained by what actually does happen; the lights reveal a room in a Victorian house or a fantastic heath, for example, and a banker or a faerie walks onto the stage. The actions of the characters form incidents—coherent units of action—that further begin to constrain what may follow. As incident follows upon incident, and patterns of cause and effect begin to be perceived, rough notions of the shape of the whole action begin to emerge; that is, people in the audience begin to have expectations about what is to come in terms of the overall plot. Where is the play going, and what is it essentially “about”?

2. For a deliciously different take on this statement, see the book Make It So by Nathan Shedroff and Chris Noessel (2012). They demonstrate how many interactive devices, forms, and affordances have been presaged—or even invented—in the media of science fiction film and television.

In Aristotelean terms, the potential of a play, as it progresses over time, is formulated by the playwright into a set of possibilities. The number of new possibilities introduced falls off radically as the play progresses. Every moment of the enactment affects those possibilities, eliminating some and making some more probable than others. When we learn, for instance, that Hamlet’s father was murdered, it becomes probable that Hamlet will try to discover the identity of the murderer. Later in the play, it becomes probable that, once he has found the villain out, Hamlet will seek revenge. But will he succeed? At each stage of the plot, the audience can perceive more than one line of probability (that is, more than one probable course of events), creating engagement and varying degrees of suspense in the audience. At the climax of a play, all of the competing lines of probability are eliminated except one, and that one is the final outcome. At the climactic moment of Hamlet, the only remaining probability is that he will die, and Fortinbras will restore order to the kingdom. In this moment—the moment when probability becomes necessity—the whole action of the play is complete. Thus, over time, dramatic potential is formulated into possibility, probability, and necessity.³

3. In the context of drama and as used in this book, the terms possibility, probability, and necessity have specific meanings that differ substantially from mathematical or scientific usage. Readers who wish to investigate the dramatic connotations further should review the Poetics, 1451a–b.

This process can be visualized (highly schematically) as the “flying wedge” in Figure 3.1. How this pattern is accomplished in a play depends, in the main, upon the playwright’s selection and arrangement of incidents and how they are causally linked. Reading the diagram from left to right shows the progression of material causality, by the way, and reading it from right to left shows formal causality at work, where the necessary end of a whole action functions as a kind of magnet, drawing the structure of the action toward itself.

The shape of potential over time in human-computer interaction is similar to the “flying wedge.” In a play, the result of this successive formulation is a completed plot—a whole action. What is the human-computer equivalent? As we noted previously, a “whole” human-computer interaction can be described, using the broad definition of a whole action, as having a beginning, middle, and end and being composed of incidents (one or more) that are parts of that whole. Thus, playing a computer game until it ends (or I end it) or a “session” with an ongoing computer game can be a whole action, and a “session” with my word processor can be a whole action (even if I don’t finish the chapter I’m writing).

With adequate magnitude along the temporal axis, human-computer activities can be seen to formulate potential in the same way that drama does—as a progression from possibility to probability to necessity. The opening display (which may or may not be multisensory) begins the process of delimiting potential. Every action taken by an agent, including both human and computer-based agent(s), creates further possibilities and constraints as the activity takes shape (see Figure 3.2). Thinking about things this way helps us to focus on how incidents can be arranged and causally linked. A human-computer activity, unlike a play, may be formulated uniquely every time it is performed. The source of variability is people, through their choices and actions, which in turn reflect different goals, styles, and capabilities. Another source may be elements like learning or randomness that are built into an activity at the level of processing.

Many of the aspects of a play’s enactment are the result of the rehearsal process, in which the director (and actors) determine where and when to move and what sorts of lighting and other technical effects should be produced. If these inventions were happening in real time rather than in the rehearsal process, plays could be seen as being far more “dynamic” in terms of the actors’ relationship to the script. The displacement is temporal, but so are the constraints. What actors and directors typically cannot do is to change the order of events or the words spoken by the characters, either in rehearsal or performance, nor can they invent new ones. A program that reformulates the potential for action, creating new possibilities and probabilities “on the fly” as a response to what has gone before, is equivalent to a playwright changing a plot in real time as a collaboration with the actors and director and communicating new portions of script to them in real time through some automagical means. In other words, the way in which human-computer interaction is more dynamic than drama is in the aspect of formulating the action, rather than in its enactment.

In some ways, human-computer interaction is quite similar to the art of improvisational theatre. Improvisational actors have freedom to introduce anything they like, but they are judged on the grace and cleverness of their choices, not simply on their novelty. People can theoretically introduce anything they like into the potential of a given human-computer activity. Introducing new potential, especially “late in the game,” has the capacity to explode the structure of the action. How can people be constrained to work only with potential that is inherent in (or amenable to) that which is already in the representational world? The problem of constraints is treated later in this book, but a key element in its solution is the deployment of dramatic probability and causality to influence (indirectly constrain) what people think of doing.

4. The notion of causality contains some cultural bias; that is, the notion of cause and effect is not so universal as Aristotle believed. Some cultures substitute temporal relations for causal ones, for instance. Likewise, many avant-garde playwrights of the twentieth century, especially the absurdists and surrealists, attempted to eliminate causality from dramatic structure. In the main, however, the notion of causality is pervasive and robust enough to justify our use of it as the basis of our theory. Of course, other theories have been formulated from the alternative views of other cultures and philosophies.

Gratuitous incidents have no direct bearing on the plot; for example, there is no reason to include a scene in which Hamlet brushes his teeth. Most of us have been annoyed by gratuitous incidents in films and TV shows, and many of us have been annoyed by the same kinds of incidents in human-computer interactions. A convention in the world of computer-aided instruction (CAI), for instance, was to ask the student to enter his or her name at the beginning of an interactive session. The most that was usually made of this incident was a message that replied, “Hello there, Jimmy,” before proceeding with the “meat” of the lesson. The name-entering incident (Jimmy and the computer saying “hi”) had nothing to do with the plot (Jimmy learns his multiplication tables). I seriously hope that this kind of educational software isn’t in use any more. Most of the time, eliciting the entry of “user” information is gratuitous for the “user,” but not for the company who owns the application.⁵

5. Try to order something online from a source you’ve never used before and escape getting email spam from them. Not to mention the Big Data problem.

This design strategy was often seen in early drill-and-practice educational programs of the sort that caused Tom Malone (1981) to write his canonical paper on intrinsic motivation. If Jimmy solves three arithmetic problems successfully, he gets to spend twenty seconds playing a starship-blaster action game. Either the math or the game segments are gratuitous, depending upon Jimmy’s understanding of the central action. The solution is either to eliminate one of the activities, or to reshape the action so that it includes both in a causally related way—e.g., a starfighter simulation in which Jimmy solves math problems in order to operate the ship.⁶

6. These three paragraphs are a slightly modified excerpt from “Interface as Mimesis,” in Norman, D. A. and Draper, S., Eds., User Centered System Design: New Perspectives on Human-Computer Interaction. Hillsdale, NJ: Lawrence Erlbaum Associates, 1986. Reprinted with permission.

Besides its function as a criterion for inclusion, playwrights deploy causality in the shaping of dramatic probability. The representation of certain causes makes certain effects probable. The possibility of conflict in the Neutral Zone (part of the dramatic potential of Star Trek) becomes a probability if a cause is represented—e.g., a Romulan incursion. In complementary fashion, the representation of effects leads people to expect that causes will be revealed—another way of constraining what is probable in the action.

A primary source of causality in dramatic incidents is the goals of the characters, that is, what the characters want and what they are trying to do (present on the level of Thought). The central action of a play is often best described in terms of the goal of its central character. The character tries various courses of action for achieving their goals. The obstacles and conflicts they encounter force changes in their behaviors and plans, and sometimes in the goals themselves. A detective character may start out trying to solve a murder and end up embroiled in an international espionage operation. Of course, by the end of the action, the audience can see that it was “about” the spy ring all along, because knowing about it makes all the details fall into place. The central character’s goal has carried them along, and the revelation of the other characters’ goals unifies seemingly unrelated incidents into a whole action through the interweaving of causality.⁷

7. One of the very best examples of the recognition of causality after the action is completed is the American film The Sixth Sense (1999). No spoilers here: If you haven’t seen it, watch it!

Likewise, the agents’ goals are most often the strongest source of causality in human-computer activity. What is each agent (human and computer-based) trying to do, get, or become? What obstacles and conflicts arise, and how do they constrain what the agents do? In human-computer activity, as in drama, goals usually lead to the formulation of plans (or strategies) for achieving them. These plans are either stated or inferred, and they provide a basis for understanding the action. The implementation, failure, revision, and formulation of plans are the “meat” of the action. To be probable, goals and plans must be plausible in terms of the characters that generate them (the “appropriateness” criterion for character, as discussed in Chapter 2).

In his dissertation, “The Dynamic Structure of Everyday Life,” (1988) AI researcher Philip Agre argues that real people do not live their lives this way; that is, goals and plans do not explain most of human behavior. His observations lead him to posit that people are primarily involved in improvising what to do next, in a moment-by-moment way, and that everyday life is “always almost wholly routine.” But everyday life is different from drama. And highly goal-oriented “real” behavior, as in the case of constructing a building or some other specific task (the kind of thing we often do with computers), can be seen to involve a greater proportion of planning activity than “everyday life” as well. Agre’s understanding of everyday activity has enabled him to arrive at AI architectures that may do a remarkable job of emulating real life, and his ideas may lead to an entirely new paradigm for representing and orchestrating human-computer interaction.

Nevertheless, I employ the notions of goals and plans in this book for several reasons. One is the desire to see human-computer interactions as “wholes” with coherent structures. Constructing them as dramatic wholes allows us to take advantage of deeply ingrained conventions about understanding representations of action. These conventions are in fact the ways in which drama is not like life: elimination of the extraneous and gratuitous, clear causal relations among things that happen, and the notions of beginnings, middles, and ends. Agre wanted artificial reality to be lifelike, but there are good reasons why, at least in some situations and for some purposes, artificial reality should be—well, artificial.

Related to Agre’s thesis is the work of Lucy Suchman. In her excellent book Plans and Situated Actions: The Problem of Human-Machine Communication (1987), Suchman contends that “purposeful” (or goal-directed) behavior is best understood, not as the execution of plans, but rather as situated actions: “actions taken in the context of particular, concrete circumstances.” Plans are fundamentally ineffective because “the circumstances of our actions are never fully anticipated and are continuously changing around us.” Suchman’s observations lead her to conclude that plans are best viewed as “a weak resource for what is primarily ad hoc activity.” Suchman does not deny the existence or use of plans, but implies that deciding what to do next in the pursuit of some goal is a far more dynamic and context-dependent activity than the traditional notion of planning might suggest. A dramatic view of human-computer interaction is amenable to the notion of situated actions in that it attempts to dynamically represent changing situational elements and to incorporate knowledge of them into both the decision-making processes of computer-based agents and the understanding of the actions of human agents in representational contexts.

In keeping with Suchman’s analysis is the fact that many factors contribute to dramatic causality by dynamically influencing agents’ choices and actions. Among them are natural forces, coincidences, situations, and conditions. Of course, “natural” forces represented in plays and imaginary worlds may be very different from those at work in the real world. Computer games select and modify the laws of physics, for instance. In computer-based simulations, scientific developments such as fractal geometry and mathematical representations of chaos theory make it possible to emulate the natural world with much greater detail and accuracy than formerly possible, but even these techniques must be deployed selectively in the process of representation-building; attempting to render the physical world (or a comparably robust alternative) completely would currently bring the world’s most powerful computers (and programmers) to their (virtual) knees. Even when selectivity is not an artistic choice, it is nevertheless a necessity in computer-based modeling of physical worlds. The important thing is to know that one is in fact exercising selectivity—to be explicit about it, and to employ a notion of the potential for action in the world one is creating as the primary selection criteria. Representing a natural force makes certain kinds of actions more probable; for instance, simulating air flow around an aircraft wing in a CAD program suggests that changes in the wing will create changes in the air flow, implying both causality and potential action. If the potential for adjusting the wing in some way is successfully represented, then the possibility of adjustment becomes more probable. Turbulence remains a chaotic problem.

Representations of functionality that do not model the physical world still employ equivalents of natural laws in the ways that things behave. Windows open and close with animated embellishments that suggest real-world physical actions; folders appear to exert a gravitational force within a limited area that sucks documents into them (when the representation of such a force is flawed, the comparison with black holes may be unintentionally evoked); windows or documents are “shoved” around with manual swipes. Whether in plays, computer games, simulations, or virtual desktops, the representation of “natural” forces must be consistent and explicit enough to allow people to incorporate them into their understanding of the particular world’s potential.

The construction of situations that possess strong dramatic potential is a central element in the playwright’s art. Situations may have both physical and character-related components (a gun on the desk; a desire for revenge). An obvious but easily overlooked element of situation building is the fact that all of the relevant aspects of the situation must be successfully represented. Watching a small child struggle with a “drawing” program on a computer is a case in point; her actions are limited by her ability to recognize the tools and the context. She is simply not able to do the kind of investigation of the environment and situation that a computer-savvy adult would be willing to undertake; she doesn’t know what rocks (or icons) to look under. For her, the representation is all there is.⁸

8. Finger-painting software works well for the little ones. Now look at these directions for an online drawing program: “Select a color (top left button), select a background color (next button). Then select the type of line you want to use, the width of the line. . . . NOTE: When selecting a color, watch carefully both circles: the main circle aims at choosing a tint and the second one aims at changing the depth of this tint.”

Coincidences can also help to establish probability, but they are ineffective when they appear to be arbitrary. Outrageously arbitrary coincidences are the stuff of comedy and farce, in which the requirements for plausibility are significantly relaxed. People commonly assume that coincidences in non-comic representations have causes that will be revealed; that is, they are more than “random” accidents. In fact, seeming coincidences stimulate people to look for causal connections. If a sword shows up just when I need one in the enchanted castle, is the wizard protecting me? Fortuitous events imply agency, and that is essentially what they are good for: implying the involvement of characters or forces in the action.

The fact that people seek to understand causality in representational worlds provides the basis for Aristotle’s definition of universality. In the colloquial view, an action is universal if everybody can understand it, regardless of cultural and other differences among individuals. This would seem to limit the set of universal actions to things that everyone on the planet does: eat, sleep, love, etc. Aristotle posits that any action can be “universalized” simply by revealing its cause; that is, understanding the cause is sufficient for understanding the action, even if it is something alien to one’s culture, background, or personal “reality.”

We need only look to works of fantasy to find obvious examples of how universalization via causality works. Actions that are patently impossible in the real world (such as a person flying) can be made believable and understandable in their dramatic context if probability is established. This fact led Aristotle to observe that in dramatic action, an impossible probability is preferable to an improbable possibility. We can believe that Peter Pan flies because of the way the potential of his world is revealed, through the way his character is established in the action, and through dramatic situations that provide him with causes to use his ability to fly. Conversely, it is possible that Peter Pan would try to have a conversation with Captain Hook instead of fighting with him (a Monty-Python-esque treatment), but the improbability of that course of action robs it of credibility. This is another reason why coincidences don’t work; it’s improbable, in all noncomic dramatic forms, for just the right thing to happen at just the right time (without some source of agency).

To summarize, probability is the key quality of dramatic action. The orchestration of probability and causality is the stuff of which dramaturgy is made. By manipulating probability, the playwright shapes the dramatic world, the plot, and (indirectly) the audience’s involvement with it. Similarly, probability can be deployed by designers of human-computer interaction to shape what people do and feel in the context of a particular virtual world. To understand more about how dramatic probability can be shaped, we can look to the structural patterns that make probability manifest.

9. Personal communication, 2012.

Drama tends to compress time, while narrative tends to extend it. Temporal compression of incidents provides strategic guidance in the inclusion or exclusion of materials, thoughts, and actions. Compressed time intensifies dramatic action. Time plays an indispensable role in the shaping of structural qualities as described in the next section.

Gustav Freytag (1898), a German critic and playwright, suggested in 1863 that the action of a play could be represented graphically, yielding a visualization of dramatic anatomy that is referred to as the “Freytag triangle” (see Figure 3.3). The notion that the action of a play could be quantified was not unfamiliar to Freytag’s contemporaries in Europe and America, whose “well-made plays” were often formulaic in the extreme (and which did not survive as examples of great drama). It is the underlying logic of Freytag’s analysis, however, and not the recipe-book flavor of his techniques, that is useful in understanding the anatomy of dramatic action.

Freytag’s visualization is based on the notions of rising and falling action.¹⁰ The rising action is all that leads up to a climax or turning point, and the falling action is all that happens from the climax to the conclusion. The rising and falling action form the sides of the triangle, of which the dramatic climax is the apex. The horizontal axis of the graph is time; the vertical axis is complication. Various structural elements occupy different locations on the triangle. Contemporary versions of Freytag’s triangle are more irregular and jagged, reflecting the differing patterns of complication and resolution within structural elements.

10. Freytag’s actual terms were “play” and “counter-play,” and they were based on Aristotle’s “complication” and “dénouement” (literally, “untying,” as a knot).

The “complication axis” of a Freytag graph represents the informational attributes of each dramatic incident. An incident that raises questions (e.g., the kidnapping of the heroine) is part of the rising action; one that answers questions (e.g., the confession of the villain) is part of the falling action. However, Freytag’s analysis was overly simplistic; each dramatic incident may raise some questions and answer others, and the questions themselves may vary in importance to the plot. Freytag’s primary contribution was to provide the beginnings of a visual representation of the shape of dramatic action.

More sophisticated Freytag-style graphs have been developed as tools for dramatic analysis. Each incident is represented as a line segment, the slope of which is derived from the relationship of the informational attributes of the incident (i.e., questions asked and answered) to its duration; for instance, a steep upward slope represents a good deal of complication in a short amount of time. We will use the following dramatic incident as an example.

A group of strangers have been invited by an anonymous person to spend the weekend in a remote mansion. During the night, one of the group (Brown) has disappeared. Some of the remaining characters are gathered in the drawing room, expressing concern and alarm. The butler (James) enters and announces that Brown has been found (see Figure 3.4).

Each informational component of the incident can be characterized in two ways. In terms of complication, the information is either positive (it asks a question) or negative (it answers a question). The importance of the information at the point at which it appears in the plot is rated on a numeric scale from 0 (completely unimportant) to 1 (extremely important). Thus an extremely significant piece of information that answers a question has a rating of –1, while a fairly insignificant piece of information that raises a question might have a rating of +.3. Figure 3.5 shows such an evaluation of the informational components of the example incident.

To represent the incident on a Freytag graph, the sum of the numeric ratings shown in Figure 3.5 can be used as the value for the variable C, representing complication. The duration of the incident in minutes (or pages of script) is used as the value of the variable T, representing time. The formula for computing the slope of the line segment that will represent the incident on the graph is: slope = C/T. In this case, C = 1.6 and T = 1 (one minute or beat of dramatic action). The sample incident is graphed in Figure 3.6.

This analytic technique can yield a detailed profile, represented numerically or graphically, of the shape of the dramatic action of a given play. The fact that this aspect of structure can be expressed quantitatively makes it potentially more amenable to computational representation. Given an informational analysis of the potential actions involved in a human-computer interaction, quantitative structural criteria could be used for orchestrating those incidents into the desired overall shape. This is possible because specific kinds of actions can be seen to have characteristic slopes or curves.

The exposition (segment A) is the part of a play that functions to reveal the context for the unfolding action. It formulates potential into possibilities, introducing characters, environments, and situations. Exposition as the revelation of information continues throughout the play, but it diminishes as the action progresses; it becomes less and less necessary or appropriate to introduce new potential.

The inciting incident (segment B) is actually a small segment rather than a point (since it has some duration); it is the action or event that begins what will become the central action of the play. On the graph, it is the point at which the curve takes its first significant upward turn. In terms of the “flying wedge,” the inciting incident initiates the first lines (vectors) of probability.

The rising action (segment C) follows the inciting incident. In this portion of the play, the characters pursue their central goals, formulating, implementing, and revising plans and meeting resistances and obstacles along the way. At some point, the action “goes critical”—that is, characters must make major decisions and take conclusive actions in pursuit of their goals.

The crisis (segment D) is a period of heightened activity and commitment, and it usually proceeds at a faster pace than the preceding action. During this segment, many lines of probability are pruned away. The climax (segment E) is the moment at which one of the lines of probability becomes necessity, and all others are eliminated. Characters either succeed or fail to achieve their goals (although those goals may have been reformulated during the course of the dramatic action). This key incident is the turning point of the action.

The falling action (segment F) represents the consequences of the climax, as they reverberate through character and situation. The slope of the falling action is characteristically rather steep; that is, things tend to fall into place quickly once the climax has been reached. The dénouement (segment G) can be described as the return to “normalcy” (the status quo of the dramatic world). In English, the word “dénouement” means “untying” or “unraveling.” The dramatic potential is exhausted; its intrinsic energy has been used up by the action.

In Hamlet, for example, the overarching concern is revenge. The pattern of the plot could be described as a battle of forces: moral thinking, impulse, and deception. All of the major characters make choices in these realms. Hamlet, in his quest to avenge his father’s death, careens between overthinking and impulsive action, resorting to deception in his arrangement of the “play within a play.” Claudius, who wishes to hide the murder of Hamlet’s father, looks first to deception, is plagued by moral thought, and succumbs to impulse in the poisoning of swords. Regardless of the characters involved, the audience typically knows something that the characters don’t. In most scenes, a plan made by a central character is thwarted suddenly by a reversal near the end of the scene. Patterns within patterns lay out these elements in different combinations. What we have, then, is a play with nested parts that rework its major themes in a particular manner involving reversal for the characters. Each of these scenes can be said to be self-similar at scale, even though the pattern of the scene or act may involve a recombination of forces.

Science tells us that such self-similarity of dimensions or parts of a thing in relation to the whole is pervasively true of natural phenomena. Richard Voss and John Clarke (1976) identified the temporal manifestation of fractals in the mathematical expression of 1 over f noise, commonly called “pink noise,” which produces the pleasure of the fractal phenomenon on the auditory level. Mandelbrot and Frame (2002) tell the story of Voss’ discoveries:

As a graduate student at Berkeley, Richard Voss was studying this problem, using signal-processing equipment and computers to produce the power spectrum of the signal from a semiconductor sample. When one sample had burned out and another was being prepared, Voss plugged his signal-analyzing equipment into a radio and computed the power spectrum. Amazingly, a 1/f spectrum appeared. Voss changed radio stations and repeated the experiment—another 1/f distribution. Classical, jazz, blues, and rock all exhibited 1/f distributions. Even radio news and talk shows gave (approximate) 1/f distributions.

Mandelbrot and Frame have documented 1/f noise in Western music as well as African, Japanese, Indian, and Russian and through a range of times, from the Medieval period through the Beatles. They conclude:

Voss uses these observations eloquently to bring closure to one of the classical Greek theories of art. The Greeks believed art imitates nature, and how this happens is relatively clear for painting, sculpture, and drama. Music, though, was a puzzle. Except for rare phenomena such as aeolian harps, few processes in Nature seem musical. Voss uses the ubiquity of 1/f noise to assert [that] music mimics the way the world changes with time.

It is remarkable to me that such patterns can be evinced in made as well as natural phenomena. This tells us something about organic wholes, but also about the human mind—both as creator and beholder.

An important thing to note about this analytical technique is that it reveals a major source of a play’s aesthetic appeal; that is, it provides some explanation of why a play feels good.¹¹ As Aristotle’s analysis of the qualitative elements of structure (discussed in Chapter 2) suggests, pattern is a powerful source of pleasure. Designers of human-computer interaction can borrow concepts and techniques from drama (and nature) to visualize and orchestrate the structural patterns of experience.

11. An interesting exercise in scientific (or artistic) visualization would be to create first-person versions of such graphs, so that one could experience them kinesthetically by “riding the curves.” Would such abstractions feel good in and of themselves? If we represented them audibly, would they sound like music? Or surf?

It is relatively easy to see the relevance of orchestrating the shape of action in story-based human-computer activities like computer games or interactive simulations. But what about more pragmatic, “computer-like” activities—say, spreadsheets? Both Heckel and Nelson have extolled the virtues of VisiCalc and its descendants (e.g., Excel). Heckel (1982) identifies one source of the product’s appeal as the immediate representation of the effects of users’ actions: “While entering formulas, the user is continuously stimulated. Similarly, when changing a number, the user is stimulated by the effect of the changes as they ripple through the spreadsheet.” This source of a good spreadsheet’s appeal can be visualized as a Freytag-style curve. Let’s say I’m using a spreadsheet to decide whether I can afford to buy a new house. Referring back to Figure 3.7, the various segments of the graph might correspond to the following actions:

A. Getting started. I enter the price of the desired house, the price that my current home is likely to fetch on the market, and any additional numerical data that I might have, such as interest rates, property taxes, and the costs of utilities.

B. Preliminary evaluation. I discover that the new house, in terms of the data already entered, will cost me $1,000 more per month. Things are looking bad, but I really want to be able to afford the house, so now I am going to start trying to think of things that will turn the picture around. Thus the “inciting incident” is the initial set of calculations, which leads to my decision to pursue a new goal: to make the numbers support the desired outcome.

C. Entering new data and formulas. Are there tax benefits that derive from the interest rates and increased debt? How will my utility bills change if I replace the new house’s electric heating system with a gas furnace? I try different strategies with positive and negative effects.

D. Making major trade-offs. Things are still looking bad to iffy; now it’s time to decide what sacrifices I am willing to make. Finally I decide that I can live without a new car, that I can forego furniture in the living room, and that I could borrow an additional chunk of down payment from my mother. Will any or all of these sacrifices be sufficient?

E. Making the decision. I “turn the crank” by implementing each of these sacrifice scenarios in turn and then in combination, until I arrive at one I can live with. Yes, there is a way to afford the new house.

F. Creating an artifact. I clean up the spreadsheet, do a little formatting, and print the whole thing out to show my husband so that he, too, will be convinced.

G. Finishing up. I save the document and exit the application.

The spreadsheet illustrates how the conception of the application and its functionality shape the action by providing elements of form. It also shows the way in which the application and the person collaborate to create a whole action with an interesting shape. It illustrates the fact that an application, in both its conception and its execution, defines the magnitude and texture of the whole action. Spreadsheets such as Excel are successful largely because they do an extremely good job of supporting whole actions with a satisfying degree of complexity, magnitude, and completeness. One could perform the same whole action as that in the previous example with a calculator, an abacus, or even a pencil and paper, but its magnitude (in the sense of duration) would be excruciatingly excessive. The action would lack organic wholeness; rather than the elegant Freytag-like curve, the action would more likely consist of long, flat-line segments of calculation punctuated by periods of analysis and planning with a completely different representational context and “feel.” In contrast, word processors, especially those that admit only text manipulations, do a comparatively poorer job of supporting actions with interesting shapes in that they focus on only part of a larger task. Programs designed to support document creation fare better in terms of dramatic shape because one is more likely to be able to do what one visualizes.

Discovery becomes more interesting when the new information is not what one might have expected; in other words, it’s a surprise (what’s that scruffy bum doing at this fancy party? Why is the house suddenly shaking? A higher interest rate may give me a tax break!). Surprises have a higher potential for complication than do run-of-the-mill discoveries; that is, they often raise more questions than they answer. Although in “real” life surprises are as often nasty as they are pleasant (why is the house suddenly shaking?), in the context of drama, they are almost always pleasurable, in that they lead to excitement, vicarious feeling, engagement, and speculation—and we are “safe” from real-world consequences (there’s an earthquake going on—don’t worry, honey, it’s only a movie). Surprise is that subspecies of discovery that is different from what one expected (or might logically have expected) to be true. Surprise is deployed by playwrights to turn up the gain on emotional and intellectual involvement—to quite literally give the audience a thrill.

A more rare and potent flavor of surprise is what Aristotle referred to as reversal: A surprise that reveals that the opposite of what one expected is true (that’s not a man, that’s a woman! The detective is actually the murderer! I thought that “formatting” would tidy up my disk, not erase it!). Reversals can cause major changes in our understanding of what is going on and our expectations about what will happen next; in other words, they can radically alter probability. In a play, an early reversal might serve as an inciting incident, causing a sharp upturn in the C/T slope (by raising a whole set of questions all at once). The climax of a play may be a reversal that causes a sharp downward turn in the slope (by answering a host of questions all at once).

In human-computer interaction, like drama, surprise and reversal are efficient and economical means for achieving radical shifts in probability. The reasons for wanting to create such a shift may be pragmatic or aesthetic. A reversal may be needed to turn a person away from an unproductive or potentially dangerous path of action. Surprise and reversal can also be deployed to create changes in the “slope” of the action in order to achieve a pleasing whole. Of course, it must be remembered that dramatic reversals have no serious real-world consequences. Obviously, one should avoid any incidents that cause actual pain or harm (such as erasing a file or destroying a document). In summary, surprises and reversals are tools for changing what people understand and expect, for stimulating interest and involvement, and for orchestrating the shape of the action.

In this chapter, we have defined elements of form and structure that are characteristic of dramatic action and shown how they relate to human-computer interaction. In the next chapter, we will consider how dramatic theory can be employed to understand and orchestrate human action in representational worlds.