Reference Levels

Mixing for cinema has long established a reference level to mix at, which is then replicated exactly in the auditoriums. Within this controlled environment, you know that audiences are going to be experiencing the film at exactly the same level that you are mixing at. Although there have been efforts to establish a common approach in games (such as an average of −23 LUFS—see this chapter’s Further Readingsection on the website for more background to this), there remains a variety of practices.

People play games in a variety of environments and circumstances and through a variety of mediums, from headphones to a mono television to a 7.1 THX-certified surround sound setup. If you are not mixing within a professional environment (and as beginners we’ll assume that most of you aren’t), then a good rule of thumb is to use other media to set a comfortable listening level—and then stick to that. Your users are typically going to be using their consoles for watching films and TV as well as listening to music, and they are not going to want to have to leap up and adjust their volume as they move between these media.

Sound Class Properties

Use the Content Browser filters to search for Sound Classes and double-click to open the Sound Class Editor. Select a Sound Mix in the Graph window to view its Details panel.

The Volume and Pitch apply a master control to anything attached to this class.

Stereo Bleed refers to the amount of a stereo sound source that should bleed into the surround speakers.

LFE Bleed is the amount of sound to bleed to the LFE channel (the low frequency effects channel that goes to the subwoofer speaker). You might want to avoid this for most sounds, saving the effect for specific instances.

Voice Center Channel Volume is the volume of the source that will go to the center channel or center speaker in a 5.1 and 7.1 speaker setup. This is traditionally reserved for dialogue in the cinema, and it can be useful in games if it is more important that the dialogue is heard than spatialized. You can also change this dynamically via a {Sound Mix}.

Radio Filter Volume and Radio Filter Volume Threshold applies a radio-like filter and setting to the sounds if the volume falls beneath a given threshold (i.e., it is heard through the center speaker and does not attenuate over distance).

Apply Effects governs whether sounds belonging to this class have effects such as filtering applied.

Always Play will prioritize the sound above others in a limited voice count situation.

IsUISound determines whether or not the sound will continue to play during the pause menu.

IsMusic designates the class as being music or not.

Reverb controls whether Reverb is applied to this class.

Sound Mix: Triggered Ducking

• Project: DemoCh09SpaceShip
• Level: SpaceShip01

You can skip between the areas of the ship using the keys 1–0.

There are a variety of circumstances where you might want to duck (attenuate) all the sounds or music in order for a specific sound to be heard, but this is most commonly used for instructional dialogue.

As you awake in the *Floor 01: Sleeping Quarters* (Bookmark 1), you’re told “Wake up, the ship’s under attack!”. The console command Stat Soundmixes will display the active soundmixes to the screen while you play and it might be useful to do this for the following sections so that you can see what’s going on.

As this dialogue is played, a {Sound Mix} is pushed on, and when it completes it is popped off.

The other sounds in this area are set to belong to the Sound Classes: {Ship_Ambient}, {Exception_ Class}, and {Tannoy}.

The Sound Mix {Duck_Ship_Ambient} is set to apply itself over 0.2 seconds and then remain on (duration settings of −1.0 stay on indefinitely). Since this mix doesn’t have a set duration, we then need to remove it (pop) when the dialogue line has finished, which we do using a <Bind Event to Audio Finished>. The effect of this is to duck down all the other sounds so that the instructional dialogue line can be clearly heard.

Sound Mix: Triggered EQ

In the *Floor 01: Corridor* you’re told “Get out of there! You need to get to the bridge!” Note how this time the ducking effect is a little subtler—instead of reducing the volume of everything, we’re applying an EQ setting via the Sound Mix to allow the dialogue to punch through the other sounds.

Although we’ve used it here for dialogue again, this technique is equally applicable when you want to highlight a big sound effect event while music is playing. Music, particularly a dense orchestral arrangement, often takes up a large range of frequencies, so the use of the EQ within the Sound Mix function is very useful in notching out a frequency range for a specific sound effect to momentarily shine through without disrupting the musical flow. The system for turning the mix on and off is the same as above.

This time we’re calling the Sound Mix that applies the EQ {Notch_for_Dialogue}.

The EQ is a three-band EQ with a high and low shelf and a parametric mid. The frequency is in Hz, the gain is from 0.0 to 1.0, and the mid frequency bandwidth ranges from 0.1 to 2.0. One of the best approaches to working out some appropriate settings is to record the audio for a sequence of your gameplay and import this audio track into your DAW. There you can play around with the EQ within a slightly friendlier environment before attempting to imitate these settings within the Sound Mix. If you look in the “Initialize” section of the Level Blueprint you’ll see that we’ve also implemented a {WakeUpFilter} mix as the level starts.

Passive/Automatic Mix Changes

Passive Mixes for Dialogue

Obviously setting up a push/pop Sound Mix system for every line of dialogue in your game is going to be a pain. Fortunately the Sound Class has the functionality that allows us to call a specific mix (when certain criteria are met) whenever a sound registered to that class is played.

After the explosion in *Floor 01: Corridor*, a hole is punched through the wall into the *Floor 01: Server Room* and you’re told “Go through the lab section to your left. Hurry—the airlocks are going to go!”

This line {Vo_Line_003} is assigned to belong to the {Ship_Dialogue_With_Passive}sound Class. As you can see in the Sound Class Editor, this has a passive Sound Mix that calls the {Duck_Ship_ Ambient} mix. Anytime a sound belonging to this Sound Class is played, it will automatically apply this Sound Mix. It will also automatically pop offthe mix when the sound has finished playing—which is handy!

Passive Sound Mixes have some additional parameters (min volume/max volume thresholds) that govern whether they actually start or not and you can set these according to the following three different usage scenarios:

• a. Always Apply the Mix

  

  If you want the mix to always be applied, then set your Min Volume Threshold to be 0.0 and your Max Volume Threshold to be high (e.g., 10.0). When a sound from the designated Sound Class plays, then its volume will always be above the Min Volume Threshold and below the Max Volume Threshold, therefore the Sound Mix you’ve referenced will be applied.

• b. Auto-ducking Other Sounds If Designated Sound Class Is Too Quiet

  

  If you only want the mix to be applied if a sound is played too quietly, then set your Max Volume Threshold to the cut off point from where you want the mix to be applied. When a sound from the designated Sound Class plays at a volume between 0.0 and max, then the Sound Mix will be applied. Typical use would be to assign this passive Sound Mix to your dialogue class so that if the dialogue is playing too quietly, it will auto-duck everything else so you can hear it.

• c. Dynamic Range

  

  If you want something to have a particular impact, then you can temporarily duck the other sounds to give it more headroom. This clears the voices that you probably won’t hear anyway and allows a greater dynamic range. If the sound plays loudly (i.e., above min threshold but below max) then the Sound Mix will be applied.

  Note that the values of min/max are derived from the volume of the sound. That is its playback volume as designated by its attenuation curve. This is not the same as the sound’s actual volume. Many systems in other engines do actually track the real volume of the sound, which is more effective for this kind of side-chain effect.

Auto-attenuation or Culling for Dynamic Range

If you go to the end of the corridor, you can hear an explosion sound with the Dynamic Range effect referred to above. The Sound Cue {Explosion_With_Passive_Cull} belongs to Sound Class {Culling_ Class}. For its passive Sound Mix, the Min Volume Threshold is set relatively high, so the culling effect will only take place when it is loud. If you walk close to it, you can hear that it is a dramatic loud explosion that momentarily blocks out all other sounds, but as you’re further away becomes more of a background sound.

As we discussed at the opening of this chapter, we notice changes and differences in volume rather than having an awareness of absolute values. If you have a big event happening that you want to have real impact, then the best effect is going to be achieved by preceding it with a quiet section to make the dynamic contrast more dramatic. This is of course not always possible, but in addition to the culling technique outlined here, you could also affect a preduck strategy that momentarily brings all the sounds down in volume immediately before loud events such as explosions or car crashes.

All platforms have some sort of built-in limiting system to stop distortion. These are often clumsy and unpredictable, and so it’s really best to avoid these systems having to be applied in the first place. There are also, however, increasingly intelligent systems of automation that can really help the overall mix of the game. Given the huge dynamic range of sounds in the physical world and the limited dynamic range that can be represented by 16 bits of information, it is not surprising that achieving the feeling of a wide dynamic range in games is a particular challenge. We’ve discussed how the level design can contribute to this, but we also need systems in place that will allow for the intelligent management of which and how many sounds are playing so that the combinations of these sounds do not cause clipping and bring in the blunt instrument of the limiter. Chirping insects may add an important element of immersion to a game environment when you are creeping around, but in the midst of battle you will not hear them, as they will be masked by the louder sounds. An automated system that monitors the current levels of the sounds at the player’s location and will cull the quieter elements, that you won’t hear anyway, will both clean up the mix in terms of frequency content and create some additional headroom for the sounds you do want to hear.

There has been a considerable amount of clever work done on interactive mixing for game audio in recent years that falls outside of the introductory remit of this book. We would encourage you to undertake some further reading in this area by following the links on the website.

Exercise 08_03: Passive Mixes

In this exercise we are going to set up an automated ducking system for whenever dialogue is played in the game.

1. Create a new Sound Class {Dialogue_Ducking} and assign all your dialogue Sound Waves or Sound Cues to it. Remember you can select and adjust multiple Sound Waves simultaneously by selecting them all and right-clicking to open a Generic Asset Editor that will apply to all.
2. In the Passive Sound Mix Modifierssection of the {Dialogue_Ducking}Details panel, add an element by using the +sign.
3. Reference the Sound Mix {Duck_Ambience} you just created in Exercise 08_01 since this is already set up to duck any sounds belonging to the ambient class.
4. Set its Min Volume Threshold to 0.0 and its Max Volume Threshold to 10. This will ensure that it is always called.
5. Play the game. You should find that every time your dialogue is played the ambient sound ducks out. You might want to create a new {Sound Mix}since the settings of the {Duck_Ambience} you used for the explosion may not work as well for dialogue ducking.
6. If certain sounds are not ducking, then it’s probably because they don’t belong to the {Ambient}sound Class. You could create additional Sound Classes for these and add them as Children of the {Ambient}sound Class. You can do this by dragging and dropping them onto the Graph of this class and connecting them as Children. Make sure the Apply to Children option is checked in your Sound Mix, and your Passive Sound Mix Modifier will now apply to these as well.
7. You might also want to set up a Passive Sound Mix Modifier for the weapons sounds so that they duck the quieter elements of the ambience (the area loops for example) or Foley. During weapons fire you probably would not hear these anyway, and by ducking them out we allow more headroom for the weapon sounds. Better still would be to only use the looping or retriggered elements of the weapon sound to instigate the duck so that the ambience is already coming back up during the tail portion, not waiting until after it has finished.

Sound Mix: Base Sound Mix and Mix Modifiers

All the Sound Mixes we’ve applied so far have actually been mix modifiers. These are mixes that you can push on or pop off at will and also apply simultaneously, as we will see in a moment. We can also apply a base Sound Mix via the <Set Base Sound Mix> node. This, however, is a permanent mix that you cannot pop off. You might have a use for this if you find that you want to make some global changes to the relative volumes of different types of sound or classes as a fixed setting across the whole game.

In the *Floor 01: Server Room* we have a problem. The airlocks are malfunctioning, and every 20 seconds or so all the air is sucked out and we are in a floating vacuum. Your helmet has a detect mode (press H) that should help you locate the passkey to the elevator at the far end.

For this scenario we need multiple Sound Mixes to be operating simultaneously, one for the air vacuum effect (which attenuates and equalizes everything out) and one for the detect mode (which highlights the beeping of the key in the mix). After a lead-in sound, {Airlock_Out_02} and the {Airlock_Malfunction} mix are implemented (this has a timed duration of 7 seconds, so doesn’t need to be popped off). This mutes everything, including the {Exception_Class} that the {KeyCard}sound Cue belongs to.

In order to bring this back up during the detect mode (instigated by the H <Key Press>), we apply another modifier mix. The adjuster settings work as multipliers, so if the {Airlock_Malfunction} mix adjusts the volume of our {Keycard} by 0.1, then our mix modifier {Detection_Mode} needs to have a Volume Adjuster value of 10.0 to bring the overall volume back to 1.0.

Note that as you can have multiple Sound Mixes at the same time, but not multiple EQs, there is also an EQ priority system to determine which EQ should take precedence. This uses the same approach to priorities that we have seen with the audio volumes and reverb (i.e., the higher the number, the greater the priority).

Sound Mixes have many uses in shaping a player’s experience, but they are also very useful when prototyping and developing your level. It is very handy to set up some mixes that solo particular types of sound (ambience, music, objects, dialogue, weapons, cats, etc.) that you can call up with some shortcut keys. See the Testing the Mixsection later in the chapter.

Character State

As you pass through the *Floor 02: Pipe Corridor* (Bookmark 2) section, you’ll get repeatedly hit by the bursts of fire and steam shooting out from the pipes that line the walls. You’ll need to pause on the way down in order to regain some health.

In order to notify the player about their current health state, we’ve applied both a Sound Mix (EQ) and a system that brings in a heartbeat sound on low health. These kind of subjective sound states that represent how a player character might be feeling through the audio can be very effective in immersing the player.

See the [BP_GAB_PipeExlosion] Blueprint and the “PlayerHealthSystem” section in the [MyCharacter] Blueprint.

NPC State

One of the most essential pieces of information that sound often provides us with is that of the current status of enemies or other NPCs in the world. In this instance (*Floor 02: Turret Section*), the NPC is actually a series of gun turrets that the player needs to sneak past. (Bookmark 3)

When the player is visible to the turrets, they power up and enter a scanning phase. If the player is still visible when this phase starts, they will fire, but if not they will power down when scanning is complete. It’s important that the player knows what state the turrets are in so they can time their movements, and it’s the audio that provides this information.

See the “Turrets” section of the [Level Blueprint] and the [BP_Turret] Blueprint.

Object State

Sound can also indicate the state of objects. In the *Floor 02: Mind the Gap*section (Bookmark 4), the player must try to cross a deep shaft by moving a series of girders into place.

The control panel is in a side chamber, so the player must rely on listening to gauge whether the objects are in position.

As you move each girder, the different sound elements change pitch to indicate the correct position to the player. This is done by reading through curves in <Timelines> to <Set Float Parameters>. The left hand button resets the system.

See the “Girders” section of the [Level Blueprint].

Game State

In the next area *Floor 02: Puzzle Pipes* (Bookmark 5), you must fix the broken pipes by finding suitable parts around the room. As you do so, you can hear the three steam sounds get muted one by one to indicate your progress.

See the [BP_GAB_Puzzle] Blueprint.

Pickups

In certain genres of games (particularly platformers), the repetitive nature of the pickup sounds serve as a kind of conditioned response to punish or reward the player. Whether it’s picking up stars, coins, or powerups, the sounds are there to reward the player with a pleasurable confirmation of achievement.

In *Floor 03: Power Cell Pickups* (Bookmark 6) you need to collect 6 healthy power cells and deposit them in the mechanism within a given time to power up the door. There are several healthy cells around the room, but unfortunately there are also some unhealthy ones too—these cause damage to the player. In the Blueprints for the two object types, there are different sounds that provide immediate positive or negative feedback to the player when they are picked up (collided with).

Although we have droned on a great deal about the need for variation in your sounds, these types of feedback sounds are an exception to the rule. These sounds are the equivalent of earcons in computer software, like the sound that tells you you’ve got mail, in that they are sounds that carry specific meaning. Although a typical coin or star pickup sound may go up in pitch if you collect more within a given period of time, for symbolic sounds like these, we are primarily interested in the information they convey rather than the sound itself, so the repetition of the same audio sample in this case is less of a problem than it has been in other circumstances. As the player is concerned with the learned meaning of the sound, rather than any information conveyed by the nature of the sound itself, we don’t want the distraction of variation where the player might be forced to think about the sound more. If there were variations in the sound, the player would be forced to consider questions like why is this sound different and what is it trying to tell me, rather than simply being able to acknowledge the information that the presence of the sound has provided.

See the [BP_DoorAccessPanelGame], [BP_DoorPowerCell_Healthy] and [BP_DoorPowerCell_ UnHealthy] Blueprints.

HUD Interactions

In the real world, sound is often used to provide a user with a confirmation of their action. Think of your cell phone or the ATM. Sound gives you the message that the button you’ve just pressed has in fact been acknowledged. This kind of confirmation sound is even more important in games, where the amount of feedback you get through their physical interfaces is often limited. Some interfaces may have rumble functionality, where you can get some (haptic) feedback to confirm your actions, but many have none and so it’s important to have the player’s interactions confirmed with sound as a representation of the tactile feedback of the physical world.

After you complete the powercell pickups task, the *Floor 03: Fuel Cell Storage Door* requires a key code (it randomizes between 592 and 437). When designing sounds to be used for this sort of menu navigation, you need to use sounds that somehow convey the notions of forwards/backwards and accept/decline.

It is difficult to generalize about the sounds to choose that might have these kind of positive (forward/accept) or negative (backward/decline) connotations. If we look at a rule of thumb from speech patterns that we could use, it might be that positive meanings are often conveyed by a rise in pitch, negative by a fall in pitch. By using slight variations on a few sound sources, it is easier to produce a positive or negative version that conveys this meaning than using completely different sounds. This also gives them an identity as a family of related sounds, rather than simply a collection of unrelated ones.

Your sounds will have more coherence and unity if you link them with the theme of the game world itself or with an aspect of the object that you’re interacting with. If your game is set within a dungeons and dragons type world, then the UI menu sounds might be made of armor and sword sounds, if set in a sci-fi world, they might be more electronic in nature. If you’re going to have menu music playing while the game is in this mode, then you might consider how the UI sounds will fit with this. You could pitch shift your UI sounds so that they were in the same key as the music so that they did not sound too abrasive against it, or you could use musical tones that fit with the style and notes of the background music. This may not be appropriate in all cases but can add a nice sense of polish to your menus.

See the [BP_DoorAccessPanelGame], [Player], and [WBP_HelmetHUDInterface] Blueprints.

Navigate

The main corridors have been blocked on *Floor 04: Air Ducts*(Bookmark 7), so you’ll have to navigate your way through the air duct system (press F for a flashlight, and C to crouch).

You can call up your tracking device (T) to help guide you through this maze of ducts. As you pass each juncture, the tracker will speed up if you’re going the right way.

See the [Player] Blueprint for the flashlight system, and the “Ventilation Shaft Tracker” section of the [Level Blueprint] for the tracker system.

Attract

Audio can act to draw attention to things, whether it be objects in the game or on-screen instructions. In the *Floor 04: Corridor Hatches* area (Bookmark 8), you need to find three audiologs that have been hidden in security hatches. If you listen carefully, you can tell when you are walking over a security hatch, as the footstep sound changes slightly. Press E to open the hatch and listen to the audiologs.

See the [BP_Floor], [Player], and [BP_AudioLog] Blueprints.

Repel

As well as attracting players, audio can also repel them. This might be through hearing the presence of NPCs or more literally in the case of *Floor 04: Radiation* (Bookmark 9), where the player must avoid the radioactive pods and mines in the room. On proximity to the pods, you can hear radioactive sounds as they start to affect your health. The mines start beeping on proximity, and if you don’t move away, they explode!

See the [GAB_Mine_BP] and [GAB_Radiation_Pod_BP] Blueprints.

Recall/Learn

You need to hack your way into the final *Generator Rooms* (Bookmark 0). Hacking (by pressing E) the computer terminal in the room on the right gives you a pitched code. You need to recall this pitch sequence for the key code to the door.

See the [WBP_MusicalCodeHUD] Blueprint and the “Musical Key Code” section of the [Level Blueprint].

Decoy

There is a guard patrolling this area. Do not attempt to tackle her directly but instead lure her over to a corner so that you can sneak past. Press R to throw an object that will make a noise and cause the guard to investigate (the player’s footsteps and gun shots will also alert the guard, so press Ctrl to sneak).

Since there is a new audio technique being used here, we’ll go into it in a little more detail. The [Player] Blueprint handles the throwing of the [ThrowObject_BP] projectile.

When this hits something, it plays an impact sound and calls an event within the [Player] that triggers a <MakeNoise> node. Only pawns can have a pawn noise emitter component or a pawn sensing component, so although the make noise event comes from the location of the hit event, this is why we have to do it in the player Blueprint.

The bot, [BotChar_NewMesh], contains a pawn sensing component that listens for the Make Noise event, and its behavior tree {BotAITree} references the Blueprint [BTS_CheckForEnemy]. Should the bot hear the make noise event, this changes the bots usual patrolling behavior so that it now goes to the location of the noise source.

Mask

In the *Floor 05: Engine Room* you need to take out the engines without the security turrets sensing your presence. Creep (hold Ctrl) and use the sound of the engines as they cycle up to their most intense to mask the sound of your shots (shoot the red control panels).

See the “Final Engines” section of the [Level Blueprint], the [Player] Blueprint and the [BP_Turret2] Blueprint.

Recap:

After working through this chapter you should now be able to:

• Understand some of the roles and functions of audio in games through the INFORM model;
• Use triggered and passive Sound Mixes;
• Implement systems using making and hearing noise.