Summary: Interactive music forms, playlists, decay tails, parallel and transitional music replacement
Project: DemoCh03MusicRacer01 Level: Racer_C
In this chapter you’ll get a very quick introduction to game music. We’ll talk briefly about some of the principles and systems, but don’t worry too much about how these are working since we will be going into them in more depth in the following chapters. For the moment this map is about getting you started with exploring some of these ideas in a practical way by writing or editing some music and importing it into the racer game we’ve provided.
If you want to jump right in and hear the final system in place, then open the Map: Racer_C. If you really want to get your music into the game straight away, you can skip to Exercise 03_05: Manual Looping and Onwards.
You can hear the music interact while you play, or you can control the system yourself manually by using the following shortcut keys:
For simplicity in all the music examples, we’ll be using stereo wave files, but there’s no reason that you couldn’t use surround music mixes (see Chapter 07 for the method of importing multichannel files), although you should be careful with this. The surround channels in games play a vital role in terms of immersion and provide the player with vital gameplay information (such as the location of NPCs), so any distinct spatialized sound will tend to be considered as being part of the game world. In situations where you want the music to unnerve the player, it can work well to play on this ambiguity, but in general the more subtle approach of putting some of the music’s reverb in the surrounds often works better.
Music is used in lots of different ways in games and for some of these, for example music for cutscenes, the process of writing and implementing music is pretty much exactly the same as it would be for film. We’re going to focus on the aspects that are unique to games and the implementation techniques that are used to deal with the two key challenges for interactive music in games:
We will be looking at the principles and implementation techniques for (a) in this chapter along with Chapters 04 and 05, and we will be dealing with (b) in the advanced music sections of Chapter 12. The chief challenge in music for games is that there is an inherent conflict between music and gameplay that we need to overcome.
Most music has a time-based structure. In order to feel satisfying, things need to happen on bars and beats and sometimes in longer musical phrases. In games the player has autonomy and is largely in charge of when things happen, and so game events more often than not do not happen to coincide nicely with bars and beats and phrases. As Clint Bajakian says, we have constant compromise between “contextual responsiveness and musical integrity.”
Here are some key musical forms that help us deal with this challenge.
This form is where we might have a long piece of background music and then introduce occasional musical stingers over the top to highlight events or actions. Sometimes we want to acknowledge an event or reward the player but do not want to necessarily instigate a large-scale change in the music. The challenge here is that the stingers can happen at any moment, so we need to write the music in such a way so that they won’t musically conflict with what might happen to be going on in the accompaniment at the time. A typical solution to this would be to use percussive stingers or to have the background music be fairly tonally static so the stinger will always fit (there are some other more complex solutions offered in Chapter 12).
Sometime referred to as vertical re-orchestration, this is music that consists of several tracks or stems playing in sync and in parallel. Game variables can then change the relative mix of the stems without interrupting or breaking the musical structure. This form is good for representing changes in states or parameters in the game since we can bring different layers in and out, and by different amounts, to represent different things. We could also use two different arrangements or performances of the same track and seamlessly crossfade between them to change the emotional intensity. However, while it is good for the flow of the music (and the flow of the game), it doesn’t provide quite the same emotional kick or reward as a nicely timed cadence or transition to a new piece.
Transitional forms are sometimes referred to as horizontal resequencing or branching forms since they represent the transition from one piece or section of music to another. These kinds of significant timed changes in music can be the most dramatic and rewarding to the player, heightening their sense of triumph. But they present the greatest challenge since the events we want to respond to rarely coincide with musically appropriate transition points.
Although one could argue that all interactive music is procedural to some extent, algorithmic forms typically work with much more granular or note-level music events. This makes them much more flexible and responsive, but they are more complex to implement. We’ll be looking at these in more detail in Chapter 12. In terms of algorithmic techniques currently applied in games, you could say that they are most effective in both representing changes in states and in providing variation in musical materials that might be playing for long periods of time.
We could implement all the following systems by referencing ambient sound Actors in the game level rather than nodes in Blueprint, and indeed you will find some games that do this. This is entirely personal preference, but we like to keep things clear. For audio that has a source in the game world, use an Actor in the game world. For audio that doesn’t have a source in the game world (in other words is played back directly to the player), use Blueprints. Consistency helps you to keep organized, remember where things are, and find them quickly.
Typically in a racing game, you might have a series of precomposed tracks that play until complete then move onto the next track. These might play in a completely random order or a particular sequence. As we noted in Chapter 01, the -Random- node, when linked up to a loop in a Sound Cue, will only choose the next sound once the previous one has finished playing. We can use this to our advantage if we want to create a randomized playlist. In the Content Browser find the Sound Cue named {Playlist_Random}. Double-click it to open the Sound Cue Editor. The -Looping- node will continue to trigger the -Random- to play these endlessly.
Play the game and listen to the randomized playlist.
The -Concatenator- node will play through its inputs in sequence and so is more useful if you want tracks to appear in a particular order (see the {Playlist_Ordered} Sound Cue). In this case the -Looping- node will restart the -Concatenator- once all tracks have been played, restarting the complete playlist.
Exercise 03_01: Playlist
We are going to take a little break from the main exercise level for this chapter and implement some of your music into a racing level. We will be returning to the exercise level in Chapter 04.
In this exercise you are going to create your own playlist for the racetrack level.
<Play Sound Attached>, -Concatenator
It might be that you want to have a single track that loops around for the whole of the level (as was common in old school games). This is easily implemented, but you need to be aware of a key technique in order to get your loops to sound smooth. When you select the start and end of your loop in your DAW and export, you might get something like this:
With a little care regarding zero crossings (see Chapter 00), this will loop alright when you import it and make it a -Looping Wave Player- in your Sound Cue.
Exercise 03_02: Looping Level Music
In this exercise we will import some of your music to use in a -Looping Wave Player-.
-Looping Wave Player
You may have noticed that your loop feels a little lumpy, with every loop around being obvious and making the music feel more repetitive than it need be. The reason why is actually down to how we edit and prepare our music. Although in your DAW the sound may appear to finish at the end bar line (specifically if you’re looking at a MIDI track), when you listen you’ll notice that the natural decay of these sounds actually lasts significantly longer. Although a few styles of music can work well when split into loops for interactive use, most have instances where the tails of notes or the decay of the reverb fall across the bar line and into the next bar. If we chop our music up at bar line points, then we won’t hear these decays and the music will sound unnatural.
One solution to get your loops sounding smoother is to integrate the decay tail back into the start of the loop. That way when it loops around, you’ll still hear the decay of the final notes.
If you’re working with an audio file, then you can edit the decay portion from the end and mix paste it back into the start (or simply put it on another track and mixdown both).
If you’re working with MIDI files, then you won’t see the decay tail, but you will hear it. The trick here is to duplicate your loop so that you have two versions, one immediately after the other. If we mixdown the whole thing then edit the resulting file in half, the second half will be our loop but will already contain the decays from the first version.
Listen now to the {Loop_Integrated} Sound Cue and compare it with the straight {Loop}.
Exercise 03_03: Integrated Loop
In this exercise you are going to import your track with an integrated decay tail for use in the game.
-Looping Wave Player
The bright sparks among you will have noticed a problem with this: The first time we start the cue, we hear the decays of sounds that haven’t even played yet! What we actually need is to play the original loop first, then loop the version that has the integrated tails.
In the Map: Racer_B Sound Cue {Play_Then_Loop_01}, you can see that we can use a -Concatenator- node to chain these two tracks together.
Unfortunately at the time of writing, the -Concatenator- node is not sample accurate, and so you may find a short pause between the two files. Another alternative way of achieving the same result is to actually start both versions at the same time but use an -Enveloper- node to keep the looped version silent until the first version has finished (you’ll often find yourself having to think of workarounds to get the results you want!)
The {Play_Then_Loop_02} cue has an example of this. Our loops in this case are 6.48 seconds long, and so the -Enveloper- is set to bring the volume of the looping version up after this time has elapsed.
The logic of the audio playback system dictates that if a sound is set to a volume of 0.0, it won’t actually start playing. This makes sense but causes a problem for us here, since it won’t actually start the loop until it starts to fade in, making it out of sync. In order to avoid this, we’ve set the initial volume of the envelope to be 0.001 instead of 0.0.
Exercise 03_04: Play then Loop
In this exercise we will chain together two versions of one of your two tracks: the first version without the decay tail integrated and then the second looping version.
-Enveloper
One way to avoid all this hassle with decay tails is to replace a simple looping system with one that actually retriggers the sound at the looping point but allows the remainder of the sound to play out. With such a system we can export our waves from the DAW with their decay tails intact and these will still be heard while the cue restarts.
Open Map: Racer_C— this is the full racer system with our temporary music in place. If you open the [Level Blueprint], you can see that now the system uses a <Timeline> node (which is set to the length of our desired loop) to retrigger the <Play Sound Attached> that contains our music cue.
This means that in this version of the game you can import your Sound Waves with their decay tails included at the end.
We’ll be looking more specifically at how these systems work in subsequent chapters, but even if you don’t get any further into game music implementation than swapping out some tracks in this level, the concepts around loops and decays are fundamental. You should practice exporting your music in a variety of ways until you’re comfortable with these ideas. That way when you are asked for loops, you can provide ones that will sound good and not clunky.
Exercise 03_05: Manual Looping
In this exercise you will set up the manual looping system that will allow you to import all the rest of your tracks into the complete system.
<Timeline>
Change the Length value to the length of your loop. Remember this is the point at which you want to restart the cue, not the total length of the cue.
Sometimes you just want to acknowledge an action or event in the game without having a major change or transition in the music. Stingers, or ornaments, are one-shot music cues that simply come in and play over the top of the current music loop.
Since these typically can occur at any moment, it’s important that you write them in such a way that they will fit musically along with anything that might be happening in the accompaniment. In the Racer_C map, there are stingers implemented to reward the player for picking up a speed boost.
Exercise 03_06: Stingers
In this exercise we will add a musical pickup stinger to the game.
Parallel forms, where we control the volume of a set of synchronized tracks or stems, are straightforward to implement since we can trigger all our stems from the same <Timeline> and then just adjust their volume in response to game events.
In the “In the Lead” and “Damage” sections of the [Level Blueprint], you can see where this takes place. A variable has been created In_the_lead_layer_volume and is <Set> to either 1.0 or 0.0 depending on your position in the race.
The damage layer works slightly differently in that it increases the volume incrementally each time the player’s vehicle gets damaged, but the principle is the same.
Exercise 03_07: Parallel Layers
In this exercise you will add two parallel layers to the music system.
Sound Cue
Transitional forms usually present more of a challenge than ornamental or parallel forms since we don’t know when things are going to happen in-game. In this game we don’t know when the player is going to cross that start line and start the second or third laps.
To keep things relatively straightforward, we’ve constrained the system to only switching at the loop point (i.e., when the timeline triggers). The laps are set by changing the variable Cue_Switch_Control in the [Level Blueprint].
This controls a -Switch- node inside the Sound Cue {All_Laps}, so when it is next triggered by the <Timeline>, it will now play the next lap (or win or lose). Don’t forget that switches, like all other nodes, consider their first input as 0, not 1.
Exercise 03_08: Transition Tracks
In this exercise you will add two new tracks for Lap 02 and Lap 03 to which the music will transition. We will also add the win and lose cues to complete the whole music system for the game.
-Switch
Looking at our Map: Racer_C, you can see that actually each lap does not just play one track, but makes a randomized choice between three variations of that track. As players might be playing for quite a while, this maintains some interest.
Exercise 03_09: Variation
Add some variation to your music.
Hopefully now you have a good understanding of the different types of interactive music forms and how to write for and prepare assets for use in them. Decay tail solutions may seem like a bit of an annoyance, but getting this right makes a lot of difference to the final polish of your soundtrack.
For further reading please see the up-to-date list of books and links on the book website.
After working through this chapter you should now be able to: