We just achieved good enough graphics quality, but we are missing an important part of the game aesthetics: the sound. Often relegated to being a final step in game development, sound is one of those things that if it's there, you won't notice its presence, but if you don't have it, you will feel that something is missing. It will help you to reinforce the ambience you want in your game and must match the graphics setting.
In this chapter, we will examine the following sound concepts:
As with graphic assets, it is important to properly set up the import settings of audio assets, which can be resource-intensive if not done properly.
In this section, we will examine the following audio importing concepts:
Let's start by discussing the different kinds of audio we can use.
There are different types of audio present in video games, which are the following:
For almost all games, we will need at least one music track, one ambient track, and several SFX to start the production of the audio. As always, we have different sources of audio assets, but we will use the Asset Store. It has three audio categories to search for the assets we need:
In my case, I also used the search bar to further filter the categories, searching for weather to find a rain effect. Sometimes, you can't find the exact audio separately; in such cases, you will need to dig in Packs and Libraries, so have patience here. In my case, I picked the three packages you can see in the following figure, but importing just some of the included sounds, all of them would weight a lot in the project. For ambience, I picked rain. Then, I picked Music – Sad Hope for music, and for SFX, I picked one gun sound effect package for our future Player's Hero Character. Of course, you can pick other packages to better suit your game's needs:
Please remember that those exact packages might not available at the moment you read this. In that case, you can either download other ones, or pick the files I used from the GitHub repo. Now that we have the necessary audio packages, let's discuss how to import them.
We have several import settings we can tweak, but the problem is that we need to consider the usage of the audio to properly set it up, so let's see the ideal settings for each case. In order to see the import settings, as always, you can select the asset and see it in the Inspector panel, as in the following screenshot:
Let's discuss the most important ones, starting with Force To Mono. Some audio may come with stereo channels, meaning that we have one sound for the left ear and another one for the right ear. This means that one piece of audio can actually contain two different audio tracks. Stereo sound is useful for different effects and instrument spatialization in the case of music, so we want that in those scenarios, but there are other scenarios where stereo is not useful. Consider 3D sound effects such as a shooting gun or some walking-pace steps. In those cases, we need the sound to be heard in the direction to the source. If the shooting of a gun happened to my left, I need to hear it coming from my left. In these cases, we can convert stereo audio to mono audio by checking the Force To Mono checkbox in the audio import settings. This will make Unity combine the two channels into a single one, reducing the size of the audio usually to almost half its size (sometimes more, sometimes less, depending on various aspects).
You can verify the impact of that and other settings at the bottom of the Audio Asset inspector, where you can see the imported audio size:
The next setting to discuss, and an important one at that, is Load Type. In order to play some audio, Unity needs to read the audio from disk, decompress, and then play it. Load Type changes the way those three processes are handled. We have the following three options here:
Maybe if you are an experienced developer you can easily determine which mode is better suited for which kind of audio, but if this is your first encounter with video games, it may sound confusing, so let's discuss the best modes for different cases:
There are other cases to consider, but those can be extrapolated based on the previous ones. Remember that the previous analysis was made by taking into account the requirements of the standard game, but this can vary a lot according to your game and your target device. Maybe you are making a game that won't consume lots of RAM but is pretty intensive in terms of CPU resources, in which case you can just put everything in Decompress on Load. It's important to consider all aspects of your game and to balance your resources accordingly.
Finally, another thing to consider is the compression format, which will change the way Unity will encode the audio in the published game. Different compression formats will give different compression ratios in exchange for less fidelity with the original audio, or higher decompression times, and all this varies a lot based on the audio patterns and length. We have three compression formats:
Which one should you use? Again, that depends on the features of your audio. Short smooth audio can use PCM, while long noisy audio can use ADPCM; the artifacts introduced by this format will be hidden in the audio itself. Maybe long smooth audio where compression artifacts are noticeable could benefit from using Vorbis. Sometimes, it's just a matter of trial and error. Maybe use Vorbis by default and when performance is reduced, try to switch to ADPCM, and if that causes glitches, just switch to PCM. Of course, the problem here is being sure that the audio processing is really what's responsible for the performance issues – maybe switching all audio to ADPCM and checking whether that made a difference is a good way to detect that, but a better approach would be to use the Profiler, a performance measurement tool that we will see later in this book.
We have other settings, such as Sample Rate Setting, that again, with a little trial and error, you can use to detect the best setting.
I have set up the audio that I downloaded from the Asset Store as you can see in the following screenshots. The first one shows how I set up the music and ambient audio files (large files):
Should be stereo (Force To Mono unchecked), use Streaming Load Type because they are large and will have just one instance playing and ADPCM Compression Format because Vorbis didn't result in a huge size difference.
This second screenshot shows how I set up the SFX files (small files):
Will be a 3D sound, so Force To Mono should be checked.. Will be short, so Decompress on Load Load Type works better. Vorbis Compression Format reduced the ADPCM size by more than a half
Now that we have our pieces of audio properly configured, we can start to use them in our scene.
We can just drag our bits of audio into our scene to start using it, but we can dig a little bit further to explore the best ways to configure them to each possible scenario.
In this section, we will examine the following audio integration concepts:
Let's start exploring AudioSources, objects that are in charge of audio playback.
AudioSources are components that can be attached to GameObjects. They are responsible for emitting sound in our game based on AudioClips, which would be the audio assets we downloaded previously. It's important to differentiate an AudioClip from an AudioSource: we can have a single - explosion AudioClip, but lots of AudioSources playing it, simulating several explosions. In this way, an AudioSource can be seen as an instance of an AudioClip.
The simplest way to create an AudioSource is to pick an AudioClip (an audio asset) and drag it to the Hierarchy window. Try to avoid dragging the audio into an existing object; instead, drag it between objects, so Unity will create a new object with the AudioSource instead of adding it to an existing object (sometimes, you want an existing object to have the AudioSource, but let's keep things simple for now):
The following screenshot shows the AudioSource generated by dragging the music asset to the scene. You can see that the AudioClip field has a reference to the dragged audio:
As you can see, the AudioSource has several settings, so let's review the common ones in the following list:
In the case of our music track, I have configured it as shown in the following screenshot. You can drag the ambient rain sound to add it to the scene and use the same settings as these because we want the same ambient effect in all our scenes. In complex scenes, though, you can have different 3D ambient sounds scattered all over the scene to change the sound according to the current environment:
Now, you can drag the shooting effect and configure it as shown in the following screenshot. As you can see, the audio, in this case, won't loop because we want the shooting effect to play once per bullet. Remember that, in our case, the bullet will be a prefab that will spawn each time we press the shoot key, so each bullet will have its own AudioSource that will play when the bullet is created. Also, the bullet is set to a 3D Spatial Blend, meaning that the effect will be transmitted through different speakers based on the position of the Audio Source relative to the position of the camera:
Something to consider in the case of 3D sounds is the Volume Rolloff setting, which is inside the 3D sound settings section. This setting controls how the volume decays over distance to the camera. By default, you can see that this setting is set to Logarithmic Rolloff, the way real-life sound works, but sometimes you don't want real-life sound decay, because sounds in real life are usually heard slightly even if the source is very far away.
One option is to switch to Linear Rolloff and configure the exact maximum distance with the Max Distance setting:
Now that we can configure individual pieces of audio, let's see how to apply effects to groups of audio instances using an Audio Mixer.
We will have several audio instances playing all over our game: the footsteps of characters, shooting, bonfires, explosions, rain, and so on. Controlling exactly which sounds are supposed to sound louder or lower depending on the case and applying effects to reinforce certain situations, such as being stunned due to a nearby explosion, is called audio mixing – the process of mixing several sounds together in a cohesive and controlled way.
In Unity, we can create an Audio Mixer, an asset that we can use to define groups of sounds. All changes to a group will affect all sounds inside it, by raising or lowering the volume, perhaps, or by applying an effect. You can have SFX and music groups to control sounds separately – as an example, you could lower the SFX volume while in the Pause menu but not the music volume. Also, groups are organized in a hierarchy, where a group can also contain other groups, so a change in a group will also apply changes to its sub-groups. As a matter of fact, every group you create will always be a child group of the master group, the group that will affect every single sound in the game (that uses that mixer).
Let's create a mixer with SFX and music groups:
Now that we have separated our sounds into groups, we can start adjusting the groups' settings. But, before doing that, we need to take into account the fact that we won't want the same settings all the time, as in the previously mentioned pause menu case, where the SFX volume should be lower. To handle those scenarios, we can create snapshots, which are presets of our mixer that can be activated via scripting during our game. We will deal with the scripting steps in Part 3 of this book, but we can create a normal snapshot for the in-game settings and a pause snapshot for the pause menu settings.
If you check the Snapshots list, you will see that a single snapshot has already been created – that can be our normal snapshot. So, let's create a pause snapshot by doing the following:
As you can see, one of the main uses of the mixer is to control group volume, especially when you see that the intensity of a group's volume is exceeding the 0 mark, indicating that the group is too loud. Anyway, there are other uses for the mixer, such as applying effects. If you've played any war game, you will have noticed that whenever a bomb explodes nearby, you hear the sound differently for a moment, as if the sound were located in another room. That can be accomplished using an effect called Low Pass, which blocks high-frequency sounds, and that's exactly what happens with our ears in those scenarios: the stress of the high-volume sound generated by an explosion irritates our ears, making them less sensitive to high frequencies for a while.
We can add effects to any channel and configure them according to the current snapshot, just as we did for the volume, by doing the following:
Aside from the Low Pass filter, you can apply several other filters, such as Echo, to create an almost dreamy effect, or a combination of Send, Receive, and Duck to make a group lower its volume based on the intensity of another group (for instance, you may want to lower SFX volume when dialog is happening). I invite you to try those and other effects and check the results to identify potential uses.
In this chapter, we discussed how to import and integrate sounds, considering the memory impact of them and applying effects to generate different scenarios. Sound is a big part of achieving the desired game experience, so take the proper amount of time to get it right.
Now that we have covered almost all of the vital aesthetic aspects in our game, let's create another form of visual communication, the user interface.