Enhancing The Player Experience with Dynamic Music

(2020), https://www.gamespot.com/games/no-straight-roads/images/

Most video games nowadays are equipped with fitting soundtracks to set the mood. Background music is an extremely easy way to enhance a game’s general atmosphere (its “vibes”, if you will). Some games also use music as a tool to supplement a narrative, like with sad violins or happy trumpets. However, game developers can only do so much by simply adding music over the gameplay. So, creators always come up with new ways to push their gameplay to the next level and make their music add to the player experience. One very common way of doing this is through dynamic music, music that changes in response to something happening in the game. This sounds cool in conversation, but how can you actually implement this in your video games?

I want to present some tricks to maximizing the positive impact your background music has on the player’s experience in your game. I have implemented dynamic music in each of my Game Jam projects over the last year, and I have learned a couple of best practices when it comes to designing such systems. I will be work-shopping this in the Unity game engine, and programming in C#; I chose these tools because they are both commonly used for Game Jams and many other independent, short-term projects. Regardless, I hope that these strategies can be applied to any game project made with any tools.

I am going to discuss the following strategies:

• Effectively looping background music, without popping, clicking, or sacrificing quality.
• Switching between parallel versions of one music track.

Unity Audio Primer

An audio source is a component that can be added to any Unity object. Each audio source can play the audio clip that is assigned to it, which can be any sound file you have imported into your project.

Music Looping

While clean looping of background music does not directly improve the player experience, lazy looping can certainly detract from it. In Unity, the built-in method of looping music simply stops the song at the end and replays it from the beginning. This may cause a “pop” to be heard at the point where this happens — if it doesn’t, the trailing sounds from the end of the song will still be cut short. To solve this, a developer could include a pause between the two iterations of the song to allow the previous one to finish. An example of this can be seen in Crash Bandicoot: The Wrath of Cortex (2001); frankly, it is ugly and distracting.

Instead, I propose a music-looping system that blends the two iterations of the song together — the one that is finishing, and the one that plays post-loop — so that the point where the loop occurs is nearly imperceptible.

In Unity, this system should be a component with the following functionality:

• Play — start playing the song from the beginning, and loop it when it gets to the end.
• Stop — stop playing the song.
• Set volume — change the volume of the song.

To help organize the code I am about to write, I prefer to express these features within an abstract class called ALoopedBGM (the A stands for abstract!):

Because this abstract class inherits MonoBehaviour(line 8), any class that extends this class will be attachable as a component to a Unity game object. So, I will create a class that extends this one, called LoopedBGM (no A - not abstract). I am including the following global class variables at the top:

This is a lot, and it will be clear what each variable is for eventually. Here is the summary of what will happen in this component:

• First, currentSource and nextSource will be created, attached as components to the game object this is attached to, and given songToPlay to play. Each AudioSource can play one music iteration at a time, so we will need two sources to blend two iterations together.
• When the song is played, currentSource will play at full volume, while nextSourceVolume will not play, and be muted.
• When currentSource has played the song for at least loopSeconds seconds, nextSourceVolume will begin playing its own iteration of the music. Its volume will be increased gradually until it is at full volume, whereas curentSource's volume will be decreased gradually until it is muted.
• currentSource will stop playing, and nextSource will continue to play the new iteration of the music. To reflect this, currentSource will be set to refer to the new source, and nextSource will be set to refer to the source that just finished.
• Throughout all of this, the values of curentSourceVolume and nextSourceVolume will be put in proportion to masterVolume.

This is ALSO quite a lot! To make it a little easier, it seems that our music player has three distinct states: when it is stopped, when it is playing normally, and when it is transitioning between iterations. I will express this using an enumerator:

For those who are unfamiliar, an enumator (enum) is data that acts almost like a traffic light. There is a set of options (green, yellow, and red) that do nothing by themselves, but can direct our code to determine what it should do next. A red light does not physically stop your vehicle, but when you see a red light, you know that that's what you should do! In the same way, setting currentState to "Stopped" will not stop our music, but we know that our music should be stopped.

So can we compress that complex bullet-point “summary” into just three different actions? Yes!

• When the player is Stopped, don’t do anything.
• When the player is Playing, check to see if the amount of seconds loopSeconds has passed, and begin the looping (transitioning) process if it has.
• When the player is Transitioning, fade nextSource in and currentSource out. When nextSource is at full volume, and when currentSource is muted, stop currentSource, swap it with nextSource, and continue playing normally.

I will express this like so:

It is good practice to include a default case, just in case the enumerator is in a state that we have not yet handled.

Note that I update the volumes of the sources regardless of the state of the player. In order to scale the volume values with masterVolume, I am changing the volumes indirectly with currentSourceVolume and nextSourceVolume rather than directly changing the volume. This ensures that we only directly deal with volumes between 0 and 1.

PlayingUpdate and TransitioningUpdate define the behaviour detailed previously:

Not very simple!

TransitioningUpdate is a little complex. However, line does exactly what I described as our actions for when the player is transitioning, in the order that I described it. The one notable thing is how I change the volume values. Modifying currentSourceVolume and nextSourceVolume does not actually modify the volumes of the audio sources. But, with SetSourceVolumes, these values are used to update these volumes at the end of every frame.

That completes our looping functionality! We need to implement the rest of our abstract class, too:

They each pretty much do what is advertised.

To actually try this out in a Unity scene, the code needs to use the Play method we defined. This can be done however you like - I personally made another C# script that calls that method at the start of the scene (see the full code at the end).

When testing this, find a good timestamp within your song at which you will begin the looping process. For best results, your music should be designed with this in mind! The best spot to put this is either right when the instruments stop playing at the end, or when the music at the end is identical to the music at the beginning. This could take some experimentation.

Dynamic Music

Dynamic music can be implemented in a variety of ways, and the mechanics of any system depends largely on the game it is being made for. For example, in No Straight Roads (2020), each piece of BGM for the boss fights has two versions, a rock version and an EDM version, and the music will switch between either version or a mix of the two depending on how well the player is doing. Dynamic music can be achieved without having multiple versions of a song, too; in almost every iteration of MarioKart, the lightning weapon will distort the music, and reaching the final lap will speed it up. Effects similar to the latter can often be achieved without writing code or designing systems; I will focus on creating a system to blend between any amount of music tracks.

For clarity, I am defining “music with different versions” as a set of songs with identical length, tempo, etc., but with other artistic differences between them (I will include examples).

This music player will have all of the functionality of the previous player (Play, Stop, and Set Volume). It will also have functionality to switch to a certain version of the music on-demand. The abstract class, AManyLoopedBGM, will look like this:

Note the following:

• This class inherits from ALoopedBGM, since it also has functionality to play, stop, and set the volume. All this means is that although it is not visible here, I will need to implement this later.
• I also have added a SetSong method to the ALoopedBGM abstract class, and have also implemented this in LoopedBGM. This is because each version of the music in this dynamic music system will use a LoopedBGM to output cleanly looped audio! So, I needed a way to set the audio clip and loop time via code.
• Finally, since we do NOT need a SetSong in AManyLoopedBGM, I overrided that method so that it simply does nothing. I will not need to implement this later.

Here are the updates made to the classes earlier:

So, I can use this template to make an implementation called ManyLoopedBGM. I can use a similar strategy that I used for the looper. However, since I am not concerned about the looping anymore, I do not need anything resembling the enumerator from earlier! This is a summary of what I will be doing:

• At the start, create a LoopedBGM for every verison of the music. Set each looper to its respective song. Keep track of which individual looper is currently active. Play and stop every looper simultaneously.
• If a given looper is active, its volume should be at maximum.
• If a given looper is not active, it should be muted.

That’s it! The set of variables I am using is very similar to before:

Each volume value in sourceVolumes corresponds to a looper in sources, and the volumes of each are set in a manner identical to how I did it in LoopedBGM.

So, first I prepare the collections of volumes and loopers:

Setting the volume of the loopers is not dependant on whether or not the music is playing. So, it is acceptable to fade each version in and out at all times. I do this here:

Exactly how I described it previously!

And finally, I implement the other features of this player:

And that should do it! Similar to the single-song looper, the Play method needs to be used in the code in order to test this.

That’s It!

The biggest benefit to programming all of these systems yourself is that you are fully in control of what happens. For example, when you loop your songs, you may want the new iteration to begin at full volume rather than being faded in. Or, you may want to allow multiple versions of a song to be played at once, rather than one at a time. I am including the complete code that was put together here, and I encourage you to experiment with it and find a dynamic music system that interests you!

Resources:

Example code: https://github.com/DThaiPome/dynamic-music-example

Dynamic music example from No Straight Roads:

Version 1: https://www.youtube.com/watch?v=UNRJ1KH0uf4

Version 2: https://www.youtube.com/watch?v=4TMEEfo9BXc