Category Archives: Research in Game Music

Research in Game Music: Silence

While working on the second installment of Research in Game Music:  Writing effective and non-repetitive game music (Part 1 here!), I stumbled upon an article from highlighting a 2007 Stanford University School of Medicine study about music.  The study (and I’m paraphrasing a bit here) basically wanted to scan people’s brains while listening to music.  It would appear “that music engages the areas of the brain involved with paying attention, making predictions and updating the event in memory.” (SD)  I mean, that makes sense.  We all get a bit mesmerized by music from time to time.  As we sit and listen, we may attempt to figure out where the music may be “going” (tonally, structure-wise, etc).  This prediction clearly requires brain power and focus.

However, what this study points out, and I think is worth further exploration, is this:  “Peak brain activity occurred during a short period of silence between musical movements – when seemingly nothing was happening.”  (DC)  Uh, wow.  The SILENCE had a profound effect, in fact, the PEAK of our listening prediction/brain activity occurs during the silence.

I also stumbled upon another article by Marcel Cobussen, a music professor at Leiden University in The Netherlands.  If you check out his website,, you can check out his entire dissertation, online: “Deconstruction in Music“.  It reads like a subway map and is full of small articles that branch off the main topic.  There were couple sections on John Cage and Silence, other composers and Silence, etc.  Out of all this, though, the article that interested me the most was, “Silence and/in Music“.  An expanded excerpt for you:

This brief and incomplete summary immediately shows the heterogeneity of silence. Silence and silence do not necessarily match. For that reason alone, silence deserves more attention. As Martin Zenck concludes in ‘Dal niente – Vom Verlöschen der Musik’ [On the Extinguishing of Music], however, the attention to silence is a peripheral moment in composition and music analysis. By no means is its status equal to sound (cf. Zenck, p.15). The pause in music, identified as an absence of sound, is the exception to the rule that has music designated as the center of the musical spectrum. Eduard Hanslick’s famous definition of music as ‘die tönend bewegte Form’ [‘form propelled by sound’] in no way indicates a music that is present in its absence, in non-sound. Sound and silence relate to each other as the essential versus the supplement, as the primary versus the secondary. It seems that not the tritonus (the augmented fourth), but rather silence is the true ‘diabolus in musica’ in Western music. Contrary to the tritonus, silence was never banned, but its raison d’etre has been thoroughly questioned up until the 20th century. Its function was mainly dramatic or rhetoric. Silence is subordinate to sound, and has for the longest time (still?) been regarded as something less significant.

Music, for the most part, has been either ON or OFF for many hundreds of years.  Notationally, how do we show “silence”?   In composition, we document silence during a piece with a rest.  In Famitracker, we simply leave it blank. It is, to us, a simple ABSENCE of sound and therefore, not AS valid.   Cobussen would agree (begrudgingly) that we do, in fact, treat silence as a second class citizen.

So how does this relate to the Stanford article?  If silence/anticipation stimulates the listeners’ brains, how can we effectively deploy these tactics in our own music?  Is is possible to write a piece that upgrades silence to first class and gives it a martini?

My brain really started to wrap around this after really digging into some chiptune tracks I’ve been working on for the past couple weeks.  With digital music, I feel like it’s even easier to convey this juxtaposition of silence and sound.  Built-in to digital sound is the ability to create both with the flick of a switch or code.  Most chiptune composers already employ this.  Imagine chiptune tracks without space, with all the channels activated and meandering.  There needs to be space (IE mistakes that all beginning composers make- “I have 7 tonal channels, I gotta use them ALL, ALL THE TIME.”  No.)

Face it.  We all love SOUND.  We all STRIVE to have wonderful sounds.  C’mon, sound is where it’s at.   LISTEN TO THIS GREAT NEW SYNTH I MADE.  DUDE, YOUR WAVE CHANNEL IS BALLIN’.  etc. etc. etc.  Still, our study shows that silence can be just powerful… and we don’t need cool wave table effects to create silence.

So where am I going with this?  Video games, much like symphonies and other predetermined and measured pieces of interactive art, find ways to employ silence as well.  In a video game, where there is already a ton of sensory overload via lights and colors and sound, does silence have an impact on us as players?  Can silence HELP our immersion and attention when playing games?

Example of Silence in Games #1:  LOADING…. LOADING…. LOADING….

Andrew High and I would agree (though we didn’t here) that loading screens definitely ruin our immersion while playing games.  Many of the older games had completely silent loading screens.  Hell, loading screens sunk entire consoles (I’m looking at you, Sega Saturn!)

Here’s a well known example of a loading screen:

If you’ve played any of the games in the Madden, you know exactly what I’m talking about.  The average delay from the game options screens (Franchise Screen, Online Screen, etc) to the actual game was usually enough for me to go downstairs, grab a beverage and some food, and come back to the screen, with time still remaining on the load.  Pretty darn awful.  On the loading screen, the game usually plays whatever music is cued up and then fades out slowly when the game is about to load.  Once it goes silent, you pretty much know the game is about to begin.  It’s a cue that we pick up on.  The silence SAYS something to us.  Interesting.

I mean, this is just one example of how the music and load screen typically line up.  A lot of the new games have music that just plays over the load screen or through it… and you’re going to need that little bit of music if you’re playing Skyim on PS3, for sure:

That’s pretty infuriating.  But at least it’s not complete silence.  Entering a new area cues up new music too, so that itself is a reward haha.

Example #2:  HARVEST MOON:  THE HERO OF LEAF VALLEY – y u no load music gud k?

Perhaps a personal example as I’m not sure how many of you guys played this but Harvest Moon:  The Hero of Leaf Valley for PSP has some major issues with the music.  I mean, the music itself is kind of nice and sweet and what you’d expect from an RPG about farming… but I’m more concerned with what happens when you change areas:

You wanna talk about “losing immersion”?  This game is a nightmare.  Every time you zone, the music cuts out.  It’s interesting though, because the music doesn’t restart… it picks back up where it left off… which makes it stutter along.  It totally drives me crazy.  I don’t understand what they were thinking.

Still, oddly, I sometimes find myself humming the next bar to the phrase during the silence.  I want my sound!  Where is my sound?  The delay creates anticipation.

Example #3:  Cue the death music!

Perfect example, again, is Super Mario Bros.  Let’s highlight this in an amusing fashion:

Dying plays the dying music… and then what happens?  There’s silence, followed by a load screen that displays how many lives remaining.  That’s your musical prep – your silence between movements.  It gives you ample time to think about what you’re going to do differently.  It’s not just Super Mario Bros too.  There are plenty of other games that have screens that give you a second to think about it after dying.

Example #4:  Dead Space IE I know there’s necromorphs here, I just can’t see them yet!

The Dead Space series employs silence to add to the drama of the game.  Nothing is more frightening than walking into a large open room with vents.  Clearly, something is going to happen.  Is it now?  How about… now!?  Let me walk a little… OH GOD NO!  MY FACE! Once a necromorph appears, the music rushes in and attempts to startle the player.  For example:

A very loose conclusion:

Obviously, there’s a lot to study about this.  My intention here was not to go into a 3000+ word paper on this but to point out some moments where silence could have some kind of affect on us.  Any thoughts out there?  Let me know and thanks for reading, as always!

Research in Game Music- A reaction to Andrew High’s Gamasutra article: “Is Game Music All It Can Be?”

Perhaps adding to what I wrote about a couple weeks ago, Gamasutra’s Andrew High writes about how the game music industry could do well to learn from film scoring.  It is an interesting read, for sure.  However, I really do not agree with most of his points and criticisms of the game industry and thus, this research article is a response.  Enjoy!

Games and films are different and must be treated as such: a reaction to Andrew High’s “Is Game Music All It Can Be?”

High states that he wants composers to approach game music with more subtlety and really think about how music interacts with how the game works.  As far as I’m concerned, this has always been a bit of how game music works.  To state that we need “film score study” to really construct music for video games is an over-simplification:  what works for a game doesn’t necessarily work for a film and vice versa.  As far as I’m concerned, game music construction is a completely different genre altogether and should be treated as such.  There is an interactive element that is present in gaming that is not present in film and while film score study would surely help in game music construction, it should not dictate the exact methods of compositions.   Here’s a snippet of what High stated:

“Video games typically have long segments of player-controlled action that require atmospheric background music on endless loop. This is certainly something that’s unique to video games, and presents its own difficulties. The composer must create audio that (a) tells the story of the area or level, (b) has enough interest to avoid fading out of the player’s notice entirely, (c) can be listened to ad nauseum without driving the player insane, and (d) stays cohesive with the rest of the music in the game. Game composers in general seem not to struggle too much with B, but A, C, and D can present serious hurdles.”

First, in order to respond to this, we’ll need to do a bit of comparison between film elements and game elements so that we have this clearly defined.

In a film, the director sets up and records a complete tapestry of scenes to be displayed for viewing.  The composer’s responsibility is to create a film score, usually through-composed, that expresses how the actors and the scenes are displayed.  These scenes play back without any variation in a set order every single time the movie is queued up.  A director would say:  Okay, I need this melody to linger a bit and then become dark because the main character is about to try to commit suicide.  Hans Zimmer or whoever would then make sure that the music, which repeats in the same order every single time, lines up with the scene.  The finished result is music that is interwoven into static viewership.  It occurs at the same time, every time.

High and I agree that video games have an interactive element that makes them different from movies.  While movies and games are essentially completed pieces of art that we view and observe, only video games allow the user to make the decisions for content.  For example, in Super Mario Bros., there are no pre-set deaths.  Death occurs ONLY when the player makes a mistake.  This, as I’ve said before, stops the current music and cues the death music.  We can assume Koji Kondo, much like a film score composer, would be told how to score the music:  When Mario dies, we want a jingle to play.  The difference is that Kondo would NOT be told when and where this music would be used- only that it needs to indicate to the player that they have died.  Kondo would write a small jingle, interrupting in nature, that indicates to the player that they’ve died.  This HAS to be abrupt and interrupt the flow of gameplay.  Subtlety MUST be ignored and this game “event” must NOT be seamless.

We should know that we suck at Super Mario Bros. The death music must play, even if it interrupts our “immersion”.

With Super Mario Bros‘s average levels being less than 30 seconds long and the music looping in varying orders depending on levels, explain to me why Super Mario Bros. is littered with boring, cliché, and just plain bad level music.”  It isn’t.  And did Koji Kondo sit there and say, “Oh yes, this is the sunny outside level so let’s make it sunny and outside.  And here’s the DARK UNDERGROUND CAVE.  I need music that captures this mood.”  I can assure you that while he did think of the game construction, he didn’t think that anyone would listen to the music outside the game and judge it on it’s ability to convey emotion or the scenes…

High continues to bolster this argument with his example from Nier.  Video game music, much like film music, is completely dependent on the entire art form.  That is, to someone who has seen a film, the score should invoke the images of the film.  Examples from Nier in which the interpretation of the music lines up with the gameplay is most likely based on the bias of actually playing the game.  He also uses Persona 4‘s music to show how a game may not have cohesion… yet the music describes to him the entire scene.  The music reminds him about what happened in the game and that’s the goal of game music.

Ask someone to listen to John Williams’ Star Wars for the first time and see what they think the movie is about.  Do they get action and space?  Do they get “a plot about Rebels or Imperial foes?”  The only reason we know how to interpret this music is our own visual recall of the opening scenes.  Unless music has a story behind it that is completely defined and mentioned to the audience before listening (program music), the audience is completely free to interpret what they hear.  The practice of saying that the music must convey an element on its own really only applies to music outside the realm of film and video games.  It is a primary consideration when composing, don’t get me wrong, but it is not the reason why some music is great and some music is awful.  The complete package (audio + visual) ultimately enhances our lasting interpretation.

A lot of High’s article focuses squarely on games that have cinematic elements where the player sits back, watches, and cannot interact.  This this is relatively new to the game industry.  In 1985, Koji Kondo was not required to write a :45 second through-composed track that plays when, say, Bowser steals the Princess and Mario angrily shakes his fist.  Older games had these stories printed in the instruction manuals or during introductions.  Today, however, a game music composer must write music that is both dynamic AND static.  The dynamic music would be for levels and interactive gameplay.  The static music would be for such cutscenes mentioned above.  This has been going on for some time, not just in the 2000s:

Composer Nobuo Uematsu and the game developers line up the music perfectly to create this cutscene… and this occurred way before the Halo series or today’s modern games.  This may look primitive by comparison, but it has all the elements that make it great from a film perspective.  Furthermore, if High wants an example of “hard transitions and visual beats“, he only look above.

We can also go back a lot further than this if we want to show Uematsu again successfully creating an cutscene with even less:

While this is only the Super Nintendo, the technique is purposeful.  Notice how the first flash of light and the first organ sound are perfectly synchronized.  The action and the music are intertwined.  Uematsu was clearly asked to produce something that reminds us of the backstory to Final Fantasy VI and while there’s no “speech” or “voice” here, we get the story and the atmosphere, dare I say, more effectively than a lot of today’s titles.  If High was looking for simplicity, he only needs to quote these older games where simplicity was king.  There’s a few perfectly lined up “beats” here and it’s not overwhelming.  It’s both atmospheric and tonal.  It’s exactly what he was looking for… and it’s 15 years older than the games he complained about.

The last point I’ll make is his article values the skills of film composers while pushing down the accomplishments of game music composers.  There are hundreds, if not thousands, of effective scores to games that do NOT fall into the criteria that High discusses.  He also completely ignores any game before about 2005 in his analysis, even citing his own article where he states that Final Fantasy XIII was the first game to use leitmotif.  Not true, by a long margin.  Leitmotif, even OPERATIC SEQUENCE, was used at least as early as Final Fantasy VI.  Each character had their own theme that appeared when the character appeared, each scene had hints of those themes, and the construction of the game was completely intertwined with those themes in mind.

In the end, this articles confuses me.  It says that it loves video game music but also pushes video game music down.  It cites examples of video game music from last 8 years and compares them to films with bigger budgets and legendary scores across the entire 120~ year history of film.  It fails to recognize why video game music is different than film score and that’s what is so troubling.  For video game music to continue as an academic study, we MUST separate it from other types of music.  We wouldn’t talk about counterpoint in an opera class, we wouldn’t discuss sonata-allegro form in a class on Hildegard von Bingen, we wouldn’t talk about hip hop in a rock music class- we cannot compare apples to oranges and ultimately, that’s what this article does.

Comments welcome!  Thanks for reading.

Research in Game Music: Technique- Writing effective and non-repetitive game music, Part 1

In this 6 part series, I will explore the common techniques used by game music composers to write effective, pleasing, and ultimately non-repetitive game music.  Each part will highlight one specific technique utilized by composers and provide numerous examples.

Part 1:  Introduction / Cue soundtrack technique (CST)

Part 2:  Story Soundtrack Technique (SST)

Part 3:  Utilizing track length

Part 4:  Variation by sheer force

Part 5:  Atmospheric and minimalist soundtracks

Part 6:  User choice

An introduction of sorts:

When I was much younger, I used to play video games on a spare TV that I would wheel out of my parents’ room and into my room.  My mom, who is not a video gamer (though she plays Dr. Mario and Luxor frequently), would always get annoyed by the music coming from the games I played and question me.  “How the heck can you keep listening to this?  It just keeps repeating over and over.  [sings the Final Fantasy VI‘s “Battle Theme”]  It’s soooooo annoying!”  My mom couldn’t understand the context of the game because she wasn’t playing, so her attribution of the game’s music being “annoying” is more that it’s “making noise”, rather than the music ITSELF is annoying.

But mom~ I’m fighting Brachosaurs and I need the music to be loud enough for everyone to enjoy!

To me, though, as the player, this music made sense.  The “Battle Theme” was for when I battled monsters,  the “World Map Theme” was for when I wandered the world, and the “Sleep Theme” was for when my characters stayed overnight at an Inn.  The music was interactive and innovative.  It was dynamic!  It changed how I changed.  Even though the music itself was no longer than 50 seconds of actual varied material at any given time, I never found myself bored or annoyed.  Games with good soundtracks got stuck in my brain and games that had awful soundtracks, well, I didn’t play them often or at all.

What is often forgotten is that game music is static (with a few exceptions).  When I load up Final Fantasy VI, the music programmed to the ROM on the cartridge loads up.  This music cannot be changed and plays at set intervals.

A picture of the 3-disc FFVI OSV

In most cases, a game’s total gameplay hours is longer than its complete soundtrack.  For example, Final Fantasy VI‘s gameplay is about 35-40 hours long (shorter if you’ve played it many times).  The complete Nobuo Uematsu soundtrack to the game is 90 minutes long.  If we are to play a game for over 35 hours and we only have roughly 1.5 hours of material from which to draw, how can we keep the music fresh?  I guess the big question here is:  how can a composer, using limited resources, under time constraints, create a static set of music that is so effective that we do not notice its repetitive nature?  How can the music augment the experience of the gameplay without ultimately becoming annoying?

I’ve spent a great deal of time thinking about how a game interacts with its music and vice versa  I broke that down and explored our own experiences as gamers.  The result is a list of six different categories of game music variation.  Each of these categories describes a technique utilized by a game to make a static, unchanging amount of music turn into music that pleases the audience for the entire duration of gameplay.  Today we’ll talk cue soundtrack technique.

Continue reading

Research in Game Music: The difference between pulse waves and square waves

So, a couple of the readers of this blog asked me (in lieu of my current 2 part series on Dr. Mario across the NES and Game Boy) to explain the difference between square waves and pulse waves.  Let’s see if I can shed some light on how this works.

Continue reading

Research in Game Music: Complete NES Audio Mapper List

I recently stumbled upon a mostly complete list of all the mappers used to create NES/Fami video/audio playback.  If you recall, I spoke about the Lost Sound Expansion Chips of the the NES a while back.  Essentially, the US NES lacked a connection pin that allowed 3rd party software companies to provide sound modules that allowed for extra sound channels and voices.  A real shame.

At any rate, I found a list from that essentially labels and spells out what each of your favorite games used for an audio mapper.  While the author,, states that this list is far from complete, it is an invaluable resource in getting a general idea in the exact formats of the audio playbacks and the added size/features of each game.  Definitely worth checking out.

Bigass NES Mapper List ver. 0.1

Some thoughts:

  • Many US NES games use the MMC1 or MMC3.  As in, most of them.  The MMC1 to allow Nintendo cartridges to save via battery backup.  The MMC3 added the ability for a screen to scroll while leaving a status bar motionless at the top of the bottom.  Hence, you can see why this was used for Super Mario 3 and the Mega Man series.  As you can see, there are some games with MMC1 that clearly had the capability to save and did not offer it- most notably Mega Man 2.  That would have been awesome.
  • This list, as the author admits, lacks many of the JP Famicom mappers.  That a shame, to be honest.  However, his reasoning is sound.  It’s tough enough for me to just get my hands on the most basic Famicom games.
  • That even being said, searching “VRC” or any of the other 3rd party chips shows some of the games for which we were not provided the complete fidelity of the music.

At any rate, enjoy the list!