Last night I attended the San Francisco Conservatory of Music’s “Game On!” conference. I believe one goal of SFCM in hosting this evening event was to solicit interest in a new music-in-media program which they are launching in Fall 2015. This program includes instruction geared to preparing the composer to work with emerging technologies and media. Applications for Fall 2015 are now being accepted, but the deadline to apply is coming up in just a few weeks.
The evening included three sections. First was a “Demo Derby” where a panel reviewed and critiqued videos of game scores by members of the audience (many but not all being current SFMC composition students). The second part was a presentation by three industry composers of what they considered to be inspiring game scores. The third part of the evening was a concert by a string quartet comprised of selections of video game music.
The mainstream game companies represented by the composers included Sony, Valve, Pyramind and there was an impressive list of game titles from companies like Lucas and Disney in the credits.
The scoring examples submitted for the “Demo Derby” tended to emphasize the more cinematographic side of game music. Composers mostly selected video clips of popular games (Final Fantasy, Lord of the Rings, are two that come to mind) and presented scores making use of orchestral sampler technology, with three exceptions: a solo piano score, a string quartet score, and an all-synthesized score. The orchestral works tended to be anthemic and rousing, with various degrees of compositional interest and sophistication.
In general, the panel response was positive, especially about the quality of the music and its fittingness to the general mood of the selection, and the “color” of the instrumentation and its sensitivity to the color of the video. We also got to hear some notes about some common concerns that arise. One was the need to maintain the mood of the scene. One score to a battle scene had high energy and an exciting and effective use of the snare. However, the music introduced a “B” theme that contrasted with a slightly lower energy level. This is a common compositional device, but as the big battle was still ongoing, the change in mood was premature.
Another common critique involved the overuse of a single key, tempo, or of an ostinato or thematic material or texture. In particular, with scene shift or energy shifts, the panel recommended making a corresponding change in the music. It is all to tempting to “play through” these emotional shifts, rather than to focus and highlight them as they should be, as these are key moments in the game play. There were also some notes about integration with sound effects, leaving aural space for them by using instrumentation that emphasized different frequency ranges.
The last presentation was a Max/MSP program that had been rigged to work with hardware that tracked hand movments and tilting. The video showed the composer and friend combatting with “swords” that were rigged to respond to the strokes, with music written to change based on the nature of the movements. This elicited some strong interest from members of the panel, even though I was having considerable trouble figuring out what gestures were leading to what musical changes. (I asked the composer about the viability of using Max/MSP or PD in a smaller scale indy game. He said that he had heard that several larger game companies were using proprietary versions of Max/MSP, and that his target was to work in that sort of situation, and not so much with the smaller games.)
In the next segment, a slightly different group of panelists presented scores which they found to be personally “inspiring.” These scores contrasted considerably with the Demo Derby submissions. Restraint was much in evidence. Instead of thematic/melodic material, we were given cues that simply heightened the sense of fear in one case, or suggested the ‘open plains’ in another case with evocations of classic spaghetti western cues. (My apologies for not writing down, and then forgetting the titles of the games!) The treatment in each case was much more akin to sound design, highly integrated with the texture and mood.
Particularly striking was a very violent “stealth” game. In it, the panelist spoke a bit about two new trends. The first is the use of “occlusion.” By this, he was referring to the muffling of sounds that originate beyond walls or in the distance. For example, a conversation in the distance by two “enemy combatants” would sound highly filtered at first, but as the protagonist gets closer, the clarity would improve. The panelist made the point that “hidden” or obscured sounds can add considerably to tension. When you enter a new room or space, the sounds are not in your face, immediately cueing you to the prescence of the enemy. Rather, there remains a threat of lurking surprises as that which is hidden or obscured emerges.
It is my experience that a simple form of occlusion has been in use for a long while, via the use of volume to convey distance. Perhaps it is the use of filtering that is new, or maybe the filtering has gotten better recently and the practice has become much more common recently, rather than only being present in a few titles up to now.
The second concept that I recall being invoked was “the looping of sound without looping.” The example score had various thematic elements, as much sound effect as music (for example, the use of dark, tense clusters of notes, a compositional device I tend to associate with Gyorgy Ligeti–e.g., the end of Kubrik’s “2001”). The cues were programmed to play intermittently. The panelist referred to an algorithm combining a bank of possible cues (including the choice of “silence”) and a randomizing count down that spaces out the cue calls over time. In effect, each cue was potentially a loop, but instead of being looped, the playback was distributed by a certain chance element. I did find the example to be particularly impressive. It makes sense that the tense, sparse sounds would fit the “stealth” mode, but surprisingly, they were also highly effective in the moments where the player springs into action for a quick scuffle, allowing the blows and grunts of pain to be the aural focus. Had another approach been used, such as ramping up a score with high energy action music–I can’t imagine it being nearly as effective.
Along these lines, another panelist made repeated uses of the term “Cagian”. This is a reference to the radical and innovative composer John Cage, who would do things like require the use of tossing I Ching coins to determine the direction of a composition, thus making the performance a result of chance operations. I can see where it is not such a leap to go from here to writing music that responds to the different game states that could arise at diverse times or in various orderings.
The panelists spoke of various compositional devices being used currently to match game state. The two I best remember are the use of branching scores, and the use of layered scores. In the first case, the composer will write two or more continuations to a score, and use game state to determine which will be the continuation in real time. In the second case, a piece is recorded in various tracks, allowing each track to be mixed in real time. One of the panelists referred to actually recording the separate groups of the orchestra (strings, winds, brass, etc.) and having access to mixing these elements in real time. A recommendation was made to pick up a free copy of FMod or another composing tool (free for non-commercial use?) and go through the experience of writing music in these two modes. Or, as one of the panelists put it: write a score, then explode it, then put it back together.
A few moments were also spent talking about the use of sound as an element in a puzzle design, the example being from Portal II. References were made to similarities with the use of sound in Art Installations.
But not all of the ‘inspiring’ game music consisted of these spare scores. We also had a reference to “Journey” with its nicely flowing, continuous and always appropriate music, and also to the classic “Tempest”, back from when games were played in arcades and the music was LOUD, and doubled as sound effects, from synthesized bullet blasts to level-changing swooshes.
I am recalling two more bits of wisdom and advice that were shared: (1) the first thing to do (before writing a single note) in scoring it to try and get a sense of the tempo of the scene; (2) think about what music has (for example, things graphics can’t do) that can be uniquely added to the mix, working towards a greater aesthetic whole.
Audience questions or comments were not solicited at the end of this presentation.
The last segment of the evening was a performance by a string quartet. I found the music difficult to sit through, but I am an impatient sort, bored easily, and wanted to get on with the meet-and-greet. The audience in general found it engaging and warmly applauded each piece, even calling the players out for a second bow at the end.
The composer-panelists were quite accessible for questions and comments in a reception held immediately after, though several did disappear to take an impromptu tour of the SFMC studio facilities.
Personal take-aways:
One of my goals was to spread information and solicit feedback on “Hexara” and my Java Audio engine. I was able to get several emails, and am in the process of sending out links. A couple of the mainstream composers were surprised to hear that Java was fast enough to handle procedural synthesis and scoring. I also found some interest among the student composers who were willing/interested to check it out. I hope I can soon successfully make the library available to JGO game programmers. I am pleased to find that much of what I put into the library is compatible with the new trends, for example, the “windchime” [SoundField object] that I have which implements “looping without loops”. This makes me think I should make the full implementation available rather than an ultra-light version that only supports playback of clips and wavs.