Game Engine Anatomy 101 May 17, 2002 Game Engine Anatomy 101, Part VI By Jake Simpson Discuss this now (26 posts) Sound and music in games are becoming increasingly important in recent years due to both advances in the game genres people are playing (where sound is an actual game play feature, such as the aural cues in Thief and other games of that ilk), and in technology. Four speaker surround systems are now both affordable and commonplace in the gamer's arsenal. Given spatialization of sound, obstruction and occlusion of noise, and the dynamic music many games employ these days to heighten the emotional responses of the player, it's no wonder that more care is being given to this area. Right now in the PC arena, there is really only one card of choice for gamers – the Sound Blaster Live!. From old time PC sound card manufacturer Creative Labs. Creative has provided their EAX sound extensions for DirectX for a number of years now, and they are a founder of the new OpenAL (Open Audio Library) initiative. OpenAL, as it sounds, is an API for sound systems in the same way that OpenGL is a graphical API. OpenAL is designed to support a number of features that are mostly common across sound cards, and provide a software alternative if a specific hardware feature is not available. For a better definition of OpenAL, I asked Garin Hiebert of Creative Labs for a definition: "Borrowing from our "OpenAL Specification and Reference" here's a definition: OpenAL is a software interface to audio hardware, providing a programmer with the ability to produce high-quality multi-channel output. OpenAL is foremost a means to generate audio in a simulated three-dimensional environment. It is intended to be cross-platform and easy to use, resembling the OpenGL API in style and conventions. Any programmer who is already familiar with OpenGL will find OpenAL to be very familiar. The OpenAL API can easily be extended to accommodate add-on technologies. Creative Labs has already added EAX support to the API, which programmers can use to add sophisticated reverberation, occlusion, and obstruction effects to their audio environment. " Soldier of Fortune II features this new system, as does Jedi Knight II: Outcast, along with the Eagle world/sound properties editor. What is Eagle? Before we get into that, let's discuss a couple of other systems, and define some sound terms. click on image for full view Miles Sound System Another system out there is the Miles Sound System. Miles is a company that produces a plug-in to your own code that will handle all the necessary talking to specific sound cards (like the Sound Blaster Live! series, or an older A3D card for example) while getting the most of each card. It's very much like an API front end, with extra features bundled in. Miles gives you access to things like MP3 decompression amongst other things. It's a nice one-stop solution, but like everything, it costs money as well as being one extra layer between your code and the hardware. For fast sound system production though, it's very useful, and they've been around for a while, so they do know their stuff. Sound Terms Let's start with obstruction and occlusion. They sound the same, but are not. Occlusion means basically that the listener has some encompassing obstacle between them and a sound being played. click on image for full view NOLF 2 Let's say you hear bad guys inside the house in this screenshot from NOLF 2. You can hear them, but their sounds are pretty muffled and muted. Obstruction is similar, but the obstacle between you and the sound is not encompassing. A good example of this is having a pillar between you and the sound source. You are still hearing the sound due to echoes in the room, but it isn't the same as the sound coming directly to your ears. Of course this does rely on knowing what is in a direct line between your ears and the sound source. And the required processing can get pretty time consuming depending on the size of the room, the distance of the sound source from you, and so on. We will talk about traces later-- suffice to say it can often be the reason for slower frame rates. The A3D code inside of Quake III does this stuff, and frame rates can often be improved by turning these options off. Tribes 2 is another sufferer from this malady. Turn off the 3D Sound options and your frame rates instantly get better, which makes sense when you consider how big the Tribes worlds are, and how far you can see. Next are sound material properties. Most sound cards give you the capability to modify sounds being played by using definable filters to act on that sound. For instance, there is a big difference between hearing a sound underwater, or in a cloth-covered room, or in a long corridor, or an opera house. It's pretty cool to be able to change the way you hear a sound depending on the environment you are in. click on image for full view Eagle Back to Eagle… This is an editor that allows most first-person shooter map designers to import their maps into the tool, and then construct simplified geometry that creates a sound map for EAX code in the actual game engine. The idea is you don't need the complex geometry of a real graphical map to simulate sound environments. You can also assign sound materials to the resulting simplified map so sound environments can change dynamically. I was witness to a demonstration of this on Soldier of Fortune and Unreal Tournament and it really is quite striking. Hearing all the sounds change when you plunge into water is a very immersive experience. OK, let's move on. For the consoles, you are more limited in your possibilities because of static hardware – although both on the PlayStation 2 and the Xbox this hardware is pretty damn cool. When I say limited, I only mean in expansion, not in what it's capable of doing. I wouldn't be the least bit surprised to see games with Dolby Digital 5.1 output very shortly from these consoles. Xbox, with its MCP audio processor can encode any game's audio into 5.1, and the game doesn't need to be specially coded to take advantage of this feature. Dolby has brought ProLogic II to the PS2, and has teamed up with Factor 5 to enable ProLogic II for GameCube games. On Xbox, game titles Halo, Madden 2002 and Project Gotham Racing all have 5.1 Dolby Digital audio content. DTS also recently unveiled an SDK for PS2 game developers to bring a reduced bit-rate version of DTS audio to games on that platform. Positional Sound--It's a Complex World Now there are some issues with sound spatialization that not many have dealt with. I speak of putting sound in a real 3D world. Having four speakers around you is a great start, but it is still only in two dimensions. Without speakers above you and below you, you really aren't getting 3D sound. There are some sound modulation filters that attempt to address this, but really there is no substitute for the real deal. Of course most games are really only played in two dimensions for the most part anyway, so it's not such a big deal. Yet. One of the most important features of any sound system is actually the mixing of the sounds together. Once you've decided what sounds you can actually hear based on where you are located, where the sounds are in space, and what the volume is for each sound, then you have to mix the sounds. Usually the sound card itself handles this, which is the primary reason for the existence of the card in the first place. However, some engines out there decide to do a 'premix' in software first. That doesn't make much sense really, until you look at a bit of history. When sound cards first came out there where many different approaches to mixing. Some cards could mix 8 sounds together, some 16, some 32, and so on. If you always want to hear 16 possible sounds, but you don't know if the card can handle it, then you fall back to the tried and tested route-- that of doing the mixing yourself in software. This is actually how the Quake III sound system works, but begs the question: "Quake III was released into a world of A3D and Sound Blaster Live! Cards, which are more standardized than ever, so why do this?" That's a good question. The sound system for Quake III is actually almost line-for-line the same sound system that was in Quake II. And Quake I, and even Doom. When you think about it, up until A3D cards and the SB Live! Card, sound systems hadn't really changed in requirements for years. Two speakers, two dimensions, and a simple volume drop off for distance. From Doom through Quake III not much has changed. And in the games industry, if it ain't broke, don't fix it. click on image for full view Miles Sound System Usually you would just use DirectSound to do your mixing for you, since it would detect the sound hardware available, or fall back to software, much as DirectX does for 3D cards. In 90% of sound cases, a software fall back doesn't really make that much difference to your frame rate. But the Doom engine was created when Direct Sound wasn't even a glimmer in some mad coder's eye. It never got updated because it never really needed to. Of course, you could use some of the clever features of the SoundBlaster Live! Card, such as a room's echoic characteristics: a rock room, or an auditorium, a cavern, a football stadium, etc. And you really should use the mixer provided by the hardware, after all, that's what it is there for. One drawback of this approach is that often the result of the mix isn't available to the program itself, as it's being done inside the card rather than in main memory. If you need to look at the resulting volumes for any reason, you are out of luck.

We haven't talked much about music generation in games. Traditionally there are two approaches, one being a straight music .wav file (or equivalent). It's pre-made, ready to run, and a minimum of fuss. However, these are expensive in terms of memory and playback time. The second approach is to code up a MIDI track using preset samples. This is often cheaper in memory, but has the drawback of having to mix several sounds together at once, thereby using up sound channels.

Dynamic music is the ability to change your music depending on the action being witnessed in the game, such as slow music for exploration, and fast for combat. One thing that's hard to do with pre-made music is to beat-match it so you can fade from one piece of music to another, which is easy with a MIDI track. Often though, if you do the fade fast enough, or fade one down before bringing up the other, you can get away with it.

click on image for full view

Cakewalk's Sonar XL Composers use tools like Cakewalk's Sonar XL to create game music soundtracks.

By the way, before we leave this subject it's worth mentioning that companies exist that specialize in creating purpose-composed music for your games. The FatMan (www.fatman.com) is one such company. It's probably easier to contract out music than anything else, which is how they exist.

Lastly of course, the in-thing for games right now is the MP3 format, which allows massive 11-to-1 compression of sound samples, yet only takes a fraction of the CPU time to decompress them before being thrown at a sound card. When I was at Raven Software, with Star Trek Voyager: Elite Force we managed to get three complete languages on one CD by use of MP3, and still have space left over for more graphics. Mainly, we MP3 only for the non-player characters (NPC) voices, since streaming an MP3 and decompressing it on the fly for all of the game's audio effects was more than the hardware could handle, although it's a definite possibility in the future. Newer formats like AAC from Dolby and WMA from Microsoft offer equal or superior audio quality at nearly twice the compression rate (well, actually half the bit rate) of MP3, and may find their way into future game titles.

That's it for this segment. Next up will be developing for networked and online gaming environments.

参考译文：

                                                            第 6 部分 声音系统，音频APIs
声音系统
由于人们玩的游戏在种类和技术上的进步，声音和音乐近几年来在游戏中正逐渐变得重要起来（声音是一个实际游戏的可玩特点，比如在Thief和其它同类游戏中的听觉提示）。现在四声道环绕系统在游戏玩家的宝库中是负担得起的和平常的事。给定空间的声音，噪音的障碍和闭塞，和动态的音乐，如今许多游戏使用这些提高玩家情绪上的反应，更多的关注投入到这个领域就不足为奇了。



现在在PC竞技场中，游戏玩家实际上只有一种声音卡可以选择 -- PC声卡制造商创新公司（Creative Labs）的Sound Blaster Live！ 从旧的时间个人计算机声音卡片制造业者有创造力的中心. 多年来创新公司已经为DirectX提供了他们的EAX声音扩展，并且他们是发起新的OpenAL（开放音频库Open Audio Library）的创立者。就如同OpenGL是一个图形API一样，OpenAL，像它起来听一样，是一个声音系统的API。OpenAL 被设计为支持大多数通常声卡的许多特征，而且在一个特定的硬件特征不可得时提供一个软件替代。



为了更好的定义 OpenAL，我向创新公司的Garin Hiebert询问了其定义:

"这里借用我们的 " OpenAL 规格和叁考" 的一个定义:

OpenAL 是对音频硬件的一个软件接口,给程序员提供一个产生高质量多通道输出的能力。OpenAL 是在模拟的三维环境里产生声音的一种重要方法。它想要跨平台并容易使用，在风格和规范上与OpenGL相似。任何已经熟悉OpenGL的程序员将发现OpenAL非常熟悉。

OpenAL API能容易地被扩展适应插件技术.创新公司已经把EAX支持加入到这套API了，程序员可以用来给他们的声音环境增加复杂的反响，比赛和障碍效果。

如同Jedi Knight II: Outcast 一样，连同Eagle 世界/声音特征编辑器,Soldier of Fortune II 以这个新系统为特征。什么是Eagle？ 在介绍这个以前，让我们讨论一些其他的系统，并定义一些声音术语。



另外的一个系统是Miles声音系统。Miles是一家公司，它为你的代码生产插件，在充分利用每块声卡时处理所有必须的到特定声音卡的说话（比如Sound Blaster Live！系列，或者老的A3D声卡）。它非常像一个API前端，捆绑了一些额外的特征在里面。 在其他事物当中Miles让你存取一些事物像MP3解压缩。 它是很好的解决方案，但像任何事一样，它花费金钱并是你的代码和硬件之间的额外一层。虽然对於快速的声音系统制造，它非常有用，而且他们有段时间了，因此他们的确精通自己的业务。



声音术语
让我们开始障碍和闭塞。它们听起来一样，但不是这样。闭塞基本上意谓着一个声音在播放时听者在他们之间有一些闭合的障碍物。



比如说，在NOLF2的一个屏幕镜头上你听到房子里面坏蛋的声音。你能听到他们，但是他们的声音相当低沉而沙哑。障碍是相似的，但是你和声音之间的障碍物并不是闭合的。一个好的例子就是在你和声源之间有一根柱子。由于房间中的回声你仍然听得到这个声音，但是它和声音直接传递到你的耳朵里是不同的。当然这确实依赖于知道在你的耳朵和声源之间的直线上是什么。而且根据房间的大小，声源到你的距离等等，需要的处理能变得相当耗时。后面我们将会谈到跟踪--足可以说它时常是比较慢的幀速率的原因。Quake III 里面的A3D 代码做了这些事情，关闭这些选项通常能够提高幀速率。Tribe 2 是这种弊病的另外一个受害者。关闭3D声音选项则你的幀速率立即好转，这在你考虑Tribes世界有多大和你能看见多远时有意义。



接着是声音物质的特征。大部分声卡可以让你能够用可定义的过滤器作用于声音从而修正播放的声音。例如，在水下，或者在一个布料遮盖的房间中，或者在一个长的走廊中，或者在歌剧院，听到的声音有着很大的不同。能够根据你所处的环境改变你听到声音的方式是相当不错的。

我们回到Eagle… 这是一个编辑器，允许多数第一人称射击游戏地图设计者将他们的地图导入到这个工具，然后构造简化的几何形体来为实际游戏引擎中的EAX代码产生一个声音地图。其思想是你不需要一个真实的图形地图的复杂几何形体来模拟声音环境。你也能够给产生的简化地图分配声音物质，这样声音环境就能够动态地改变。我亲眼目睹了这在Soldier of Fortune和Unreal Tournament上的示范，确实相当引人注目。 我这在财富和 Unreal 巡回赛和它的军人上真的对示范是证人相当醒目. 当你跳入水中时，听到所有的声音改变，这是一个非常令人沉浸的经历。

好, 让我们继续吧。

对于游戏机，由于静态的硬件，你的各种可能性会更受限制 — 尽管在PlayStation 2和Xbox上，硬件相当不错。我说的限制，仅仅是指扩展，而不是它所能够做的。我一点也不会感到惊讶看到这些游戏机上的游戏很快支持杜比数字5.1（Dolby Digital 5.1）输出。Xbox ，由于它的 MCP 音频处理器，能够将任何游戏音频编码为5.1，并且游戏不需要特别编码就能利用这个特征。杜比（Dolby）把ProLogic II 带到了 PS2 上，并与Factor 5合作为GameCube游戏实现了ProLogic II。在 Xbox 之上，Halo, Madden 2002 和 Project Gotham Racing等游戏都有5.1杜比数字音频内容。DTS最近也为 PS2 游戏开发者发布了SDK，为这个平台上的游戏带来了降低了比特率的DTS音频版本。



位置的声音--一个复杂的世界
现在有一些很少有处理的声音空间化问题。我说的是把声音放在一个真实的3D世界中。有四个扬声器在你周围是一个很棒的开始，但这仍然只是在二维方向。在你的上方和下方没有扬声器，你没有真正获得3D声音。有一些声音调制过滤器试图解决这个问题，但实际上没有真实东西的代替物。当然真实地大多数游戏多半只是在二维方向上，因此这仍然不是太大的问题。

实际上任何声音系统最重要的特征之一是把声音混合在一起。根据你所处的位置，空间中声音的位置，每个声音的音量大小，一旦你决定了实际上你能够听到的声音，然后你必须混合这些声音。通常声音卡自己处理这些，这首先是声音卡存在的主要原因。然而，外面有一些引擎决定首先用软件做一次‘预混合’。直到你着眼于一点点历史以前，这并没有真正地带来多大的意义。



当声音卡最初问世的时候，有许多不同的混合方法。一些声卡可以混合8种声音，一些单位16种，一些32种，等等。 如果你总想听到16种可能的声音，但你不知道声音卡是否能够处理，那么你回到了尝试和试验的道路上 — 就是你自己用软件混合。这实际上是Quake III声音系统的工作方式，但提一个问题:"Quake III是为A3D和Sound Blaster Live！声卡世界发布的，这比以前更加标准化，为什么还这样做？" 这是个好问题。实际上Quake III的声音系统几乎每行代码都和Quake II中的声音系统一样。而且Quake I，甚至Doom也是这样。你想一想，向上直到 A3D 声卡和 SB Live! 声卡，许多年来声音系统的需求没有真正地改变过。两个扬声器，二维方向，音量简单地随着距离减小。从Doom一直到Quake III没有发生太大变化。而且在游戏行业中，如果不是迫不得已，别理会它。

通常你会仅仅使用DirectSound为你做声音混合，因为它会可以使用的声音硬件，或者转而依靠软件，很多地方就像DirectX为3D显示卡所做的一样。在 90% 的声音情形中，依靠软件混合对你的幀速率没有真正发生太多不同。当DirectSound在一些狂热的编码者眼中甚至还不是一丝光线时，Doom引擎就已经产生了。它从来没有得到更新过，因为它从来就没有真的需要更新。

当然，你可以使用 SoundBlaster Live！声卡的一些聪明特征，例如房间的回声特性: 一块石窟，或一个礼堂，一个巨穴, 一个足球体育馆等。而且你真的应该使用由硬件提供的混合器，毕竟，那是它存在的目的。这种方法的一个不足之处是程序本身时常无法获得混合结果，因为混合是在声卡内部完成而不是在主存。如果由于某种原因你需要看到产生的音量，你是运气不好。



Music Tracks in Games（游戏中的音轨）
我们没有过多的谈到游戏中的音乐生成。传统的有两种方法，一种是简单的音乐 .wav 文件(或同等物)。它被预先制作做好，准备运行，和最小忙乱。然而，这些在内存和回放时间方面很昂贵。第二种方式用预设的样本编码MIDI音轨。这时常比较节省内存，但缺点是必须同时把一些声音混合在一起，因而会把声音通道用光。



动态音乐就是根据在游戏中目睹的行动改变你的音乐的能力，比如探险用慢节奏的音乐，战斗用快节奏的音乐。预先制作的音乐的一个困难之处是要合拍，因此你可以从一段音乐渐弱到另一段音乐，这对于MIDI音轨比较容易。尽管时常你足够快速地淡出，或者一段音乐在播放另一段音乐之前已经消失了，你能侥幸不被察觉。



在我们离开这个主题之前，顺便说一下，值得一提的是存在一些公司专门为你的游戏创作特定意义的音乐。FatMan(www.fatman.com) 就是一家这样的公司。音乐可能比其他别的东西更加容易外包，这是他们存在的方式。

最后，游戏现在的事情自然是MP3格式，允许巨大的11 ：1的声音样本压缩，然而在送到声音卡之前只花费CPU很少的时间解压缩。当我在Rave Software工作时，在Star Trek Voyager: Elite Force 中，我们设法用MP3在一张CD上面完全支持三种语言，仍然为较多的图形留有空间。主要地，我们 MP3 只用于非玩家角色（NPC）的语音，由于游戏的全部音频效果MP3流和动态解压缩超出了硬件的处理能力，虽然在将来这是肯定可能的。比较新的格式，如来自 Dolby 的 AAC 和来自微软的WMA，以将近两倍MP3的压缩率提供了相等或者更高的音频质量（实际上一半的比特率），可能应用到将来的游戏中。

以上是这一章节的内容，下面将是网络和连线游戏环境的开发。