Microphones and audio processing have become a fixed part of game development, not only for music and singing games, but also for team based action and strategy titles, which require multiple players to coordinate their actions in order to succeed. It doesn't matter whether you want to make your game voice controlled, detect the pitch of someone singing or implement a voice chat, you will almost always need to get access to an audio device. In this short recipe, you will learn how to open an audio source for recording and do some simple signal processing on the received audio data.
Set up a new Panda3D Python project as shown in Setting up the game structure found in Chapter 1 before going on with this recipe.
Complete these tasks to enable and handle input from a microphone:
- Open
Application.pyand replace its contents with the following code:from direct.showbase.ShowBase import ShowBase from panda3d.core import * import audioop class Application(ShowBase): def __init__(self): ShowBase.__init__(self) self.addSmiley() self.addGround() self.setupMicrophone() self.cam.setPos(0, -50, 10) def setupMicrophone(self): for i in range(MicrophoneAudio.getNumOptions()): print i, MicrophoneAudio.getOption(i) if MicrophoneAudio.getNumOptions() > 0: index = raw_input("choose device: ") opt = MicrophoneAudio.getOption(0) self.cursor = opt.open() taskMgr.add(self.update, "update audio") def addSmiley(self): self.smiley = loader.loadModel("smiley") self.smiley.reparentTo(render) self.smiley.setZ(10) def addGround(self): input handling, Panda3Daudio data, reading from microphonecm = CardMaker("ground") cm.setFrame(-500, 500, -500, 500) ground = render.attachNewNode(cm.generate()) ground.setColor(0.2, 0.4, 0.2) ground.lookAt(0, 0, -1) def update(self, task): if self.cursor.ready() >= 16: data = self.cursor.readSamples(self.cursor.ready()) rms = audioop.rms(data, 2) minmax = audioop.minmax(data, 2) intensity = float(rms) / 32767.0 self.win.setClearColor(Vec4(intensity, intensity, intensity, 1)) print rms, minmax currentZ = self.smiley.getZ() self.smiley.setZ(currentZ - 0.3 + intensity) if self.smiley.getZ() <= 1: self.smiley.setZ(1) return task.cont - Start the program. The window background color will get closer to white and the smiley will start to lift off the ground as you provide louder input:
To open an audio recording device for reading data in Panda3D we have to retrieve a cursor object of the type MovieAudioCursor. For this, we choose a device option and open it. In Panda3D's MicrophoneAudio API terminology, an option describes a combination of input device, jack, channel, and sampling rate. First, we print a list of all available options. Then we create a console prompt that asks for the number of the device to use.
To read raw audio data from the recording device, we need to continuously poll the cursor, checking if enough samples have been buffered. To access the audio data we use the readSamples() method. This returns a string of audio bytes, which we can directly use for some simple audio processing.
Using the audioop module that is part of the Python programming libraries, we calculate the root mean square of the audio signal. This is used for getting the power of the signal, which means you will get lower values when whispering into the microphone and higher ones when screaming and shouting. We just use the root mean square to create a very simple visualization of input loudness, but we could also use it to encourage players of a karaoke game to sing louder or do some simple beat detection for audio visualizations by watching how this value changes over time.
For a more thorough analysis of the input data, we would need fast Fourier transforms, among other techniques. An efficient and easy to use implementation of fast Fourier transforms can be found in the NumPy library, for example, which can be downloaded for free from numpy.scipy.org.