To begin with, we need to add that data capture to our VisualizerBox class. We will start by adding the variables we'll need:

    public static byte[] fftBytes, fftNorm;
    public static float[] fftPrep;
    public static int fftTexture = -1;

We need to allocate the FFT data arrays, and to do that we need to know their size. We can ask the Android Visualizer API how much data it's capable of giving us. For now, we'll choose the minimum size and then allocate the arrays as follows:

    public VisualizerBox(final CardboardView cardboardView){
        . . .
        fftPrep = new float[captureSize / 2];
        fftNorm = new byte[captureSize / 2];
        ...

Capturing FFT data is similar to capturing waveform data. But we'll do some preprocessing on it before saving it. According to the Android Visualizer API documentation, (http://developer.android.com/reference/android/media/audiofx/Visualizer.html#getFft(byte[]) the getFfT function provides data specified as follows:

The layout in the returned byte array is as follows:

Likewise, we'll prepare the incoming captured data into a normalized array of values between 0 and 255. Our implementation is as follows. Add the onFftDataCapture declaration immediately after the onWaveFormDataCapture method (within the OnDataCaptureListener instance):

            @Override
            public void onFftDataCapture(Visualizer visualizer, byte[] bytes, int samplingRate) {
                fftBytes = bytes;
                float max = 0;
                for(int i = 0; i < fftPrep.length; i++) {
                    if(fftBytes.length > i * 2) {
                        fftPrep[i] = (float)Math.sqrt(fftBytes[i * 2] * fftBytes[i * 2] + fftBytes[i * 2 + 1] * fftBytes[i * 2 + 1]);
                        if(fftPrep[i] > max){
                            max = fftPrep[i];
                        }
                    }
                }
                float coeff = 1 / max;
                for(int i = 0; i < fftPrep.length; i++) {
                    if(fftPrep[i] < MIN_THRESHOLD){
                        fftPrep[i] = 0;
                    }
                    fftNorm[i] = (byte)(fftPrep[i] * coeff * 255);
                }
                loadTexture(cardboardView, fftTexture, fftNorm);
            }

Note that our algorithm uses a MIN_THRESHOLD value of 1.5 to filter out insignificant values:

    final float MIN_THRESHOLD = 1.5f;

Now in setup(), initialize fftTexture with a generated texture, as we do for the audioTexture variable:

    public void setup() {
        audioTexture = genTexture();
        fftTexture = genTexture();
        if(activeViz != null)
            activeViz.setup();
    }

Now we need to write the shader programs.

If necessary, create a resources directory for the shaders, res/raw/. The fft_vertex.shader is identical to the waveform_vertext.shader created earlier, so you can just duplicate it.

For the fft_fragment shader, we add a bit of logic to decide whether the current coordinate is being rendered. In this case, we are not specifying a width and just rendering all pixels below the value. One way to look at the difference is that our waveform shader is a line graph (well, actually a scatterplot), and our FFT shader is a bar graph.

File: res/raw/fft_fragment.shader

precision mediump float;        // default medium precision
uniform sampler2D u_Texture;    // the input texture

varying vec2 v_TexCoordinate;   // interpolated texture coordinate per fragment
uniform vec4 u_Color;

void main() {
    vec4 color;
    if(v_TexCoordinate.y < texture2D(u_Texture, v_TexCoordinate).r){
        color = u_Color;
    }
    gl_FragColor = color;
}

The code for the FFTMaterial class is very similar to what we did for the WaveformMaterial class. So for brevity, just duplicate that file into a new one named FFTMaterial.java. And then, modify it as follows.

Ensure that the class name and constructor method name now read as FFTMaterial:

public class FFTMaterial extends Material {
    private static final String TAG = "FFTMaterial";
    ...

    public FFTMaterial(){
    ...

We decided to change the borderColor array to a different hue:

    public float[] borderColor = new float[]{0.84f, 0.65f, 1f, 1f};

In setupProgram, ensure that you're referencing the R.raw.fft_vertex and R.raw.fft_fragment shaders:

        program = createProgram( R.raw.fft_vertex, R.raw.fft_fragment);

Then, make sure that the appropriate shader-specific parameters are getting set. These shaders use u_Color (but not a u_Width variable):

    //Shader-specific parameters
    textureParam = GLES20.glGetUniformLocation(program, "u_Texture");
    MVPParam = GLES20.glGetUniformLocation(program, "u_MVP");
    colorParam = GLES20.glGetUniformLocation(program, "u_Color");
    RenderBox.checkGLError("FFT params");

Now, in the draw method, we're going to draw with the VisualizerBox.fftTexture value (instead of VisualizerBox.audioTexture), so change the call to GLES20.glBindTexture as follows:

GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, VisualizerBox.fftTexture);

Ensure that the colorParam parameter is set (but unlike the WaveformMaterial class, there is no width parameter here):

GLES20.glUniform4fv(colorParam, 1, borderColor, 0);

We can now add the visualization for the FFT data. In the visualizations/ folder, duplicate the WaveformVisualization.java file into a new file named FFTVisualization.java. Ensure that it's defined as follows:

    public class FFTVisualization extends Visualization {

In its setup method, we'll create a Plane component and texture it with the FFTMaterial class like this, (also note modifying the position and rotation values):

    public void setup() {
        plane = new Plane().setMaterial(new FFTMaterial()
                .setBuffers(Plane.vertexBuffer, Plane.texCoordBuffer, Plane.indexBuffer, Plane.numIndices));

        new Transform()
                .setLocalPosition(5, 0, 0)
                .setLocalRotation(0, -90, 0)
                .addComponent(plane);
    }

Now in onCreate of MainActivity, replace the previous visualization with this one:

visualizerBox.activeViz = new FFTVisualization(visualizerBox);

When you run the project, we get a visualization like this, rotated and positioned over to the right:

This simple example illustrates that FFT data separates spatial frequencies of the audio into discrete data values. Even without understanding the underlying mathematics (which is nontrivial), it's often sufficient to know that the data changes and flows in sync with the music. We used it here to drive a texture map. FFT can also be used like we used waveform data in the first example to drive attributes of 3D objects in the scene, including position, scale, and rotation, as well as parametrically defined geometry. In fact, it is generally a better data channel for such purposes. Each bar corresponds to an individual frequency range, so you can specify certain objects to respond to high frequencies versus low frequencies.