What Is the Android NDK ?

The NDK is an addition to the Android SDK that lets you write C/C++ and assembly code that you can then integrate into your Android application. The NDK consists of a set of Android-specific C libraries, a cross-compiler toolchain based on the GNU Compiler Collection (GCC) that compiles to all the different CPU architectures supported by Android (ARM, x86, and MIPS), and a custom-built system that should make compiling C/C++ code easier when compared to writing your own makefiles.

The NDK doesn’t expose most of the Androids APIs, such as the UI toolkit. It is mostly intended to speed up slow Java methods by rewriting them in C/C++ and calling them from within Java. Since Android 2.3, Java can be bypassed almost completely by using the NativeActivity class instead of Java activities. The NativeActivity class is specifically designed to be used for games with full window control, but it does not give you access to Java at all, so it can’t be used with other Java-based Android libraries. Many game developers coming from iOS choose that route because it lets them reuse most of the C/C++ on Android without having to go too deep into the Android Java APIs. However, integration of services such as Facebook authentication or ads still needs to be done in Java, so designing the game to start in Java and call into C++ via the JNI is often the most compatible way. With that said, how does one use the JNI?

The Java Native Interface

The Java Native Interface (JNI) is a way to let the virtual machine (and hence Java code) communicate with C/C++ code. This works in both directions; you can call C/C++ code from Java, and you can call Java methods from C/C++. Many of Android’s libraries use this mechanism to expose native code, such as OpenGL ES or audio decoders.

Once you use the JNI, your application consists of two parts: Java code and C/C++ code. On the Java side, you declare class methods to be implemented in native code by adding a special qualifier called native. This could look like this:

package com.badlogic.androidgames.ndk;
public class MyJniClass {
     public native int add(int a, int b);
}

As you can see, the method we declared doesn’t have a method body. When the VM running your Java code sees this qualifier on a method, it knows that the corresponding implementation is found in a shared library instead of in the JAR file or the APK file.

A shared library is very similar to a Java JAR file. It contains compiled C/C++ code that can be called by any program that loads this shared library. On Windows, these shared libraries usually have the suffix .dll; on Unix systems, they end in .so.

On the C/C++ side, we have a lot of header and source files that define the signature of the native methods in C and contain the actual implementation. The header file for our class in the preceding code would look something like this:

/* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class com_badlogic_androidgames_ndk_MyJniClass */

#ifndef _Included_com_badlogic_androidgames_ndk_MyJniClass
#define _Included_com_badlogic_androidgames_ndk_MyJniClass
#ifdef __cplusplus extern "C" {

#endif
/*
 * Class:       com_badlogic_androidgames_ndk_MyJniClass
 * Method:      add
 * Signature: (II)I
 */
JNIEXPORT jint JNICALL Java_com_badlogic_androidgames_ndk_MyJniClass_add
(JNIEnv *, jobject, jint, jint);

#ifdef __cplusplus
}
#endif
#endif

This header file was generated with a JDK tool called javah. The tool takes a Java class as input and generates a C function signature for any native methods it finds. There’s a lot going on here, as the C code needs to follow a specific naming schema and needs to be able to marshal Java types to their corresponding C types (e.g., Java’s int becomes a jint in C). We also get two additional parameters of type JNIEnv and jobject. The first can be thought of as a handle to the VM. It contains methods to communicate with the VM, such as to call methods of a class instance. The second parameter is a handle to the class instance on which this method was invoked. We could use this in combination with the JNIEnv parameter to call other methods of this class instance from the C code.

This header file does not contain the implementation of the function yet. We need a corresponding C/C++ source file that implements the function:

#include "myjniclass.h"

JNIEXPORT jint JNICALL Java_com_badlogic_androidgames_ndk_MyJniClass_add
     (JNIEnv * env, jobject obj, jint a, jint b) {
     return a + b;
}

These C/C++ sources are compiled to a shared library, which we then load through a specific Java API so that the VM can find the implementation of the native methods of our Java classes:

 System.loadLibrary("myjnitest");
int result = new MyJniClass().add(12, 32);

The call to System.loadLibrary() takes the name of the shared library. How it finds the corresponding file is up to the VM implementation to some degree. This method is called only once, at startup, so that the VM knows where to find the implementations of any native libraries. As you can see, we can invoke the MyJniClass.add() method just like any other Java class method!

Enough of all this talk. Let’s get our hands dirty with a little C code on Android! We will write a few simple C functions that we want to invoke from our Java application. We’ll guide you through the process of compiling the shared library, loading it, and calling into the native methods.

Setting Up the NDK

Before we start, we have to install the NDK. This is actually a rather simple process.

1. Head over to http://developer.android.com/tools/sdk/ndk/index.html , click on Download, and pick the archive for your platform.

2. Extract the archive to a location of your liking and note down its location.

3. Add the base NDK directory to your system path:

   1. On Linux or Mac OS X, open a shell and add the path of the NDK installation directory to your $PATH environment variable. Generally, it’s in .profile and the line looks like this: export PATH = $PATH:/ path/to/your/ndk/installation

   2. On Windows, choose Control Panel ➤ System and Security ➤ System ➤ Advanced System Settings ➤ Environment Variables ➤ System Variables. Select Path in the System Variables list, click Edit, and add the directory to the end of the variable value, starting with a semicolon (e.g., ;c:Android_NDK).

4. To verify that your NDK has been installed successfully, issue the command ndk-build in your terminal. It should spit out a few comments about the lack of an Android project.

5. In Chapter 2, we set up the JDK and added its bin/ directory to our path.

6. Make sure the tools in that directory are still available by issuing the command javah. It should print out the usage information for that tool. We’ll need it later.

Setting Up an NDK Android Project

As in the previous coding chapters, you have to create a new Android project. Copy over all the framework code from Chapter 12. Then, create a new package called com.badlogic.androidgames.ndk, place into it a copy of one of the starter activities from previous chapters, rename it to NdkStarter, and make it the launcher activity. As always, remember to add any new tests to both the manifest file and the starter activity.

To make things easier, you should open your terminal now and navigate to that new project’s root directory. Make sure your PATH still contains the correct entries so that the ndk-build and javah tools can be invoked.

Creating Java Native Methods

As we saw previously, it’s rather simple to specify which methods of a Java class are implemented in native code. However, we need to be careful when defining what types we pass into the method and what return type we get out of the method.

While we can pass any Java type to a native method, some types are considerably harder to handle than others. The easiest types to work with are primitive types like int, byte, boolean, char, and so on, which correspond directly to the equivalent C types. The next easiest types to handle on the C/C++ side are one-dimensional arrays of primitive types, such as int[] or float[].

These arrays can be converted to C arrays or pointers with methods provided by the JNIEnv type we saw earlier. Next on the easiness scale are direct ByteBuffer instances. As with arrays, they can be easily converted to a pointer. Depending on the use case, strings can be easy to use as well. Objects and multidimensional arrays are a lot harder to handle. Working with these on the C/C++ side is similar to working with the reflection APIs on the Java side.

We can also return any Java type from native methods. Primitive types are again rather easy to handle. Returning other types usually involves creating an instance of that type on the C/C++ side, which can be pretty complicated.

We’ll only look into passing around primitive types, arrays, ByteBuffer instances, and strings a little bit. If you want to know more about how to handle types via the JNI, we refer you to the (online) book Java Native Interface 5.0 Specification, found at http://docs.oracle.com/javase/1.5.0/ docs/guide/jni/spec/jniTOC.html.

For our JNI experiment, we’ll create two methods. One will copy a float[] to a direct ByteBuffer in C code, and the other will print out a string to logcat. Listing 13-1 shows our JniUtils class.

Listing 13-1. JniUtils.java: We keep it simple

package com.badlogic.androidgames.ndk;

import java.nio.ByteBuffer;

public class JniUtils {
     static {
     System.loadLibrary("jniutils");
     }

     public static native void log(String tag, String message);

     public static native void copy(ByteBuffer dst, float[] src, int offset, int len);
}

The class starts with a static block. The code in this block will be invoked the first time the VM encounters a reference to the JNIUtils class. It’s the perfect place to call System.loadLibrary(), which will load the shared library we’ll compile in a bit. The parameter we pass to the method is the pure name of the shared library. As we’ll see later, the actual file is called libjniutils.so. The method will figure this out on its own.

The log() method mimics the Android Java Log.d() method. It takes a tag and a message that will get printed to logcat.

The copy() method is actually useful. In Chapter 8, we investigated the performance problems for the FloatBuffer.put() method. We resorted to using a pure Java implementation that used IntBuffer and some nasty tricks so we could speed up the copying of a float array to a direct ByteBuffer in the Vertices class. We’ll now implement a method that takes a direct ByteBuffer and a float array and copies the array to the buffer. This is a lot faster than using the corresponding Java APIs. We can later modify our Vertices and Vertices3 classes to use this new functionality.

Note that both methods are static methods instead of instance methods. This means we can call them without having an instance of the class JniTest! This also has minor effects on our C signatures, as we’ll see in a bit.

Creating the C/C++ Header and Implementation

The first thing we do when we start to write the C/C++ code is generate the header file via the javah JDK command-line tool. It takes a few parameters that are useful to us:

• The name of the output file, which in our case is jni/jniutils.h. The javah tool will create the jni/ folder for us if it doesn’t exist yet.

• The path containing the .class file of the Java class for which it should generate a C header. This will be bin/classes if we invoke javah from the root directory of our project. It’s the output path for the Eclipse compiler when it compiles any of our Android project’s source files.

• The fully qualified name of the class, which is com.badlogic. androidgames.ndk.JniUtils in our case.

Open the terminal or command prompt and navigate to the root folder of the Android project. Make sure that the NDK and JDK are in your $PATH as described earlier. Now execute the following command:

javah -o jni/jniutils.h -classpath bin/classes com.badlogic.androidgames.ndk.JniUtils

This will create a file called jniutils.h in the jni/ folder of our Android project. Listing 13-2 shows its content.

Listing 13-2. jniutils.h, containing the C functions that implement our native methods

/* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class com_badlogic_androidgames_ndk_JniUtils */

#ifndef _Included_com_badlogic_androidgames_ndk_JniUtils
#define _Included_com_badlogic_androidgames_ndk_JniUtils
#ifdef __cplusplus extern "C" {

#endif
/*
 * Class:     com_badlogic_androidgames_ndk_JniUtils
 * Method:    log
 * Signature: (Ljava/lang/String;Ljava/lang/String;)V
 */
JNIEXPORT void JNICALL Java_com_badlogic_androidgames_ndk_JniUtils_log
(JNIEnv *, jclass, jstring, jstring);

/*
 * Class:     com_badlogic_androidgames_ndk_JniUtils
 * Method:    copy
 * Signature: (Ljava/nio/ByteBuffer;[FII)V
 */
JNIEXPORT void JNICALL Java_com_badlogic_androidgames_ndk_JniUtils_copy
(JNIEnv *, jclass, jobject, jfloatArray, jint, jint);

#ifdef __cplusplus
}
#endif
#endif

Time to implement those functions. First, we create a new file called jniutils.cpp in the jni/ folder. Listing 13-3 shows its content.

Listing 13-3. jniutils.cpp, the implementation of the JniUtils native methods

#include <android/log.h>
#include <string.h>
#include "jniutils.h"

We need a few C includes, namely log.h, which is provided by the NDK; string.h; and our own jniutils.h. The first include lets us use the native Android logging functions. The second include lets us use memcpy(). The last one imports the signatures of our native methods as well as jni.h, which contains the JNI APIs.

JNIEXPORT void JNICALL Java_com_badlogic_androidgames_ndk_JniUtils_log
   (JNIEnv *env, jclass clazz, jstring tag, jstring message) {
   const char *cTag = env-> GetStringUTFChars(tag, 0);
   const char *cMessage = env-> GetStringUTFChars(message, 0);

      __android_log_print(ANDROID_LOG_VERBOSE, cTag, cMessage);

   env-> ReleaseStringUTFChars(tag, cTag);
   env-> ReleaseStringUTFChars(message, cMessage);
}

This function implements the JniUtils.log() method, which takes a JNIEnv and a jclass as the first two parameters. The env parameter allows us to work directly with the JVM. The jclass parameter represents the JniUtils class. Remember that our methods are static methods. Instead of getting a jobject, as in the previous example, we get the class. The tag and message parameters are the two strings we pass in from Java.

The log.h header defines a function called android_log_print, which is similar to the standard C printf function. It takes a log level and two char* pointers representing the tag and the message. Our tag and message parameters have the type jstring and can’t be cast to char* pointers. Instead, we have to temporarily convert them to char* pointers via methods exposed by the env parameter. This is done in the first two lines of the function via calls to

env-> GetStringUTFChars().
env-> ReleaseStringUTFChars().

Next, we simply call the logging method, passing in the parameters. Finally, we need to clean up the converted strings so that we don’t leak memory. This is done via

JNIEXPORT void JNICALL Java_com_badlogic_androidgames_ndk_JniUtils_copy
   (JNIEnv *env, jclass clazz, jobject dst, jfloatArray src, jint offset, jint len) {
     unsigned char* pDst = (unsigned char*)env-> GetDirectBufferAddress(dst);
     float* pSrc = (float*)env-> GetPrimitiveArrayCritical(src, 0); memcpy(pDst,
     pSrc + offset, len * 4);
     env-> ReleasePrimitiveArrayCritical(src, pSrc, 0);
}

The second function takes a direct ByteBuffer, a float array, an offset into the float array, and the number of floats we want to copy. Note that the ByteBuffer has the type jobject! Whenever you pass in anything other than a primitive type or an array, you will get a jobject. Your C/C++ code needs to know which type to expect! In our case, we know that we are getting a ByteBuffer instance. ByteBuffer instances are just thin wrappers around a native memory area. They are super easy to handle in C; we can simply fetch a pointer to their memory address via env-> GetDirectBufferAddress().

Our float array is a bit more difficult to handle. The env-> GetPrimitiveArrayCritical() method will lock the array and return a pointer to its first element. Using this function is dangerous; you should not try to concurrently modify the array in Java. Calling any other JNI method from this point on is prohibited as well. Otherwise, you’ll get hard-to-debug behaviors in your C/C++ code!

Once we have the pointers, we simply use memcpy() to copy the contents of the float array to the ByteBuffer. Note that we do not perform any kind of bounds checking, which means the Java code calling this method must be bulletproof. Trying to copy more floats into the ByteBuffer than we allocated might result in a nasty segmentation fault. The same is true when we specify offsets and lengths that are outside of the float array we passed in. The general takeaway is that you have to know what you are doing when using JNI and the NDK. If you don’t, your application will just explode with hard-to-debug errors!

At the end of the function, we unlock the float array again via a call to env-> ReleasePrimitiveArrayCritical(). You must call this method under all circumstances; otherwise, you’ll run into all kinds of issues.

With our C/C++ header and source files in place, it’s time to build the shared library.

Building the Shared Library

As mentioned earlier, the NDK comes with its own build system. While it still uses standard makefiles under the hood, you as a user don’t have to deal with their complexity. Instead, you write two files: an Application.mk file specifying which CPU architectures you want to target, and an Android.mk file defining which other libraries you want to link to, which source files to compile, and how the final shared library should be called.

The Application.mk file is placed into the jni/ folder. Listing 13-4 shows its content.

Listing 13-4. Application.mk, defining the CPUs we want to target

APP_ABI := armeabi armeabi-v7a x86 mips

And that’s all there is to it! All it defines are the four architectures our native code should run on. The ARM architectures are the most common targets—pretty much all current Android devices have an ARM CPU. The x86 architecture can be found in soon-to-be-released Intel devices, according to rumors. There are also some emulator images that support this architecture. The MIPS architecture is currently used by a handful of low-end Android tablets.

Having defined the architectures, we can now move on to the Android.mk file, which specifies how our native code should be built. This file is also located in the jni/ folder. Listing 13-5 shows the content.

Listing 13-5. Android.mk, specifying our build

LOCAL_PATH:= $(call my-dir)
include $(CLEAR_VARS)

LOCAL_MODULE    := jniutils
LOCAL_LDLIBS    := − llog
LOCAL_ARM_MODE  := arm
LOCAL_SRC_FILES := jniutils.cpp

include $(BUILD_SHARED_LIBRARY)

The first two lines are pretty much boilerplate. They make sure the paths are correctly handled and that various variables are reset.

The next line defines our shared library’s name, which is jniutils in this case.

Next, we specify which libraries we want to link to. We use the native Android logging facilities, so we link to the liblog library provided by the NDK.

The next line is specific to the ARM architecture. It tells the build system that we want to generate non-thumb code. ARM processors have two modes in which they operate: thumb and 32-bit ARM. The former results in smaller code, but it is often slower. For our C/C++ code, it doesn’t really matter that much, but it’s good practice to enable this option.

Next, we specify the C/C++ source files that should be compiled. We only have a single file. To specify additional files, just add them to the same line, separated with a space, or add them on a new line. If you choose the second option, you have to append a backslash at the end of the previous line.

The last line tells the build system to generate a shared library. We could also let it generate a static library, which we would then compile into a shared library with other static libraries. This is a mechanism used by more complicated JNI projects. We are happy with a simple shared library.

Now, it’s time to build our shared library. To do that, open your terminal, make sure your PATH environment variable is in order, navigate to the root directory of your project, and issue the following command:

ndk-build

If all went well, you should be greeted with the following output:

Compile++ arm  : jniutils <= jniutils.cpp
In file included from jni/jniutils.h:2:0,
                 from jni/jniutils.cpp:2:
D:/workspaces/book/android-ndk-r8b/platforms/android-14/arch-arm/usr/include/jni.h:592:13: note:
the mangling of 'va_list' has changed in GCC 4.4
StaticLibrary : libstdc++.a
SharedLibrary : libjniutils.so
Install : libjniutils.so => libs/armeabi/libjniutils.so
Compile++ arm  : jniutils <= jniutils.cpp
In file included from jni/jniutils.h:2:0,
                 from jni/jniutils.cpp:2:
D:/workspaces/book/android-ndk-r8b/platforms/android-14/arch-arm/usr/include/jni.h:592:13: note:
the mangling of 'va_list' has changed in GCC 4.4
StaticLibrary : libstdc++.a
SharedLibrary : libjniutils.so
Install : libjniutils.so => libs/armeabi-v7a/libjniutils.so
Compile++ x86 : jniutils <= jniutils.cpp
StaticLibrary : libstdc++.a
SharedLibrary : libjniutils.so
Install : libjniutils.so => libs/x86/libjniutils.so
Compile++ mips : jniutils <= jniutils.cpp
StaticLibrary : libstdc++.a
SharedLibrary : libjniutils.so
Install : libjniutils.so => libs/mips/libjniutils.so

This cryptic output tells us a few things. First, our code got compiled for each of the four CPU architectures separately. The resulting shared libraries are all called libjniutils.so. They are placed in the libs/ folder. Each architecture has a subdirectory (e.g., armeabi or x86). When we compile our APK, those shared libraries get packaged with our application. When we call System.loadLibrary(), as in Listing 13-1, Android knows where to find the correct shared library for the architecture on which our application is currently running. Great, let’s put this to the test!

Putting It All Together

We now have everything in place to test our native methods. Let’s create a test that invokes both of the JniUtils methods. We call the class JniUtilsTest; it extends the GLGame class and contains a GLScreen implementation, as usual. It simply copies a float[] array into a direct ByteBuffer and then outputs the contents of the ByteBuffer to logcat via the other native method. Listing 13-6 shows the entire code. Don’t forget to add it to the NdkStarter class and the manifest file.

Listing 13-6. JniUtilsTest.java: Testing our native methods

package com.badlogic.androidgames.ndk;

import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.FloatBuffer;

import com.badlogic.androidgames.framework.Game; import com.badlogic.androidgames.framework.Screen; import com.badlogic.androidgames.framework.impl.GLGame;
import com.badlogic.androidgames.framework.impl.GLScreen;

public class JniUtilsTest extends GLGame {

     public Screen getStartScreen() {
         return new JniUtilsScreen(this);
     }

     class JniUtilsScreen extends GLScreen {
           public JniUtilsScreen(Game game) {
               super(game);
               float[] values = { 1.231f, 554.3f, 348.6f, 499.3f }; ByteBuffer buffer =
               ByteBuffer.allocateDirect(3 * 4);
               buffer.order(ByteOrder.nativeOrder());

               JniUtils.copy(buffer, values, 1, 3);
               FloatBuffer floatBuffer = buffer.asFloatBuffer();
               for(int i = 0; i < 3; i++) {
                    JniUtils.log("JniUtilsTest", Float.toString(floatBuffer.get(i)));
               }
           }

           @Override
           public void update(float deltaTime) {
           }

           @Override
           public void present(float deltaTime) {
           }

           @Override
           public void pause() {
           }

           @Override
           public void resume() {
           }

           @Override
           public void dispose() {
           }
     }
}

All the important stuff happens in the screen’s constructor. We start off by creating a small float[] array with some dummy values and a direct ByteBuffer instance that can hold 12 bytes, or 4 floats. We also make sure that the ByteBuffer instance uses native byte order so that we don’t run into some nasty issues when we pass it to our C/C++ code.

We then copy three floats from the float[] array, starting at index 1, over to our ByteBuffer instance. The final few lines output the copied floats via our native logging method. Executing this code on a device will output the following in logcat:

08–15 17:28:31.953: V/JniUtilsTest(1901): 554.3
08–15 17:28:31.953: V/JniUtilsTest(1901): 348.6
08–15 17:28:31.953: V/JniUtilsTest(1901): 499.3

Exactly what we were expecting.

Now you can use the JniUtils class to copy the float[] array to the direct ByteBuffer instance in your game code. Note that we set the limit and position of the buffer manually. This is necessary because our JNI method doesn’t manipulate the position and limit fields of the buffer. The OpenGL ES methods we pass that buffer to might use that information, though.

By using our native copy() method, we can gain quite a lot of speed, especially when we draw many sprites via the SpriteBatcher class. Having to move vertices from the CPU to the GPU every frame costs in performance quality, while doing so by making a copy first and converting everything to ints is even worse. Our new JniUtils reduces the amount of copying considerably, while also being faster than the IntBuffer trick we employed earlier!

To test our implementation, simply copy all the necessary assets from previous chapters. Hook them up with the NdkStarter activity and try them out on your Android device.

Summary

We only scratched the surface of what’s possible with the NDK. But, as you witnessed, even lightweight use of the NDK can pay off quite a bit. Our copy() method was simple to implement but let us display more sprites per frame by being a lot faster than the old, Java-based implementation. This is a recurring theme with NDK development: identify your bottlenecks and implement only very small pieces of code on the native side. Don’t go overboard, as calling native methods has overhead. Just making everything a native method is unlikely to increase the performance of your application.

It’s now time to look into marketing considerations for our game.