Wait, there's more! Just one more thing to add. Building interactive applications require us to be able to determine whether the user is gazing at a specific object. We can put this into RenderObject, so any objects in the scene can be gaze detected.

The technique that we'll implement is straightforward. Considering each object we render is projected onto a camera plane, we really only need to determine whether the user is looking at the object's plane. Basically, we check whether the vector between the camera and the plane position is the same as the camera's view direction. But we'll throw in some tolerance, so you don't have to look exactly at the center of the plane (that'd be impractical). We will check a narrow range. A good way to do this is to calculate the angle between the vectors. We calculate the pitch and yaw angles between these vectors (the up/down X axis angle and left/right Y axis angle, respectively). Then, we check whether these angles are within a narrow threshold range, indicating that the user is looking at the plane (more or less).

This method is just like the one used in Chapter 3, Cardboard Box, although at that time, we put it in MainActivity. Now, we'll move it into the RenderObject component.

Note that this can get inefficient. This technique is fine for our projects because there is a limited number of objects, so the calculation isn't expensive. But if we had a large complex scene with many objects, this setup would fall short. In that case, one solution is to add an isSelectable flag so that only those objects that should be interactive in a given frame will be interactive.

If we were using a fully-featured game engine, we would have a physics engine capable of doing a raycast to precisely determine whether the center of your gaze intersects the object with a high degree of accuracy. While this might be great in the context of a game, it is overkill for our purposes.

At the top of RenderObject, add a Boolean variable for an isLooking value. Also, add two variables to hold the yaw and pitch range limits to detect the camera viewing angle, and a modelView matrix that we'll use for calculations:

    public boolean isLooking;
    private static final float YAW_LIMIT = 0.15f;
    private static final float PITCH_LIMIT = 0.15f;
    final float[] modelView = new float[16];

The implementation of the isLookingAtObject method is as follows. We convert the object space to the camera space, using the headView value from onNewFrame, calculate the pitch and yaw angles, and then check whether they're within the range of tolerance:

    private boolean isLookingAtObject() {
        float[] initVec = { 0, 0, 0, 1.0f };
        float[] objPositionVec = new float[4];

        // Convert object space to camera space. Use the headView // from onNewFrame.
        Matrix.multiplyMM(modelView, 0, RenderBox.headView, 0, model, 0);
        Matrix.multiplyMV(objPositionVec, 0, modelView, 0, initVec, 0);

        float pitch = (float) Math.atan2(objPositionVec[1], -objPositionVec[2]);
        float yaw = (float) Math.atan2(objPositionVec[0], -objPositionVec[2]);

        return Math.abs(pitch) < PITCH_LIMIT && Math.abs(yaw) < YAW_LIMIT;
    }

For convenience, we'll set the isLooking flag at the same time the object is drawn. Add the call at the end of the draw method:

    public void draw(float[] view, float[] perspective){
        . . . 
        isLooking = isLookingAtObject();
    }

That should do it.

For a simple test, we'll log some text to the console when the user is gazing at the cube. In MainActivity, make a separate variable for the Cube object:

    Cube cubeObject;

    public void setup() {
        cube = new Transform();
        cubeObject = new Cube(true);
        cube.addComponent(cubeObject);
        cube.setLocalPosition(2.0f, -2.f, -5.0f);
    }

Then, test it in postDraw, as follows:

    public void postDraw() {
        if (cubeObject.isLooking) {
            Log.d(TAG, "isLooking at Cube");
        }
    }