OpenMAX components have input buffers to give data to the component and output buffers to return data either to the application or to the next component in a pipeline. This chapter looks at buffer management both using OpenMAX and using the IL Client library on the Raspberry Pi.
Building Programs
You can build the programs in this chapter using the following Makefile:
DMX_INC = -I/opt/vc/include/ -I /opt/vc/include/interface/vmcs_host/ -I/opt/vc/include/interface/vcos/pthreads -I/opt/vc/include/interface/vmcs_host/linuxEGL_INC =OMX_INC = -I /opt/vc/include/ILOMX_ILCLIENT_INC = -I/opt/vc/src/hello_pi/libs/ilclientINCLUDES = $(DMX_INC) $(EGL_INC) $(OMX_INC) $(OMX_ILCLIENT_INC)CFLAGS=-g -DRASPBERRY_PI -DOMX_SKIP64BIT $(INCLUDES)CPPFLAGS =DMX_LIBS = -L/opt/vc/lib/ -lbcm_host -lvcos -lvchiq_arm -lpthreadEGL_LIBS = -L/opt/vc/lib/ -lEGL -lGLESv2OMX_LIBS = -lopenmaxilOMX_ILCLIENT_LIBS = -L/opt/vc/src/hello_pi/libs/ilclient -lilclientLDLIBS = $(DMX_LIBS) $(EGL_LIBS) $(OMX_LIBS) $(OMX_ILCLIENT_LIBS)all: il_buffer il_decode_image
Buffers
Data is transferred into and out of components using buffers, which are just byte arrays. Each component has ports, and the ports each have a number of buffers. The program portinfo of Chapter 8 can be used to list how many buffers a port has, how many it must minimally have, and what the recommended buffer size is.
For example, running this on the Broadcom video_decodecomponent’s input port gives the following:
$./portinfo OMX.broadcom.video_decode 130Component name: OMX.broadcom.video_decode:46 version 0.0, Spec version 1.1Port 130is input portDomain is VideoBuffer count 20Buffer minimum count 1Buffer size 81920 bytes
When a component is created, it has an array of pointers to buffer headers (a placeholder for buffers and information about them) but doesn’t actually have any memory allocated either for the headers or for the buffers. These have to be allocated. In the next section, you’ll look at the OpenMAX calls to do this, and then in the following section you’ll see how the IL Client library manages this.
OpenMAX Buffer Allocation
There are two OpenMAX calls to get buffers for a port. The first is OMX_AllocateBuffer where the IL client asks the component to do all the work for them (section 3.2.2.15 of the specification). Typical code is as follows:
for (i = 0; i < pClient-> nBufferCount; i++) {OMX_AllocateBuffer(hComp,&pClient->pBufferHdr[i],pClient->nPortIndex,pClient,pClient->nBufferSize);}
The component must be in the Loaded state or the port must be disabled. Typically, an application will disable the port and then allocate the buffers.
The other call is OMX_UseBuffer (section 3.2.2.14). This takes a buffer allocated either by the client or by another component and uses that as its own buffer.
The IL Client Library
You don’t need to worry about buffer allocation. The library does it for you. The function ilclient_enable_port_buffers does the relevant buffer allocation for that port. The following is a typical call:
ilclient_enable_port_buffers(component, <port_index>,NULL, NULL, NULL);
(You can give your own malloc function, but it is easier to let the library use its own.)
Once buffers have been allocated, the port can be moved to an enabled state as follows:
ilclient_enable_port(component, <port_index>)This is a blocking call and will complete when the port has changed state .
You can modify the state change program of the previous chapter to create buffers for its ports. For simplicity, you hard-code a component (in this example, the Broadcom image_encode component), which has one input port 340 and one output port 341. Note that you have to set the component create flags to the following:
ILCLIENT_DISABLE_ALL_PORTS|ILCLIENT_ENABLE_INPUT_BUFFERS|ILCLIENT_ENABLE_OUTPUT_BUFFERS
If you don’t, you may get obscure errors such as the following:
assertion failure:ilclient.c:747:ilclient_change_component_state():error == OMX_ErrorNoneProgram received signal SIGABRT, Aborted.0xb6e41bfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6
If this occurs, it’s best to use a debugger such as gdb to step into ilclient_change_component_state to find out what the actual error is.
The program is il_buffer.c.
#include <stdio.h>#include <stdlib.h>#include <OMX_Core.h>#include <OMX_Component.h>#include <bcm_host.h>#include <ilclient.h>void printState(OMX_HANDLETYPE handle) {OMX_STATETYPE state;OMX_ERRORTYPE err;err = OMX_GetState(handle, &state);if (err != OMX_ErrorNone) {fprintf(stderr, "Error on getting state ");exit(1);}switch (state) {case OMX_StateLoaded: printf("StateLoaded "); break;case OMX_StateIdle: printf("StateIdle "); break;case OMX_StateExecuting: printf("StateExecuting "); break;case OMX_StatePause: printf("StatePause "); break;case OMX_StateWaitForResources: printf("StateWait "); break;case OMX_StateInvalid: printf("StateInvalid "); break;default: printf("State unknown "); break;}}char *err2str(int err) {switch (err) {case OMX_ErrorInsufficientResources: return "OMX_ErrorInsufficientResources";case OMX_ErrorUndefined: return "OMX_ErrorUndefined";case OMX_ErrorInvalidComponentName: return "OMX_ErrorInvalidComponentName";case OMX_ErrorComponentNotFound: return "OMX_ErrorComponentNotFound";case OMX_ErrorInvalidComponent: return "OMX_ErrorInvalidComponent";case OMX_ErrorBadParameter: return "OMX_ErrorBadParameter";case OMX_ErrorNotImplemented: return "OMX_ErrorNotImplemented";case OMX_ErrorUnderflow: return "OMX_ErrorUnderflow";case OMX_ErrorOverflow: return "OMX_ErrorOverflow";case OMX_ErrorHardware: return "OMX_ErrorHardware";case OMX_ErrorInvalidState: return "OMX_ErrorInvalidState";case OMX_ErrorStreamCorrupt: return "OMX_ErrorStreamCorrupt";case OMX_ErrorPortsNotCompatible: return "OMX_ErrorPortsNotCompatible";case OMX_ErrorResourcesLost: return "OMX_ErrorResourcesLost";case OMX_ErrorNoMore: return "OMX_ErrorNoMore";case OMX_ErrorVersionMismatch: return "OMX_ErrorVersionMismatch";case OMX_ErrorNotReady: return "OMX_ErrorNotReady";case OMX_ErrorTimeout: return "OMX_ErrorTimeout";case OMX_ErrorSameState: return "OMX_ErrorSameState";case OMX_ErrorResourcesPreempted: return "OMX_ErrorResourcesPreempted";case OMX_ErrorPortUnresponsiveDuringAllocation: return "OMX_ErrorPortUnresponsiveDuringAllocation";case OMX_ErrorPortUnresponsiveDuringDeallocation: return "OMX_ErrorPortUnresponsiveDuringDeallocation";case OMX_ErrorPortUnresponsiveDuringStop: return "OMX_ErrorPortUnresponsiveDuringStop";case OMX_ErrorIncorrectStateTransition: return "OMX_ErrorIncorrectStateTransition";case OMX_ErrorIncorrectStateOperation: return "OMX_ErrorIncorrectStateOperation";case OMX_ErrorUnsupportedSetting: return "OMX_ErrorUnsupportedSetting";case OMX_ErrorUnsupportedIndex: return "OMX_ErrorUnsupportedIndex";case OMX_ErrorBadPortIndex: return "OMX_ErrorBadPortIndex";case OMX_ErrorPortUnpopulated: return "OMX_ErrorPortUnpopulated";case OMX_ErrorComponentSuspended: return "OMX_ErrorComponentSuspended";case OMX_ErrorDynamicResourcesUnavailable: return "OMX_ErrorDynamicResourcesUnavailable";case OMX_ErrorMbErrorsInFrame: return "OMX_ErrorMbErrorsInFrame";case OMX_ErrorFormatNotDetected: return "OMX_ErrorFormatNotDetected";case OMX_ErrorContentPipeOpenFailed: return "OMX_ErrorContentPipeOpenFailed";case OMX_ErrorContentPipeCreationFailed: return "OMX_ErrorContentPipeCreationFailed";case OMX_ErrorSeperateTablesUsed: return "OMX_ErrorSeperateTablesUsed";case OMX_ErrorTunnelingUnsupported: return "OMX_ErrorTunnelingUnsupported";default: return "unknown error";}}void error_callback(void *userdata, COMPONENT_T *comp, OMX_U32 data) {fprintf(stderr, "OMX error %s ", err2str(data));}int main(int argc, char** argv) {int i;char *componentName;int err;ILCLIENT_T *handle;COMPONENT_T *component;componentName = "image_encode";bcm_host_init();handle = ilclient_init();if (handle == NULL) {fprintf(stderr, "IL client init failed ");exit(1);}if (OMX_Init() != OMX_ErrorNone) {ilclient_destroy(handle);fprintf(stderr, "OMX init failed ");exit(1);}ilclient_set_error_callback(handle,error_callback,NULL);err = ilclient_create_component(handle,&component,componentName,ILCLIENT_DISABLE_ALL_PORTS|ILCLIENT_ENABLE_INPUT_BUFFERS|ILCLIENT_ENABLE_OUTPUT_BUFFERS);if (err == -1) {fprintf(stderr, "Component create failed ");exit(1);}printState(ilclient_get_handle(component));err = ilclient_change_component_state(component,OMX_StateIdle);if (err < 0) {fprintf(stderr, "Couldn't change state to Idle ");exit(1);}printState(ilclient_get_handle(component));// input portilclient_enable_port_buffers(component, 340,NULL, NULL, NULL);ilclient_enable_port(component, 340);// the input port is enabled and has input buffers allocated// output portilclient_enable_port_buffers(component, 341,NULL, NULL, NULL);ilclient_enable_port(component, 341);// the output port is enabled and has output buffers allocatederr = ilclient_change_component_state(component,OMX_StateExecuting);if (err < 0) {fprintf(stderr, "Couldn't change state to Executing ");exit(1);}printState(ilclient_get_handle(component));exit(0);}
The output from running ./il_buffer video_decode is as follows:
StateLoadedStateIdleOMX error OMX_ErrorSameStateOMX error OMX_ErrorSameStateStateExecuting
The “errors” called OMX_ErrorSameState aren’t errors at all and can be ignored.
Writing to and Reading from Buffers
The whole point of using OpenMAX on the RPi is to get data into and out of the Broadcom GPU using the OpenMAX components. You do this by loading up the input buffers of a component with data and asking the component to empty them with a call OMX_EmptyThisBuffer. When the component has processed the data, it signals an OMX_FillBufferDoneevent for an output buffer, at which point your application will take the processed data and do something with it.
When a component has finished processing its input buffer data, it signals back to the client with an OMX_EmptyBufferDoneevent. At this point, the client can put more data into the buffer for processing by the component.
Similarly, when a client has finished doing something with the output data, it returns the buffer to the component by calling the function OMX_FillThisBuffer.
One input buffer may produce none or more buffers of output data. An output buffer can consume data from one or more input buffers. There is no direct correlation between emptying an input buffer and filling an output buffer. Essentially these processes should be done concurrently by the client.
whenever there is an empty input buffer put data into the input buffer call EmptyThis Buffer | Whenever an output buffer is full process its data call FillThisBuffer |
Now of course things are not that simple. Unless you want to get into Video Core or Posix threads , a sequential version of this is easier. It looks like this:
while there is more input dataif there is an empty input bufferput data into the input buffercall EmptyThis Bufferif there is a full output bufferprocess its datacall FillThisBuffer// cleanup any remaining output datawhile there is another full output bufferprocess its datacall FillThisBuffer
Of course, this begs the question of how you know that there are empty input buffers and full output buffers. There are IL Client calls to do this.
ilclient_get_input_bufferilclient_get_output_buffer
These look after the background event processing for OMX_EmptyBufferDone and OMX_FillBufferDone. These calls can be blocking or nonblocking. Block if you know that there must be a buffer available before you can continue, but don’t block otherwise (busy wait).
Sequence of Actions
The pseudo-code for a simple application would ideally look like this:
create component in loaded state with all ports disabledmove the component to idle stateenable input ports (creating input port buffers)enable output ports (creating output port buffers)move the component to executing statewhile there is more input dataif there is an empty input bufferput data into the input buffercall EmptyThis Bufferif there is a full output bufferprocess its datacall FillThisBufferwhile there is another full output bufferprocess its datacall FillThisBuffer
The EOS Flag
The client will generally know when there is no more data. It will have finished reading from a file, a socket connection will have closed, and so on. It will need to pass this information into the component so that it in turn can pass it back out to the client to signal that there are no more output buffers.
The client signals this by setting the flag OMX_BUFFERFLAG_EOS in the flags field of the buffer header. Typical code to read data from a file into input buffers, setting EOS on completion, looks like this:
OMX_ERRORTYPE read_into_buffer_and_empty(FILE *fp,COMPONENT_T *component,OMX_BUFFERHEADERTYPE *buff_header,int *toread) {OMX_ERRORTYPE r;int buff_size = buff_header->nAllocLen;int nread = fread(buff_header->pBuffer, 1, buff_size, fp);printf("Read %d ", nread);buff_header->nFilledLen = nread;*toread -= nread;if (*toread <= 0) {printf("Setting EOS on input ");buff_header->nFlags |= OMX_BUFFERFLAG_EOS;}r = OMX_EmptyThisBuffer(ilclient_get_handle(component),buff_header);if (r != OMX_ErrorNone) {fprintf(stderr, "Empty buffer error %s ",err2str(r));}return r;}
When getting information from the output buffers, the client will be watching for that flag. Minimal code to just print the size of output data and exit on end-of-stream is as follows:
OMX_ERRORTYPE save_info_from_filled_buffer(COMPONENT_T *component,OMX_BUFFERHEADERTYPE * buff_header) {OMX_ERRORTYPE r;printf("Got a filled buffer with %d, allocated %d ",buff_header->nFilledLen,buff_header->nAllocLen);// do something here, like save the data - do nothing this time// quit if no more data comingif (buff_header->nFlags & OMX_BUFFERFLAG_EOS) {printf("Got EOS on output ");exit(0);}// otherwise refill itr = OMX_FillThisBuffer(ilclient_get_handle(component),buff_header);if (r != OMX_ErrorNone) {fprintf(stderr, "Fill buffer error %s ",err2str(r));}return r;}
A Hiccup: Port Settings Changed
The little flies in this ointment are these:
Most components need to know at least something about their input data before they can begin to handle it. For example, to handle image data, the image_decode component must be told that the data is in JPEG format (for example).
Even if the component knows the input data format, it won’t know up front how much space to allocate for output buffers. It can’t know this until it has processed at least some of the input data.
The first point will be examined in detail in later chapters. In the following program, just ignore set_image_decoder_input_format for now.
The second problem is dealt with by OpenMAX generating an OMX_PortSettingsChanged event. When this occurs is problematic: it may occur after (or during) the processing of the first block. This seems to occur with JPEG images, for example. But there may need to be a number of blocks read. This seems to occur with H.264 video files.
The IL Client library has two relevant functions to look for OMX_PortSettingsChanged events.
The call ilclient_wait_for_event is a blocking call with an error timeout. This can be used if you know there is (or should be) an event coming within the timeout period. The return value will tell you whether it succeeded.
The call ilclient_remove_event checks the previous event list and returns success if it is able to remove an event from this list. This mechanism waits until the event has already occurred and then detects it.
The blocking ilclient_wait_for_event works on a JPEG image. The second method works on an H.264 movie. I haven’t found the middle ground yet.
To handle the case of a blocking wait, the following changes need to be made to the pseudo-code:
Enable the input ports but not the output ones
Send a single block of data into the component’s input port
Wait for a PortSettingsChanged event on the component’s output port
Enable the output port
Continue emptying and filling buffers
The pseudo-code now looks like this:
create component in loaded state with all ports disabledmove the component to idle stateenable input ports (creating input port buffers)move the component to executing stateput data into an input buffercall EmptyThis Bufferwait for port settings changed on the output portenable output ports (creating output port buffers)while there is more input dataif there is an empty input bufferput data into the input buffercall EmptyThis Bufferif there is a full output bufferprocess its datacall FillThisBufferwhile there is another full output bufferprocess its datacall FillThisBuffer
Example Image Decoding
To make this more concrete, the following program decodes an image using the decode_image component. You aren’t (at this point) interested in decoding images, so ignore the function void set_image_decoder_input_format, which just lets the decoder know the input format (JPEG data).
The program almost follows the pseudo-code of the previous section. The divergence is caused by what seems to be a bug to me. When the client has finished its data, it adds OMX_BUFFERFLAG_EOS. However, if the total data is less than the size of the image_decode’s input buffers, the component doesn’t pass this through. The program hangs. The workaround is to reset the data and send it again.
The program is il_decode_image.c.
The output on a typical JPEG file from, for example, ./il_decode_image jan-us-2006.jpg looks like this:
StateLoaded
StateIdle
Setting image decoder format
OMX error OMX_ErrorSameState
StateExecuting
Read 81920
OMX error OMX_ErrorSameState
Read 81920
Got a filled buffer with 0, allocated 10616832
Read 81920
Read 81920
...
Read 81920
Read 3901
Setting EOS on input
Getting last output buffers
Got eos event
Got a filled buffer with 10616832, allocated 10616832
Got EOS on outputThe program is
#include <stdio.h>#include <stdlib.h>#include <sys/stat.h>#include <OMX_Core.h>#include <OMX_Component.h>#include <bcm_host.h>#include <ilclient.h>void printState(OMX_HANDLETYPE handle) {OMX_STATETYPE state;OMX_ERRORTYPE err;err = OMX_GetState(handle, &state);if (err != OMX_ErrorNone) {fprintf(stderr, "Error on getting state ");exit(1);}switch (state) {case OMX_StateLoaded: printf("StateLoaded "); break;case OMX_StateIdle: printf("StateIdle "); break;case OMX_StateExecuting: printf("StateExecuting "); break;case OMX_StatePause: printf("StatePause "); break;case OMX_StateWaitForResources: printf("StateWait "); break;case OMX_StateInvalid: printf("StateInvalid "); break;default: printf("State unknown "); break;}}char *err2str(int err) {return "error deleted";}void eos_callback(void *userdata, COMPONENT_T *comp, OMX_U32 data) {fprintf(stderr, "Got eos event ");}void error_callback(void *userdata, COMPONENT_T *comp, OMX_U32 data) {fprintf(stderr, "OMX error %s ", err2str(data));}int get_file_size(char *fname) {struct stat st;if (stat(fname, &st) == -1) {perror("Stat'ing img file");return -1;}return(st.st_size);}static void set_image_decoder_input_format(COMPONENT_T *component) {// set input image formatOMX_IMAGE_PARAM_PORTFORMATTYPE imagePortFormat;memset(&imagePortFormat, 0, sizeof(OMX_IMAGE_PARAM_PORTFORMATTYPE));imagePortFormat.nSize = sizeof(OMX_IMAGE_PARAM_PORTFORMATTYPE);imagePortFormat.nVersion.nVersion = OMX_VERSION;imagePortFormat.nPortIndex = 320;imagePortFormat.eCompressionFormat = OMX_IMAGE_CodingJPEG;OMX_SetParameter(ilclient_get_handle(component),OMX_IndexParamImagePortFormat, &imagePortFormat);}OMX_ERRORTYPE read_into_buffer_and_empty(FILE *fp,COMPONENT_T *component,OMX_BUFFERHEADERTYPE *buff_header,int *toread) {OMX_ERRORTYPE r;int buff_size = buff_header->nAllocLen;int nread = fread(buff_header->pBuffer, 1, buff_size, fp);printf("Read %d ", nread);buff_header->nFilledLen = nread;*toread -= nread;if (*toread <= 0) {printf("Setting EOS on input ");buff_header->nFlags |= OMX_BUFFERFLAG_EOS;}r = OMX_EmptyThisBuffer(ilclient_get_handle(component),buff_header);if (r != OMX_ErrorNone) {fprintf(stderr, "Empty buffer error %s ",err2str(r));}return r;}OMX_ERRORTYPE save_info_from_filled_buffer(COMPONENT_T *component,OMX_BUFFERHEADERTYPE * buff_header) {OMX_ERRORTYPE r;printf("Got a filled buffer with %d, allocated %d ",buff_header->nFilledLen,buff_header->nAllocLen);if (buff_header->nFlags & OMX_BUFFERFLAG_EOS) {printf("Got EOS on output ");exit(0);}// do something here, like save the data - do nothing this time// and then refill itr = OMX_FillThisBuffer(ilclient_get_handle(component),buff_header);if (r != OMX_ErrorNone) {fprintf(stderr, "Fill buffer error %s ",err2str(r));}return r;}int main(int argc, char** argv) {int i;char *componentName;int err;char *img;ILCLIENT_T *handle;COMPONENT_T *component;FILE *fp;int toread;OMX_BUFFERHEADERTYPE *buff_header;if (argc < 2) {fprintf(stderr, "Usage: %s jpeg-image ", argv[0]);exit(1);}img = argv[1];if ((fp = fopen(img, "r")) == NULL) {fprintf(stderr, "Can't open: %s ", img);exit(2);}if (( toread = get_file_size(img)) == -1) {fprintf(stderr, "Can't stat: %s ", img);exit(2);}componentName = "image_decode";bcm_host_init();handle = ilclient_init();if (handle == NULL) {fprintf(stderr, "IL client init failed ");exit(1);}if (OMX_Init() != OMX_ErrorNone) {ilclient_destroy(handle);fprintf(stderr, "OMX init failed ");exit(1);}ilclient_set_error_callback(handle,error_callback,NULL);ilclient_set_eos_callback(handle,eos_callback,NULL);err = ilclient_create_component(handle,&component,componentName,ILCLIENT_DISABLE_ALL_PORTS|ILCLIENT_ENABLE_INPUT_BUFFERS|ILCLIENT_ENABLE_OUTPUT_BUFFERS);if (err == -1) {fprintf(stderr, "Component create failed ");exit(1);}printState(ilclient_get_handle(component));err = ilclient_change_component_state(component,OMX_StateIdle);if (err < 0) {fprintf(stderr, "Couldn't change state to Idle ");exit(1);}printState(ilclient_get_handle(component));// must be before we enable buffersset_image_decoder_input_format(component);// input portilclient_enable_port_buffers(component, 320,NULL, NULL, NULL);ilclient_enable_port(component, 320);err = ilclient_change_component_state(component,OMX_StateExecuting);if (err < 0) {fprintf(stderr, "Couldn't change state to Executing ");exit(1);}printState(ilclient_get_handle(component));// Read the first block so that the component can get// the dimensions of the image and call port settings// changed on the output port to configure itbuff_header =ilclient_get_input_buffer(component,320,1 /* block */);if (buff_header != NULL) {read_into_buffer_and_empty(fp,component,buff_header,&toread);// If all the file has been read in, then// we have to re-read this first block.// Broadcom bug?if (toread <= 0) {printf("Rewinding ");// wind back to start and repeatfp = freopen(IMG, "r", fp);toread = get_file_size(IMG);}}// wait for first input block to set params for output portilclient_wait_for_event(component,OMX_EventPortSettingsChanged,321, 0, 0, 1,ILCLIENT_EVENT_ERROR | ILCLIENT_PARAMETER_CHANGED,10000);// now enable output port since port params have been setilclient_enable_port_buffers(component, 321,NULL, NULL, NULL);ilclient_enable_port(component, 321);// now work through the filewhile (toread > 0) {OMX_ERRORTYPE r;// do we have an input buffer we can fill and empty?buff_header =ilclient_get_input_buffer(component,320,1 /* block */);if (buff_header != NULL) {read_into_buffer_and_empty(fp,component,buff_header,&toread);}// do we have an output buffer that has been filled?buff_header =ilclient_get_output_buffer(component,321,0 /* no block */);if (buff_header != NULL) {save_info_from_filled_buffer(component,buff_header);}}while (1) {printf("Getting last output buffers ");buff_header =ilclient_get_output_buffer(component,321,1 /* block */);if (buff_header != NULL) {save_info_from_filled_buffer(component,buff_header);}}exit(0);}
Conclusion
Buffers are the means of communicating data to and from components. This chapter has looked at how to manage buffers. You briefly looked at the OpenMAX calls but spent the most time using the Broadcom IL library.
Resources
OpenMAX IL: The Standard for Media Library Portability: https://www.khronos.org/openmax/il/
OpenMAX IL 1.1.2 Specification: https://www.khronos.org/registry/omxil/specs/OpenMAX_IL_1_1_2_Specification.pdf
Raspberry Pi OpenMAX forum: www.raspberrypi.org/forums/viewforum.php?f=70
VMCS-X OpenMAX IL Components: www.jvcref.com/files/PI/documentation/ilcomponents/
Source code for ARM side libraries for interfacing to Raspberry Pi GPU: https://github.com/raspberrypi/userland