13

Advanced Features

Camera

Smart Interaction

File API

HTML5/CSS3

Summary

The previous chapters already described the basic Samsung SmartTV application features. This chapter will describe more complex additional features such as the camera and SI (Smart Interaction).

Camera

Samsung SmartTV can use both the internal and an external camera. Let's see how the cameras are controlled in an application.

The SDF provides intensive camera-related API guides, including how to set initial position and size of the camera window, that are sufficient for developing an application with the camera feature.

“Fitness VOD” is a good example that uses the camera feature. It offers “Virtual Mirror” that shows a user's action on the screen using the camera feature, so that the user can closely compare a professional coach and herself. The application is available on the Samsung SmartTV App Store.

A user can also control sizes of VOD and the Virtual Mirror (camera) windows with three preset modes.

The SDF's DTV Web Device API provides all camera-related API functions. Unlike the IME or AVPlayer, no complex exception handling is needed to add the camera capability. The API simply supports turning on or off a camera. See below for the list of camera controlling functions.

Checking Camera Status

A camera's current status can be checked using the GetCameraState() function, which returns one of four statuses listed below.

The change in the camera status can be monitored using the RegisterEventCallback function, which returns one of the following three events.

• PL_CAMERA_EVENT_DISCONNECTED: Camera is disconnected
• PL_CAMERA_EVENT_CONNECTING: Camera is connecting
• PL_CAMERA_EVENT_CONNECTED: Camera is connected

Turning on a camera

Use the StartCamVideo() function to start a camera with initial cam video parameters, including distance from the TV screen's top-left edge, the cam video region's width and height, and screen resolution and quality level.

Turning off a camera

Turning off a camera is easier than turning it on since no parameter is necessary. Use the StopCamVideo() function to turn off a camera. The function returns a value that indicates if the camera was successfully turned off.

The next example demonstrates turning on a camera and then turning it off.

if(it == 0)
{

     webapis.camera.StartCamVideo(21,0,968,680, webapis.camera.PL_CAMERA_
RESOLUTION_VGA, webapis.camera.PL_CAMERA_QUALITY_HIGH);
}
else if(it == 1)
{
    webapis.camera.StopCamVideo();
}

Smart Interaction

What is SI (Smart Interaction)?

While a SmartTV's standard input device is a remote controller, the Samsung SmartTV supports additional TV controlling options with voice and (movement) gesture sensing. Samsung calls this technology Smart Interaction.

Preparation for using the SI

The Web Device API library is needed to use the SI. Add the below statement in index.html to enable the application to use the library.

<script type=“text/javascript” src=“$MANAGER_WIDGET/Common/webAPI/1.0/
webapis.js”>
</script>

Checking the SI Capability

Not all Samsung SmartTVs support the SI. Only the 7000 series or higher models that were marketed after 2012 have the SI feature. Therefore, it is vital to check if a SmartTV has the capability before using voice or gesture recognition.

The Web API's Is RecognitionSupported() member function returns if the current SmartTV supports the SI feature.

The SI Voice Recognition

The voice recognition feature starts when a user speaks a preset start command, or presses the voice recognition button on a smart controller. The start command can be changed on the Samsung SmartTV's configuration menu.

The following chart demonstrates how the SI API handles a user's voice recognition command.

Activating Voice Recognition

Once the above preparation is done, set the voice element to “y” in the config.xml file to enable the application of the voice recognition feature.

<voice itemtype=“string”>y</voice>

Register Event

Use the Web API SubscribeExEvent() function to register voice recognition.

Event Unregister

Use the UnsubscribeExEvent() function to unregister an event that was previously registered with the SubscribeExEvent() function.

An application must use the UnsubscribeExEvent() function and unregister registered events on exit. Set up this exception handling in the window.onunload handler.

<body onload=“Main.onLoad();” onunload=“Main.onUnload();”>

Main.onUnload = function(){
      webapis.recognition.UnsubscribeExEvent(webapis.recognition.PL_
RECOGNITION_TYPE_VOICE, ‘testApp’);
};

Event Handling

Samsung SmartTV runs the registered callback function when voice recognition is started. The function receives the event object as its parameter, which returns event type, result, and a few other properties.

The next table lists the event types.

The result value holds the recognized user voice input.

Configuring the Voice HelpBar

The voice helpbar shows available voice commands or status information on the bottom of the Samsung SmartTV screen.

Use the SetVoiceHelpbarInfo() function to control the voice helpbar. The helpbar has two modes—embedded and server guide—and receives string type parameters.

Each helpbar mode uses a different method to process the voice data. The embedded one processes the voice data internally, and tries to match it with one of the registered commands in the helpbar. A maximum of seven commands can be registered.

The server guide mode uses an external voice recognition service. The TV sends the recorded sounds file to the server, and the server returns a recognized result back to the TV. This is a more powerful mode, but it requires an Internet connection.

SI – Gesture Recognition

Before explaining gesture recognition in detail, it will be helpful to start with an example application with the feature. The below is RoyPoy, a Handstudio-developed educational art application.

Warning

The Samsung SmartTV's mouse event handling is similar to a common JavaScript mouse event handling. However, it has a major potential problem of losing the focus through a mouse onClick event. This process needs to be even more carefully handled while using multiple input devices.

Configuration

Once the above preparation is done, set the mouse element to “y” in the config.xml file to enable the application of the gesture recognition feature.

<mouse itemtype=“string”>y</mouse>

Register Event

Use the Web API SubscribeExEvent() function to register the gesture recognition. See the previous voice recognition event registration for details.

Event Unregister

Use the UnsubscribeExEvent() function to unregister an event that was previously registered with the SubscribeExEvent() function. See the previous voice recognition event registration for details.

Event Handling

Samsung SmartTV runs the registered callback function when voice recognition is started. The function receives the event object as its parameter, which returns event type, result, and a few other properties.

The following table lists the event types.

The result value depends on each event type.

Configuring the Gesture HelpBar

The gesture helpbar shows available gestures or status information on the bottom of the Samsung SmartTV screen.

Like the voice recognition, the gesture recognition process depends on the helpbar mode. Use the SetGestureHelpbarInfo() function to control the gesture helpbar.

The parameter helpbarInfo object has the following structure. While the below chart treats it as if it is an object data type to help understanding, it is actually a String data type.

Non-standard mouse events—Slap or Thumbs up—need to be registered in the helpbarItemList to be recognized by the TV. As in the voice recognition, the helpbarItemList has a maximum number of items—four.

File API

The File API is used to permanently store user data (ID, log) to the TV's internal storage (file system) in a SmartTV application. The permanently stored data can be used with File I/O for cache purposes.

All applications share the same directory and can cause collisions by using the same filename as well. To solve this potential problem, assign a subdirectory for the application using its unique IDs.

Let's review the most commonly used functions of the File API using an example.

List of the File API Interfaces

The File API provides the following interfaces.

The File API

Let's review each of the above File APIs in detail.

File API Example

Let's create an example to demonstrate the File API.

Loading the File API

// Obtain a file system instance
var fileSystemObj = new FileSystem();
// Data path
var path;

var load = function(){
    // Confirm if the directory exists
    if (fileSystemObj.isValidCommonPath(curWidget.id) != 1)
    {
        // Make one if none exists
       fileSystemObj.createCommonDir(curWidget.id);
    }

    path = curWidget.id + “/data.dat”;
};

First, create a FileSystem instance and curWidget.id's own directory to use the File API. This example will create the file data.dat in the curWidget.id folder.

Read and Write a File

var read = function()
{
    var result, jsFileObj;

    // Open file
    jsFileObj = fileSystemObj.openCommonFile(path, “r”);

    if (jsFileObj) {
         // Read data if the file exists
        result = jsFileObj.readAll();


         // Close the file
        fileSystemObj.closeFile(jsFileObj);
    } else {
        result = false;
    }
    return result;
};

var write = function(val){
    var jsFileObj;

    // Write a new file
    jsFileObj = fileSystemObj.openCommonFile(path, ‘w’);
    jsFileObj.writeAll(val);

    // Close the file
    fileSystemObj.closeFile(jsFileObj);

    return result;
};

Note that different parameters are used for the openCommonFile() interface and depend on the file IO mode. Running the code on an emulator will create the file in an SDK subdirectory, instead of a TV. The existing folder is located below.

Delete a File

Once the application is deleted, the curWidget.id folder and files in the folder must also be deleted. Add the next line in the config.xml to accommodate this task.

<deleteJS>ResetDB</deleteJS>

The ResetDB is a .js file that must be added in the top of an application. The file must include a logic that deletes all files in the curWidget.id that are created by the File API.

var ResetDB = function() {
      var fileSystemObj = new FileSystem();
      var directory = curWidget.id;

      return {
          reset: function() {
              if (fileSystemObj.isValidCommonPath(directory) != 1) {
                   return true;
              }

              var path = curWidget.id + ‘/data.dat’;
              alert(‘ResetDB.reset: filepath: ’ + path);

              if (fileSystemObj.deleteCommonFile(path)) {
                   alert(‘delete complete: ’ + path);
              }
          }
     };
}();

HTML5/CSS3

The popular HTML5/CSS3 standard is covered by Samsung SmartTV also. Since Samsung SmartTV application development is based on a web browser, the new standard is well supported.

HTML5

The SDF mentioned that 2012 SmartTV models support HTML5. Its own UI components, Slider and Date Picker, are based on the HTML5 technology. Samsung SmartTV openly supports the following HTML5 elements.

The Canvas Tag

The canvas element is used to create dynamic 2D shapes and bit images.

<canvas id=“example” width=“200” height=“200”>
    This text is displayed if your browser does NOT support HTML5 canvas.
</canvas>

Control the created canvas element using JavaScript.

var example = document.getElementById(‘example’);
var context = example.getContext(‘2d’);
context.fillStyle = “rgb(255,0,0)”;
context.fillRect(30, 30, 50, 50);

The Audio Tag

The audio element is used to play music or streaming audio files.

<audio src=“example-sound.mp4”>
    Your browser does NOT support the audio element.
</audio>

The Video Tag

There was no standard method that can embed a movie on a web page. It required an external plug such as Flash or ActiveX. HTML5 now provides a standard interface to play movies.

<video src=“movie.mp4” width=“320” height=“240”>
    Your browser does not support the video tag.
</video>

The following attributes can be configured in a video element.

In addition to the above three elements, Samsung SmartTV supports the following HTML5 elements.

CSS3 Support

Samsung SmartTV also supports CSS3. See below for the details.

3D Transforms

See the below SDF guide for additional information on HTML5 and CSS3.