The previous chapters already described the basic Samsung SmartTV application features. This chapter will describe more complex additional features such as the camera and SI (Smart Interaction).
Samsung SmartTV can use both the internal and an external camera. Let's see how the cameras are controlled in an application.
The SDF provides intensive camera-related API guides, including how to set initial position and size of the camera window, that are sufficient for developing an application with the camera feature.
“Fitness VOD” is a good example that uses the camera feature. It offers “Virtual Mirror” that shows a user's action on the screen using the camera feature, so that the user can closely compare a professional coach and herself. The application is available on the Samsung SmartTV App Store.
A user can also control sizes of VOD and the Virtual Mirror (camera) windows with three preset modes.
The SDF's DTV Web Device API provides all camera-related API functions. Unlike the IME or AVPlayer, no complex exception handling is needed to add the camera capability. The API simply supports turning on or off a camera. See below for the list of camera controlling functions.
A camera's current status can be checked using the GetCameraState() function, which returns one of four statuses listed below.
The change in the camera status can be monitored using the RegisterEventCallback function, which returns one of the following three events.
Use the StartCamVideo() function to start a camera with initial cam video parameters, including distance from the TV screen's top-left edge, the cam video region's width and height, and screen resolution and quality level.
Turning off a camera is easier than turning it on since no parameter is necessary. Use the StopCamVideo() function to turn off a camera. The function returns a value that indicates if the camera was successfully turned off.
The next example demonstrates turning on a camera and then turning it off.
if(it == 0) { webapis.camera.StartCamVideo(21,0,968,680, webapis.camera.PL_CAMERA_ RESOLUTION_VGA, webapis.camera.PL_CAMERA_QUALITY_HIGH); } else if(it == 1) { webapis.camera.StopCamVideo(); }
While a SmartTV's standard input device is a remote controller, the Samsung SmartTV supports additional TV controlling options with voice and (movement) gesture sensing. Samsung calls this technology Smart Interaction.
The Web Device API library is needed to use the SI. Add the below statement in index.html to enable the application to use the library.
index.html
<script type=“text/javascript” src=“$MANAGER_WIDGET/Common/webAPI/1.0/ webapis.js”> </script>
Not all Samsung SmartTVs support the SI. Only the 7000 series or higher models that were marketed after 2012 have the SI feature. Therefore, it is vital to check if a SmartTV has the capability before using voice or gesture recognition.
The Web API's Is RecognitionSupported() member function returns if the current SmartTV supports the SI feature.
The voice recognition feature starts when a user speaks a preset start command, or presses the voice recognition button on a smart controller. The start command can be changed on the Samsung SmartTV's configuration menu.
The following chart demonstrates how the SI API handles a user's voice recognition command.
Once the above preparation is done, set the voice element to “y” in the config.xml file to enable the application of the voice recognition feature.
config.xml
<voice itemtype=“string”>y</voice>
Use the Web API SubscribeExEvent() function to register voice recognition.
Use the UnsubscribeExEvent() function to unregister an event that was previously registered with the SubscribeExEvent() function.
An application must use the UnsubscribeExEvent() function and unregister registered events on exit. Set up this exception handling in the window.onunload handler.
index.html
<body onload=“Main.onLoad();” onunload=“Main.onUnload();”>
main.js
Main.onUnload = function(){ webapis.recognition.UnsubscribeExEvent(webapis.recognition.PL_ RECOGNITION_TYPE_VOICE, ‘testApp’); };
Samsung SmartTV runs the registered callback function when voice recognition is started. The function receives the event object as its parameter, which returns event type, result, and a few other properties.
The next table lists the event types.
The result value holds the recognized user voice input.
The voice helpbar shows available voice commands or status information on the bottom of the Samsung SmartTV screen.
Use the SetVoiceHelpbarInfo() function to control the voice helpbar. The helpbar has two modes—embedded and server guide—and receives string type parameters.
Each helpbar mode uses a different method to process the voice data. The embedded one processes the voice data internally, and tries to match it with one of the registered commands in the helpbar. A maximum of seven commands can be registered.
The server guide mode uses an external voice recognition service. The TV sends the recorded sounds file to the server, and the server returns a recognized result back to the TV. This is a more powerful mode, but it requires an Internet connection.
Before explaining gesture recognition in detail, it will be helpful to start with an example application with the feature. The below is RoyPoy, a Handstudio-developed educational art application.
The Samsung SmartTV's mouse event handling is similar to a common JavaScript mouse event handling. However, it has a major potential problem of losing the focus through a mouse onClick event. This process needs to be even more carefully handled while using multiple input devices.
Once the above preparation is done, set the mouse element to “y” in the config.xml file to enable the application of the gesture recognition feature.
Use the Web API SubscribeExEvent() function to register the gesture recognition. See the previous voice recognition event registration for details.
Use the UnsubscribeExEvent() function to unregister an event that was previously registered with the SubscribeExEvent() function. See the previous voice recognition event registration for details.
Samsung SmartTV runs the registered callback function when voice recognition is started. The function receives the event object as its parameter, which returns event type, result, and a few other properties.
The following table lists the event types.
The result value depends on each event type.
The gesture helpbar shows available gestures or status information on the bottom of the Samsung SmartTV screen.
Like the voice recognition, the gesture recognition process depends on the helpbar mode. Use the SetGestureHelpbarInfo() function to control the gesture helpbar.
The parameter helpbarInfo object has the following structure. While the below chart treats it as if it is an object data type to help understanding, it is actually a String data type.
Non-standard mouse events—Slap or Thumbs up—need to be registered in the helpbarItemList to be recognized by the TV. As in the voice recognition, the helpbarItemList has a maximum number of items—four.
The File API is used to permanently store user data (ID, log) to the TV's internal storage (file system) in a SmartTV application. The permanently stored data can be used with File I/O for cache purposes.
All applications share the same directory and can cause collisions by using the same filename as well. To solve this potential problem, assign a subdirectory for the application using its unique IDs.
Let's review the most commonly used functions of the File API using an example.
The File API provides the following interfaces.
Let's review each of the above File APIs in detail.
Let's create an example to demonstrate the File API.
// Obtain a file system instance var fileSystemObj = new FileSystem(); // Data path var path; var load = function(){ // Confirm if the directory exists if (fileSystemObj.isValidCommonPath(curWidget.id) != 1) { // Make one if none exists fileSystemObj.createCommonDir(curWidget.id); } path = curWidget.id + “/data.dat”; };
First, create a FileSystem instance and curWidget.id's own directory to use the File API. This example will create the file data.dat in the curWidget.id folder.
var read = function() { var result, jsFileObj; // Open file jsFileObj = fileSystemObj.openCommonFile(path, “r”); if (jsFileObj) { // Read data if the file exists result = jsFileObj.readAll(); // Close the file fileSystemObj.closeFile(jsFileObj); } else { result = false; } return result; }; var write = function(val){ var jsFileObj; // Write a new file jsFileObj = fileSystemObj.openCommonFile(path, ‘w’); jsFileObj.writeAll(val); // Close the file fileSystemObj.closeFile(jsFileObj); return result; };
Note that different parameters are used for the openCommonFile() interface and depend on the file IO mode. Running the code on an emulator will create the file in an SDK subdirectory, instead of a TV. The existing folder is located below.
▸ Note: Samsung TV SDK 4/Emulator/(emulator being used)/commonlib/(application name)
Once the application is deleted, the curWidget.id folder and files in the folder must also be deleted. Add the next line in the config.xml to accommodate this task.
<deleteJS>ResetDB</deleteJS>
The ResetDB is a .js file that must be added in the top of an application. The file must include a logic that deletes all files in the curWidget.id that are created by the File API.
var ResetDB = function() { var fileSystemObj = new FileSystem(); var directory = curWidget.id; return { reset: function() { if (fileSystemObj.isValidCommonPath(directory) != 1) { return true; } var path = curWidget.id + ‘/data.dat’; alert(‘ResetDB.reset: filepath: ’ + path); if (fileSystemObj.deleteCommonFile(path)) { alert(‘delete complete: ’ + path); } } }; }();
The popular HTML5/CSS3 standard is covered by Samsung SmartTV also. Since Samsung SmartTV application development is based on a web browser, the new standard is well supported.
The SDF mentioned that 2012 SmartTV models support HTML5. Its own UI components, Slider and Date Picker, are based on the HTML5 technology. Samsung SmartTV openly supports the following HTML5 elements.
The canvas element is used to create dynamic 2D shapes and bit images.
<canvas id=“example” width=“200” height=“200”> This text is displayed if your browser does NOT support HTML5 canvas. </canvas>
Control the created canvas element using JavaScript.
var example = document.getElementById(‘example’); var context = example.getContext(‘2d’); context.fillStyle = “rgb(255,0,0)”; context.fillRect(30, 30, 50, 50);
The audio element is used to play music or streaming audio files.
<audio src=“example-sound.mp4”> Your browser does NOT support the audio element. </audio>
There was no standard method that can embed a movie on a web page. It required an external plug such as Flash or ActiveX. HTML5 now provides a standard interface to play movies.
<video src=“movie.mp4” width=“320” height=“240”> Your browser does not support the video tag. </video>
The following attributes can be configured in a video element.
In addition to the above three elements, Samsung SmartTV supports the following HTML5 elements.
Samsung SmartTV also supports CSS3. See below for the details.
▸ Windows Internet Explorer Test Center: http://samples.msdn.microsoft.com/ietestcenter
See the below SDF guide for additional information on HTML5 and CSS3.
This chapter covered the camera function that supports virtual mirror, the File API that can store permanent data, and the new standard HTML5. There are many other advanced features that the SmartTV supports. An App Store–published application is much more complex than the simple gallery-style app that was described here. The SDF provides detailed guides for the Samsung SmartTV's advanced application features.