The Android operating system is an open source mobile platform which is widely used on smart phones and tablets. Its design is based on the Linux kernel, and its primary user control interface is via a touchscreen. This user interface consists of touch actions, such as swiping, tapping, or pinching elements on the screen. Tapping a virtual keyboard is one of the methods of entering text input. Various types of game control screens allow user interaction similar to joysticks and pushbuttons. The screen interface typically adjusts from portrait display to landscape based on the orientation in which the device is held. Sensors such as accelerometers, gyroscopes, and proximity sensors are usually available on the mobile device platform and used by the application software.
To provide this type of interface for the user of a robot, the ROS Android metapackage has been developed and made available for use and further development. But before we dive into an explanation of this metapackage, we will spend some time playing with the existing Android-TurtleBot application manager, Rocon Remocon.
Android apps have been developed for TurtleBot to perform the following operations:
- Listener
- Teleop
- Map Navigation
- Make a Map
- TurtleBot Follower
- TurtleBot Panorama
In addition to these, more apps could be added in the future. These apps can be downloaded from the Google Play Store on any Android device connected to the Internet. The primary manager for these apps is Rocon Remocon (Indigo version).
The following instructions will lead you through the process of installing the software and running your TurtleBot around the room:
- Downloading the Rocon Remocon app:
On your Android device, select the icon for the Google Play Store (see the following image). In the search box, enter
Rocon Remoconand tap search. From the choices displayed, select the Indigo version of Rocon Remocon. The other TurtleBot apps can be downloaded through the Rocon Remocon app when it is up and running.The following screenshot shows the Google Play Store and Rocon Remocon icons:
Google Play Store and Rocon Remocon icons
- Network connectivity and TurtleBot setup:
Power on the TurtleBot netbook and decide over which Wi-Fi network you wish to connect to the TurtleBot and your Android device. Refer to the Networking the netbook and remote computer section in Chapter 3, Driving Around with a TurtleBot. The
ROS_MASTER_URIand theROS_IPorROS_HOSTNAMEvariables should be set on TurtleBot in the.bashrcfile in order to reflect the TurtleBot's IP address on the chosen network. (Refer to the Netbook network setup section in Chapter 3, Driving Around with a TurtleBot).Also, connect your Android device to the same network. Find the settings for the Wi-Fi network and connect to it. If you are using a router, you may wish to register the URI address for the TurtleBot and your Android device so that they will not change when disconnected and reconnected to the router.
- Configuring the Rocon Remocon app:
Tap on the Rocon Remocon icon to open it and select the ADD A MASTER button. As shown in the following screenshot (left side), the master URI of the TurtleBot should be added to replace the
localhostportion of thehttp://localhost:11311/URI. A button SCAN THE LOCAL NETWORK will search for the TurtleBot, but this will only work if the TurtleBot is up and running with minimal launch.When you have entered the TurtleBot's URI, press the ENTER button and you should see an image similar to the image shown on the right side of the following screenshot. A triangular warning symbol will appear because TurtleBot is not up and running yet.
The TurtleBot's URI information will be saved by the app and will be available for use each time the app is started. If this URI is always registered to the TurtleBot, then this setup is only required once:
Rocon Remocon screens
- Launching TurtleBot:
Now it is time to bring TurtleBot to life! Power on the TurtleBot base, open a terminal window on the netbook, and type the following command:
$ roslaunch turtlebot_bringup minimal.launchAfter TurtleBot has played its little bringup tune, you can close the lid on the netbook. All other operations will be done on the Android device.
- Android control:
On your Android device, press the REFRESH button and see the Turtlebot icon appear in the place of the triangular warning symbol. Next, touch the bar where the Turtlebot icon and the URI are displayed. A pop-up window should appear that has Choose your role heading displayed with options for Android Pairing and Documentation. Select the circle next to Android Pairing and the following screen image should appear. This screen shows all the apps currently available for the Turtlebot Remocon. (The number and arrangement of your icons may differ.):
TurtleBot – Android Pairing screen
- Android Teleop:
Now we will try the Teleop app by touching the Teleop icon. A pop-up window should appear that confirms that you are using Teleop with an Android device: Lovely, you are now allowed to launch this interaction. Touch the Launch button at the bottom.
If you are successful, a screen similar to the following screenshot should appear. The camera image will be different, of course. If you see a pattern of nine dots, be patient and wait for the camera sensor of the TurtleBot to start transmitting images. If an image does not appear after a few seconds, check the connections to your vision sensor to make sure the sensor is installed properly.
The following screen should appear after you press the Launch button:
Teleop screen
Now you can drive the TurtleBot around by placing your finger inside the circle on the right and sliding it in the direction you want TurtleBot to go. Remember, you are controlling TurtleBot as if you were standing behind it. If you swipe your finger to the bottom of the screen, TurtleBot will back up and swiping-to-the-left will cause it to rotate left. At the top of the screen are monitors for the battery levels for the robot base and the netbook.
When you are ready to shut down the app, touch the Back button on the Teleop screen and the Leave button on the Android Pairing screen. Then, you should shut down the apps from one of your main app managers or the menus on your device.
Exploring the rest of TurtleBot Remocon apps is left to you. Some troubleshooting help is provided, in case you get stuck.
The following troubleshooting procedures are provided to assist with getting your Turtlebot Remocon apps up and running:
- Connection Failure:
If the Connection Failure message is displayed on the Rocon Remocon screen, the Remocon app was not able to connect with the TurtleBot. Here is a list of things to check:
- TurtleBot's base should be powered on
- TurtleBot's netbook should be powered on and connected to the network shared by the TurtleBot and Android device
- TurtleBot's
.bashrcshould identify the properROS_MASTER_URIandROS_IPorROS_HOSTNAMEvariables - TurtleBot should be running
minimal.launch - The Android device should be connected to the network shared by Android and the TurtleBot
- The TurtleBot's URI should be entered as the master URI in Rocon Remocon
- Failed to start the paired rapp:
This message appears when another Turtlebot Remocon app is running or has not been shut down properly. For this error, you should tap the Cancel button and wait for the app that is running to shut down.
If an app is not responding, it is best to shut it down entirely and start over.
- Other troubleshooting method:
For additional troubleshooting help, access the tutorial at How to Run Turtlebot Android Application, which can be found at http://wiki.ros.org/turtlebot/Tutorials/indigo/Android%20Interactions/How%20to%20Run%20Turtlebot%20Andorid%20Application.
Now we can develop an Android interface for any of our ROS robots. The next section will describe how to interface an ROS robot with an Android device and develop a user control interface unique to the robot.
If you are interested in developing additional apps for the TurtleBot similar to the ones in the previous section, refer to the TurtleBot Android Apps Dev Tutorials at http://wiki.ros.org/turtlebot/Tutorials/indigo/Android%20Interactions/Turtlebot%20Android%20Apps%20Dev%20Tutorials.
In this section, we begin by downloading the software to become an Android developer using ROS.
There are two non-ROS software packages to download in order to become an ROS-Android developer. First, download the Oracle Java Standard Edition Development Kit (JDK). This JDK is recommended over the openjdk Java development kit at this time due to problems with performance and the user interface. Download the Oracle JDK from http://www.oracle.com/technetwork/java/javase/overview/index.html.
A JDK is a development environment for building Java applications, applets, and components. After the .tar.gz archive file is downloaded, you may unpack the tarball in a location of your choice. We have chosen to install it at the system level in the /opt directory where administrator privileges are necessary to add these directories.
Verify the software version is 1.8 (or later) using this command:
$ javac -version
Next, you will need Android Studio, which is the official integrated development environment (IDE) for Android. Android Studio provides all the tools to build apps for every type of Android device. Download the Android Studio package as a ZIP file from http://developer.android.com/sdk/index.html.
Scroll to the bottom of the web page for the Linux platform distribution.
It is recommended that you install this package also in the /opt directory (as /opt/android-studio). Next, run the setup script using the following command:
$ /opt/android-studio/bin/studio.sh
A setup wizard will guide you through the setup process. When prompted to import previous Android Studio settings, you may select I do not have a previous version of Studio and click on OK. Select the Standard installation type. The Android SDK components will be downloaded as part of the setup process.
For systems that run a 64-bit version of Ubuntu or if the message Unable to run mksdcard SDK tool is displayed on the screen, the following 32-bit libraries will need to be installed using the command in a terminal window:
$ sudo apt-get install lib32z1 lib32ncurses5 lib32bz2-1.0 lib32stdc++6
Execute the following two commands to add Android Studio to your environment variables:
$ echo export PATH=${PATH}:/opt/android-sdk/tools:/opt/android-sdk/platform-tools:/opt/android-studio/bin >> ~/.bashrc $ echo export ANDROID_HOME=/opt/android-sdk >> ~/.bashrc
With the two non-ROS packages installed, the ROS metapackages for rosjava and android are needed. The installation of these metapackages is described in the upcoming section.
Our instructions for creating an ROS-Android development environment will be to use a Debian installation of rosjava with a catkin workspace built on top. To install the rosjava metapackage, type this command:
$ sudo apt-get install ros-indigo-rosjava
Next, to create an empty catkin workspace overlay on top of your ros-indigo files, type the following commands:
$ mkdir –p ~/myjava/src $ cd ~/myjava/src $ source /opt/ros/indigo/setup.bash $ catkin_init_workspace $ cd ~/myjava $ catkin_make
If you have your own Java source code and you wish to use it in this workspace, refer to further instructions at http://wiki.ros.org/rosjava/Tutorials/indigo/Deb%20Installation.
You can also start from here and create your own rosjava packages. Follow the tutorials at http://wiki.ros.org/rosjava_build_tools/Tutorials/indigo/Creating%20Rosjava%20Packages.
For Android development, we continue by installing the ready-made core android libraries. Use the following commands to create an android workspace, which overlays the rosjava workspace created previously:
$ mkdir –p ~/myandroid $ wstool init -j4 ~/myandroid/src my_android_sources.rosinstall $ source ~/myjava/devel/setup.bash $ cd ~/myandroid $ catkin_make
Other options for installing the source code for ROS Android core libraries can be found at http://wiki.ros.org/android/Tutorials/indigo/Installation%20-%20ROS%20Development%20Environment.
Knowing the following terms should help as you learn more about the rosjava and android development environments:
Ant: An Apache Ant is a Java library and command-line tool used to build Java applications. Developers either build antlibs that contain Ant tasks and types, or they have access to numerous ready-made antlibs. Apache Ant is a term trademarked by The Apache Software Foundation, which provides support for open source projects.
Gradle: Gradle is an automated build system that combines with Ant and Maven to offer a method of declaring a project's configuration identifying the order of its build tasks. The system can handle large multiproject builds and supports incremental builds for only the portion of a project that is dependent on what has been changed.
JAR: A Java Archive (JAR) is the package format that is used to combine Java class files and the metadata to be distributed as a software application or library. JARs are compressed into ZIP file format for archiving purposes.
Maven: As a software project management and compression tool, Maven manages the state of a project's development, including its build, reporting, and documentation aspects. For our purposes, a repository in Maven will be used to hold build artifacts and dependencies for ROS-Android applications. Apache Maven is another open source project of The Apache Software Foundation.
The division between the /myjava and /myandroid workspaces is strategic to the development. In the /myjava workspace, you can create and build custom rosjava message jars and artifacts. You can also use this space to build and test rosjava algorithms for use in your Android applications. To dive into rosjava development, refer to the list of tutorials (for ROS Indigo) at http://wiki.ros.org/rosjava.
The /myandroid workspace contains the official ROS android stacks as follows:
android_core: This package contains Damon Kohler's core libraries and examples on Android developmentandroid_extras: This package consists of various extra functionalities for peripheral devices (such as a hokuyo scanner) or utilities (such as imaging qrcodes)android_apps: This package includes development libraries and apps for robot pairing (as with TurtleBot)android_remocons: This package contains remote control clients for appable robots
The following diagram highlights the dependencies between the rosjava and android libraries:
From this point, you are ready to begin with the creation of Android packages using rosjava scripts. To begin with, tutorials on creating Android packages and applications are available at http://wiki.ros.org/rosjava_build_tools/Tutorials/indigo/Creating%20Android%20Packages.
For a complete list of ROS Android tutorials, refer to the list of tutorials (for ROS Indigo) at http://wiki.ros.org/android.