Client applications are installed onto mobile wireless devices such as mobile phones. They are able to provide users with more functionality, better integration, and better performance than browser applications. Client applications may be written in a portable programming language (e.g., J2ME or FlashLite) or written as native code (e.g., Symbian C++ for Nokia phones).
Client applications can be divided into two categories: portable applications and native applications. Portable applications run in (or on) a virtual machine, such as the Java Virtual Machine. Native applications are written to run directly on particular architecture or platform of the target devices; for instance Nokia's Series 60 3rd edition platform.
Portable applications are written to run on an idealized platform, which should provide a consistent environment regardless of the actual details of the physical device. They are therefore less likely to have access to the latest and greatest features offered by a particular device. However, one application should run, essentially unchanged, on hundreds of different devices and models.
Native applications, in contrast, do have access to specific features provided by a physical device, and have access to things like the underlying file system, the contacts list, etc. They tend to run faster, but distinct, custom versions of an application need to be written to suit different target platforms. A target platform may be broad or narrow depending on the amount of integration with particular features provided by the devices. For instance, a target platform for Nokia phones could be “Series 60” that covers around 50 models, or subdivided into “Series 60 1st edition,” “Series 60 2nd edition,” and “Series 60 3rd edition” depending on whether the software needs to use specific features offered by later versions of the handset models.
Sun was one of the first companies to try to provide a standard platform across many mobile devices where a program written and compiled for that platform can be installed on any of those devices. Sun's platform is known as Java Micro Edition (Java ME). Until recently Java ME was called Java 2 Micro Edition (J2ME), which is still the more common term for the platform and programming language.
Subsequently, there have been several other “portable” platforms for mobile devices including:
I will limit my discussion to Java ME as I have no experience of automated testing for the other platforms.
Although Java ME is intended to be a common, consistent platform, in practice many of the features were optional and manufacturers did not implement all of them, and some features were implemented partially. Many features are documented through Java Specification Requests (JSRs), e.g., JSR75 for access to the file system.
The software is bundled as a JAR file, together with a JAD file that describes the JAR file, e.g., the file size and the names of the main class. In practice a JAR file's internal structure is similar to a zip file and can be opened with the same software utilities.
Java ME is a reduced version of Java (standard edition). Java ME lacks some of the features of Java for development and debugging. For example, many devices offer only limited stack traces, and custom class-loaders are not available.
Native applications can take advantage of all the features offered by particular phone models, e.g., with very well integrated user interfaces, access to contacts, the camera, etc.
Native applications are often developed in C/C++ or a similar low-level programming language. Binary Runtime Environment for Wireless (BREW) is also considered a native platform, which is implemented on a relatively small number of devices.
The compilers and development environments are more demanding than the equivalent tools for J2ME applications. For instance we install at least four heavyweight SDKs, an emulator, and Visual Studio in order to build an application for Windows Mobile phones.
Some of the software tools used to build native applications are easy to incorporate into automated builds and testing. However, others require significant effort just to run and require additional checking after the event to find out whether something went wrong or not.
Typically, testing client applications are done interactively, even when the tests are automated for various reasons, such as:
In terms of an automation strategy for client applications, we combine automation with manual (human) checking, particularly for on-screen rendering.
Custom test applications are a good way of learning about the key characteristics of a device or its virtual machine environment (e.g., for Java ME). These can often run unattended and report results directly to a server for processing and analysis.