Survey, Comparison and Evaluation of Cross

Platform Mobile Application Development Tools

Isabelle Dalmasso, Soumya Kanti Datta, Christian Bonnet, Navid Nikaein
Mobile Communication Department, EURECOM
Sophia Antipolis, France
{dalmasso, dattas, bonnet, nikaeinn}@eurecom.fr

Abstract—Mobile application development is becoming more SDK, user experience, stability of framework, ease of
challenging with diverse platforms and their software updating, cost of development for multiple platform and time
development kits. In order to reduce the cost of development and to market an app. Most of the developers would like to release
reach out to maximum users across several platforms, developers apps for major mobile platforms (iOS, Android) and provide a
are migrating to cross platform application development tools. In
consistent user experience (UX) across the platforms.
this paper, we provide several decision criteria beyond the
portability concerns for choosing suitable cross platform tool for Developing an app for separate mobile platforms require in-
application development. The desirable requirements in a cross depth knowledge of them and their SDKs. This increases the
platform framework are identified. A general architecture for cost of development, ease of updating and time to market an
cross platform application development is discussed. Then a app. This is where the cross platform development tools come
survey of several write once run anywhere tools (PhoneGap, into picture. Table I provides an in-depth comparison among
Titanium, Sencha Touch) are provided along with a classification native, mobile web and cross platform app development
and comparison among the tools. To examine the performance in approaches.
terms of CPU, memory usage, power consumption, Android test
applications are developed using such tools. It is found that
PhoneGap consumes less memory, CPU and power since it does
Cross platform tools (e.g. PhoneGap, Titanium, Rhomobile)
not included dedicated UI components. Finally the paper allow implementing an app and its user interface (UI) using
summarizes the contributions and concludes with some future web technologies like HTML, CSS. Then the app can be built
directions. for several mobile platforms (e.g. iOS, Android, Windows
Phone 7, BlackBerry). This process is helpful only when a
Keywords-mobile development; cross platform tools; developer is willing to compromise user experience and more
PhoneGap; Titanium; Android. importance is given to launching of the app in several
platforms to reach to maximum users. This approach allows
I. INTRODUCTION developing app for multiple mobile platforms at the same
The landscape of mobile platforms has seen major time. Thus the cost of development and time to market the app
evolution in recent past. While BlackBerry, Bada and is reduced.
Symbian failed to reach out to the masses, iOS and Android
TABLE I. DECISION FACTORS
have won the war of mobile platforms. In the era of Decision Native Mobile web Cross platform
smartphones and tablets, mobile applications are providing criterion approach approach approach
added value to several industries including transport, Quality of Excellent Very good Not as good as
ecommerce, net banking, travel, retail and enterprise services. UX native apps
Quality of High Medium Medium to low
Developers are exploiting the state-of-the-art functionalities of
apps
the smart devices to offer revolutionizing user experience. In- Potential Limited to a Maximum Large - as it
turn they are becoming the engine for innovation. Thus it is of users particular including reaches to users
prime importance for a mobile platform provider to attract mobile smartphones, of different
more and more developers in order to boast external platform tablets and other platforms
feature phones
investment and revenue via them. Not only the mobile App High Low Medium to low
platform owners and handset manufacturers but also network development
service providers and chipset makers are investing heavily to cost
develop and release software development kits to reach out to Security of Excellent Depends on Not good
app browser security
the developers. Also there are several tool vendors like cross- Supportabili Complex Simple Medium to
platform tool vendors (PhoneGap, Titanium), app diagnostic ty complex
tool vendors (BugSense) and more who are also trying to Ease of Complex Simple Medium to
catch the attention of mobile application developers. updating complex
Time-to- High Medium Short
market
But the diversity of mobile platforms and the variety SDKs App Yes Yes Yes
and other tools pose unique challenges. They include choice of extension

The research is sponsored by the French project WL-Box 4G Pole SCS.

978-1-4673-2480-9/13/$31.00 ©2013 IEEE 323

 This paper presents several criteria beyond portability  Accessing built-in features: The tools must be able to
concerns to choose an appropriate cross platform tool for access the built-in features of a smart device. Use of
development. Several such tools are present at the moment. We camera, sensors, geo-localization and more features
have put forward the requirements of cross platform framework helps to provide better user experience.
and the high level architecture of the same. In-depth survey of
several such Write Once Run Anywhere (WORA) enlightens  Open source: It attracts more of application
about the API & documentation, development environment, developers and the developer community can
deployment, advantages and weakness. To evaluate the participate in bug-fixes and further development. It is
performance of such tools, we have developed Android apps to be noted that this is not a technical requirement.
with four such tools and measured the CPU usage, memory
usage and power consumption. During the test, it is found that III. GENERAL ARCHITECTURE OF CROSS PLATFORM
PhoneGap consumes less CPU, memory and power than other APPLICATION DEVELOPMENT
tools. But the app developed using PhoneGap does not have a This section provides a general architecture of cross
very good UI. To create a better UI, additional tools are platform application development portrayed in Figure 1.
required. In that case, Sencha 2.0 stands out among others.
The rest of the paper is as follows. Section II outlines the
basic requirements of a cross platform development
framework. Section III portrays a general architecture of cross
platform mobile application development. Section IV presents
a detail survey of WORA tools. Section V classifies WORA
tools and provides a comparison among the tools. Section VI
demonstrates the performance testing results. Finally the paper
concludes with some future directions.

II. REQUIREMENTS OF A CROSS-PLATFORM FRAMEWORK

We have identified the desirable requirements of any cross-
platform framework as stated below:
 Multiple mobile platform support: The framework
must support several mobile platforms. Support for
Figure 1: General architecture of cross platform mobile application
Android and iOS are very essential since they have the development.
largest share in the application markets.
The application developer implements the business logic or
 Rich user interface: Currently the cross platform tools the application functionalities using web technologies. The
can not provide rich user interface (UI) as native apps. cross platform framework allows implementing user interface
Since the success of an application highly depends on and access storage facility and device features (sensors, camera
user experience of the interface, rich UI development and contacts) which interacts with a JavaScript API. The API
should be incorporated. Support for sophisticated will in turn interact with the native API of a mobile platform.
graphics (2D, 3D), animation, multimedia are The application is then built separately to generate the
necessary. executables for different platforms. The APIs for the mobile
 Back-end communication: Mobile devices promote platforms actually allow generating the respective application.
an "always-connected" model in which the users are Thus the generated application can be run in corresponding
sharing material in social networking sites, watching mobile device.
videos, communicating via live chat, gathering
information 24X7. There smooth support for backend IV. SURVEY OF WRITE ONCE RUN ANYWHERE TOOLS
communication protocols and data formats are absolute There are several WORA tools available in the market e.g.
mandatory. PhoneGap, Titanium, Rhomobile, JQuery Mobile and more.
 Security: Applications developed by cross platform This paper presents a survey of these tools to provide a quick
tools are not highly secure. Proper research needs to be overview to application developers.
carried out to secure the tools and applications.
A. PhoneGap
 Support for app-extensions: It is required to install
The PhoneGap open source framework provides a decent
app extensions on top of existing applications like in-
app purchase/billing capability. toolbox for building native mobile applications using only
HTML, JavaScript and CSS [1], [11], [12], [13]. It’s quite
 Power consumption: It is an important issue now-a- popular among users mainly because of its flexibility,
days with thousands of smartphones and tablets are straightforward architecture and ease of use.
being activated daily. The generated applications must  API & Documentation: It offers a pure JavaScript /
be optimized for power. HTML / CSS API (Webkit framework) together with

324
 a library wrapping material's resources. PhoneGap E. Application Craft
also provides documentation for the proprietary API. It is a cloud based development environment for mobile,
 Environment Dev/Build: It provides a Eclipse plug- tablet and desktop. Application Craft permits to design a web
in but can also be used from other IDEs to build application by connecting the platform through a browser,. The
mobile applications. code generated is stored to the servers, with private/public
 Deployment: The binary file generated at the build address to run the application. The code generated is
cannot be published on any mobile application downloadable.
market. The developer obtains the final release
through the pricing PhoneGap Build service. F. Appcelerator Titanium Studio 2.0
 Framework stability: It is a mature framework.  API & Documentation: It provides a rich API and
 Advantages: All native wrapper source code is low level objects like TCP Sockets. UI objects are
provided so it can be customized further. Broad customizable through a JavaScript API. There is no
ranges of platforms are supported by PhoneGap. HTML and CSS coding here. The tools presents good
Apps are built purely in HTML, JavaScript and CSS API documentation but the SDK tools usage
allowing web developers to adopt the tool easily. (compiler etc) are not exposed [2].
 Weakness: There is lack of support for native UI  Environment Dev/Build: The IDE Titanium Studio
components, design patterns and dev tools. However, (based on Eclipse IDE) is mandatory. It embeds the
the developers are free to combine PhoneGap with SDK and tools. The generation to native code then
another tool like JQuery or Sencha to produce a the compilation and packaging to a native application
better UI for the apps. is made thanks a python script (for Android).
B. Sencha Touch 2.0  Framework stability: It is a mature framework.
 Advantages: The native code output is very quick
 API & Documentation: Sencha Touch 2.0 is a and smoothly executes on mobile devices. The setup
powerful yet a complex framework. The HTML code is easy for new developers. The documentation is
is generated by JavaScript APIs. In theory Sencha excellent and potentially supports tablet
SDK Tools are self-contained development platform development.
providing an access to a subset of phone native API  Weakness: It has restrictive APIs and small set of
(e.g. camera, notification, connection, orientation). It phones are supported.
also offers to build native packaging deployable on
iOS and Android application markets [4]. V. CLASSIFICATION OF WRITE ONCE RUN ANYWHERE
 Framework stability: It is a mature framework. TOOLS
 Alternatively Sencha may be installed as JavaScript/ In this paper we concentrate on WORA tools as they
CSS resources of a PhoneGap project. This is a provide the opportunity to implement an application for several
workaround but permits to develop the UI using mobile platforms. Efforts have been made to broadly
Sencha API. categorize the tools based on the functionalities they provide.
C. PhoneGap + Sencha Touch 2.0 Five high level factors are listed that can guide developers to
select a possible category.
 API & Documentation: : A complete JavaScript
framework with MVC concept and complete API A. Application development tools
documentation are available for developers.  Hybrid App: This category of tools provides a
 Environment Dev/Build: Uses imported main CSS/ platform-specific shell application which has the
JavaScript files. capability of rendering pre-packaged HTML pages. It
 Framework stability: It is a mature framework. extends the HTML capabilities through APIs which
allow access to device specific features. Some Tools
D. PhoneGap + JQuery mobile include libraries to render platform specific UI.
JQuery mobile is a light-weight API and development of AppMobi and PhoneGap are such category tools.
any application querying web services is quite easy [3].  Mobile Web: These tools are primarily JavaScript
 API & Documentation: JQuery Mobile is built on libraries which in combination with suitable HTML5
top of JQuery API and offers a JavaScript library of and associated CSS are rendering your mobile website
CSS and other components. Good documentation of on different types of device. Some of these tools can
the mobile API allows developers to familiarize with work in conjunction with Hybrid App tool and the
the tools quickly. result can be packaged into a native application. For
 Environment Dev/Build: The environment uses example, JQTouch, JQurey Mobile and Sencha Touch.
imported CSS and JavaScript file for a project and
 Generator: These are tools where a developer writes
builds as any regular Android application.
the application in a specific language (JavaScript or
 Framework stability: It is a mature framework. PHP) and the tool translates it into a deployable native
application for different specific platforms. The

325
 deployable application may include a runtime engine Some applications developed by Titanium are: Legoland
or a virtual machine. The main difference between a Parc California (multimedia /entertainment / ecommerce),
Generator and a Hybrid App is the potential Ebay Corporate (business). New York Senate (multimedia) and
performance gain using the generated native code ZipCar (ecommerce).
instead of translating HTML for graphic objects VI. PERFORMANCE EVALUATION
access. Appcelerator and Rhomobile[9] are typical
examples of such tools. To examine the performance, Android test applications
have been developed using cross platform method. They have
B. User Interface Tools the following features and can be categorized as a part of a
 Visual Tools: They provide a visual interface where 'business' type application.
elements / widgets are dropped into the screen and  The UI presents buttons, each of which has request
the internal application plumbing is taken care by the type. When clicking on a button the application queries
tool. The result, depending on the tool is either a free web services (available on the internet) over
native application or a mobile website. Some AJAX, REST and SOAP technologies.
visualization tools may be used in conjunction with
 Parse and display the different formats of responses:
application development tools (mainly the Hybrid
Text, XML, JSON.
App tools). Dragon rad [5], Application Craft [6],
July Systems [7], NetBiscuit [8] are some examples The applications are installed and tested on an Archos tablet
of visual tools. running Ice Cream Sandwich (Android 4.0.4). The testing
environment consists of Linux Fedora, Eclipse IDE Indigo
C. Guildeline for developers Classic, web development Plug-ins and Android ADT 16.
We provide a guideline for developers to select an above Figure 2 depicts the user interfaces of the test applications. It is
mentioned category. to be noted that no measure has been taken to make a
sophisticated user interface and the default version is adopted.
 Native user experience: If the resulting application is
desired to have user experience similar to native The above applications are representative of mobile
applications, then "Generator" must be chosen. When machine-to-machine applications. Within the framework of the
user experience can be compromised, any other project WL-Box 4G [19], we are developing mobile
category is sufficient. applications that receive data from various sensors via a device
gateway. The data is processed on-the-fly and the UI of the
 Offline/online usage: Mobile web applications only applications are updated.
allow online usage through a browser. Generator and
hybrid apps create installable applications which
A. Memory usage
permit offline usage.
It is found that such type of applications are developed in
 Compatibility: The mobile devices are evolving very Titanium. But when the 'Guidelines for developers' are
fast in terms of hardware, mobile platforms and considered, the choice points to PhoneGap because of the
SDKs. If an cross platform application is desired for a application requirement. So the application is developed using
long term, mobile web is the least risky as mobile the following tools and then the memory & CPU usage and
devices are all supporting HTML5 and newer power consumption are noted.
versions.
 Only PhoneGap.
 Limited access of built-in features: In this case,
hybrid apps can be preferred. But if applications  PhoneGap & JQuery mobile
require frequent access of built-in features, generator  PhoneGap & Sencha Touch 2.0
should be chosen although native approach will more
suitable.  Titanium
 Security: Hybrid and generator are likely to be more The memory usage information is obtained from DDMS
secure than pure mobile web applications. Better tool of ADT.
security can be offered by native approach.
 Proportional set size (PSS): PSS is the amount of
D. Types of applications developed using WORA Tools memory shared with other processes, account in a way
Several types of mobile applications are generated using the that the amount is divided evenly between the
WORA tools: business, games, multimedia, tools, processes that share it. This is memory that would not
entertainment, ecommerce and social. be released if the process was terminated, but is
indicative of the amount that this process is
Some applications developed by PhoneGap are: Wikipedia “contributing” to overall memory load.
(multimedia), BBC Olympics (multimedia), UnTapp (social),
Facebook (social) and Zinga Game - Mafia Wars Mobile Game  Unique set size (USS): USS is the set of pages that
(game). are unique to a process. This is the amount of memory
that would be freed if the application gets terminated.

326
 Table II provides the memory usage metrics. TABLE III. CPU USAGE METRIC
Developed app CPU usage from CPU usage from 'top'
TABLE II. MEMORY USAGE METRICS snapshot approach command approach
Developed app PSS USS PhoneGap + 81.92771% Max: 10%
HTML + CSS
PhoneGap only 12091 6036 tools Min: 0%
PhoneGap + Jquery Mobile 14730 9424 Average: 2%
PhoneGap + Sencha Touch 2.0 24526 20164 PhoneGap + 80.26316% Max: 42
JQuery + HTML
Titanium 17500 8676 Min: 0%
Average: 10%
From the above table it is clear that the application written PhoneGap + 44.0% Max: 32%
in PhoneGap only has smallest PSS and USS value. This is due Sencha Touch 2.0
Min: 0%
to the fact that PhoneGap by design does not use any styling
element or tools for betterment of UI. So we can conclude that Average: 8%
the memory usage will increase with the addition of features to
generate better UI. C. Power consumption
PhoneGap integrated with JQuery Mobile or Sencha Touch Power consumption of mobile applications has received
2.0 is a complete environment for better UI development for much attention of the researchers recently [14], [15], [16]. To
application. In this case the memory usage increases with the effectively use the battery of mobile devices, the apps
use of HTML and JavaScript files. developed using the cross platform tools should be power
efficient. We have measured the power consumption of the
Titanium comprises of a full SDK and the memory usage is same apps using "Power Tutor' [17], [18]. It is a very popular
thus on the higher side. Android app that reports power consumption of individual apps
installed in a mobile device. Table IV tabulates the results. It is
B. CPU usage to be noted that the reported values are average power
To measure the CPU usage,, the apps are developed using consumption of the apps.
PhoneGap and other tools. The code segments to measure the TABLE IV. POWER CONSUMPTION METRIC
CPU usage are added to the Android activity through Developed app Power consumption (mW)
PhoneGap. But no app could be developed using Titanium as it
does not allow to add the activity. A plug-in could be PhoneGap + HTML + 107
CSS tools
developed for this purpose but that is time consuming. We
have followed the two different avenues to record CPU usage PhoneGap + JQuery + 168
as follows: HTML
PhoneGap + Sencha 120
 The first approach takes a CPU snapshot at each state Touch 2.0
of the Activity life cycle (i.e. onCreate, onStart,
onPause, onStop and onDestroy) of the apps.
 The other approach is to read a 'top' result every second Again the result points out that the first app consumes least
during the whole life cycle of the apps. Then an power among the three apps. The reason is attributed to the fact
average for each state of the Activity life cycle is that the UI is very simple. The app developed with PhoneGap
computed. and Sencha Touch 2.0 also works efficiently since it consumes
120mW power.
The developed apps perform the same functionalities as the
previous test. Table III portrays the results of the test. From the above evaluations, it is quite clear that PhoneGap
and Sencha Touch 2.0 work efficiently in terms of CPU usage
The values obtained from CPU snapshot approach is and power consumption. But currently there is no leader cross
computed when the app is doing much computation and is thus platform development tool in the market that fits all the general
represented using very high values. Also it is to be noted that requirements. However the most popular tool among the
these values may vary a lot from a millisecond to another since developers are PhoneGap and Titanium. We identify the main
they are snapshot at CPU usage for very short amount of time. differences between them in the Table V.
The values obtained from 'top' result shows varying CPU The above results are also analyzed with respect to the
usages. The min value is always 0 as - once the app fetches the requirements of cross platform tools mentioned in Section II.
requested page and shows it, the app does not use any CPU
waiting for the next system input. The average value is  The chosen tools are open source and can build
computed using the total elapsed time. From this approach, it is applications for multiple operating systems. This paper
clear that the first app utilizes very less CPU but the user describes the performance of the Android applications
experience is not very sophisticated. When Sencha Touch 2.0 and iPhone applications will be developed and
is used along with PhoneGap, the CPU usage is more but the evaluated in future works.
user experience is significantly better.

327
  PhoneGap does not contain sophisticated UI building In future we are targeting to identify several other
tools. Thus the memory, CPU and power requirements performance test cases. At the same time, the other cross
are less than PhoneGap combined with JQuery or platform tools need to be examined using the memory, CPU
Sencha Touch 2.0. and power consumption metrics described in this paper.
Applications will be developed for iOS platform. Further
 Back-end communications are supported by all the experiments will be carried out to examine the security and app
tools. extension capabilities of the cross platform tools. One
 Rest of the requirements like security, app-extension important issue that has not been addressed yet is access to
will be evaluated in future work. context information from sensors as current mobile computing
heavily depends on it. Such information mining using cross
TABLE V. DIFFERENCE BETWEEN PHONEGAP AND TITANIUM platform approach has to be researched further.
PhoneGap Titanium
REFERENCES
 A PhoneGap application is  A Titanium application is
written in HTML5 and runs in written in JavaScript and [1] http://phonegap.com
an native container. complies into a native [2] http://www.appcelerator.com
application and utilizes native [3] http://jquerymobile.com
 Developers can utilize HTML5, controls.
CSS and JavaScript also. [4] http://www.sencha.com
 It is not an HTML5 application [5] http://dragonrad.com
 The UI elements may be running in a web container.
rendered using JQuery Mobile, [6] http://www.applicationcraft.com
Snecha Touch or some other  By design, this tool can provide [7] http://julysystems.com
JavaScript web mobile better performance and better [8] http://www.netbiscuits.com
development framework. user experience.
[9] http://www.rhomobile.com
[10] http://www.appmobi.com
VII. CONCLUSION [11] https://github.com/remy/PhoneGap-Plugin-WebSocket
In a nutshell, we have concentrated on cross platform [12] https://github.com/phonegap/phonegap-
development tools since they build apps for several platforms plugins/tree/master/iPhone/InAppPurchaseManager
and the development cost and time to market are less. Although [13] https://github.com/anismiles/websocket-android-phonegap
the user experience is not as good as native apps, but the apps [14] S. K. Datta, “Android stack integration in embedded systems,” in
can be released in several platforms at once to reach out to International Conference on Emerging Trends in Computer &
Information Technology, Coimbatore, India, 2012.
most of the potential users. We have described the general
requirements of cross platform framework and its general [15] Datta, S.K.; Bonnet, C.; Nikaein, N., "Android power management:
Current and future trends," First IEEE Workshop on Enabling
architecture. Then a detail survey is presented that covers
Technologies for Smartphone and Internet of Things (ETSIoT),
several aspects of the tools allowing developers to gain insight
pp.48,53, 18 June 2012.
about the tools. Some high level guidelines to choose among
the categories are provided for the developers. Example of [16] Lee, J.; Hyunwoo Joe; Hyungshin Kim, "Smart phone power model
applications developed by PhoneGap and Titanium are given. generation using use pattern analysis," 2012 IEEE International
The performance of the Android test apps are measured in Conference on Consumer Electronics (ICCE), pp. 412,413, 13-16 Jan.
terms of memory, CPU usage and power consumption. The app 2012.
written in PhoneGap is found to use minimum memory, CPU [17] L. Zhang, et al. “Accurate online power estimation and automatic battery
and power but provides a very simple user experience. It is also behavior based power model generation for smartphones.” In Proc. Of
reported that PhoneGap with Sencha Touch 2.0 work ACM CODES+ISSS’10, Arizona, USA, 2010, pp. 105-114.
significantly well when available memory is not an issue and [18] https://play.google.com/store/apps/details?id=edu.umich.PowerTutor
better UI is desired. [19] http://www.pole-scs.org/projet/wl-box4g

PhoneGap Sencha JQueryMobile Titanium

Figure 2: User interfaces of the test Android applications.

328