Android Permissions:

User Attention, Comprehension, and Behavior

Adrienne Porter Felt*, Elizabeth Ha† , Serge Egelman*,

Ariel Haney† , Erika Chin*, David Wagner*
*Computer Science Department †School of Information
University of California, Berkeley
{apf,egelman,emc,daw}@cs.berkeley.edu, {lizzy,arielhaney}@ischool.berkeley.edu

ABSTRACT data or sends premium SMS messages for profit. Grayware and
Android’s permission system is intended to inform users about the malware have both been found in the Android Market, and the rate
risks of installing applications. When a user installs an application, of new malware is increasing over time [17, 46].
he or she has the opportunity to review the application’s permission Google does not review or restrict Android applications. Instead,
requests and cancel the installation if the permissions are excessive Android uses permissions to alert users to privacy- or security-
or objectionable. We examine whether the Android permission sys- invasive applications. When a user initiates the process of installing
tem is effective at warning users. In particular, we evaluate whether an application, he or she is shown the list of permissions that the
Android users pay attention to, understand, and act on permission application requests. This list identifies all of the phone resources
information during installation. We performed two usability stud- that the application will have access to if it is installed. For exam-
ies: an Internet survey of 308 Android users, and a laboratory study ple, an application with the SEND_SMS permission can send text
wherein we interviewed and observed 25 Android users. Study par- messages, but an application without that permission cannot. If the
ticipants displayed low attention and comprehension rates: both the user is not comfortable with the application’s permission requests,
Internet survey and laboratory study found that 17% of participants then he or she can cancel the installation. Users are not shown per-
paid attention to permissions during installation, and only 3% of In- missions at any time other than installation.
ternet survey respondents could correctly answer all three permis- In this paper, we explore whether Android permissions are us-
sion comprehension questions. This indicates that current Android able security indicators that fulfill their stated purpose: “inform the
permission warnings do not help most users make correct security user of the capabilities [their] applications have” [5]. We base our
decisions. However, a notable minority of users demonstrated both inquiry on Wogalter’s Communication-Human Information Process-
awareness of permission warnings and reasonable rates of compre- ing (C-HIP) model, which provides a framework for structuring
hension. We present recommendations for improving user attention warning research [44]. The C-HIP model identifies a set of steps
and comprehension, as well as identify open challenges. between the delivery of a warning and the user’s final behavior. We
connect each step with a research question:

Categories and Subject Descriptors 1. Attention switch and maintenance. Do users notice permis-
H.5.2 [Information Interfaces and Presentation]: User Interfaces; sions before installing an application? A user needs to switch
D.4.6 [Software]: Security and Protection—Access controls focus from the primary task (i.e., installation) to the per-
mission warnings, and she needs to focus on the permission
warnings for long enough to read and evaluate them.
General Terms 2. Comprehension and memory. Do users understand how per-
Human Factors, Security missions correspond to application risks? Users need to un-
derstand the scope and implications of permissions.
Keywords 3. Attitudes and belief. Do users believe that permissions accu-
rately convey risk? Do users trust the permission system to
Android, smartphones, mobile phones, usable security limit applications’ abilities?
4. Motivation. Are users motivated to consider permissions?
1. INTRODUCTION Do users care about their phones’ privacy and security? Do
Android supports a booming third-party application market. As they view applications as threats?
of July 2011, the Android Market included more than 250, 000 5. Behavior. Do permissions influence users’ installation deci-
applications, which have been downloaded more than six billion sions? Do users ever cancel installation because of permis-
times [34]. Unfortunately, the growth in the Android platform has sions? Users should not install applications whose permis-
triggered the interest of unscrupulous application developers. An- sions exceed their comfort thresholds.
droid grayware collects excessive amounts of personal information
(e.g., for aggressive marketing campaigns), and malware harvests Each step is critical: a failure of usability at any step will render all
subsequent steps irrelevant.
We performed two usability studies to address the attention, com-
Copyright is held by the author/owner. Permission to make digital or hard prehension, and behavior questions. First, we surveyed 308 An-
copies of all or part of this work for personal or classroom use is granted droid users with an Internet questionnaire to collect data about their
without fee.
Symposium on Usable Privacy and Security (SOUPS) 2012, July 11-13, understanding and use of permissions. Next, we observed and in-
2012, Washington, DC, USA. terviewed 25 Android users in a laboratory study to gather nuanced
data. The two studies serve to confirm and validate each other. We
do not study attitudes or motivation because we find that most users
fail to pass the attention and comprehension steps.
Our primary findings are:

• Attention. In both the Internet survey and laboratory study,

17% of participants paid attention to permissions during a
given installation. At the same time, 42% of laboratory par-
ticipants were unaware of the existence of permissions.
• Comprehension. Overall, participants demonstrated very low
rates of comprehension. Only 3% of Internet survey respon-
dents could correctly answer three comprehension questions.
However, 24% of laboratory study participants demonstrated
competent—albeit imperfect—comprehension.
• Behavior. A majority of Internet survey respondents claimed
to have decided not to install an application because of its
permissions at least once. Twenty percent of our laboratory Figure 1: On the left, a screenshot of the Android Market’s final installation
study participants were able to provide concrete details about page, displaying the application’s permission requests. On the right, the
times that permissions caused them to cancel installation. permission dialog that appears if a user clicks on a permission warning.
functionality and requirements. Users can weigh the permissions
Our findings indicate that the Android permission system is nei- against their trust of the application and personal privacy concerns.
ther a total success nor a complete failure. Due to low attention The official Android Market provides every application with two
and comprehension rates, permissions alone do not protect most installation pages. The first installation page includes a description,
users from undesirable applications (i.e., malware or grayware). user reviews, screenshots, and a “Download” button. After press-
However, a minority of laboratory study participants (20%) demon- ing “Download,” the user arrives at a final installation page that
strated awareness of permissions and reasonable rates of under- includes the application’s requested permissions (Figure 1). Per-
standing (comprehension grades of 70% or higher). This minority missions are displayed as a three-layer warning: a large heading
could be sufficient to protect others if their opinions about appli- that states each permission’s general category, a small label that
cation permissions could be successfully communicated via user describes the specific permission, and a hidden details dialog. If
reviews. We also found that some people have altered their be- an application requests multiple permissions in the same category,
havior based on permissions, which demonstrates that users can be their labels will be grouped together under that category heading.
receptive to security and privacy warnings during installation. If a user clicks on a permission, the details dialog opens. The de-
Contributions. We contribute the following: tails dialog may include examples of how malicious applications
can abuse the permission (e.g., “Malicious applications can use this
• Android permissions are intended to inform users about the to send your data to other people”). The permission system gives
risks of installing applications [5]. We evaluate whether An- users a binary choice: they can cancel the installation, or they can
droid permissions are effective security indicators. accept all of the permissions and proceed with installation.
• Researchers have speculated that Android permission warn- On most phones, Android users can also download applications
ings are ignored by users [18, 15]. We perform two studies from non-Google stores like the Amazon Appstore. When a user
to investigate how people use permissions in practice; to our selects an application through an unofficial store, that store might
knowledge, we are the first to provide quantitative data. not present permission information. However, Android’s installa-
• We explore the reasons why users do not pay attention to tion system will always present the user with a permission page
or understand Android permissions, and we identify specific before the application is installed on the phone. Like the final in-
problems with the way permissions are presented. stallation page in the Android Market, the installer displays permis-
sions as a multi-layer warning. This paper focuses on the Android
Market’s installation process because the official Android Market
2. BACKGROUND AND RELATED WORK is the primary distributor of Android applications.
In this section, we provide an overview of Android permissions
and the installation process. We then present some of the relevant 2.2 Smartphone Privacy
literature on smartphone privacy and the effectiveness of warnings. Past studies on smartphone users’ privacy concerns have primar-
ily focused on location tracking and sharing [6, 10, 29, 24, 36].
2.1 Android Permissions Although location sharing is an important aspect of smartphone
In order to protect Android users, applications’ access to phone privacy, only 2 of 134 Android permissions pertain to location.
resources is restricted with permissions. An application must ob- Concurrently, Roesner et al. [35] studied user expectations for lo-
tain permissions in order to use sensitive resources like the camera, cation, copy-and-paste, camera, and SMS security. Our study en-
microphone, or call log. For example, an application must have the compasses all permissions and focuses on how users perceive the
READ_CONTACTS permission in order to read entries in a user’s existing permission warnings.
phonebook. Android 2.2 defines 134 permissions. In concurrent and independent work, Kelley et al. [25] performed
Obtaining permissions is a two-step process. First, an applica- twenty semi-structured interviews to explore Android users’ feel-
tion developer declares that his or her application requires certain ings about and understanding of permissions. However, the scope
permissions in a file that is packaged with the application. Second, of our study is much broader: we collected large-scale quantita-
the user must approve the permissions requested before installa- tive results, performed an observational study, and experimentally
tion. Each application has its own set of permissions that reflects its measured comprehension with multiple metrics. Their study ex-
clusively reported qualitative data and did not address attention or 3. METHODOLOGY
behavior. Additionally, we designed our study to identify specific We surveyed 308 Android users with an Internet survey and in-
problems with the way permissions are presented. terviewed 25 Android users in a laboratory study. We designed
Android privacy researchers have built several tools to help users the two studies to validate each other. We recruited Internet survey
avoid privacy violations. Most research has focused on identifying respondents with AdMob advertisements and laboratory study par-
malicious behavior [15, 19, 14, 13, 46, 33], without considering ticipants with Craigslist advertisements; although both recruitment
how to help users make informed security decisions. However, two procedures might introduce bias, it is unlikely that they introduced
sets of researchers have focused on usability. Howell and Schechter the same biases. We piloted our studies with 50 AdMob-recruited
proposed the creation of a sensor-access widget, which visually no- Internet respondents and interviews of acquaintances.
tifies the user when a sensor like the camera is active [22]. Roesner
et al. proposed user-driven access control: rather than asking users 3.1 Internet Survey
to review warnings, this approach builds permission-granting into
In September 2011, we recruited Android users to answer an In-
existing user actions [35]. We focus on the usability of the existing
ternet survey about Android permissions. The purpose of this sur-
system, rather than providing new tools or user interfaces.
vey was to gauge how widely users understand and consider An-
droid permissions. To recruit respondents, we commissioned an
2.3 Warning Research advertising campaign using AdMob’s Android advertising service.
Wogalter proposed a model of how humans process warning Our advertisement was displayed in applications on Android de-
messages, known as the Communication-Human Information Pro- vices in the U.S. and Canada. (The advertisement did not appear
cessing (C-HIP) model [44]. The model formalizes the steps of a on web sites.) As an incentive to participate, each person who com-
human’s experience between being shown a warning message and pleted a survey received a free MP3 download from Amazon.com.
deciding whether or not to heed the warning. C-HIP assumes that The advertisement included our university’s name and said, “Sur-
the user is expected to immediately act upon the warning, which vey for free Amazon MP3.” We recruited people with AdMob
is appropriate for research on computer security dialogs. (Other advertisements because doing so restricted survey respondents to
researchers have focused on situations in which consumers need to those using applications on Android devices.
recall warnings for later use [30].) Researchers in the area of usable We paid AdMob $0.116 per click and received 31, 984 visitors,
security have begun to use Wogalter’s model to analyze the specific of which 1, 994 (1%) began and 350 (17.5%) completed the survey.
ways in which computer security dialogs can fail users. The rate at which people began the survey was likely influenced by
Cranor used the C-HIP model as the basis for her “human in the the high rate of accidental clicks on advertisements on mobile de-
loop” framework, which addresses problems for designers of inter- vices [2] and our request that only people age 18 and over take the
active systems [11]. Egelman et al. used the C-HIP model to exam- survey. Among people who started the survey, the completion rate
ine the anti-phishing warnings used by two popular web browsers was likely influenced by the difficulty of completing a survey on a
to determine how they could be improved [12]. They recommended phone. We ran the advertisement for two hours, and respondents
differentiating severe warnings from less severe ones, providing completed it in an average of seven minutes.
recommendations to the user, and eliminating jargon. Sunshine et We filtered out respondents who (1) stated that they were under
al. performed a followup study using the C-HIP model to examine 18, (2) had non-Android user-agent strings, or (3) appeared
web browser certificate warnings [40]. They concluded that warn- to be duplicates based on their IP addresses and user-agent
ings should be designed based on the severity of the threat model, strings. This left us with 326 unique responses. We designed our
and that it is important to take context into account when offering survey to make cheating (i.e., false responses to receive the MP3)
suggestions to the user. Some of these lessons could be applied to easy and obvious by making every question optional and providing
Android permission warnings to improve them. an “I don’t know” option for each question. Survey responses fell
The Facebook Platform’s security warnings are similar to An- into two distinct groups: responses in which all but two or three
droid’s, in that a permission dialog is triggered when a third-party questions were complete, and responses in which only one or two
application requests access to personal data. King et al. asked par- questions were complete. (Complete questions are neither blank
ticipants whether they noticed the permission dialog before enter- nor “I don’t know.”) We filtered out responses in the latter group.
ing their survey, and only a minority responded affirmatively [26]. This resulted in a total of 308 valid responses.
However, this result is not necessarily generalizable; the partici- The 308 respondents reported that they were 50% male and 49%
pants knew the survey application had been created by a privacy female, with the remainder declining to report their gender. Re-
researcher, which likely decreased their interest in security indica- spondents indicated that their age distribution was: 28% between
tors. They also presented survey participants with general compre- the ages of 18 and 28, 28% between the ages of 29 and 39, 22%
hension questions about the Facebook platform, such as whether between the ages of 40 and 50, 15% between the ages of 51 and
Facebook applications are created by Facebook. Half of partici- 61, and 5% over the age of 62. This age distribution is in line with
pants were able to answer each of these questions correctly. Android age demographics [1], although the gender breakdown of
Technology users’ feelings about privacy are complicated and our survey is more balanced than overall Android demographics.
often contradictory. When asked directly about their privacy pref- The survey was nine pages long and meant to be completed on an
erences, most surveys have found that people are very protective Android smartphone. Each page filled a standard phone screen. We
of their personal data [3, 9]. However, users’ actions do not al- used the first three pages to ask respondents about Android usage
ways correspond to their professed preferences [4, 23]. This may information: how long they had owned an Android phone, from
be because users overestimate their privacy concerns or do not un- where they had downloaded Android applications, and the factors
derstand the ramifications of their actions (i.e., the user does not they considered when downloading applications. On each of the
understand that the action violates his or her privacy preferences). three subsequent pages, we randomly displayed 1 of 11 Android
As such, we design our inquiry into Android permissions to be ro- permission warnings and asked respondents to indicate what the
bust to over-reporting of security concerns by directly observing permission allows the application to do. We gave respondents four
users and asking questions about users’ past actions. choices, in addition to “none of these” and “I don’t know.” We then
 Figure 2: Screenshot of a quiz question from the Internet survey. Figure 3: Screenshot of permissions on an application’s Settings page.

asked respondents to complete the three Westin index questions,1 with the newer version of the Android Market. Google released
tell us about their past actions relating to Android permissions, and a new version of the Market in August 2011, and not all phones
provide demographics information (age and gender). had been upgraded yet. We decided to focus on users with the new
Figure 2 depicts one of the quiz questions from the survey, and version of the Market to reduce study variability.
Table 3 lists the 11 quiz questions and choices. We designed the Our Craigslist advertisement yielded 112 eligible participants. In
permission quiz questions to include one completely incorrect choice order to match our participants’ ages to Android demographics [1],
and one choice to test fine-grained comprehension (e.g., whether we grouped applicants by age and selected a random proportion
they understood that a permission to read calendar events does not of people from each age group. We scheduled interviews with 30
include the privilege to edit the calendar). The set of 11 quiz ques- participants. Three people failed to attend and two people had tech-
tions included two questions about the READ_SMS permission: one nical problems with their phones, leaving us with 25 completed in-
to test the distinction between reading and sending SMS messages, terviews (12 women and 13 men). The age distribution was close to
and another to test respondents’ familiarity with the “SMS” acronym. overall Android age demographics by design, with 20% of partici-
Survey respondents received only one of these two related ques- pants between 18 and 24, 32% between 25 and 34, 20% between 35
tions, so scores for these questions were independent of each other.2 and 44, 16% between 45 and 54, and 12% older than 55. None of
All of the quiz questions had one or two correct choices, with the participants were affiliated with our institution, although some
the exception of the question about the CAMERA permission. This of the younger participants were students at other universities.
permission controls the ability to take a new photograph or video Each interview took 30–60 minutes and had six parts:
recording; it does not control access to the photo library. However,
we later discovered that all applications can view or edit the photo 1. General Android usage questions (e.g., how many applica-
library without any permission. Consequently, the correct answer tions they have installed).
to the CAMERA permission question is to select all four choices. 2. Participants were instructed to find and install an applica-
tion from the Android Market, using their own phones. We
3.2 Laboratory Study prompted them to install a “parking finder app that will help
In October 2011, we recruited 25 local Android users for a lab- [the user] locate your parked car.” This task served to con-
oratory study. The primary purpose of the laboratory study was to firm that participants were familiar with installing applica-
supplement the Internet survey with detailed and explanatory data. tions from the Android Market.
We also designed the attention and behavior portions of the inter- 3. Participants were instructed to find and install a second ap-
view to avoid any over-reporting problems that might have influ- plication from the Android Market using their own phones.
enced the Internet survey. We prompted them to:
To recruit participants, we posted a Craigslist ad for the San Pretend you are a little short on cash, so you want
Francisco Bay Area. Our advertisement offered people $60 to par- to install a coupons app. You want to be able
ticipate in an hour-long interview about how they “choose and use to find coupons and sales for groceries, your fa-
Android applications.” In order to be eligible for the laboratory vorite electronics, or clothes while you’re out shop-
study, we required that participants owned an Android phone and ping. If you already have a coupons app, pretend
used applications. We also asked study applicants to look at a you don’t like it and want a new one.
screenshot and tell us whether they had the new or old version of
the Android Market; we then secretly limited eligibility to users All of the top-ranked applications for search terms related
to this scenario had multiple permissions. During this appli-
1
The Westin index is a set of three questions designed to segment cation search process, we asked participants to tell us what
users into three groups: Privacy Fundamentalists, Privacy Pragma- they were thinking about while using the Market. We also
tists, and Privacy Unconcerned [42]. The Westin index is widely observed what user interface elements they interacted with.
used in surveys to gauge users’ attitudes towards privacy [27]. 4. Westin index questions.1
Buchanan et al. validated the Westin index for use in a computing
context by showing that it correlates with users’ privacy concerns 5. We asked participants about an application on their phone
and behavior on the Internet [9]. that they had installed and recently used. We then opened
2 the application’s information page in Settings (Figure 3) and
In the remainder of this paper, we refer to these two questions as
READ_SMS1 and READ_SMS2 , as depicted in Table 3. asked them to describe and explain the permissions.
 6. We asked participants for specific details about past permission- Attention to Permissions Number of users 95% CI
related behaviors, such as whether they have ever looked up Looked at the permissions 4 17% 5% to 37%
permissions or decided not to install an application because Didn’t look, but aware 10 42% 22% to 63%
of its permissions. Is unaware of permissions 10 42% 22% to 63%

Table 1: Attention to permissions at installation (Lab Study, n = 24)

Two researchers performed each interview, with one acting as the
interviewer and the other acting as a notetaker. we did not mention permissions unless the participant verbally in-
To promote a casual atmosphere, we held the interviews at a cof- dicated that he or she was reading them. After each participant
fee shop and offered participants coffee, tea, or water. Participants passed through the page with permissions, we asked him or her to
used their own phones to encourage them to behave as they would describe what had been on the previous page.
in the real world. We made an effort to not prime participants to We categorized participants into three groups:
security or privacy concerns until the fourth task, at which point we
specifically asked them about their attitudes towards privacy. We • Participants who looked at permissions during the installa-
introduced ourselves as computer science students and did not re- tion. These participants either told us that they were looking
veal that we were security researchers until the end of the study. at permissions while on the page with permissions or they
We prevented participants from determining the security focus of were later able to provide specific details about the contents
the study in advance by posting the Craigslist advertisement in the of that page. They were also able to discuss permissions in
name of a researcher with no online presence or prior publications. general, indicating that the laboratory study was not the first
time that they had viewed permissions. For example, one
participant opened the page with permissions and stated,
4. ATTENTION DURING INSTALLATION
The only thing I started doing recently, is kinda
Do users notice Android permissions before installing an appli-
looking at these – is there anything really weird.
cation? Attention is a prerequisite for an effective security indica-
tor: a user cannot heed a warning that he or she does not notice. When questioned, that participant described concern over
In our Internet survey we asked respondents whether they looked “the network stuff.”
at permissions during installation. To supplement this self-reported • Participants who did not look at the permissions for this spe-
statistic, we empirically determined whether laboratory study par- cific application, but were able to tell us that the final instal-
ticipants were aware of permission warnings. We also report users’ lation page listed permissions. In order to answer our ques-
attention to user reviews, which are shown during installation. tion, these participants must have paid attention to permis-
sions at some point in the past. For example, one participant
4.1 Permissions in this category responded,
I’ve seen a lot of them...A lot of ’em have full net-
4.1.1 Internet Survey work access, access to your dialer, your call logs,
In our Internet survey, we asked respondents, “The last time you and GPS location also.
downloaded an Android application, what did you look at before • Participants who were unaware that the final installation page
deciding to download it?” Respondents were able to select multi- included a list of permissions. For example, one participant
ple choices from a set of options that included “Market reviews,” said, “I don’t remember. I just remember ‘Download and in-
“Internet reviews,” “screenshots,” and “permissions.” stall’.” Another said, “I don’t ever pay attention. I just accept
We found that 17.5% of our 308 respondents (95%CI: [13.5%, and download it.”
22.3%]) reported looking at permissions during their last appli-
cation installation. Respondents who can be classified as Privacy We did not require knowledge of the term “permissions”; partici-
Fundamentalists using the Westin index were significantly more pants typically used other phrases (e.g., “little warning things”) to
likely to report looking at permissions than other respondents (p < describe what they saw or remembered.
0.0005; Fisher’s exact test). While statistically significant, the pro- Table 1 shows the number of study participants who fell into
portion of Privacy Fundamentalists who claimed to look at permis- each of the three categories. Fourteen participants (58% of 24) no-
sions was still a minority: 40.5% of the 42 Privacy Fundamentalists ticed permissions during the experimental installation or reported
reported looking at permissions, whereas 13.9% of the remaining paying attention to permissions in the past.3 The remaining par-
266 respondents reported looking at permissions. ticipants were unaware of the presence of permissions on the final
This self-reported question suffers from two limitations: some installation page in the Market. We did not observe a relationship
people over-report security concerns, and others may read permis- between Westin indices and participants’ attention to permissions.
sions without knowing the technical term that refers to them. We Of the ten participants who did not look at permissions during the
asked survey respondents specifically about their “last installation” study but were aware of them, three volunteered that they used to
to discourage over-reporting, but people may still guess when they look at permissions but no longer do. For example, one participant
cannot remember. Our laboratory study served to confirm the re- said, “I used to look...I just stopped doing that.” These participants
sults of the survey on a second population with a different metric. might have experienced warning fatigue, since users see permission
warnings for about 90% of applications [18]. One participant said
4.1.2 Laboratory Study that she used to be concerned about the location permission, but
In the follow-up laboratory study, we performed an experiment gradually lost her concern because so many of the applications that
to empirically determine whether users noticed permissions during she installed requested this permission.
installation. We instructed study participants to talk us through the Of the ten participants who had never paid attention to permis-
process of searching for and installing a coupon application. We sions, two knew that they were accepting an agreement on the final
recorded whether they clicked on or mentioned the permissions on 3
For this statistic, we omit one participant who had never previ-
the final Market installation page. To avoid priming participants, ously completed an installation without help.
 Importance Read reviews Didn’t read reviews A few participants reported that they read reviews but simply
A lot 68% 4% treated them as one factor among many, rather than using them
Somewhat 16% 4% as their primary decision-making factor. One of these participants
Mistrust 4% 0% described the rating system as “a starting point,” and another said
Unknown 0% 4% that reviews are “just a place to start.” One of the 25 participants
Total 88% 12% actively mistrusts positive reviews because she has written reviews
for her company’s products on websites. Despite this, she still looks
Table 2: We observed whether users read reviews, and later asked how much at reviews to identify negative traits of applications.
importance they place on reviews (Lab Study, n = 25)
At the end of the study, we asked participants whether they had
ever tried to find out what a permission means or why an appli-
installation page. They both described the page as containing legal cation was asking for it. Eight of the study participants (32% of
terms of use, with one incorrectly elaborating that the text specified 25) responded affirmatively, with six (24% of 25) people stating
legal restrictions on the use of the application. Due to their lack of that they found this information in some type of review. Three of
interest in legal text, neither had ever read the screen so they were these participants stated that they had read user reviews to deter-
unaware that the text pertains to security and privacy. mine whether an application’s permissions were appropriate, and
The self-reported survey and observational study results both another two said that they had read news articles that reviewed ap-
suggest that 17% of users routinely look at permissions when in- plications’ permissions. Another participant said that he read about
stalling an application. We also found that 42% of study partici- permission information in reviews, but that he had never noticed
pants could not possibly benefit from permission information be- that the same permission information was available on the final in-
cause they had never noticed it. The remaining 42% of participants stallation page. One of the six said,
were aware of permissions but do not always consider them. If I’m not sure about an app I’ll research it and see
what other people say about the permissions. Like, ‘It
4.2 Reviews does this,’ ...and, ‘It’s necessary.’ And there will be
Like permissions, user reviews have the ability to convey privacy an argument or a discourse about the permissions that
and security information during installation. User reviews can warn need to be on there or don’t need to be.
people about undesirable or privacy-invasive applications.
This suggests that reviews are an important part of communicating
permission information, especially for users who do not understand
4.2.1 Internet Survey permission warnings on their own.
We asked survey respondents, “The last time you downloaded an
Android application, what did you look at before deciding to down- 5. COMPREHENSION OF PERMISSIONS
load it?” A total of 219 survey respondents (71.1% of 308; 95%CI:
[65.5%, 76.2%]) reported looking at some type of review before Do users understand how permissions correspond to application
installation. Of these, 193 respondents (62.7% of 308; 95%CI: privileges? Users can only make correct security decisions based
[57.0%, 68.1%]) indicated that they looked at Market reviews dur- on permissions if they understand what the permission warnings
ing their last application installation, and 42 respondents (13.6% mean. We used three metrics to measure subjects’ understanding
of 308; 95%CI: [10.0%, 18.0%]) stated that they had looked at of permission warnings. First, we tested Internet survey respon-
other reviews on the Internet. Twenty-six respondents (8.4% of dents with multiple-choice questions (Section 5.1). Second, we
308; 95%CI: [5.6%, 12.1%]) reported that they had looked at both graded laboratory study participants’ ability to describe the per-
Internet and Market reviews. We did not find that any age, gender, mission warnings of a familiar application (Section 5.2). Third, we
or Westin group was more or less likely to look at reviews. asked study participants whether the application’s set of permis-
sions gave it the ability to send text messages (Section 5.3).
4.2.2 Laboratory Study 5.1 Permission Comprehension Quiz
In our follow-up laboratory study, we observed whether partic- Internet survey respondents answered three randomly-selected
ipants actually considered reviews during application installation. quiz questions from the set of eleven questions in Table 3. Six
We instructed participants to tell us what they were reading and respondents omitted one or more questions; we filtered those par-
considering while selecting and installing a coupon application. We ticipants out of this analysis, leaving us with 302 respondents who
did not mention reviews or ratings unless the participant first spoke answered three quiz questions. Overall, respondents did not ran-
of or clicked on them. After participants mentioned reviews or rat- domly select their responses: the observed median differs from
ings, we asked them how much importance they placed on reviews 1/16, which would be the median if respondents were equally likely
and whether they trusted them to be correct. If a participant did not to select any choice or set of choices (one-sample Wilcoxon Signed
consider reviews during installation, we asked the participant for Rank test, p < 0.0005).
his or her opinion of reviews after the installation task. Eight respondents (2.6% of 302) answered all three questions
Table 2 shows participants’ opinions of reviews and whether correctly. On average, respondents correctly answered 21% of the
they considered reviews during the installation. All but three par- three questions. We considered the relationship between respon-
ticipants mentioned application reviews during installation; of the dent scores and demographics:
three that did not read reviews, two later claimed when questioned
that they read reviews in some situations. The majority of partici- • We did not observe a correlation between respondent scores
pants placed a lot of importance on reviews. For example, and the length of Android phone ownership.
• No significant differences were observed between the gen-
[Reviews] let me know if it’s a decent app or not. Be- ders or with regard to Westin index classifications.
cause most people will put on there whether it’s a good • There was a negative correlation between age and the number
app or a bad app. of correct answers (r = −0.257, p < 0.0005); younger
people were more likely to understand permissions.
 Permission n Options Responses
4 Send information to the application’s server 45 41.3%
INTERNET 4 Load advertisements 30 27.5%
Category: Network communication 109 7 None of these 16 14.7%
Label: Full Internet access 7 Read your text messages 13 11.9%
7 Read your list of phone contacts 11 10.1%
I don’t know 36 33.0%
4 Read your phone number 41 47.7%
READ_PHONE_STATE 7 See who you have called 37 43.0%
Category: Phone calls 85 4 Track you across applications 20 23.3%
Label: Read phone state and identity 7 Load advertisements 11 12.8%
7 None of these 10 11.6%
I don’t know 15 17.4%
4 Place phone calls 30 35.3%
CALL_PHONE 7 Charge purchases to your credit card 27 31.8%
Category: Services that cost you money 83 7 None of these 16 18.8%
Label: Directly call phone numbers 7 See who you have made calls to 14 16.5%
7 Send text messages 11 12.9%
I don’t know 16 18.8%
4 Read other applications’ files on the SD card 41 44.6%
WRITE_EXTERNAL_STORAGE 4 Change other applications’ files on the SD card 39 42.4%
Category: Storage 92 7 None of these 16 17.4%
Label: Modify/delete SD card contents 7 See who you have made phone calls to 15 16.3%
7 Send text messages 11 12.0%
I don’t know 15 16.3%
4 Keep your phone’s screen on all the time 49 60.5%
WAKE_LOCK 4 Drain your phone’s battery 37 45.7%
Category: System tools 81 7 None of these 7 8.6%
Label: Prevent phone from sleeping 7 Send text messages 4 4.9%
7 Delete your list of contacts 4 4.9%
I don’t know 13 16.0%
4 Turn your WiFi on or off 36 52.9%
CHANGE_NETWORK_STATE 7 Send information to the application’s server 13 19.1%
Category: System tools 66 7 Read your calendar 7 10.3%
Label: Change network connectivity 7 None of these 7 10.3%
7 See who you have made calls to 5 7.4%
I don’t know 17 25.0%
4 Read text messages you’ve sent 30 54.5%
READ_SMS2 4 Read text messages you’ve received 25 45.5%
Category: Your messages 54 7 Send text messages 10 18.2%
Label: Read SMS or MMS 7 Read your phone’s unique ID 6 10.9%
7 None of these 4 7.3%
I don’t know 11 20.0%
4 Read text messages you’ve received 44 56.4%
READ_SMS1 7 Read e-mail messages you’ve received 30 38.5%
Category: Your messages 77 7 Read your call history 13 16.7%
Label: Read SMS or MMS 7 None of these 8 10.3%
7 Access your voicemail 8 10.3%
I don’t know 13 16.7%
4 Read your calendar 56 53.3%
READ_CALENDAR 7 None of these 18 17.1%
Category: Your personal information 101 7 Add new events to your calendar 12 11.4%
Label: Read calendar events 7 Send text messages 12 11.4%
7 Place phone calls 9 8.6%
I don’t know 19 18.1%
4 Read your list of contacts 52 60.5%
READ_CONTACTS 4 Read your call history 19 22.1%
Category: Your personal information 86 7 None of these 14 16.3%
Label: Read contact data 7 Delete your list of contacts 9 10.5%
7 Place phone calls 5 5.8%
I don’t know 14 16.3%
4 Take pictures when you press the button 27 37.0%
CAMERA 4 Take pictures at any time 27 37.0%
Category: Hardware controls 72 4 See pictures taken by other applications 16 21.9%
Label: Take pictures 4 Delete pictures taken by other apps 13 17.8%
7 None of these 13 17.8%
I don’t know 17 23.3%
Table 3: Survey respondents were each asked three multiple choice questions, randomly selected from this set. Respondents could select “None,” “I don’t
know,” or one or more of the four definitional choices. This table orders the choices by the number of respondents who selected it. The 4 symbol marks
correct responses, and the 7 symbol marks incorrect responses.
 Permission n Correct Answers model is READ_SMS2 ; most respondents were able to correctly de-
1 Choice READ_CALENDAR 101 46 45.5% termine that the READ_SMS2 permission grants the ability to read
CHANGE_NETWORK_STATE 66 26 39.4%
but not send text messages.
READ_SMS1 77 24 31.2%
CALL_PHONE 83 16 19.3%
WAKE_LOCK 81 27 33.3% 5.2 Free-Form Permission Descriptions
2 Choices

WRITE_EXTERNAL_STORAGE 92 14 15.2% We hypothesized that users might understand permission warn-

READ_CONTACTS 86 11 12.8% ings better when the permissions are associated with a familiar
INTERNET 109 12 11.0%
READ_PHONE_STATE 85 4 4.7% application. For example, a user who does not understand the
READ_SMS2 54 0 0% INTERNET permission in isolation might know that the permis-
4 CAMERA 72 7 9.7% sion is needed to fetch news from the Internet when he or she sees
that the permission is associated with a news application. As such,
Table 4: The number of people who correctly answered a question. Ques- we designed our follow-up laboratory study to ask users about the
tions are grouped by the number of correct choices. n is the number of meaning of permissions in the context of a familiar application.
respondents. (Internet Survey, n = 302)
During the laboratory study, we asked each participant to view
the permissions of an application that he or she had recently used
• We compared the scores of respondents who did and did on his or her phone. The participant was therefore familiar with
not report looking at permissions in a past application in- the application’s functionality. We asked participants to read each
stallation. Respondents who reported looking at permissions permission aloud and explain what it meant. We gave participants
scored higher on average (30.3% vs. 18.6%). The differ- three chances to demonstrate their understanding of the permis-
ence was statistically significant (U = 5, 293.0, p < 0.007, sions: we asked what the permissions meant, why the application
r = 0.16) but small in absolute terms. had them, and whether each permission was necessary or unneces-
• In the survey, we asked respondents whether they typically sary for the respective application.
used the Android Market or unofficial application stores. Re- To evaluate user understanding, we graded participants’ free-
spondents who typically used the Android Market were sig- form descriptions of permissions as follows:
nificantly more likely to understand the permissions (U =
2, 474.0, p < 0.001, r = 0.20). The 28 respondents who did
not use the Market had an average score of 4.7%, whereas the • Correct. A correct answer completely explains the mean-
remaining 274 respondents had an average score of 22.3%. ing of a permission. For example, one participant correctly
stated that the BLUETOOTH_ADMIN permission allowed the
Although we found statistically significant differences between cer- application to “create a Bluetooth connection” and “discon-
tain groups, no group performed well on an absolute scale. nect Bluetooth to save battery.”
Despite their poor overall scores, many respondents demonstrate • Correct but overly broad. This type of answer contained cor-
a limited understanding of the permission warnings. As shown in rect information, but the participant believed that the permis-
Table 3, a plurality of respondents selected at least one correct sion granted more privileges than it actually does. For ex-
choice for every question, and a majority of respondents selected ample, one participant understood that the INTERNET per-
at least one correct choice for six questions. Respondents’ low mission could be used to send or retrieve data, but he also
scores reflect the fact that we only consider an answer correct if believed it gave the application the ability to “check my GPS
that respondent specified all of the correct choices and no incorrect or see where I’m going.” (In this example, the application
choices. Although only 21% of the 906 answers4 were completely did not have a location permission.)
correct, 53% of the answers contain at least one correct choice: • Incomplete. Incomplete answers show that the participant
17% of answers contain a subset of the correct choices, and 15% had a partial understanding of the permission, but lacked
of answers contain extra incorret choices. This type of error indi- comprehension of an important aspect of the permission. For
cates that the respondent can identify (part of) the definition, but example, one participant understood that the RECEIVE_SMS
he or she also incorrectly believes that the permission’s scope is permission pertains to text messages but did not know how.
narrower or broader than it actually is. • Incomplete and overly broad. This type of answer includes
Seven questions had multiple correct choices. Users performed wrong information in addition to an incomplete but correct
significantly worse on questions with multiple correct choices (r = explanation of a permission. For example, one participant
−0.59, p < 0.028; one-tailed), so we can only directly compare described the READ_PHONE_STATE permission as,
permissions with the same number of possible correct choices. Ta-
ble 4 depicts the eleven permissions and the number of survey re- Phone calls is probably like the call log or the
spondents who got each one completely correct. phone calls that are made. It tells you their names
We hypothesize that some respondents made decisions based pri- and maybe a picture.
marily on the category headings, which are featured in a much
This description is partially correct because the permission
larger font than the specific permission labels. This may have led
relates to call state, but it is incomplete because the permis-
respondents to overstate the meanings of permissions (i.e., select
sion also provides access to the participant’s own identity.
incorrect as well as correct choices). Respondents’ answers to all
The participant also incorrectly stated that the permission
but one of the permissions seem consistent with this hypothesis (Ta-
grants access to contacts’ names and pictures.
ble 3). For example, the CALL_PHONE permission illustrates this
• Wrong. The participant provided an incorrect description
type of error: the large category heading says “Services that cost
of a permission that included substantially more privileges
you money,” and nearly as many respondents selected the incorrect
than the truth. For example, several participants stated that
answer of “Charge purchases to your credit card” as the correct an-
the READ_PHONE_STATE permission applications listen to
swer of “Place phone calls.” The one question that does not fit this
their phone calls. This confusion likely occurred because the
4
906 answers: 302 respondents provided three answers each. category heading for that permission is “Phone Calls.”
 90 - 99! Participant response: Yes No Unsure
80 - 89! Correct 4% 32% -
70 - 79! Incorrect 44% 12% -
Total 48% 44% 8%
% Correct!

60 - 69!
50 - 59! Table 6: Can an application send text messages? The correct answer de-
40 - 49! pends on the application that the given user selected. (Lab Study, n = 25)
30 - 39!
20 - 29!
We observed that participants often placed more emphasis on the
10 - 19!
category heading than the specific permission text, which caused
0 - 9! them to err in the direction of overstating the privilege associated
0! 1! 2! 3! 4! 5! 6! with permissions. (Figure 1 shows examples of categories and spe-
Number of Users! cific permissions.) Descriptions were overly broad 29% of the time,
Figure 4: A histogram of participants’ grades. (Lab Study, n = 25) and all but 3 of the overly-broad responses could be attributed to
the category heading. For example, the READ_CONTACTS per-
mission is under the heading of “Personal Information.” Upon see-
• Unable to answer. We placed responses in this category when
ing that warning, one participant stated that the permission pro-
the participant read the permission aloud and then stated that
vided access to his passwords, and another believed that the per-
he or she could not describe the permission.
mission encompassed all of the data on her phone. Similarly, the
• Omitted. Participants often skipped permissions that were
READ_PHONE_STATE permission is under the heading of “Phone
present on the screen, and we were not always able to prompt
Calls.” Participants inferred that the warning referred to a wide
them to address the skipped permission. In these cases, we
variety of phone-related behavior, such as giving a company per-
have no way of knowing whether the participant would have
mission to make telemarketing calls to the participant.
been able to answer correctly.

Figure 4 depicts each laboratory study participant’s grade, where

5.3 Specific Permission Comprehension
a person’s grade is the percentage of descriptions that were correct. After each participant described the set of permissions, we asked
We calculated this percentage after filtering out permissions that him or her whether the selected application had the ability to send
they omitted; omitted permissions are excluded because we do not text messages without his or her knowledge. If the participant asked
know why they were omitted. Two participants received grades of for clarification, we elaborated that we wanted to know whether
0%, the highest grade was 83%, and the average was 39%. Con- the application can send text messages, not whether it does. This
trary to our initial hypothesis, comprehension rates are still low privilege is granted with the SEND_SMS permission, which is in
when permissions are associated with a familiar application. the “Services that cost the user money” category with the specific
Six participants received grades of 70% or higher. We observed permission label of “Send SMS messages.” This question was de-
two of the six high scorers looking at permissions during installa- signed to gauge whether people can determine the tasks that an ap-
tion (Section 4.1), and another three indicated that they had looked plication can do on their phones, given its permissions. We chose
at permissions in the past. The permission system could potentially the SEND_SMS permission for this question because we thought
help these five participants (20% of 25) because they sometimes that all participants would be familiar with text messages, and the
pay attention to and understand permission warnings. The sixth permission is associated with malware [17, 46].
high scorer expressed some familiarity with permissions but did Table 6 presents participants’ responses. The correct answer de-
not know that the Market displayed them prior to installation. pends on the application that the given user selected. Only nine
Other participants commonly said that they did not know what of the participants (36% of 25) answered correctly. Participants’
the warnings meant: 25% of the times that a participant read a answers were not significantly different from guessing: twelve re-
permission, he or she was completely unable to describe it. Several sponded affirmatively and eleven negatively.
participants mentioned that they understood all of the vocabulary Four participants selected applications with the SEND_SMS per-
but did not know how that information pertained to their phones. mission, and three of them incorrectly stated that the application
As one participant said, could not send text messages. One of these participants had asked
us about the meaning of the SEND_SMS permission during the pre-
I think I know what they mean as a person who has zero vious step of the laboratory study, and she correctly repeated our
electronics or programming training, just in terms of explanation. Despite this, she still responded that the application
what I think the words mean...I know what they mean could not send text messages. She re-examined the permission
in terms of the face value of the words. I don’t really warning after our question and stated,
know what they mean in terms of complicated, in terms
of technicalities... Well, I don’t know now. Cause it said that it could – I
don’t know. I’m going to say no.
Participants’ grades are not directly comparable to each other
because each participant viewed a different application’s permis- Another participant was aware that her chosen application could
sions. However, a few popular permissions were present in 11 or send text messages because she had used the application, but she
more participants’ applications. Table 5 shows how participants still believed that it was not capable of sending text messages with-
performed when describing these permissions. Notably, partici- out her expressed approval. She seemed to believe that all appli-
pants performed better on the three permissions that refer to general cations require user approval to send text messages, regardless of
computer concepts: Internet access, hard drive storage, and putting the permissions. The third person looked at the category heading
the phone to “sleep.” Participants were less able to describe the two (“Services that cost the user money”) and incorrectly decided that
smartphone-specific permissions. it referred to Internet data and phone calls but not text messages.
 READ_CONTACTS WAKE_LOCK WRITE_EXTERNAL_STORAGE READ_PHONE_STATE INTERNET
Correct 0% 54% 47% 0% 68%
Correct but overly broad 9% 9% 0% 0% 4%
Incomplete [and overly broad] 18% 0% 18% 45% 9%
Wrong 45% 0% 23% 20% 9%
Unable to answer 27% 36% 12% 35% 9%
Total number of participants 11 11 17 20 22

Table 5: The grades of free-form participant responses for popular permissions. (Lab Study, n = 25)

Twenty-one participants selected applications that do not have

Self-Reported Behavior Respondents
the SEND_SMS permission. Of those, eleven participants incor-
rectly thought that their applications could send text messages. When Yes 56.7%
Didn’t like permissions 32.6%
asked why, six participants explained that various other permissions
Too many permissions 16.0%
allow this behavior. Two people said that the INTERNET permis- Both 8.1%
sion (listed under the “Network communication” category heading) No/I don’t know 43.3%
allows an application to send a text message. For example,
Table 7: Respondents who claimed they did not install an application due
It has access to my network, so I assume it could send to permissions. (Internet Survey, n = 307)
a message if it wanted to.
Four people believed that the READ_PHONE_STATE permission Self-Reported Behavior Participants
(listed under the “Phone calls” category heading) grants the ability Yes 5 20%
to send text messages. For example, Probably 2 8%
No 18 72%
Well, yeah, because of the phone calls. Because of the
phone calls, they can read the phone calls, so obvi- Table 8: Participants who claim they did not install an application due to
ously they can. permissions. The two “Probably” responses refer to participants who could
not provide confirming details. (Lab Study, n = 25)
A sixth participant believed that the application could combine the
personal information, phone calls, and network communication cat-
A respondent could select both of the affirmative options, and the
egories together to send a text message.
answers were not randomly ordered.
One of our participants said he had a small amount of experience
We received 307 responses. Table 7 shows the results: 56.7% of
as an Android developer. He was among the eleven participants
respondents (95%CI: [52.1%, 62.3%]) claimed to have decided not
who incorrectly stated that an application could send text messages.
to install an application because of its permissions. We found that
When asked for an explanation,
respondents who can be classified as Privacy Fundamentalists using
I’ve done some programming but I don’t know all the the Westin index were more likely than other respondents to report
permissions. ... I just don’t know if the permissions are not installing an application due to its permissions (χ2 =5.6161,
so fine grained that they make texting a special permis- p = 0.016): 73.8% of the 42 Privacy Fundamentalists (95%CI:
sion that you have to add. [60.5%, 87.1%]) responded affirmatively, compared to 53.9% of
the 265 remaining respondents (95%CI: [47.9%, 59.9%]).
The participant then reasoned that two other permissions likely in- The number of affirmative responses to this question may be ar-
clude that ability. Without knowing the full list of possible Android tificially inflated because of position bias; people display a slight
permissions, it is difficult for a user – even a highly experienced, preference for the first choice over later choices [8]. Survey respon-
technically competent user – to determine whether an application dents viewed this question after seeing the permission quiz ques-
cannot perform an action. In other words, users need to know what tions, which also may have increased their likeliness to respond af-
permissions their application does not have in order to comprehend firmatively. We asked survey respondents about a past action rather
the scope of the permissions that it does have. than a preference to mitigate over-reporting, but people may err
with a bias when they cannot remember the answer.
6. INFLUENCE ON USER BEHAVIOR
Do permissions influence users’ installation decisions? Users are
6.2 Laboratory Study
shown permissions on the final installation page of the Market so In our follow-up laboratory study, we asked participants the same
that they can refrain from downloading an application if they dislike question: “Have you ever not installed an app because of permis-
its requested permissions. We asked users whether they have ever sions?” However, we designed the laboratory study question to
decided not to install an application because of its permissions. avoid over-reporting. If a person responded affirmatively, we asked
for detailed information about the application and why he or she ob-
6.1 Internet Survey jected to the permissions. Although people often over-report their
The survey asked, “Have you ever not installed an app because of security concerns when asked abstract questions, we feel it is un-
permissions?” Respondents were shown the following four choices: likely that a participant would fabricate specific details of his or her
application installation history in an in-person interview.
• Yes, I didn’t like the permissions Table 8 shows how study participants responded to this question.
• Yes, there were too many permissions We asked the five affirmative participants to explain why and how
• No often they had decided not to install certain applications based on
• I don’t know their permissions. Here, we excerpt their concerns:
 • One person decided not to install a social networking appli- 7.2 Short-Term Recommendations
cation because “with exact location then they could post that Our studies identified several factors that contribute to the low
on my page or something like that.” attention and comprehension rates. We now present a set of short-
• “At least five. I felt it was asking for too much, or it was term recommendations aimed at addressing these problems by mod-
going to do too much data, and I didn’t feel comfortable.” ifying the existing install-time warnings, although future research
• One participant became alarmed after reading a Wall Street would be needed to design and implement these recommendations.
Journal article about Android applications’ permissions and
privacy policies [43]. “I haven’t really downloaded very many Categories. We find that category headings widely confused users.
apps since... And there have been a few I haven’t downloaded As Figure 1 shows, the final installation page uses a multi-layer
because they asked for a bunch of accesses.” user interface to convey permissions. The large category headings
• “In the zone of maybe one out of four, roughly. Mostly most are short, simple, and non-technical; below them, the smaller text
of them look fairly benign to me in terms of my concerns, includes more information about the specific permissions. Multi-
but there are some of them that just look like they’re overkill. layer user interfaces are intended to simultaneously satisfy novice,
I must say that in the beginning of installing apps, I – and I average, and expert users by providing subsequently more infor-
believe most people – are more hesitant about installing apps mation at each layer of the user interface [39, 28]. However, the
that reveal your location.” category headings are currently so broad that they cause users to
• Another person was aware of permissions but did not read overestimate the scope and risk of the requested permissions. Over-
them on his own. Instead, he would look for reviews about estimation undermines the warning system because it causes users
certain permissions pertaining to battery life. “Some of the to believe that they are granting dangerous permissions to more ap-
ones that people say, ‘It runs at startup,’ and, ‘You can’t stop plications than they are. This likely has a negative impact on the
it,’ or something like that...then I won’t download it.” amount of attention that users pay to permissions; there is little rea-
son to read individual permission warnings if one believes that all
Two of the five participants who said that they had not installed applications receive dangerous privileges.
an application because of permissions scored very poorly on the We recommend re-organizing and re-naming categories to shape
comprehension study (Section 5.2). One was unable to describe user expectations more appropriately. In particular, the “Personal
any permissions correctly, and the other described only two of seven Information” and “Phone Calls” categories misled many of the users
permissions correctly. This shows that people may act on permis- in our studies. Although the category headings need to be re-
sion information even if they do not correctly understand it. designed, we do not recommend removing them; the categories
Through our attention and comprehension studies, we identified reduce warning fatigue by decreasing the number of warnings that
five participants who were aware of and understood permissions are shown on the screen. (E.g., a user sees only three warnings for
relatively well. Two of those participants said that they had can- eight permissions if the eight permissions fall into three categories.)
celled installation due to permissions in the past. In other words, Risks, Not Resources. We find that many users cannot connect
8% of 25 participants paid attention to, understood, and previously permission warnings to risks, even if they understand all of the
acted on permissions. It is unclear why the other three participants technical terms in a permission warning. Currently, most of the
who paid attention to and understood permissions have never can- warnings are resource-centric and value-neutral (e.g., “full Internet
celled installation because of permissions; it is possible that they access” and “read phone state and identity”). Users are left to de-
lack motivation, lack trust in the permission system, or have sim- cide on their own how the resources might be used, which causes
ply not yet encountered a suspicious application. them to underestimate or overestimate the risks of permissions. It
is important for warnings to clearly convey specific risks [45]. We
7. IMPLICATIONS cannot expect non-expert users to understand the relationship be-
We evaluated whether the Android permission system can help tween resources and risks, and users cannot provide informed con-
users avoid security- and privacy-invasive applications. We now as- sent if they do not realize the risks. The long explanation dialog
sess the significance of our findings and several recommendations specifies the risks for a few permissions, but the majority lack risk
for improving the usability of permissions. information; also, we did not observe any users reading the long
dialogs. We recommend that permission warnings focus wholly on
7.1 Effectiveness of Permissions risks (i.e., potential negative outcomes) instead of resources. For
Our studies demonstrated that the majority of Android users do example, “full Internet access” could be replaced with “use your
not pay attention to or understand permission warnings. Nearly data plan.” To balance the risks with benefits, developers could be
half of the laboratory study participants were completely unaware given space in the UI to justify why they need the permissions.
that permission warnings are displayed in the Market. Since at- Low-Risk Warnings. We observed evidence of users experiencing
tention and comprehension are prerequisites for informed security warning fatigue. Warning fatigue is exacerbated by unnecessary
decisions, our study indicates that the current Android permission warnings. To avoid devaluing the warnings, we recommend that
system does not help most users make good security decisions. permissions without clear risks should not be shown to users. For
However, we also found that permissions are effective at con- example, the ability to connect to a Bluetooth device is unlikely to
veying security information to a minority of users: 20% of the lab- cause a user harm. Warnings that do not convey real risks teach
oratory study participants were aware of permissions and scored the user that all warnings are unimportant [41, 12], and there are
above 70% on the comprehension metric. It is possible that this is limits on how much information people can process when making
sufficient; a small fraction of expert users could write negative re- decisions [7, 20]. Currently, some permissions are not displayed
views when they encounter troubling permission requests, thereby to users unless they choose to “See more” because the permissions
protecting other consumers. Researchers have found that negative are considered non-dangerous; we recommend that more permis-
reviews can influence sales in other contexts [38, 47], and 24% of sions should be classified as non-dangerous (and hidden by de-
laboratory study participants said that they had relied on reviews or fault). Users should be surveyed to identify low-concern risks.
news reports to provide them with permission information.
Absent Permissions. Our SMS comprehension study demonstrated Customization. We hypothesize that different users have different
that people cannot reason about the absence of permissions. A user types of privacy and security concerns. For example, a mother told
cannot say with certainty that a permission does not encompass a us that she worried a lot about people knowing her daughter’s loca-
privilege unless the user knows that another permission exists to tion via their shared phone, whereas another user said he was con-
address that privilege or no permission permits the action. Conse- cerned only about whether applications will excessively drain his
quently, users overestimate the scope and risk of the permissions phone’s battery. When users read permissions aloud to us for the
that are present. Currently, it is infeasible for any user to remem- comprehension study, they often told us (without prompting) that
ber all of the permissions, given that Android has more than 100 they did not care about certain permissions. Warnings will likely be
permissions. We recommend coalescing or paring down the list of more effective if they are relevant to users’ specific concerns about
permission warnings to a set that is small enough for users to look applications. The challenge is to identify users’ concerns without
up and remember with accuracy. expecting all users to fill out surveys or provide feedback. It might
be possible to learn which warnings are likely to be relevant to par-
Optional Permissions. Several researchers have suggested that
ticular users, classes of users, or users generally.
users should be able to grant or deny an application’s permissions
individually, rather than as a bundle [32, 33]. This would give users
finer-grained control over the resources that applications have ac- 8. CONCLUSION
cess to. We do not recommend adopting this proposal until user un- This paper represents a first step in understanding the effective-
derstanding of permissions can be improved with other measures. ness of Android permissions. Our two studies indicate that Android
The low comprehension rates suggest that users cannot currently permissions fail to inform the majority of users, but permissions
make informed decisions about individual permissions. Even the are not wholly ineffective despite researchers’ predictions [18, 15].
users that displayed comprehension competency during the labora- A minority of users demonstrated awareness and understanding
tory study did not receive perfect comprehension scores. As such, of permissions, and we found that permissions helped some users
individual permission granting would add complexity to the user avoid privacy-invasive applications. This motivates continued ef-
interface without increasing user control. fort towards the goal of usable permissions. However, low rates of
user attention and comprehension indicate that significant work is
7.3 Open Problems needed to make the Android permission system widely accessible.
Larger changes are needed to improve the relevance of permis- We identified a set of issues that are impeding awareness and
sion warnings and reach users who are currently unaware of per- comprehension. In particular, category headings are confusing,
mission warnings. Future permission system designers should re- some users cannot connect resource-based warnings to risks, some
consider the ways in which permissions are displayed to users. We users cannot reason about the absence of permissions, and some
present a set of open problems and future research directions that users are experiencing warning fatigue. We provide a set of rec-
are motivated by our studies. ommendations to address these issues. Our results also support
Timing. Android shows users permission information during in- three directions of future work for improving permission systems:
stallation instead of when they are using the application. This de- connecting reviews to permissions, customizing warnings to users’
sign decision was made because “over-prompting the user causes concerns, and investigating new types of warning dialogs.
the user to start saying ‘OK’ to any dialog that is shown” [5]. In-
deed, many studies have shown that users click through security Ethics and Safety
dialogs that are presented when the user is trying to perform a The studies described within this paper received approval from the
task with an application [31, 41, 37]. However, we find that the Institutional Review Board (IRB) at the University of California,
install-time permission dialog is similarly dismissed by most users. Berkeley, prior to the commencement of the research. All Internet
Additionally, install-time permissions lack context; unlike dialogs survey data was collected anonymously, and the IP address logs
shown at runtime, there is no way to know what application func- were deleted once responses had been de-duplicated. Laboratory
tionality the install-time permissions correspond to. This suggests study participants were not interviewed anonymously, but all data
that completely new solutions that avoid dialogs, such as sensor- was coded so that only the lead researcher could connect consent
access widgets [22] or access-control gadgets [35], may be needed. forms to participant data. We gave laboratory study participants
Reviews. We identified a small minority of “expert” users who an opportunity to ask clarifying questions about permissions and
could potentially protect others by sharing their concerns about per- Android security after their interviews.
missions. One direction is to re-think how a system could support
the sharing of privacy and security concerns. How can we incen- Acknowledgments
tivize writing reviews about permissions? How can we help inter-
We thank Angie Abbatecola for her help with survey and study
ested users determine what applications are doing with permissions logistics, such as ensuring that laboratory study participants were
so that they can write useful reviews? How can other readers con- paid in a timely fashion. We also thank Jennifer King for her in-
firm claims about privacy and security? Currently, Android does sightful comments and discussion. This material is based upon
not provide any way to audit an application’s permission usage, al-
work supported by Facebook and National Science Foundation Grad-
though security researchers have developed applications and tools
uate Research Fellowships. Any opinions, findings, conclusions,
to do so [13, 16, 21]. However, users with interests in privacy and or recommendations expressed here are those of the authors and do
security are not necessarily computer scientists, despite some fa- not necessarily reflect the views of Facebook or the National Sci-
miliarity with smartphone technology; none of our “expert” users
ence Foundation. This work is also partially supported by National
had any formal technical education, and we do not expect that they
Science Foundation grant CCF-0424422, a gift from Google, and
would be able to use any of the existing research tools. the Intel Science and Technology Center for Secure Computing.
9. REFERENCES 2011.
[1] AdMob Mobile Metrics Report. AdMob Blog, 2010. [19] A. Fuchs, A. Chaudhuri, and J. Foster. SCanDroid:
[2] How Consumers Interact with Mobile App Advertising. Automated Security Certification of Android Applications.
Harris Interactive Survey, December 2011. Technical report, University of Maryland, 2009.
[3] M. Ackerman, L. Cranor, and J. Reagle. Privacy in [20] G. J. Gaeth and J. Shanteau. Reducing the Influence of
e-commerce: examining user scenarios and privacy Irrelevant Information on Experienced Decision Makers.
preferences. In Proceedings of the ACM Conference on Organizational Behavior and Human Performance, 33, 1984.
Electronic Commerce, 1999. [21] P. Hornyack, S. Han, J. Jung, S. Schechter, and D. Wetherall.
[4] A. Acquisti. Privacy in electronic commerce and the These Aren’t the Droids You’re Looking For: Retrofitting
economics of immediate gratification. In Proceedings of the Android to Protect Data From Imperious Applications. In
ACM Electronic Commerce Conference (ACM EC), 2004. Proceedings of the ACM Conference on Computer and
[5] Android Open Source Project. Android Security Overview, Communication Security, 2011.
2012. [22] J. Howell and S. Schechter. What you see is what they get. In
[6] L. Barkhuus and A. Dey. Location-based services for mobile Proceedings of the IEEE Workshop on Web 2.0 Security and
telephony: a study of users’ privacy concerns. In Privacy (W2SP), 2010.
Proceedings of the International Conference on [23] C. Jensen, C. Potts, and C. Jensen. Privacy practices of
Human-Computer Interaction, 2003. Internet users: Self-reports versus observed behavior. In
[7] J. R. Bettman. An Information Processing Theory of International Journal of Human-Computer Studies, 2005.
Consumer Choice. Addison-Wesley Publishing Company, [24] P. Kelley, M. Benisch, L. Cranor, and N. Sadeh. When are
1979. users comfortable sharing locations with advertisers? In
[8] N. J. Blunch. Position Bias in Multiple-Choice Questions. Proceedings of the ACM CHI Conference on Human Factors
Journal of Marketing Research, 1984. in Computing Systems, 2011.
[9] T. Buchanan, C. Paine, A. N. Joinson, and U.-D. Reips. [25] P. G. Kelley, S. Consolvo, L. F. Cranor, J. Jung, N. Sadeh,
Development of measures of online privacy concern and and D. Wetherall. A Conundrum of Permissions: Installng
protection for use on the Internet. Journal of the American Applications on an Android Smartphone. In Proceedings of
Society for Information Science and Technology, 2007. the Workshop on Usable Security (USEC), 2012.
[10] S. Consolvo, I. E. Smith, T. Matthews, A. LaMarca, [26] J. King, A. Lampinen, and A. Smolen. Privacy: Is There An
J. Tabert, and P. Powledge. Location disclosure to social App for That? In Proceedings of the Symposium on Usable
relations: why, when, & what people want to share. In Privacy and Security (SOUPS), 2011.
Proceedings of the ACM CHI Conference on Human Factors [27] P. Kumaraguru and L. F. Cranor. Privacy Indexes: A Survey
in Computing Systems, 2005. of Westin’s Studies. Technical report, Carnegie Mellon
[11] L. F. Cranor. A Framework for Reasoning about the Human University CMU-ISRI-5-138, 2015.
in the Loop. In Proceedings of the Conference on Usability, [28] R. Leung, L. Findlater, J. McGrenere, P. Graf, and J. Yang.
Psychology, and Security. USENIX Association, 2008. Multi-Layered Interfaces to Improve Older Adults’ Initial
[12] S. Egelman, L. F. Cranor, and J. Hong. You’ve Been Warned: Learnability of Mobile Applications. ACM Transactions on
An empirical study of the effectiveness of web browser Accessible Computing (TACCESS), 2010.
phishing warnings. In Proceedings of the ACM CHI [29] J. Lindqvist, J. Cranshaw, J. Wiese, J. Hong, and
Conference on Human Factors in Computing Systems, 2008. J. Zimmerman. I’m the mayor of my house: examining why
[13] W. Enck, P. Gilbert, B.-G. Chun, L. P. Cox, J. Jung, people use Foursquare - a social-driven location sharing
P. McDaniel, and A. N. Sheth. TaintDroid: An application. In Proceedings of the ACM CHI Conference on
Information-Flow Tracking System for Realtime Privacy Human Factors in Computing Systems, 2011.
Monitoring on Smartphones. In Proceedings of the [30] W. A. Magat, W. K. Viscusi, and J. Huber. Consumer
Symposium on Operating Systems Design and Processing of Hazard Warning Information. Journal of Risk
Implementation (OSDI), 2010. and Uncertainty, 1, 1988.
[14] W. Enck, D. Octeau, P. McDaniel, and S. Chaudhuri. A study [31] S. Motiee, K. Hawkey, and K. Beznosov. Do windows users
of Android application security. In Proceedings of the follow the principle of least privilege?: investigating user
USENIX Security Symposium, 2011. account control practices. In Proceedings of the Symposium
[15] W. Enck, M. Ongtang, and P. McDaniel. On lightweight on Usable Privacy and Security (SOUPS), 2010.
mobile phone application certification. In Proceedings of the [32] K. Mueller and K. Butler. Flex-P: Flexible Android
ACM Conference on Computer and Communication Security Permissions. IEEE Symposium on Security and Privacy,
(CCS), 2009. Poster Session, 2011.
[16] A. P. Felt, E. Chin, S. Hanna, D. Song, and D. Wagner. [33] M. Nauman, S. Khan, M. Alam, and X. Zhang. Apex:
Android Permissions Demystified. In Proceedings of the Extending Android Permission Model and Enforcement with
ACM Conference on Computer and Communication Security User-defined Runtime Constraints. In ACM Symposium on
(CCS), 2011. Information, Computer and Communications Security
[17] A. P. Felt, M. Finifter, E. Chin, S. Hanna, and D. Wagner. A (ASIACCS), 2010.
Survey of Mobile Malware in the Wild. In Proceedings of the [34] P. Nickinson. Android Market now has more than a
ACM Workshop on Security and Privacy in Mobile Devices quarter-million applications, 2011.
(SPSM), 2011. [35] F. Roesner, T. Kohno, A. Moshchuk, B. Parno, H. Wang, and
[18] A. P. Felt, K. Greenwood, and D. Wagner. The Effectiveness C. Cowan. User-Driven Access Control: Rethinking
of Application Permissions. In Proceedings of the USENIX Permission Granting in Modern Operating Systems. In
Conference on Web Application Development (WebApps),
 Proceedings of the IEEE Conference on Security and
Privacy, 2012.
[36] N. Sadeh, J. Hong, L. Cranor, I. Fette, P. Kelley,
M. Prabaker, and J. Rao. Understanding and capturing
people’s privacy policies in a mobile social networking
application. Personal and Ubiquitous Computing, 2009.
[37] S. E. Schechter, R. Dhamija, A. Ozment, and I. Fischer. The
Emperor’s New Security Indicators. In Proceedings of the
IEEE Symposium on Security and Privacy, 2007.
[38] S. Sen and D. Lerman. Why are you telling me this? An
examination into negative consumer reviews on the web.
Journal of Interactive Marketing, 21, 2007.
[39] B. Shneiderman. Promoting universal usability with
multi-layer interface design. In Proceedings of the
Conference on Universal Usability (CUU), 2003.
[40] J. Sunshine, S. Egelman, H. Almuhimedi, N. Atri, and L. F.
Cranor. Crying Wolf: An Empirical Study of SSL Warning
Effectiveness. In Proceedings of the USENIX Security
Symposium, 2009.
[41] J. Sunshine, S. Egelman, H. Almuhimedi, N. Atri, and L. F.
Cranor. Crying Wolf: An Empirical Study of SSL Warning
Effectiveness. In Proceedings of the USENIX Security
Symposium, 2009.
[42] H. Taylor. Most People are “Privacy Pragmatists” Who,
While Concerned about Privacy, Will Sometimes Trade It
Off for Other Benefits. Harris Interactive, March 2003.
[43] S. Thurm and Y. I. Kane. Your apps are watching you, 2010.
[44] M. S. Wogalter. Communication-Human Information
Processing (C-HIP) Model. In Handbook of Warnings.
Lawrence Erlbaum Associates, 2006.
[45] M. S. Wogalter. Purpose and scope of warnings. In
Handbook of Warnings. Lawrence Erlbaum Associates,
2006.
[46] Y. Zhou, Z. Wang, W. Zhou, and X. Jiang. Hey, You, Get Off
of My Market: Detecting Malicious Apps in Official and
Alternative Android Markets. In Proceedings of the Network
and Distributed System Security Symposium (NDSS), 2012.
[47] F. Zhu and X. Zhang. Impact of Online Consumer Reviews
on Sales: The Moderating Role of Product and Consumer
Characteristics. Journal of Marketing, 74, 2010.