BIG IOT AND SOCIAL NETWORKING DATA FOR SMART CITIES Alg.docx

BIG IOT AND SOCIAL NETWORKING DATA FOR SMART
CITIES:
Algorithmic improvements on Big Data Analysis in the context
of RADICAL city
applications
Evangelos Psomakelis12,Fotis Aisopos1, Antonios Litke1,
Konstantinos Tserpes21, Magdalini
Kardara1 and Pablo Martínez Campo3
1Distributed Knowledge and Media Systems Group, National
Technical University of Athens, Zografou Campus, Athens,
Greece
2Informatics and Telematics Dept, Harokopio University of
Athens, Greece
3Communications Engineering department, University of
Cantabria, Santander, Spain
{fotais, litke, nkardara, tserpes,
vpsomak}@mail.ntua.gr,[email protected]
Keywords: Internet of Things, Social Networking, Big Data
Aggregation and Analysis, Smart City applications,
Sentiment Analysis, Machine Learning
Abstract: In this paper we present a SOA (Service Oriented
Architecture)-based platform, enabling the retrieval and

analysis of big datasets stemming from social networking (SN)
sites and Internet of Things (IoT) devices,
collected by smart city applications and socially-aware data
aggregation services. A large set of city
applications in the areas of Participating Urbanism, Augmented
Reality and Sound-Mapping throughout
participating cities is being applied, resulting into produced sets
of millions of user-generated events and
online SN reports fed into the RADICAL platform. Moreover,
we study the application of data analytics such
as sentiment analysis to the combined IoT and SN data saved
into an SQL database, further investigating
algorithmic and configurations to minimize delays in dataset
processing and results retrieval.
1 INTRODUCTION
Modern cities are increasingly turning towards
ICT technology for confronting pressures associated
with demographic changes, urbanization, climate
change (Romero Lankao, 2008) and globalization.
Therefore, most cities have undertaken significant
investments during the last decade in ICT
infrastructure including computers, broadband

connectivity and recently sensing infrastructures.
These infrastructures have empowered a number of
innovative services in areas such as participatory
sensing, urban logistics and ambient assisted living.
Such services have been extensively deployed in
several cities, thereby demonstrating the potential
benefits of ICT infrastructures for businesses and the
citizens themselves. During the last few years we
have also witnessed an explosion of sensor
deployments and social networking services, along
with the emergence of social networking (Conti et al.,
2011) and internet‐of‐things technologies (Perera et
al., 2013; Sundmaeker et al., 2010) Social and sensor
networks can be combined in order to offer a variety
of added‐value services for smart cities, as has
already been demonstrated by various early internet‐
of‐things applications (such as WikiCity(Calabrese et
al., 2007), CitySense(Murty et al., 2007),
GoogleLatitude(Page and Kobsa, 2010)), as well as
applications combining social and sensor networks

(as for example provided by (Breslin and Decker,
2007; Breslin et al., 2009) and (Miluzzo et al., 2007).
Recently, the benefits of social networking and
internet‐of‐things deployments for smart cities have
also been demonstrated in the context of a range of
EC co‐funded projects (Hernández-Muñoz et al.,
2011; Sanchez, 2010).
Current Smart City Data Analysis implies a wide
set of activities aiming to turn into actionable data the
outcome of complex analytics processes. This
analysis comprises among others: i) analysis of
thousands of traffic, pollution, weather, waste, energy
and event sensory data to provide better services to
the citizens, ii) event and incident analysis using near
real-time data collected by citizens and devices
sensors, iii) turning social media data related to city
issues into event and sentiment analysis , and many

others. Combining data from physical
(sensors/devices) and social sources (social
networks) can give more complete, complementary
data and contributes to better analysis and insights. In
overall, smart cities are complex social systems and
large scale data analytics can contribute into their
sustainability, efficient operation and welfare of the
citizens.
Motivated by the modern challenges in smart
cities, the RADICAL approach (RADICAL, 2016)
opens new horizons in the development, deployment
and operation of interoperable social networking and
Internet of Things services in smart cities, notably
services that could be flexibly and successfully
customized and replicated across multiple cities. Its
main goal is to provide the means for cities and SMEs
to rapidly develop, deploy, replicate, and evaluate a
diverse set of sustainable ICT services that leverage

established IoT and SN infrastructures. Application
services deployed and pilotedinvolve: i) Cycling
Safety Improvement, ii) Products Carbon Footprint
Management, iii) Object‐driven Data Journalism, iv)
Participatory Urbanism, v) Augmented Reality, vi)
Eco‐ consciousness, vii) Sound map of a city and viii)
City-R-Us: a crowdsourcing app for collecting
movement information using citizens smartphones.
The RADICAL platform is an open platform
having as added value the capability to easily
replicate the services in other smart cities, the ability
to co-design services with the involvement of cities'
Living Labs, and the use of added value services that
deal with the application development, the
sustainability analysis and the governance of the
services.
The RADICAL approach emphasizes on the
sustainability of the services deployed, targeting both
environmental sustainability and business viability.

Relevant indicators (e.g., CO2 emissions, Citizens
Satisfaction) are established and monitored as part of
the platform evaluation.End users (citizens) in
modern smart cities are increasingly looking for
media‐rich services offered under different space‐,
context‐, and situational conditions. The active
participation and interaction of citizens can be a key
enabler for successful and sustainable service
deployments in future cities. Social networks hold the
promise to boost such participation and interaction,
thereby boosting participatory connected governance
within the cities. However, in order to enable smart
cities get insight information on how citizens think,
act and talk about their city it is important to
understand their opinion and sentiment polarity on
issues related to their city context. This is where
sentiment analysis can play a significant role. As
social media data bring in significant Big Data
challenges (especially for unstructured data streams)

it will be important to find effective ways to analyse
sentimentally those data for extracting value
information and within specific time windows.
This paper has the following contributions:
uniform social and IoT big data aggregation
and combination.
t Analysis
techniques efficiency, to reduce record,
retrieval, update and processing time.
-grams storage and
frequency representation in the context of big
data Sentiment Analysis.
The rest of the paper is structured as follows:
Section 2 gives an overview of related and similar
works that can be found in the international literature
and in projects funded by the European Commission.
Section 3 presents the RADICAL architecture and
approach. Section 4 presents details about the
Sentiment Analysis problem and related experiments,

while in section 5 we provide the future work to be
planned in the context of RADICAL and the
conclusions we have come into.
2 RELATED WORK
Recently, various analytical services such as
sentiment analysis found their way into Internet of
Things (IoT) applications. With the devices that are
able to convey human messages over the internet
meeting an exponential growth, the challenge now
revolves around big data issues. Traditional
approaches do not cope with the requirements posed
from applications for analytics in e.g. high velocity
rates or data volumes. As a result, the integration of
IoT with social sensor data put common tasks like
feature extraction, algorithm training or model
updating to the test.
Most of the algorithms are memory-resident and
assume a small data size (He et al., 2010) and once
this threshold is exceeded, the algorithms’ accuracy
and performance degrades to the point they are
useless. Therefore even if we focused solely on
volume challenges, it is intuitively expected that the
accuracy of the supervised algorithms will be
affected. An attempt from (Liu et al., 2013) to use
Naïve Bayes in an increasingly large data volume,
showed that a rapid fall of the algorithms accuracy is
followed by a continuous, smooth increase

asymptotically tending from the lower end to the
baseline (best accuracy under normal data load).
Rather than testing the algorithm’s limitations,
most of the other approaches are focusing on
implementing parallel and distributed versions of the
algorithms such as (He et al., 2010; Read et al., 2015).
In fact most of them rely on the Map-Reduce
framework so as to achieve high throughput
classification (Amati et al., 2014; Sakaki et al., 2013;
Wang et al., 2012; Zhao et al., 2012) whereas a
number of toolkits have been presented with
implementations of distributed or parallel versions of
machine learning algorithms such as (“Apache
Mahout: Scalable machine learning and data mining,”
n.d., “MEKA: A Multi-label Extension to WEKA,”
n.d.; Bifet et al., 2010). While these solutions put
most of the emphasis in the model and the
optimization of the classification task in terms of
accuracy and throughput, there is a rather small body
of research dealing with the problem of feature
extraction in high pace streams. The standard solution
that is considered is the use of a sliding window and
the application of standard feature extraction
techniques in this small set. In cases where the
stream’s distribution is variable, a sliding window
kappa-based measure has been proposed (Bifet and
Frank, 2010).
As reported in (Strohbach et al., 2015), another
domain of intense research in the area of scalable
analytics is for an architecture that combines both
batch and stream processing over social and IoT data
while at the same time considering a single model for
different types of documents (e.g. tweets Vs

blogposts). Sentiment analysis is a typical task that
requires batch modeling in order to generate the
golden standards for each of the classes. This process
is also the most computationally intense, as the
classification task itself is usually a CPU bound task
(i.e. run the classification function). In a data
streaming scenario the golden standards must be
updated in a batch mode, whereas the feature
extraction and classification must take place in real
time.
Perhaps the most prominent example of such an
architecture is the Lambda Architecture (Marz and
Warren, 2015) pattern which solves the problem of
computing arbitrary functions on arbitrary data in
realtime by combining a batch layer for processing
large scale historical data and a streaming layer for
processing items being retrieved in real time from an
input queue or analytics in e.g. high velocity rates or
data volumes. As a result, the integration of IoT with
social sensor data put common tasks like feature
extraction, algorithm training or model updating to
the test.Most of the algorithms are memory-resident
and assume a small data size (He et al., 2010) and
once this threshold is exceeded, the algorithms’
accuracy and performance degrades to the point they
are useless. Therefore even if we focused solely on
volume challenges, it is intuitively expected that the
accuracy of the supervised algorithms will be
affected. An attempt from Liu et al (Liu et al., 2013)
to use Naïve Bayes in an increasingly large data
volume, showed that a rapid fall of the algorithms
accuracy is followed by a continuous, smooth
increase asymptotically tending from the lower end to
the baseline (best accuracy under normal data load).

3. THE RADICAL APPROACH
The RADICAL platform integrates components
and tools from (SocIoS, 2013) and (SmartSantander,
2013) projects, in order to support innovative smart
city services, leveraging information stemming from
Social Networks (SN) and Internet of Things devices.
Using the aforementioned tools, it can collect,
combine, analyze, process, visualize and provide
uniform access to big datasets of Social Network
content (e.g. tweets) and Internet of Things
information (e.g. sensor measurements or citizen
smartphone reports).
The architecture of the RADICAL platform is
depicted in Figure 1. As can be observed, all IoT data
are pushed into the platform through the respective
Application Programming Interfaces (IoT API and
Repository API) and are forwarded to the RADICAL
Repository, comprised by a MySQL database, formed

based on the RADICAL Object Model. The device-
related data, as dictated by this object model, are
saved in the form of Observations and Measurements.
Observations correspond to general IoT events
reported (e.g. a sensor report or bicycle "check-in"
event), while Measurements to more specific metrics
included in an Observation (e.g. Ozone
measurements (mpcc) or bicycle current speed
(km/h)). On the other hand, SN data are accessed in
real time from the underlying SN adaptors, by
communicating with the respective Networks’ APIs.
In cases of Social Networks like Foursquare that
provide plain venues and statistics, the adaptor-like
data structures do not make sense, thus relevant
Social Enablers are used to retrieve venue-related
information data.
On top of the main platform, RADICAL delivers
a set of tools (Application Management layer) that

allow end users to make better use of the RADICAL
platform, such as configuring the registered IoT
devices or extracting general activity statistics,
through the RADICAL Configuration API. Lastly,
the RADICAL Data API allows smart city services to
access the different sources of information (social
networks, IoT infrastructures, city applications),
combine data and perform data analysis by using the
appropriate platform tools.
Figure 1: RADICAL Platform Architecture
As can be seen in the Service Application Layer,
in the context of RADICAL a wide range of Smart
City services of various scopes has been developed:
izen journalism and Participatory
Urbanism: Those two interrelated services
allow citizens reporting events ofinterest in the

city, by posting images, text and metadata
through their smartphones.
sensors can report the situation in the city
streets through their smartphones.
products, people and services: By using a
range of sensors, the CO2 emissions in specific
places in a city may be monitored.
d Reality in Points of Interest
(POI): Tourists use their smartphones to
identify and receive information about points
of interest in a city.
-consciousness:Leverages
on the viral effect in the propagation of
information in the social networks as well as
the recycling policy of a city, through
monitoring and reporting relevant actions on
citizens' smartphones.
-Orientated Urban Noise Decibel

Measurement Application: Noise sensors are
employed throughout the city and citizens are
able to report and comment noise-related
information through SNs under a hashtag.
Urban Services: This service gathers sensory
data along with SN check-ins in city venues, to
construct a traffic map throughout the city,
leveraging the process load of anycentralized
decision making process.
The aforementioned services are piloted in six
European participating cities: Aarhus, Athens,
Genoa, Issy les Moulineaux, Santander and the region
of Cantabria. Figure 2illustrates a screenshot example
of the RADICAL Cities' Dashboard, where general
statistics on device registration and activity for a
service throughout different cities in a specific time
RADICAL
Data

Repository
City
Applications
Radical Data API
Service Application Layer
Sound MAPP
Municipality
Cockpit
City R-US
Service
aggregator
Repository
Gateway
Application Management Tools
Governance
Toolkit
Application
Development
Toolkit
Sustainability
services
Resource

Directory
Service
Storer
IoT Manager Event Broker
Node
Manager
Data Rep.
Configurator
Radical Configuration API
IoT devices
Repository API
Configuration Tools
Social Media
…
Register
Manager
IOT API SN Adaptors
SocIoS Core Service
SN Enablers
Event
Detection

Sentiment
Analysis
GWFPSens GW4IoT Devices GW4Serv…
Platform Tools
Carbon
Footprint
Augmented
Reality
Eco-
Conciousness
Object Driven
Journalism
Participatory
Urbanism
Cycling
Safety
Social Networks -
Venues Services
period is provided. In overall, during the last pilot

iteration, RADICAL Repository had captured a total
of 5.636 active IoT devices sending 728.253
Observations and 5.461.776 Measurements.
Most of the services above depend on the
aggregation of those IoTdata with social data
stemming from online Social Network sites. E.g. in
the Participatory Urbanism service, citizens' reports
sent through smartphones and saved in RADICAL
Repository are combined with relevant tweets (under
a city service hashtag), as well as POI information
that can be collected from similar SNs.
Figure 2: RADICAL Cities Dashboard presents smartphone
registrations and measurements for the AR service in the cities
of Santander and Cantabria over a period
Thus, given the size of the datasets acquired by
smart city services, along with the rich social media
content that can be retrieved through the RADICAL

platform adaptors, big data aggregation and analysis
challenges arise. Data Analysis tools are the ones that
further process the data in order to provide
meaningful results to the end user, i.e. Event
Detection or Sentiment Analysis.
When it comes to Big Data, as in the RADICAL
case, where millions of user-reported events are
aggregated along with millions of SN posts and an
extraction of general results is required, the challenge
accrued is two-fold: First, the tool must ensure the
accuracy of the analysis, in the sense that data
classification is correct to a certain and satisfactory
extent, and second, processing time must be kept
under certain limits, so that results retrieval process
delay is tolerable by end-users. Moreover, it is
apparent in such analysis that a trade-off between
effectiveness and efficiency exists. The latter is a
most crucial issue in Big Data analysis and apart from

the policy followed in data querying (e.g. for queries
preformed in an SQL database), it is also related to
the algorithmic techniques employed foranalyzing
those datasets.
In the context of this work, we focus on the
Sentiment Analysis on the big IoT and SN related
datasets of RADICAL, as this was the most popular
functionality among participating cities and almost all
of the RADICAL Smart City services presented
above make use of it. The goal of the Sentiment
Analysis service is to extract sentiment expressive
patterns from user-generated content in social
networks or IoT-originated text posts. The service
comes to the aid of the RADICAL city administrators,
helping them to categorize polarized posts, meaning
sentimentally charged text, e.g. analyse citizens’
posts to separate subjective from objective opinions
or count the overall positive and negative feedback,

concerning a specific topic or event in the city.
4. SENTIMENT ANALYSIS
EXPERIMENTS AND
PERFORMANCE IMPROVEMENT
4.1 Introduction
The term Sentiment Analysis refers to an
automatic classification problem. Its techniques are
trying to distinguish between sentences of natural
language conveying positive (e.g. happiness, pride,
joy), negative (e.g. anger, sadness, jealously) or even
neutral (no sentiment texts like statements, news,
reports)emotion (called sentiment for our purposes)
(Pang et al., 2002).
A human being is capable of understanding a
great variety of emotions from textual data. This
process of understanding is based on complicated
learning procedures that we all go through while
using our language as a means of communication, be
it actively or passively. It requires imagination and
subjectivity in order to fully understand the meaning
and hidden connections of each word in a sentence,
two things that machines lack.
The most common practice is to extract numerical
features out of the natural language (Godbole et al.,
2007). This process translates this complex means of
communication into something the machine can
process.

4.2 Natural Language Processing
In order to process the natural language data, the
computer has to take some pre-processing actions.
These actions include the cleansing of irrelevant,
erroneous or redundant data and the transformation of
the remaining data in a form more easily processed.
Cleansing the data has become a subjective task,
depending on the purposes of each researcher and the
chosen machine learning algorithms. The
transformation of the sentences in another form now
is clearly studied and each approach has some
advantages and disadvantages. This paper will detail
three approaches, two widely used and one that had
some success in improving the accuracy of the
algorithms: the bag of word, N-Gramsand N-Gram
Graphs(Aisopos et al., 2012; Fan and Khademi, 2014;
Giannakopoulos et al., 2008; Pang and Lee, 2008).
The bag of words approach is perhaps the most
simple and common one. It regards each sentence as
a set of words, disregarding their grammatical
connections and neighbouring relations. It splits each
sentence based on the space character (in most
languages) and then forms a set of unrelated words (a
bag of words as it is commonly called). Then each
word in this bag can be disregarded or rated by a
numerical value, in order to create a set of numbers
instead of words.
The N-Grams are a bit more complex. They also
form a bag of words but now each sentence is split
into pseudo-words of equal length. A sliding window

of N characters is rolling on the sentence creating this
bag of pseudo-words. For example if N=3 the
sentence “This is a nice weather we have today!” will
be split in the bag {‘Thi’, ‘his’, ‘is ’, ‘s i’, ‘ is’, ‘is ’,
‘s a’, ‘ a ’, ‘a n’, ‘ ni’, ‘nic’, ‘ice’, ‘ce ’, ‘e w’, ‘ we’,
‘wea’, ‘eat’, ‘ath’, ‘the’, ‘her’, ‘er ’, ‘r w’, ‘ we’, ‘we
’, ‘e h’, ‘ ha’, ‘hav’, ‘ave’, ‘ve ’, ‘e t’, ‘ to’, ‘tod’,
‘oda’, ‘day’, ‘ay!’}.
This technique takes into regard the direct
neighbouring relations by creating a continuous
stream of words, it still ignores the indirect relations
between words and even the relations between the
produced N-Grams. Of course it is impossible to have
a predefined set of numerical ratings for each one of
these pseudo-words because each sentence and each
N number (which is defined arbitrarily by the
researcher) produces a different set of pseudo-
words(Psomakelis et al., 2014). So machine learning
is commonly used to replace these words with
numerical values and create sets of numbers which
can be aggregated to sentence level.
An improvement on that approach aims to take
into consideration the neighbouring relations between
the produced N-Grams. This approach is called N-
Gram Graphs and its main concept is to create a graph
connecting each N-Gram with its neighbours in the
original sentence. So each node in this graph is an N-
Gram and each edge is a neighbouring
relation(Giannakopoulos et al., 2008). This approach
gives a variety of new informationto the researchers
and to the machine learning algorithms, including
information about the context of words, making it a
clear improvement of the simple N-Grams(Aisopos et
al., 2012). The only drawbacks are the complexity it

adds to the process and the difficulties of storing,
accessing and updating a graph of textual data.
4.3 Dataset Improvements
At the core of sentiment analysis is its dataset. We
are gathering and employing bigger and bigger
datasets in order to better train the algorithms to
distinguish what is positive and what is negative.
Classic storage techniques are proving more and more
cumbersome for large datasets. ArrayLists and most
Collections are adding a big overhead to the data so
they are not only enlarging the space requirements for
its storage but they are also delaying the analysis
process. So new techniques for data storage and
retrieval are needed, techniques that will enable us to
store even bigger datasets and access them with even
smaller delays.
The most commonly used such technique is the

Hash List(Fan and Khademi, 2014), which first
hashes the data in a certain, predefined amount of
buckets and then creates a List in each bucket to
resolve any collisions. This method’s performance is
heavily dependent on the quality of the hash function
and its ability to equally split the data into the buckets.
The target is to have as small lists as possible. That is
the case because finding the right bucket for a certain
piece of data is done in O(1) time but looking through
the List in that bucket for the correct spot to store the
piece of data is done in O(n) time where n is the
number of data pieces in the List.
Moreover, in Java which is the programming
language that we are using, each List is an object
containing one object for each data piece. All these
objects create an overhead that is not to be ignored. In
detail the estimated size that a hash list will occupy is
calculated as:

12 + ((B − E) ∗ 12) + (E ∗ 4)
+ (U ∗ (N ∗ 2 + 72))
Equation 1: Size estimation of Hash List where N=NGram
Length, U=Unique NGrams, B=Bucket Size, E=Empty
Buckets.
The worst case for storage but best for access time
is when almost each data piece has its own bucket. In
this case, for N=5, S=11881376, U=S, B=(26^N)*2,
E=11914220, we have a storage size of 1110 MB. The
best case for storage but worse for access time is when
all data pieces are in a small number of buckets, in big
lists. In this case for N=5, S=11881376, U=1,
B=(26^N)/2, E=200610 we have a storage size of 23
MB. In an average case of N=5, S=11881376,
U=7510766, B=26^N, E=2679046 we have 682 MB
of storage space needed. The sample for the above
examples was the complete range of 5-Grams for the
26 lowercase English characters which are 26^5 =

11881376.
Our proposed technique now, the one that we call
Dimensional Mapping, has a standard storage space,
depending only on the length of the N-Grams. The
idea is to store only the weight of each N-Gram with
the N-Gram itself being the pointer to where it is
stored. That is achieved by creating an N-dimensional
array of integers where each character of the N-Gram
is used as an index. So, in order to access the weight
of the 5-Gram ‘fdsgh’ in the table DM we would just
read the value in cell DM[‘f’][‘d’][‘s’][‘g’][‘h’]. A
very simple mapping is used between the characters
and an integers: after a very strict cleansing process
where we convert all characters in lowercase and
discard all characters but the 26 in the English
alphabet, we are just subtracting the ASCII value of
‘a’. Due to the serial nature of the characters that
gives us an integer between 0 and 26 that we can use

as an index. A more complex mapping can be used in
order to include more characters or even punctuation
that we now ignore.
The Dimensional Mapping has a standard storage
size requirement, dependent only on the length of the
N-Grams as we mentioned before. The size it
occupies can be estimated by the following formula:
(26� ) ∗ 4 + ∑ ((26�−� ) ∗ 12)
�=�
�=1
Equation 2: Dimensional Mapping size estimation with N
being the length of N-Grams.
This may seem large but for the 5-Grams the
estimated size is just 51 MB. Compared to the worst
case of Hash Lists (1110 MB) or even the average
case (682 MB) it seems like a huge improvement.
This is caused due to the fact that the

multidimensional array stores primitive values and
not objects, which reduces the overhead greatly.
Moreover, we can now say that accessing and
updating a certain data piece can be done in O(1) time
with absolute certainty, with no dependency on the
data itself or a hash function. This had significant
results in speeding up the execution times of the
analysis, enabling us to look into streaming data and
semi-supervised machine learning algorithms.
4.4 Results
We measured three main KPIs for the result
comparison. Two of them (success ratio, kappa
variable) were measuring the success ratio of
classification and one (execution time) the
algorithmic improvement. We present them bellow.
We run experiments on 5-Grams stored in classic
ArrayList format, in Hash Lists and in Dimensional
Mapping. After storing the N-Grams in these formats

we applied a 10-fold cross validation on each one of
the seven machine learning algorithms we chose:
Naïve Bayesian Networks, C4.5, Support Vector
Machines, Logistic Regression, Multilayer
Perceptrons, Best-First Trees and Functional Trees.
Then we recorded the three KPIs for each one of these
21 experiments. The results for the first two KPIs are
shown in the bar chart that follows. In the same chart
we have included the KPIs for a threshold based
classification, using an arbitrarily set threshold.
Figure 3: A comparison of the three KPIs as shown in the
sentiment analysis experiments
As of the execution times the following table
contains a summary of the results:
Table 1: Execution time in seconds summary - comparing

for the various algorithms and techniques
ArrayLists Hash
List
Dimensional
Mapping
Thresholds 1691 5 4
Naïve
Bayes
12302 7 7
C4.5 21535 9 8
SVM 20662 147 177
Logistic
Regression
22251 9 11
MLP 21224 41 48
BFTree 23319 25 19
FTree 22539 16 16
5. CONCLUSIONS

RADICAL platform, as presented in the current
work, successfully combines citizens' posts retrieved
through smartphone applications and Social
Networks in the context of smart city applications, to
produce a testbed for applying multiple analysis
functionalities and techniques. The exploitation of
resulting big aggregated datasets pose multiple
challenges, with timely-efficient analysis being the
most important. Focusing on data storage and
representation, multiple techniques were examined in
the experiments performed, in order to come up with
the optimal algorithmic approach of Dimensional
Mapping. In the future the authors plan to use even
larger and more complex datasets, further leveraging
on the effectiveness of these social networking
services.
ACKNOWNLEDGEMENTS
This work has been supported by RADICAL and

Consensus projects and has been funded by the
European Commission's Competitiveness and
Innovation Framework Programme under grant
agreements no 325138 and 611688 respectively.
REFERENCES
Aisopos, F., 0001, G.P., Tserpes, K., Varvarigou,
T.A., 2012. Content vs. context for
sentiment analysis: a comparative analysis
over microblogs., in: Munson, E.V.,
Strohmaier, M. (Eds.), HT. ACM, pp. 187–
196.
Amati, G., Angelini, S., Bianchi, M., Costantini, L.,
Marcone, G., 2014. A scalable approach to
near real-time sentiment analysis on social
networks. Inf. Filter. Retr. 12.
Apache Mahout: Scalable machine learning and data
mining [WWW Document], n.d. URL
http://mahout.apache.org/ (accessed 4.8.15).

Bifet, A., Frank, E., 2010. Sentiment Knowledge
Discovery in Twitter Streaming Data, in:
Proceedings of the 13th International
Conference on Discovery Science, DS’10.
Springer-Verlag, Berlin, Heidelberg, pp. 1–
15.
Bifet, A., Holmes, G., Kirkby, R., Pfahringer, B.,
2010. MOA: Massive Online Analysis. J.
Mach. Learn. Res. 11, 1601–1604.
Breslin, J., Decker, S., 2007. The Future of Social
Networks on the Internet: The Need for
Semantics. IEEE Internet Comput. 11, 86–
90. doi:10.1109/MIC.2007.138
Breslin, J.G., Decker, S., Hauswirth, M., Hynes, G.,
Phuoc, D.L., Passant, A., Polleres, A.,
Rabsch, C., Reynolds, V., 2009. Integrating
Social Networks and Sensor Networks, in:

Proceedings on the W3C Workshop on the
Future of Social Networking.
Calabrese, F., Kloeckl, K., Ratti, C., 2007. Wikicity:
Real-time location-sensitive tools for the
city IEEE Pervasive Computing, 390–413.
Conti, M., Passarella, A., Pezzoni, F., 2011. A model
for the generation of social network graphs.,
in: WOWMOM. IEEE Computer Society,
pp. 1–6.
Fan, M., Khademi, M., 2014. Predicting a Business
Star in Yelp from Its Reviews Text Alone.
CoRR abs/1401.0864.
Giannakopoulos, G., Karkaletsis, V., Vouros, G.A.,
Stamatopoulos, P., 2008. Summarization
system evaluation revisited: N-gram graphs.
TSLP 5.
Godbole, N., Srinivasaiah, M., Skiena, S., 2007.
Large-Scale Sentiment Analysis for News

and Blogs, in: Proceedings of the
International Conference on Weblogs and
Social Media (ICWSM).
GrowSmarter, 2016. Grow Smarter project [WWW
Document]. URL http://www.grow-
smarter.eu/home/
Gubbi, J., Buyya, R., Marusic, S., Palaniswami, M.,
2012. Internet of Things (IoT): A Vision,
Architectural Elements, and Future
Directions. CoRR abs/1207.0203.
He, Q., Zhuang, F., Li, J., Shi, Z., 2010. Parallel
Implementation of Classification
Algorithms Based on MapReduce, in: Yu, J.,
Greco, S., Lingras, P., Wang, G., Skowron,
A. (Eds.), Rough Set and Knowledge
Technology, Lecture Notes in Computer
Science. Springer Berlin Heidelberg, pp.
655–662.

Hernández-Muñoz, J.M., Vercher, J.B., Muñoz, L.,
Galache, J.A., Presser, M., Gómez, L.A.H.,
Pettersson, J., 2011. Smart Cities at the
Forefront of the Future Internet., in:
Domingue, J., Galis, A., Gavras, A.,
Zahariadis, T.B., Lambert, D., Cleary, F.,
Daras, P., Krco, S., Müller, H., Li, M.-S.,
Schaffers, H., Lotz, V., Alvarez, F., Stiller,
B., Karnouskos, S., Avessta, S., Nilsson, M.
(Eds.), Future Internet Assembly, Lecture
Notes in Computer Science. Springer, pp.
447–462.
Liu, B., Blasch, E., Chen, Y., Shen, D., Chen, G.,
2013. Scalable sentiment classification for
Big Data analysis using Na #x00EF;ve
Bayes Classifier, in: Big Data, 2013 IEEE
International Conference on. pp. 99–104.
doi:10.1109/BigData.2013.6691740

Marz, N., Warren, J., 2015. Big Data: Principles and
best practices of scalable realtime data
systems, 1 edition. ed. Manning
Publications, Westampton.
MEKA: A Multi-label Extension to WEKA [WWW
Document], n.d. URL
http://meka.sourceforge.net/ (accessed
3.27.15).
Miluzzo, E., Lane, N.D., Eisenman, S.B., Campbell,
A.T., 2007. CenceMe - Injecting Sensing
Presence into Social Networking
Applications., in: Kortuem, G., Finney, J.,
Lea, R., Sundramoorthy, V. (Eds.),
EuroSSC, Lecture Notes in Computer
Science. Springer, pp. 1–28.
Murty, R., Gosain, A., Tierney, M., Brody, A., Fahad,
A., Bers, J., Welsh, M., 2007. Harvard
University Technical Report. TR‐13‐07.

Page, X.W., Kobsa, A., 2010. Navigating the social
terrain with google latitude. iConference
Urbana‐Champaign, p.174‐178.
Pang, B., Lee, L., 2008. Opinion mining and
sentiment analysis. Found. Trends Inf. Retr.
2, 1–135.
Pang, B., Lee, L., Vaithyanathan, S., 2002. Thumbs
up? Sentiment Classification Using Machine
Learning Techniques, in: emnlp2002.
Philadelphia, Pennsylvania, pp. 79–86.
Perera, C., Zaslavsky, A.B., Christen, P.,
Georgakopoulos, D., 2013. Sensing as a
Service Model for Smart Cities Supported
by Internet of Things. CoRR abs/1307.8198.
Psomakelis, E., Tserpes, K., Anagnostopoulos, D.,
Theodora, V., 2014. Comparing Methods for
Twitter Sentiment Analysis. Proc. 6th Int.
Conf. Knowl. Discov. Inf. Retr. - KDIR
2014 225–232.

RADICAL, 2016. Rapid Deployment for Intelligent
Cities and Living, FP7 EU funded Research
project [WWW Document]. URL
http://www.radical-project.eu
Read, J., Martino, L., Olmos, P.M., Luengo, D., 2015.
Scalable multi-output label prediction: From
classifier chains to classifier trellises.
Pattern Recognit. 48, 2096–2109.
doi:10.1016/j.patcog.2015.01.004
Romero Lankao, P., 2008. Urban areas and climate
change: Review of current issues and trends.
Issues Pap. 2011 Glob. Rep. Hum. Settl.
Sakaki, T., Okazaki, M., Matsuo, Y., 2013. Tweet
analysis for real-time event detection and
earthquake reporting system development.
Knowl. Data Eng. IEEE Trans. On 25, 919–

931.
Sanchez, L., 2010. SmartSantander: Experimenting
The Future Internet in the City of the Future.
Presented at the PIMRC2010, Istanbul,
Turkey.
SmartSantander, 2013. SmartSantander, FP7 EU
funded Research project [WWW
Document]. URL
http://www.smartsantander.eu/
SocIoS, 2013. Exploiting Social Networks for
Building the Future Internet of Services, FP7
EU funded Research project [WWW
Document]. URL
http://www.sociosproject.eu/
Strohbach, M., Ziekow, H., Gazis, V., Akiva, N.,
2015. Towards a big data analytics
framework for IoT and smart city
applications, in: Modeling and Processing

for Next-Generation Big-Data
Technologies. Springer, pp. 257–282.
Sundmaeker, H., Guillemin, P., Friess, P., Woelfflé,
S. (Eds.), 2010. Vision and Challenges for
Realising the Internet of Things.
Publications Office of the European Union,
Luxembourg.
Wang, H., Can, D., Kazemzadeh, A., Bar, F.,
Narayanan, S., 2012. A System for Real-
time Twitter Sentiment Analysis of 2012
U.S. Presidential Election Cycle., in: ACL
(System Demonstrations). The Association
for Computer Linguistics, pp. 115–120.
Zhao, J., Dong, L., Wu, J., Xu, K., 2012. MoodLens:
an emoticon-based sentiment analysis
system for chinese tweets., in: 0001, Q.Y.,
Agarwal, D., Pei, J. (Eds.), KDD. ACM, pp.
1528–1531.

Smart cities research:
Your paper should meet the following requirements:
• Be approximately 3-5 pages in length, not including the
required cover page and reference page.
• Follow APA guidelines. Your paper should include an
introduction, a body with fully developed content, and a
conclusion.
• Support your response with the readings from the course and
at least five peer-reviewed articles or scholarly journals to
support your positions, claims, and observations. The UC
Library is a great place to find resources.
• Be clear with well-written, concise, using excellent grammar
and style techniques. You are being graded in part on the
quality of your writing
Please go through below details:
Welcome to another exciting concept in our course – big data
and smart cities. Here we will be discussing smart cities in more
detail while watching a lecture including videos of some cities
that have implemented some smart city technology approaches.
So what is a smart city? Even today, there's no agreed
definition. One thing that's really clear about smart cities is
they're very specific to a city. The problems we have in one city
will be different than in another. Although the categories are
similar, how we approach them, the extent of the issue, and the
solutions are going to be very specific to each individual city.
Cities are changing rapidly—growing in size and density,
creating more waste, and using more resources. To solve the
problems these pressures create, city governments need to think
different. Smart cities need to use 21st century thinking and
technology to enable a better quality of life for their citizens.
Please watch the video lecture, read the Big IoT and Social
Networking for Smart Cities article, and complete the research
paper (to be submitted prior to Sunday for credit with no

plagiarism score(SafeAssign)).
This week's reading centered around how Big Data analytics can
be used with Smart Cities. This is exciting and can provide
many benefits to individuals as well as organizations. For this
week's research assignment, you are to search the Internet for
other uses of Big Data in RADICAL platforms. Please pick an
organization or two and discuss the usage of big data in
RADICAL platforms including how big data analytics is used in
those situations as well as with Smart Cities. Be sure to use the
UC Library for scholarly research. Google Scholar is the 2nd
best option to use for research.
.

BIG IOT AND SOCIAL NETWORKING DATA FOR SMART CITIES Alg.docx

Recommended

More Related Content

Similar to BIG IOT AND SOCIAL NETWORKING DATA FOR SMART CITIES Alg.docx (20)

More from jasoninnes20 (20)

Recently uploaded (20)

BIG IOT AND SOCIAL NETWORKING DATA FOR SMART CITIES Alg.docx