Visualizing CDR Data

Center for Research and Application for Satellite Remote Sensing
Yamaguchi University
Visualizing CDR Data

CDR data associating the information with the position information of the used
base station ID (Cell ID), it is possible to estimate the position when a certain
mobile phone communicates. Through such data processing, it is possible to
trace the movement / trajectory of a user of a certain mobile phone.
The format of CDR data differs depending on the provider, but basically
includes
● ID
● Time / timestamp,
● Geographical coordinates (latitude and longitude).
Data Format

Even if personal information is excluded, CDR data is still business confidential
data for mobile phone companies, and if personal information related to call
records leaks by any chance, it can lead to serious social impact. Therefore,
obtaining CDR data is not easy because it requires careful negotiations and
agreements between mobile phone companies and ministries.
As an alternative, the project investigated open data, which has a format
similar to that of CDR data, and represents the trajectory of people.
Data Acquisition Method

Reality Mining Dataset
Data collected by MIT Human Dynamics Lab in 2004, which showing the
trajectory of 100 individuals over a 9-month period.
This is the result of recording the Cell ID and Bluetooth transmission / reception
by the smartphone application for data collection and linking it to the location
information. You can register and download and use it from the project website.

How to download data?
1. Go to website:
http://realitycommons.media.mit.edu/realitymining4.html
1. Fill out the information requested by the website.
2. After submit this section, then you will receive an email in
your email address with a link to the requested dataset.
NOTE:
● The data from Reality Commons cannot use immediately, and it
must be convert format file first.
● https://opencellid.org/ This link is “The world's largest Open
Database of Cell Towers” that stored dataset with cell tower
locations.

iTIC Open Data Archive
● Taxi probe data published by the Intelligent Traffic Information Center Foundation,
a group of Thailand's automobile and traffic-related operators.
● It is containing 1-2 second frequency GPS logs of approximately 4,000 vehicles over
the period June to December 2017.
Although the method, accuracy, and frequency of location information acquisition are
different from CDR data, it is considered to be useful in terms of handling a large amount
of movement trajectory.cal

1. Go to website: https://www.iticfoundation.org/download
2. Fill out in the red box.
1
2

3. After press “confirm” button in first
page, then iTic Open Data Archives page
will appear.
4. Click the information that you want to
download.
5. Index of/data/probe-data page will
appear.
6. Click the data for download.
3
4
5
6

Because of the large amount of data, it is difficult to calculate and visualize
with general spreadsheet software. The following is a list of useful software.
● 3.3.1 PostgreSQL/PostGIS, Spatialite
● 3.3.2 MobMap
Data Analysis Methods

PostgreSQL/PostGIS, Spatialite
By using PostGIS, which extends the function of handling spatial data to
PostgreSQL, which is a typical relational database management system, the
aggregation and weighing of a large amount of movement trajectory data
can be made more efficient. Spatialite is a spatial data extension of SQLite,
which is a simple database management system.
The procedure of analyzing the movement trajectory data is mainly done by
sorting by ID and time / timestamp.

MobMap
MobMap is software that specializes in
visualizing such trajectory data and
provides the function of expressing
individual movements in moving images.
It runs on a web browser and can be
used free of charge.

Note: For data to visualize in Mobmap,
use data from iTic.
For Prepare subsetting data use method
as follow;
1. Download "A bundle of command-
line tools for managing SQLite
database files..." from
https://sqlite.org/download.html
2. Extract the executable binary files.
1

3. Prepare a SQL script on notepad (or other text editor) with the codes in (a). The [input file name],
[date], [output file name] shall be replaced, and tbl.id_str LIKE 'a%' is for filtering the data to reduce data
size, indicating filtering id_str starting with 'a'. The script might be saved with ".sql" format.
NOTE: Here is the code for prepare SQL script.
CREATE TABLE tbl (id_str varchar, valid integer, lat double, lon double, t
timestamp without time zone, speed integer, heading integer, for_hire_light
integer, engine_acc integer);
.separator ,
.import [input file name] tbl
CREATE TABLE hash (id_int INTEGER PRIMARY KEY, id_str varchar);
INSERT INTO hash (id_str) SELECT DISTINCT id_str FROM tbl;
.output [output file name]
SELECT id_int, tbl.id_str, t, lat, lon FROM tbl
LEFT JOIN hash ON tbl.id_str = hash.id_str
WHERE tbl.t LIKE "[date YYYY-MM-DD] %"
AND tbl.id_str LIKE 'a%'
;
3
a

4. Execute sqlite3.exe by go to “Command Prompt” and enter the command to access the
program's directory (a). Type command for use sqlite and print output (b).
5. Load the output on Mobmap .
a b
NOTE: Input file and SQL script should be locate in the same directory with sqlite.exe

1. Go to website: https://shiba.iis.u-tokyo.ac.jp/member/ueyama/mm/
2. Press “Launch” button.
1
2

3. After pressed “Launch” button, this page will
appear.
4. Press “Add moving data” icon (in the
position of the Start from here’s arrowhead) for
import file data(.csv).
5. Select data that you want to visualize.
6. Then specify the column including id, XY
coordinate, time.
7. Click “Start loading” button.
3
4
6
7

c. You can edit properties of your data by press the “Open configuration” button.
a. Play and Stop button for
visualize your trajectories data.
b. You can choose
which type of
background you want
to open with your
data by press this
button, which will
have the following
options ------>
a
c
b

Exposure population analysis using CDR data
A long-term analysis of CDR data allows us to observe the immigration
situation. With this data, the situation of evacuation and return after a
disaster can be accurately grasped, and cooperation with evacuation
destination infrastructure development, administrative services, and
reconstruction activities in the stricken area can be promoted efficiently and
effectively.
Use Case

Exposure population analysis in the 2015 Nepal earthquake
Analysis of anonymized CDR data of 12 million people after the 2015 Nepal
earthquake revealed shifting migration patterns of the affected people. In
particular, it was estimated that 390,000 people migrated out from the
Kathmandu Valley and moved to the surrounding areas and south-central
Nepal. These data are useful for planning humanitarian assistance activities.
Ref: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4779046/
Use Case

Flood Disaster Management
UN Global Pulse attempted to manage flood disasters by observing people's response to
flood disasters based on the frequency of calls from CDR data. The results are as follows.
● CDR data is extremely useful as a proxy indicator of population distribution.
● Public alerts are not always effective in alerting people.
● The trajectories of people read from the CDR data are useful for understanding the
process of flood impacts.
● Most of the calls made during the disaster were in the most affected areas.
These results indicate that CDR data can be useful in measuring the impacts of floods on
people and infrastructure, and their attention to disasters.
Use Case

Visualizing CDR Data

Recommended

More Related Content

Similar to Visualizing CDR Data (20)

More from NopphawanTamkuan (20)

Recently uploaded (20)

Visualizing CDR Data