Mingli Yue and Yihai Sun Oppgave
Mingli Yue and Yihai Sun Oppgave
By
Mingli Yue, Yihai Sun
Abstract:
A great interest of boat manufacturers is to improve their products by knowing how the boats
are used after sale. In order to gather information about the condition of usages, a system
needs to be developed in order to collect data from different marine electronics mounted on
the boat. Through this thesis work, we developed such data collecting system for leisure boats
which support CAN Bus the message-based protocol. The data collection system has been
developed and installed on a Linux-based embedded system connected to the CAN Bus
network through a gateway in our laboratory. Through the data collection system, all data
generated from different marine electronics in the network can be captured, filtered,
transmitted, displayed and then stored in the system. For data transmission and access, we
have implemented three methods through wired or wireless networks, i.e., the fixed Internet,
3G/LTE cellular networks and Wi-Fi networks.
Furthermore, the prototype implementation has been extensively tested in both lab and real-
life environment.
Keywords: CAN Bus, NMEA 2000, embedded system, data filtering, data transmission
Page 2 of 64 1.0
Marine Data Collection and Transmission
Preface
This report is the result of the master thesis IKT 590 (30 ECTS) which is part of our fourth
semester MSc study at the Faculty of Engineering and Science, University of Agder (UiA) in
Grimstad, Norway. The work on this project started from 1 January 2013 and ended on 3
June 2013. We have completed the main goal of our project “Marine Data Collection based
on Embedded System with Wired and Wireless Transmission”.
This project is part of the ECO-Boat MOL project which is funded by the Research council of
Norway. We would like to thank our project supervisors Dr. Lei Jiao and Professor Frank Y.
Li for the guidance in giving feedback on technical and content of report throughout this
project. Through this thesis work, we learnt a lot about project content and technical report
writing. We also thank Mr. Ahmad Noor for his constructive suggestions and technical
support. At last, we would like to give our thanks to Marex AS for providing their boats for
real-life experiments.
Grimstad
3 June 2013
Mingli Yue and Yihai Sun
1.0 Page 3 of 64
Marine Data Collection and Transmission
Contents
Abbreviations ........................................................................................................................ 9
1 Introduction ................................................................................................................ 11
1.1 Background and Motivation ................................................................................... 11
1.2 Problem Statement ................................................................................................ 11
1.3 Approaches ........................................................................................................... 11
1.4 Thesis Outline ........................................................................................................ 12
2 Technology Background ............................................................................................. 13
2.1 CAN Bus and NMEA 2000 ..................................................................................... 13
2.2 Access Modes ....................................................................................................... 13
2.2.1 Wired connection .......................................................................................... 13
2.2.2 Wi-Fi ............................................................................................................. 14
2.2.3 Cellular networks .......................................................................................... 14
2.3 Web Service .......................................................................................................... 14
2.4 Secure Shell .......................................................................................................... 15
2.5 Netcat .................................................................................................................... 15
2.6 MySQL .................................................................................................................. 16
2.7 Secure Sockets Layer ............................................................................................ 16
2.8 Advanced Encryption Standard.............................................................................. 17
3 Requirements and System Design ............................................................................. 18
3.1 Requirements ........................................................................................................ 18
3.2 System Architecture............................................................................................... 18
3.2.1 Local component .......................................................................................... 19
3.2.2 Remote component ...................................................................................... 19
3.3 Embedded System ................................................................................................ 19
3.3.1 Hardware ...................................................................................................... 19
3.3.2 Software ....................................................................................................... 21
3.4 Remote Server ...................................................................................................... 22
3.4.1 Hardware ...................................................................................................... 22
3.4.2 Software ....................................................................................................... 22
4 Implementation of Basic Functionalities...................................................................... 24
4.1 Data Collection ...................................................................................................... 24
4.2 Data Filtering ......................................................................................................... 25
4.2.1 Filtering by device ......................................................................................... 25
4.2.2 Filtering by type of message ......................................................................... 27
4.2.3 Filtering by message update interval ............................................................ 28
4.3 Data Storage ......................................................................................................... 28
4.4 Data Transmission ................................................................................................. 29
4.4.1 Real-time data transmission ......................................................................... 30
4.4.2 Historical data transmission .......................................................................... 30
4.5 Automatic Operation of Data Collection and Transmission .................................... 31
4.6 Data Browsing ....................................................................................................... 33
4.6.1 Data browsing on the embedded system ...................................................... 33
4.6.2 Data browsing on the remote server ............................................................. 33
4.6.3 Data browsing from external Internet connection .......................................... 35
5 Implementation of Advanced Features ....................................................................... 39
5.1 Data Security ......................................................................................................... 39
5.2 Transmission Security ........................................................................................... 40
5.2.1 Secure connection for MySQL ...................................................................... 40
5.2.2 Secure connection for FTP ........................................................................... 40
5.3 Local Access and Control by Using SSH ............................................................... 41
5.4 Reverse Control ..................................................................................................... 42
1.0 Page 5 of 64
Marine Data Collection and Transmission
Page 6 of 64 1.0
Marine Data Collection and Transmission
Figures
Figure 1 Application scenario of data collection and transmission system ...................12
Figure 2 Description of command parameters in Netcat ..............................................16
Figure 3 System architecture .......................................................................................19
Figure 4 BeagleBoard-xM and PandaBoard ES ...........................................................20
Figure 5 Types of RESTful Web service in Netbeans ..................................................22
Figure 6 Translate raw data to human-readable message ...........................................24
Figure 7 Priority of filtering criteria ...............................................................................25
Figure 8 Messages from specified device ....................................................................26
Figure 9 Messages from devices not in the blacklist ....................................................27
Figure 10 Messages with a specified PGN ....................................................................28
Figure 11 Table structure in local database ...................................................................28
Figure 12 Two modes for data transmission ..................................................................29
Figure 13 Configuration of startup applications ..............................................................32
Figure 14 Add startup program ......................................................................................32
Figure 15 List of startup programs .................................................................................32
Figure 16 Table structure on remote server ...................................................................34
Figure 17 Viewing data on remote server ......................................................................34
Figure 18 Viewing data in Netbeans ..............................................................................35
Figure 19 Functions of Web service...............................................................................36
Figure 20 Inquery for single message ............................................................................37
Figure 21 Inquery for multiple messages .......................................................................37
Figure 22 Message count ..............................................................................................38
Figure 23 AES encryption and decryption in MySQL .....................................................39
Figure 24 Encrypted data and transmission ...................................................................41
Figure 25 Flow chart of reverse control..........................................................................42
Figure 26 Netcat listening port .......................................................................................42
Figure 27 Reverse control on the embedded system .....................................................44
Figure 28 Control terminal on the remote server ............................................................44
Figure 29 Testbed .........................................................................................................46
Figure 30 Access modes ...............................................................................................47
Figure 31 Data received in the remote server ................................................................47
Figure 32 Connect the embedded system to mobile phone through Wi-Fi .....................48
Figure 33 Login screen ..................................................................................................49
Figure 34 Log in the embedded system .........................................................................49
Figure 35 Local access ..................................................................................................50
Figure 36 Real-life experiment .......................................................................................51
Figure 37 Real-time data in real-life experiment ............................................................51
Figure 38 PuTTY configuration ......................................................................................58
Figure 39 Execute the shell script ..................................................................................59
Figure 40 Set message update interval .........................................................................59
Figure 41 Select filtering mode ......................................................................................60
Figure 42 Specify device ID ...........................................................................................60
Figure 43 coordinator and wireless sensor ....................................................................61
1.0 Page 7 of 64
Marine Data Collection and Transmission
Tables
Table 1 Technical specifications of PandaBoard ES .................................................20
Table 2 Filtering criteria and application scenarios ...................................................25
Table 3 Rule 1: Specified device...............................................................................26
Table 4 Rule 2: Blacklist ...........................................................................................26
Table 5 Filtering by PGN ..........................................................................................27
Table 6 Filtering by update interval ...........................................................................28
Table 7 Internet-dependent functionalities under different access modes .................47
Table 8 Compatibility of operating system on the embedded system ........................53
Page 8 of 64 1.0
Marine Data Collection and Transmission
Abbreviations
AP Access Point
CA Certificate Authority
1.0 Page 9 of 64
Marine Data Collection and Transmission
1 Introduction
In Section 1.1, the motivation of this thesis is introduced. Section 1.2 illustrates the problems
need to solved while Section 1.3 describes our solution. At last, Section 1.4 gives the outline
of this thesis.
Now marine sensors are widely implemented on leisure boats to provide monitoring
information. However, as far as we know, there is not any data collection and management
system on the market. Thus, we aim at developing such a system for boat manufacturers
and owners.
These requirements are the functional demands of the system, which will be described in the
next chapter.
1.3 Approaches
In this thesis, we design a marine data collection and transmission system. It is based on an
ARM-based single-board computer, which is small in size but provides excellent performance.
Figure 1 illustrates the overall structure of the data collection and transmission system. The
system can be implemented on boats, being connected to CAN Bus. In the CAN Bus network,
there are NMEA 2000-compliant sensors generating monitoring data and broadcasting it
through the bus. The embedded system is designed to collect the data, and store it after
filtering. Furthermore, the data will be transmitted to the designated remote server through
Internet. It is worth mentioning that, the embedded system supports flexible connectivity, so
that we can choose wired or wireless connection according to circumstances.
1.0 Page 11 of 64
Marine Data Collection and Transmission
• To set up a server which receive the data from embedded system and publish the
data on the Internet.
• Chapter 3 describes the functional requirements of the system as well as the system
architecture.
• Chapter 4 and 5 present the basic functionalities and advanced features of the
system respectively.
• Discussions is in Chapter 7
Page 12 of 64 1.0
Marine Data Collection and Transmission
2 Technology Background
As mentioned in the previous chapter, this data collection system is designed for NMEA
2000-compliant devices in a CAN Bus network. In the following paragraphs, we will introduce
these two standards. Then we will give a description on the cellular network and Web service
technology, which enable the data transmission and data publishing.
The main advantage of a CAN Bus is that devices connected to it may exchange data in all
directions. And the transmission via CAN Bus is very quick and fulfill the requirements for
most devices. It is worth mentioning that the CAN Bus is a worldwide standard, which means
that all CAN Bus-compliant devices can exchange data regardless of manufacturer. The
CAN Bus system operates in a manner that all devices may listen to the messages
transmitted in CAN Bus. Devices only accept the messages which are needed, while discard
the others [2].
NMEA 2000, which is defined by National Marine Electronic Association, is a data network
for communications between marine electronic devices. It is based on CAN Bus which
connects devices together in a common channel. It means that different devices such as
temperature sensors, GPS and fuel monitor can exchange the data between each other. The
main goal of the standard is to share marine information in an easy way. We can say that the
NMEA 2000 is a language defined based on CAN Bus. In an NMEA 2000 network, the data
transmitted in CAN Bus should follow the frame structure defined in NMEA 2000 standard.
In the NMEA family, there are NMEA 0183 and NMEA 2000. NMEA 0183 standard is the
predecessor of the other and it is not based on CAN Bus. One advantage of the new
standard is that it has higher data rate, i.e., 256000 bps compared with 4800 bps in NMEA
0183. Another advantage is that more compact binary messages are used in NMEA 2000,
which makes it more efficient than NMEA 0183. In this project, all the devices we use comply
with NMEA 2000 standard.
Normally, wired connection can support higher data transmission rate than wireless
connection. Take optical network for example, the data rate in uplink can reach 2.4 Gbps and
1.2 Gbps in downlink according to the standard of GPON [3].
However, when sailing, it is not possible to obtain wired connection. Thus we need to adopt
wireless technology which is suitable for mobile devices.
1.0 Page 13 of 64
Marine Data Collection and Transmission
2.2.2 Wi-Fi
Nowadays Wi-Fi is widely used in our daily life. It enables electronic devices to exchange
data wirelessly at a high data rate. For example, according to IEEE 802.11n standard, the
maximum data rate can reach 600 Mbps in theory by increasing the transmission bandwidth
and adopting MIMO.
Although Wi-Fi is a good solution of wireless transmission, the problem lies in the limitation of
transmission range and interference due to license free. In the outdoor environment, the
transmission range is usually around 160 m. When sailing, it will be natural that boats sail out
of the range. And because the frequency band of Wi-Fi is free to use, so it may be shared by
many users and interference may occur.
Besides the Internet access, Wi-Fi can be used for local communication, e.g., establishing
connection between the embedded system and a mobile router or a laptop.
The transmission range in cellular networks is about 10 to 15 km. By deploying more base
stations, the coverage of cellular network is close to 100% coverage along Norwegian coasts,
i.e., nearly all the offshore areas where leisure boats usually sail are under the coverage of
cellular networks.
The data rate of cellular network is quite high. Take 3G networks for example, the theoretical
maximum data rate is 384 kbps while moving. And under practical situation, the data rate is
normally higher than 200 kbps. In an LTE network, the data rate is even much higher.
Since it is easy for us to get access to the Internet via cellular network and the data rate and
coverage meet the requirements of data transmission, the cellular network can be
considered as the most popular Internet access mode.
Page 14 of 64 1.0
Marine Data Collection and Transmission
In our case, we used Restful Web service [5] as application for collect and check real-time
data from embedded system.
• By standard URIs, resources are addressable and can be used as hypermedia links
• It is layered system
• Uniform interface
RESTful Web service allows that resources have different representations, such as JSON,
TXT and XML. The RESTful client can send request for specific representations via the
HTTP protocols.
However, these protocols basically meet an inevitable problem, which is the security risk.
When you transmit any important files through ftp, a potential intruder can intercept and
obtain the data. Moreover, if you use telnet to access another machine and remotely execute
an application, your username and password can be intercepted during the transmission.
To improve security, SSH, the secure shell was standarized by IETF, which is a popular,
software-based approach [6]. Whenever the data transport through the network, SSH
automatically encrypts it. After the data reached its destination, SSH automatically decrypts it.
Although it has encryption and decryption during the transmission, the users can work
normally and locally regardless of the process of transmission. In addition, SSH uses secure
and modern encryption algorithms to provide enough protection during transmission.
2.5 Netcat
Netcat is a network debugging tool, which helps in reading and writing data cross network
connections. In Netcat, it uses TCP/UDP for its functionality working across the networks. In
addition, it is configurable and can be driven by scripts. There are a lot of inbuilt commands
that can add different features to the utility.
1.0 Page 15 of 64
Marine Data Collection and Transmission
The following figure shows the description of various command parameters when Netcat is
working:
2.6 MySQL
MySQL is a relational database management system that can helps us to create a database
with tables, columns and indexes. It can be used to store, sort, manage and display data
content, which is reliable, fast, easy to use and suitable for application of any size.
• MySQL has fast working performance and also works well with large data sets.
• MySQL widely opens interface to PHP, which contributes a lot in Web service
development
Page 16 of 64 1.0
Marine Data Collection and Transmission
encrypted communication for Web traffic. It provides the security in terms of message
integrity, authentication and confidentiality [7].
• Confidentiality, by encrypting the data message, only application endpoints
understand the data
• Integrity, where the protocol will detects if any data was changed or loss during the
transmission
There are 6 different ways to use symmetric key in AES encryption method, which are
named Modes of Operation. The following list shows different modes of operation that can be
used in AES encryption [8].
• Counter (CTR)
• Counter (CTR)
In the next chapter, we will introduce the requirements to the data collection and
transmission system and the structure of the system.
1.0 Page 17 of 64
Marine Data Collection and Transmission
3.1 Requirements
As the main goal of this project is to develop a system used to collect, filter and store the
data of marine sensors as well as to transmit data to a remote server from which people can
obtain the data for analysis. The system should fulfill the following functional requirements.
Basic functionalities:
1. Data collection: to collect NMEA 2000 messages from CAN Bus network and
translated the binary raw message in to the format which is human readable.
3. Data storage: the filtered data should be properly stored and managed.
4. Data transmission: to transmit data from the embedded system to the server.
5. Data publishing: to publish the data on the Internet, users can access the appointed
web page for the data.
For data transmission, there are two different working modes: real-time data transmission
and historical data transmission. The former occurs immediately after the data collection and
filtering while the latter is performed after the startup of Ubuntu.
Advanced features:
3. Local control: to access and control the embedded system by using SSH.
4. Reverse control: to enable the remote server to obtain the control on the embedded
system.
Basic functionalities are the needs for implementing basic operation of the system, and the
advanced features are the improvements on security, local and remote control. The
requirements on them are the main goals of our design, which will be introduced in detail in
the next two chapters. In the following section, we will present the architecture of the system.
Page 18 of 64 1.0
Marine Data Collection and Transmission
3.3.1 Hardware
In the initial stage, two development boards are tested as the hardware platform of our data
collection system as shown in Figure 4. One is BeagleBoard-xM and the other is
PandaBoard ES. Both of them have good enough performance for running the Linux-based
system.
1.0 Page 19 of 64
Marine Data Collection and Transmission
adapter. We check this problem on Internet and find that it is very common among Ubuntu
users. It also happens to other wireless USB adapters.
After reinstalling the drivers and manually configuring the wireless network, the problem
remains. Since the stability of wireless network is important for data transmission, we have to
abandon it and look for a new platform.
With the help of on-board wireless adapter, the wireless connection between PandaBoard
ES and the wireless access point is stable. Even when disconnecting from the network, the
on-board wireless adapter can automatically re-establish the wireless connection if the
access point remains.
Due to the high performance, small size and low power consumption, the PandaBoard ES is
adopted as the hardware platform for the embedded system.
Page 20 of 64 1.0
Marine Data Collection and Transmission
3.3.2 Software
The software to be installed on PandaBoard ES includes the operating system, database,
FTP client and the data collection software developed by us.
Operating system
We choose Ubuntu as our operating system as it is a popular Linux-based system. After
installing different version on PandaBoard ES and BeagleBoard-xM, we find that only Ubuntu
12.04 LTS (desktop version for OMAP board) is available on them. The image of install CD
can be found on the official website of Ubuntu [9]. There are two versions for OMAP3 and
OMAP4 respectively. According to the model of processor, we choose OMAP4 version for
PandaBoard ES.
Database
We adopt MySQL (version 5.5.22 for Ubuntu) to store and manage the data on the
embedded system. Hereinafter, the database installed on the PandaBoard ES is called local
database while the one on the remote server is called remote database.
In Ubuntu system, MySQL can be downloaded in the Software Center. After installing it on
PandaBoard ES, we need to log in the database and create a device list table by typing the
following command before perform the data storage.
CREATE TABLE device(id INT NOT NULL AUTO_INCREMENT UNIQUE KEY, deviceid INT UNSIGNED);
The data collection system will record the IDs assigned to the sensors in the NMEA 2000
network in this table. According to this table, the data collection system can detect new
devices in the network and create new tables for storing the data generated by them.
FTP client
We use FTP tool to transfer the data stored on the PandaBoard ES to the remote server.
Since the data contains no real-time information, it is hereinafter called historical data.
In this project, we use LFTP for historical data transmission. Because it is a command-line
FTP client, by shell scripting, we can have automatic data uploading which will be introduced
in Section 4.5.
All the message formats are defined in NMEA 2000 Appendix B – PGN (NMEA Network
Messages) Database [11]. We have purchased NMEA 2000 Appendix B in order to use it for
data translation.
We also add new features to analyzer. One is data filtering, which helps users to concentrate
on the data of interest and reduce the traffic of unnecessary data transmission. Furthermore,
by invoking the MySQL API for C, the filtered data will be store in the local database.
Meanwhile, if the embedded system and remote server are both online, the data will also be
transmitted to the remote server and stored in server’s database. Since the data is
transmitted right after being received, it is hereinafter called real-time data.
1.0 Page 21 of 64
Marine Data Collection and Transmission
All functionalities of the software will be described in details in Chapter 4 as well as the
usages.
PHP
Since we need to run a PHP script to establish the connection with the remote server for
reverse control, we have to install the PHP developing environment beforehand by using the
command
sudo apt-get install php5 libapache2-mod-php5 php5-cli php-pear php5-mysql php5-pgsql
3.4.1 Hardware
In thesis, we use our own laptop as a server. The laptop is powered by an Intel Core i5
processor, and with4GB 1333 MHz RAM and 500GB storage, which support all the software
we need and work smoothly.
3.4.2 Software
On the remote server, we need to install the following software and applications.
Operating system
The default operating system on the laptop is Windows 7 Professional (64-bit).
From Java EE 6 and Java EE 5, Web service standards are supported by Netbeans. We use
GlassFish open source edition as our server to make service access available from external
Internet. Since RESTful service is stateless for server and have a simple architecture
between client and server, we decide to use it as our Web service style. Moreover, Netbeans
IDE assists us to create RESTful Web services directly from MySQL, thus this functionality
helps us to wrap entity beans and provide easy CRUD (create, read, update and delete)
operation. Figure 5 shows the different architecture types for RESTful Web service in
Netbeans.
Page 22 of 64 1.0
Marine Data Collection and Transmission
Database
Similar to the embedded system, MySQL (version 5.5.31 for Windows 7) is installed on the
remote server for receiving and storing the data transmitted from the embedded system.
Since the default setting of access control in MySQL only allow the access from local IP
address 127.0.0.1, we need to make the database can be accessed from any IP address. By
using the following commands after log in MySQL:
using mysql;
update user set host = ‘%’ where user = ‘root’ and host = ‘localhost’
flush privileges;
FTP server
For historical data transmission, we need to install a FTP server on the remote server. In our
case, Serv-U (version 14.0.0.6) is adopted for providing FTP service.
In this chapter, we have presented the requirements and architecture of the system. And the
requirements are divided into two parts: basic functionalities and advanced features, which
will be introduced in Chapter 4 and 5 respectively.
1.0 Page 23 of 64
Marine Data Collection and Transmission
3. Data storage: the filtered data should be properly stored and managed.
4. Data transmission: to transmit data from the embedded system to the server. It
consists of two parts: real-time data transmission and historical data transmission.
5. Data publishing: to publish the data on the Internet, users can access the appointed
web page for the data.
With these functionalities, the embedded system and the server constitute an integral system
which provides data collection and transmission service.
As shown in Figure 6, the raw data mainly consist of a group of 8-bit binary data which can
be translated according to the definitions in NMEA 2000 Appendix B [11].
Page 24 of 64 1.0
Marine Data Collection and Transmission
We develop the system in the way that data can be filtered according to different criteria set
by users. In Table 2, we can see that there are criteria apply to different scenarios.
Table 2 Filtering criteria and application scenarios
The filtering criteria can be adopted individually or concurrently. If multiple criteria are
designated simultaneously, they will be performed according to the priority illustrated in
Figure 7.
1.0 Page 25 of 64
Marine Data Collection and Transmission
from different devices can be distinguished according to the transmitters’ IDs. We define two
different filtering rules for filtering by device.
Specified device
Rule 1 is used to retrieve data only from the specified device. Refer to Table 3 for the usage.
Table 3 Rule 1: Specified device
For instance, this rule applies to the scenario that only GPS information are needed, and the
device ID of GPS receiver is known (ID is 32), the following steps should be performed:
Step 1: Open the terminal and go to the directory of actisense-serial and analyzer.
Blacklist
Rule 2 is used to discard the data from the device(s) listed in the blacklist, the remaining of
the data will be captured and displayed.
Table 4 Rule 2: Blacklist
Purpose To extract the messages from devices which are not in the blacklist
actisense-serial /dev/ttyUSB0 | analyzer –nsrc <Device ID1>< Device ID2
Usage
(optional)>…
Page 26 of 64 1.0
Marine Data Collection and Transmission
Take our testbed for example, there are four devices connected by a CAN bus: a DST110
Triducer (ID is 112), a GPS receiver (ID is 32), a TLM100 tank level monitor (ID is 35) and an
Actisense NGT-1 USB Gateway (ID is 0). In this NMEA 2000 network, if users are only
interested in the messages from the tank level monitor and Actisense NGT-1 USB Gateway.
Then the following command should be used for collecting data only from these two devices.
./actisense-serial -r /dev/ttyUSB0 | ./analyzer -nsrc 32 112
By listing all the unwanted devices in the blacklist, the messages sent by the GPS receiver
and DST110 Triducer will be discarded. On contrary, the messages from the tank level
monitor and NGT-1 Gateway, which are not in the list, are retrieved and displayed on the
screen as shown in Figure 9.
1.0 Page 27 of 64
Marine Data Collection and Transmission
The message update interval can be set in the range from 0 to 59 (in seconds). Set 0 for
storing and displaying all the messages. If another value n is set, message will be updated
only once during each update interval (which is equal or larger than n seconds). For example,
as shown in Figure 10, the messages updates every 5 seconds. If we set the interval to 10
second, within the same duration, the output data will be half-sized.
Page 28 of 64 1.0
Marine Data Collection and Transmission
All the tables for storing data are named by the ID assigned to devices in NMEA 2000
network, e.g., the data generated by GPS receiver (ID is 32) will be put into the table src32
while the information of engine (ID is 17) will be stored in table src17. These tables have an
identical structure which is shown in Figure 11.
The definitions of the parameters in table are as follows:
• dst: ID of the device to which the message is sent. 255 is the broadcast ID;
Except the message serial number, all the other parameters are intercepted from messages.
It is useful for the user who searches for specified messages.
1.0 Page 29 of 64
Marine Data Collection and Transmission
As shown in Figure 12, the collected data will be immediately transmitted to the remote
server and simultaneously stored in the local database on the embedded system. The data in
local database is called historical data and the one transmitted in real time is called real-time
data.
1 MYSQL re_connection;
2 int res;
3 mysql_init(&re_connection);
4 if (mysql_real_connect(&re_connection, <IP of remote database>, <username>, <password>,
<name of database>, 0, NULL, CLIENT_FOUND_ROWS))
5{
6 char sql_insert_re[8400];
7 sprintf(sql_insert_re, "INSERT INTO b_001 values(NULL, '%s', '%s', '%s', %u, %u, %u, %u,
'%s');",<boatname>, strdate, strtime, prio_sql, src_sql, dst_sql, pgn_sql, mbuf);
8 res = mysql_query(&re_connection, sql_insert_re);
9 if (res)
10 {
11 printf("Fail to write to REMOTE database\n");
12 }
13 }
14 else
15 {
16 printf("Fail to connect to REMOTE database\n");
17 }
18 mysql_close(&re_connection);
Each filtered message will be passed to the function with above codes for real-time
transmission. The first line of the codes defines a MySQL connection, and then it is initialized
in Line 3. In line 4, function mysql_real_connect() is used to establish the connect to the
remote database with correct IP of remote database, username, password and database
name. The 7th line is for insert the message to the remote database according the data
structure. After transmitting the message, use mysql_close() to close the connection. The
process will repeat when another message is passed to the function.
If no available connection between the embedded system and remote server, after a few
seconds, the operation will be time-out and return an error message -- Fail to connect to
REMOTE database. However, the message will still be stored into the local database and
then the system will begin to process the next message.
Page 30 of 64 1.0
Marine Data Collection and Transmission
information, it includes complete information on the running state of boats which holds value
to boat manufacturers and owners for data analysis.
In order to simplify the operation of FTP uploading, we develop the following shell script
upload.sh.
1 #!bin/bash
2 mysqldump –u<MySQL username> -p<MySQL password> b_001 > <directory of exported
data>/b_001-$(date +%y%m%d).sql
3 lftp -e "set net:timeout 10;set net:max-retries 3" <<EOF
4 open <IP of FTP server>:<port no>
5 user <FTP username> <FTP password>
6 put <directory of exported data>/b_001-$(date +%y%m%d).sql
7 bye
8 EOF
9 rm <directory of exported data>/b_001-$(date +%y%m%d).sql
10 rm <directory of exported data>/b_001-$(date +%y%m%d -d "1 day ago").sql
The first line defines the script compiler. The second one is used to export the data from
MySQL and save it as an sql file under the appointed directory. Line 3 is used to set the
timeout and retry limit for connecting to FTP server in avoiding infinity reconnection. Line 4, 5
and 6 are for logging in FTP server login and transferring the file of exported data. Finally,
the last two lines are used for deleting the files created on the day and the day before.
With the shell script, we still need to type the following command manually in the terminal to
start the operation.
cd <directory of upload.sh>
sh upload.sh
In order to make the embedded system self-running, we find a solution which will be
described in the next section.
1 #!bin/bash
2 cd <directory of actisense-serial and analyzer>
3 ./actisense-serial /dev/ttyUSB0 | ./analyzer -itv 30
Step 2: Click the system button on the top right corner and select Startup Applications
as shown in Figure 13.
1.0 Page 31 of 64
Marine Data Collection and Transmission
Step 3: Click Add button, then input the name and command for adding data.sh.
Page 32 of 64 1.0
Marine Data Collection and Transmission
Now, after system startup, the data collection and real-time transmission will be started and
keep running in background. Meanwhile, historical data will be exported from database and
uploaded to the FTP server. The process will be terminated when either the exported data is
successfully transmitted or connecting requests exceed the limit.
Step 3: There could be multiple tables in the selected database. Show all tables by
typing:
show tables;
By using the following command, we can see table structure in Figure 16.
describe b_001;
1.0 Page 33 of 64
Marine Data Collection and Transmission
As shown in Figure 18, by opening the Netbeans and choosing Database in the left bar, to
launch java database driver ecoboat and choose the Tables under the ecoboat directory.
There are 2 tables in the list: b_001 and src32, right click on b_001 and view data, we can
observe that all the data that will be display in the IDE.
Page 34 of 64 1.0
Marine Data Collection and Transmission
In order to publish the Web service, we install the Netbeans 7.3 IDE and make several trials
under Java developing environment. Here are our steps to publish the RESTful Web service:
Firstly, we create a MySQL table class, which defines the different sensors’ parameters such
as id, boatname, date, pgn and src.
Then we can invoke these parameters to see the data by the command NamedQueries. For
example, @NamedQuery(name = "B001.findById", query = "SELECT b FROM B001 b
WHERE b.id = :id").
Next, we compile table b_001 into RESTful Web service class. By using 4 operations (GET,
POST, PUT, DELETE), we can easily invoke the specific parameter and view its content.
Here we choose three main episodes in my codes to explain how the RESTful service works
with MySQL database.
@Stateless
@Path("boattest.b001")
public class B001FacadeREST extends AbstractFacade<B001> {
@PersistenceContext(unitName = "boattestPU")
private EntityManager em;
public B001FacadeREST() {
super(B001.class);
}
This segment of codes illustrates that class B001FacadeREST is defined to have the same
method as table class B001 and makes the class B001FacadeREST to be stateless.
1.0 Page 35 of 64
Marine Data Collection and Transmission
@POST
@Override
@Consumes({"application/xml", "application/json"})
public void create(B001 entity) {
super.create(entity);
}
The segment of codes above shows that a selection function is added into class, which we
can decide the type of data content in RESTful Web service. In our case, there are two types
of format: xml and json.
@GET
@Path("count")
@Produces("text/plain")
public String countREST() {
return String.valueOf(super.count());
}
The last segment of codes is invoking the GET operation to count the total number of data in
our database.
After the RESTful Web service class setup, we need to design our Web service interface. It
is a web-page that the clients can access from external Internet and choose which data they
want to view. Because of the time limitation, we used the Netbeans default RESTful web-
page in our thesis. As shown in Figure 19, it has 3 functions: select a single id to view, select
a range of id to view and count the total number of data.
Once we want to check a specific data, for example the data content of id 38. We can use
the {id} function: to type 38 in id blank, then click Test. It shows entire information of id 38, as
shown in Figure 20.
Page 36 of 64 1.0
Marine Data Collection and Transmission
1.0 Page 37 of 64
Marine Data Collection and Transmission
Page 38 of 64 1.0
Marine Data Collection and Transmission
3. Local control: to access and control the embedded system by using SSH.
4. Reverse control: to enable the remote server to control the embedded system.
Function 1 is implemented by using the AES encryption and decryption provided by MySQL.
Using the SSL encryption on FTP and MySQL, we can implement transmission security.
Local control is based on SSH while reverse control is implemented by using port listening.
MySQL supports data encryption and decrytion based on AES, which is a standard for the
encryption of electronic data. AES is a reliable solotion for data security and has been widely
adopted.
1.0 Page 39 of 64
Marine Data Collection and Transmission
As shown in Figure 23, when storing the data into MySQL, we can use aes_encrypt() to
encrypt the message string (AES_ENCRYPT_TEST) with a key string (KEY). Then the
unreadable encrypted data will be stored in the table. If anyone wants to decrypt the data, he
must have the key.
By implementing AES encryption, the data stored in MySQL is secured as only the key
holders can decrypt it.
As we have two different connection modes, the introductions will be given respectively.
Step 2: Move CA certificate, server certificate and server key files to the server side.
Step 3: Move CA certificate, client certificate and client key files to the client side.
Step 5: Start server and client with their SSL files respectively.
For more details, please refer to the reference manual on the official website for MySQL [13].
After finishing the above operations, we also need to modify the source code in analyzer.c.
Before calling mysql_real_connect(), use the mysql_ssl_set() function to specify the path to
the certificate and key files for the client side.
1 MYSQL re_connection;
2 int res;
3 mysql_init(&re_connection);
4 mysql_ssl_set(&re_connection, <path name to the key file of client>, <path name to the
certificate file of client>, <path name to the CA file>, NULL, NULL);
5 if (mysql_real_connect(&re_connection, <IP of remote database>, <username>, <password>,
<name of database>, 0, NULL, CLIENT_FOUND_ROWS))
…
Page 40 of 64 1.0
Marine Data Collection and Transmission
Step 2: Move CA certificate, server certificate and server key files to the server side.
Step 3: Move CA certificate, client certificate and client key files to the client side.
Step 4: Configure Serv-U to enable SSL connection. Refer to the website of Serv-U for
details [14].
Step 5: Configure LFTP to enable SSL connection by typing the following commands in
terminal.
lftp
set ftp:ssl-force true
set ftp:ssl-protect-data true
set ssl:verify-certificate no
set ssl:ca-file < path name to the CA file>
set ssl:key-file <path name to the client key file>
set ssl:cert-file <path name to the client certificate file>
By now, all the transmission between the embedded system and remote server has been
secured by SSL encryption.
As illustrated in Figure 24, with the encryption on both data and transmission, the data is
always under the protection. Only the key holders, which could be either human or
applications) can decrypt the data.
1.0 Page 41 of 64
Marine Data Collection and Transmission
As mentioned in Chapter 3, the reverse control enables the administrative users who are
close to the server to remotely control the embedded system even it does not have the public
IP address. Figure 25 illustrates the basic procedures of reverse control: firstly, the remote
server opens a specific port as scanner and listener. Once the embedded system actively
launches a connection to the server, the in-coming connection will be detected by the server.
A notice of successful connection will pop out in the command line in which we can send
commands to control the embedded system.
In PHP scripts, we firstly defined the server’s public IP address, port number, chunk size and
daemon process as following code episode:
Page 42 of 64 1.0
Marine Data Collection and Transmission
set_time_limit (0);
$VERSION = "1.0";
$ip = '128.39.202.145';
$port = 4431;
$chunk_size = 1400;
$write_a = null;
$error_a = null;
$shell = 'uname -a; w; id; /bin/sh -i';
$daemon = 0;
$debug = 0;
Then we open the reverse control and spawn the shell process, which is achieved by
episodes:
$sock = fsockopen($ip, $port, $errno, $errstr, 30);
if (!$sock) {
printit("$errstr ($errno)");
exit(1);
}
// Spawn shell process
$descriptorspec = array(
0 => array("pipe", "r"), // stdin is a pipe that the child will read from
1 => array("pipe", "w"), // stdout is a pipe that the child will write to
2 => array("pipe", "w") // stderr is a pipe that the child will write to
);
$process = proc_open($shell, $descriptorspec, $pipes);
if (!is_resource($process)) {
printit("ERROR: Can't spawn shell");
exit(1);
}
If nothing blocks the above process, we will have a prompt on the client side to inform users
whether this connection is success or not. It is achieved by following codes:
stream_set_blocking($pipes[0], 0);
stream_set_blocking($pipes[1], 0);
stream_set_blocking($pipes[2], 0);
stream_set_blocking($sock, 0);
printit("Successfully opened reverse control to $ip:$port");
while (1) {
// Check for end of TCP connection
if (feof($sock)) {
printit("ERROR: Connection terminated");
break;
}
// Check for end of STDOUT
if (feof($pipes[1])) {
printit("ERROR: Process terminated");
break;
}
1.0 Page 43 of 64
Marine Data Collection and Transmission
Once the PHP script is running on the embedded system, the prompt of successfully
connection will automatically pop out, which is shown in Figure 27. Then Figure 28 shows
that we can enter any Linux commands on the server side as we do in the Ubuntu terminal.
Page 44 of 64 1.0
Marine Data Collection and Transmission
We have tested this function in three ways of Internet connection of embedded system: cable
connection, Wi-Fi connection and 3G/4G Router respectively, more details will be given in
Chapter 6. It works well in all scenarios, which the connection is stable and the control
performance is effective.
1.0 Page 45 of 64
Marine Data Collection and Transmission
In the first three cases described in the following sections, the tests are concentrated on the
critical functionalities and feature. The fourth one is an on-site test on the boat provided by
Marex AS, the overall performance of the system is validated in practical use.
Figure 29 Testbed
The testbed is used in the following experiments.
• Experiment on access modes
• Experiment on Boat
In the following sections, the test scenarios will be introduced in detail.
Page 46 of 64 1.0
Marine Data Collection and Transmission
The real-time data is received in the remote server as well as the historical data in the file
b_001-130515.sql as shown in Figure 31.
1.0 Page 47 of 64
Marine Data Collection and Transmission
The results indicate that available connections to the Internet are established and the
Internet-dependent features function correctly under all the three access modes.
Step 3: Start the embedded system with connection to the cellular network by using a
mobile router.
Step 4: Connect the embedded system to the NMEA 2000 network and then launch the
data collection system.
After repeating the above steps for 20 times, it is observed that the filtered data on the
embedded system can be transmitted to the remote server through cellular network.
Step 2: Connect the embedded system to the access point through Wi-Fi and note
down the IP address.
Page 48 of 64 1.0
Marine Data Collection and Transmission
Step 3: Start a SSH client application on the phone (In this example, we use VX
ConnectBot for Android which is free and can be found in the application market)
and log in by typing:
Step 4: After inputting the password, the login information will be displayed on the
phone.
1.0 Page 49 of 64
Marine Data Collection and Transmission
Now we have obtained the control on the embedded system and can access the data in the
local database or perform FTP uploading. Through SSH, all operations can be performed by
inputting corresponding commands on the phone in the same way as they are done on the
PandaBoard ES.
The remote server is placed in the campus of University of Agder in Grimstad while the boat
sails from Fevik to Arendal and go back to Fevik.
The embedded system is connected to a NMEA 2000 network through the NGT-1 gateway
and Telenor’s cellular network via the 3G/4G router. And in the NMEA 2000 network, there
are other two devices: a GPS receiver and an engine.
During the voyage, we can see the collected data on the screen as illustrated in Figure 36.
The data are generated by either the GPS receiver or the engine. Meanwhile, we contact our
partner who is monitoring the remote server in the campus by mobile phone for verifying
whether the data transmission is successfully performed.
Page 50 of 64 1.0
Marine Data Collection and Transmission
1.0 Page 51 of 64
Marine Data Collection and Transmission
From the figure above, we can see the state of the engine (ID is 17), GPS coordinates as
well as other information. For more information collected in this experiment, please refer to
Appendix E.
The system performance shown in this experiment is inspiring as it gives evidence that this
marine data collection system has the potential to be put into commercial use.
To summarize, the results of the tests described in this chapter reflect that the system has
good availability and stability, which strengthen our confidence to make it a commercial
product.
Page 52 of 64 1.0
Marine Data Collection and Transmission
7 Discussions
There are several topics in which we are interested. During the system development, we
come across difficulties which even lead to changes in design. After repeated attempt and
modification, the difficulties are solved and the design goal is fulfilled. In addition, the
experience we have gained may be helpful to improve our work in the future.
1.0 Page 53 of 64
Marine Data Collection and Transmission
Page 54 of 64 1.0
Marine Data Collection and Transmission
8.1 Contributions
Our main contributions in this thesis include:
• Design and develop the system architecture
Furthermore, we can invoke Google Map API in the remote server side, so that boats can be
located with knowing the real-time GPS coordinates. Or with a set of historical GPS data, we
can even illustrate the sailing route on the map and estimate the current location.
In addition, we may adopt satellite communication as a new Internet access mode in the
future so that the transmission range will be greatly extended.
1.0 Page 55 of 64
Marine Data Collection and Transmission
References
[1] NMEA 2000 and CANbus, http://www.whichmarineelectronics.com/?p=5, [access on Feb.
14, 2013]
[6] Iyappan, P.; Arvind, K. S.; Geetha, N.; Vanitha, S., "Pluggable Encryption Algorithm In
Secure Shell(SSH) Protocol,"Emerging Trends in Engineering and Technology (ICETET),
2009 2nd International Conference on, vol., no., pp.808,813, 16-18 Dec. 2009
[7] Suresh, V. M.; Karthikeswaran, D.; Sudha, V. M.; Chandraseker, D.M., "Web server load
balancing using SSL back-end forwarding method," Advances in Engineering, Science and
Management (ICAESM), 2012 International Conference on , vol., no., pp.822,827, 30-31
March 2012
[11] NMEA 2000 Appendix B – PGNs (NMEA Network Messages) Full Database,
http://www.nmea.org/store/index.asp?show=pdet&pid=325&cid=7, [access on January 23,
2013]
[12] Frank Cassidy, NMEA 2000 Explained - The Latest Word, March 2, 1999
[15] Future Technology Devices International Ltd., Technical Note TN_101 Implementing
Custom FTDI VID and PID Codes using Linux, Oct. 30, 2008.
Page 56 of 64 1.0
Marine Data Collection and Transmission
The guide for installing the NGT-1 Gateway on a Linux embedded system is shown as
follows.
Step 1 Connect Actisense NGT-1 to the embedded system through USB interface. At
this stage, the virtual COM (ttyUSB) for NGT-1 has not yet been created.
Step 2 Add product information to ftdi_sio by typing the following command in Ubuntu
terminal:
Step 3 Add ftdi_sio to the system modules for the change to take effect.
Step 5 The virtual COM port (/dev/ttyUSB0) can be found under the directory /dev by
using the command below:
ls -al dev/ttyUSB*
Step 6 If you do not log in as a root user, please remember to add your username to
the dialout group, so that all these changes will take effective in your account.
BRLTTY is a Linux/Unix-console access application for blind users. Please uninstall it first if it
is installed on the system, or else the virtual COM port will not be accessible [15].
1.0 Page 57 of 64
Marine Data Collection and Transmission
Step 2: Once it successfully connects to the embedded system, a control terminal will
pop up, in which we need to enter the password of remote embedded system.
Step 3: After the password is verified, we can type any command to control the remote
embedded system.
However, in practical scenario, the cellular network is often selected as the access mode
when sailing at sea. In cellular networks, the IP address of the embedded system is usually a
dynamic address assigned by the operator. It is difficulty for the remote server side to obtain
the address. In addition, operators of cellular networks may restrict the SSH access in terms
of security consideration. Consequently, SSH is not a proper approach for our case.
Page 58 of 64 1.0
Marine Data Collection and Transmission
1.0 Page 59 of 64
Marine Data Collection and Transmission
Step 3: Enter the device ID of the device, in which the user is interested.
This script can also be used in the scenarios of local control by using SSH.
Page 60 of 64 1.0
Marine Data Collection and Transmission
The WSN system developed by another two master students is used for collecting data from
wireless sensors installed on boats. A coordinator needs to be connected to the embedded
system in order to receive the data from the sensors through wireless transmission.
The GUI for the remote server is an enhancement to the remote server. It provides more
user-friendly interface and better performance on data search.
1.0 Page 61 of 64
Marine Data Collection and Transmission
The following is part of the collected data stored in the remote server.
Page 62 of 64 1.0
Marine Data Collection and Transmission
1.0 Page 63 of 64
Marine Data Collection and Transmission
Page 64 of 64 1.0