21018

Examensarbete 15 hp
Juni 2021

Anomaly Detection on Satellite

Time-series

Lovisa Ekeroot
Moa Tennberg
 Abstract
Anomaly Detection on Satellite Times-series

Lovisa Ekeroot, Moa Tennberg

Teknisk- naturvetenskaplig fakultet

UTH-enheten In this thesis, anomalies are defined as data points whose value differs
significantly from the normal pattern of the data set. Anomalous
Besöksadress: observations on time series measured on satellites has a growing need of
Ångströmlaboratoriet
Lägerhyddsvägen 1 being detected directly on board the space-orbit systems to for example
Hus 4, Plan 0 prevent malfunction and have efficient data management. Unibap's service
Spacecloud Framework (SCFW) is developed to allow the deployment of
Postadress: machine learning applications directly on the satellite systems. Neural
Box 536
751 21 Uppsala Networks (NNs) is therefore a candidate for the possibility to predict
anomalies on satellite time series. The work described in this report
Telefon: aims to implement and create a benchmark for Convolutional Autoencoder
018 – 471 30 03 NN (CNN) and a Long Short-term Memory Autoencoder NN (LSTM). These
Telefax: implementations are used to determine which NN can be applied in
018 – 471 30 00 Unibap's SCFW and detect anomalies with accuracy.
Hemsida: The NNs are trained and tested using a public data-sets which contain
http://www.teknat.uu.se/student
real and artificial time-series with labelled anomalies. The anomalies
are detected by reconstructing the time series and creating a threshold
between the output and the input. The algorithms classify a data point
as an anomaly if it lies above the threshold. The networks are evaluated
based on accuracy, execution time and size, to assess whether they are
suited for implementation in SCFW. The results from the NNs indicate
that CNN is best suited for further application. On this basis, an
attempt to implement CNN in SCFW is performed, but failed due to time
and documentation limitations. Therefore, further research is needed to
identify whether CNN can be implemented in SCFW and successfully detect
anomalies.

Handledare: Elisabeth Schöld Linnér

Ämnesgranskare: Maria Strömme
Examinator: Martin Sjödin
ISSN: 1401-5757, 21018
Preamble
The authors of this thesis wants to mention and appreciate some people that have made
this project possible to execute. Without them, the students would not have been able
to complete the project with the results presented in this thesis. Therefore the authors
wants to thank:

• Elisabeth Schold Linnér

• Unibap AB (publ)

• Oliver Petri

• Gabriel Fornaeus

• Samuel Eliasson Godonou

ii
Acronyms
CNN Convolutional Neural Network
COTS Commercial-Off-The-Shelf
CPU Central Processing Unit
FGPA Field Programmable Gate Array
FN False Negative
FP False Positive
GPU Graphical Processing Unit
IDE Integrated Development Environment
LSTM Long Short-term Memory
NN Neural Network
NAB Numenta Anomaly Benchmark
OBC On-Board Computer
RNN Recurrent Neural Network
SCFW SpaceCloud Framework
SGS Satellite Ground Station
SoC System on Chip
SWaP Size, Weight and Power
TN True Negative
TP True Positive

iii
Contents
1 Introduction 1
1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Problem description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.3 Project objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.4 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Theory 3
2.1 Time series analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2 Satellites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2.1 General Composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2.2 Communication with Earth . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2.3 On-Board Computer . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 Environmental Effects on the Satellite Structure . . . . . . . . . . . . . . . . . 8
2.4 Algorithms for Anomaly Detection . . . . . . . . . . . . . . . . . . . . . . . . 9
2.4.1 Choice of Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.4.2 Convolutional Neural Network (CNN) . . . . . . . . . . . . . . . . . . 10
2.4.3 Long Short-term Memory (LSTM) Neural Network . . . . . . . . . . . 10
2.4.4 Autoencoders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.5 Unibap’s SpaceCloud Framework . . . . . . . . . . . . . . . . . . . . . . . . . 12

3 Methodology 13
3.1 General procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.2 Considered Time Series with Anomalies . . . . . . . . . . . . . . . . . . . . . . 13
3.2.1 Desired Data of Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.2.2 Data of Machine Temperature . . . . . . . . . . . . . . . . . . . . . . . 14
3.2.3 Data of CPU Utilisation . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.3 Data preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.4 Experiment setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.4.1 Software used for Implementation . . . . . . . . . . . . . . . . . . . . . 18
3.4.2 Public Implementations . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.4.3 Choice of Evaluation Indices . . . . . . . . . . . . . . . . . . . . . . . . 19

4 Results 21
4.1 Evaluation of CNN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4.1.1 Machine Temperature . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4.1.2 CPU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.2 Evaluation of LSTM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.2.1 Machine Temperature . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.2.2 CPU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.3 Implementation for SCFW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

5 Discussion 27
5.1 Comparison of Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
5.2 Time-depending Impacts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
5.3 Future Developments and Improvements . . . . . . . . . . . . . . . . . . . . . 28
5.3.1 Anomaly detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
5.3.2 Evaluation of SCFW . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

6 Conclusion 30

iv
1 Introduction
1.1 Background
When data is collected over time, some data points will deviate significantly from the
majority of the data. These data points are henceforth called anomalies. Anomalies exists
in all forms of data, but the time series with the collected data has to be analysed for
the anomalies to be discovered. A more detailed description of time series and anomalies
are presented in Section 2.1 in this thesis.

In the space industry, which provides the world with for example weather forecasts and
documentation of the globe based on satellite observations, data is often collected in the
form of time series. The collected time series are often large, with an extreme amount
of data points, which can make the anomaly detection time consuming and computer
capacity demanding due to its volume, more in Section 2.2.2. Since the conditions in space
differs from the environmental conditions on Earth, such as temperature and pressure,
satellite hardware needs to be able to manage these changes without jeopardising its
performance and reliability. Section 2.3 presents a more detailed description of the
environmental effects of space.

1.2 Problem description

Transmitting large time series collected by satellites to Earth, for processing and decision
making, is sometimes a waste of capacities, both in time and computational resources. By
implementing algorithms on Unibap AB’s service SpaceCloud Framework (SCFW) with
the limitations of satellite hardware in mind, anomaly detection in time series directly
on the satellite’s hardware is going to be investigated. This will provide Unibap with a
foundation for future development within this kind of on-board data processing.

1.3 Project objective

This project aims to have an algorithm working in Unibap’s SpaceCloud Framework,
that is able to detect at least one correct anomaly from a public data set of time series
with annotated anomalies. To do so, the following question of issues are to be considered:

• How do satellites operate?

• What are the limitations of satellite hardware that need to be considered?

• What kind of data is going to be considered from the satellites?

• Are there any public implementations of algorithms that can be applied to this
problem? Can they be modified to work in SpaceCloud Framework?

1
1.4 Delimitations
In this thesis, delimitations are being made to be able to execute the project during the
limited time span. The following delimitations are the foundation for the rest of the
project:

• All satellites mentioned in this report are artificial satellites (spacecraft), and not
natural satellites (for example the Moon).

• The description of the environment of space, together with a satellite’s functionality

and composition, are in this thesis mainly presented on a general level. This is due
to that the data from the time series should be independent from which kind of
satellite that has collected it, and that the environment of a specific orbit in space
are the same for all satellites in that orbit.

• Since the report is written together with Unibap, their hardware/software are
presented as an example, if available. This gives the report a intuitive composition,
since the project aims to use Unibap’s SpaceCloud Framework in the final results.

• The machine learning algorithms are implemented using TensorFlow, due to its
TensorFlow Lite library, which can compress the model and make it more efficient.
This limits the choice of model, and can result in different results compared to
using another framework.

• The algorithms are implemented and trained on the students computers. If there is
time and availability, the students have the opportunity to implement the algorithms
on Unibap’s iX5 computer, using the service SpaceCloud Framework, to see if the
algorithms can function with Unibap’s satellite hardware.

• Due to lack of public data set with annotated anomalies, the available data used
in this thesis is limited.

2
2 Theory
2.1 Time series analysis
In time series, where one collects data for the same subject over time, the data will
provide a standard and deviations within the resulting data set. The deviations that
differs significantly from the majority are in this project called anomalies [1]. An example
of a marked anomaly in a time series is presented in Figure 1.

Figure 1: A time series with a marked anomaly. Credits: Khanin Artur [1].

The anomalies can differ more or less from the standard, and therefore be more or less
extreme, depending on the subject investigated. For example, during investigation of the
temperature of a surface on Earth covered by forest, the occasion of a forest fire would
result in a anomaly. This is due to that the fire’s temperature deviate from the forest’s
standard temperature [2]. The temperature of the fire itself can also vary depending on
the position of the measurement (is it on the edge or in the middle), which indicates the
extremeness of the anomaly.

By analysing the collected data over a longer period of time and compare it with itself,
it is possible to see a positive or negative trend, or no trend at all [1]. Continuing with
the example of the area covered by forest on Earth, a positive or negative trend could
indicate that the temperature of this area increases respectively decreases due to for
example climate changes and season of the year. This would not necessarily indicate
that there is an ongoing forest fire or snowstorm, due to that these events (for an area
where snow is unusual) would have larger deviations from the standard and therefore
be presented as anomalies. If there is no trend at all, the temperature of the forest
remains the same. All these three scenarios are a result, which provides the reader with
information about the data. From the given information, the reader could then make
predictions of the evolution of occasions in time.

3
2.2 Satellites
2.2.1 General Composition
Satellites are constructed by the same fundamental principle, regardless of what informa-
tion or service they provide to Earth. The principle is that a satellite can be divided
into two main sections: the payload and the platform (or bus) [3].

The payload is the equipment connected to the platform, which specifies the intention of
the specific satellite, and what data it should collect. The range of different payloads is
great, but common sensors are optical and hyperspectral imaging sensors, together with
antennas, transmitters and navigation signal generators [3]. An example of a satellite
is the Jason-2 satellite, which is shown in Figure 2. This satellite is designed to help
scientists study the ocean topography, and is therefore equipped with an altimeter to
measure the distance to the ocean surface [4, 5].

Figure 2: The Jason-2 satellite. Credits: NASA [4].

The development of sensor technology is constantly trying to reach improvements, which

drives the payload sizes to become smaller [6]. This implicates that several payloads
can be connected simultaneously to the satellite’s platform, the section of the satellite
that all spacecraft must have to be able to operate in space, making it possible to collect
different kinds of data. The platform has to be able to analyse and connect to all the
different kinds of payloads, which indicates that the platform has to be constructed in a
general way, with some major components that makes the satellite operate [3].

The general construction of a satellite’s platform includes a power source (often in terms
of a solar panel and batteries), and an antenna, to transmit and receive information
from Earth [4]. Beyond the need of power and the ability of signal transmission, the
platform also has a need for telemetry, tracking and control subsystem, thermal control,
and a structure which contains and protects the hardware in the satellite. These different
subsections are for protecting the satellite against the extreme conditions in space, to
track and regulate the movement, altitude and velocity of the satellite, together with the

4
satellites characteristics and health [3]. To do required calculations and to be able to
make decisions, the satellite also needs some form of an on-board computer [7], which
description is presented in Section 2.2.3.

2.2.2 Communication with Earth

As already mentioned, satellites communicate with equipment such as antennas and
radios, by sending radio waves to antennas on other devices, either on Earth or another
satellite [8]. The radio waves contain information, often in terms of scientific data, health
data of the satellite or the satellite’s location.

The principle of communication between a device on Earth and a satellite in space is

presented in Figure 3. The signals are transmitted from the Earth station by using a
certain frequency range, the so called uplinks, so that the satellite can receive information.
The information given by the signal is analysed on the satellite, and the new signal,
before being re-transmitted, is also amplified. The signal, which might contain different
information than the one received by the satellite, are re-transmitted in another frequency
range, the so called downlinks, toward the receiving Earth station [9]. This procedure is
the same, regardless if the satellite is used for science, voice transmissions, navigating
systems or something else.

Figure 3: Principle of sending signals with a satellite. Credits: ESOA [9].

The Earth Stations, also called Satellite Ground Station (SGS), are structured to allow
a variety of users and applications to stream and collect remote sensing satellite data
[10]. They can connect to different satellites, and it is therefore not necessary for each
satellite actor to have their own SGS. The actors have the opportunity to use a SGS
only when necessary, by renting the feature of a SGS for a specific time slot [10].

With the limitation of not having a SGS available at all times, actors want to minimise
the time necessary to be connected to the SGS, and the associated costs. The actors
cannot evade the requirement of collecting data, and a problem occurs when the data is
too heavy to downlink in the limited time. The actors could then try to extend the time
slot, but seeing that the SGS are occupied and operated during most hours of the day
all year around, the extent of time slot could be expensive [11].

5
By streaming and collecting the data by using a SGS, for further data processing and
storing, there is a time delay for returning to the satellite with actions that needs to
happen quickly. Sometimes, it is not the data that is of major interest, but for example
if the satellite needs to change it’s position, due to critical surroundings such as extreme
temperature or pressure that is harmful for the satellite’s components. Since the data
needs to first be downlinked and then analysed, before a decision can be made and
instructions uplinked to the satellite again for analysis and execution, the time delay
could be unnecessarily large [12]. This shows a need of investigating machine learning
algorithms possible to apply directly onboard the satellite to prevent delays in actions.

2.2.3 On-Board Computer

The On-board Computer (OBC) is the device that runs the satellite’s on-board software,
in order to manage the satellite’s vital functions and makes the system perform as desired
[7]. To mention some of the OBC’s functions, the management of telecommunication
actions, attitude and orbit control, failure detection, isolation and recovery are often
included. On-board data processing is also a function of an OBC.

To be able to manage the desired functions, an OBC consists of a microprocessor, memory

banks and interfacing chips for interaction with sub-systems [7]. It can either be an
integrated unit on a satellite’s platform, or a modular device connected as a payload to
the platform. Regardless of this choice, the OBC must be within the satellite’s limit for
Size, Weight and Power (SWaP) [6]. The radiation of space, described in Section 2.3,
together with the SWAP condition, cause constrains in the possible hardware composition
of the OBC. It is desired to have components that are tolerant to the radiation of space
for a longer period of time before being significantly damaged, without jeopardising the
hardware’s performance and reliability or the satellite’s SWaP limit [13].

In the composition of an OBC, the components are often commercial/Commercial-Off-

The-Shelf (COTS). The COTS components has a low cost due to the availability on the
market, since they correspond to the hardware used in a regular computer on Earth.
The constant evolution of technology on Earth, provides a high performance of the
COTS components. Since the COTS components has very little, if any at all, protection
and insulation for the radiation of space, they often tend to have a very short life span
before being significantly damaged [14]. It is therefore not reliable to only have COTS
components in the composition of an OBC, due to the ignorance of the components
possible life span. Beyond COTS components, the OBC also has specialised components
with applied radiation hardening processes. These processes makes the components
more radiation tolerant and provides them with a low probability for suffering from
significant issues due to the radiation [14]. With applied radiation hardening processes,
the performance of the specialised components are more capacity demanding since they
require more time, energy and storage than the COTS components.

With combinations of components that are both specialised and COTS, hybrid OBCs
are becoming more frequently used to more easily meet the performance requirements
for energy, speed and storage, without jeopardising the reliability or the life span of
the device. In systems that are hybrid or heterogeneous, any combination of hardware
specialised for on-board processing are used, for example Graphical Processing Unit

6
(GPU), Central Processing Unit (CPU) and Field Programmable Gate Array (FGPA)
[14]. The different hardware has different aims. For example, to identify anomalies in
images, the detection could be delegated to the GPU since it is a graphical issue, even
though this component has a high requirement of power and therefore has a rare usage.
With the evolution that the GPU’s are integrated as System on Chip (SoC) systems,
which has been done on Unibap’s iX5, the requirement of power is more easily met,
which changes the possible use cases for GPUs on OBCs [14].

Table 1 lists OBCs that are potentially capable of running algorithms for anomaly
detection, simultaneously with standard commands and data handling. It includes only
OBCs with public hardware specifications. The fields that are left blanked is supposed
to mark that the information either could not be found or is not relevant for the specific
OBC. It is not unrealistic that other OBCs on the market, beyond the ones presented in
Table 1, could be relevant for on-board anomaly detection.

Table 1: Overview of some high performance on-board computers.

Manufacturer Model CPU GPU FGPA RAM Storage
Unibap iX5 x4 SoC Radeon 72 2 GiB 64+240 GB
Unibap iX10 x4 8t SoC Radeon 924 2 GiB -
ibeos EDGE x4 192 core - 2 GB 4 GB NAND
Innoflight CFC-500 x4 192 core 1-4 Xilinx 4 Gb 16 (SoC) 16 GB
Moog Bandera > 1 GHz - Yes > 24 GB > 48 GB
Moog G-series 1.2 GHz Yes Xilinx 2 GB 64 GB
SBC KU060

From the data in Table 1, it is clear that the OBCs vary in components, capability
and architecture. A typical satellite computer is therefore difficult to identify, but it
seems that some components are more frequently used than others, for example the x4
in the choice of CPU. Since multi-core CPUs are, as observed, more frequently used,
just as multi-core GPUs, the created SoC systems indicates that implementation and
running of efficient machine learning algorithms on OBCs should be possible [14]. As
presented in Table 1, FPGAs are common in OBCs and could therefore potentially be
an alternative for the implementation and running of the on-board algorithms. The
alternative regarding FPGAs is not considered in this thesis since FPGAs are used for
running the satellite when using Unibap’s OBC iX5 through the service SpaceCloud
Framework (more in section 2.5). The option of using FPGAs are still mentioned to
present the possibility to use different components of the OBC for anomaly detection.
Further topics that should be contemplated in the choice of operating OBC, beyond the
processing capability and what kind of processor, are power requirements, reliability of
software, memory (storage and RAM), inter-operability and interfacing capabilities [7].

Proceeding from Table 1, the OBCs produced by Unibap should be suitable for running
anomaly detection algorithms. Since this thesis is in collaboration with Unibap, it is
henceforth their OBCs and hardware that are going to be used in this project together
with Unibap’s SpaceCloud Framework, more in Section 2.5.

7
2.3 Environmental Effects on the Satellite Structure
The environment in space imposes threats upon satellites when launched into Earth
orbit, such as temperature extremes, radiation and high-vacuum pressure. Due to the
environmental conditions, specific materials and devices are required to prolong the
satellite’s mission to prevent system failures [3]. Damage to the satellite components
can cause anomalies to appear in the collected data. This imposes a degree of error
which can compromise the use of the data. For example, if the satellite monitors sea
level changes for tsunami detection, an anomaly due to satellite damage may alert the
system of a tsunami when there in reality is none.

The environment in Earth orbit varies depending on the distance from Earth’s surface.
Satellites are launched into different orbits depending on the mission and mission dura-
tion. Satellites are commonly launched into six different types of orbits: Geostationary
orbit (GEO), low Earth orbit (LEO), medium Earth orbit (MEO), sun-synchronous
orbit (SSO), geostationary transfer orbit (GTO) and Lagrange points (L-points) [15].
Examples of satellite orbits are displayed in Figure 4.

(a) Low Earth orbit (LEO) (b) Sun-synchronous orbit (SSO)

(c) Geostationary orbit (GEO) (d) Geostationary transfer orbit (GTO)

Figure 4: Example of Earth orbit types. Credits: ESA [15].

L-points orbits are used to map and observe deep space and GTO are used for moving a
satellite from one orbit to another. Thus, these orbits are not of interest in this project.
LEO lie at an altitude between 160 km and 1000 km above Earth. These orbits are
commonly used for high resolution satellite imaging stationary relative to Earth’s surface,

8
and are used for telecommunication and meteorology [15]. MEO denotes a range of orbits
between LEO and GEO that are used for navigation satellites, for example to track
aeroplanes. SSO lie at altitudes between 200 km to 1000 km and are synchronised to be
in a fixed position relative to the Sun. Thus, SSO satellites observe the same location
at the same time of the day, and these satellites are commonly used to detect change
relative to time [15].

In LEO, GEO, MEO and SSO the satellites are exposed to extreme environmental
circumstances. For example, in LEO the hard vacuum space ranges from 10 9 torr to
10 6 torr (where 1 torr = 133 Pa). This can cause outgassing, which is when excess gas is
released within a solid. This gas can accumulate on the optical instruments and sensors
of the satellite and its electrical circuits. High levels of outgassing and contamination
can cause electrostatic discharge which affects the charging and discharging of a satellites
battery. Therefore, the materials that can be used in a satellite’s instruments are limited
to minimise the risk of outgassing and electrostatic discharge [3].

In all orbits, satellites are subjected to high levels of radiation. The radiation effects on a
satellite further limits the electronic devices that can be used on the mission. Radiation
damage to electronic devices range from degradation in parametric performance to
complete functional failure of the device [16]. The computers that withstand radiation and
damage the most efficiently are outdated with respect to performance. These limitations
need to be taken into account when designing algorithms. For example, models with
large computational costs cannot be applied on the current satellite hardware.

2.4 Algorithms for Anomaly Detection

2.4.1 Choice of Algorithms
As described in Section 2.2.3, the OBC is the part of the satellite that preforms the
on-board processing. Anomaly detection is one of the possible processes that can be
executed on the OBC, depending on its components and their obligations. For the
anomaly detection to work, the algorithms needs to be selected with respect to the chosen
OBC and its corresponding limitations.

Two groups of algorithms that the students believes to be able to function on Unibap’s
OBC iX5 are the Convolutional Neural Network (CNN) and the Long Short-Term Mem-
ory (LSTM) Neural Network. These networks are chosen due to their compatibility with
time-series forecasting and prediction [17, 18]. There are other possible algorithms to
implement for anomaly detection on satellite time-series such as Principal Component
Analysis (PCA) and One Class Support Vector Machine (SVM). However, neural net-
works are chosen since they tend to outperform classical machine learning methods [19].
There might be other possible algorithms to implement for anomaly detection on satellite
time-series beyond the ones mentioned, but the investigation of which these are and how
to implement them will not be included in this thesis. With respect to the limitation of
time for the project, the students constrained the number of neural networks down to
two, to hopefully complete the thesis within its time range.

9
2.4.2 Convolutional Neural Network (CNN)
Convolutional Neural Networks (CNN) emerged within image recognition during the
1980s, and their structure is derived from the human brain’s visual cortex and humans’
ability to recognise patterns [20]. Therefore, CNNs are commonly used for image classifi-
cation and recognition.

The structure of CNNs revolves around its convolutional layers. The neurons in the
first convolutional layer are not connected to every pixel in the input image, but to
the pixel that exists in their receptive fields [20]. This allows the second convolutional
layer to connect only to the neurons that are positioned in a small portion of the input
layer. The neural network (NN) is then allowed to perceive low-level features in the
first convolutional layer and increase the feature level in the subsequent convolutional
layers. Normally, the CNN architecture consists of a few convolutional layers then a
pooling layer, followed by another few convolutional layer and then another pooling layer,
until the NN is deep enough to achieve high-level learning of the input system [20]. An
example of the architecture for a typical CNN is presented in Figure 5. However, the
architecture and depth of the CNN depends on the task.

Figure 5: Example architecture of a typical CNN. Credits: C Galván-Trejada, V Maeda-

Gutiérrez [21]

2.4.3 Long Short-term Memory (LSTM) Neural Network

Long Short-term memory (LSTM) is a class of the Recurrent neural network (RNN) which
can predict future data points in for example time-series. A RNN consists of connections
pointing from the input to the output (feedforward connections) and connections pointing
backwards (feedbackward connections). For example, a neuron in an RNN may produce
a output and send it forward but also backward to itself as an input, this is called a
recurrent neuron. An example of this kind of RNN is presented in Figure 6. Thus, a
collection of recurrent neurons form a memory system for the NN. In contrast to CNN,
RNNs input consists of sequences of arbitrary lengths instead of fixed-sized inputs [22].
Problems with the typical RNN structure arise, when the distance between the input
and the output is too large, since RNNs are cannot learn to connect to information that
require "long-term dependencies" [22]. However, the LSTM class of RNN type networks
are capable of learning long-term dependencies. The structure of an LSTM allows the
network to learn what information to store in the long-term memory state, what to throw
away and what to read from it [22].

10
Figure 6: Example architecture of a typical RNN. Where a block of the NN, A, observes
an input, x, and outputs a value h. Credits: Christopher Olah [23]

Figure 7: Example structure of an LSTM cell. Where h represent the inputs, c

represents the cells outputs. Credits: Christopher Olah [24]

At each time step in the LSTM cell some memories are dropped and others are added.
This is shown in Figure 7. Thus, LSTM is able to learn what information (data-points)
is important in training the Neural Network on the data set. The NN can then preserve
significant information and use it when it is needed. Thus, the LSTM class can capture
long-term patterns in for example time-series, and recreate sequences of it, which makes
it useful in anomaly detection.

2.4.4 Autoencoders
For the implementation of CNN and LSTM NN, autoencoders were used as the main
architecture. This is further discussed in Section 3.4.2, where the used implementations
are further discussed.

Autoencoders are used for unsupervised machine learning, where the NN tries to learn
the representation of a data set to be able to recreate it [25]. Autoencoders are a type of
artificial NN, which is capable of performing unsupervised learning to accurately represent
the input data. Where, an artificial NN is a network designed to function similarly to the
human brain’s ability to process and store information [26]. Autoencoders are commonly
used to reduce the size of the input data, which makes these networks useful for systems
that require low dimensionality [26]. Autoencoders are useful for anomaly detection since
they are able to reconstruct the input data, where the output closely resembles its input.
Thus, by creating a threshold the autoencoder can predict where anomalies are most
likely to occur.

11
2.5 Unibap’s SpaceCloud Framework
Unibap AB’s service SpaceCloud Framework (SCFW) is developed to among other things
circumvent undesired problems on the satellite or the need of a SGS, and to create a
higher value data-product by only downlinking the relevant data [27]. This is done by
enabling on-board data processing on the satellite’s OBCs. The purpose of SCFW is to
allow costumers to deploy machine learning applications on spacecraft such as satellites,
without the need of actually implementing the application directly for the spacecraft
itself [27]. The applications developed by costumers are deployed on the spacecraft by
being uplinked from a SGS.

SCFW provides the opportunity to reuse hardware already in orbit, since new appli-
cations can be implemented and activated from Earth. Together with that SCFW is
a multi-node system, several active applications are allowed at the same time. The
enabling of shared resources makes is possible to run different applications on the same
spacecraft, equipped with the vital payloads, to collect, analyse and downlink different
kinds of data simultaneously [27].

In this thesis, it is going to be investigated whether or not anomaly detection together

with SCFW can be applied to satellite hardware. If it is possible, then anomalies can be
detected directly on the satellite’s OBC. In the occasion of the OBC finding one or more
anomalies in the collected data set by the satellite’s payloads, warning information about
anomalies in significant data sets can be sent to the user of the SGS during communication
with Earth. This indicates that the user gets information about a potential danger, and
could downlink and analyse the specific data set as quick as possible. This scenario
indicates that the potential danger in the data set would be found and analysed quicker
than if the user would analyse all data sets collected by the satellite before even finding
the anomalies.

12
3 Methodology
3.1 General procedure
To implement an algorithm for anomaly detection on a time-series data set, previous
implementations were investigated. Various available algorithms were investigated for
possible application. Two different algorithms are implemented with respect to anomaly
detection, different algorithms are considered to have a greater likelihood of success for at
least one working algorithm. In the best case scenario two algorithms can be compared
against each other.

The procedure for both CNN and LSTM NN autoencoders are similar, however, the
approach is different. Both algorithms are trained on time-series where the output is
a reconstruction of the input, that follows the general trend and pattern of the input.
Thereafter, the error between the input and output is determined on the test data. The
algorithm then classifies a data point as an anomaly if the reconstruction error is above
a static threshold.
v
u N
u 1 X 2
Threshold = |x ⇤ x| + 3 t (x⇤ x)i |x⇤ x| , (1)
N 1 i=1

where x⇤ represent the predicted value, x the true value and N is the number of samples
in the data set.bThe static threshold, which is calculated in Equation 1, determines how
easily the neural networks reconstructs or predicts the data. It is done by measuring the
deviation between the predicted and the true values. The static threshold is determined
by using the three-sigma statistic rule [28], which is used to determine the percentage of
values that lies in an interval estimate on a normal distribution.

3.2 Considered Time Series with Anomalies

3.2.1 Desired Data of Usage
Initially, the investigated data sets came primarily from climate research. By utilising
Googles "Dataset search" application [29], several data sets were found. The most
promising data was found from the CEDA archive [30], which contains a monthly grid
ranging from 1993 to 2014 of mean sea level changes (the sea level deviations of world
mean sea level) at varying latitudes and longitudes. The CEDA archive data set was of
interest since the data was easily accessible and contained many data points. Although
this data is a valuable real-life scenario in which detected anomalies can prematurely
identify possible tsunamis, the lack of labelled anomalies in the data set makes it difficult
for training the neural networks. Therefore, real satellite time series were not used.
Instead the time series were chosen depending on if they were suited for supervised
learning and if similar analysis could be done on a satellite.

A important characteristic of the data set is its quality. To yield the most accurate
predictions when training and testing the algorithms several factors are significant. The
most evident factor is the size of the data set, such that the data set can be split into a
large training set, and a validation and testing set [31]. Where the validation set is used

13
for tuning the algorithm. A large data set is necessary to avoid overfitting or underfitting
the data, the standard training data set consists of 75% of the original data set. For
anomaly detection it is important that the majority of the data set follows a trend and
that there is no random element to the data points. This allows the algorithms to detect
patterns and single out anomalous behaviour. These factors where also considered when
choosing appropriate time series.

3.2.2 Data of Machine Temperature

The considered time series was found from the Numenta Anomaly Benchmark (NAB)
repository [32], which contains 58 time series with labelled anomalies where the time
series are either artificial or real data. In the repository several data sets were of interest,
however, the neural networks are mainly be trained on NAB’s "Machine temperature
data". This data set is shown with labelled anomalies in Figure 8.

Figure 8: Temperature sensor data of an internal component of a large, industrial

machine, with annotated anomalies.

The data set contains 22696 data values from a temperature sensor of an internal compo-
nent of a large, industrial machine. This scenario is also applicable to satellites, since
their internal components need to be monitored to prevent system failure due to for
example over heating. The causes of the anomalies are known. The second anomaly
represents a planned shutdown of the machine, whereas the third is unexpected and
leads to the fourth anomaly, which represents a system failure [32]. This time series was
chosen not only due to its applications but also due to the position of the anomalies.
The location of the anomalies allows an even distribution of anomalies when dividing
the data set into a training and testing data set.

3.2.3 Data of CPU Utilisation

To evaluate if the neural networks can detect anomalies accurately in a range of data
sets, an additional data set was used. These time series were also found from the NAB
repository [32]. In contrast to the temperature sensor data, two time series of similar size
and range of values were used. These time series represent Amazon Web Services (AWS)

14
server metrics collected by the Amazon Cloudwatch service. The Amazon Cloudwatch is
a service used for monitoring and observing applications and get a view of operational
health in a system [33]. The metrics used in these time series is CPU utilisation. These
data sets are shown with labelled anomalies in Figure 9 and 10.

Figure 9: Data-set one, measuring CPU utilisation from AmazonCloud watch with
labelled anomalies.

Figure 10: Data-set two, measuring CPU utilisation from AmazonCloud watch with
labelled anomalies.

The data sets each contain 4032 data values measuring CPU utilisation. CPU utilisation
refers to a computer’s or system’s usage of processing resources [34]. This metric is
therefore applicable to satellite monitoring, since some system components require heavy
CPU utilisation whilst others require less. By monitoring CPU utilisation on satellites,
CPU utilisation degradation can be detected in advance as anomalies. Furthermore,

15
the location of the labelled anomalies in Figure 9 and 10 allow an even distribution of
anomalies when dividing the data set into a training and testing data set

3.3 Data preparation

To ensure that the time series is internally consistent the data values need to be stan-
dardised with a standard deviation of 1 and a mean of 0. This will allow the neural
networks to better generalise the data and prevents the network from making inaccurate
assumptions.

Furthermore, the data sets were divided into a training and testing data set. The data
measured using a temperature sensor was divided into a 75% training and 25% testing, to
allow the neural network to train on a sufficiently large data set. For the data gathered
from the AWSCloudwatch data set, the dataset in Figure 9 was used for training and
Figure 10 was used for testing. The standardised train and test data set with labelled
anomalies for both kinds of data sets are shown in Figure 11, 12, 13 and 14.

Figure 11: Normalised Temperature sensor data of an internal component of a large,

industrial machine used for training, with annotated anomalies.

16
 Figure 12: Normalised Temperature sensor data of an internal component of a large,
industrial machine used for testing, with annotated anomalies.

Figure 13: AWSCloudwatch standardised training data-set, with annotated anomalies.

17
Figure 14: AWSCloudwatch standardised testing data-set, with annotated anomalies.

3.4 Experiment setup

3.4.1 Software used for Implementation
To be able to implement an algorithm, it is required to have software installed that
allows programming of neural networks. In this thesis, the Integrated Development
Environment (IDE) JupyterLab in the package manager Anaconda Navigator is used for
implementation, training, testing and evaluation of both algorithms. JupyterLab is the
chosen IDE due to its ability to use the language Python and the platform TensorFlow in
the implementations. It is possible to chose another IDE, such as for example PyCharm,
if desired.

The manual for implementing applications for SCFW is followed when trying to work
with SCFW. This manual is found on Unibap’s webbpage [27]. For further explanation
about the implementation for SCFW, it is recommended to read the manual. The main
parts that can be said is that it is required to pull an already created Docker image from
Unibap’s repository, and adapt this one for the desired purpose. When the application is
correctly build, the new Docker container is then shipped to Unibap’s repository again
before the application can be applied on satellite hardware by using a specific VPN
provided by the company.

3.4.2 Public Implementations

In this thesis, the students investigates public implementations of the chosen algorithms
LSTM and CNN. A public implementation using TensorFlow is found for LSTM [35],
but applied to finance and not data from satellite time series. Similarly, the public
implementation of CNN, using Tensorflow, is found [36], originally intended for noise
in a room. These public implementations are managed and rewritten such that the
same results are achieved as the original authors, but with the students own code. The
structure of the codes are the foundations for the implementation intended for data from

18
satellite time series.

The students tried to find public implementations without this kind of artificial NN,
but were not able to find any. They therefore proceeded to use the implementations
including Autoencoders, with the belief of an advantage due to that the goal is for a
satellite’s OBC to detect an anomaly in a data set unsupervised for later implementation.
However, a supervised approach is considered in this report. Both final implementations
of CNN and LSTM for this thesis is attached as a repository on GitHub [37].

3.4.3 Choice of Evaluation Indices

The implementation of two different algorithms provides the opportunity to compare the
final results. For a fair comparison, the performance of the algorithms is investigated
using the same evaluation indices.

In this thesis, the evaluation of the algorithms is based on the visual presentation of
the algorithms’ performance, together with their respective confusion matrix of the
result. The visual presentations in forms of plots provides an intuitive understanding of
which data points the algorithms concluded to be anomalies. By comparing the plots of
the algorithms’ results to the original plots with the correct annotated anomalies, it is
possible to observe if an algorithm correctly finds anomalies.

By investigating a confusion matrix, such as the one presented below in Table 2, it is

possible to observe if the corresponding algorithm has determined a data point correctly.
The different kinds of results are presented as either True or False Positive (TP/FP)
respectively Negative (TN/FN). The sum of TP or TN are the amount of correctly
determined anomalies, compared to the labelled ones. The number of FP or FN presents
the amount of incorrect determined anomalies by the algorithm.

Table 2: Presentation of a general confusion matrix.

Predicted Anomalies
Anomaly Non-anomaly
Anomaly True Positive (TP) False Negative (FN)
True Anomalies
Non-anomaly False Positive (FP) True Negative (TN)

It is also possible to use evaluation parameters, such as precision and recall, to evaluate
the performance of the algorithms if desired. Both precision and recall depend on the
values of the confusion matrix.

TP TP
P recision = (2) Recall = , (3)
TP + FP TP + FN
where precision presents the rate of correctly determined anomalies compared to all
determined anomalies, and recall is the rate of correctly determined anomalies compared
to the actual amount of annotated anomalies [38]. Due to the fact that the available
public data sets only includes a few annotated anomalies, the students believes that the

19
amount of these are not large enough to have an impact on the presented examples of
evaluation parameters. Therefore, these kinds of evaluation parameters are not included
in this thesis’s evaluation of algorithms. By using data sets including more annotated
anomalies, evaluation parameters such as precision and recall could be of larger interest
to investigate the performance of the algorithms. Since the project objective of the
thesis is to correctly find an annotated anomaly in the data set, it is believed that visual
presentations and confusion matrices are sufficient to evaluate performance.

20
4 Results
4.1 Evaluation of CNN
4.1.1 Machine Temperature
The training of the implementation of CNN took on average approximately 20 seconds
when running the model on a Dual-Core Intel Core i5 processor. On the same processor
the execution time of applying the trained model on the test data set (time interference)
took approximately 3 seconds, however, the interference depends on the size of the test
data. The trained CNN algorithm has a size of 158 kilo-bytes. These results are the
same, regardless of which kind of data that was investigated.

Figure 15 presents the results on the testing data from the "Machine Temperature" data,
which is previously presented in Figure 12. The confusion matrix which represent the
performance of the CNN is presented in Table 3.

Figure 15: Predicted anomalies using CNN autoencoder on Temperature sensor data.

Table 3: Confusion matrix for CNN autoencoder of predicted results in Figure 15.

Predicted Anomalies
Anomaly Non-anomaly
Anomaly 1 0
True Anomalies
Non-anomaly 191 4347

From Figure 15 and Table 3, one can observe that the NN is able to correctly predict the
anomaly, and all predicted anomalies are in the same region as the correctly classified one.
Furthermore, as can be seen in the confusion matrix, the NN predicts more anomalies
than labelled in the test data-set. This can be a result of the threshold chosen to detect
the anomalies.

21
4.1.2 CPU
Figure 16 presents the results of CNN on the testing data from the "AWS Cloud watch -
CPU utilisation" data, which is previously presented in Figure 14. The confusion matrix
which represents the performance of the CNN is presented in Table 4.

Figure 16: Predicted anomalies using CNN autoencoder on AWS CPU utilisation data.

Table 4: Confusion matrix for CNN autoencoder of predicted results in Figure 16.

Predicted Anomalies
Anomaly Non-anomaly
Anomaly 0 2
True Anomalies
Non-anomaly 983 3048

From Figure 16 and Table 4 it can be observed that the NN is not able to correctly
predict the two anomalies presented in the public data set. In comparison to the results
from Figure 15, the NN predicts anomalies in a cluster. The NN fails at recognising
anomalous behaviour by the first anomaly, however, it recognises a cluster of anomalies
an instance after the presence of the second anomaly.

4.2 Evaluation of LSTM

4.2.1 Machine Temperature
The execution time of the training of the implementation of LSTM NN is on average ap-
proximately 2.5 minutes when running the model on a Dual-Core Intel Core i5 processor.
On the same processor the execution time of applying the trained model on the test data
set (time interference) took approximately 10 seconds, however, the interference depends
on the size of the test data. The trained LSTM algorithm has a size of 2.4 mega-bytes.
These results are the same, regardless of which kind of data that was investigated.

Figure 17 presents the LSTM results regarding the testing data from "Machine Temper-
ature", previously presented in Figure 12. The confusion matrix which represents the
performance of the LSTM implementation is presented in Table 5.

22
Figure 17: Predicted anomalies using LSTM autoencoder on Temperature sensor data.

Table 5: Confusion matrix for LSTM autoencoder of predicted results in Figure 17.

Predicted Anomalies
Anomaly Non-anomaly
Anomaly 0 1
True Anomalies
Non-anomaly 35 4503

From Figure 17 and Table 5, one can observe that the LSTM NN is not able to correctly
predict the anomaly included in the data set. It does predict anomalies, but the range of
these are wide and there are no obvious relation between the predicted anomalies. By
investigating the confusion matrix, it is also here obvious that the NN predicts more
anomalies than what is annotated in the test data set.

4.2.2 CPU
Figure 18 presents the LSTM results regarding the testing data from "AWS Cloud Watch
- CPU utilisation", previously presented in Figure 14. The confusion matrix which
represents the performance of the LSTM is presented in Table 6.

Figure 18: Predicted anomalies using LSTM autoencoder on AWS CPU utilisation data.

23
 Table 6: Confusion matrix for LSTM autoencoder of predicted results in Figure 18.

Predicted Anomalies
Anomaly Non-anomaly
Anomaly 1 1
True Anomalies
Non-anomaly 1400 2631

From Figure 18 and Table 6, one can observe that the LSTM NN is able to partly
correctly predict the anomalies included in the data set. By investigating Figure 18 it
is clear that the algorithm predicts anomalies within the correct region, but is despite
this fact unable to predict the first anomaly. The figure also shows that the predicted
anomalies are clustered around the true anomalies. It might be seen as a premonition
for an anomaly to appear, and it also classifies the history of the second anomaly in the
data-set as anomalies. By investigating the confusion matrix, it is clear that the NN
predicts more anomalies than what is annotated in the test data set. This can also be a
result of the threshold chosen to detect anomalies.

4.3 Implementation for SCFW

To create an application with the algorithms for SCFW that is applied on satellite hard-
ware, the students first tried to install the software Docker and a corresponding operating
system, which is Ubuntu-18.04. By using the Linux-commands "sudo apt update" and
"sudo apt install python3-pip", and installing the correct Windows Subsystem for Linux
and running the application as admin, Docker finally worked.

With Docker installed, the next step is to build the application for SCFW. This is done
by building a Docker-image in Ubuntu-18.04, by following the manual from Unibap.
The students where able to build the example Docker-image provided by Unibap, which
displays the text "Hello SpaceCloud!" when executed. The Docker-file for the application
provided by Unibap is presented in Figure 19, and the building and execution is presented
in Figure 20. The commands used for building and pushing the application to Unibap’s
repositories are presented in Figure 21.

Figure 19: Example architecture of a Docker-file which prints "Hello SpaceCloud!".

24
 Figure 20: Result of running the application in Figure 19.

Figure 21: Result of pushing the application in Figure 19 to Unibap’s repositories.

To build a Docker-file for one of the algorithms, the students tried to reuse the structure
from "Hello SpaceCloud!", with some additional parts regarding importation of the
used data and necessary packages. The structure for the application regarding the CNN
algorithm on the "Machine Temperature" data-set, which is the students’ first try, is
presented in Figure 22. As earlier mentioned, the structure is the same as for the "Hello
SpaceCloud!" application, but the chosen algorithm and data-set is copied into the
application for the execution.

Figure 22: Final architecture of a Docker-file for the CNN algorithm and data regarding
Machine Temperature.

25
 Figure 23: Result of trying to build the application in Figure 22.

With the application built as Figure 22 presents, the students tried to build the Docker
file. With calling "docker build ." while standing in the correct folder, the error messages
presented in Figure 23 arose. These error messages indicates that the data set and
algorithm attached to the application could not be found. The usage of the building
command in Figure 21 resulted in the same error messages. With a lack of time and
resources from Unibap, the students cancelled the implementation of SCFW to complete
the rest of the project. This decision was supported by the students’ supervisor from
Unibap.

26
5 Discussion
5.1 Comparison of Performance
From Figure 15 and 17, it can be observed that CNN accurately predicts the anomaly
and only detects anomalies in the positional region of the labelled anomaly, whereas
LSTM is unable to detect anomalies in the correct region of the true anomaly. However,
from Figure 16 and 18, it can be observed that CNN only detects anomalies in the region
following the second labelled anomaly. LSTM detects anomalous behaviour for both
the first and second anomaly, and correctly predicts the presence of the second. There-
fore, CNN has the best performance with regard to the "Machine system temperature"
data set, whereas LSTM has the best performance with regard to "CPU utilisation".
Furthermore, from the confusion matrices 3, 4, 5 and 6, it can be observed that both
implementations predict more anomalies than what is annotated in the test data set.
The reason for this is could be the lack of enough annotated anomalies in the training
data. One consequence of using a training data which are not good enough, is that the
threshold chosen to detect anomalies becomes incorrect for accurate predictions.

The performance of the algorithms can not only be measured using the confusion matrix.
Since the algorithms have to be operational on a satellite, the execution time and size
of the algorithms needs to be taken into account in the evaluation. CNN is able to
train and detect anomalies on both data-sets in approximately 30 seconds, by having
a size of 158 kilo-bytes. LSTM took longer to train and detect anomalies, and on
average took 3 minutes to predict anomalies, and has a size of 2.4 mega-bytes. The
training time for LSTM was dependent on the number of data points considered in
predicting future data points in reconstructing the training data set. Thus, LSTM’s
training took originally between 10 minutes and 40 minutes. Therefore, with regard to
execution time of the time interference, CNN is more efficient than LSTM. By comparing
the sizes of the algorithms, it is also clear that CNN is the most suitable to apply
on satellite hardware due to its small size. Since satellite hardware has limitations
within for example storage, it is believed that a smaller size of a NN would be more
suitable for a possible application on SCFW, if the performance of the algorithms allow it.

Considering the algorithms’ performance with regard to their accuracy in predicting

anomalies and their execution time, the CNN autoencoder can be considered as the
better algorithm in detecting anomalies on satellite time series. Despite not recognising
anomalies correctly in the "CPU utilisation" data set, which can be a result of not
training the algorithm on a larger data set, the faster execution time in comparison to
LSTM has to be considered. For satellite anomaly detection it is pivotal that anomalies
are recognised on time and not 40 minutes after the anomaly has occurred, to avoid
system failure. This is due to that the concept of on-board anomaly detection is for the
system to detect anomalies rapidly.

5.2 Time-depending Impacts

Regardless of which algorithm that is going to be implemented for this problem, a fact
to be aware of is that unknown problems are, with a high probability, going to appear,
that might affect the time plan. Some tasks might differ in time for different people, due

27
to for example hardware, operating system, troubleshooting and previous knowledge. It
is therefore important to keep in mind that the steps performed in this report might
take shorter or longer time for another person to do, despite following the same general
procedure for the chosen algorithm.

During this thesis, the students encountered one problem before the implementations,
regarding the installation of TensorFlow, which was more time demanding than planned.
Installation and activation of TensorFlow is required for both algorithms, since both
implementations depends on this platform’s ability to apply neural networks for machine
learning. Before starting the implementation of an algorithm, it is encouraged to install
TensorFlow on the computer that is used for the NNs in the end. This is done in a
terminal command prompt by writing "pip3 install tensorflow", whilst online. The
students found that for a computer using Windows 10 as operating system, it might be
required to have the latest supported Visual C++ downloads installed [39].

Another problem affecting the time schedule is the installation of the software necessary to
implement SCFW. For the students, this part was very time consuming. The installation
of Docker and learning how to use the Linux-distribution Ubuntu-18.04 required more
time than planned. It is concluded that this depended on the lack of previous knowledge
within this software and its usage before this project. The implementation of the
application for SCFW was also time consuming, due to the lack of documentation
and how the Linux-distribution worked. More about the evaluation of the application
implementation for SCFW in Section 5.3.2. For a person with larger knowledge about
SCFW, Linux-commands and Docker, the implementation of SCFW should not take as
much time as for the students.

5.3 Future Developments and Improvements

5.3.1 Anomaly detection
The choice of public data influences the performance of the algorithms to detect anomalies.
There are several factors that impacts the results, such as the number of annotated
anomalies and the size of the data set. For the algorithms to perform with accuracy
without overfitting or underfitting on the training data, the training data needs to be
large. The rest of the data is then used for validation and testing, where the model is
tuned using the validation data set. The testing data is in the end used for the final
evaluation of the algorithm. However, both the "Machine system temperature" and
"CPU utilisation" data set were not large enough to perform accurate training of the NN’s.

The training data-set also need to contain enough samples of anomalies to accurately
predict their position in the testing data. Since the chosen data sets contained few
annotated anomalies in comparison to its size, it was prioritised to have the majority of
anomalies in the training data. This lead to a lack of anomalies in the testing data, and
removed the possibility of including a validation data set. The results of the algorithms
can therefore be challenged, since both NN’s were tuned with regard to the test data.

For future development, larger data sets that contain more annotated anomalies should
be used to improve the training of the algorithms. This can also improve the results

28
of the algorithms, since it can provide annotated data for an equal division between
training, testing and validation data.

5.3.2 Evaluation of SCFW

During the implementation for SCFW, the students encountered many difficulties. Partly
due to their lack of previous knowledge within the Linux operating system, and partly
due to the lack of documentation of SCFW. Since the students were new to using Linux
commands, even the installation of Docker resulted in error messages concerning for
example the lack of necessary packages. The majority of encountered difficulties is
believed to depend on the lack of how a Linux distribution works, how to edit files and
operate in the system. The research after correct commands were time-demanding, and
can therefore be observed as a large factor in the ability to implement the application
within the limited time.

The first specified difficulty that occurred was to understand what a Docker was, why it
was needed and how to install it. But with Docker correctly installed, the automatic login
to Unibap’s repositories is the part of the implementation for SCFW that is accomplished
without any difficulties. During the construction of the application "Hello Spacecloud!",
the students had difficulties with pushing the Docker-image to Unibap’s repositories.
This was later solved by the code presented in Figure 21. The second specified difficulty
was to create copies of the data set and algorithms to the desired application. It was at
this stage that the implementation was cancelled.

Since the implementation for SCFW was cancelled, the implementations of the anomaly
detection was unfortunately not able to be applied on satellite hardware. It is therefore
still unknown if the anomaly detection actually works on Unibap’s OBC iX5. With
some more time and resources, the students believes that they would have been able
to create applications for the different implementations of anomaly detection. Since
the implementation sizes are in terms of kilo- or mega-bite, the students believes that
the anomaly detection should be able to work on satellite hardware with an operative
application.

According to the students, a good addition for the implementation for SCFW could be
a document with Linux commands that are frequently used. This might simplify the
beginning of implementation for a person without previous knowledge. Another augment
could be different ways to tag and push Docker files to Unibap’s repositories, since this
was one of the larger difficulties that appeared for the students. Unfortunately, the
implementation for SCFW was cancelled in the beginning of the application creation. This
indicates that difficulties and problems are able to appear later in the implementations,
but these did not appear for this thesis.

29
6 Conclusion
Unibap’s SCFW allows deployment of NNs on the satellite systems, which permits
anomaly detection on satellite time series. Anomaly detection aboard satellites is pivotal
since anomalous behaviour can be a result of system malfunction. Furthermore the
detection of anomalies can be used in detecting abnormal environmental behaviour on
Earth. Due to the hazardous environment in space, robust but primitive computers with
respect to performance need to be used. This limits the algorithms that can be applied
aboard the satellite. To evaluate which algorithm is appropriate for implementation on
SCFW a CNN autoencoder and a LSTM autoencoder were applied to the same data
sets.

By comparing the confusion matrices of the CNN and the LSTM, their execution time and
size, it is concluded that CNN is best suited for the problem. Both networks successfully
detected anomalies on both data sets, but with varying accuracy. The implementation of
the CNN autoencoder did not result in the most accurate predictions on either dataset.
However, since CNN was significantly faster and had a lower dimensionality than LSTM
it is more applicable to satellite hardware. The implementation of the CNN autoencoder
for SCFW was unsuccessful. The failure of creating an application for anomaly detection
on SCFW was a result of time limitation and lack of knowledge of Linux operating
system. Nonetheless, it is hypothesised that the CNN autoencoder is functional on the
iX5, since it has a low dimensionality.

In the case of anomaly detection and the implementation of the NNs for SCFW, future
developments need to be made. Foremost, the data-sets used for the implementation of
the networks was flawed, which negatively affected the training of the CNN and LSTM.
In the usage of a larger sized data set, that includes more labelled anomalies than the
ones used in this thesis, is believed to improve the performance of the networks and
to conduct accurate training, validation and testing of the algorithms. Therefore, it
is concluded that future development and implementation needs to be made to prove
CNN’s functionality on the satellite hardware by using the service SCFW.

30
References

[1] Khanin Artur. Time Series and How to Detect Anomalies in Them — Part I. (Last
Used: 2021-04-12). Nov. 2020. url: https://becominghuman.ai/time-series-
and-how-to-detect-anomalies-in-them-part-i-7f9f6c2ad32e.
[2] Mahsa Salehi and Lida Rashidi. “A Survey on Anomaly detection in Evolving Data:
[with Application to Forest Fire Risk Prediction]”. In: ACM SIGKDD Explorations
Newsletter 20 (May 2018). doi: 10.1145/3229329.3229332.
[3] Christopher Leonard. “Challenges for Electronic Circuits in Space Applications”.
In: (). Analog Devices, Inc. url: https : / / www . analog . com / en / technical -
articles/challenges- for- electronic- circuits- in- space- applications.
html.
[4] Sandra May. What Is a Satellite? NASA. Aug. 2017. url: https://www.nasa.gov/
audience/forstudents/5- 8/features/nasa- knows/what- is- a- satellite-
58.html.
[5] Earth Esa. Jason-2. (Last Used: 2021-04-26). url: https://earth.esa.int/web/
eoportal/satellite-missions/j/jason-2.
[6] Nandinbaatar Tsog et al. “Intelligent data processing using in-orbit advanced
algorithms on heterogeneous system architecture”. In: (2018). doi: 10.1109/AERO.
2018.8396536.
[7] SatSearch. An overview of on-board computer (OBC) systems available on the
global space marketplace. (Last Used: 2021-04-08). Mar. 2021. url: https : / /
blog.satsearch.co/2020-03-11-an-overview-of-on-board-computer-obc-
systems-available-on-the-global-space-marketplace.
[8] Ashley Campbell. How do satellites communicate? NASA. Aug. 2017. url: https:
/ / www . nasa . gov / directorates / heo / scan / communications / outreach /
funfacts/txt_satellite_comm.html.
[9] ESOA. Operating in Space. (Last Used: 2021-04-08). url: https://www.esoa.
net/technology/operating-in-space.asp.
[10] Earth Sciences (ESS). Satellite Ground Stations – Everything you wanted to know.
(Last Used: 2021-04-08). Oct. 2020. url: https://www.essearth.com/satellite-
ground-stations/#:~:text=Satellite\%20Ground\%20Stations\%20(SGS)\%
20are,or\%20research\%20centres\%20like\%20CSIRO..
[11] Swedish Space Corporation (SSC). SSC’s global ground station network. (Last Used:
2021-04-08). url: https://sscspace.com/ssc- worldwide/ground- station-
network/..
[12] Unibap AB. SpaceCloud. (Last Used: 2021-04-06). url: https://unibap.com/en/
our-offer/space/.
[13] Fredrik Bruhn et al. “Introducing radiation tolerant heterogeneous computers for
small satellites”. In: (2015). doi: 10.1109/AERO.2015.7119158.

31
[14] Alan D. George and Christopher M. Wilson. “Onboard Processing With Hybrid
and Reconfigurable Computing on Small Satellites”. In: Proceedings of the IEEE
106.3 (2018). doi: 10.1109/JPROC.2018.2802438.
[15] European Space Agency (ESA). Types of Orbits. (Last Used: 2021-04-07). Mar.
2020. url: https://www.esa.int/Enabling_Support/Space_Transportation/
Types_of_orbits.
[16] European Space Agency. ESA - Radiation: satellites unseen enemy. Last Used:
2021-05-19. July 2011. url: https://www.esa.int/Enabling_Support/Space_
Engineering_Technology/Radiation_satellites_unseen_enemy.
[17] Jason Brownlee. How to Develop Convolutional Neural Network Models for Time Se-
ries Forecasting. (Last Used: 2021-04-07). Nov. 2018. url: https://machinelearningmastery.
com/how- to- develop- convolutional- neural- network- models- for- time-
series-forecasting/.
[18] Jason Brownlee. How to Develop LSTM Models for Time Series Forecasting. (Last
Used: 2021-04-07). Nov. 2018. url: https://machinelearningmastery.com/how-
to-develop-lstm-models-for-time-series-forecasting/.
[19] Jahnavi Mahanta. Introduction to Neural Networks, Advantages and Applica-
tions. (Last Used: 2021-04-04). July 2017. url: https://towardsdatascience.
com / introduction - to - neural - networks - advantages - and - applications -
96851bd1a207?fbclid=IwAR2a_ILFOcfodeyDJKWz-6eZhZulw8WP9GUuyBbpD9WbBHRTkqaIa7Ybnh
[20] Aurélien Géron. “Hands-on Machine Learning with Scikit-Learn & TensorFlow”.
In: O’Reilly, 2017. Chap. 13.
[21] Carlos E.Galván-Tejada Valeria Maeda-Gutiérrez. “Figure 2. Representation of
the architecture of a convolutional neural network (CNN)”. In: Comparison of
Convolutional Neural Network Architectures for Classification of Tomato Plant
Diseases (2019). Last Used: 2021-04-28. url: https://www.mdpi.com/2076-
3417/10/4/1245/htm.
[22] Aurélien Géron. “Hands-on Machine Learning with Scikit-Learn & TensorFlow”.
In: O’Reilly, 2017. Chap. 14.
[23] colah’s blog. “An unrolled recurrent neural network”. In: Understanding LSTM
Networks (2015). Last Used: 2021-05-20. url: https://colah.github.io/posts/
2015-08-Understanding-LSTMs/.
[24] Christopher Olah. “A LSTM cell”. In: Understanding LSTM Networks (2015).
Last Used: 2021-05-20. url: https : / / colah . github . io / posts / 2015 - 08 -
Understanding-LSTMs/.
[25] Will Badr. Auto-Encoder: What Is It? And What Is It Used For? (Part 1). (Last
Used: 2021-05-18). Apr. 2019. url: https://towardsdatascience.com/auto-
encoder-what-is-it-and-what-is-it-used-for-part-1-3e5c6f017726.
[26] Aurélien Géron. “Hands-on Machine Learning with Scikit-Learn & TensorFlow”.
In: O’Reilly, 2017. Chap. 15.
[27] Unibap AB. SpaceCloud® Framework - manual. (Last Used: 2021-04-12). url:
https://unibap.com/en/support/spacecloud-framework-manual/.

32
[28] Wikipedia. 69-95-99.7 rule - Wikipedia. Last Used: 2021-05-17. 2021. url: https:
//en.wikipedia.org/wiki/68\%E2\%80\%9395\%E2\%80\%9399.7_rule.
[29] Google. Google Dataset Search Engine. Last Used: 2021-06-09. url: https://
datasetsearch.research.google.com/.
[30] CEDA Archive. ESA Sea Level Climate Change Initiative (Sea_Level_cci): Times
series gridded Sea Level Anomalies (SLA), Version 1.1. Last Used: 2021-05-05.
2018. url: https://data.ceda.ac.uk/neodc/esacci/sea_level/data/L4/
MSLA/v1.1.
[31] Janet Williams. Factors That Make a Great Machine Learning Dataset. Last Used:
2021-06-09. 2017. url: https://www.promptcloud.com/blog/what-to-look-
for-in-training-dataset/.
[32] Numenta. The Numenta Anomaly Benchmark (NAB). Last Used: 2021-05-20. 2020.
url: https://github.com/numenta/NAB.
[33] Amazon. Amazon Cloudwatch. Last Used: 2021-05-20. url: https://aws.amazon.
com/cloudwatch/.
[34] Jeff Layton. Getting the Most from Your Cores. Last Used: 2021-05-20. 2021.
url: https://www.admin- magazine.com/HPC/Articles/Determining- CPU-
Utilization.
[35] Susan Li. Time Series of Price Anomaly Detection with LSTM. Last Used: 2021-
05-12. 2020. url: https://towardsdatascience.com/time-series-of-price-
anomaly-detection-with-lstm-11a12ba4f6d9.
[36] Pavithra Vijay. Timeseries anomaly detection using an autoencoder. Last Used:
2021-05-10. 2020. url: https://keras.io/examples/timeseries/timeseries_
anomaly_detection/.
[37] Moa Tennberg and Lovisa Ekeroot. Anomaly detection on satellite timeseries.
(Last Used: 2021-05-20). url: https://github.com/moa-elisabeth/Anomaly-
detection-on-satellite-time-series.
[38] David M W Powers. Evaluation: From Precision, Recall and F-Factor to ROC,
Informedness, Markedness Correlation. Last Used: 2021-06-11. 2007. url: https:
/ / web . archive . org / web / 20191114213255 / https : / / www . flinders . edu .
au / science _ engineering / fms / School - CSEM / publications / tech _ reps -
research_artfcts/TRRA_2007.pdf.
[39] Microsoft. The latest supported Visual C++ downloads. (Last Used: 2021-04-28).
url: https://support.microsoft.com/en-us/topic/the-latest-supported-
visual-c-downloads-2647da03-1eea-4433-9aff-95f26a218cc0.

33
Populärvetenskaplig sammanfattning
Detta projekt går ut på att implementera algoritmer med hjälp av företaget Unibaps ser-
vice SpaceCloud Framework (SCFW). Syftet med algoritmerna är att detektera anomalier
i tidsserier op satelliters hårdvara i rymden. Anomalier är datapunkter som avviker från
resterande i en given datamängd. För att lyckas hitta lämpliga algoritmer för problemet
krävs förståelse av satelliters funktion och uppbyggnad, vilka begränsningar de har och
hur datahanteringen ser ut i dagsläget.

Satelliter består av en kombination av olika sensorer, som tillsammans nyttjas för att
uppfylla satellitens syfte. Dessa sensorer sammankopplas med hjälp av en dator ombord
på satelliten, en on-board computer (OBC). Den utvalda OBCn är konstruerad med
hårdvara som ska tåla rymdens miljö i form av bland annat kosmisk strålning och ex-
trema temperaturer. OBC kan med hjälp av mjukvara hantera den insamlade datan från
sensorerna och analysera kommandon mottagna från jorden. Unibap har konstruerat iX5
som är en OBC som anses vara möjlig att applicera algoritmer för anomalitet detektion på.

I dagsläget sker datahanteringen mellan satelliter och jorden med hjälp av rymdstationer
på marken (Satellite Ground Stations, SGS). Företag kan hyra in sig under en tid på
dessa för att antingen ta emot eller skicka data, men måste själva analysera datan som
de fått från satelliterna. Denna process är både tidsmässigt och ekonomiskt krävande.
Därmed är det aktuellt att undersöka ifall anomali detektion kan ske direkt på satelliter-
nas hårdvara för att antingen självmant kunna fatta ett beslut om nödvändigt, eller i
tidigt skede varna företaget om anomalier vid kontakt med en SGS.

I projektet implementerades algoritmerna Convolutional Neural Network (CNN) autoen-

coder och Long Short-Term Memory (LSTM) Neural Network autoencoder för anomali
detektion. Dessa användes för att försöka förutspå anomalier i datamängder berörande
dels temperatur hos en maskin på en satellit och dels satellitens CPU (Central Processing
Unit). För anomali detektion direkt på studenternas datorer med tillgänglig hårdvara
för jorden lyckades båda implementationerna hitta anomalier, men presterade olika bra
på de olika datamängderna. Den stora skillnaden är att implementationen för CNN var
både mindre till storlek och klarade träningen på kortare tid än vad LSTM gjorde. Dessa
faktorer är önskvärda för att få en implementation att fungera på begränsad hårdvara
(som hos satelliter) och snabbt detektera eventuella anomalier.

Med tidsbrist och bristande dokumentation berörande applikationsbyggandet för SCFW

lyckades inte implementationerna testas på Unibaps hårdvara. Det är däremot troligt att
någon med mer erfarenhet inom hanteringen av ett Linux-distribuerat operativ system
än vad studenterna har, kunna bygga en fungerande applikation för CNN på ix5. Det
finns därmed möjlighet att fortsätta arbetet med anomali detektion direkt på satelliters
hårdvara tillsammans med SCFW, med goda förutsättningar för att lyckas om man
har tid och erfarenhet inom Linux-kommandon. För att uppfylla projektets mål är det
därmed SCFW applikationen som saknas.

34