This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

Article

Improved Target Signal Source Tracking and

Extraction Method Based on Outdoor Visible
Light Communication Using an Improved Particle
Filter Algorithm Based on Cam-Shift Algorithm
Zhipeng Liu1,⟊, Weipeng Guan2,⟊,*, Shangsheng Wen3
1. School of Electronic and Information Engineering, South China University of Technology,
Guangzhou, 510640, China
2. School of Automation Science and Engineering, South China University of Technology, Guangzhou
510640, China
3. School of Materials Science and Engineering, South China University of Technology, Guangzhou,
510640, China
* Correspondence: [email protected](W.G.)
⟊ These authors contributed equally to this work, Zhipeng Liu and Weipeng Guan are co-first authors of the
article.

Abstract: An improved particle filter algorithm based on Cam-Shift algorithm applied to

outdoor visible light communication (outdoor-VLC) is presented in this paper. In the
outdoor-VLC system, accurately and completely tracking and extracting the target signal
source Light Emitting Diode (LED) area is the premise for realizing communication.
However, few existing studies pay attention to it. In the dynamic outdoor environment,
there will be a lot of different environmental interferences (such as background
interference, similar object interference, etc.), which greatly increases the difficulty of
tracking and extracting the target signal source LED area. Therefore, in this paper, an
improved tracking algorithm is proposed to solve the problem of how to track and extract
the target signal source LED area accurately, stably and in real time in the outdoor-VLC
system under various environmental interferences, so as to increase the feasibility of
practical application of VLC system in outdoor scenes. This improved algorithm combines
the particle filter algorithm and Cam-Shift algorithm originally. Experimental results show
that the proposed algorithm has good accuracy, robustness and real-time performance
under the environment of multiple interference factors. Accordingly, the proposed
algorithm can be applied to the outdoor-VLC system with various environmental
interferences, and can realize the actual first step of communication in VLC system based
on image sensor well, laying a foundation for feature extraction, data transmission and
other subsequent steps.

Keywords: outdoor visible light communication (outdoor-VLC); complementary metal-

oxide-semiconductor (CMOS) sensor; premise of communication; improve particle filter
algorithm; Cam-Shift algorithm; robustness; in real time; non-linear and non-gaussian
environment

1. Introduction
Visible light communication (VLC) is a wireless communication method which

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

applies the visible light as a carrier for the data. Many applications based on VLC have
been created, such as indoor communication [1,2], indoor positioning [3,4], intelligent
public illumination [5], intelligent transport systems [6,7], etc. But, in the field of VLC, most
of the studies focuses on indoor applications, and there are few studies on outdoor
applications [8]. The VLC system is mainly consisted by a transmitter and a receiver. The
transmitter modulates the Light Emitting Diode (LED) produced light. The receiver
receives the modulated light beam to demodulate and extract the data information.
Currently, there are two ordinary receivers used in VLC: one is based on a photodiode (PD)
and the other is based on an image sensor. A VLC system based on PD receives the light
intensity of different LEDs to realize communication so that it costs less and has a fast
response time. However, there is a serious flaw in PD that it is greatly affected by the
environment, especially the ambient light [9], which makes it difficult for the PD to be used
in an outdoor environment. In our previous work [10-14], we have completed a series of
research projects on PD-based VLC. And it can be found that when the interference of the
direction of the light beam, the background light or the reflection of surrounding objects
occurs on PD, the PD-based communication would be greatly affected and even fail. The
other receiver, the image sensor can directly capture images to obtain the angle and
distance of the light source and other information. It can also separate the signals from
different light sources so that it is difficult to be affected by the ambient light. According
to the characteristics of PD and image sensor above, the image sensor performs better in
outdoor-VLC application field.
A complete and practical image-sensor-based outdoor-VLC system contains many
components, including signal source information modulation, signal source tracking and
extraction, signal source image area features extraction and demodulation, image
processing and denoising, etc. These components are of great research value and have
great room for improvement. As far as we know, the main direction of the existing studies
on image-sensor-based outdoor-VLC is to increase the transmission speed, reduce the BER
(bit error rate), and solve other normal communication problems. In most existing image-
sensor-based VLC studies, such as in references [15-17], transmitter (signal source) is fixed
physically, receiver (camera) is fixed physically, or received image background has already
been cleared. After adjusting the exposure time of the camera, only the signal source LED
is left in the image and the information can be demodulated directly from these images.
However, in an image-sensor-based VLC, different interferences, such as reflective objects,
are left in the image, especially in the dynamic outdoor scenes. That is to say, the premise
of communication in an image-sensor-based VLC system with different environmental
interferences is to track and extract the target signal source LED accurately and completely
from the received images. Yet, it is generally ignored in the existing studies. Moreover, a
CMOS sensor is used as the receiver in this paper due to the numerous attractive
advantages of complementary metal-oxide-semiconductor (CMOS) sensors [18].
In addition, the field of VLC-based visible light positioning (VLP) is most closely
related to the studies of tracking algorithm. However, there are few studies consider the
accuracy, robustness and real-time ability at the same time. In reference [19], the proposed
VLP method takes the accuracy and real-time performance of system into account. Its
positioning accuracy was 7.5cm and its maximum allowable movement speed of the

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

positioning terminal was 18km/h. However, the robustness is ignored. As it is based on

pixel intensity detection, the communication would even fail when the blur effect and
shielding effect occurred. In reference [20], MiniMax filter, which is used to estimate and
predict the trajectory of terminal (receiver), can only support low-speed motion. Besides,
it is only theoretically simulated. Reference [21] proposed a probability-based algorithm
for tracking the signal source LED in the case of motion blur, which is similar to our
previous work [3]. However, in reference [21], there was no actual experimental result. So
it is only theoretically workable. None of these works takes environmental interference
into consideration, which is the main obstacle to the application of the outdoor-VLC
system. In our previous work [22], we considered the problem of environmental
interference and proposed using particle filter algorithm to track the target signal source
LED. The algorithm has good robustness and can track and extract target signal sources
under different interferences. However, it cannot obtain good accuracy and good real-time
performance at the same time because the particle filter algorithm has a large amount of
calculation. In fact, video target tracking algorithm provides a feasible solution for tracking
and detecting the LED signal area. It can estimate the position of the LED area in the
subsequent image based on the information of the existing LED signal area, thereby
narrowing the search range and improving the real-time performance of the positioning.
Additionally, it has good robustness and good accuracy. In our previous work [3,22], we
have studied the application of video target tracking algorithm in VLP. Inspired by these
studies, we innovatively apply video target tracking algorithm to outdoor-VLC to solve
the problem of tracking and extracting target signal LED source in this paper.
Motivated by important and urgent problems, which are often overlooked, in this
paper, we propose an improved particle filter algorithm based on the Cam-Shift algorithm.
The particle filter algorithm is a classical method to solve non-linear filtering problem. And
it is widely used in target tracking field because of its non-linear and non-gaussian
characteristics. It predicts the next state by extracting a large number of samples (also
called particles) near the target and calculating and comparing their weight. The particle
filter algorithm can improve the tracking robustness because it is appropriate for non-
linear and non-gaussian environment. However, the particle filter algorithm’s
shortcoming is that it has a large amount of calculation, which means it cannot obtain good
accuracy and good real-time performance at the same time. In order to make up for it, the
algorithm proposed in this paper combines the Cam-Shift algorithm originally. The Cam-
Shift algorithm converts the received image into the color probability distribution map
through the color histogram model of the signal source LED, and adjusts the position and
size of the search window adaptively according to the results obtained from the previous
frame. The proposed algorithm uses Cam-Shift algorithm to drift the particles, calculate
the weight of each particle, resample and output the predicted target position after
extracting numerous particles near the target. And it can be used to track and extract the
target signal source LED of VLC system accurately, stably and in real time under different
environment interferences, which improve the feasibility of outdoor-VLC and fill the gap
in this research field. All in all, this paper has three main contribution. First, it solves the
problem of how to track and detect the target signal LED source accurately and stably in
real time, which is the premise of realizing outdoor-VLC. Second, due to the advantages

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

of video target tracking algorithm, we innovatively apply it to outdoor-VLC. Third, the

proposed improved algorithm combines the particle filter algorithm and Cam-Shift
algorithm originally. The rest of this paper is organized as follows. Detailed theories of
outdoor-VLC and the proposed algorithm are introduced in Section 2. In Section 3,
experimental setup and result analysis are presented. Finally, this paper is summarized in
section 4.
2. Theory
2.1. Outdoor Visible Light Communication
Figure 1 describes the geometric model of the outdoor-VLC system based on a CMOS
sensor. The transmitter modulates the data by a certain method and transmits the
information by the modulated LEDs’ light. The optical signal is transmitted through the
air to the receiver. Then the CMOS sensor captures the image after adjusting its exposure
time. After that, the target signal source LED region is extracted by the receiver. Finally,
the receiver extracts the LED region’s features and demodulates the data with a
corresponding method. In this way, data transmission can be realized.

Figure 1. the geometric model of the outdoor-VLC system based on a CMOS sensor
The proposed outdoor-VLC system uses a CMOS sensor as a receiver to achieve data
demodulation, whose working principle is shown in our previous work [3]. According to
our previous work [3], “the exposure and data readout are performed row by row, the data
of one row read out immediately when the exposure of this row is finished. This is known
as the rolling shutter mechanism. By the rolling shutter mechanism of CMOS sensor,
turning on and off the LED light during a period of exposure would result in bright and

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

dark stripes on the image captured by CMOS sensor,” and the schematic diagrams of the
rolling shutter of the CMOS sensor is shown in Figure 2. Moreover, in our previous work
[16, 18, 23], the detailed processing of modulation and demodulation have been described,
and it will not be discussed in this paper. For readers interested in or having questions
about detailed modulation and demodulation based on CMOS sensors, please refer to our
previous report [16, 18, 23].

Figure 2. Schematic diagrams of the rolling shutter of the CMOS sensor.

Additionally, in the practical application of outdoor-VLC based on CMOS sensor,
there are three optical channel models:(1) transmitter is fixed and receiver is moving; (2)
the receiver is fixed and the transmitter is moving; (3) both the transmitter and the receiver
are moving. Obviously, these three models can explain almost all practical application
scenarios. Besides, communication between two vehicles can correspond to all three
channel models and it is shown in Figure 1. In practice, the motion state of the transmitter
and receiver is random, so the tracking algorithm should be applicable to all three optical
channel models. In fact, under different circumstances, how to track and extract the target
signal source LED area can have different choices. When the receiver is fixed and the
transmitter is moving, the background modeling method can be used to track and extract
the signal source area, which can have a good performance even under serious background
interference. However, when the receiver is moving, corresponding to optical channel
model (2) and (3) mentioned above, the background modeling or frame difference method
may not be applicable, because the background of the received image is always changing.
Therefore, considering the computational complexity and making the algorithm
applicable to all optical channel models, VLC system requires a more general tracking
algorithm. Consequently, we propose an improved particle filter algorithm based on cam-
shift algorithm for tracking and extracting the target signal source LED area in this paper.
In the following sections, we will introduce the basic theory and detailed process of the
proposed algorithm, and elucidate the superior performance of the algorithm.

2.2. Improved Particle Filter Algorithm Based on Cam-Shift Algorithm

The particle filter algorithm can improve the tracking robustness because it is
appropriate for non-linear and non-gaussian environment. In order to make up for the
particle filter algorithm’s shortcoming, which is large computation, and improve real-time

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

performance of tracking, the proposed improved algorithm in this paper combines the
Cam-Shift algorithm. In order to explain the improved algorithm better, the traditional
particle filter algorithm and the traditional Cam-Shift algorithm are introduced first.
2.2.1. Traditional Particle Filter Algorithm
The particle filter algorithm is a well-known method to solve the nonlinear filtering
problem. Assume that the state space model of a nonlinear dynamic system is
𝑥𝑘 = 𝑓𝑘 (𝑥𝑘−1 |𝑣𝑘−1 ) (1)

𝑧𝑘 = ℎ𝑘 (𝑥𝑘 |𝑢𝑘 ) (2)

Where 𝑥𝑘 is the state of the system at time 𝑘 , 𝑧𝑘 is the observation vector at time 𝑘 ,
𝑓𝑘 : 𝐴́𝑛𝑥 × 𝐴́𝑛𝑣 → 𝐴́𝑛𝑥 is the state transfer function of the system, ℎ𝑘 : 𝐴́𝑛𝑥 × 𝐴́𝑛𝑢 → 𝐴́𝑛𝑧 is the
system measurement function, and 𝑣𝑘 and 𝑢𝑘 are the process noise and observed noise
of the system, respectively. Assume that the system process is an 𝑚-order Markov process.
According to the interpretation of the nonlinear filtering problem, our goal is to recursively
estimate the posterior probability density 𝑝(𝑥0:𝑘 |𝑧1:𝑘 ) of nonlinear system states based on
the observations with noise. 𝑥0:𝑘 = {𝑥0 , 𝑥1 , ⋯ , 𝑥𝑘 } represents the sequence of states
generated by the system at time 𝑘, and 𝑧1:𝑘 = {𝑧1 , 𝑧2 , ⋯ , 𝑧𝑘 } represents the sequence of
observations.
Base on 𝑚-order Markov assumption, the recursive update formula of the posterior
probability density function 𝑝(𝑥0:𝑘 |𝑧1:𝑘 ) is derived as follows:
𝑝(𝑧𝑘 , 𝑥0:𝑘 , 𝑧1:𝑘−1 )
𝑝(𝑥0:𝑘 |𝑧1:𝑘 ) = (3)
𝑝(𝑧𝑘 , 𝑧1:𝑘−1 )

𝑝(𝑧𝑘 |𝑥0:𝑘 , 𝑧1:𝑘−1 )𝑝(𝑥0:𝑘 , 𝑧1:𝑘−1 )

= (4)
𝑝(𝑧𝑘 , 𝑧1:𝑘−1 )

𝑝(𝑧𝑘 |𝑥0:𝑘 , 𝑧1:𝑘−1 )𝑝(𝑥0:𝑘 |𝑧1:𝑘−1 )

= (5)
𝑝(𝑧𝑘 |𝑧1:𝑘−1 )

𝑝(𝑧𝑘 |𝑥0:𝑘 , 𝑧1:𝑘−1 )𝑝(𝑥𝑘 , 𝑥0:𝑘−1 |𝑧1:𝑘−1 )

= (6)
𝑝(𝑧𝑘 |𝑧1:𝑘−1 )

𝑝(𝑧𝑘 |𝑥0:𝑘 , 𝑧1:𝑘−1 )𝑝(𝑥𝑘 |𝑥0:𝑘−1 , 𝑧1:𝑘−1 )𝑝(𝑥0:𝑘−1 |𝑧1:𝑘−1 )

= (7)
𝑝(𝑧𝑘 |𝑧1:𝑘−1 )

∝ 𝑝(𝑧𝑘 |𝑥0:𝑘 , 𝑧1:𝑘−1 )𝑝(𝑥𝑘 |𝑥0:𝑘−1 , 𝑧1:𝑘−1 )𝑝(𝑥0:𝑘−1 |𝑧1:𝑘−1 ) (8)

= 𝑝(𝑧𝑘 |𝑥𝑘 )𝑝(𝑥𝑘 |𝑥𝑘−𝑚:𝑘−1 , 𝑧1:𝑘−1 )𝑝(𝑥0:𝑘−1 |𝑧1:𝑘−1 ) (9)

The basic idea of the particle filter algorithm is to construct a sample-based posterior
probability density function. The set {𝑥𝑖0:𝑘 , 𝜔𝑖𝑘 }𝑁 𝑖=1 composed of 𝑁 particles is used to
represent the system posterior probability density function 𝑝(𝑥0:𝑘 |𝑧1:𝑘 ) ,where
{𝑥𝑖0:𝑘 , 𝜔𝑖𝑘 }𝑁𝑖=1 is the sample (particle) set, extracted from the state space of the posterior
probability distribution. The weight of each sample particle is {𝜔𝑘𝑖 , i = 1, ⋯ , N} ,and
satisfies ∑𝑖 𝜔𝑖𝑘 = 1. According to this set of weighted particles, the posterior probability
density of time 𝑘 can be approximated as:

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

𝑝(𝑥0:𝑘 |𝑧1:𝑘 ) ≈ ∑ 𝜔𝑘𝑖 𝛿(𝑥0:𝑘 − 𝑥0:𝑘

𝑖
) (10)
𝑖=1

According to this approximation, complex integral operations can be converted into

summation operations. For example, the expected value of the function of interest
g(𝑥0:𝑘 ) is solved as follows:

E(𝑔(𝑥0:𝑘 )) = ∫ 𝑔(𝑥0:𝑘 ) 𝑝(𝑥0:𝑘 |𝑧1:𝑘 )𝑑𝑥0:𝑘 (11)

Its approximate solution based on the sample is:

𝑁

E(𝑔(𝑥0:𝑘 )) = ∑ 𝜔𝑘𝑖 𝑔(𝑥0:𝑘

𝑖
) (12)
𝑖=1

In practical applications, it is very difficult to extract valid samples directly from the
posterior probability distribution. Therefore, how to effectively extract the samples of the
posterior distribution is the key to reduce the statistical estimation variance and improve
the filtering performance of the particle filter algorithm.
Introducing the importance sampling method can improve the sampling efficiency. In
this method, after using an importance sampling density q(𝑥0:𝑘 |𝑧1:𝑘 ) , equation (11) can
be written as follows:

E(𝑔(𝒙0:𝑘 )) = ∫ 𝑔(𝑥0:𝑘 ) 𝑝(𝑥0:𝑘 |𝑧1:𝑘 )𝑑𝑥0:𝑘

𝑝(𝑥0:𝑘 |𝑧1:𝑘 )
= ∫ 𝑔(𝑥0:𝑘 ) 𝑞(𝑥0:𝑘 |𝑧1:𝑘 )𝑑𝒙0:𝑘
𝑞(𝑥0:𝑘 |𝑧1:𝑘 )

= ∫ 𝑔(𝑥0:𝑘 ) 𝜔∗ (𝑥0:𝑘 )𝑞(𝑥0:𝑘 |𝑧1:𝑘 )𝑑𝑥0:𝑘

= 𝐸𝑞(∙) [𝑔(𝑥0:𝑘 )𝜔∗ (𝑥0:𝑘 )] (13)

𝑝(𝑥 |𝑧 𝑖)
In the above calculation, 𝜔∗ (𝑥0:𝑘 ) = 𝑞(𝑥0:𝑘 |𝑧1:𝑘 ) . Extract 𝑁 sample particles {𝑥0:𝑘 ,𝑖 =
0:𝑘 1:𝑘

0, ⋯ , 𝑁} independently from importance sampling density, and then we can obtain the
following approximation:
𝑁
𝑖
E(𝑔(𝑥0:𝑘 )) = ∑ 𝑔(𝑥0:𝑘 ̃𝑘𝑖
)𝜔 (14)
𝑖=1

In equation (12), ̃𝑘𝑖

𝜔 ̃𝑘𝑖 = 𝜔𝑘𝑖 / ∑𝑗 𝜔𝑘𝑖 , and 𝜔𝑘𝑖 is calculated
is the normalized weight, 𝜔
according to the following formula:
𝑖
𝑝(𝑥0:𝑘 |𝑧1:𝑘 )
𝜔𝑘𝑖 = ω(𝑥0:𝑘
𝑖
)∝ 𝑖
(15)
𝑞(𝑥0:𝑘 |𝑧1:𝑘 )

If the set of particles approximated by the posterior probability density

𝑝(𝑥0:𝑘−1 |𝑧1:𝑘−1 ) at time 𝑘-1 has been obtained, the next step is to approximate the posterior
density 𝑝(𝑥0:𝑘 |𝑧1:𝑘 ) at time 𝑘 with the new set of particles.
To get a recursive calculation method, we can decompose the importance density
function into the following form:

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

𝑖
𝑞(𝑥0:𝑘 |𝑧1:𝑘 ) = 𝑞(𝑥𝑘 |𝑥0:𝑘−1 , 𝑧1:𝑘 )𝑞(𝑥0:𝑘−1 |𝑧1:𝑘 ) (16)

Then, by adding the new particle set 𝑥𝑘𝑖 ~𝑞(𝑥0:𝑘 |𝑥0:𝑘−1 , 𝑧1:𝑘 ) obtained from the
𝑖
importance sampling density to the known particle set 𝑥0:𝑘−1 ~𝑞(𝑥0:𝑘−1 |𝑧0:𝑘−1 ) , a new
𝑖
particle set 𝑥0:𝑘 ~𝑞(𝑥0:𝑘 |𝑧1:𝑘 ) is obtained.
According to the 𝑚-order Markov assumption, equation (15) can be written as
equation (17):
𝑞(𝑥0:𝑘 |𝑧1:𝑘 ) = 𝑞(𝑥𝑘 |𝑥𝑘−𝑚:𝑘−1 , 𝑧𝑘 )𝑞(𝑥0:𝑘−1 |𝑧1:𝑘−1 ) (17)

Substituting (9) and (17) into equation (15),

𝑝(𝑧𝑘 |𝑥𝑘𝑖 )𝑝(𝑥𝑘𝑖 |𝑥𝑘−𝑚:𝑘−1
𝑖 𝑖
)𝑝(𝑥0:𝑘−1 |𝑧1:𝑘−1 )
𝜔𝑘𝑖 = (18)
𝑞(𝑥𝑘𝑖 |𝑥𝑘−𝑚:𝑘−1
𝑖 𝑖
, 𝑧𝑘 )𝑞(𝑥0:𝑘−1 |𝑧1:𝑘−1 )

𝑖
𝑝(𝑧𝑘 |𝑥𝑘𝑖 )𝑝(𝑥𝑘𝑖 |𝑥𝑘−𝑚:𝑘−1
𝑖
)
= 𝜔𝑘−1 (19)
𝑞(𝑥𝑘𝑖 |𝑥𝑘−𝑚:𝑘−1 , 𝑧𝑘 )
In the above calculation, 𝑝(𝑧𝑘 |𝑥𝑘𝑖 ) is a likelihood function, 𝑝(𝑥𝑘𝑖 |𝑥𝑘−𝑚:𝑘−1
𝑖
) is a
𝑖
probability transfer density function, and 𝑞(𝑥𝑘 |𝑥𝑘−𝑚:𝑘−1 , 𝑧𝑘 ) is an importance sampling
density. From this formula, it can be known that the weight of the updated particle can be
recursively calculated by selecting an appropriate importance sampling density 𝑞(∙). And
the posterior filter density 𝑝(𝑥𝑘−𝑚+1:𝑘 |𝑧1:𝑘 ) can be approximated as:
𝑁

𝑝(𝑥𝑘−𝑚+1:𝑘 |𝑧1:𝑘 ) ≈ ∑ 𝜔𝑘𝑖 𝛿(𝑥𝑘−𝑚+1 − 𝑥𝑘−𝑚+1:𝑘

𝑖
) (20)
𝑖=1

According to the above analysis, the basic 𝑚-order particle filtering algorithm’s major
steps are as follows:
(1) Extract N particles from the importance sampling density function:
𝑥𝑖𝑘 ~𝑞(𝑥𝑘 |𝑥𝑖𝑘−𝑚:𝑘−1 , 𝑧𝑘 ) (21)

And get a new particle collection 𝑥𝑖𝑘 ~(𝑥𝑖0:𝑘−1 , 𝑥𝑖𝑘 ), 𝑖 = 1, ⋯ , 𝑁.

(2) Calculate the weights of the N particles using equation (19).
(3) Normalize the weights:
𝑗
̃𝑘𝑖 = 𝜔𝑘𝑖 ⁄∑ 𝜔𝑘 , 𝑖 = 1, ⋯ , 𝑁
𝜔 (22)
𝑗
(4) 𝑘=𝑘+1, and then return to step 1.
However, the common problem of the basic particle filtering algorithm described
above is the degradation phenomenon. Since the variance of particle weight increases with
time, degradation is unavoidable. After several iterations, except for a very small number
of particles, the weight of other particles is small enough to be negligible. Degradation
means that if it continues to iterate, a large amount of computing resources is consumed
in dealing with those insignificant particles. It not only causes waste of resources, but also
affects the final estimation results. In order to reduce the impact of degradation, this paper
chooses the particle filter algorithm using the resampling method. The resampling method
is to discard those particles with smaller weight and multiply the particles with larger
weight to produce a plurality of particles with equal weight, which can effectively reduce
the effect of degradation.

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

2.2.2. Traditional Cam-Shift Algorithm

The Cam-Shift algorithm is a target tracking algorithm, which mainly achieves the
purpose of tracking through the color information of moving objects in the video image. It
uses the target's color histogram model to convert the image into a color probability
distribution map, initializes the size and position of a search window, and adaptively
adjusts the position and size of the search window based on the results obtained in the
previous frame to locate the center position of the target in the current image. It includes
three parts: calculating color projection (back projection), Mean-Shift iteration, and Cam-
Shift tracking.
2.2.2.1 Back Projection
The RGB (red, green and blue) color space is sensitive to the changes in illumination
brightness. Therefore, the image is firstly converted from RGB space to HSV space to
reduce the effect of this change on the tracking effect. In the HSV color space, H, S, and V
represent hue, saturation, and value, respectively. Secondly, the H component is used as a
histogram, which represents the probability of occurrence of different H component values
or the number of pixels in the histogram. Finally, the color probability distribution map is
produced by replacing the value of each pixel in the image with the probability pair in
which the color appears. This process is called back projection.
2.2.2.2. Mean-Shift Iteration
The Mean-Shift iteration is a robust non-parametric method for climbing density
gradients to find the target by iteratively finding the extreme value of the probability
distribution. The algorithm process is as follows:
(1) Select the search window in the color probability distribution map.
(2) Calculate the zero moment 𝑀00 , first moments 𝑀01, 𝑀10, and the center of mass
(𝑥𝑐 , 𝑦𝑐 ):
𝑀00 = ∑ ∑ 𝐼(𝑥, 𝑦) (23)
𝑥 𝑦

𝑀01 = ∑ ∑ 𝑦𝐼(𝑥, 𝑦) (24)

𝑥 𝑦

𝑀10 = ∑ ∑ 𝑥𝐼(𝑥, 𝑦) (25)

𝑥 𝑦
𝑀10 𝑀01
𝑥𝑐 = ; 𝑦𝑐 = (26)
𝑀00 𝑀00
(3) Move the center of the search window to the center of mass.
(4) Exit the program when the moving distance of the search window is less than a given
threshold or reaches the maximum number of iterations; otherwise, return to step 2.
Specifically, the Cam-Shift algorithm adjusts the search window with the orientation
information, which is obtained using a Mean-Shift iteration and determines the size
of the search window in the next frame.
2.2.2.3. Cam-Shift Tracking
The key point of the Cam-Shift algorithm is that by using it, the size of the search
window can be adjusted. And it can keep tracking when the size of the target area changes.
Specifically, the Cam-Shift algorithm adjusts the search window according to the

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

orientation information, which is obtained using a Mean-Shift iteration and determines the
size of the search window in the next frame. And the direction information of the target is
reflected by the second moments 𝑀11 , 𝑀20 and 𝑀02 of the search window. 𝑀11 , 𝑀20
and 𝑀02 are calculated by using the following formulas:
𝑀11 = ∑ ∑ 𝐼(𝑥, 𝑦)𝑥𝑦 (27)
𝑥 𝑦

𝑀20 = ∑ ∑ 𝐼(𝑥, 𝑦) 𝑥 2 (28)

𝑥 𝑦

𝑀02 = ∑ ∑ 𝐼(𝑥, 𝑦) 𝑦 2 (29)

𝑥 𝑦
The length and width of the next frame search window are calculated in closed form
as follows. Define the intermediate variables a, b and c:
𝑀20
a= − 𝑥𝑐2 (30)
𝑀00
𝑀11
b = 2( − 𝑥𝑐 𝑦𝑐 ) (31)
𝑀00
𝑀02
c= − 𝑦𝑐2 (32)
𝑀00
The direction angle, length, and width are calculated by:
1 𝑏
θ = tan−1 (33)
2 𝑎−𝑐

(𝑎 + 𝑐) + √𝑏2 + (𝑎 − 𝑐)2
𝑙=√ (34)
2

(𝑎 + 𝑐) − √𝑏2 + (𝑎 − 𝑐)2
𝑤=√ (35)
2
Therefore, the initial size of the search window for the next frame is:
𝑤𝑖𝑑𝑡ℎ = 𝑚𝑎𝑥(𝑙 cos 𝜃 , 𝑤 sin 𝜃) (36)

ℎ𝑒𝑖𝑔ℎ𝑡 = 𝑚𝑎𝑥(𝑙 sin 𝜃 , 𝑤 cos 𝜃) (37)

2.2.3. Improved Algorithm

The traditional Cam-Shift algorithm implements tracking a specific target based on a
color histogram of the target area, and can adaptively change the size of the search window
to accurately track the target. Although the traditional Cam-Shift algorithm performs well
in a simple background, it is easy to lose the target when the background becomes
complicated. This is because under a more complex background, there are many external
factors that affect the color characteristics of the target, such as interference of similar
colors, occlusion, deformation, and so on. If the traditional Cam-Shift algorithm is used in
these situations, it will lead to the wrong location of the target, and eventually the target
tracking fails.
The particle filter algorithm is a well-known method for solving the nonlinear filtering
problem. It predicts the next state by extracting numerous samples near the target and
calculating and comparing their weights. In outdoor-VLC systems with different

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

interference, target signal source LED area tracking may be affected by various factors.
Therefore, the proposed algorithm combines the traditional Cam-Shift and particle
filtering algorithms using the resampling method. The improved algorithm not only
shortens the search path of the Cam-Shift algorithm, but also reduces the influence of
external factors and improves the robustness of the algorithm.
A detailed process of the proposed tracking algorithm is shown in Figure 3. First,
select the target source signal LED area as the tracking target and initialize the search
window. At the same time, a particle set is randomly distributed near the target initial state.
Second, update the current time particle set to obtain the particle set at the next moment
through the set state transition model. Third, use Cam-Shift algorithm to make all the
particles in the particle set drift to the target position. Fourth, determine whether the
drifted result returned by Cam-Shift algorithm is empty or overflow. The reason for this
step is that the similarity between the particle area and the target signal source LED area
is used as the particle weight in this paper, while the function called compareHist that
calculates the similarity in OpenCV cannot support the input parameter to be empty or
overflow. Therefore, when the result returned by the cam-shift algorithm is judged to be
empty or overflow, we use the particle set of the previous frame as the particle set of the
current frame. Fifth, calculate the weight of each particle based on the similarity between
the current image of each particle position and the target image and then initialize the
weight. Fifth, resample based on the weights of each particle. Finally, the average position
of the particles in the resampled particle set is used as the tracking result.

Figure 3. Detailed procession of the proposed tracking algorithm

3. Experimental Setup and Result Analysis

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

3.1. Experimental Setup

The experimental system is shown in Figure 4, consisting of two parts: the transmitter
and the receiver. The transmitter contains direct current (DC) voltage sources, a drive
circuit board, a LED, and a STM32(STMicroelectronics32) development board. The receiver
contains an industry camera, a ROS (Robot Operating System) robot, and a computer (Dell
Inspiron 5557, Windows 10, 4G RAM, Intel (R) Core (TM) i5-6200U CPU @ 2.4GHz). DC
voltage sources supply power to the LED and boards. The LED emits a modulated optical
signal to the receiver. The STM32 development board generates modulation information.
The drive circuit board drives the whole transmitter circuit system. The industrial camera
which is used to receive modulation information from the transmitter is mounted on the
ROS robot which is used to simulate the motion of the vehicle. In order to track the LED
signal source in real time, the computer uses the proposed tracking algorithm to process
the received image from the industrial camera. The open source computer vision library
(OpenCV, which was founded by Intel in 1999 and is now supported by Willow Garage.
And version OpenCV 3.4.0 is used in this paper which was released on December 23, 2017)
is used to complete processing process with C++ language. The specifications of industrial
cameras, LED, ROS robot, STM32 development board and driver circuit board are shown
in Table 1.
Table 1. Parameters of the experimental system.
Camera Specifications
Model MV-U300
Spectral Response Range/nm 400~1030
Resolution 800×600
Frame Rate/FPS 46
Dynamic Range/dB >61
Signal-to-noise Ratio/dB 43

Pixel(H×V) 2048×1536
Pixel Size/μm² 3.2×3.2
Time of Exposure/ms 0.0556-683.8
Sensitivity 1.0V/lux-sec 550nm
Optical Filter 650nm Low Pass Optical
Filter
Type of Shutter Electronic Rolling Shutter
Acquisition Mode Successive and Soft Trigger
Working Temperatures/°C 0-50
Support Multiple Visual Software OpenCV, LabView
Support Multiple Systems Vista, Win7, Win8, Win10
ROS Robot Specifications
Size (L×W×H)/mm3 138×178×141
Weight (+ SBC + Battery + Sensors)/kg 1
SBC (Single Board Computer) Raspberry Pi 3 Model B
32-bit ARM Cortex®
MCU -M7 with FPU (216
MHz, 462 DMIPS)
3.3V / 800mA
Power connectors 5V / 4A
12V / 1A
Input: 100-240V, AC
Power adapter (SMPS) 50/60 Hz, 1.5A @max
Output: 12V DC, 5A

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

Actuator Dynamixel XL460-

W250
PC connection USB
STM32 Development Specifications
CPU STM32F407ZGT6, LQFP144
FLASH/SRAM 1024K / 192K
Power supply port 5V / 3.3V
Drive Circuit Board Specifications
Drive chip DD311
Drive current/A 0.1
Drive voltage/V 28
LED Specifications
Diameter of the LED/mm 150
Power of the LED/W 6
The half-power angles of LED/deg(𝜑1/2 ) 60
The video sequence is recorded using the industrial camera described above to reflect
the results of the proposed algorithm better in different environments in the application of
CMOS-based VLC systems. By the way, the camera’s frame rate is 46 FPS and its resolution
is 800*600. The performance of the camera is not very outstanding, which can be achieved
by many mobile phones with high performance at present. In the experiment, the three
channel models which have been introduced in 2.1 are actually identical when applying
our algorithm because the proposed algorithm does not involve background modeling
which includes some method, such as background subtraction or a frame difference
method. The transmitter simulates a traffic light and the receiver, which is an ROS robot
with a computer with an industrial camera, simulates a moving vehicle. Different
interferences were added to the video recording, including occlusion, background
interference and similar object interference, to test the performance of the proposed
algorithm.

STM32 LED
Development Board
DC Voltage
Sources
Drive
Circuits

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

Personal Computer

Industry Camera

ROS Robot

Figure 4. The experimental system configuration.

3.2. Result and Analysis

3.2.1. Accuracy Performance
In order to maximize the accuracy performance of the proposed tracking algorithm,
101 consecutive frames (6th through 106th) video sequences (with gray LED sources) were
selected for processing. The total tracking error can be defined as:
2
𝑇𝑜𝑡𝑎𝑙 𝑡𝑟𝑎𝑐𝑘𝑖𝑛𝑔 𝑒𝑟𝑟𝑜𝑟 = √(𝑥 − 𝑥𝑟 )2 + (𝑦 − 𝑦𝑟 )2 (38)

𝑒𝑟𝑟𝑜𝑟 𝑜𝑓 𝑥(𝑦) = |𝑣𝑎𝑙𝑢𝑒𝑡𝑟𝑎𝑐𝑘𝑖𝑛𝑔 𝑎𝑙𝑔𝑜𝑟𝑖𝑡ℎ𝑚 − 𝑣𝑎𝑙𝑢𝑒𝑎𝑐𝑡𝑢𝑎𝑙 | (39)

In the equation, (𝑥, 𝑦) represents the calculated coordinates of the target signal
source LED of the proposed algorithm, and (𝑥𝑟 , 𝑦𝑟 ) represents the actual coordinate of the
target LED. According to the proportional relationship between the image coordinates and
the actual coordinates, the actual distance can be obtained by calculation. The tracking
results are shown in Figure 5, where the blue dots represent the actual locations and the
red dots are the coordinates calculated by the proposed algorithm. Figure 5.a is the total
tracking result, and Figure 5.b, 5.c is respectively the tracking result in the x and y axis
directions. As can be seen from the results in Figure 5, the performance of the tracking is
very good and the error is small. To further illustrate the results and show the accuracy of
the proposed algorithm in this round, the total tracking error, the error of x-coordinate and
the error of y-coordinate are shown in Figure 6. In Figure 6.a, the blue line represents the
total tracking error. In Figure 6.b, the pink line represents the error of the x-coordinate and
the error of the y-coordinate is represented by the green line in Figure 6.c. From the
experimental results, the average total tracking error of the proposed algorithm is 0.4475
cm and the maximum total tracking error is 1.4391 cm. Furthermore, in the x-coordinate
direction, the average error is 0.2598 cm, and the maximum error is 1.2495 cm. And in the
y coordinate direction the average error is 0.3022 cm, and the maximum error is 1.0710 cm.
There are many reasons for such tracking errors, such as image noise, camera shake,

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

brightness vibration of the target signal source LED, etc., which affect the back-projection
image, causing the color probability distribution map to be disturbed. However, in the
practical application of outdoor-VLC systems, the tracking error of a few centimeters or
even 1 cm is obviously acceptable. Therefore, it can be concluded that the proposed
algorithm has good tracking accuracy and can be used to track and extract the target signal
source LED area accurately, which is the premise of realizing communication in a CMOS-
based VLC system.

(a)

(b)

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

(c)
Figure 5. The calculated coordinate value and the actual coordinate value of the target
signal source LED: (a) tracking result of the target signal source LED, (b) Tracking result
of the x coordinates, and (c) tracking result of the y coordinates.

(a)

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

(b)

(c)
Figure 6. Tracking error of the target signal source LED: (a) total tracking error, (b) tracking
error of x coordinate, and (c) tracking error of y coordinate.

The cumulative distribution function (CDF) is the integral of the probability density
function, used to represent the probability of the occurrence of an event. Specifically, the
y-coordinate of any point on the CDF curve indicates the probability that the variable is
less than or equal to the corresponding x-coordinate of that point. The CDF is defined as:
𝐹𝑋 (𝑥0 ) = 𝑃(𝑋 ≤ 𝑥0 ) (40)

If variable is discrete, the CDF can be expressed as:

𝐹𝑋 (𝑥0 ) = 𝑃(𝑋 ≤ 𝑥0 ) = ∑ 𝑃𝑖 (41)
𝑥𝑖 ≤𝑥0
where 𝑝𝑖 is the probability of event 𝑥𝑖 .

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

The CDF is a commonly used error analysis index, such as in reference [3,4,24]. When
tracking or positioning accuracy is evaluated, the x coordinates represent the error and the
y coordinates represent the probability. In addition, when the ordinate value is fixed, the
smaller the x value is, the smaller the error value will be, and the more accurate the
algorithm will be. Usually, the y value is 90% or 95% as the criterion.
Therefore, we use the CDF to further elaborate the accuracy performance of the
proposed algorithm in this paper. As shown in Figure 7, when the probability was 95%,
total tracking error, tracking error of x coordinate and tracking error of y coordinate were
1.104 cm, 0.8925 cm and 0.714 cm, respectively. In other words, 95% of the total tracking
error was less than 1.2 centimeters. Consequently, it can be concluded that the proposed
algorithm showed good accuracy for tracking the target signal source LED area.

Figure 7. The cumulative distribution function (CDF) of the tracking error.

3.2.2. Robustness Performance
The proposed tracking algorithm needs to have good stability, which is reflected by
robustness performance. And also, it is the premise of realizing communication in a
CMOS-based VLC system. As for a CMOS-based VLC, background influence, similar
object influence and shielding effect are the main obstacles for a tracking algorithm. It
greatly influences the success of tracking and extracting the target signal source LED area,
and even leads to communication failure. Therefore, the above four influencing factors
were added to our experimental environment to test the stability of the tracking algorithm
in this paper. The video recorded by the green LED signal source is used to display the
experimental results of this part and the detailed results are as follows.
Firstly, when the target signal source LED area is complete and no environmental
interference occurs, the tracking target is selected and the search window is initialized. At
the same time, the set of particles is initialized. It can be seen from Figure 8 that the
proposed algorithm tracks the target signal source LED with a small error and can
completely extract the target area. Obviously, the proposed algorithm can achieve high
precision tracking in this situation.

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

Figure 8. Target signal source LED selection and tracking algorithm initializing without
environmental interference.
Secondly, when the target signal source LED is covered, the search window shrinks
as the effective color feature information decreases. If the traditional Cam-Shift algorithm
is used, the search window will shrink to an area of 0 and stay in place. When the target
source reappears in future frames, the search window is unable to track it and the tracking
is failed. When the algorithm proposed in this paper is used, numerous particles are
updated near the target area, and the similarity between these particles and the target area
is used to predict the target position of the next frame. Therefore, as shown in Figure 9, the
proposed algorithm still has a good tracking performance in this situation.

(a) (b)

(c) (d)

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

Figure 9. Tracking performance of the proposed algorithm when occlusion occurs:

(a)frame number 50, (b)frame number 55, (c)frame number 64, and (d)frame number 70

Thirdly, because of the reflection of sunlight, some outdoor scenes appear in the image.
Such images are usually so bright that they cannot be removed by adjusting the exposure
of industrial cameras. Therefore, it is necessary to examine this kind of interference. The
traditional Cam-Shift algorithm would mistakenly regard outdoor scenes as a target and
track it, which leads to the tracking failure. As shown in Figure 10, the proposed algorithm
combines particle filter and regards the particle which has the highest similarity with the
target signal source area from the particles set drifted by Cam-Shift algorithm as the
tracking result so that this algorithm can stabilize the search window near the target signal
source area and achieve a better tracking effect.

(a) (b)
Figure 10. Tracking performance of the proposed algorithm under background
interference: (a)frame number 179, (b)frame number 189
Fourth, similar target interference occurred. Our experiment uses a light green LED
which is the same shape as the target signal source LED and makes it enter the field of
view of the industry camera. When the target source LED approaches the LED with similar
color, the traditional Cam-Shift algorithm would mistakenly regard the LED as a tracking
target, resulting in tracking failure. But the proposed algorithm uses the particle which has
the highest similarity with the target signal source area from the particles set drifted by
Cam-Shift algorithm as the tracking result. As can be seen from Figure 11, the algorithm
can accurately track and extract the target signal source LED area and has good robustness

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

when the similar target interference occurs, laying foundations for communication.

(a) (b)
Figure 11. Tracking performance of the proposed algorithm under similar color
interference: (a)frame number 236, (b)frame number 263

3.2.3. Real-Time Performance

Since tracking and extracting the target signal source LED area is the first step in
communication, the processing time required for tracking and the complexity of the
algorithm largely determine the real-time performance of CMOS-based VLC system
communication. Therefore, the real-time performance of the algorithm largely determines
whether the tracking algorithm can be used in an actual VLC system. We experimented
and took the processing time t of 400 consecutive frames. And then we calculated that the
improved particle filter algorithm based on Cam-Shift algorithm proposed in this paper
has an average time of 69.5474ms for processing each frame.
It is difficult for us to judge the real-time quality only by looking at such a simple
number. So, we compare other algorithms to help us judge it. According to our previous
work [3] in the field of tracking and positioning, for instance, which uses optical flow
detection and a Bayesian forecast algorithm for target tracking, its average running time
was 162 ms. Obviously, the algorithm proposed in this paper is much faster than it is while
the accuracy of the two algorithms is basically the same. Although there is a small gap of
the two algorithms, it is acceptable, especially in the outdoor-VLC system. Besides, in our
another work [25], which use the Cam-Shift algorithm with a Kalman filter, its average
running time was 42 ms. It is a little faster than the proposed algorithm. However, in
reference [25], the proposed algorithm’s average total tracking error is 0.85 cm, which is
greater than our proposed algorithm in this paper. Therefore, the algorithm is suitable for
real-time communication of VLC. Moreover, compared with other works, the algorithm
proposed in this paper also has a good real-time performance. References [26-28] can
provide different degrees of positioning accuracy, which can reach decimeter-level, within
ten centimeters, or even millimeter-level. However, these works only focus on the accuracy
of positioning and ignore real-time performance. In Reference [19], the computational time
of each frame is only about 0.023 s for single-luminaire, which is faster than our processing
time. But the method proposed in Reference [19] can only provide accuracy of 7.5 cm,
which is greater than the accuracy of the proposed algorithm. Furthermore, Reference [21],
whose focus was similar to ours, proposed a probability-based tracking algorithm and
performed well in tracking error. However, it lacked a practical experiment and real-time

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

analysis, meaning that it is not enough to justify its feasibility in practical application
where real-time ability is required. In summary, we can conclude that the improved
particle filter algorithm based on Cam-Shift algorithm we proposed has good real-time
performance.

4. Conclusions
In this paper, an improved tracking algorithm, which combines particle filtering
algorithm and Cam-Shift algorithm, is proposed for visible light communication to make
up for the lack of existing VLC research on tracking and extracting target signal source
LED area. Extracting the target signal source LED area is the premise of realizing visible
light communication in practical application. Yet, most of the existing VLC studies ignore
it or assume that there is no background interference in the received images. In fact, in the
dynamic outdoor environment, there is interference from other luminous or reflective
objects in the received image. Therefore, we propose an improved algorithm, combining
the advantages of particle filter algorithm and Cam-Shift algorithm, which can accurately,
stably and in real time track and extract the target signal source LED area under different
environmental interferences.
Experimental results showed that the tracking error of the proposed algorithm was
1.335 cm, which means that the proposed algorithm had a high accuracy. Besides, the
average processing time of each frame was 69.5474ms, which means the proposed
algorithm had good real-time performance. More importantly, in the actual situation of an
image-sensor-based outdoor-VLC system considered in this paper, the proposed
improved algorithm showed good robustness and can solve the problem of tracking and
extracting the target signal source LED area in the images under different interferences,
which is the premise for the outdoor-VLC system to achieve communication in a dynamic
scenario. Consequently, the proposed improved algorithm can be applied to the image-
sensor-based VLC system, including the communication in a motion state, and can be
applied to indoor and outdoor applications both, which improves the feasibility of
outdoor-VLC application and fills the gap in this research field.

Funding: This research was funded by College Students Innovation and Entrepreneurial
Training Program number S201910561205, National Undergraduate Innovative and
Entrepreneurial Training Program number 201510561003, 201610561065, 201610561068,
201710561006, 201710561054, 201710561057, 201710561058, 201710561199, 201710561202,
201810561217, 201810561195, 201810561218, 201810561219, 201910561166, 201910561164
Special Funds for the Cultivation of Guangdong College Students’ Scientific and
Technological Innovation(“Climbing Program” Special Funds) grant number
pdjh2017b0040, pdjha0028, pdjh2019b0037 and Guangdong science and technology project
grant number 2017B010114001. , “The One Hundred-Step Ladder Climbing Program”
grant number j2tw201902166, j2tw201902189.

References
1. Hussein, A.T., Alresheedi, M.T., Elmirghani, J.M.H. 20 Gb/s Mobile Indoor Visible
Light Communication System Employing Beam Steering and Computer Generated

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

Holograms J. Lightwave Technol. 2015, 33, 5242-5260.

2. Wu, S; Wang, H.; Youn, C.H. Visible light communications for 5G wireless networking
systems: From fixed to mobile communications. IEEE Netw. 2014, 28, 41-45.
3. Guan, W.; Chen, X.; Huang, M.; Liu, Z.; Wu, X.; Chen, Y. High-Speed Robust Dynamic
Positioning and Tracking Method Based on Visual Visible Light Communication Using
Optical Flow Detection and Bayesian Forecast. IEEE Photonics J. 2018.
4. Cai, Y.; Guan, W.; Wu, Y.; Xie, C.; Chen, Y.; Fang, L. Indoor High Precision Three-
Dimensional Positioning System Based on Visible Light Communication Using Particle
Swarm Optimization. IEEE Photonics J. 2017.
5. Kumar, N. Smart and intelligent energy efficient public illumination system with
ubiquitous communication for smart city. In Proceedings of the IEEE International
Conference on Smart Structures and Systems, Chennai, India, 28–29 March 2013; pp.
152–157.
6. Wang, N.; Liu, C.; Lu, Y.; Shen, J. A Visible Light Communication (VLC) based
Intelligent Transportation System for lorry fleet. In Proceedings of the IEEE
International Conference on Optical Communications and Networks, Mumbai, India,
24–26 February 2017.
7. Siddiqi, K.; Raza, A.D.; Muhammad, S.S. Visible light communication for V2V
intelligent transport system. In Proceedings of the IEEE International Conference on
Broadband Communications for Next Generation Networks and Multimedia
Applications, Graz, Austria, 14–16 September 2016; pp. 1–4.
8. Ndjiongue, A.R.; Ferreira, H.C. An Overview of Outdoor Visible Light
Communications. Trans. Emerg. Telecommun. Technol. 2018.
9. Trong-Hop Do; Myungsik Yoo Potentialities and challenges of VLC based outdoor
positioning. 2015 International Conference on Information Networking (ICOIN). 2015,
2015-.
10. Peng, Q.; Guan, W.; Wu, Y.; Cai, Y.; Xie, C.; Wang, P. Three-dimensional high-precision
indoor positioning strategy using Tabu search based on visible light communication.
Opt. Eng. 2018, 57.
11. Chen, H.; Guan, W.; Li, S.; Wu, Y. Indoor high precision three-dimensional positioning
system based on visible light communication using modified genetic algorithm. Opt.
Commun. 2018, 413, 103–120.
12. Guan, W.; Wu, Y.; Wen, S.; Chen, H.; Yang, C.; Chen, Y.; Zhang, Z. A novel three-
dimensional indoor positioning algorithm design based on visible light
communication. Opt. Commun. 2017, 392, 282–293.
13. Guan, W.; Wu, Y.; Xie, C.; Chen, H.; Cai, Y.; Chen, Y. High-precision approach to
localization scheme of visible light communication based on artificial neural networks
and modified genetic algorithms. Opt. Eng. 2017, 56.
14. Guan, W.; Wu, Y.; Wen, S.; Yang, C.; Chen, H.; Zhang, Z.; Chen, Y. High precision three-
dimensional iterative indoor localization algorithm using code division multiple
access modulation based on visible light communication. Opt. Eng. 2016, 55.
15. Chow, C.W.; Shiu, R.J.; Liu, Y.C.; Liao, X.L.; Lin, K.H.; Wang, Y.C.; Chen, Y.Y. Using
advertisement light-panel and CMOS image sensor with frequency-shift-keying for
visible light communication. Opt. Express 2018, 26, 12530.

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JPHOT.2019.2940773, IEEE
Photonics Journal

16. Xie, C.; Guan, W.; Wu, X.; Fang, L.; Cai, Y. The LED-ID Detection and Recognition
Method based on Visible Light Positioning using Proximity Method. IEEE Photonics J.
2018, 10.
17. Do, T.H.; Yoo, M. Visible light communication based vehicle positioning using LED
street light and rolling shutter CMOS sensors. Opt. Commun. 2018, 407, 112–126.
18. Guan, W.; Wu, Y.; Xie, C.; Fang, L.; Liu, X.; Chen, Y. Performance analysis and
enhancement for visible light communication using CMOS sensors. Opt. Commun.
2018, 410, 531–545.
19. Fang, J.; Yang, Z.; Long, S.; Wu, Z.; Zhao, X.; Liang, F.; Jiang, Z.L.; Chen, Z. High-speed
indoor navigation system based on visible light and mobile phone. IEEE Photon. J.
2017, 9, 2, 8200711
20. Alsalami, F.M. Game theory MiniMax filter design for indoor positioning and tracking
system using visible light communications. In Proceedings of the IEEE International
Conference on Information Communication and Management, Hertfordshire, UK, 29–
31 October 2016.
21. Huynh, P.; Do, T.H.; Yoo, M. A Probability-Based Algorithm Using Image Sensors to
Track the LED in a Vehicle Visible Light Communication System. Sensors 2017, 17, 347.
22. Wu, Y.X.; Guan, W.P.; Zhang, X.J.; Huang, M.X.; Cao, J.Y. Visible light positioning
system based on CMOS image sensor using particle filter tracking and detecting
algorithm. Optics Communications. 2019,444,9-20.
23. Li, J.Y.; Guan, W.P. The Optical Barcode Detection and Recognition Method Based on
Visible Light Communication Using Machine Learning. APPLIED SCIENCES-BASEL.
2018, 8, 12, 2425.
24. Wu, Y.; Liu, X.; Guan, W.; Chen, B.; Chen, X.; Xie, C. High-speed 3D indoor localization
system based on visible light communication using differential evolution algorithm.
Opt. Commun. 2018, 424, 177–189.
25. Huang, M.X.; Guan, W.P.; Fan, Z.B.; Chen Z.H.; Li J.Y.; Chen, B.D. Improved Target
Signal Source Tracking and Extraction Method Based on Outdoor Visible Light
Communication Using a Cam-Shift Algorithm and Kalman Filter. Sensors.
2018,18,12,1-21.
26. Zhang, R.; Zhong, W.-D.; Kian, Q.; Zhang, S. A single LED positioning system based
on circle projection. IEEE Photonics J. 2017, 9, 7905209.
27. Kim, J.-Y.; Yang, S.-H.; Son, Y.-H.; Han, S.-K. High-resolution indoor positioning using
light emitting diode visible light and camera image sensor. IET Optoelectron. 2016, 10,
184–192.
28. Hossen, M.S.; Park, Y.; Kim, K.-D. Performance improvement of indoor positioning
using light-emitting diodes and an image sensor for light-emitting diode
communication. Opt. Eng. 2015, 54, 035108.

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.