entropy

Article
On the Genuine Relevance of the Data-Driven Signal
Decomposition-Based Multiscale Permutation Entropy
Meryem Jabloun *, Philippe Ravier and Olivier Buttelli

Laboratoire Pluridisciplinaire de Recherche en Ingénierie des Systèmes, Mécanique, Énergétique (PRISME),

University of Orleans, 45100 Orleans, France
* Correspondence: [email protected]

Abstract: Ordinal pattern-based approaches have great potential to capture intrinsic structures of
dynamical systems, and therefore, they continue to be developed in various research fields. Among
these, the permutation entropy (PE), defined as the Shannon entropy of ordinal probabilities, is an
attractive time series complexity measure. Several multiscale variants (MPE) have been proposed in
order to bring out hidden structures at different time scales. Multiscaling is achieved by combining
linear or nonlinear preprocessing with PE calculation. However, the impact of such a preprocessing
on the PE values is not fully characterized. In a previous study, we have theoretically decoupled the
contribution of specific signal models to the PE values from that induced by the inner correlations
of linear preprocessing filters. A variety of linear filters such as the autoregressive moving average
(ARMA), Butterworth, and Chebyshev were tested. The current work is an extension to nonlinear
preprocessing and especially to data-driven signal decomposition-based MPE. The empirical mode
decomposition, variational mode decomposition, singular spectrum analysis-based decomposition
and empirical wavelet transform are considered. We identify possible pitfalls in the interpretation
of PE values induced by these nonlinear preprocessing, and hence, we contribute to improving the
PE interpretation. The simulated dataset of representative processes such as white Gaussian noise,
Citation: Jabloun, M.; Ravier, P.;
fractional Gaussian processes, ARMA models and synthetic sEMG signals as well as real-life sEMG
Butelli, O. On the Genuine Relevance signals are tested.
of the Data-Driven Signal
Decomposition-Based Multiscale Keywords: time series; multiscale permutation entropy; data-driven decomposition; nonlinear
Permutation Entropy. Entropy 2022, filtering; electromyography
24, 1343. https://doi.org/
10.3390/e24101343

Academic Editor: Luca Faes

1. Introduction
Received: 31 July 2022
Ordinal pattern-based approaches have recently gained attention because of their
Accepted: 14 September 2022
natural and efficient way to transform a time series into a sequence of symbols with a
Published: 23 September 2022
finite alphabet [1–37]. This ordinal pattern sequence is simply obtained by comparing
Publisher’s Note: MDPI stays neutral the values of a finite number of neighboring samples and ranking them. This operation
with regard to jurisdictional claims in is threshold-free and does not require knowledge of the data range. Moreover, from a
published maps and institutional affil- practical point of view, the obtained ordinal patterns are robust to linear perturbations and
iations. moderate noise levels [1,2]. The probability distribution of ordinal patterns (OP) implicitly
captures information about the temporal structure of the time series and thus allows
the extraction of several promising ordinal pattern-based indicators. These indicators
are useful in a growing number of applications including biomedical signal and image
Copyright: © 2022 by the authors.
processing [6,10,11,15,31,33,38], fault bearing diagnosis [30,39–44], financial time series
Licensee MDPI, Basel, Switzerland.
analysis [7] and engineering physics [18,35,45–51].
This article is an open access article
Among the OP based-indicators, the permutation entropy (PE) was the first to be
distributed under the terms and
conditions of the Creative Commons
proposed [1]. It was introduced as a complexity measure of weakly stationary time series
Attribution (CC BY) license (https://
and was defined as the Shannon entropy [52] of the OP probability distribution. By con-
creativecommons.org/licenses/by/ struction, the PE benefits from the robustness and simplicity of the OP theory and takes
4.0/). into account information on the temporal organization of time series contrary to classical

Entropy 2022, 24, 1343. https://doi.org/10.3390/e24101343 https://www.mdpi.com/journal/entropy

Entropy 2022, 24, 1343 2 of 17

entropies. The PE is theoretically related to the well-known measure of dynamical systems

complexity, the Kolmogorov–Sinai entropy [9]. A large number of time series processing
tasks were performed using the PE tool [16,26,38,47].
As the PE has the ability to capture hidden dynamics of time series, various extensions
of the PE, called multiscale PE (MPE), have been proposed to explore in depth the internal
structure of signals at different time scales. Among the MPE techniques, we cite coarse-
graining MPE [53], composite MPE [54], refined composite MPE [12], downspamling PE
(DPE) [55], composite DPE [31], refined composite DPE [31], hierarchical PE [56], and
data-driven decomposition-based MPE [38–51,57–59]. The main idea of these PE-based
techniques is to combine PE with a preprocessing step of the signal of interest in order to
reduce the number of patterns related to noise. These MPE techniques can be roughly split
into two categories regarding the preprocessing step. The first category applies a linear fil-
tering [12,31,53–55], whereas the second category uses a nonlinear processing [38–51,57–59].
Despite the widely proven usefulness of these techniques, there is still a need to understand
the impact of the preprocessing step on the MPE values and how this impact can signifi-
cantly alter the interpretation of MPE results. In [3], a study investigated the shortcomings
of coarse-grained MPE, which can be viewed as a moving average filtering procedure
followed by an M-factor downsampling operator. At high M-scales, this MPE suffers from
spurious patterns reflecting spectral aliasing as Shannon’s theorem is no longer respected.
In [27], we have theoretically characterized the effect of a linear filtering preprocessing
on the PE. By using the Wiener–Khinchin theorem, the linear filter’s intrinsic PE and its
contribution to the PE of the original signal were theoretically dissociated. This result was
validated by means of simulated signals (white Gaussian noise and ARMA processes) sub-
jected to a variety of linear filters such as the moving average, Butterworth, and Chebyshev
type I filters. However, it is important to note that since the PE is a biased statistic [60], we
have considered sufficiently long signals in order to neglect the bias effect. Therefore, the
work presented in [27] provided an appropriate framework for characterizing the linear
filter impact on the PE, which helped improve the interpretation of the PE by identifying
possible artefact information introduced by this preprocessing step.
In the current paper, we aim at characterizing the data-driven decomposition-based
MPE as a new challenge in the understanding of the impact of nonlinear preprocess-
ing in the PE. To that end, four decomposition methods are considered: the empiri-
cal mode decomposition (EMD) [39,40,43,48,57], the variational mode decomposition
(VMD) [38,41,45–47,49,58], the singular spectrum signal analysis-based decomposition
(SSA) [50,59] and the empirical wavelet transform [42,44,51]. We test a variety of simulated
signals including white Gaussian noise, fractional Gaussian noise, ARMA processes, simu-
lated surface electromyography signals (sEMG) and multicomponent sinusoidal signals.
We find that the multiscaling effect of these methods should be interpreted with respect to
the mean frequency of the components obtained and not as a function of the rank of these
obtained components. The observed decrease of the PE of each component is not related to
changes on the PE but only to the spectrum shift of those components. This may invalidate
some interpretation of the results of such tools, especially when applied to real data.
The paper is organized as follows. Section 2 recalls the theoretical background of PE.
Section 3 elaborates on the data-driven decomposition-based MPE and the results that
bring light on the MPE are presented. Discussion is provided in Section 4. Section 5 draws
conclusions and arguably.

2. Permutation Entropy—Review of Existing Theory

This section briefly recalls the concepts and tools useful for understanding the PE
and MPE.

2.1. Permutation Entropy Definition

Let us consider N samples of a weakly stationary and uniformly sampled time series
xt with t = 0, 1, . . . , N − 1. A symbol sequence, called pattern Π = r1 r2 . . . rd of dimension
Entropy 2022, 24, 1343 3 of 17

(order) d, can be obtained by sorting in ascending order every d consecutive samples

xt , xt+1 , xt+2 , . . . , xt+d−1 . This can be represented by a permutation function v(.) of the set
{1, 2, . . . , d} defined by: v(1) = r1 , v(2) = r2 , v(3) = r3 , . . ., and v(d) = rd . The maximum
number of permutation of this set is d!, and so is the maximal number of distinct patterns.
For example, for d = 3, if xt < xt+1 < xt+2 , we obtain the pattern Π = 123 with r1 = 1,
r2 = 2, r3 = 3 and v(.) is the identity, whereas if xt+1 < xt < xt+2 , we obtain the pattern
Π = 213 where r1 = 2, r2 = 1 and r3 = 3.
The probability that the pattern Π of type r1 r2 . . . rd occurs can be estimated [2],
subject to N >>> d! by

#t |{ xt , xt+1 , . . . , xt+d−1 } of type r1 r2 . . . rd

 
pr1 r2 ... rd = , (1)
N−d−1

where # denotes the cardinal number.

The normalized PE, which was introduced in [1] as a natural measure of the complexity
of the signal xt , was defined as the Shannon entropy applied to the pattern distribution (1),
and it can be estimated as:
∑ pΠ log( pΠ )
H=− . (2)
log(d!)
The PE reaches its minimum value 0 for a monotone waveform, whereas it reaches its
maximum value of 1 for a uniform pattern distribution (pΠi = 16 for d = 3 and pΠi = 24 1

for d = 4). The PE robustness to ectopic or aberrant values and to moderate noise level is
well established [60–62].

2.2. Permutation Entropy of Known Signals

Several studies have been conducted on the properties and behavior of PE on known
signals [1,2,7,55,60,62–64]. For a comparison purpose, we here recall the results obtained
on stationary real Gaussian processes and pure tone signals.

2.2.1. Stationary Real Gaussian Processes

In [2], the PE (2) of a stationary real Gaussian process xt of zero mean and unit
variance was evaluated using the exact expressions of the probability of ordinal patterns
Π = r1 r2 . . . rd provided that 2 ≤ d ≤ 4. We propose to merge all these expressions into
one written as follows:
!
1 1 1 αi,j
pr1 r2 ...rd = d−1 + d−2
2
∑
2 π i,j∈{1,2,...,d−1}
arcsin
2 β i,j
(3)
i< j
 
where αi,j = ∑ (−1)`+h ρ xx v(i + `) − v( j + h) (4)
`,h∈{0,1}
    
β2i,j = 1 − ρ xx v(i + 1) − v(i ) 1 − ρ xx v( j + 1) − v( j) , (5)

and ρ xx (τ ) denotes the normalized auto-correlation function (ACF) of the real time series
xt defined as:
E [ xt xt+τ ]
ρ xx (τ ) = ρ xxτ = (6)
E[ xt2 ]
for τ = − N + 1, − N + 2, . . . , N − 1. Note that ρ xxτ is symmetric and that both (4) and (5)
only depend on the values of ρ xx1 , ρ xx2 and ρ xx3 .
Thus, knowing these three point values of the ACF of any stationary Gaussian signal
is sufficient to access to its corresponding theoretical PE value.
Among the stationary Gaussian processes considered in [2,60,62], there are
Entropy 2022, 24, 1343 4 of 17

• Autoregressive AR(1) processes defined by xt = a xt−1 + et where the parameter

| a| < 1, and et is an i.i.d. zero-mean and unit variance Gaussian noise. Its ACF is
given by

1 if τ = 0,
ρ xxτ = (7)
a|τ | otherwise.
• AR(2) processes defined by xt = axt−1 + bxt−2 + et with parameters a and b verifying
| a + b| < 1, | a − b| < 1 and |b| < 1. Its ACF is calculated through

 1 if τ = 0,
a
ρ xxτ = 1− b if τ = 1, (8)
aρ xxτ −1 + bρ xxτ−2 otherwise.


q
• Moving average processes MA(q) defined by xt = ∑`=0 a` et−` with a0 = 1, and et is a
unitary centered Gaussian noise. Its ACF is evaluated using the following expression:
 q
 ∑`=τ a` a`−τ
q if |τ | ≤ q,
ρ xxτ = ∑ a2 (9)
 0 `=0 ` otherwise.

• Fractional Gaussian noise (fGn) is defined as the increment of fractional Brownian

motion (fBm) and is a zero-mean stationary process. Its ACF is given by
 
1 2h 2h 2h
ρ xxτ = | τ + 1| − 2| τ | + | τ − 1| . (10)
2

where h ∈]0, 1[is referred to as the Hurst exponent [55]. Recall that for h = 0.5, the fGn
reduces to a white Gaussian noise with equal probabilities for the d! order patterns of
dimension d and hence a permutation entropy H = 1.
• Synthetic sEMG signals simulated as Gaussian processes, whose power spectral
density (PSD) is modeled according to [65] by:

f h4 ν2
PSD (ν) = k
2 (11)
ν2 + f l2 ν2 + f h2

where k is a normalization constant and f l and f h are the normalized low and high
cut-off frequencies of the sEMG PSD model. Its ACF expression was derived in [27]:

f h2 + f l2 −2π f |τ |
 
2 fh −2π f l |τ | 2 2 −2π f h |τ |
ρ xxτ = − f l e + e h + π ( f h − f l ) | τ | e . (12)
( f h − f l )2 2 fh

• Correlated Gaussian processes defined as the output of a linear filter whose input is a
white Gaussian noise. Such processes can be encountered in studies limited to one
specific or useful bandwidth as it is the case in [66]. The ACF of such processes is
directly linked to the ACF of the used linear filter as described in [27]. For example,
an ideal band-pass filter of a normalized bandwidth is equal to BW = [νl , ν f ], whose
transfer function is defined as:

1 if ν ∈ [νl , νh ],
HF (ν) = , (13)
0 otherwise

with 0 ≤ νl < νh < 0.5 and ν is the normalized frequency, which leads to an ACF of
the filter output signal as follows:
   
ρ xxτ = cos π (νh + νl )τ sinc π (νh − νl )τ . (14)
Entropy 2022, 24, 1343 5 of 17

Note that by setting νl = 0, we have the ACF of the output signal of ideal low-pass
filters. Another example, a Gaussian filter whose transfer function is defined by
 
HF (ν) = exp −2πν2 σ2 (15)

leads to an ACF of the following filter output signal

−τ 2
 
ρ xxτ = exp . (16)
2σ2

2.2.2. Sinusoidal Signals

Let us consider a pure tone signal xt = cos(2πν0 t) with a normalized frequency
f
ν0 = F0s and a large number of samples in order to avoid bias estimation of the PE [60]. The
PE value can be approximated by the following expressions for Fs >>> 2 f 0 :

2νp log(νp ) + (1 − 2νp ) log( 12 − νp )

H=− for d = 3, (17)
log(3!)
4νp log(νp ) + (1 − 4νp ) log( 12 − 2νp )
H=− for d = 4, (18)
log(4!)
ν0
with νp = . (19)
1 − ν0 (d − 1)

When the sampling frequency Fs tends to be very high in comparison to the signal frequency
f 0 , i.e., νp ≈ 0, the PE H reaches the theoretical PE of a continuous time series H = 0.387 and
H = 0.218 for d = 3 and d = 4, respectively [67]. It turns out that the PE value H = 0.387
is also that obtained for d = 3 when the sampling frequency is equal to the Nyquist rate
Fs = 2 f 0 . Indeed, in this last case, only two equiprobable patterns are available.
Now, let us consider a sum of two sinusoidal signals xt = cos(2πν0 t) + ar cos(2π f r ν0 t +
φ), where parameters ar and f r are the amplitude and frequency ratios of the two consid-
ered components, and φ is a uniform random phase identically distributed on [−π, π ].
The average of this signal PE, < H > φ , with respect to φ is a function of the parameters ar
and f r as shown in Figure 1a,b, where f 0 = 1 Hz and Fs = 20 Hz. The dark area corresponds
to a PE close to that of the dominant component cos(2πν0 t) and equal to 0.581 and 0.441
for d = 3 and d = 4, respectively.

1 1
0.6
0.9 0.9
0.46

0.8 0.58 0.8

0.44
0.7 0.7
0.56 0.42
0.6 0.6
fr

fr

0.5 0.5
0.54 0.4

0.4 0.4
0.38
0.3 0.52 0.3

0.2 0.2 0.36

0.5
0.1 0.1
0.34

0 0
-2 -1.5 -1 -0.5 0 0.5 1 1.5 2 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2

log a r log a r

(a) (b)
Figure 1. PE values of the sum of two sinusoidal signals xt = cos(2πν0 t) + ar cos(2π f r ν0 t + φ)
as a function of the parameters ar and f r . The signal is noise free, and the sampling frequency is
Fs = 20 Hz. (a) d = 3; (b) d = 4.

Note that this area edge and the PE values are sensitive to the chosen sampling
frequency [67] and to the signal-to-noise ratio (SNR), as illustrated in Figure 2a,b.
Entropy 2022, 24, 1343 6 of 17

1 1

0.9 0.6 0.9 0.6

0.8 0.8
0.58 0.58
0.7 0.7

0.6 0.56 0.6 0.56

fr

fr
0.5 0.5

0.4 0.54 0.4 0.54

0.3 0.3
0.52 0.52
0.2 0.2

0.1 0.5 0.1 0.5

0 0
-2 -1.5 -1 -0.5 0 0.5 1 1.5 2 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2

log a r log a r

(a) (b)
Figure 2. PE values of the sum of two sinusoids (17) as a function of the parameters ar and f r : PE
values are dependent of the sampling frequency and the SNR. (a) d = 3, Fs = 40 Hz and SNR=∞;
(b) d = 3, Fs = 20 Hz and SNR = 20 dB.

2.3. Multiscale Permutation Entropy

Multiscale variants of PE (MPE) have been proposed in the literature in order to
identify hidden structures in long memory processes or/and over-sampled signals. The
main idea of MPE methods is to apply a preprocessing step to the signal of interest in order
to reduce the number of noise-related patterns before proceeding to PE estimation.

2.3.1. Linear Preprocessing-Based MPE

Many of the MPE are based on linear preprocessing [12,30,31,53–55] such as lag oper-
ator and/or linear filters and/or downsampling. Firstly, it should be noted that at high
scales, the downsampling step of this MPE class may induce spurious patterns reflecting
spectral aliasing as Shannon’s theorem is no longer respected [3]. Secondly, the constraint
N
M >> d!, where N is the signal sample number and M is the scale, must be verified in order
to ensure a reduced bias and variance of the estimate of the pattern distribution (13) [60,62].
Finally, the linear filtering has an impact on the PE, which has been theoretically and exper-
imentally characterized in [27]. More precisely, in [27], the contribution of the linear filter to
the PE was theoretically dissociated from that of the original signal model. Therefore, the
work presented in [27] helped improve the interpretation of the PE by identifying possible
artefact information introduced by this linear preprocessing step.

2.3.2. Nonlinear Preprocessing-Based MPE

Other MPE methods that use a nonlinear preprocessing have been proposed in
order to overcome the shortcomings of the aforementioned linear filtering-based MPE
class [38–51,57–59].
Among these nonlinear preprocessing-based MPE, we cited four data-driven signal
decomposition-based MPE that are of great interest in many engineering fields: the em-
pirical mode decomposition (EMD) [39,40,43,48,57], the variational mode decomposition
(VMD) [38,41,45–47,49,58], the singular spectrum signal analysis-based decomposition
(SSA) [50,59] and the empirical wavelet transform [42,44,51].
The EMD is a well-known data-driven approach that was proposed by Huang et al. [68]
to recursively decompose a signal into a sum of oscillating functions called intrinsic mode
functions (IMF). The empirical scheme used by the EMD is based on a sifting process that
iteratively extracts the envelops of signal maxima and minima and removes their mean
from the signal. An IMF is extracted when a stopping criterion of the sifting process is
reached. The original stopping criterion was defined as the total number of extrema and the
number of zero-crossings differs by at most one. The obtained IMFs should have separated
spectral bands and be well-behaved with the Hilbert transform. The aspect of the EMD
acting as a filter bank has been also studied in depth in [69]. In practice, the EMD is known
Entropy 2022, 24, 1343 7 of 17

to be sensitive to noise, sampling frequency and mode mixing of signal components with
close frequencies. Other considerations or/and modifications of the original EMD have also
been proposed to overcome the EMD limitations [70–72]. In our study, we use the EMD
implementation proposed in [70] where a Cauchy-type stop criterion is considered in order
to relax the original IMF definition and to extract AM-FM modes with physical significance.
The VMD proposed in [73] is a non-recursive signal decomposition method that breaks
down the signal into a small number of narrow-band IMFs well adapted to Hilbert trans-
form. Unlike EMD, the VMD has a consolidated mathematical theory, since the IMFs and
their respective center frequencies are extracted concurrently by optimizing a constrained
variational problem. All IMFs obtained are amplitude and frequency-modulated (AM-
FM) cosine waveforms whose instantaneous frequencies vary slowly in a non-decreasing
pattern. The VMD exhibits interesting performances compared to the EMD, such as the
attenuation of the mixing mode and the improvement of the noise robustness. The VMD
behavior as a filter bank was investigated in [74]. In order to increase the performance of
the VMD, improved versions of the VMD were also proposed [75–77].
The empirical wavelet transform (EWT) proposed in [78] is a data-driven decom-
position method that extracts different oscillatory modes from a signal by designing an
adaptive wavelet filter bank. The upper and lower passband cut-off frequencies of the
wavelet filters are automatically selected as the midpoint of two adjacent local maxima
of the Fourier spectrum of the signal. The extracted modes are narrow time-frequency
sub-bands. As with VMD, the EWT has a theoretical basis. It also combines the advantages
of EMD and the wavelet transform. Further refinements of EWT have also been proposed
in [79–81] to improve the performance of EWT.
Singular spectrum analysis (SSA), introduced in [82] and featuring an unsupervised
classification algorithm, is also a data-driven decomposition method, but it differs in
that its theoretical basis is close to that of the principal component analysis. SSA is able
to identify principal processes and hidden periodicities. The components extracted are
physically significant and correspond either to orthogonal oscillatory components with a
narrow spectral band (periodic or quasi-periodic components), trends or noise. Various
improvements to the original SSA have also been proposed to overcome the redundancy of
the original SSA decomposition and to take into account the possibility of a time-varying
number of components [83,84].
Despite the widely proven usefulness of the permutation entropy combined with these
data-driven decomposition-based techniques, there is still a need to understand the impact
of such a preprocessing on the MPE values and how this impact can significantly alter the
interpretation of the MPE results.

3. Impact of Nonlinear Preprocessing in the PE Values

In this section, we aim to characterize data-driven decomposition-based MPE as a new
challenge in understanding the impact of nonlinear preprocessing in PE. We compare the
four signal decomposition-based methods: EMD, VMD, EWT and SSA.
We present results using simulations of different models, including white noise, frac-
tional Gaussian noise (fGn), AR(q) and MA(q) models, and simulated sEMG signals.
An application to real-life sEMG data is also provided.
For all curves, the markers indicate the actual calculated PE values of extracted
components (IMFs), while the lines are an interpolation of these values.

3.1. Results on White Noise

A total of 100 time series of uncorrelated white Gaussian noise were simulated.
The sampling frequency was set to Fs = 1000 Hz and the number of samples was equal to
N = 1000. At least eight components (IMFs) were extracted from each time series and for
each method. The PE was then calculated for each obtained component. We also propose
Entropy 2022, 24, 1343 8 of 17

to compute the normalized center frequency of each component for comparison purposes.
This can be achieved using a PSD estimation, PSD (ν), of the IMF:
Z
νc = ν PSD (ν) dν. (20)

Figure 3 shows a comparison of the PE obtained from simulations. For all considered
methods, the results are presented in two ways. The first one is the classical one: PE as a
function of the extraction rank of the components. In the second way, PE is presented as a
function of the normalized center frequency of the IMFs.

d=3 d=3

0.9 0.9

0.8 0.8

0.7 EMD 0.7

VMD
EWT

PE
PE

EMD
SSA VMD
0.6 0.6
EWT
SSA

0.5 0.5

0.4 0.4

0 5 10 15 20 25 10-2 10-1 100

Component extraction rank Center frequency

(a) (b)
d=4 d=4
0.9 0.9
0.8 0.8

0.7 0.7

0.6 0.6
EMD
0.5 VMD 0.5
PE
PE

EWT
SSA
0.4 0.4
EMD
VMD
EWT
SSA
0.3 0.3

0 5 10 15 20 25 10-2 10-1 100

Component extraction rank Center frequency

(c) (d)
Figure 3. Impact of nonlinear preprocessing on the MPE values of white Gaussian noise: EMD, VMD,
EWT or SSA decomposition precedes the calculation of PE with a pattern dimension d = 3 and d = 4.
(a,c) PE as a function of the extraction rank of the components in contrast to (b,d) PE as a function of
the center frequency of each extracted mode. The average results of 100 simulated time series are
depicted. At least 8 components are extracted by each method.

As can be seen in Figure 3a,c, the four methods considered seem to act differently on
the PE, but looking at Figure 3b,d, the variation and decrease of the PE with respect to
the center frequency of extracted components seem to be similar for almost all methods.
The results are very close especially for the components extracted at high or moderate
frequencies with VMD and EWT preprocessing methods. The SSA method sticks to a
single spectral band, and the first 25 extracted components all share the same narrow band.
The lowest number of extracted components is obtained by EMD, but it covers a larger
spectral range.
Note that these obtained results remain valid even if the uncorrelated noise is not
Gaussian but uniformly distributed.
Next, to further investigate the impact on the MPE and to obtain an idea of the
reliability of these results, we present a theoretical approximation of the EMD and VMD-
based MPE. To this end, recall that both the EMD and VMD act as a filter bank but in two
Entropy 2022, 24, 1343 9 of 17

distinct ways [69,74]. Figure 4a,d illustrates this filter-bank aspect where the DSPs of the
first eight IMFs extracted from Gaussian noise are depicted. We propose to approximate
their normalized PSD by Gaussian curves:

PSD (ν) ≈ exp(−4π 2 (ν − νc )2 σ2 ). (21)

From this point, each IMF can be viewed as the output of a linear filter whose input is
white Gaussian noise and transfer function is the square root of the expected spectra (21).
An approximation of the normalized ACF is then given by (16), and an approximation
of the theoretical PE of the IMF can be obtained using (3). The theoretical PE obtained
are shown in Figure 4b,e for a pattern dimension d = 3 and in Figure 4c,f for a pattern
dimension d = 4. As shown in these figures, the theoretical and experimental PE appear to
have a close shape, especially at high and moderate center frequencies. At very low center
frequencies, the Gaussian approximation of the PSD IMF may no longer be valid, leading
to a divergence between the experimental and theoretical PE.

d=3 d=4
1 1
1
0.9 0.9
0.9 0.8 0.8

0.8 0.7 0.7

Normalized PSD

0.7 0.6 0.6

0.6 0.5 0.5

PE

PE
0.5
0.4 0.4
0.4
simulated PE
0.3
0.3 Approx. theor. PE 0.3 simulated PE
0.2 Approx. theor. PE
0.1

0 0.2 0.2
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 10-2 10-1 10-2 10-1
Normalized frequency Center frequency Center frequency

(a) EMD (b) EMD (c) EMD

d=3 d=4
1 1
1
0.9 simulated PE 0.9
0.9
Approx. theor. PE
0.8 0.8 0.8 simulated PE
Normalized PSD

0.7 Approx. theor. PE

0.7 0.7
0.6
PE

PE

0.5
0.6 0.6
0.4

0.3
0.5 0.5
0.2

0.1

0 0.4 0.4
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 10-2 10-1 10-2 10-1
Normalized frequency Center frequency Center frequency

(d) VMD (e) VMD (f) VMD

Figure 4. Impact of nonlinear preprocessing on PE values of white Gaussian noise: EMD (or VMD)
precedes the calculation of PEs with a pattern dimension d = 3 and d = 4. (a,d) Estimation of the
IMF PSDs (solid line) and the Gaussian approximation of these PSD (dotted line). (b,e) Average
of PEs of the extracted IMFs based on 100 simulated Gaussian white noise superimposed to their
approximated theoretical PEs computed using (16) and (2) for a pattern dimension d = 3. (c,f) same
as (b,c) but with a pattern dimension d = 4. All calculated PEs are plotted against the average of
center frequencies of each extracted mode.

3.2. Results on Fractional Gaussian Noise

The application of the data-driven decomposition-methods (EMD and VMD) on
fractional Gaussian noise (fGn) is illustrated in Figures 5 and 6. In total, 100 simulated time
series with a sample number N = 1000 were generated for each Hurst exponent.
As can be seen in Figure 5a,c, the two methods considered appear to act differently on
the fGn PE values. In addition, the fGn with the long-range dependence (h = 0.9) has the
lowest IMF PE values.
However, as shown in Figures 5b,d and 6a, the decay and variation of the IMF PE
curves with respect to the IMF center frequencies seem to be very close or even identical
to those of the white Gaussian noise (h = 0.5), especially for IMFs of high and moderate
center frequencies. Furthermore, Figure 6a shows that the shape of the IMF PE curves is
Entropy 2022, 24, 1343 10 of 17

non-sensitive to the Hurst exponent, but the actually calculated PE values that are indicated
by the (o) marker are, however, sensitive to a frequency shift.

d=3 d=3
1
fGn Hurst expon.=0.5
0.9 0.9
fGn Hurst expon.=0.9
0.8 fGn Hurst expon.=0.1 0.8

0.7 0.7

PE
PE

0.6
0.6

0.5
0.5 fGn Hurst expon.=0.5
fGn Hurst expon.=0.9
fGn Hurst expon.=0.1
0.4
0.4
1 2 3 4 5 6 7 8 10-2 10-1
Component extraction rank Center frequency

(a) EMD (b) EMD

d=3 d=3
1

0.95
0.9
0.9
0.8
0.85

0.8 0.7

PE
PE

0.75
0.6

0.7 fGn Hurst expon.=0.5

fGn Hurst expon.=0.9
0.5 fGn Hurst expon.=0.5
0.65 fGn Hurst expon.=0.1
fGn Hurst expon.=0.9
fGn Hurst expon.=0.1
0.6
0.4
1 2 3 4 5 6 7 8 10-2 10-1
Component extraction rank Center frequency

(c) VMD (d) VMD

Figure 5. Impact of the data-driven decomposition on the PE values of white Gaussian noise (-o)
compared to that of fractional Gaussian processes with Hurst exponent 0.1 (diamond) and 0.9 (star):
(a,b) PE of a pattern dimension d = 3 with EMD preprocessing. (c,d) same as (a,b) but with VMD
preprocessing. A total of 100 times series were simulated for each Hurst exponent.

d=3
1 0.5

0.9 0.45

0.4
0.8
0.35
fc of fGn IMFs

0.7 0.3
identity
PE

fGn Hurst=0.1 0.25 fGn Hurst=0.1

fGn Hurst=0.2 fGn Hurst=0.2
0.6
fGn Hurst=0.3 0.2 fGn Hurst=0.3
fGn Hurst=0.4 fGn Hurst=0.4
fGn Hurst=0.5 0.15 fGn Hurst=0.5
fGn Hurst=0.6 fGn Hurst=0.6
0.5
fGn Hurst=0.7 0.1 fGn Hurst=0.7
fGn Hurst=0.8 fGn Hurst=0.8
fGn Hurst=0.9 0.05 fGn Hurst=0.9

0.4 0
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
10-2 10-1
Center frequency fc of AWGN IMFs

(a) (b)
Figure 6. Impact of the EMD preprocessing on the PE values of white Gaussian noise (-o) versus
fractional Gaussian processes: (a) PE of a pattern dimension d = 3 and Hurst exponent varying
from 0.1 to 0.9 in 0.1 steps. (b) Average of center frequencies of fGn IMFs versus average of center
frequencies of AWGN IMFs. A total of 100 times series are simulated for each Hurst exponent.

Therefore, if we model the PE curve of AWGN IMFs by a function H = Fct(νc ) of

the normalized center frequency νc , we can easily obtain the PE of fGN IMFs by scaling:
Entropy 2022, 24, 1343 11 of 17

H f Gn = H (αh νc ), where αh is a slope that depends on the Hurst exponent. These slopes are
actually depicted in Figure 6b where we can see the linear relationship between normalized
center frequencies of the fGn IMFs and those of AWGN IMFs.

3.3. Results on Signal Models

We compute the PE of correlated signals in order to assess the contribution of the
data-driven decomposition method to the original PE of the signal models. We use white
Gaussian noise as a benchmark for comparison. We simulate 100 time series of length
N = 1000 for each of the following models.
• AR(2) process with a = 0.8 and b = 0.15 and whose PE is evaluated by (8) and (3) [27].
• AR(7) processes used in [85] to model the PSD of Heart Rate Variability (HRV) recorded
in the tilt and rest positions. Table III in [85] shows the AR model parameters used for a
sampling rate of Fs =1 Hz and error term variances equal to 404 × 10−6 and 137 × 10−6
for rest and tilt, respectively.
• MA(2) process with parameters a0 = 0.4 and a1 = 0.6 and whose PE is evaluated
by (9) and (3) [27].
• sEMG models: sEMG signals are obtained by filtering white Gaussian noise with
the inverse Fourier transform of the square root of expected spectra (11). Two sets
of parameters ( f l , f h ) are tested: (30 Hz, 60 Hz) and (50 Hz, 150 Hz) for a sampling
frequency Fs = 1024 Hz as proposed in [65]. The PE of such signals is evaluated using
(12) and (3) [27].
We also propose theoretically investigating the impact of the EMD-based MPE on the
contribution of those models by using the same theoretical Gaussian approximation of
IMFs DSP, as detailed in Section 3.1.
The application of EMD-based MPE to the signal models considered above is illus-
trated in Figure 7. The comparison between the PE of AR(2) IMFs and the AWGN IMFs
(see Figure 7e) shows superimposed PE curves, but the actually calculated PE values that
are indicated by the (o) marker are shifted. So, we notice the same PE scaling effect with
respect to the center frequencies of IMFs as in the case of fGn. This remark also holds for
the MA(2) model, as shown in Figure 7d, and for the IMF PE curves of the AR(7) models of
HRV in tilt and rest position, as shown in Figure 7c. The IMF PE of simulated sEMG signals
also exhibits the same behavior (PE scaling with respect to center frequency), as shown
in Figure 7e. However, we notice that this latter figure shows that the highest frequency
extracted components are slightly sensitive to the signal model.
In Figure 7b, we have also added the results of a linear filtering on the PE values
where, unlike EMD-based MPE, there is a gap between the filter intrinsic PE and the AR(2)
signal model filtered at high frequencies.
Finally, we propose to check the impact of the data-driven decomposition on the PE
values of the sum of two sinusoidal signals xt = cos(2πν0 t) + ar cos(2π f r ν0 t + φ) as a
function of the parameters ar and f r.
To this end, we calculate the relative percentage error between the theoretical PE (17)
and the computed PE of pattern dimension d = 3 for the first extracted component. This
error is plotted in Figure 8 as a function of the amplitude and frequency ratios. As can
be seen in Figure 8, the error variation in PE values reflects the ability of the methods
considered (EMD, VMD, EWT and SSA) to separate the components.
Entropy 2022, 24, 1343 12 of 17

d=3 d=3
1 1
simulated PE - AWGN PE - AWGN
0.9 0.9
Approx. th. PE - AWGN PE - AR(2)
0.8 simulated PE - AR(2) 0.8 PE - Butter. filt. AWGN
Approx. th. PE - AR(2) PE - Butter. filt. AR(2)
0.7 0.7

PE

PE
0.6 0.6

0.5 0.5

0.4 0.4

10-2 10-1 10-2 10-1

Center frequency Center frequency

(a) AR(2) versus AWGN—EMD (b) Comparison of MPE

d=3 d=3
1 1

0.9 0.9
simulated PE - AWGN
Approx. th. PE - AWGN
0.8 0.8 simulated PE - MA(2)
Approx. th. PE - MA(2)
0.7 0.7
PE

PE
0.6 0.6

0.5 simulated PE - rest 0.5

Approx. theor. PE - rest

simulated PE - tilt
0.4 Approx. theor. PE - tilt 0.4

10-2 10-1 10-2 10-1

Center frequency Center frequency

(c) AR(7) rest versus tilt—EMD (d) MA(2) versus AWGN—EMD

d=3
1

0.9

0.8

0.7
PE

0.6

0.5 simulated PE - AWGN

Approx. th. PE - AWGN
simulated PE sEMG 30-60 Hz
Approx. th. sEMG 30-60 Hz
0.4
simulated PE sEMG 50-150 Hz
Approx. th. sEMG 50-150 Hz
10-2 10-1
Center frequency

(e) Simulated sEMG (11)—EMD

Figure 7. Impact of EMD decomposition on the PE values of signal models. (a) EMD-based MPE of
AR(2) processes (-∆ ) versus white Gaussian noise(-o): experimental PE (solid line) versus theoretical
PE approximation (dashed line). (b) EMD-based MPEs of both AR(2) model and AWGN superim-
posed to linear preprocessing-based MPE: 9th order Butterworth filter. (c) EMD-based MPE of AR(7)
during rest and tilt positions: EMD-based MPE of AR(2) model (-∆ ) versus white Gaussian noise(-o):
experimental PE (solid line) versus theoretical PE approximation (dashed line). (d) same as (a) but
with MA(2) processes. (e) EMD-based MPE applied to two sEMG models (11).
Entropy 2022, 24, 1343 13 of 17

d=3 d=3 d=3 d=3

1 1 3 1 1
2
0.9 2 0.9 0.9 0.9
2.5 0
0.8 0.8 0.8 0.8 0
0

0.7 0.7 2 0.7 0.7

-5 -2
-2
0.6 0.6 0.6 0.6
1.5
-4
fr

fr

fr

fr
0.5 -4 0.5 0.5 0.5
-10
1
0.4 0.4 0.4 0.4 -6
-6
0.3 0.3 0.3 0.3
0.5 -15
-8
-8
0.2 0.2 0.2 0.2
0
0.1 0.1 0.1 0.1 -10
-10 -20

0 0 0 0
-2 -1.5 -1 -0.5 0 0.5 1 1.5 2 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2
log ar log ar log ar log ar

(a) (b) (c) (d)

Figure 8. Impact of the data-driven decomposition on the PE values of a sum of two sinusoidal signals:
the relative error between theoretical PE (17) in percent and computed PE of pattern dimension d = 3
of the first component obtained after (a) EMD, (b) VMD, (c) EWT and (d) SSA decomposition.

3.4. Results on Real-Life sEMG Signals

We here consider two sEMG signals recorded from the short head of the biceps brachii
during exercises according to the protocol detailed in [86]. The force level was imposed at
20% and at 80%. From Figure 9, we notice that when plotted against the center frequencies
of the IMFs, the EMD or VMD-based MPE is very close to that of white Gaussian noise.
Only the lowest frequency IMFs have discarded PE. The most noticeable effect is again the
frequency shift or the PE scaling effect, as shown in Figure 9c,f.

1 200

0.9
IMF PE of AWGN 0.9 180
IMF PE of sEMG 20% IMF PE of AWGN
0.8 160
0.8 IMF PE of sEMG 80% IMF PE of sEMG 20%
0.7 IMF PE of sEMG 80% 140
0.7
120

Delta fc
0.6
PE
PE

0.6 100
0.5
80

0.5 60 fc 20%
0.4 fc 80%
40

0.4 20

0.3 0
1 2 3 4 5 6 7 8 100 101 102 103 0 100 200 300 400 500 600 700

IMF extraction rank Center frequency fc of AWGN IMF

(a) EMD (b) EMD (c) EMD

1 500
0.95 0.9 450
0.9
0.8 400
0.85
0.7 350
0.8
300
0.75 0.6
fc
PE
PE

250
0.7
0.5
200 fc 20%
0.65
150
fc 80%
0.6 IMF PE of AWGN 0.4 IMF PE of AWGN
100
IMF PE of sEMG 20% IMF PE of sEMG 20%
0.55 IMF PE of sEMG 80% IMF PE of sEMG 80% 50

0.3 0
1 2 3 4 5 6 7 8 100 101 102 103 0 100 200 300 400 500 600 700

IMF extraction rank Center frequency fc of AWGN IMF

(d) VMD (e) VMD (f) VMD

Figure 9. Impact of EMD and VMD decomposition on the PE values of real-life sEMG signals versus
AWGN: (a,c) IMF PE of a pattern dimension d = 3 as a function of IMF extraction rank. (b,d) IMF PE
of a pattern dimension d = 3 as a function of center frequencies of extracted IMFs. Center frequencies
of extracted IMFs from sEMG signals versus those of AWGN.

4. Discussion
The data-driven decomposition-based methods we considered extract components
with different frequency panels. When applied to AWGN or to correlated processes with
different signal models (AR, MA, simulated sEMG), all the considered methods behaved
almost identically to a white Gaussian noise with respect to the center frequencies of the
extracted components. Indeed, the PE values of extracted components are located on the
same curve used as a benchmark whatever the model and for each of the data-driven
decomposition methods used. However, the actually calculated PE values which are of
the same number of extracted IMFs seem to be shifted frequency with respect to those
of AWGN.
Entropy 2022, 24, 1343 14 of 17

Unlike linear preprocessing MPE, it can be concluded that the variation in the data-
driven decomposition-based MPE may mainly reflect the frequency shift (frequency band-
width of extracted components) of components more than a concrete complexity variation
related to the signal model. The obtained results suggest that the PE decay or variation
should not be interpreted from the perspective of rank of the extracted component but
rather from the perspective of their frequency location. These decay and variation are
sensitive to the frequency shift of extracted components.

5. Conclusions
We need to accurately outline the impact of nonlinear preprocessing in the PE and
identify its contribution to PE from the information content of signal models. In this work,
we found that data-driven decomposition-based MPE should be analyzed and interpreted
based on the center frequency of the extracted components rather than their extraction rank.
Unlike the linear filtering-based MPE, the variations of these data-driven decomposition-
based MPE mainly reflect the frequency shift of the bandwidth of each IMF more than the
contribution of the signal model. This work allows us to correctly interpret the PE values
over an arbitrary time series. Future research directions may also involve highlighting the
impact in special cases such as existing persistent forbidden patterns. This is a topic for
future research to explore.

Author Contributions: Conceptualization, methodology, validation, formal analysis, resources,

writing—original draft preparation and visualization, M.J.; visualization and project administration,
P.R. and O.B. All authors read and agreed to the published version of the manuscript.
Funding: This research has been funded by the University of Orléans (France).
Institutional Review Board Statement: The study was conducted according to the guidelines of the
Declaration of Helsinki, and approved by the Institutional Pedagogical and Ethical Committee of
“Institut National de Formation Supérieure en Sciences et Technologies du Sport (INFS/STS) de Dély
Ibrahim, Alger”, Algeria (2004).
Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.
Data Availability Statement: Not applicable.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Bandt, C.; Pompe, B. Permutation Entropy: A Natural Complexity Measure for Time Series. Phys. Rev. Lett. 2002, 88, 174102.
[CrossRef] [PubMed]
2. Bandt, C.; Shiha, F. Order Patterns in Time Series. J. Time Ser. Anal. 2007, 28, 646–665. [CrossRef]
3. Valencia, J.F.; Porta, A.; Vallverdu, M.; Claria, F.; Baranowski, R.; Orlowska-Baranowska, E.; Caminal, P. Refined Multiscale
Entropy: Application to 24-h Holter Recordings of Heart Period Variability in Healthy and Aortic Stenosis Subjects. IEEE Trans.
Biomed. Eng. 2009, 56, 2202–2213. [CrossRef] [PubMed]
4. Xiao-Feng, L.; Yue, W. Fine-grained permutation entropy as a measure of natural complexity for time series. Chin. Phys. B 2009,
18, 2690–2695. [CrossRef]
5. Zunino, L.; Soriano, M.C.; Fischer, I.; Rosso, O.A.; Mirasso, C.R. Permutation-information-theory approach to unveil delay
dynamics from time-series analysis. Phys. Rev. E 2010, 82, 046212. [CrossRef]
6. Liu, Y.; Chon, T.S.; Baek, H.; Do, Y.; Choi, J.H.; Chung, Y.D. Permutation entropy applied to movement behaviors of Drosophila
melanogaster. Mod. Phys. Lett. B 2011, 25, 1133–1142. [CrossRef]
7. Zanin, M.; Zunino, L.; Rosso, O.A.; Papo, D. Permutation Entropy and Its Main Biomedical and Econophysics Applications: A
Review. Entropy 2012, 14, 1553–1577. [CrossRef]
8. Rosso, O.A.; Olivares, F.; Zunino, L.; De Micco, L.; Aquino, A.L.L.; Plastino, A.; Larrondo, H.A. Characterization of chaotic maps
using the permutation Bandt-Pompe probability distribution. Eur. Phys. J. B 2013, 86, 116. [CrossRef]
9. Unakafova, V.; Unakafov, A.; Keller, K. An approach to comparing Kolmogorov-Sinai and permutation entropy. Eur. Phys. J. Spec.
Top. 2013, 222, 353–361. [CrossRef]
10. Amigó, J.M.; Keller, K.; Unakafova, V.A. Ordinal symbolic analysis and its application to biomedical recordings. Philos. Trans. R.
Soc. A Math. Phys. Eng. Sci. 2015, 373, 20140091. [CrossRef]
11. Porta, A.; Bari, V.; Marchi, A.; De Maria, B.; Castiglioni, P.; di Rienzo, M.; Guzzetti, S.; Cividjian, A.; Quintin, L. Limits of
permutation-based entropies in assessing complexity of short heart period variability. Physiol. Meas. 2015, 36, 755–765. [CrossRef]
Entropy 2022, 24, 1343 15 of 17

12. Humeau-Heurtier, A.; Wu, C.W.; Wu, S.D. Refined Composite Multiscale Permutation Entropy to Overcome Multiscale
Permutation Entropy Length Dependence. IEEE Signal Process. Lett. 2015, 22, 2364–2367. [CrossRef]
13. Humeau-Heurtier, A. The Multiscale Entropy Algorithm and Its Variants: A Review. Entropy 2015, 17, 3110–3123. [CrossRef]
14. Quintero-Quiroz, C.; Pigolotti, S.; Torrent, M.C.; Masoller, C. Numerical and experimental study of the effects of noise on the
permutation entropy. New J. Phys. 2015, 17, 93002. [CrossRef]
15. Liu, T.; Yao, W.; Wu, M.; Shi, Z.; Wang, J.; Ning, X. Multiscale permutation entropy analysis of electrocardiogram. Phys. A Stat.
Mech. Appl. 2017, 471, 492–498. [CrossRef]
16. Zunino, L.; Kulp, C.W. Detecting nonlinearity in short and noisy time series using the permutation entropy. Phys. Lett. A 2017,
381, 3627–3635. [CrossRef]
17. Makarkin, S.A.; Starodubov, A.V.; Kalinin, Y.A. Application of permutation entropy method in the analysis of chaotic, noisy, and
chaotic noisy series. Tech. Phys. 2017, 62, 1714–1719. [CrossRef]
18. Toomey, J.P.; Argyris, A.; McMahon, C.; Syvridis, D.; Kane, D.M. Time-Scale Independent Permutation Entropy of a Photonic
Integrated Device. J. Light. Technol. 2017, 35, 88–95. [CrossRef]
19. Watt, S.J.; Politi, A. Permutation entropy revisited. Chaos Solitons Fractals 2019, 120, 95–99. [CrossRef]
20. Bandt, C. Small Order Patterns in Big Time Series: A Practical Guide. Entropy 2019, 21, 613. [CrossRef]
21. Cuesta-Frau, D. Using the Information Provided by Forbidden Ordinal Patterns in Permutation Entropy to Reinforce Time Series
Discrimination Capabilities. Entropy 2020, 22, 494. [CrossRef]
22. Myers, A.; Khasawneh, F.A. On the automatic parameter selection for permutation entropy. Chaos Interdiscip. J. Nonlinear Sci.
2020, 30, 033130. [CrossRef]
23. Boaretto, B.R.R.; Budzinski, R.C.; Rossi, K.L.; Prado, T.L.; Lopes, S.R.; Masoller, C. Evaluating Temporal Correlations in Time
Series Using Permutation Entropy, Ordinal Probabilities and Machine Learning. Entropy 2021, 23, 1025. [CrossRef]
24. Bandt, C. Order patterns, their variation and change points in financial time series and Brownian motion. Stat. Pap. 2020,
61, 1565–1588. [CrossRef]
25. Humeau-Heurtier, A. Multiscale Entropy Approaches and Their Applications. Entropy 2020, 22, 644. [CrossRef]
26. Neuder, M.; Bradley, E.; Dlugokencky, E.; White, J.W.C.; Garland, J. Detection of local mixing in time-series data using permutation
entropy. Phys. Rev. E 2021, 103, 22217. [CrossRef]
27. Dávalos, A.; Jabloun, M.; Ravier, P.; Buttelli, O. The Impact of Linear Filter Preprocessing in the Interpretation of Permutation
Entropy. Entropy 2021, 23, 787. [CrossRef]
28. Gutjahr, T.; Keller, K. On Rényi Permutation Entropy. Entropy 2021, 24, 37. [CrossRef] [PubMed]
29. Soriano, M.C.; Zunino, L. Time-Delay Identification Using Multiscale Ordinal Quantifiers. Entropy 2021, 23, 969. [CrossRef]
[PubMed]
30. Dávalos, A.; Jabloun, M.; Ravier, P.; Buttelli, O. Improvement of Statistical Performance of Ordinal Multiscale Entropy Techniques
Using Refined Composite Downsampling Permutation Entropy. Entropy 2020, 23, 30. [CrossRef] [PubMed]
31. Ravier, P.; Dávalos, A.; Jabloun, M.; Buttelli, O. The Refined Composite Downsampling Permutation Entropy Is a Relevant Tool
in the Muscle Fatigue Study Using sEMG Signals. Entropy 2021, 23, 1655. [CrossRef]
32. Kang, H.; Zhang, X.; Zhang, G. Phase permutation entropy: A complexity measure for nonlinear time series incorporating phase
information. Phys. A Stat. Mech. Appl. 2021, 568, 125686. [CrossRef]
33. Morel, C.; Humeau-Heurtier, A. Multiscale permutation entropy for two-dimensional patterns. Pattern Recognit. Lett. 2021,
150, 139–146. [CrossRef]
34. Amigó, J.M.; Dale, R.; Tempesta, P. Complexity-based permutation entropies: From deterministic time series to white noise.
Commun. Nonlinear Sci. Numer. Simul. 2022, 105, 106077. [CrossRef]
35. Shibasaki, Y. Permutation-Loewner entropy analysis for 2-dimensional Ising system interface. Phys. A Stat. Mech. Appl. 2022,
594, 126943. [CrossRef]
36. Huang, X.; Shang, H.L.; Pitt, D. A model sufficiency test using permutation entropy. J. Forecast. 2022, 41, 1017–1036. [CrossRef]
37. Wan, L.; Ling, G.; Guan, Z.H.; Fan, Q.; Tong, Y.H. Fractional multiscale phase permutation entropy for quantifying the complexity
of nonlinear time series. Phys. A Stat. Mech. Appl. 2022, 600, 127506. [CrossRef]
38. Xiao, F.; Yang, D.; Lv, Z.; Guo, X.; Liu, Z.; Wang, Y. Classification of hand movements using variational mode decomposition and
composite permutation entropy index with surface electromyogram signals. Future Gener. Comput. Syst. 2020, 110, 1023–1036.
[CrossRef]
39. Zhang, X.; Liang, Y.; Zhou, J.; Zang, Y. A novel bearing fault diagnosis model integrated permutation entropy, ensemble empirical
mode decomposition and optimized SVM. Measurement 2015, 69, 164–179. [CrossRef]
40. Zhao, L.; Yu, W.; Yan, R. Gearbox Fault Diagnosis Using Complementary Ensemble Empirical Mode Decomposition and
Permutation Entropy. Shock Vib. 2016, 2016, 3891429. [CrossRef]
41. An, X.; Pan, L. Bearing fault diagnosis of a wind turbine based on variational mode decomposition and permutation entropy.
Proc. Inst. Mech. Eng. Part O J. Risk Reliab. 2017, 231, 200–206. [CrossRef]
42. Chen, X.; Yang, Y.; Cui, Z.; Shen, J. Wavelet Denoising for the Vibration Signals of Wind Turbines Based on Variational Mode
Decomposition and Multiscale Permutation Entropy. IEEE Access 2020, 8, 40347–40356. [CrossRef]
43. Ying, W.; Zheng, J.; Pan, H.; Liu, Q. Permutation entropy-based improved uniform phase empirical mode decomposition for
mechanical fault diagnosis. Digital Signal Process. 2021, 117, 103167. [CrossRef]
Entropy 2022, 24, 1343 16 of 17

44. Gao, J.; Wang, X.; Wang, X.; Yang, A.; Yuan, H.; Wei, X. A High-Impedance Fault Detection Method for Distribution Systems
Based on Empirical Wavelet Transform and Differential Faulty Energy. IEEE Trans. Smart Grid 2022, 13, 900–912. [CrossRef]
45. Li, Y.; Li, Y.; Chen, X.; Yu, J. A Novel Feature Extraction Method for Ship-Radiated Noise Based on Variational Mode Decomposi-
tion and Multi-Scale Permutation Entropy. Entropy 2017, 19, 342. [CrossRef]
46. Xie, D.; Esmaiel, H.; Sun, H.; Qi, J.; Qasem, Z.A.H. Feature Extraction of Ship-Radiated Noise Based on Enhanced Variational
Mode Decomposition, Normalized Correlation Coefficient and Permutation Entropy. Entropy 2020, 22, 468. [CrossRef]
47. Xie, D.; Sun, H.; Qi, J. A New Feature Extraction Method Based on Improved Variational Mode Decomposition, Normalized
Maximal Information Coefficient and Permutation Entropy for Ship-Radiated Noise. Entropy 2020, 22, 620. [CrossRef] [PubMed]
48. Tian, Z.; Li, S.; Wang, Y. A prediction approach using ensemble empirical mode decomposition-permutation entropy and
regularized extreme learning machine for short-term wind speed. Wind Energy 2020, 23, 177–206. [CrossRef]
49. Xie, D.; Hong, S.; Yao, C. Optimized Variational Mode Decomposition and Permutation Entropy with Their Application in
Feature Extraction of Ship-Radiated Noise. Entropy 2021, 23, 503. [CrossRef] [PubMed]
50. Xia, X.; Chen, B.; Zhong, W.; Wu, L. Correlation Power Analysis for SM4 based on EEMD, Permutation Entropy and Singular
Spectrum Analysis. In Proceedings of the 2021 IEEE 5th Advanced Information Technology, Electronic and Automation Control
Conference (IAEAC), Chongqing, China, 12–14 March 2021 ; pp. 1478–1485. [CrossRef]
51. Sharma, S.; Tiwari, S.; Singh, S. Integrated approach based on flexible analytical wavelet transform and permutation entropy for
fault detection in rotary machines. Measurement 2021, 169, 108389. [CrossRef]
52. Shannon, C.E. A mathematical theory of communication. Bell Syst. Tech. J. 1948, 27, 379–423. [CrossRef]
53. Aziz, W.; Arif, M. Multiscale Permutation Entropy of Physiological Time Series. In Proceedings of the 2005 Pakistan Section
Multitopic Conference, Karachi, Pakistan, 24–25 December 2005; pp. 1–6. [CrossRef]
54. Wu, S.D.; Wu, C.W.; Lee, K.Y.; Lin, S.G. Modified multiscale entropy for short-term time series analysis. Phys. A Stat. Mech. Appl.
2013, 392, 5865–5873. [CrossRef]
55. Zunino, L.; Pérez, D.; Kowalski, A.; Martín, M.; Garavaglia, M.; Plastino, A.; Rosso, O. Fractional Brownian motion, fractional
Gaussian noise, and Tsallis permutation entropy. Phys. A Stat. Mech. Appl. 2008, 387, 6057–6068. [CrossRef]
56. Yang, C.; Jia, M. Hierarchical multiscale permutation entropy-based feature extraction and fuzzy support tensor machine with
pinball loss for bearing fault identification. Mech. Syst. Signal Process. 2021, 149, 107182. [CrossRef]
57. Huang, S.; Wang, X.; Li, C.; Kang, C. Data decomposition method combining permutation entropy and spectral substitution with
ensemble empirical mode decomposition. Measurement 2019, 139, 438–453. [CrossRef]
58. Liu, X.; Wang, Z.; Li, M.; Yue, C.; Liang, S.Y.; Wang, L. Feature extraction of milling chatter based on optimized variational mode
decomposition and multi-scale permutation entropy. Int. J. Adv. Manuf. Technol. 2021, 114, 2849–2862. [CrossRef]
59. Yang, H.; Zhang, A.; Li, G. A New Singular Spectrum Decomposition Method Based on Cao Algorithm and Amplitude Aware
Permutation Entropy. IEEE Access 2021, 9, 44534–44557. [CrossRef]
60. Dávalos, A.; Jabloun, M.; Ravier, P.; Buttelli, O. On the Statistical Properties of Multiscale Permutation Entropy: Characterization
of the Estimator’s Variance. Entropy 2019, 21, 450. [CrossRef]
61. Review of local mean decomposition and its application in fault diagnosis of rotating machinery. J. Syst. Eng. Electron. 2019,
30, 799. [CrossRef]
62. Dávalos, A.; Jabloun, M.; Ravier, P.; Buttelli, O. Theoretical Study of Multiscale Permutation Entropy on Finite-Length Fractional
Gaussian Noise. In Proceedings of the 2018 26th European Signal Processing Conference (EUSIPCO), Rome, Italy, 3–7 September
2018; pp. 1087–1465. ISSN 2076-1465. [CrossRef]
63. Little, D.J.; Kane, D.M. Permutation entropy of finite-length white-noise time series. Phys. Rev. E 2016, 94, 022118. [CrossRef]
64. Davalos, A.; Jabloun, M.; Ravier, P.; Buttelli, O. Multiscale Permutation Entropy: Statistical Characterization on Autoregressive
and Moving Average Processes. In Proceedings of the 2019 27th European Signal Processing Conference (EUSIPCO), A Coruna,
Spain, 2–6 September 2019; pp. 1–5. [CrossRef]
65. Farina, D.; Merletti, R. Comparison of algorithms for estimation of EMG variables during voluntary isometric contractions. J.
Electromyogr. Kinesiol. 2000, 10, 337–349. [CrossRef]
66. Cashaback, J.G.; Cluff, T.; Potvin, J.R. Muscle fatigue and contraction intensity modulates the complexity of surface electromyog-
raphy. J. Electromyogr. Kinesiol. 2013, 23, 78–83. [CrossRef]
67. Davalos Trevino, A. On the Statistical Properties of Multiscale Permutation Entropy and its Refinements, with Applications on
Surface Electromyographic Signals. Ph.D. Thesis, Orleans University, Orleans, France, 2020; PHD thesis supervised by O. Buttelli
and M. Jabloun, Orleans University-Signal Processing 2020.
68. Huang, N.E.; Shen, Z.; Long, S.R.; Wu, M.C.; Shih, H.H.; Zheng, Q.; Yen, N.C.; Tung, C.C.; Liu, H.H. The empirical mode
decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proc. R. Soc. Lond. Ser. A Math.
Phys. Eng. Sci. 1998, 454, 903–995. [CrossRef]
69. Flandrin, P.; Rilling, G.; Goncalves, P. Empirical Mode Decomposition as a Filter Bank. IEEE Signal Process. Lett. 2004, 11, 112–114.
[CrossRef]
70. Wang, G.; Chen, X.Y.; Qiao, F.L.; Wu, Z.; Huang, N.E. On intrinsic mode function. Adv. Adapt. Data Anal. 2010, 2, 277–293.
[CrossRef]
71. Pustelnik, N.; Borgnat, P.; Flandrin, P. Empirical mode decomposition revisited by multicomponent non-smooth convex
optimization. Signal Process. 2014, 102, 313–331. [CrossRef]
Entropy 2022, 24, 1343 17 of 17

72. Qiao, N.; Wang, L.h.; Liu, Q.y.; Zhai, H.q. Multi-scale eigenvalues Empirical Mode Decomposition for geomagnetic signal
filtering. Measurement 2019, 146, 885–891. [CrossRef]
73. Zosso, D.; Dragomiretskiy, K. Variational Mode Decomposition. IEEE Trans. Signal Process. 2013, 62, 531–544. [CrossRef]
74. Wang, Y.; Markert, R. Filter bank property of variational mode decomposition and its applications. Signal Process. 2016,
120, 509–521. [CrossRef]
75. Lian, J.; Liu, Z.; Wang, H.; Dong, X. Adaptive variational mode decomposition method for signal processing based on mode
characteristic. Mech. Syst. Signal Process. 2018, 107, 53–77. [CrossRef]
76. Rehman, N.U.; Aftab, H. Multivariate Variational Mode Decomposition. IEEE Trans. Signal Process. 2019, 67, 6039–6052.
[CrossRef]
77. Liu, S.; Yu, K. Successive multivariate variational mode decomposition. Multidimens. Syst. Signal Process. 2022. [CrossRef]
78. Gilles, J. Empirical Wavelet Transform. IEEE Trans. Signal Process. 2013, 61, 3999–4010. [CrossRef]
79. Deng, F.; Liu, Y. An Adaptive Frequency Window Empirical Wavelet Transform Method for Fault Diagnosis of Wheelset Bearing.
In Proceedings of the 2018 Prognostics and System Health Management Conference (PHM-Chongqing), Chongqing, China, 26–28
October 2018; pp. 1291–1294. [CrossRef]
80. Wang, D.; Zhao, Y.; Yi, C.; Tsui, K.L.; Lin, J. Sparsity guided empirical wavelet transform for fault diagnosis of rolling element
bearings. Mech. Syst. Signal Process. 2018, 101, 292–308. [CrossRef]
81. Hu, Y.; Li, F.; Li, H.; Liu, C. An enhanced empirical wavelet transform for noisy and non-stationary signal processing. Digit.
Signal Process. 2017, 60, 220–229. [CrossRef]
82. Broomhead, D.; King, G.P. Extracting qualitative dynamics from experimental data. Phys. D Nonlinear Phenom. 1986, 20, 217–236.
[CrossRef]
83. Harmouche, J.; Fourer, D.; Auger, F.; Borgnat, P.; Flandrin, P. The Sliding Singular Spectrum Analysis: A Data-Driven Non-
Stationary Signal Decomposition Tool. IEEE Trans. Signal Process. 2017, 66, 251–263. [CrossRef]
84. Rodrigues, P.C.; Lourenço, V.; Mahmoudvand, R. A robust approach to singular spectrum analysis. Qual. Reliab. Eng. Int. 2018,
34, 1437–1447. [CrossRef]
85. Mateo, J.; Laguna, P. Improved heart rate variability signal analysis from the beat occurrence times according to the IPFM model.
IEEE Trans. Bio-Med. Eng. 2000, 47, 985–996. [CrossRef]
86. Ravier, P.; Buttelli, O.; Jennane, R.; Couratier, P. An EMG fractal indicator having different sensitivities to changes in force and
muscle fatigue during voluntary static muscle contractions. J. Electromyogr. Kinesiol. 2005, 15, 210–221. [CrossRef]