This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts

for publication in the IEEE ICC 2009 proceedings

Sensitivity Analysis of Burst Detection and RF

Fingerprinting Classification Performance
R.W. Klein, M.A. Temple, M.J. Mendenhall and D.R. Reising
Air Force Institute of Technology
Wright-Patterson AFB, Ohio 45433 USA
Email: [email protected]

Abstract—There has been a recent shift toward improving uniqueness exists and is attributable to various manufacturing,
wireless access security within the OSI PHY layer by exploiting RF aging, and environmental factors [3]. While several process-
features that are inherently device specific and difficult to replicate ing steps are required to effectively exploit RF fingerprints,
by an unintended party. This work addresses the extraction and
exploitation of RF “fingerprints” to classify emissions and provide transient detection is perhaps the most important [6], [8]. In
device-specific identification. Burst transient detection precedes this context, transient detection includes both the transient
RF fingerprint extraction and is generally the most critical start time and signal duration over which fingerprints are ex-
step in the overall process. This work provides a much needed tracted. Both of these factors are important given that improper
sensitivity analysis of burst detection capability. The analysis is selection of either can bias the processing to favor channel
conducted using instantaneous amplitude responses with both
Fractal-Bayesian Step Change Detection (Fractal-BSCD) and Vari- noise effects or steady-state signal effects [3]. Burst transients
ance Trajectory (VT) processes. The performance of each method can be estimated using various emission features. However,
is evaluated under varying SNR conditions using experimentally instantaneous amplitude and instantaneous phase features are
collected 802.11a OFDM signals. The impact of transient detection perhaps the most extensively investigated [3], [6]–[8]. With the
error on signal classification performance is then demonstrated exception of more recent work in [12] and [13], these previous
using RF fingerprints and Multiple Discriminant Analysis (MDA)
with Maximum Likelihood (ML) classification. The VT technique efforts lack a detailed sensitivity analysis of burst detection and
emerges as the better alternative for all SNRs considered and fingerprint classification performance under varying channel
yields MDA-ML classification accuracy that is consistent with noise conditions.
“perfect” transient estimation performance.
This type of analysis is imperative for determining the
I. I NTRODUCTION minimum acceptable collected SNR that will provide consistent
and accurate results. Establishing the minimum acceptable SNR
Considerable research has been conducted on detecting also allows determination of the maximum transmitter-receiver
and/or mitigating spoofing within the Medium Access Con- separation distance which would aide in laying out the physical
trol (MAC) layer of the Open Systems Interconnection (OSI) hardware for network security. Noise sensitivity performance
stack [1], [2]. There has been a recent shift toward providing can also provide a good discriminator for comparing various
added security at the OSI Physical (PHY) layer by exploiting detection and classification techniques. For the work presented
RF features that are inherently unique to a specific device and here, noise sensitivity analysis for transient detection perfor-
that are difficult to replicate by an unintended party. For ex- mance is conducted for three noise-signal conditions, including:
ample, some efforts have investigated Received Signal Strength 1) noise only effects using a single collected 802.11a burst and
(RSS) (a power-based metric) for detecting and/or locating a multiple noise realizations, 2) signal only effects incorporating
spoofing node [1], [2]. Both of these efforts demonstrated some burst-to-burst signal variability with a single noise realization,
success at detecting spoofing using experiments conducted with and 3) combined noise-signal effects using multiple burst and
different hardware and in different physical environments. noise realizations. The impact of transient detection error on
RF fingerprinting work provides an alternative PHY layer signal classification performance is then demonstrated using
approach but is dismissed in [2] for “scale” reasons. For Multiple Discriminant Analysis with Maximum Likelihood
applications where size constraints may not be a dominant classification (MDA-ML).
factor, RF fingerprinting remains a viable alternative and is
considered in this work. Collectively, related works in RF fin- II. BACKGROUND
gerprinting, electromagnetic signatures, intrapulse modulation,
and unintentional modulation [3]–[11], form a solid basis for A. Fractal-Bayesian Step Change Detector
developing techniques that may be applicable to commercial It has been demonstrated that transient detection can be
communication devices. accomplished using the fractal dimension measure followed by
If the inherent RF fingerprints are repeatedly extractable and a Bayesian Step Change Detector [7]. This process is denoted
unique, they may be used to identify the specific make, model, here as Fractal-BSCD. The fractal derivation can be found in
or serial number of a device. Previous work suggests that this [14] and can be calculated using the following Higuchi method.

U.S. Government work not protected by U.S. copyright

This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE ICC 2009 proceedings

Given data time series {X(1), X(2), ..., X(NX )}, the curve 0.01

Amplitude
length is defined as: 0.005

X̄(NX − 1)
Lm (k) = , (1) 0
0 500 1000 1500 2000 2500 3000
k 2 NL Sample Number

NL
 1

Fractal
X̄ = |X (m + ik) − X (m + (i − 1) k)| ,
i=1 0.5

where NL = (NX − m)/k, • is the floor operator, k is the 500 1000 1500 2000 2500
Sample Number
interval index number, and m ∈ [1, k] is the start time index 0.4
number. For this work, the signal of interest was divided into

PDF
windowed regions containing NX = 20 samples. 0.2

The average of Lm (k) over m is denoted here as L (k) 0

0 500 1000 1500 2000 2500 3000
and defines the curve length for time interval k. By varying k Sample Number

over [1, kmax ] and plotting L (k) versus k on a log-log scale,

Fig. 1. Instantaneous Signal Amplitude (Top), Fractal d (Middle), and A-
the data ideally forms a straight line, with a proper selection Posteriori PDF (Bottom).
of kmax . The fractal dimension d is defined as the negative of
the line slope, which can be calculated using a least squares
1+(m−1)Ns +Nw
method. kmax is empirically chosen. If it is too large, the data 1  2
plotted on the log-log scale will not be linear. If it is too small, Wa (m) = [a(k) − µw ] , (4)
Nw
there will not be enough data points for an accurate linear fit. k=1+(m−1)Ns
For this work, a value of kmax = 10 is chosen for all fractal where i = 1, 2, ..., Lw − 1, m = 1, 2, ..., Lw , Lw =
calculations. (Na − Nw )/Ns  + 1, Nw is the window extent, and Ns is
Using the fractal dimension vector d formed across all the number of samples the window advances between calcu-
data windows, BSCD is applied to determine the a-posteriori lations. The µw factor in (4) is the sample mean of {aw (k)}
probability that a given fractal dimension dm ∈ d represents the which is the subsequence of consecutive elements from {a(k)}
data change point corresponding to the transient start. The a- contained in the window.
posteriori Probability Distribution Function (PDF) for m given Fig. 2 shows a representative amplitude response and cor-
d is [15] responding {V Ta (i)} sequences for two different SN Rs. As
   −1
 NF −2 shown, there is a distinct VT peak response corresponding
p ({m} |d, I) ∝ m (NF − m) × d¯ 2
, (2) to the burst transient start which becomes less discernable as
m  2
SN R decreases. Given optimization is not addressed under
NF 2 NF
 1  1  this work, a simple sliding window average and thresholding
d¯ = d2i − di − di , technique is used to automatically estimate the transient start
i=1
m i=1
NF − m i=m+1
based on locating the initial peak response in {V Ta (i)}.
where NF is the length of d, I denotes prior information,
and m is the potential change point being evaluated. The C. MDA-ML Classification
value of m corresponding to max[p ({m} |d, I)] establishes the
transient start sample number. The work in [3], [6] shows that MDA is an extension of Fisher’s Linear Discriminant (FLD)
abrupt, non-gradual feature changes are important for the BSCD process for more than two classes [16]. Classification is demon-
to work successfully. Signals that possess more gradually strated here using an MDA-ML process [17]. For the 3-class
changing amplitude responses require a different technique for problem, the MDA process projects higher-dimensional data
transient location. A representative signal, fractal dimension, onto a 2-dimensional “Fisher plane” that maximizes interclass
and a-posteriori PDF is shown in Fig. 1. distances while simultaneously minimizing intraclass distances.
In principle, this method cannot improve classification po-
B. Variance Trajectory (VT) tential. However, it provides good class separation and vi-
sualization of data having dimensionality greater than three.
The work in [6] analyzed Bluetooth signals using Fractal- Using lower-dimensional data, ML decision boundaries are
BSCD with instantaneous amplitude and showed some im- determined assuming normally distributed input data, equal
provement over BSCD using a VT process with instanta- costs or risk and uniform prior probabilities. To discriminate
neous phase. The work presented here generates VT sequence c classes using d-dimensional input data, the input vector x is
{V Ta (i)} using instantaneous amplitude sequence {a(k)}, linearly projected onto a (c − 1)-dimensional space using
k = 1, 2, ..., Na , to estimate the burst transient start. The ith
element of sequence {V Ta (i)} is given by [12] y = Wt x , (5)
V Ta (i) = |Wa (i) − Wa (i + 1)| , (3)
This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE ICC 2009 proceedings

5
Amplitude

4
3
2 200
1

500 1000 1500 2000 2500 3000 150

Sample Number
1.5

Bin Count
100
1
VT

0.5
50
0
0 500 1000 1500 2000 2500 3000
Sample Number 0
2 2
1.5 1 30
24 27
VT

1 0
15 18 21
−1 9 12
0.5 6
3
−2 −3 0
Error (secs * 10−5) SNR (dB)
500 1000 1500 2000 2500
Sample Number
Fig. 3. Impact of Channel Noise variation: Fractal-BSCD transient estimation
Fig. 2. Instantaneous signal amplitude (Top) and corresponding V Ta for: using 200 noise realizations and a given 802.11a RF Burst.
(Middle) SN R = 30 dB (Middle) and (Bottom) SN R = 3 dB.

impact, the two methods perform similarly at higher SNRs.

where y is the vector of projected values and W is a d×(c−1) Differences arise at lower SNRs, with the VT method degrading
projection matrix. Classification is performed using unknown as before and producing missed detections. The Fractal-BSCD
data and the 2-dimensional trained decision boundaries. The response degrades differently than before, becoming multi-
process classifies each unknown input data set by projecting modal at lower SNRs and producing a significant number
it onto the trained “Fisher plane” using (5). Projected points of detections in the noise-only portion of the signal. The
falling within the correct region are correctly classified while modes are attributable to anomalous spikes in a specific noise
those falling outside the correct region are misclassified. The realization. This is consistent with results in [6] and [3] given
percentage of correct classification is determined based on the that BSCD processing is most effective when non-gradual
total number of unknown trials. parameter changes occur. At lower SNRs the amplitude change
is too gradual in some bursts for the BSCD method to reliably
III. R ESULTS : T RANSIENT D ETECTION detect them.
1) Channel Noise Variability: These results illustrate the 3) Combined Noise-Signal Variability: These results illus-
effect of channel noise variation for a given RF burst using 200 trate the combined effects of noise and burst-to-burst signal
AWGN realizations scaled and added to achieve the desired variability. In this case, 200 AWGN realizations are scaled for
analysis SNR. Fractal-BSCD and VT estimation results are each SNR and added to each of the 200 collected bursts – a
shown in Fig. 3 and Fig. 4, respectively.
At higher SNRs the two methods perform similarly as the
noise power varies, with primary differences beginning at
SN R ≈ 9 dB. Fractal-BSCD degradation is directly attributed
to the a-posteriori PDF degradation, as calculated per (2) and
shown in Fig. 1. The PDF loses the strong peak response and
becomes more uniformly distributed as noise power increases. 150
VT degradation is attributed to, and affected by, threshold
Bin Count

selection criterion. For the non-optimum method implemented 100

here, the threshold criterion is not always satisfied and a default

transient start value is assigned – a missed detection. The 50

number of missed detections at lower SNRs can be reduced by

0
changing the threshold. However, this also reduces estimation 2
accuracy and precision at higher SNRs. 1 27 30
0 18 21 24
2) Burst-to-Burst Variability: These results illustrate the ef- −1 9 12 15
3 6
fect of burst-to-burst variation using a given AWGN realization −5
Error (secs * 10 )
−2 −3 0
SNR (dB)
scaled to achieve the desired analysis SNR. Results for Fractal-
BSCD and VT estimation using 200 collected bursts are shown Fig. 4. Impact of Channel Noise variation: VT transient estimation using 200
in Fig. 5 and Fig. 6, respectively. As with the channel noise noise realizations and a given 802.11a RF Burst.
This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE ICC 2009 proceedings

Bin Count * 104

150
2
Bin Count

100
1

50
0
2
0
2
1 27 30
0 18 21 24
30
1
24 27 −1 9 12 15
0
15 18 21 3 6
−1 9 12 −5 −2 −3 0
6 Error (secs * 10 ) SNR (dB)
3
−2 −3 0
Error (secs * 10−5) SNR (dB)
Fig. 7. Impact of Combined variation:Fractal-BSCD transient estimation using
Fig. 5. Impact of RF Burst variation: Fractal-BSCD transient estimation using 200 noise realizations per 200 802.11a bursts.
200 802.11a bursts and fixed noise realization.

data represents the signal “fingerprint” used by the MDA-

total of 40,000 unique AWGN realizations per SNR. Results ML process and includes variance, kurtosis and skewness
for the Fractal-BSCD and VT method are shown in Figs. 7 statistics calculated over the preamble instantaneous amplitude,
and Fig. 8, respectively. In this combined channel noise and frequency and phase responses.
burst-to-burst variability effect, the channel noise is dominant. A five-fold cross validation and Monte Carlo process was
This is evident in that channel noise effect results in Fig. 3 used to ensure statistical significance. The overall process
and Fig. 4 are almost identical to the combined effects results included: 1) generate AWGN, scale to achieve desired SNR,
Fig. 7 and Fig. 8. and add independent realizations to each burst, 2) estimate
transient start using the method being evaluated, 3) generate
IV. R ESULTS : S IGNAL C LASSIFICATION I MPACT W per Section II.C using the first 160 bursts from each
device for MDA training, 4) use remaining 40 bursts from
A total of 200 802.11a bursts were collected from three
each device as “unknown” input data, transform/project them
different devices and used to demonstrate the impact of tran-
using W, and classify each per ML criteria, 5) store/accumulate
sient estimation on MDA-ML classification performance. After
classification results, 6) circularly shift (re-order) the collected
locating the transient start, statistical waveform feature data
bursts by 40 and repeat Step 2 through Step 6 four times,
is extracted from the next 16 µSec of the burst (802.11a
7) repeat Step 1 through Step 6 200 times using different in-
preamble region [18]) and MDA-ML classification performed
dependent realizations of AWGN for each iteration, 8) average
in accordance with Section II.C. The multi-dimensional input

3
Bin Count * 104

150
Bin Count

2
100

1
50

0 0
2 2
1 30 1 27 30
24 27 0 21 24
0 18 21 18
12 15
−1 9 12 15
−1 6 9 6
3 3
−5 −2 −3 0 −5 −2 −3 0
Error (secs * 10 ) SNR (dB) Error (secs * 10 ) SNR (dB)

Fig. 6. Impact of RF Burst variation: VT transient estimation using 200 Fig. 8. Impact of Combined variation:VT transient estimation using 200 noise
802.11a bursts and fixed noise realization. realizations per 200 802.11a bursts.
This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE ICC 2009 proceedings

accumulated classification results from Step 5 to obtain final ACKNOWLEDGMENT

classification performance. This process is repeated in 3 dB
steps for SN R ∈ [−3, 30] dB. This research supported by the Sensors Directorate, Air Force
Research Laboratory, and the Tactical SIGINT Technology
Fig. 9 shows average MDA-ML classification accuracy (TST) Program.
including burst transient detection error effects for Perfect,
Perfect with random error, Fractal-BSCD, and VT transient
detection methods. In this case, “perfect” results are obtained “The views expressed in this article are those of the author(s) and
do not reflect official policy of the United States Air Force, Department
using a transient start location based on visual inspection of of Defense or the U.S. Government.”
each collected burst. To determine if “perfect” provides best
possible MDA-ML classification accuracy, a uniform randomly
distributed error was added to the perfect transient location R EFERENCES
estimates and results generated for comparison. As shown, the
Perfect with Random Error results are consistent with Perfect [1] Y. Chen, W. Trappe and R. Martin, “Detecting and localizing wireless
spoofing attacks.” IEEE Conf on Sensor, Mesh and AdHoc Comm and
results and marginally better/poorer for SN R below/above ap- Nets (SECON), pp. 193-202, Jun 2007.
proximately 14 dB, respectively. The VT technique marginally [2] Y. Sheng, K. Tan, G. Chen, D. Kotz, and A. Campbell, “Detecting 802.11
outperforms Perfect estimation for SN R ∈ [6, 12] dB but pro- MAC layer spoofing using received signal strength.” IEEE 27th Annual
Conf on Computer Comm (INFOCOM), Apr 2008.
vides considerable performance improvement when compared [3] O. Ureten and N. Serinken, “Wireless security through RF fingerprinting,”
to the Fractal-BSCD technique at lower SN Rs. Canadian Journal of Electrical and Computer Engineering, vol. 32, no. 1,
pp. 27 – 33, Winter 2007.
[4] L.E. Langley, “Specific Emitter Identification (SEI) and classical param-
eter fusion technology, pp. 277-381.” IEEE Western Electronics Show
V. C ONCLUSION and Conference (WESCON), Sep 1993.
[5] J. Hall, M. Barbeau and E. Kranakis, “Using transceiverprints for anomaly
based intrusion detection.” 3rd IASTED Int’l Conf on Comm, Internet
Using experimentally collected emissions from 802.11a and Info Technology (CIIT), Nov 2004.
OFDM devices, a sensitivity analysis was conducted for burst [6] ——, “Detection of transient in radio frequency fingerprinting using
signal phase.” IASTED Int’l Conf on Wireless and Optical Commu-
transient detection and RF fingerprinting classification perfor- nications, May 2003.
mance. Transient detection error of Fractal-BSCD and VT pro- [7] O. Ureten and N. Serinken, “Detection of radio transmitter turn-on
cesses was characterized and its impact on signal classification transients,” IEE Electronics Letters, vol. 35, no. 23, pp. 1996 – 1997,
Nov 1999.
evaluated using MDA-ML. Overall, the VT technique provided [8] ——, “Bayesian detection of WiFi transmitter RF fingerprints,” IEE
MDA-ML classification performance that was consistent with Electronics Letters, vol. 41, no. 6, pp. 373 – 374, Mar 2005.
“perfect” transient estimation results and provided considerable [9] N. Serinken and O. Ureten, “Generalised dimension characterization of
radio transmitter turn-on transients,” IEE Electronics Letters, vol. 36,
performance improvement when compared with the Fractal- no. 12, pp. 1064 – 1064, Jun 2000.
BSCD technique at lower SN Rs. In addition, the VT method is [10] J. Dudczyk, J. Matuszewski and M. Wnuk, “Applying the radiated
algorithmically less complex and requires orders-of-magnitude emission to specific emitter identification.” Int’l Conf on Microwaves,
Radar and Wireless Comm (MIKON), pp. 431-434, May 2004.
less computation time, making it a viable approach for auto- [11] A. Kawalec, T. Rapacki, S. Wnuczek, J. Dudczyk, and R. Owczarek,
matic transient detection and classification applications. “Mixed method based on intrapulse data and radiated emission to emitter
sources recognition,” May 2006, pp. 487–490.
[12] W.C. Suski, M.A. Temple, M.J. Mendenhall and R.F. Mills, “Using
spectral fingerprints to improve wireless network security,” in Proceedings
1
of the 2008 IEEE Global Communications Conference, Mar 2008.
[13] ——, “Radio Frequency (RF) fingerprinting commercial communication
devices to enhance electronic security,” Int. J. of Electronic Security and
0.9
Digital Forensics, Dec 2008.
[14] T. Higuchi, “Approach to an irregular time series on the basis of the
fractal theory,” Phys. D, vol. 31, no. 2, pp. 277–283, 1988.
Classification Accuracy

0.8
[15] J. O Ruanaidh and W. Fitzgerald, Numerical bayesian methods applied to
signal processing. Statistics and Computing Series, New York: Springer,
1996.
0.7
[16] R.A. Fisher, “The use of multiple measurements in taxonomic problems,”
Annals of Eugenics, vol. 7, pp. 179 – 188, 1936.
Fractal−BSCD
[17] R. Duda, P. Hart, and D. Stork, Pattern Classification, 2nd ed. New
0.6
Perfect w/Random error
York: John Wiley & Sons, Inc., 2001.
Perfect
[18] 802.11a, WLAN MAC and PHY layer specs: high speed PHY extension
VT in the 5 GHz band, IEEE, Piscataway, NJ 08855-1331, USA, Sep 16,
0.5
1999, revised 2003.

0.4
−5 0 5 10 15 20 25 30
SNR (dB)

Fig. 9. Average MDA-ML classification accuracy including burst transient

detection error effects for various burst detection methods.