Get the full ebook with Bonus Features for a Better Reading Experience on ebookgate.

com

Local Approximation Techniques in Signal and Image

Processing Jaakko Astola

https://ebookgate.com/product/local-approximation-
techniques-in-signal-and-image-processing-jaakko-astola/

OR CLICK HERE

DOWLOAD NOW

Download more ebook instantly today at https://ebookgate.com

Instant digital products (PDF, ePub, MOBI) available
Download now and explore formats that suit you...

Information Fusion in Signal and Image Processing Digital

Signal and Image Processing 1st Edition Isabelle Bloch

https://ebookgate.com/product/information-fusion-in-signal-and-image-
processing-digital-signal-and-image-processing-1st-edition-isabelle-
bloch/
ebookgate.com

Optimisation in Signal and Image Processing 1st Edition

Patrick Siarry

https://ebookgate.com/product/optimisation-in-signal-and-image-
processing-1st-edition-patrick-siarry/

ebookgate.com

Digital Signal and Image Processing 1st Edition Tamal Bose

https://ebookgate.com/product/digital-signal-and-image-processing-1st-
edition-tamal-bose/

ebookgate.com

Digital Image and Signal Processing for Measurement

Systems 1st Edition J. Richard Duro

https://ebookgate.com/product/digital-image-and-signal-processing-for-
measurement-systems-1st-edition-j-richard-duro/

ebookgate.com
Advanced Image Processing Techniques and Applications 1st
Edition N. Suresh Kumar

https://ebookgate.com/product/advanced-image-processing-techniques-
and-applications-1st-edition-n-suresh-kumar/

ebookgate.com

Applications in Time Frequency Signal Processing

Electrical Engineering Applied Signal Processing Series
1st Edition Antonia Papandreou-Suppappola
https://ebookgate.com/product/applications-in-time-frequency-signal-
processing-electrical-engineering-applied-signal-processing-
series-1st-edition-antonia-papandreou-suppappola/
ebookgate.com

The Digital Signal Processing Handbook Video Speech and

Audio Signal Processing 2nd Edition Vijay K. Madisetti

https://ebookgate.com/product/the-digital-signal-processing-handbook-
video-speech-and-audio-signal-processing-2nd-edition-vijay-k-
madisetti/
ebookgate.com

DSP for MATLAB and LabVIEW I Fundamentals of Discrete

Signal Processing Synthesis Lectures on Signal Processing
Isen
https://ebookgate.com/product/dsp-for-matlab-and-labview-i-
fundamentals-of-discrete-signal-processing-synthesis-lectures-on-
signal-processing-isen/
ebookgate.com

Image and Video Processing in the Compressed Domain

Jayanta Mukhopadhyay

https://ebookgate.com/product/image-and-video-processing-in-the-
compressed-domain-jayanta-mukhopadhyay/

ebookgate.com
Bellingham, Washington USA
Library of Congress Cataloging-in-Publication Data

Katkovnik, V. IA. (Vladimir IAkovlevich).

Local approximation techniques in signal and image processing / Vladimir Katkovnik,
Karen Egiazarian, and Jaakko Astola.
p. cm.
Includes bibliographical references and index.
ISBN: 0-8194-6092-3 (alk. paper)
1. Signal processing--Mathematics. 2. Image processing--Mathematics. 3.
Approximation theory. I. Egiazarian, K. (Karen), 1959- II. Astola, Jaakko. III. Title.

TK5102.9.K38 2006
621.382'2--dc22
2006042318

Published by

SPIE—The International Society for Optical Engineering

P.O. Box 10
Bellingham, Washington 98227-0010 USA
Phone: +1 360 676 3290
Fax: +1 360 647 1445
Email: [email protected]
Web: http://spie.org

Copyright © 2006 The Society of Photo-Optical Instrumentation Engineers

All rights reserved. No part of this publication may be reproduced or distributed

in any form or by any means without written permission of the publisher.

The content of this book reflects the work and thought of the author(s).
Every effort has been made to publish reliable and accurate information herein,
but the publisher is not responsible for the validity of the information or for any
outcomes resulting from reliance thereon.

Printed in the United States of America.

The cover image: Spring in Tampere, Finland – May 2006 by Alessandro Foi.
Contents

Preface xi
Notations and Abbreviations xv

1 Introduction 1
1.1 Linear Local Approximation 2
1.1.1 Windowing 2
1.1.2 Nonparametric estimation 3
1.1.3 Scale 5
1.1.4 Ideal scale 8
1.1.5 Adaptive varying scale 9
1.2 Anisotropy 11
1.2.1 Univariate case 11
1.2.2 Multivariate case 12
1.3 Nonlinear Local Approximation 14
1.3.1 Likelihood and quasi-likelihood 14
1.3.2 Robust M-estimation 16
1.3.3 Adaptive varying scale 16
1.4 Multiresolution Analysis 17
1.5 Imaging Applications 17
1.6 Overview of the Book 18

2 Discrete LPA 21
2.1 Introduction 22
2.1.1 Observation modeling 22
2.1.2 Classes of signals 23
2.1.3 Multi-index notation 26
2.2 Basis of LPA 28
2.2.1 Idea of LPA 28
2.2.2 Windowing and scale 34
2.2.3 Estimate calculation 36
2.2.4 Multivariate estimates 37
2.2.5 Examples 38
2.3 Kernel LPA Estimates 47
2.3.1 Estimate of signal 47
2.3.2 Estimate of derivative 48
2.3.3 Reproducing polynomial kernels 48
v
vi Contents

2.4 Nonparametric Regression 50

2.4.1 Regression function 50
2.4.2 Nadaraya-Watson estimate 51
2.5 Nonparametric Interpolation 53

3 Shift-Invariant LPA Kernels 57

3.1 Regular Grid Kernels 57
3.2 Vanishing Moments 60
3.3 Frequency Domain 62
3.3.1 Frequency response 62
3.3.2 Discrete-time Fourier transform 65
3.3.3 Vanishing moments 67
3.3.4 Discrete Fourier transform 68
3.3.5 Examples 71
3.4 Numerical Shift-Invariant Kernels 75
3.4.1 Square uniform window 76
3.4.2 Gaussian window 80
3.5 Numerical Differentiation 83
3.5.1 Univariate differentiation 84
3.5.2 Multivariate differentiation 87

4 Integral LPA 91
4.1 Integral Kernel Estimators 91
4.1.1 Integral LPA 91
4.1.2 Multivariate kernels and estimates 96
4.1.3 Limit LPA estimates 97
4.1.4 Frequency domain 97
4.2 Analytical Kernels 100
4.2.1 1D case 100
4.2.2 2D case 104
4.3 Generalized Singular Functions∗ 107
4.3.1 Gaussian smoothing kernels 107
4.3.2 Univariate Dirac delta function 108
4.3.3 Multivariate Dirac delta function 110
4.3.4 Dirac’s delta sequences 111
4.4 Potential Derivative Estimates 114

5 Discrete LPA Accuracy 121

5.1 Bias and Variance of Estimates 122
5.1.1 Main results 122
5.1.2 Discussion 125
5.2 Ideal Scale 126
5.2.1 Varying scale 126
5.2.2 Invariant scale 131
5.2.3 Example 132
5.3 Accuracy of Potential Differentiators∗ 136
Contents vii

6 Adaptive-Scale Selection 137

6.1 ICI Rule 139
6.1.1 Foundations 139
6.1.2 LPA-ICI algorithm 145
6.1.3 Complexity 147
6.1.4 Convergence 147
6.1.5 Threshold adjustment 148
6.2 Multiple-Window Estimation 151
6.2.1 Multiwindow estimate 152
6.2.2 Combined-window estimate 153
6.3 Denoising Experiments 154
6.3.1 Univariate signals 154
6.3.2 Binary imaging 167

7 Anisotropic LPA 173

7.1 Directional Signal Processing 173
7.1.1 Recent developments 173
7.1.2 Directional LPA-ICI approach 180
7.2 Directional LPA 187
7.2.1 Polynomials 188
7.2.2 Directional windowing 189
7.2.3 Rotation 190
7.2.4 Calculations 193
7.2.5 Shift-invariant kernels 194
7.2.6 Frequency domain 196
7.3 Numerical Directional Kernels 198
7.3.1 Sectorial kernels 198
7.3.2 Line-wise kernels 205

8 Anisotropic LPA-ICI Algorithms 207

8.1 Accuracy Analysis∗ 207
8.1.1 Polynomial smoothness 207
8.1.2 Vanishing moments 211
8.1.3 Taylor series 212
8.1.4 Basic estimates 213
8.1.5 Rotated estimates 217
8.2 Adaptive-Scale Algorithms 219
8.2.1 Calculations 219
8.2.2 Recursive algorithms 221
8.2.3 Algorithm complexity 224
8.3 Directional Image Denoising 224
8.3.1 Criteria and algorithms 224
8.3.2 Experiments 227
8.4 Directional Differentiation 236
8.4.1 Anisotropic gradient 237
8.4.2 Examples 242
viii Contents

8.5 Shading from Depth 253

8.6 Optical Flow Estimation 256
8.6.1 Optical flow equation 257
8.6.2 Velocity estimation 259
8.6.3 Experiments 261

9 Image Reconstruction 263

9.1 Image Deblurring 264
9.1.1 Blur modeling and deblurring 264
9.1.2 LPA deblurring 267
9.2 LPA-ICI Deblurring Algorithms 273
9.2.1 ICI adaptive scale 273
9.2.2 Algorithms 274
9.2.3 Performance study 277
9.3 Motion Deblurring 280
9.4 Super-Resolution Imaging 282
9.5 Inverse Halftoning 285
9.5.1 Error diffusion 286
9.5.2 Linear model of error diffusion 287
9.5.3 Anisotropic deconvolution 289
9.5.4 Simulation results 291
9.6 3D Inverse 294
9.6.1 Sectioning microscopy 294
9.6.2 Observation model 295
9.6.3 3D spatially adaptive inverse 299
9.6.4 Simulation 304

10 Nonlinear Methods 307

10.1 Why Nonlinear Methods? 307
10.1.1 Basic hypotheses 307
10.1.2 M-estimation 309
10.1.3 Statistical approach 311
10.1.4 Some distributions 312
10.2 Robust M-Estimation 319
10.2.1 Median estimates 320
10.2.2 Performance of M-estimates 323
10.2.3 Median in Gaussian noise (simulation) 326
10.2.4 Theory of minimax estimation∗ 330
10.2.5 Median in non-Gaussian noise (simulation) 337
10.3 LPA-ICI Robust M-Estimates 339
10.3.1 Performance of LPA M-estimates 340
10.3.2 ICI algorithm for M-estimates 342
10.4 Nonlinear Transform Methods 343
10.4.1 Model simplification 343
10.4.2 Variance stabilization 345
Contents ix

11 Likelihood and Quasi-Likelihood 351

11.1 Local Maximum Likelihood 351
11.1.1 Parametric ML 351
11.1.2 Nonparametric ML 353
11.1.3 Theory for local ML 355
11.1.4 LPA-ICI algorithms 358
11.2 Binary and Counting Observations 361
11.2.1 Bernoulli regression 361
11.2.2 Poisson regression 364
11.3 Local Quasi-Likelihood 366
11.3.1 Parametric quasi-likelihood 367
11.3.2 Nonparametric quasi-likelihood 369
11.4 Quasi-Likelihood LPA-ICI Algorithms 371
11.4.1 Zero-order regression 371
11.4.2 High-order regression 372

12 Photon Imaging 379

12.1 Direct Poisson Observations 381
12.1.1 Anscombe transform 381
12.1.2 Local quasi-likelihood 382
12.1.3 Recursive LPA-ICI(AV) algorithm 382
12.1.4 Numerical experiments 386
12.2 Indirect Poisson Observations 390
12.2.1 ML pure inverse 390
12.2.2 Richardson-Lucy method 391
12.2.3 EM pure inverse 393
12.2.4 Regularized inverse 396
12.2.5 LPA-ICI filtering 397
12.3 Local ML Poisson Inverse 400
12.3.1 Local ML for indirect observations 400
12.3.2 Linear inverse plus LPA-ICI filtering 401
12.3.3 Numerical experiments 405
12.4 Computerized Tomography 407
12.4.1 Emission tomography 407
12.4.2 Transmission tomography 410
12.4.3 Adaptive LPA-ICI algorithms 422

13 Multiresolution Analysis 427

13.1 MR Analysis: Basic Concepts 427
13.1.1 Orthogonal series 428
13.1.2 Wavelets 430
13.2 Nonparametric LPA Spectrum 440
13.2.1 Finite-difference LPA spectrum 441
13.2.2 Linear independence 443
13.2.3 Gram-Schmidt orthogonalization 444
13.2.4 Orthogonal LPA spectrum analysis 445
x Contents

13.2.5 Perfect reconstruction 446

13.2.6 MR analysis for differentiation 449
13.2.7 Examples of MR kernels 449
13.3 Thresholding 452
13.3.1 Basic concepts 452
13.3.2 Control of error rate 457
13.3.3 Image processing 458
13.4 Parallels with Wavelets 463

14 Appendix 467
14.1 Analytical Regular Grid Kernels 467
14.1.1 1D kernels 467
14.1.2 2D kernels 475
14.2 LPA Accuracy 480
14.2.1 Proof of Proposition 4 480
14.2.2 Proof of Proposition 7 481
14.3 ICI Rule 487
14.3.1 Proof of Proposition 10 487
14.4 Cross Validation 490
14.4.1 Basic approach 491
14.4.2 Algorithm 492
14.4.3 Shift-invariant kernels 495
14.5 Directional LPA Accuracy 496
14.5.1 Proof of Proposition 12 496
14.5.2 Convergence rate for signal estimation 502
14.5.3 Convergence rate for derivative estimation 506
14.6 Random Processes 510
14.6.1 Spectrum of random process 510
14.6.2 Minimization in frequency domain 512
14.6.3 Vector random processes 513
14.7 3D inverse 516
14.7.1 Regularized inverse 517
14.7.2 LPA regularized inverse 517
14.7.3 LPA regularized Wiener inverse 519
14.8 Nonlinear Methods 521
14.8.1 Proof of Proposition 13 521
14.8.2 Proof of Proposition 14 524
14.8.3 Proof of Proposition 15 529

References 535
Index 547
Preface

This book deals with a wide class of novel and efficient adaptive signal process-
ing techniques developed to restore signals from noisy and degraded observations.
Imaging provides a clear example of this type of problem. Digital images produced
by a variety of physical units, including still/video cameras, electron microscopes,
radar, x-rays, ultrasound devices, etc., are used for various purposes, including
entertainment, medical, business, industrial, military, civil, security, and scientific
applications. In today’s computer networks, the number of circulating digital images
is enormous and continues to increase rapidly. Digital images can become distorted
for many reasons, such as blurring and changes in illumination, motion, and process-
ing, including transmission, quantization, smoothing, compression, and geometric
transformations.
In many cases useful information and high quality must be extracted from the
imaging. However, often raw signals are not directly suitable for this purpose and
must be processed in some way. Such processing is called signal reconstruction.
This book is devoted to a recent and original approach to signal reconstruction
based on combining two independent ideas: local polynomial approximation and
the intersection of confidence interval rule. Local polynomial approximation is
applied for linear and nonlinear estimation using a polynomial data fit in a sliding
window. The window size, which is also interpreted as a scale, is one of the key
parameters of this technique. The terms “window size,” “bandwidth,” and “scale”
are synonymous here.
The local polynomial approximation is combined with an adaptive window size
selection procedure called the intersection of confidence interval rule. The idea of
this adaptation is as follows. The algorithm searches for the largest local area of the
point of estimation where the local polynomial approximation assumptions fit well
to the data. The estimates are calculated for a grid of window sizes and compared.
The adaptive window size is defined as the largest of those windows in the set for
which the estimate does not differ significantly from the estimators corresponding
to the smaller window sizes.
In the approach considered in this book, the intersection of confidence interval
rule defines the best scale for each point/pixel of the signal. In this way, we arrive
at a varying adaptive-scale signal processing (imaging) with a pointwise adaptive-
scale selection. The adaptive estimator is always nonlinear, even for a linear local
polynomial approximation, because the nonlinearity of the method is incorporated
xi
xii Preface

in the intersection of the confidence interval rule itself. It is proved that these
adaptive estimators allow one to get a near-optimal quality of the signal recovery.
The local polynomial approximation (LPA) plus the intersection of confidence
interval (ICI) rule is an efficient tool for signal (image) processing, especially for
denoising, interpolation, differentiation, and inverse problems.
The local approximation (nonparametric regression) techniques considered
here signify that no prior limit is placed on the number of unknown parameters
used to model the signal. Such theories of estimation are necessarily quite different
from traditional (parametric regression) statistical models with a small number of
parameters specified in advance.
Experiments demonstrate the state-of-art performance of the new algorithms,
which on many occasions visually and quantitatively outperform the best existing
methods.
Historically, the nonparametric regression technique is a predecessor of
wavelets. It demonstrates a tremendous breakthrough in adaptive methods overall.
This current development is almost unknown to the signal processing community,
which is mainly dominated by the wavelet paradigm.
In this book we present basic concepts, methodology, and theory of this modern,
spatially adaptive (nonparametric-regression-based) signal and image processing.
The advanced performance of the algorithms is illustrated by applications to various
image processing problems.
We present and interpret the nonparametric regression LPA as a filter design
technique.
The ICI adaptive-scale selection concerns adaptive neighborhood filtering. The
ICI as well as the nonparametric regression approach overall admit the derivation
of the adaptive algorithms from the optimization of the estimation accuracy. In this
way we develop a regular approach to the adaptive neighborhood filtering design
that is universally applicable to various problems.
We focus on material that we believe is fundamental and has a scope of appli-
cations that is not limited to solutions of specialized problems. Therefore, the book
is prepared for a broad readership. To avoid mathematical technicalities, we prefer
to postpone the presentation of proofs until the Appendix.
In addition to explanations of the formal mathematics required for background,
all methods are discussed in light of their implementations and applications.
Local polynomial approximation is an old idea. However, when combined
with the new adaptation technique, it becomes a novel and powerful tool. The
new approach and new algorithms are mainly illustrated for imaging applications.
However, they are quite general in nature and can be applied to any scalar or multi-
dimensional data. These new methods can be exploited as independent tools as well
as jointly with conventional techniques.
The emphasis of the book is on multivariate problems. However, the introduc-
tion guides the reader through the most important ideas and techniques relating to
univariate signals.
The mathematics used in the book is motivated by and restricted to the necessity
of making the ideas and algorithms clear. Its complexity remains at a level well
within the grasp of college seniors and first-year graduate students who have had
Preface xiii

courses in mathematical analysis, vectors, matrices, probability, and statistics. All

statements are explained in the text, and the proofs presented in the Appendix are
not necessary for understanding the algorithms. The more formal mathematical
sections marked by asterisks may be omitted on a first reading.
The book is written as a self-contained text and includes review comments on
the basic literature in each of the subjects treated.
Numerical experiments are necessary to fully understand the algorithms and
methods developed in this book. We provide MATLAB1 implementation of the basic
algorithms presented as well as programs used in the illustrating examples. These
programs are available at http://www.cs.tut.fi/∼lasip. We support this software by
updating the programs and including those programs that implement new algorithms
and methods.
We hope that readers will find this book highly valuable in their research, data
processing and analysis, and data interpretation.
The primary target readers are researchers, engineers, and graduate and post-
graduate students working in signal and image processing fields who develop and
study methods and algorithms. We trust that the book will be interesting to those
coming to the area for the first time as well as to readers who are more familiar
with the field. The book can be useful for teaching in electrical engineering depart-
ments. The secondary beneficiaries of the book are readers who develop and use
algorithms for specific applications, in particular, specialists working in the areas
of signal and image processing, communications, geoscience and remote sensing,
medicine, and biology imaging.
The aim of this book is to advocate the modern adaptive nonparametric tech-
niques that have proven efficient for image and signal processing and to provoke
further research in both theory and applications.
Many people have been of great help to our work on this book. In particular, we
thank Professors Alexandre Tsybakov (University 6, Paris) and Vladimir Spokoiny
(Weierstrass-Institute, Berlin) for fruitful discussions.
We especially thank our students, Alessandro Foi and Dmitry Paliy, for ded-
icated participation in the book project. We note the significant contribution of
Alessandro Foi for developing ideas and algorithms for directional anisotropic and
recursive image processing.

Tampere, Finland
April 2006
V. Katkovnik
K. Egiazarian
J. Astola

1
MATLAB
is a registered trademark of The Mathwork, Inc.
Notation and Abbreviations

Signals, variables, parameters

y true signal
z observed signal
h scalar/vector nonnegative scale parameter of estimate
ŷh estimate of y of scale h
ŷh(r) estimate of the derivative y (r) of scale h
x argument variable (integer or continuous)
Xs coordinates of observations
m order (power) of LPA
r order of the derivative
∆ sampling interval
φ vector of basis function in LPA
w window function
v kernel of blurring (convolution) operator
Γ threshold in the ICI rule
d dimensionality of signals
r = (r1 . . . , rd ) multi-index order of the derivative
m = (m1 , . . . , md ) multi-index order of LPA
h = (h1 , . . . , hd ) multi-index scale parameter
∆ = (∆1 , . . . , ∆d ) multi-index sampling interval

Symbols

g kernel of smoothing operator (filter)

g (r) kernel of rth differentiating operator (filter)
∂xr derivative operator: derivative on x of order r
(g  ∗ y)(x) convolution (discrete/continuous) of g and y
arg maxx f (x) the value of x that maximizes f (x)
arg minx f (x) the value of x that minimizes f (x)
σz standard deviation of random z
var{z} variance of random z
E{z} expected value of random z
cov{x, y} covariance of random x and y
xv
xvi Notation and Abbreviations

(·)T transpose of vector or matrix

(·)∗ complex conjugate of vector or matrix
(·)H Hermitian transpose of vector or matrix
δk,l Kronecker delta
δ(x) Dirac delta
O(x) a small value of order x
o(x) a small value of the order smaller than x

equal by definition
 equal approximately
∼ proportional
||x|| Euclidian norm of a vector x
Cα continuous differentiable functions with bounded pth-order
derivatives
Hα,γ Hölder class of functions
[z]+ [z]+ = z if z ≥ 0 and [z]+ = 0 if z < 0
a, b inner product of vectors a and b
·


integer part of number
R d = d-dimensional integration
1(x) = 1 indicator function
B −1 inverse matrix
B# pseudoinverse matrix
det B determinant of matrix

Sets
R scalar real numbers
R+ nonnegative scalar real numbers
Z integer numbers
Z+ nonnegative integer numbers
Rd space of real d-dimensional vectors
Rd+ space of non-negative real d-dimensional vectors
Zd space of d-dimensional vectors with integer elements
Zd+ space of d-dimensional vectors with integer nonnegative elements
L2 space of squared integrable functions
l2 (Zd ) space of squared summable sequences defined in Zd -space

Abbreviations
IFT integral Fourier transform
DFT discrete Fourier transform, discrete in both spatial and
frequency variables
IDFT inverse discrete Fourier transform
Notation and Abbreviations xvii

DTFT discrete time Fourier transform, discrete in time/spacial

variables and continuous in frequency variables
LSM least-square method
ML maximum likelihood
MSE mean-square error
LPA local polynomial approximation
ICI intersection of confidence intervals
LPA-ICI adaptive local polynomial approximation algorithm
equipped with ICI rule
LPA-ICI(AV) LPA-ICI algorithm with the variance-adaptive ICI
LPA(AV)-ICI(AV) LPA-ICI algorithm with the variance-adaptive ICI and
with variance-adaptive LPA estimation
N (m, σ2 ) Gaussian distribution with expectation m and variance σ2
i.i.d. independent identically distributed
p.d.f. probability density function
Chapter 1

Introduction

The book is devoted to signal and image reconstruction based on two independent
ideas: local approximation for the design of linear and nonlinear filters (estimators)
and adaptation of these filters to unknown smoothness of the signal of interest.
As flexible universal tools we use local polynomial approximation (LPA) for
approximation and intersection of confidence intervals (ICI) for adaptation.
The LPA is applied for linear filter design using a polynomial fit in a sliding
window. The window as well as the order of the polynomial defines a desirable
filter.
The window size is considered as a varying adaptation parameter of the filter.
The ICI is an adaptation algorithm. It searches for a largest local window size
where LPA assumptions fit well to the observations. It is shown that the ICI adaptive
LPA is efficient and allows for a nearly optimal quality of estimation.
To deal with anisotropic signals, in particular those that are typical in imaging,
narrowed nonsymmetric directional windows are used. The corresponding direc-
tional (multidirectional) LPA filters equipped with the ICI adaptive window sizes
demonstrate an advanced performance.
The LPA combined with the maximum likelihood and quasi-likelihood are used
for the design of nonlinear filters (estimators). The ICI rule used for the window
size selection of these filters defines nonlinear filters as adaptive to the unknown
smoothness of the reconstructed signal.
LPA-ICI algorithms are powerful recent tools in signal and image processing.
These algorithms demonstrate the state-of-art performance and can visually and
quantitatively outperform some of the best existing methods.
The ICI rule is universal and can be applied with many existing linear and
nonlinear filters (not only with the LPA-based filters) where the bias-variance trade-
off is valid for selecting tunable parameters.
The LPA-ICI techniques define a wide class of pointwise spatially/time adaptive
filters applicable in many scientific and engineering fields.
In this introduction we start with motivation, ideas, and constructive comments
to review basic elements of the techniques presented in the book. These com-
ments are presented for univariate signals while the book is addressed mainly to
multidimensional problems.
1
2 V. Katkovnik et al.

1.1 Linear Local Approximation

1.1.1 Windowing

Let y be a scalar function of a scalar argument x. We wish to reconstruct this

function using the observations

zs = y(Xs ) +
s , s = 1, . . . , n, (1.1)

where the observation coordinates Xs are known and the
s are zero-mean random
errors.
We make no global assumptions about y but assume that locally it can be well
approximated with members of a simple class of parametric functions. Taylor’s
series can serve as a good approximation of a smooth function.
In a neighborhood of x this series gives for y(Xs ):

y(Xs )
y(x) − y (1) (x)(x − Xs ) + y (2) (x)(x − Xs )2 /2 + · · · . (1.2)

Since the function y and the derivatives y (1) and y (2) are unknown in Eq. (1.2),
we may look to fit data y(Xs ) in the form

y(Xs )
C0 − C1 (x − Xs ) + C2 (x − Xs )2 /2 + · · · , (1.3)

where the coefficients C0 , C1 , and C2 in Eq. (1.3) are used as estimates for y(x),
y (1) (x), and y (2) (x), respectively.
Modeling in the form (1.3) allows this fruitful interpretation of the coefficients
Ck as the derivatives of y.
Note that we use the first-order polynomial x in Eq. (1.3) with the minus in
order to have C1 as the estimator of y (1) (x). Of course, Eq. (1.3) can be written with
positive signs for all polynomials,

y(Xs )
C0 + C1 (x − Xs ) + C2 (x − Xs )2 /2 + · · · , (1.4)

but then the estimate of y (1) (x) is −C1 .

In order to preserve the meaning of the coefficients Ck as the estimators of the
derivatives we always use the odd-order polynomials in the model (1.3) with the
negative signs.
How do we formalize a local fit? The weighted least-square method gives the
following criterion:


J (x, C) = w(x − Xs )es2 , es = zs − y(x, Xs ),
s

y(x, Xs ) = C0 − C1 (x − Xs ) + C2 (x − Xs )2 /2, (1.5)

where w is a window function; y(x, Xs ) denotes a polynomial approximation; es

is a residual between the observation zs and the approximation y(x, Xs ); and C is
a vector with components C0 , C1 , and C2 .
Introduction 3

The window function w is “centered” on the point of interest x. The coordinate

x positions the window, and w(x − Xs ) in Eq. (1.5) defines a set of neighborhood
observations Xs and their weights in estimation for x.
Figure 1.1 illustrates this sliding window concept showing w(x − Xs ) for a few
x and corresponding sets of observations in Eq. (1.5).
Minimizing Eq. (1.5) on C, one may find an estimate of the parameter C:

C(x) = arg min J (x, C). (1.6)
C

It is clear that, in general, the coefficients C are different for each x. The
 emphasizes that the estimate of C depends on x. The sliding window
notation C(x)
estimation makes the coefficients C in the model (1.3) varying on x.
The model (1.3) can be zero, first, second, or higher orders depending on the
power of used polynomials. The parameter C(x)  immediately gives estimates of
(r)
the function y and the derivatives y :
0 ,
ŷ(x) = C 1 ,
ŷ (1) (x) = C 2 .
ŷ (2) (x) = C (1.7)

1.1.2 Nonparametric estimation

The sliding window estimates (1.5)–(1.6) belong to the class of so-called nonpara-
metric estimates. Let us clarify what we mean here by parametric and nonparametric
estimation.
For a parametric estimation, the criterion (1.5) and the estimates (1.6) are
transformed to


J (C) = ws es2 , es = zs − y(Xs ),
s (1.8)
y(Xs ) = C0 − C1 Xs + C2 Xs2 /2,

Figure 1.1 A Gaussian sliding window centered at the point x shows data sets entered in
the estimate for the point x and the weights of the included observations.
4 V. Katkovnik et al.

where the weights ws are fixed and

 = arg min J (C).
C (1.9)
C

A zero-order parametric model with y(x) = C0 gives an estimate ŷ(x) = C 0

constant for all x. The first-order parametric model y(x) = C0 − C1 x gives the
estimate ŷ(x) = C 0 − C1 x as linear on x, and the second-order parametric model
y(x) = C0 − C1 x + C2 x 2 /2 obviously gives the estimate as quadratic on x.
The order of the polynomial model completely defines the type of estimation
curve (constant, linear, or quadratic).
It is quite different for the sliding window estimates (1.5)–(1.6), where the order
of the polynomial does not specify explicitly the type of estimation curve, which
essentially depends on observed data.
The windowed estimate (1.6) cannot be represented as a polynomial function
of the order given in the local model. In slightly different terms, the estimate curve
(1.5)–(1.6) cannot be parameterized as a polynomial function of the given order,
thus explaining the terms parametric and nonparametric estimation.
As an important technical point defining the difference between the parametric
and nonparametric estimates, note that the weights ws are constant in Eq. (1.8) and
vary with x in Eq. (1.5).
The models of the signal are used in very different ways in the nonparametric
and parametric estimations. In the parametric approach, the model ŷ(x) = C 0 −
C 
1 x + C2 x /2 is global and used in this form for all x. Thus, if we need to calculate
2

the derivatives, they are ŷ (1) (x) = −C 1 + C

2 x and ŷ (2) (x) = C
2 .
In the nonparametric approach we use the model ŷ(x, Xs ) = C 0 (x) −
1 (x)(x − Xs ) + C
C 2 (x)(x − Xs )2 /2 in Eq. (1.5) for Xs = x only, giving

0 (x).
ŷ(x) = ŷ(x, Xs )|x=Xs = C

For the estimation of the derivatives we have

1 (x)
ŷ (1) (x) = ∂Xs ŷ(x, Xs )|x=Xs = C

and
2 (x).
ŷ (2) (x) = ∂X2 s ŷ(x, Xs )|x=Xs = C

Thus, in the nonparametric estimation, the model is explicitly local and used
in a local pointwise manner. The coefficients in Eqs. (1.5–1.6) are calculated for a
fixed x and the model is used for this x only.
The dependence on x in the parametric case is defined by the basis functions
(polynomials) of the model and the coefficients are fixed. In the nonparametric
case, everything is completely different: The dependence on x is defined by the
1 (x), C
0 (x), C
varying coefficients C 2 (x) while the basis functions are used with a
fixed zero value of the argument.
This nonparametric nature of the windowed estimation follows from the tech-
nique used for each x in Eq. (1.5) when fit with different weights for different data.
Introduction 5

The localization relaxes the sensitivity of the standard parametric models to the
order of the model and transforms the polynomial fit into a much more flexible tool
than can be expected from the standard parametric approach.
The estimate ŷ(x) = C 0 (x) (as well as the estimates of the derivatives) can be
given in the form of the kernel operator (filter) linear on the following observations:


ŷ(x) = g(x, Xs )zs . (1.10)
s

The following notation is used here:

g(x, Xs ) = w(x − Xs )φT (x − Xs )Φ−1 φ(0),

(1.11)
Φ= w(x − Xs )φ(x − Xs )φT (x − Xs ),
s

where φ(x) = (1, −x, x 2 /2, . . .)T is a vector of polynomials of the LPA. This is
a general formula where the length of the vector φ is equal to the order of the
polynomials plus one.

1.1.3 Scale

The size of the window function w in Eq. (1.5) is an important and natural parameter
of the local approximation. If the window is large and prescribes equal weights to
all residuals, then the nonparametric estimate coincides with the parametric one.
The difference between the parametric and nonparametric approaches disappears.
In this case, the estimation curve is constant, linear, or quadratic, depending on the
polynomial degree used in the model, and enables a maximal smoothing of random
noise in observations.
If the window is of a minimal size, then the estimation curve goes exactly
through the nearby observation points and there is no smoothing of the data. In this
way the window size controls the level of smoothing enabled by the estimator.
The curves in Figs. 2.6 and 2.9 in Chapter 2 illustrate the role of window size
(with notation h for this parameter). The zero-order LPA is used for filtering a finite
number of observations in Fig. 2.6. For a large window size (large h) the estimate is
a constant equal to the mean value of the observations. For a very small window size,
the estimate is a piecewise constant going exactly through the observed values of the
signal. For intermediate values of the window size, we obtain curves demonstrating
different smoothness and different closeness to the observations.
The first-order LPA is used for filtering the same observations in Fig. 2.9. For
the large window size, the estimate is a linear function corresponding to the linear
least square fit of the signal. For a very small window size, the estimate is nearly
piecewise linear going through the observation points. For intermediate values of
the window size, we have curves demonstrating different smoothness and different
closeness to the observations.
6 V. Katkovnik et al.

Mathematically, the window size is introduced by a special scale parameter h

with the window function w in Eq. (1.5) replaced by
wh (x) = w(x/ h)/ h, h > 0. (1.12)
A smaller or larger h narrows or widens the window wh (x), respectively.  The
scalar factor
 1/ h in Eq. (1.12) serves for normalization of the window. If w(x)dx =
1, then w(x/ h)/ hdx = 1 for all h > 0.
With this scale parameter, the estimate (1.10) is transformed to


ŷh (x) = gh (x, Xs )zs (1.13)
s

with the notation

gh (x, Xs ) = wh (x − Xs )φTh (x − Xs )Φ−1
h φh (0),

(1.14)
Φh = wh (x − Xs )φh (x − Xs )φTh (x − Xs ).
s

We use the scale parameter h in the window function wh and for the polynomials.
In the above formulas,
φh (x) = φ(x/ h).
Thus, the polynomials in φh (x) are also scaled by h.
The index h in ŷh and gh emphasizes the special role of the scale parameter h
for the estimate as well as for the kernel of the estimator.
With this notation, the local polynomial model in Eq. (1.5) has the form
yh (x, Xs ) = φTh (x − Xs )C. (1.15)
Under some appropriate assumptions, the discrete local estimator (1.13)
becomes shift-invariant, and for the continuous data it can be represented in the
form of the integral kernel transformation:
   
1 x−u
ŷh (x) = g z(u)du = g(u)z(x − hu)du, (1.16)
h h
where g is a kernel depending on the window w and the order of the polynomials,
g(x) = wh (x)φT (x)Φ−1 φ(0),

Φ = w(x)φ(x)φT (x)dx. (1.17)

The integral form (1.16) makes obvious an interpretation of the window size h
as the scale parameter of the estimate and of the LPA.
In the first formula in Eq. (1.16), h defines the scale of the kernel g((x −
u)/ h)/ h convolved with the signal z(u) independent on h, while in the second
formula h defines a scale of the signal z(x − hu) convolved with the kernel g(u)
independent on h.
Introduction 7

The following moment conditions hold for the LPA kernels of the order m:

g(x)dx = 1, for m = 0,
 
g(x)dx = 1, xg(x)dx = 0, for m = 1, (1.18)
  
g(x)dx = 1, xg(x)dx = 0, x 2 g(x)dx = 0, for m = 2.

Using the Taylor series,

y(x − hu) = y(x) − hy (1) (x)u + h2 y (2) (x)u2 /2 − h3 y (3) (x)u3 /6,
and inserting it into the second formula in Eq. (1.16) instead of z(x − hu), we find

ŷh (x) = y(x) − hy (x) g(u)udu + 0(h2 ),
(1)
for m = 0,

ŷh (x) = y(x) + h2 y (2) (x) g(u)u2 du/2 + 0(h3 ), for m = 1, (1.19)

ŷh (x) = y(x) − h y (x) g(u)u3 du/6 + 0(h4 ),
3 (3)
for m = 2.

All of these estimates give the true value of y(x) as the main term of the series.
The error of the estimation is proportional to h for m = 0, to h2 for m = 1, and to
h3 for m = 2. For small h this shows the order of the error, which depends on the
order of the polynomial approximation. In any case, if h → 0, the estimate becomes
accurate as ŷh (x) → y(x).
The parameter h controls the smoothing of y. Larger and smaller h result in
stronger and weaker smoothing of y, respectively.
In the frequency domain, the Eq. (1.16) becomes
h (λ) = G(λh)Y (λ),
Y (1.20)
where the capital letters denote the integral Fourier transform of the corresponding
variable; in particular, we have for y(x)

Y (λ) = F {y(x)} = y(x) exp(−iλx)dx.

The filter G(λ) = F {g(x)} obtained by the local polynomial approximation is

a typical low-pass filter such that G(0) = 1 and |G(λ)| → 0 as |λ| → ∞.
The parameter h scales/dilates the frequency characteristic of the filter
(Fig. 1.2), and defines a bandpass width of the filter, which is narrowed with h → ∞
and widened with h → 0.
Various terms are used in publications for the parameter h, such as window
size, scale, bandwidth, or dilation. In this book the term scale is mainly used.
The integral form of the local approximation operator clearly reveals the scaling
role of the parameter h.
8 V. Katkovnik et al.

Figure 1.2 Amplitude frequency characteristics of the continuous kernels for different scales
h1 < h 2 < h 3 .

1.1.4 Ideal scale

If the observation model includes the random noise in Eq. (1.1), the estimation error
ey (x, h) = y(x) − ŷh (x) = mŷh (x, h) + ey0 (x, h)

consists of the systematic mŷh (x, h) and random ey0 (x, h) components

mŷh (x, h) = y(x) − E{ŷh (x)},

ey0 (x, h) = ey (x, h) − mŷh (x, h).
In the asymptotic analysis the systematic error is evaluated according to
Eq. (1.16) as

mŷh (x, h) = y(x) − g(u)y(x − hu)du.

Assuming that the noise is independent with an identical distribution for all
observations, the variance of the random components is defined as

2∆
σŷh (x, h) = σ Bg ,
2
where Bg = g 2 (u)du, (1.21)
h
∆ is a small sampling period, and σ2 is the variance of the noise in Eq. (1.1).
According to Eq. (1.19), we may conclude that the main term of the systematic
error is
|mŷh (x, h)| = hm+1 |y (m+1) (x)|Ag , (1.22)

where
 m is the order of the polynomial approximation and Ag =
| g(u)um+1 du|/(m + 1)!.
Introduction 9

The estimation variance (1.21) is a monotonically decreasing function of h

while the bias (1.22) is the monotonically increasing function of h. This means that
there is a bias-variance balance giving the optimal value h∗ to the scale h.
The deviation from this optimal value means that the bias is increasing and
the variance is decreasing if h > h∗ , and the bias is decreasing and the variance is
increasing if h < h∗ .
This optimal h can be found by minimizing the mean-square error:
lŷh (x, h)
E{ey2 (x, h)} = m2ŷh (x, h) + σŷ2h (x, h).
Inserting Eqs. (1.21) and (1.22) into this formula, we have the mean-square
risk function,
 2 ∆
lŷh (x, h) = hm+1 Ag y (m+1) (x) + σ2 Bg ,
h
convex on h.
The equation ∂lŷh (x, h)/∂h = 0 gives, for the optimal scale,
 1/(2m+3)
σ2 Bg ∆
h∗ = . (1.23)
2(m + 1)[Ag y (m+1) (x)]2
This optimal solution is called the ideal or oracle scale and depends on the derivative
y (m+1) (x), which is unknown. The ideal scale cannot be used in practice without
estimation of this derivative. Nevertheless, it gives important information about the
behavior of the optimal scales and estimates. In particular, it shows that the optimal
scale at point x should be large or small for small or large |y (m+1) (x)|, respectively.
Another conclusion is that the varying scale is crucial in the areas where the
behavior of y is essentially varying on x because it is identified by the derivative
y (m+1) (x). In the areas where this derivative is more or less invariant, the invariant
scale can be exploited.
Figure 1.3 illustrates the idea of the bias-variance trade-off. Note that the ideal
scale h∗ is always smaller than the h corresponding to the equality
|mŷh (x, h)| = σŷh (h).
Thus, in the ideal balance of the bias versus the variance, the bias is smaller than
the standard deviation of error.

1.1.5 Adaptive varying scale

Theoretical analysis and experiments show that the local approximation estimates
can be efficient only with a correct selection of the scale h. It can be varying or invari-
ant, but it should be properly selected. The scale selection is a necessary element
of a successful local approximation estimation. Without automatic scale selection,
the local approximation, even though it is a reasonable idea, does not work.
In signal processing and statistics, scale selection as it is defined above is the
subject of many publications exploiting different ideas and techniques.
10 V. Katkovnik et al.

Figure 1.3 The squared bias |mŷh (x, h)|2 and the variance ŷ2h (x, h) illustrating the bias-
variance trade-off. The “ideal” balance is achieved at h = h∗ minimizing the mean-square
loss function lŷh (x, h).

Recently, a novel class of algorithms known under the generic name Lepski’s
approach has been introduced in statistics and shown to be efficient in theory and
practice. These algorithms are proposed for the pointwise varying-scale adaptive
nonparametric estimation. In this book we develop as the basic adaptation algorithm
one version of this approach, called the intersection of confidence interval (ICI) rule.
The algorithm is simple in implementation. Let H be a set of the ordered scale
values H = {h1 < h2 < · · · < hJ }. The estimates ŷh (x) are calculated for h ∈ H
and compared.A special statistic is exploited to identify a scale close to the ideal one.
This statistic needs only the estimates ŷh (x) and the variances of these estimates,
σŷ2h (x, h), both calculated for h ∈ H .
A layout of the LPA-ICI adaptive algorithm is shown in Fig. 1.4. The LPA
designed filters with the kernels gh1 , . . . , ghJ are applied to the input signal z. The
outputs of these filters ŷh1 , . . . , ŷhJ are inputs of the ICI algorithm, which defines
the adaptive-scale estimates ŷh+ (x) in a pointwise manner for each x.

Figure 1.4 A layout of the adaptive-scale LPA-ICI algorithm.

Introduction 11

1.2 Anisotropy

1.2.1 Univariate case

The local polynomial approximation motivated by the Taylor series in Eq. (1.2)
is based on the assumption that the function y is smooth and differentiable. This
means that (at least locally) the behavior of the function is more or less the same
in all directions around the point of interest x. This sort of isotropy assumes, in
particular, that the following three definitions of the derivative are equivalent:
y(x + h) − y(x)
∂y/∂x = lim
h→0 h
y(x) − y(x − h)
= lim
h→0 h
y(x + h/2) − y(x − h/2)
= lim .
h→0 2h
Thus, for derivative estimation we can use the right, left, or symmetric finite-
differences and the results are approximately equal. This simple hypothesis, which
is used by default in most signal processing algorithms, does not hold at singular
points or so-called change points, where the behavior of y changes rapidly.
Directional local approximation is developed to deal with the anisotropy of y.
For univariate y, the directionality of estimation assumes the use of right and
left windows. We say that the window is right or left if w(x) = 0 for x < 0 or
x > 0, respectively. This allows us to get good approximations near the jump points
of curves.
Let us look at a simple example. The signal is piecewise constant having zero
value for x < a and a value of 1 for x ≥ a (Fig. 1.5a). The observation noise is
an independent standard Gaussian. Let the estimator be a sample mean (symmetric
window estimate) with the estimate for the point x 0 in the form
1

ŷh (x 0 ) = zs ,
# 0|Xs −x |≤h

where # is a number of observations used in the estimate.

Figure 1.5(a) shows the estimation intervals for different h. It is obvious that
for x 0 < a the best ideal estimator should have h = a − x 0 , because in this case the
sample mean estimate is unbiased (the signal has a constant value on this interval)
with the minimum variance while the estimation interval is maximal. Figure 1.5(b)
illustrates the nonsymmetric sample means when the right/left means (right/left
windows) are used with varying window sizes:
1

ŷhright (x 0 + 0) = zs ,
# 0 0
x ≤Xs ≤x +h

1

ŷhlef t (x 0 − 0) = zs .
#
x 0 −h≤Xs <x 0
12 V. Katkovnik et al.

Figure 1.5 Symmetric, right/left estimation, and scale selection.

In order to emphasize their differences with respect to the symmetric window

estimate, we consider the estimation at the jump point x = x 0 . The plus and minus
zero in the argument of the estimate mean the plus and minus of some small values,
respectively. This is used to make the function well-defined at the point of estima-
tion. The argument x 0 + 0 says that the estimation point is from the right-hand side
of the jump point, while the argument value x 0 − 0 corresponds to estimation for
the point from the left-hand side of the jump.
It is clear that the right window estimator is well-suited for estimation at x =
x 0 + 0, because it can use a large window size estimate with a maximum size
b − x0.
Both the symmetric and left window estimates are not able to work well in this
case because they can use only small windows (which can be very noisy) or the
larger window estimates (with a definitely large bias error).
In a similar way the left window estimate works well for x = x 0 − 0 with the
maximum window size x 0 − c, while the right and symmetric estimators are not
able to give good results because they can use only very small windows.

1.2.2 Multivariate case

Points, lines, edges, and textures are specific elements in images. They are locally
defined by position, orientation, and size. Often being of small size, these specific
features encode a great proportion of information contained in images. The cor-
responding image intensity is a typical example of an anisotropic function y and
points, lines, and edges are singular change points and change surfaces in images.
Introduction 13

What makes the problem more specific is that reconstruction of y near the change
points is of prime interest because these points present the most valuable imaging
information.
For a multivariate y for directional estimation we consider a ball of radius h
defining a spherical neighborhood of x,

Bh = {u : ||x − u|| ≤ h}. (1.24)

Next we introduce a sectorial partition of this ball with K nonoverlapping sectors

Sθh having x as a common vertex. Here θ gives a direction of the sector and h is a
length of the sector.
One can exploit the LPA to design the estimate and the estimate kernel for
x using the sectors as the estimate support. In this way, we obtain K estimates
ŷh,θi (x), i = 1, . . . , K, with K nonoverlapping supports Sθhi covering the ball Bh .
The ICI is used to find the adaptive varying scale for each sector inde-
pendently. Thus, we arrive at K adaptive-scale estimators ŷh,θi (x), which have
h+ (θ )
different adaptive scales/sizes for each sectorial support Sθi i (Fig. 1.6a). The
union of the supports of these sectors is not a ball Bh of radius h, but a star-
shaped set that can have a sophisticated configuration (Fig. 1.6b). Being defined
by the ICI rule, this star-shaped set is a shape-adaptive neighborhood of the x
used for estimation. In this way we obtain an adaptive pointwise segmentation of
the image.
Figure 1.7 illustrates the implementation of the concept of the adaptive star-
shaped neighborhoods for Lena test images for a few pixels in the image. These
adaptive sets cover nearly perfect neighborhoods with more or less homogeneous
behavior of the intensity function where the smooth polynomial approximation
works well.
The adaptive scaling and segmentation can be used for many purposes—in
particular, in edge detection, motion estimation, pattern recognition, and so on.
In denoising, the final estimate is calculated as a weighted mean of the sectorial
estimates. Thus, we obtain a highly efficient anisotropic adaptive-scale estimator
with the angular resolution depending on a number of sectors K.

(a) (b)

Figure 1.6 A star-shaped adaptive neighborhood used in anisotropic directional estimation.

14 V. Katkovnik et al.

Figure 1.7 An LPA-ICI pointwise adaptive segmentation of the Lena test image.

1.3 Nonlinear Local Approximation

1.3.1 Likelihood and quasi-likelihood

Our previous discussion of the LPA concentrated on the object function y, mainly
ignoring the noise in the observation model (1.1). However, the noise in Eq. (1.1) is
an equally important element of the observation model that becomes an extremely
sensitive issue when noise characteristics (e.g., the variance) are signal-dependent.
To deal with this sort of complex scenario, local versions of the likelihood and
quasi-likelihood methods are developed. These methods allow us to exploit explicit
probabilistic characteristics of observations to obtain optimal noise reduction
procedures.
In statistics, the observation model usually means a joint probability density
f (z, y) of the observation pairs {zs , Xs }s=1,...,n . In this probability density
the argument z corresponds to observations and y is a parameter of interest,
usually y = E{z}. This modeling is quite different from the more restrictive
standard additive model (1.1).
The maximum likelihood (ML) is a universal statistical technique that is used if
the distribution of observations is known. The optimality of the ML estimate means
that this method is able to extract full information about the estimated parameters
available in the given observations.
The local ML is a nonparametric counterpart of the widely used parametric
ML. It extends the scope of the parametric ML to a broader class of functions y.
The local likelihood does not assume a global parametric modeling of y but fits this
model locally within the sliding window.
The local likelihood and local log-likelihood use the window function w for
localization in a neighborhood of the point of interest x and replace the global
parametric model with a local one.
Using the polynomial linear model (1.15), the local log-likelihood can be
presented in the form


Lh (x, C) = ln f (zs , yh (x, Xs ))wh (x − Xs ),
s
(1.25)
yh (x, v) = C T φh (x − v), φh (x) = φ(x/ h),
wh (x) = w(x/ h)/ h,
Introduction 15

where h stays for the scale parameter used both in the window function w and for
the scaling of the polynomials φ.
For y we use the model yh (x, v) linear on C. This type of model is known in
statistics as a generalized linear model. For the local log-likelihood it becomes a
local (nonparametric) generalized linear model.
The standard nonlocal likelihood uses the global parametric model on C, which
is assumed to be valid for all x. The local likelihood relaxes this assumption and
assumes that the used linear model is valid only locally in some neighborhood of
the estimation point x.
Following the idea of the local nonparametric estimation, we use the criterion
Lh (x, C) for computing C. The model yh (x, v) is exploited in the pointwise manner
only. In this way we arrive at the following local ML version of the LPA:
 = arg max Lh (x, C),
C
C
(1.26)
T
ŷh (x) = C φh (0).

For Gaussian noise with a constant variance, this approach gives the linear LPA
estimates in Eq. (1.10).
In general, the estimates in Eq. (1.26) are nonlinear. This nonlinearity concerns
two different aspects of the problem. First, the equation

∂Lh (x, C)/∂C = 0

is nonlinear with respect to C, which means that there is no analytical solution for
 and numerical methods should be used for maximization in Eq. (1.26). Second,
C
the estimate ŷh (x) is nonlinear with respect to the observations zs .
There exist many circumstances in which, even though the full likelihood is
unknown, one can specify the relationship between the mean and the variance. In this
situation, estimation of the mean (regression, local regression) can be achieved by
replacing the log-likelihood by a quasi-likelihood Q(z, y). The quasi-likelihood has
been proposed as a distribution-free method (note that the ML is a distribution-based
method).
One assumes that we have independent observations with the expectations y
and the variance of these observations is modeled as

σ2 (y) = σ2 ρ(y), (1.27)

where σ2 is unknown and ρ is a given positive function of y called a variance

function. Thus, the variance is considered as a product of some given function of
the estimated y and the unknown invariant scalar σ2 .
In many applications the noise that corrupts the signal is signal-dependent,
the most widely encountered types being multiplicative, Poisson, film-grain, and
speckle noise. Their common feature is that the variance of the noise is related to
the magnitude of the signal.
There are many adaptive-neighborhood approaches to filtering signals and
images corrupted by signal-dependent noise. Instead of using fixed-sized, fixed-
shaped neighborhoods, statistics of noise and signals are computed within
16 V. Katkovnik et al.

variable-sized, variable-shaped neighborhoods that are selected for every point of

estimation.
In the developed quasi-likelihood approach we are concerned with this class
of problems. The quasi-likelihood is used to derive the estimate relevant to the
observation model and then the ICI enables the neighborhood adaptivity close to
the ideal one.

1.3.2 Robust M-estimation

The term “M-estimate” is used for estimates obtained by minimizing a sample

mean of a nonquadratic residual loss function. The M-estimation nonlinear version
of the linear LPA estimate (1.5) is different by a nonquadratic residual loss function
ψ(·):
 h) = arg min Jh (x, C),
C(x,
C∈RM


Jh (x, C) = wh (x − Xs )ψ(es ), (1.28)
s

es = zs − y(x, Xs ), yh (x, Xs ) = C T φh (x − v),

with the function estimate in the usual form,
T φh (0).
ŷh (x) = C (1.29)
Here yh (x, Xs ) is the LPA of the signal y as it is exploited in Eq. (1.5). The vector C is
an unknown parameter of this model found by minimization of the nonquadratic cri-
terion Jh (x, C) in Eq. (1.28). Equations (1.28)–(1.29) define a LPA nonparametric
M-estimate of y(x).
The median estimates corresponding to the standard median with y(x, Xs ) =
C0 in Eq. (1.28) and the local polynomial median with the power m > 0 for y(x, Xs )
are particular cases of M-estimates with
ψ(e) = |e|. (1.30)
There are various ideas behind the selection of the loss function ψ for M-estimation.
One of them is to design a robust estimate. The concept of robustness has numerous
and different interpretations in literature. Huber’s minimax estimation theory pro-
poses a regular approach for designing the robust estimates in the sense that these
estimates are insensitive (to some extent) with respect to variations of the unknown
noise distribution.

1.3.3 Adaptive varying scale

Under unrestrictive assumptions the ICI adaptive-scale selection rule is applicable

for the nonlinear nonparametric estimates (1.26) and (1.28). The estimates ŷh and
their variances should be calculated for a set scale, h ∈ H , as the input of the ICI rule.
Introduction 17

In general, the scale-adaptive nonlinear estimates are much more demanding

than the linear ones in terms of the required computational load, in that they concern
calculation of both the estimates and the variances. For one particular case, when
the LPA of the zero power is used, the quasi-likelihood and the maximum likelihood
estimates (for the exponential class of distributions) become linear estimates. The
ICI yields neighborhoods where the unknown signal is nearly constant. Thus, by
using the linear estimates in conjunction with the ICI rule, the pointwise segmenta-
tion of the data is produced into neighborhoods with nearly constant values of the
underlining signal.
Fast algorithms are developed for nonlinear estimates of Eqs. (1.26) and (1.28)
that exploit this idea. The linear estimates are calculated for all scales, and the
ICI algorithm selects the maximum scales where the signal continues to be nearly
constant. In this way the difficulties of nonlinear estimations are overcome and
algorithms that are incomparably faster than any standard nonlinear adaptive-scale
estimators are developed.

1.4 Multiresolution Analysis

Based on the LPA linear filters we introduce a novel multiresolution (MR) non-
parametric regression analysis. This MR approach gives varying-scale adaptive
estimators that are quite different from those defined by the ICI algorithm. Instead
of selecting the estimate with a single best scale h, we build a nonlinear estimator
using the estimate ŷh with all available scales h ∈ H .
The denoising is produced in two steps. The first step transforms the data into
noisy spectrum coefficients (MR analysis). In the second step, these noisy estimates
of the spectrum are filtered by thresholding procedures and then exploited for esti-
mation (MR synthesis). In this way an extension of the conventional nonparametric
regression approach is achieved and a wider class of adaptive-scale estimators with
potentially better performance is obtained.

1.5 Imaging Applications

One fundamental branch in signal processing concerns image reconstruction

problems—the restoration of an optically and electronically corrupted digital image
or a sequence of digital images. Corruption can be the result of some indirect obser-
vations (experiments) when what we observe is treated as a linear or nonlinear
transformation of the object of interest. The problem is to reconstruct the image
relevant to the observed phenomenon.
We illustrate performance of the developed techniques considering the follow-
ing imaging problems:

• Denoising with invariant variance Gaussian noise,

• Denoising with signal-dependent noise,
• Differentiation of noisy images and edge detection,
18 V. Katkovnik et al.

• Motion estimation from an image sequence,

• Deblurring of noisy images,
• Inverse halftoning,
• Super-resolution imaging from a sequence of images,
• 3D image deblurring (3D microscopic imaging),
• Photon-limited imaging with direct and indirect (blurred) observations subject
to a Poisson distribution,
• Computerized tomography.

The developed imaging techniques are universal and can also be applied in the
following areas where our current interests are concentrated:

• Blind deblurring with parametric and nonparametric uncertainty of the point

spread function;
• Focus-from-defocus, depth-from-defocus, and 3D imaging from defocus;
• Imaging for PET and SPET tomography;
• SAR and ISAR image processing;
• Time-frequency analysis;
• Digital holography.

1.6 Overview of the Book

In Chapters 2 and 3, basic local approximation ideas and techniques are presented,
and the roles of the window function and scale are discussed. The polynomial
smoothness of LPA kernels is defined in terms of the vanishing moment equations
and in the frequency domain. It is shown that the designed filters are reproducing
(accurately) for the polynomial signals. The nonparametric approach is exploited
for filtering, interpolation, and differentiation.
Shift-invariant kernel estimates are derived for regular grid signals. All results
are given in multivariate notation equally applicable for uni- and multivariate
signals.
In Chapter 4, the integral local polynomial approximation is studied. These
continuous variable filters and transformations are important on their own as well
as limit representations for the discrete ones. The limit integral forms are used for
accuracy analyses of discrete filters and for accuracy optimization. The integral
LPA estimates equipped with scale parameters are treated as delta-sequences for
estimation and differentiation. Corresponding links with a theory of generalized
functions are also discussed.
The accuracy analysis of the discrete LPA estimates is the subject of Chapter 5.
Formulas for the bias are derived for some classes of signals, and the ideal scale
balancing the bias and variance of the estimates is studied.
Adaptive-scale selection is the subject of Chapter 6. The varying adaptive-scale
selection is considered to be an efficient tool to deal with nonstationary signals
having spatially varying properties. Lepski’s general approach and the intersection
Introduction 19

of confidence (ICI) rule are introduced as basic adaptation tools for spatially varying
adaptation. The ICI rule is derived from optimization of the mean-square error. The
idea as well as the implementation and properties of the ICI rule are presented.
Simulation experiments demonstrate a high sensitivity of the ICI rule to singularities
and change points in signals and its efficiency for adaptive filtering.
The directional LPA is presented in Chapter 7 as a further development of the
local approximation idea. It is proposed to deal with the anisotropy of signals having
different behaviors in different directions. The directionality of the approximation
is enabled by narrowed sectorial windows. The directional kernels use polynomials
of different orders and different scales for different directions. The designing of
this sort of filter for estimation (smoothing) and differentiation is given.
The directional ICI adaptive-scale estimation is the subject of Chapter 8. The
main goal is to obtain a shape-adaptive neighborhood for the best approximation
of anisotropic signals. The approach is based on a sectorial partition of the local
neighborhood with adaptive sizes of the sectors defined by the ICI rule. A bank of
scale-adaptive directional filters is used with the final estimate found by aggregation
of the directional estimates.
Recursive LPA-ICI adaptive algorithms are proposed as a special class of adap-
tive filters with a cyclic repetition of the multidirectional LPA-ICI filtering. The
advance performance of the LPA-ICI directional algorithms is demonstrated for
image denoising problems.
For differentiation, a novel concept of the anisotropic gradient is developed.
The algorithm looks for the adaptive neighborhood where the signal allows a linear
local approximation and the gradient is calculated using observations only from
this neighborhood. Specific features of this concept are illustrated for shading-
from-depth problems.
Scale-adaptive image deblurring techniques are considered in Chapter 9. A
basic topic concerns the deblurring of a noisy image provided that the point-spread
function of the blur is shift-invariant. Developed algorithms combine the regularized
inverse and regularized Wiener inverse filtering procedures, both using the LPA-ICI
scale-adaptive technique. Fast implementation of the algorithms is based on using
convolution operations for computing.
The ICI scale adaptivity is proved to be very efficient. It is shown that the newly
developed algorithms demonstrate state-of-the-art performance. This technique is
generalized for 3D deblurring problems.
Nonlinear nonparametric estimation (filtering) is the subject of Chapter 10.
Observation modeling and performance criterion are considered as the main reasons
to replace linear estimates by more demanding nonlinear ones. At first, a class of
nonparametric robust estimates based on LPA and the M-estimation is considered.
Links between the robust and minimax estimations are discussed.Accuracy analyses
and scale optimizations show that the ICI rule is applicable for the adaptive varying-
scale selection. Adaptive-scale median filters are considered as a special class of
these adaptive M-estimates.
The nonlinear Box-Cox and Anscombe transformations allow the stabilization
of the variance in observations. In particular, this approach is efficient to deal with
the Poisson distribution data. Using this concept we develop adaptive algorithms
20 V. Katkovnik et al.

where the nonlinear estimation is reduced to a linear one with nonlinear transforms
before and after filtering.
A general approach to nonlinear estimation problems based on the local like-
lihood and quasi-likelihood is considered in Chapter 11. The LPA of the signal
of interest is used as the basic design tool. The accuracy results for the bias and
variance of the estimates are obtained. It is shown that the only difference versus
the corresponding results for the linear estimate concerns the formulas for the vari-
ance. It follows that the ICI rule can be modified and used for the nonlinear adaptive
varying-scale filtering.
The quasi-likelihood is a distribution-free version of the likelihood that does
not require the accurate probabilistic observation model. The reweighted least-
square algorithms are used for the quasi-likelihood estimate calculation. The quasi-
likelihood-based methods are relevant to a wide class of applications where the
variance of observations is signal-dependant.
Fast algorithms implementing the quasi-likelihood scale-adaptive estimates are
developed based on linear convolution estimates completed by the ICI rule. As a
particular application of the developed adaptive nonlinear estimates, we consider
the counting (Poisson distribution) and binary (Bernoulli distribution) observations.
Image reconstruction problems concerning Poisson distribution data are the
subject of Chapter 12. Important areas of application include digital photogra-
phy, biology, and medical imaging problems. In many cases crucial high-quality
imaging requirements are in clear contradiction to very noisy raw data, with the
noise variance depending on the signal. Nonlinear methods based on likelihood and
quasi-likelihood are standard tools for this sort of problem.
A number of novel algorithms are proposed using the nonlinear LPA joint with
the ICI adaptive-scale selection. Special attention is paid to the development of more
computationally productive methods based on fast linear estimates completed by
the ICI adaptive-scale selection. It is shown that these methods are able to give high
quality with a reasonable complexity of computing.
A novel multiresolution (MR) nonparametric regression analysis is considered
in Chapter 13 as a further development and an alternative to the LPA-ICI adap-
tive technique. This processing includes the following three successive steps: MR
analysis, thresholding (filtering in the transform domain), and synthesis. The final
estimate is a composition of the estimates of different scales. In contrast to it, the
ICI rule defines a unique best scale in a pointwise manner for each estimation point.
Mathematical results concerning proofs of propositions as well as derivations
of some formulas are collected in the Appendix.
Chapter 2

Discrete LPA

The idea of local smoothing and local approximation is so natural that it is not
surprising it has appeared in many branches of science. Citing [145] we men-
tion early works in statistics using local polynomials by the Italian meteorologist
Schiaparelli (1866) and the Danish actuary Gram (1879) (famous for developing
the Gram-Schmidt procedure for the orthogonalization of vectors).
In the 1960s and 1970s the idea became the subject of intensive theoretical study
and applications, in statistics by Nadaraya [158], Watson [230], Parzen [175], and
Stone [215], and in engineering sciences by Brown [19], Savitzky and Golay [196],
Petersen [179], Katkovnik [98], [99], [101], and Cleveland [29].
The local polynomial approximation as a tool appears in different modifica-
tions and under different names, such as moving (sliding, windowed) least square,
Savitzky-Golay filter, reproducing kernels, and moment filters. We prefer the term
LPA with a reference to publications on nonparametric estimation in mathematical
statistics where the advanced development of this technique can be seen.
In this chapter the discrete local approximation is presented in a general mul-
tivariate form. In the introductory section (Sec. 2.1), we discuss an observation
model and multi-index notation for multivariate data, signals, and estimators.
Section 2.2 starts with the basic ideas of the LPA presented initially for 2D
signals typical for image processing. The window function, the order of the LPA
model, and the scaling of the estimates are considered in detail. Further, the LPA is
presented for the general multivariate case with estimates for smoothing and differ-
entiation. Estimators for the derivatives of arbitrary orders are derived. Examples
of 1D smoothers demonstrate that the scale (window size) parameter selection is
of importance.
The LPA estimates are given in kernel form in Sec. 2.3. The polynomial smooth-
ness of the kernels and their properties are characterized by the vanishing moment
conditions.
The LPA can be treated as a design method for linear smoothing and
differentiating filters. Links of the LPA with the nonparametric regression con-
cepts are clarified in Sec. 2.4, and the LPA for interpolation is discussed briefly in
Sec. 2.5.
21
22 V. Katkovnik et al.

2.1 Introduction

2.1.1 Observation modeling

Since the LPA is a method that is valid for signals of any dimensionality, we
mainly use a multidimensional notation because it is universal and convenient. For
instance, we say that the signal y(x) depends on x ∈ Rd , i.e., the argument x is a
d-dimensional vector, and we use it with d = 1 for scalar signals in time and
with d = 2 for 2D images. The multidimensional notation allows us to present
results in a general form valid for both scalar time and 2D image and signal
processing.
The symbol d is reserved for the dimensionality of the argument-variable x.
Thus, x is a vector with elements x1 , . . . , xd .
Suppose that we are given observations of y in the form ys = y(Xs ), s =
1, . . . , n. Here Xs stays for a location of the sth observation, and the coordinates
of this location are
Xs = [x1 (s), . . . , xd (s)].

The integers s number observations and their locations.

Noisy observations with additive noise can be given in the following standard
model commonly used for image and signal processing:

zs = y(Xs ) +
s , s = 1, . . . , n, (2.1)

where the additive noise
s is an error of the sth experiment usually assumed to be
random, zero-mean independent for different s with E{
s } = 0, E{
s2 } = σ2 .
Equation (2.1) is a direct observation model where a signal y is measured
directly and the additive noise only defines a data degradation.
Suppose we are able to observe (y
v)(x), where v is a blurring kernel or point
spread function (PSF) of a linear convolution. The blurring phenomenon modeled
by the PSF v (continuous or discrete) is very evident in many signal and image
applications. Moreover, we assume that the data are noisy, so that we observe zs
given by
zs = (y
v)(Xs ) +
s . (2.2)

This equation is an indirect observation model with a signal y measured not directly
but after some transformation.
A data degradation is defined by this transformation as well as by the additive
noise. It is assumed in image processing that all signals in Eqs. (2.1) and (2.2) are
defined on a 2D rectangular regular grid:

X = {Xs : s = 1, . . . , n}

with pixel (grid node) coordinates

Xs = [x1 (s), x2 (s)]. (2.3)

Discrete LPA 23

In this notation the image pixels are numbered by s taking values from 1 through
n. The coordinates of these pixels can be given explicitly as
X = {k1 ∆1 , k2 ∆2 : k1 = 1, . . . , n1 , k2 = 1, . . . , n2 }, n = n1 n2 , (2.4)
where ∆1 and ∆2 are the sampling intervals for x1 and x2 , respectively.
In signal processing literature, square brackets are sometimes used for dis-
crete arguments to differentiate them from continuous arguments. For example,
the notation y(k1 ∆1 , k2 ∆2 ) is for a continuous argument function and y[k1 , k2 ] is
for a discrete one with
y(k1 ∆1 , k2 ∆2 ) = y[k1 , k2 ]. (2.5)
For this discrete argument y we also can use the general notation y(Xs ) provided
that Xs = (k1 ∆1 , k2 ∆2 ).
The LPA considered in the book is applicable to observations given on irregular
grids, when the coordinates Xs and the distances between Xs can be arbitrary. In
image processing this sort of situation appears when data are destroyed or lost.
Figure 2.1 illustrates the cases of regular and irregular grids for 2D data.

2.1.2 Classes of signals

A basic objective of data processing is to reconstruct (estimate) y from the noisy

observations {zs } with pointwise errors that are as small as possible. It is assumed
that y is an unknown deterministic. In comparison with stochastic models of y, this
means that the main intention is to obtain the best results for every realization of y,
even if they are generated by a probabilistic phenomenon.
The first category is that of parametric models y(x, θ), where θ is a vector of
unknown parameters. These models seek a signal y with parameter θ that gives an
approximation of the relation between x and y. We predict y based on the argument
variable x assuming that θ is given. The estimation is concentrated around methods
for finding θ from observations
zs = y(Xs , θ) +
s , s = 1, . . . , n.

Figure 2.1 Types of data grids: (a) regular with sampling intervals 1 and 2 on the argu-
ments x1 and x2 ; (b) regular with missing data; and (c) irregular with varying intervals between
observations Xs .
24 V. Katkovnik et al.

The second category is that of nonparametric models, which assume that a

parametric representation of y with a reasonably small number of parameters θ is
unknown or does not exist. This y can be quite arbitrary, say, with discontinuities
and varying curvature. Examples of nonparametric and parametric curves (linear
y(x, θ) = θ1 + θ2 x and harmonic y(x, θ) = θ1 cos(θ2 x + θ3 ) + θ4 ) are shown in
Fig. 2.2.
In this book we deal with the nonparametric category of signals y. Usually it
is assumed that there is some continuous argument signal y(x), and the observed
y(Xs ) are sampled values of this continuous signal. Different classes of continuous
argument signals can be exploited. We restrict our analysis to the class Cα of con-
tinuous differentiable signals y of d variables with bounded αth-order derivatives,
which are defined mathematically as follows:

  
α  r1 
C = y : max ∂x1 . . . . ∂xd y(x) = Lα (x) ≤ Lα , ∀x ∈ R ,
rd d
(2.6)
r1 +···+rd =α

where Lα (x) and Lα are finite and α = r1 + · · · + rd is the order of the derivative
used in this definition.
The class Cα is composed of signals having bounded all possible derivatives of
the order α.
Let d = 1. Then the class Cα is defined as

Cα = {y : |∂xα1 y(x1 )| = Lα (x1 ) ≤ Lα , ∀x1 ∈ R}.

The class includes all signals with the bounded αth-order derivative.

Figure 2.2 Parametric and nonparametric models: (a) linear parametric model, (b) cosine
parametric model, and (c) nonparametric function with unknown parametric representation.
Discrete LPA 25

Let d = 2. Then
   
α  r1 r2 
C = y : max ∂x1 ∂x2 y(x1 , x2 ) = Lα (x1 , x2 ) ≤ Lα , ∀(x1 , x2 ) ∈ R2 .
r1 +r2 =α

In this definition, the maximum for each x = (x1 , x2 ) is calculated over all
derivatives of the order α = r1 + r2 . For α = 1, it assumes that
max(|∂x1 y(x)|, |∂x2 y(x)|) = L1 (x),
while for α = 2 it means
max(|∂x21 y(x)|, |∂x22 y(x)|, |∂x1 ∂x2 y(x)|) = L2 (x).
Definition (2.6) is global because the conditions hold for all x. For images where
singularities and discontinuities of y carry the most important image features, a local
smoothness of y is more realistic. Therefore, the class Cα can be replaced by
   
 
CαA = y : max ∂xr11 . . . . ∂xrdd y(x) = Lα (x) ≤ Lα , ∀x ∈ A ⊂ Rd , (2.7)
r1 +···+rd =α

where the argument x is restricted to a subset A.

The following piecewise smooth models are useful for many applications.
Let an area where y is defined be separated into q regions Ai , i = 1, . . . , q, and
each of these regions be a connected set with edges (boundaries Gi ). The signal y
is assumed to be smooth differentiable within Ai :

q

y(x) = yi (x)1[x ∈ Ai ], (2.8)

i=1

where 1[x ∈ Ai ] is an indicator of the region Ai , 1[x ∈ Ai ] = 1 if x ∈ Ai and

zero otherwise. More specifically, yi belongs to a class of continuous differentiable
signals CαAi . The parameters α, Lα (x), and Lα can be different for the different yi
in Eq. (2.8).
The piecewise constant y,

q

y(x) = ai 1[x ∈ Ai ], (2.9)

i=1

is a particular case of Eq. (2.8) with constant values within each region Ai . Figure 2.3
shows an example of a piecewise constant y defined according to Eq. (2.9) as well
as the corresponding regions Ai .
In the models (2.8) and (2.9) yi (x), and ai , as well as the regions Ai are usually
unknown. The boundaries Gi define change points of the piecewise smooth y in
Eq. (2.8). The estimation of y can be produced in different ways. One possible
approach deals with a two-stage procedure that includes estimating the boundaries
Gi at the first stage, which defines the regions Ai . The second stage is a parametric
or nonparametric fitting yi in Ai .
Another approach is connected with the concept of spatially adaptive estima-
tion. In this context, change points can be viewed as a sort of anisotropic behavior
26 V. Katkovnik et al.

Figure 2.3 A piecewise constant 2D function.

of the estimated signal. One may therefore apply the same procedure for all x,
for instance, nonlinear wavelet, ridgelet, or curvelet estimators, and the analysis
focuses on the quality estimation when change-points are incorporated into the
model. With this approach, the main intention is to estimate the signal but not the
locations of the change-points, which are treated as elements of the signal surface.
In this book we will follow the second approach. The objective is to develop a
method that simultaneously adapts to the anisotropic smoothness of the estimated
curve and is also sensitive to the discontinuities in the curves and their derivatives.
In general, conditions (2.6) and (2.7) define nonparametric signals that do not
have parametric representations. The following important class of parametric y can
be interpreted as a special case of the nonparametric class Cα .
Let Lm+1 (x) ≡ 0 for x ∈ Rd . Then y is polynomial of the degree m. Indeed, the
condition Lm+1 (x) ≡ 0 means that y is a solution of the set of differential equations
∂xr11 . . . . ∂xrdd y(x) = 0
for all ri such that r1 + · · · + rd = m + 1, and this solution is a polynomial. For
this class of polynomials we use the notation
P m = {y : ∂xr11 . . . . ∂xrdd y(x) = 0, r1 + · · · + rd = m + 1, ∀x ∈ Rd }, (2.10)

where m stays for the power of the polynomial. It is obvious that P m ⊂ Cm+1
with Lm+1 = 0.

2.1.3 Multi-index notation

Multi-index notation is used to shorten expressions that contain many indices.

Let x be a d-dimensional vector, x ∈ Rd , and y be a scalar function of x,
Discrete LPA 27

y(x) = y(x1 , . . . , xd ). A multi-index r is a d-tuple of nonnegative integers rj ,

j = 1, . . . , d,
r = (r1 , r2 , . . . , rd ),
d
where rs ≥ 0. The notation |r| is used to denote the length s=1 rs .
For polynomial expressions in coordinates xj , the multi-index notation provides
a convenient shorthand:
k k k
x k  x1 1 x2 2 · · · xd d ,

where x and k are d-tuples. Further,

k k k
xk x1x2 xd
 1 2 ··· d , (2.11)
k! k1 ! k2 ! kd !
where k! means a product of kj !, j = 1, . . . , d, and x k /k! is a product of the ratios
k
xj j /kj !, j = 1, . . . , d.
The differentiation operator of the order r can be written in the following
equivalent forms:
(r ,r ,...,r )
∂xr y(x) = ∂xr11 ∂xr22 . . . ∂xrdd y(x) = ∂(x11 ,x22 ,...,xdd ) y(x), (2.12)

where the elements of the tuple r in the superscript of ∂xr are the orders of the
partial derivatives calculated on the variables given by the d-tuple x shown as the
subscript. This subscript x can be omitted when the meaning of the differentiation
variables is obvious. Then the derivative is written as ∂ r y(x).
The differentiation of the polynomial x k is defined by
x k−r
∂xr x k = k! if k ≥ r, (2.13)
(k − r)!
where k ≥ r means that k1 ≥ r1 , . . . , kd ≥ rd . In the standard scalar notation,
Eq. (2.13) means

k k k1 ! k −r kd ! k −r
∂xr x k = ∂xr11 x1 1 · . . . · ∂xrdd xd d = x1 1 1 · . . . · xd d d . (2.14)
(k1 − r1 )! (kd − rd )!
For the polynomial (2.11), this differentiation gives the following formula from
Eq. (2.13):
xk 1 x k−r
∂xr = ∂ r (x k ) = if k ≥ r. (2.15)
k! k! (k − r)!
Consider the Taylor series of the d variables in the scalar notation,

∞ ∞ r r
u11 · · · ud2 (r1 ,...,rd )
y(x1 + u1 , . . . , xd + ud ) = ··· ∂ y(x1 , . . . , xd ). (2.16)
r1 =0
r ! · · · rd ! (x1 ,...,xd )
r =0 1
d
28 V. Katkovnik et al.

Using the multi-index notation and tuples, this can be given in the compact form

∞
ur
y(x + u) = ∂xr y(x), (2.17)
r=0
r!
∞ ∞ ∞
where r=0 = r1 =0 · · · rd =0 .
Expressions (2.11)–(2.17) demonstrate that, in multi-index notation,
multivariate formulas look like usual scalar ones. However, the multivariate nature
of these formulas is intact. In fact, the multi-index notation corresponds exactly to
the so-called array arithmetic operations as they are defined in MATLAB. These
operations are carried out element-by-element and can be used with multidimen-
sional data arrays. In MATLAB the period character (.) distinguishes the array
operations from the usual ones.

2.2 Basis of LPA

The idea of the LPA is simple and natural. It is assumed that the signal y is well
approximated by a polynomial in some neighborhood of the point of interest x.
We find the coefficients of the polynomial fit by the weighted least-square method
(LSM) and use this approximation to calculate the estimate for the point of interest
x, called “center” of the LPA. Note that the term “center” does not assume a central
position of x with respect to neighboring observations. This term aims to stress
that the LPA is applied to obtain the estimate for this particular point x. The local
expansion is applied to calculate the estimate for the given x only. The calculations
are repeated for each point. This pointwise procedure determines the nonparametric
character of the LPA estimation.
The LPA technique is valid for any dimensionality d of x. Starting from d =
2, we deal first with the 2D images. These provide a good opportunity for clear
illustrations. At the same time, d = 2 is an obvious multidimensional case that is
not so trivial as the scalar one, d = 1.

2.2.1 Idea of LPA

Let x ∈ R2 be a “center” (desired point) of the LPA. The estimate for the point
v ∈ R2 in a neighborhood of the center x is of the form
yh (x, v) = C T φh (x − v), φh (x) = φ(x/ h), (2.18)
φ(x) = [φ1 (x), φ2 (x), . . . , φM (x)]T ,
(2.19)
C = (C1 , C2 , . . . , CM )T , x = (x1 , x2 ), v = (v1 , v2 ),
where φ ∈ RM is a vector of linearly independent 2D polynomials
k k
x1 1 x2 2
(−1)k1 +k2 (2.20)
k1 ! k2 !
Discrete LPA 29

with the polynomial degrees k1 + k2 from 0 up to m, and C ∈ RM is a vector of

the parameters. The minus sign in this equation enables proper signs to estimates
of the derivative, which are obtained using this model. Details concerning the sign
selection in this equation will be discussed later.
Through this book, the character h, h > 0, is reserved to denote a scale. It
can be a scale of the estimate or a scale of variables, but it is always a scale. In
φh (x) = φ(x/ h), it is a scale of the polynomials. Used as a subscript, h should not
be confused with integer subscripts denoting the items of the vector φ in Eq. (2.19).
The elements of the vector φh (x) are
φi,h (x) = φi (x/ h), i = 1, . . . , M.
The total number of polynomials in φ = (φ1 , . . . , φM )T and the unknown
parameters in the vector C = (C1 , . . . , CM )T are equal to
(m + 2)!
M= = (m + 2)(m + 1)/2.
2 · m!
The corresponding φi , i = 1, . . . , M, form a complete set of the polynomials of
the power m. By definition, the completeness for the polynomials means that for
any x and v
φ(x − v) = A(v)φ(x), (2.21)
where A(v) is an M × M matrix depending on v only. Therefore, Eq. (2.21) means
that any polynomial of degree equal to or less than m can be written as a lin-
ear combination of {φi }. For m = 0, 1, 2, and 3, complete sets of the 2D linearly
independent polynomials (2.20) are of the form
φ1 = 1, for m = 0;
φ2 = −x1 , φ3 = −x2 , for m = 1;
(2.22)
φ4 = x12 /2, φ5 = x22 /2, φ6 = x 1 x 2 , for m = 2;
φ7 = −x13 /6, φ8 = −x23 /6, φ9 = −x12 x2 /2, φ10 = −x1 x22 /2, for m = 3;
with M = 1 for m = 0, M = 3 for m = 1, M = 6 for m = 2, and M = 10 for
m = 3. For each m the complete set of the polynomials in Eq. (2.22) includes the
row corresponding to this m as well as all previous rows with smaller values of m.
It is a good exercise to check the completeness of Eq. (2.21) for the sets of poly-
nomials in Eq. (2.22) corresponding to different m and to obtain the corresponding
matrices A(v).
Formula (2.18) can be interpreted as a truncated Taylor series that looks like
the following in the scalar form:

M
yh (x, v) = Ci φi,h (x − v). (2.23)
i=1

Model (2.18) is exploited to fit the discrete observations zs as they are introduced
in Eq. (2.1).
30 V. Katkovnik et al.

The following weighted least-square criterion Jh (x, C) is used in the LPA

to find the coefficient C in Eq. (2.18) (e.g., [31], [46], [77], [98], [99], [100],
[102], [145]):

Jh (x, C) = wh (x − Xs )[zs − yh (x, Xs )]2 , (2.24)
s

where the window function w formalizes the localization of fitting with respect to
the center x and
1 x
wh (x) = 2 w . (2.25)
h h
The scale parameter h > 0 determines the size of the window used in Jh (x, C).
Let us clarify the meaning of the local model used in Eq. (2.24). Consider the
residual zs − yh (x, Xs ) from the criterion (2.24) assuming that the observations are
noiseless and the model yh (x, Xs ) fits zs = ys = y(Xs ). According to the general
formula (2.16), the Taylor series for y(Xs ) with the reference point x is
zs = y(Xs )
= y(x) − ∂x1 y(x) · [x1 − x1 (s)] − ∂x2 y(x) · [x2 − x2 (s)]
+ ∂x21 y(x) · [x1 − x1 (s)]2 12 + ∂x22 y(x) · [x2 − x2 (s)]2 21
+ ∂x1 ∂x2 y(x) · [x1 − x1 (s)][x2 − x2 (s)] + · · · . (2.26)
In parallel, consider the model yh (x, Xs ) in Eq. (2.23) with the polynomials of
the corresponding powers:
yh (x, Xs ) = C1 − C2 · [x1 − x1 (s)]/ h − C3 · [x2 − x2 (s)]/ h
1 1
+ C4 · [x1 − x1 (s)]2 2
+ C5 · [x2 − x2 (s)]2 2
2h 2h
1
+ C6 · [x1 − x1 (s)][x2 − x2 (s)] 2 + · · · .
h
Using Eq. (2.26), the residuals zs − yh (x, Xs ) in Eq. (2.24) can be given in
the form
1
zs − yh (x, Xs ) = [y(x) − C1 ] + C2 − ∂x1 y(x) [x1 − x1 (s)]
h
1 1
+ C3 − ∂x2 y(x) [x2 − x2 (s)] + ∂x21 y(x) − C4 2
h h
1
· [x1 − x1 (s)]2 /2 + ∂x22 y(x) − C5 · [x2 − x2 (s)]2 /2
h2
1
+ ∂x1 ∂x2 y(x) − C6 · [x1 − x1 (s)][x2 − x2 (s)]. (2.27)
h2
Because the polynomials in Eq. (2.27) are linearly independent, minimizing the
weighted residuals in Eq. (2.24) is equivalent to minimizing the squared [y(x) −
Discrete LPA 31

C1 ]2 , [C2 / h − ∂x1 y(x)]2 , [C3 / h − ∂x2 y(x)]2 , and so on. In this way C1 gives the
estimate of y(x), C2 / h gives the estimate of the derivative ∂x1 y(x), C3 / h gives the
estimate of the derivative ∂x2 y(x), and so on. The formulas (2.27) show that the use
of the polynomials of Eq. (2.20) in Eq. (2.23) results in the coefficients C of this
expansion giving the estimates of the derivatives of y.
The negative sign for the odd powers k1 + k2 in Eq. (2.20) enables the correct
signs for the derivative estimates. The parameter h is an important element of
the estimates scaling the elements of the vector C. The expansions (2.26)–(2.27)
provide an initial important flavor of the LPA approach as well as the accurate
meaning of its elements.
Let us go further to some general aspects of this technique. In the model yh (x, v)
used in Eq. (2.24), the argument x is considered as a fixed invariant parameter
defining the center of the approximation, and the approximations for different Xs
are obtained by using this model with the varying second argument v assuming
v = Xs . Figure 2.4 illustrates the idea and the meaning of the LPA center.
The data are given on the grid X = {Xs , s = 1, 2, . . . , n} for Eq. (2.4) and a
location of the center (LPA reference point) x defines the type of estimation problem
considered. If x = Xk ∈ X, then the LPA estimate approximates the values yk or
smooths the noisy data zk . If x ∈ / X and x is inside of image boundaries, then
the LPA estimate interpolates the function y(x) for the argument value x using the
observations given on the grid X. If x ∈ / X and x is outside of the image boundaries,
the LPA estimate extrapolates the signal y(x) outside the area covered by X.
The size of the neighborhood used in the estimation depends on the scale h.
Larger values of h result in a larger neighborhood.
Depending on the x value in Eq. (2.24) we can treat the LPA estimates as
approximating/smoothing, interpolating, or extrapolating the data defined on the
data grid X.

Figure 2.4 The LPA center x and its neighborhood. The scale parameter h taking values
h0 < h1 < h2 defines the size of the square neighborhood of the center x.
Exploring the Variety of Random
Documents with Different Content
 Vielä kerran mentiin miehissä, koulumestarikin mukana,
tarkastamaan "Markettaa", jota Juoseppi nyt kiitteli kiittämästä
päästyään niin, että jo alkoi vanhasta ja viekkaasta Markusta pahalta
tuntua ja hän mietti: "Taisivat pojanpahalaiset minua pettää.", mutta
kun hän muisti, kuinka pehmeitä kaikki nuorat ovat, kuinka harvoina
ja harmaina purjeet, kuinka kulunut ankkuriketju ja että kaarissa on
mätää enemmän kuin ostajat luulivat, tuli hän jälleen miltei entiselle
tuulelle.

Alkoi jo hämärtää, ennenkuin Sameli ja Juoseppi pääsivät

lähtemään. Useat ryypyt pitkin päivää olivat vaikuttaneet sen, että
elleivät he olleetkaan sanottavasti humalassa, eivät he olleet aivan
selviäkään. He olivat rikkaita, nuoria, voimakkaita ja älykkäitä
kauppiaita ja merimiehiä, laivanpäämiehiä, jotka olivat eräässä
köyhässä kalastajakylässä, jota kutsutaan Ruokolahdeksi, käyneet
jaalanostossa ja läksivät nyt komealla orhilla ajamaan kotiinsa että
kaviot vain kapsahtavat, reen partaat rusahtelevat tupien, aittojen tai
saunojen nurkkiin ahtaissa kujasten käänteissä ja orhin harja vain
hulmuaa ja rekiloimen liepeet liehuvat pohjatuulen puhaltaessa
vastaan. Koiratkin ymmärtävät, että herroja lähtee kylästä, ja
saattavat kilpaa haukkuen ja ulvoen parikymmenlukuisena laumana.
"Tytöt — katsos Sameli — tirkistelevät ikkunoista!!" —

Hevonen juoksi rivakasti ylös kilometrin pitkää, loivaa ja

mutkikasta ylämäkeä, rinnettä, joka vietti etelään ja kaakkoon ja
jossa oli niittyjä — nyt lumilakeuksia. Sitten saavuttiin metsään.
Sitten tuli taas niittyjä ja sen jälkeen Vetel-järvi, sitten niittyjä ja
sitten painuttiin korpeen. Jylhiä honkia kasvoi kahden puolen tietä.
Tie oli niin kapea — sehän ei paljon muuta ollut kuin karjanpolku —
että oli vaara ajaa reki puuhun, sillä täällä syvällä metsän sisällä oli
pimeämpi kuin lakealla. Ajettiin hiljemmin.
 Siihen asti olivat molemmat istuneet reessä vaiti. Sitten sanoi
Sameli, nykäisten jälleen hevosen parempaan juoksuun:

— Sen minä sanon sinulle, Juoseppi, että kun minä olen nyt sinua
auttanut, niin minä vaadin, minä voin sinulta vaatia, että sinä pidät
silmäsi auki etkä pyri tallaamaan minun varpailleni, muuttui sitten
elämä miksi hyvänsä.

— Mitäs sinä oikein tarkoitat?

— Ymmärrä itse. Pidä silmäsi auki, sanon minä vielä kerran. Pidä
silmäsi auki, ja kyllähän sitte näet milloin olet tallaamaisillasi minun
varpailleni ja väistä silloin ajoissa. Jumal'auta, jos et niin tee!

— Sinä olet humalassa. Älä huuda!

— En minä huuda, enkä minä ole humalassa. Minä vain varoitan

sinua ajoissa, ettei sinun tarvitse katua. Ymmärrätkö sinä?! Niin sinä
olet tuhma kuin ryssän mahlikas, mutta sitä vartenhan minun sinua
on varoitettavakin. Minun nokalleni älä yritäkään hypätä, mies! Pane
se mieleesi!!

— Sinä haastat riitaa. Kuulehan! Minä viskaan sinut reestä, niin

isäntä kuin oletkin, ellet ole hiljaa!

— Piki-Juoseppi! — Piki-Juoseppi! — Älä anna käsien valtaa — älä

anna käsien valtaa! Ystäviähän me sentään ollaan! Eikös niin!

Saavuttiin äkkiä Liivalahdenjärven jäälle. Punakeltainen kuu nousi

sinipunaisella, tähtikirkkaalla tummalla taivaalla näkyviin itäisten
vuorten takaa. Edessä oli järven lakea jää. Sen takaa kohosi
Havukkavuoren metsäinen rinne, ja jylhä, huimaavan korkea
Havukkavuoren nenä, huippu, jonka ohi tie metsän läpi kulki.
Lännessä kuulsi vielä yhä himmenevän ja sammuvan iltaruskon
loisto.

Jäällä päästettiin hevonen täyteen raviin. Pian alettiin nousta

rinnettä ylöspäin, läpi synkän metsän, jossa tie pujoitteleikse ohi ja
välitse aitan suuruisten kivien ja suurten kuusten, joiden oksat
tapasivat milloin ylhäällä hevosen luokkaan, joka kaatoi niistä lunta
alas hevosen selkään ja miesten päälle, milloin aisoihin ja rekeenkin,
kun sattuivat olemaan alempana. Hevosta täytyi ajaa kävellen yhä
pahenevan nousun ja epätasaisuuksien vuoksi.

— Kuulehan, Juoseppi, sanoi Sameli ja pysäytti hevosen kokonaan.

— No?

— Meille tulee tästä hyvin paha kohta vastaan. Mitenkähän

päästään siitä ohi.

— Jaa tämä Havukkavuoren pykälä?

— Niin.

— Mikäs siitä on mennessä, kun on näin paljon lunta ja kivet ovat

peitossa.

— Ei se mitään…

— Ja voidaanhan pahimmassa paikassa nousta reestä pois ja…

— Mutta kun se pahin paikka on juuri siinä Ukon luolien suulla,

siinä, mihin kuuluu se kultaseppien vasarain läiske ja miltä paikalta
kesällä nähdään savun nousevan, kun tänne mereltä katselee — kun
Havukkavuoren Ukko leipiä paistaa — ymmärrätkö sinä.
 — Ole joutavoimatta!

— Ole itse!

— Ei se ole henkeä keitään vienyt.

— Vai ei! Etkö sinä ole kuullutkaan, miten kävi ennen sille suurelle
tietäjälle — kuului olleen minun isä-vainajani äidin sukua — joka
yritti loitsurunoillaan ja taioillaan manata aarteet itselleen
Havukkavuoren Ukolta. Oli asettunut siihen vastakkaiselle vuorelle,
Tervakalliolle, johon oli sytyttänyt pienen uhritulen ja sen ääressä
luki lukujaan ja loihti loihtimisiaan. Luki niinikään että:

Luolehetar luonnon tyttö, Manun eukon palkkalainen, joka

oot vuoren vartijana, rahapurnun paimenena,

ja sillä tavalla, ja kun ei sitte muistanutkaan koko lukua, järähti

kallio, Havukkavuoren nenä suitsi savua, ja se ei enään ollutkaan
mitään leivinuunin savua, ja tietäjä oli tukehtua ja kiitti onneaan, kun
pääsi vähissä hengin ja juosten alas rantaan Liivalahteen omalle
haapiolleen, jolla alkoi soutaa kotiin päin. Kaiken aikaa oli
Havukkavuoren Ukko syytänyt savua hänen päällensä, että oli eksyä
selvälle meren selälle, kun ei ollut kompassiakaan mukana — ja
lieneekö noita ollut silloin vielä kompasseja olemassakaan. —
Soutamalla niin likellä rantaa, että airot kolkki kaiken aikaa
rantakallioihin, osasi hän tulla kotiin. Vasta lähellä kylää pääsi siitä
savupilvestä kirkkaaseen ilmaan ja sai jälleen hengähtää. Kun sitte
tuli kotiin, oli siellä ryskänyt ja reistaillut koko ajan joku näkymätön
ja vienyt kaikki kullat ja hopeat — yksin tietäjän vaimon sormista
kultasormukset ja korvista kultaiset korvarenkaat oli siepponut ja niin
oli tietäjä entistäänkin tullut köyhemmäksi eikä uskaltanut enää
toista kertaa yrittää Ukon aarteita etsimään. Isä-vainaja toi aina
jouluaattoiltoina Ukolle siankinkun ja juhannusaattona maitoa uhriksi
ja ohikulkiessaan aina vuoli tähän hopeaa ja sanoi:

Pysy siellä, kun pysytän! Ole siellä, kun oletan!

ja pysyi Ukon kanssa loppuun asti hyvänä miehenä. — So, Polle!

Seisopas vähän!

Sameli pysäytti hevosen kokonaan.

Pahimpaan paikkaan oli enää tuskin viittäkymmentä syltä. Ohi

suurten, oksaisten kuusten tyvien ja suurien kivien näkyi se
kirkkaassa kuun paisteessa.

Kuului kultavasarain läiske.

— Sattuuko sinulla olemaan yhtään hopearahaa mukanasi?, kysyi

Sameli.

— Eikä ole, ja pakanaako sinä sillä tekisit?

— Eikö sinulla ole yhtään hopearahaa?!

— No ei kun ei. Päästäneen tästä nyt ilman tulliakin!

Kuu pistäytyi välillä pohjatuulen ajamaan pilveen ja valaisi taas

sen perästä sitäkin kirkkaammin.

Kammottavina seisoivat tummat kuuset ja aavemaisina lepäsivät

niiden varjot lumella, ja vielä kammottavampana kohosi puiden
takaa louhikko ja Havukkavuoren jyrkkä rinne ja huippu.

— No, koetetaan, sanoi Sameli ja ajoi hevosen jälleen

kävelemään. —
Ei tästä ilkeä takaisinkaan ajaa, eikä kiertää voi.

Äänettöminä alkoivat miehet jatkaa matkaansa.

Sameli tuskin uskalsi hengittää, kun olivat pahimmalla kohdalla,

missä päivän aikana näkyvät vuoren sisään johtavien luolien
kymmenet isommat ja pienemmät suut, joista ei tiedä kukaan, mikä
niistä on oikea ja mikä valereikä, sillä kukaan ei ole uskaltanut niitä
tutkia ja uuria, ja joista tie aukenee vuoren sisällä oleviin
aarrekammioihin.

Piki-Juoseppia oli peloittanut koko ajan pahemmin kuin Samelia,

vaikka hän sanoin koetti esiintyä rohkeana.

— Soo Polle, kävelepäs rivakammin!, sai Sameli hevoselleen juuri

kuiskatuksi, kun hän tunsi, että joku vetää hänen altaan hevoslointa
ja heinäkulia.

Se raastoi voimiensa perästä.

Sen huomasi Piki-Juoseppikin, ja taakseen katsomatta ja

vaistomaisesti, kuten vaarassa aina, hän mitään ennättämättä
harkita tai ajatella, sillä hänen ajatuksensa olivat kauhun vallassa,
sieppasi reen keulapuolelta hevosruoskan ja löi sillä hevosta
voimiensa takaa kerran, löi toisen, kolmannen ja neljännen ja
viidennen että siima vinkui ja hevosen lautaset läiskyivät. Samaa teki
Sameli oikealla puolella anteli ohjaksen perillä hevosta kupeisiin
minkä jaksoi. Miehet olivat nousseet tätä varten polvilleen rekeen.
Hevonen laukkasi, se miltei lensi ylös rinnettä ja reki heittelehti kuin
jolla aallokossa. Töin tuskin pysyi reki pystyssä ja miehet reessä ja
puiden oksat raapivat heidän kasvojaan.
 Lopulta olivat he jälleen tasaisella maalla, eräällä suolla, joka on
ylhäällä vuoristossa, Ylitiensuolla.

Hevonen juoksi tasaista ravia.

Näytti, kuin joku jalkamies tulisi vastaan.

Sameli pysäytti hevosen.

Mies näytti pysähtyneen.

Hevonen huohotti ja puhalsi höyryä sieraimistaan, kuin hopeapilvi

se leijaili kuun kirkkaassa valossa.

Miehet istuivat tyhjän rekensä partailla ja katselivat, että mikäs

mies sieltä oikein tulee.

Mies käveli jälleen ja sanoi kohdalle päästyään: "Hyvää iltaa",

johon
Sameli ja Juoseppi vastasivat.

Mies oli Jyrin Tuomas.

Tunsivathan he nyt jo miehen, vaikka ensin pelkäsivät, että tämä

olisi
Havukkavuoren Ukon metkuja.

— Mihinkäs Tuomas näin myöhään menee, kysyi Sameli

varmuuden vuoksi.

— Naapurikylään ja sieltä Lounatriviin, loistolle, kahdeksalta alkaa

minulla siellä vartiovuoro, loistolla nimittäin. Kun on näin kirkas
kuunpaiste, niin en välittänyt lähteä aikaisemmin.
 Jyrin Tuomas palveli siihen aikaan Lounatrivissä loistonvartijana.

Hän jatkoi matkaansa.

Sameli antoi hevosensa vielä seistä ja levätä.

Suon itäsivulla kohosi tasainen mänty- ja kuusimetsä ja länsisivulla

korkea Havukkavuoren seljänne.

Suolla itsessään kasvoi mataloita vaivaismäntyjä ja

varpukasvullisuutta, joka oli lumen peitossa.

— Jokos nyt uskot, sanoi Sameli Juosepille.

— Täytyyhän minun uskoa, kun omin silmin näin.

— Näitkö sinä sen itsensä?!

— No vaikken ihan sitä itseään, niin näen kuitenkin, että tässä ei

ole lointa eikä heinäkulia. Ne se vei kuin veikin, paholainen! Vei ihan
sulin käsin ja elävien miesten alta. Ei ole turhaa se puhe siitä
Haukkavuoren Ukosta!

— Ei ole. Sen minä olen aina uskonut!

— Mitähän tuo tuolla heinäkulilla ja loimella on tekevinään?! —

— Älä luulekaan että se kulia ja lointa ajaa! Minua se pakana

tarkoitti! Minut se olisi siepannut, koska täältä minun puoleltani kuli
ja loimi ensin alkoivat kulkea. Se tietysti yritti siepata minua turkin
liepeestä ja luuli sen jo saaneensa, hyvänkin saaliin, kun kouraisikin
erehdyksessä kyntensä loimeen ja heinäkuliin. Pettyipäs kerran
sekin, itse vanha pejuoni ja konstimestari! Pettyi kuin pettyikin, ja
nyt tässä kokonaan toiset miehet nauravat partaansa. Mutta
helkkarin varovainen saa tästä lähin olla tätä tietä kulkiessaan. Ei
tiedä, mitä voi tapahtua, kun ei edes osaa loihtia.

Sameli veti ohjaksista.

— Soo hevonen! Alas taas ravata!

*****

Seuraava päivä oli leuto, kirkas ja auringonpaisteinen.

Puhalteli lounattuuli.

Sameli ja Piki-Juoseppi olivat taas matkalla ylitietä. Juoseppi

matkalla naapurikylään hakemaan jaalaansa, kun nyt on nousumeri
ja ummikas lohkonut jäitä rannoilta ja satamasta ja on siis hyvä
tilaisuus ajaa jaala mereen, panna siihen purjeet päälle ja tuoda se
kotiin ja vetää omille teloille kotirantaan.

Sameli oli lähtenyt kyytiin, ainakin Havukkavuoren pykälään asti,

nähdäkseen ja tarkastaakseen, onko eileniltaisesta tapauksesta
jäänyt lumeen mitään merkkejä, Ukon jalanjälkiä, tai ainakin kulin ja
loimen jälki, kun se niitä perässään luolaan raahasi ja — tärkein
seikka kaikesta — mistä luolan suusta se on mennyt sisään.

Suuri oli molempien miesten ällistys, kun he paikalle saavuttuaan

näkivät sekä loimen että kulin riippuvan aivan tien vieressä olevan
kuusen oksassa.

Siinä oli kasvanut useissa kuusissa oksia, jotka olivat salvanneet

tien, ja joku oli niitä kirveellä rapsinut poikki, raivannut niitä pois
tieltä. Kuusten runkoihin, tien puolelle, oli näin jäänyt pitempiä ja
lyhempiä teräväpäisiä oksantynkiä, niinkuin esimerkiksi tuokin, jossa
riippui kuli ja loimi. Se oksa oli reen partaan korkuisella ja se kuusi
aivan tiessä kiinni. Siinä oksassa oli kiinni kuli ja loimi.

— No tähänhän ne ovat tarttuneet!, huudahti Juoseppi.

— Älä sano vielä mitään, virkkoi Sameli. — Ajattele ensin mitä

sanot! Olisihan me toki nähty ne, ja tuo kuusi ja huomattu, jos kuli
ja loimi olisivat siihen tarttuneet. Älä luule, että asia niin
yksinkertainen on! Ei ensinkään! Se oli vain niin viekas ukko, että
kun se huomasi pettyneensä, ja siepanneensa väärästä liepeestä ja
saaneensa toista kuin pyysi, tahtoo se johtaa meidät harhaan ja
ripusti tavarat — kun huomasi ne arvottomiksi — tähän oksaan,
meille silmän lumeiksi ja oman erehdyksensä peittämiseksi. Viisas
olet olevinasi, mutta kyllä elkeesi älytään!

Sellainen oli Samelin ensimäinen omakohtainen kokemus

Havukkavuoren
Ukosta!

VI luku.

LASKIAINEN KIRKKOTARHASSA.

Talvi oli kulunut laskiaiseen.

Samelin elämään oli tullut kokonainen uusi maailma tänä talvena.

Se maailma piti hallussaan hänen ajatuksiaan aina.

 Kun hän heräsi aamulla, oli se maailma hänen edessään: Anna-
Marja hymyilevine kasvoineen.

Se seurasi häntä pitkin päivää kauppoja tehdessä, jutellessa

ihmisten kanssa ja illalla rahoja lukiessa — venäläisissä
paperirahoissa hän tarkasti erästä rouvan kuvaa nähdäkseen, olisiko
siinä Anna-Marjan näköä: Ei ollut. Anna-Marja ei ole noin lihava ja
noin vanha, eikä Anna-Marjalla ole noin monta leukaa, eikä hänen
katseensa ole noin röyhkeä ja lihallinen — se sana ei Samelia
miellyttänyt, mutta parempaa ei löytynyt — eikä Anna-Marjasta
vanhetessaankaan tulle tuon näköistä leveää rouvasihmistä.

Rahoja laskiessaan ajatteli Sameli: "Jos olisi minun vallassani,

toisin puotiin kerääntyneet rahat joka ilta sinulle, Anna-Marja, ja
sanoisin: Nämä ovat sinun!" —

Kun puoti tyhjeni hetkiseksi ostajista ja apulainen oli mennyt

hakemaan
jauhosäkkiä, tupakkakulia, tai muuta alhaalta ranta-aitoista, jäi
Sameli yksin puotiinsa tuijottamaan ulos ikkunasta merelle ja ajatteli
Anna-Marjaa ja ihmetteli usein, miksi hän vasta viime syksynä löysi
Anna-Marjan, jonka hän on nähnyt lapsesta asti.

Hän koetti etsiä muistinsa jo miltei sammuneista komeroista

löytääkseen edes yhden tapauksen, jolloin — jo muka vuosia sitte, jo
rippikoulussa — olisi huomannut Anna-Marjan ja sanonut hänelle
hyvän sanan ja auttanut häntä ja ollut hänelle hyvä ja jolloin Anna-
Marja olisi hymyillyt hänelle samalla tavalla kuin viime syksynä
tansseissa Esalla, kun hän, Sameli, Jumala ties mistä
onnenpotkauksesta, joutui viemään Anna-Marjaa tanssimaan
hampuria. Sameli olisi toivonut, että sellaisia tapauksia olisi löytynyt
paljon, loppumattomasti, että he ensi kerran taas kohdatessaan
olisivat voineet toinen toisilleen niistä kertoa.

Ei ainoaakaan löytynyt!

Sameli koetti usein tekaista sellaisen jutun, jutun jossa ulkopiirteet

olisivat totta ja johon hän, Sameli, puhaltaisi hengen.

— — Esimerkiksi — — keväällä monta vuotta sitte — —

verkkorannalla — — hailinkutuaikana maan takana, länsirannalla
Simonapajalla, nuotiolla, valoisan keskiyön aikaan, aamuapajan
vetoa odotellessa, — — toiset pojat pilkkasivat sinua ja minä
puolustin sinua ja sinä otit minua kädestä ja katsoit minua silmiin —
— muistatko —. Minä olen sen aina muistanut ja olen pitänyt sinua
aina mielessäni, enkä nyt vasta…

Sameli oli pitkin syksyä ja alkutalvea ja joulukisojen aikana ja

niiden jälkeen aina tähän asti etsinyt tilaisuutta päästä Anna-Marjan
kanssa kahdenkesken ja haastaa hänelle sillä tavalla ja vielä kysyä
Anna-Marjalta, "ajatteleeko hänkin samalla tavalla minusta, eikä voi
minua mielestään pois ajaa — eikä tahdokaan sitä tehdä." —

Sameli tahtoi päästä varmuuteen.

Viis siitä jos hänen omaisuutensa karttuu ja hänellä on varmuus

siitä mitä syö ja juo ja millä itsensä vaatettaa, kun hänellä ei ole
varmuutta, mitä Anna-Marja ajattelee ja ketä ajattelee — niin, niin —
ketä ajattelee.

Hän katui tuhmuuttaan, ettei puhunut suutaan puhtaaksi silloin

kun
Anna-Marjan kanssa kahden joutui.
 Joulunaikoina hän saattoi Anna-Marjaa kotiin kaksikin kertaa, eikä
kummallakaan kerralla puhunut sanaakaan.

Tänään on laskiainen ja Esalla on laskiaistanssit.

Tänä iltana pitää kaiken tulla selväksi.

Näin ei saa jatkua!

Vuoden päästä on luultavasti kaikki jo ohi.

Jospa nyt jo tietäisin, mitä tiedän vasta vuoden päästä!!

Viime laskiaisena en tietänyt mitään siitä, mitä nyt tiedän.

Viime laskiaisena ja kaikkina aikoina ennen viime syksyä, oli

minulle yhdentekevää, kenen kanssa Anna-Marja tanssii, kenen
kanssa hän juttelee, kenelle hymyilee ja kuka häntä kotiin saattaa —
vaikka se olisi sitte itse Piki-Juoseppi!

Niin, niin!

Piki-Juoseppi saisi pysyä loitolla Anna-Marjasta.

Hän saisi ruvetakin kulkemaan toisen kylän tansseissa ja kisoissa!

Parasta olisi, että hän purjehtisi meren pohjaan heti ensi

matkallaan jaalalla, jonka hänelle ostin.

Sitävarten sen hänelle ostinkin!

Tänä iltana, se olkoon sanottu, minä kiellän Anna-Marjaa

sanomasta sanaakaan enää Piki-Juosepille, hymyilemästä hänelle
samalla tavalla kuin minulle, tanssimasta hänen kanssaan, vielä
vähemmin suostumasta hänen saatettavakseen.

Tänä iltana minä käsken Anna-Marjan sanomaan Piki-Juosepille,

mokomallekin naisten viettelijälle, että: "En tule sinun kanssasi!
Sameli saattaa minut kotiin!"

Niin, siitä sait! — Mokomakin Piki-Juoseppi!

Ja Piki-Juosepille minä sanon myös selvät sanat!

Sille en sano muuta kuin: "Jumal'aut! Älä pyri varpailleni!!" —

Ymmärtäköön siitä itse lopun!

*****

Sameli katsoi niin tiukasti että ikkuna oli puhjeta ja siruset lentää
kauas meren jäälle, ja hänen oikea kätensä kouraisi monta arkkia
ruskeaa käärepaperia, joka ratisten rutistui yhdeksi palleroksi hänen
nyrkkinsä sisään.

*****

Siitä on sinun, Piki-Juoseppi, ymmärrettävä, että pysy loitolla!

Syytä itseäsi, ellet siitä ymmärrä mikä sinua odottaa!

Että sinua odottaa kuolema suolaiseen meriveteen, että sinusta

tulee kilkkien ruokaa, jos kerrankaan, jos kerrankaan astut
varpailleni, jos kerrankaan Anna-Marja hymyilee sinulle samalla
tavalla kuin minulle!

Ymmärrä ja pidä varasi!

 Minä ostin sinulle veijarille vanhan jaalahaiskan. Vielä minä annan
sinulle romurautaakin jaalan täyden toisensa perästä ja vaanin
merellä sumussa, milloin voin karata kimppuusi.

Näin se käy:

Jumala lähettää sumua meren päällyksen täyteen ja sinulle

vastatuulen.

Sumu on niin sakea, ettet sinä näe edes jaalasi keulapurjeita.

Silloin minä olen "Anna-Marjoineni" saapuvilla.

Muista se, ettet sinä yksin seilaa, vaikk'ei sumutorvesi ääneen

kukaan vastaakaan.

Muista se, että minä itse tähystelen silloin laivani keulamaston

huipussa, siellä ylhäällä, missä sumua ei ole ja näen missä sinun
jaalasi kaksi mustaa mastonnokkaa piirtelee sumun ylempiä harsoja.

Sieltä ylhäältä, ymmärrätkö, minä annan ohjeita perämiehelleni ja

annan
"Anna-Marjani" syöksyä täysin purjein sinun mastojesi väliin.

Kuuletko, miten ryskää ja rutisee, suhisee ja kohisee ja tunnetko

vauhtia, jolla putoat jaaloinesi pohjaan ja kenties pullahdat vielä
kerran veden pinnalle, etkä näe enää mitään ja haukot vettä suuhusi
ja vedät sieraimiisi ja painut vihoviimeisen kertasi!

Tiedä ettei minun tarvitse edes odottaa syksyä!

Kevät on matalain ja sakeain sumujen aika!

Niin!
 Tänä iltana en sano sinulle kuin pari kolme kovaa sanaa ja
ymmärrä loput itse!!

Jumal'aut! Älä pyri varpailleni, jos henkesi on kallis!

*****

Samassa tuli apulainen sisään. Reestä he nostivat yhdessä säkit ja

kulin puotiin ja sen päätyttyä sanoi Sameli apulaiselleen:

— Kuules, Riko?

— No?

— Eik' se olisi mukavaa, jos ihminen tietäisi, mitä tuleva vuosi tuo
mukanaan?

— Miks'ei olisi! Olishan se!

— Tietäisi, mitenkä asiat ovat ensi laskiaisena.

— Saahan sen tietää.

— Saako!?

— Miksikäs ei? Elää vain rauhassa vuoden kaikki päivät ja niin on

taas laskiainen ja sittepähän tietää, mitä se on. Eikö vain liene
niinkuin tämäkin laskiainen: Myydään tavaraa ja kootaan rahaa.
Lunta on, ja pakkasta tai lämmintä, lumituiskua tai poutaa, tyyntä tai
tuulta. Tietäähän tämän nyt. Jos taas sitä ennen kuollaan, niin ei
tiedetä, eikä ole sitte tarvis tietääkään. No?

— Sinä olet lysti poika!

 — Tiedän minä muutakin.

— No?

— Monet ovat ensi vuonna naimisissa, jotka nyt eivät ole. Monta
vainajana, joka nyt parsii tai kutoo verkkoa, sahaa tai halkoo puita,
kulkee jäällä hylkeenpyynnissä ja niin edespäin. Moni uusi ihminen
on syntynyt, jota nyt ei ole. Monella rahaa enemmän kuin nyt. Kaikki
viisaampia kuin nyt…

— Niinhän sinulta tulee kuin papilta! Tuon kuitenkin tietää joka

mies.
En minä sitä tahdo tietää.

— No mitäs sitte?!

— Sitähän minä juuri sanoinkin, ettei sitä voi kukaan tietää, eikä
sitä voi keneltäkään kysyä.

— No sitte minä tiedän mitä ainakin tänä laskiaisena on!

— No?

— Sinä olet — hunningolle menossa!

— Kuinka niin?

— Sellaista, mitä ei kukaan tiedä, ei kukaan kysy muulloin kuin

vähää ennen hulluutta, tai muuta tapaturmaa. Setä-vainaja sanoi
niin ja ettei laivassa kysytä, että mihinkähän tästä oikein mennään,
ennenkuin eksyksissä ja vähää ennen matalalle tärähtämistä. Ja se
oli viisas mies. Ota siis tarkka peilaus, missä olet ja heitä luoti
mereen ja hiljennä vauhtia, ettet aja matalalle.
 — Et sinä ymmärrä, mistä on kysymys! Ehkä joskus ymmärrät, jos
satut samaan hätään, missä minä nyt olen.

— Hätään!? Onko jotain varastettu?

— Ei mitään. Ole huoletta! Ei puhuta tästä enää. Tänään on

laskiainen, ja kun nyt on jo hämärä ja kukaan ei tulle enää puotiin,
saat sinä mennä minne haluat.

Sameli itse päätti mennä Anna-Marjan luo, nyt heti, hänen

kotiinsa.

Kun ei tilaisuutta hänen tapaamiseensa ja puhumiseen näy

odottaen tulevan, niin se on ottaen otettava, päätteli Sameli.

Vaikeinta oli keksiä asia.

Jos nimittäin Anna-Marjan isä ja äiti tai jompikumpi sattuu myös

olemaan kotona, on hänellä oltava jokin oikea asia.

Hän päätti mennä Anna-Marjan isältä pyytämään lainaksi — niin

mitä hän sitten lainaisi?

Kirves, näveri, purjekinnas, höylä (minkälainen hyvänsä), vasara,

hohtimet…

Ei sovi!

Tätä mietti Sameli mennessään, ja aika loppui kesken, sillä hän oli
jo Anna-Marjan kodin portilla. Siinä pälkähti hänelle päähän sepittää
seuraava juttu:

Jo tänään hän, Sameli, oli aikonut ruveta paikkaamaan "Anna-

Marjan" huippupurjetta, mutta ei löytänyt purjekinnastaan — ei niin
mistään. Aamulla olisi työhön viimeistään käytävä ja täytyy nyt
naapurista pyytää lainaksi tuo kinnas.

Hyvä on.

Rohkeasti vain sisään!

Onneksi oli Anna-Marja yksin sisällä.

Samelin suuret ja laajat aikomukset puhua Anna-Marjalle kaikki

asiat selviksi supistuivat sangen vähiin.

Hän vain kysyi:

— Kuules, Anna-Marja?

— No?

— Lähdetkö sinä tänä iltana laskiaistansseihin Esalle?

— Menetkö sinä?

— Menen.

— Minä en voi mennä.

— Miksi et, eivätkö vanhempasi laske?

— Ehkä niinkin, mutta minua ei itseänikään haluta lähteä

mihinkään.

— Lähde vain. Mennään nyt jo ulos kävelemään. Olisi niin paljo

puhuttavaa.
 — Ei. Ei. Ei. — Puhu nyt vain tässä. En minä lähde. En minä lähde
mihinkään — kuuletko?

— Miksi?

— Siksi, että en lähde. — Minun on pää kipeä. En jaksa. Käyn

maata aikaisin. En välitä koko laskiaisista. Olkoot ja pitäkööt. Tyttöjä
on käynyt ja juossut tässä koko päivän pyytämässä mukaansa
laskiaispannukakkuja paistamaan ja leikkimään ja selittäneet, että
nyt pidetään laskiaista tänä talvena monta päivää. — Pitäkööt —
"Pitäkööt minun puolestani", — sanoin heille. — Jo sunnuntaina
kävivät ensimäiset pyytäjät. Kehuivat jo alkaneensa laskiaista pitää
sunnuntaista asti ja jatkavansa ainakin tämän viikon. Minä sanoin,
että pitäkää vaikka helluntaihin asti, ei se minua liikuta. — En minä
mene mihinkään.

— Et mihinkään!?

— En. Pysyn kotona, ja kun saan tämän sukan parsituksi, käyn

maata. Älä sinäkään mene mihinkään. Mene sinäkin kotiisi ja käy
maata. Niin on parempi. Antaa toisten pitää lystiä!

— Mutta olethan sinä aina ennen ollut — joka laskiainen.

— Mutta nyt en. En, vaikka tekisit mitä!

— Ja kun minä ajattelin näinikään, että kun minä en ole sinun

kanssasi ollut koskaan samoja laskiaisia viettämässä…

— Et ole ollut?!

— Niin. En ole ollut, niin mentäisiin nyt.

Welcome to Our Bookstore - The Ultimate Destination for Book Lovers
Are you passionate about books and eager to explore new worlds of
knowledge? At our website, we offer a vast collection of books that
cater to every interest and age group. From classic literature to
specialized publications, self-help books, and children’s stories, we
have it all! Each book is a gateway to new adventures, helping you
expand your knowledge and nourish your soul
Experience Convenient and Enjoyable Book Shopping Our website is more
than just an online bookstore—it’s a bridge connecting readers to the
timeless values of culture and wisdom. With a sleek and user-friendly
interface and a smart search system, you can find your favorite books
quickly and easily. Enjoy special promotions, fast home delivery, and
a seamless shopping experience that saves you time and enhances your
love for reading.
Let us accompany you on the journey of exploring knowledge and
personal growth!

ebookgate.com