Visit https://ebookfinal.

com to download the full version and

explore more ebook

Bayesian Brain Probabilistic Approaches to Neural

Coding Computational Neuroscience Kenji Doya

_____ Click the link below to download _____

https://ebookfinal.com/download/bayesian-brain-
probabilistic-approaches-to-neural-coding-
computational-neuroscience-kenji-doya/

Explore and download more ebook at ebookfinal.com

Here are some recommended products that might interest you.
You can download now and explore!

Bayesian Rationality The Probabilistic Approach to Human

Reasoning 1st Edition Mike Oaksford

https://ebookfinal.com/download/bayesian-rationality-the-
probabilistic-approach-to-human-reasoning-1st-edition-mike-oaksford/

ebookfinal.com

Memory and the computational brain why cognitive science

will transform neuroscience 1st Edition C. R. Gallistel

https://ebookfinal.com/download/memory-and-the-computational-brain-
why-cognitive-science-will-transform-neuroscience-1st-edition-c-r-
gallistel/
ebookfinal.com

The Probabilistic Mind Prospects for Bayesian Cognitive

Science 1st Edition Nick Chater

https://ebookfinal.com/download/the-probabilistic-mind-prospects-for-
bayesian-cognitive-science-1st-edition-nick-chater/

ebookfinal.com

Advances in Neural Population Coding 1st Edition M.A.L.

Nicolelis (Eds.)

https://ebookfinal.com/download/advances-in-neural-population-
coding-1st-edition-m-a-l-nicolelis-eds/

ebookfinal.com
Bayesian nets and causality Philosophical and
computational foundations Williamson

https://ebookfinal.com/download/bayesian-nets-and-causality-
philosophical-and-computational-foundations-williamson/

ebookfinal.com

Brain Renaissance From Vesalius to Modern Neuroscience 1st

Edition Marco Catani

https://ebookfinal.com/download/brain-renaissance-from-vesalius-to-
modern-neuroscience-1st-edition-marco-catani/

ebookfinal.com

The Computational Brain Patricia S. Churchland

https://ebookfinal.com/download/the-computational-brain-patricia-s-
churchland/

ebookfinal.com

Computational Neuroscience in Epilepsy 1st Edition Ivan

Soltesz

https://ebookfinal.com/download/computational-neuroscience-in-
epilepsy-1st-edition-ivan-soltesz/

ebookfinal.com

Coaching the Brain Practical Applications of Neuroscience

to Coaching 1st Edition Joseph O'Connor

https://ebookfinal.com/download/coaching-the-brain-practical-
applications-of-neuroscience-to-coaching-1st-edition-joseph-oconnor/

ebookfinal.com
Bayesian Brain Probabilistic Approaches to Neural
Coding Computational Neuroscience Kenji Doya Digital
Instant Download
Author(s): Kenji Doya, Shin Ishii, Alexandre Pouget
ISBN(s): 9780262042383, 026204238X
Edition: Kindle
File Details: PDF, 67.15 MB
Year: 2007
Language: english
Bayesian Brain
PROBABILISTIC APPROACHES TO
NEURAL CODING

edited by
KENJl DOYA, SHIN ISHII,
ALEXANDRE POUGET,
AND RAJESH P. W. RAO
Bayesian Brain
Computational Neuroscience
Terrence J. Sejnowski and Tomaso A. Poggio, editors

Neural Nets in Electric Fish, Walter Heiligenberg, 1991

The Computational Brain, Patricia S. Churchland and Terrence J. Sejnowski, 1992
Dynamic Biological Netzuorks: The Stomatogastic Nervous System, edited by Ronald
M. Harris-Warrick, Eve Marder, Allen I. Selverston, and Maurice Moulins,1992
The Neurobiology ofNeural Networks, edited by Daniel Gardner, 1993
Large-Scale Neuronal Theories of the Brain, edited by Christof Koch and Joel L.
Davis, 1994
The Theoretical Foundations of Dendritic Function: Selected Papers of Wilfrid Rall
with Commentaries, edited by Idan Segev, John Rinzel, and Gordon M. Shep-
herd, 1995
Models of Information Processing in the Basal Ganglia, edited by James C. Houk,
Joel L. Davis, and David G. Beiser, 1995
Spikes: Exploring the Neural Code, Fred Rieke, David Warland, Rob de Ruyter
van Steveninck, and William Bialek, 1997
Neurons, Networks, and Motor Behavior, edited by Paul S. Stein, Sten Grillner,
Allen I. Selverston, and Douglas G. Stuart, 1997
Methods in Neuronal Modeling: From Ions to Networks, second edition, edited by
Christof Koch and Idan Segev, 1998
Fundamentals of Neural Network Modeling: Neuropsychology and Cognitive Neu-
roscience, edited by Randolph W. Parks, Daniel S. Levine, and Debra L. Long,
1998
Neural Codes and Distributed Representations: Foundations ofNeura1 Computation,
edited by Laurence Abbott and Terrence J. Sejnowski, 1999
Unsupervised Learning: Foundations of Neural Computation, edited by Geoffrey
Hinton and Terrence J. Sejnowski, 1999
Fast Oscillations in Cortical Circuits, Roger D. Traub, John G. R. Jefferys, and
Miles A. Whittington, 1999
Computational Vision: Information Processing in Perception and Visual Behavior,
Hanspeter A. Mallot, 2000
Graphical Models: Foundations ofNeural Computation, edited by Michael I. Jordan
and Terrence J. Sejnowski, 2001
Self-organizing Map Formation: Foundation ofNeural Computation, edited by Klaus
Obermayer and Terrence J. Sejnowski, 2001
Neural Engineering: Computation, Representation, and Dynamics in Neurobiological
Systems, Chris Eliasmith and Charles H. Anderson, 2003
The Computational Neurobiology of Reaching and Pointing, edited by Reza Shad-
mehr and Steven P. Wise, 2005
Dynamical Systems in Neuroscience, Eugene M. Izhikevich, 2006
Bayesian Brain: Probabilistic Approaches to Neural Coding, edited by Kenji Doya,
Shin Ishii, Alexandre Pouget, and Rajesh P. N. Rao, 2007
Bayesian Brain
Probabilistic Approaches to Neural Coding
Kenji Doya, Shin Ishii, Alexandre Pouget, and Rajesh P. N. Rao

The MIT Press

Cambridge, Massachusetts
London, England
O 2007 Massachusetts Institute of Technology
All rights reserved. No part of this book may be reproduced in any form by
any electronic or mechanical means (including photocopying, recording, or in-
formation storage and retrieval) without permission in writing from the pub-
lisher.

MIT Press books may be purchased at special quantity discounts for business
or sales promotional use. For information, please email [email protected]
or write to Special Sales Department, The MIT Press, 55 Hayward Street, Cam-
bridge, MA 02142.

This book was typeset by the authors using LATEX2Eandfbookstyle file by Christo-
pher Manning. Printed and bound in the United States of America.

Library of Congress Cataloging-in-Publication Data

Bayesian brain : probabilistic approaches to neural coding / Kenji Doya ... [et
al.].
p. cm. - (Computational neuroscience)
Includes bibliographical references.
ISBN-10: 0-262-04238-X (alk. paper)
ISBN-13: 978-0-262-04238-3 (alk. paper)

1. Brain. 2. Neurons. 3. Bayesian statistical decision theory. I. Doya, Kenji.

Contents

Series Foreword ix

Preface xi

I Introduction 1
1 A Probability Primer
Kenji Doya and Shin Ishii 3
1.1 What Is Probability? 3
1.2 Bayes Theorem 6
1.3 Measuring Information 6
1.4 Making an Inference 8
1.5 Learning from Data 10
1.6 Graphical Models and Other Bayesian Algorithms 13

I1 Reading Neural Codes 15

2 Spike Coding
Adrienne Fairhall 17
2.1 Spikes: What Kind of Code? 17
2.2 Encoding and Decoding 25
2.3 Adaptive Spike Coding 42
2.4 Summary 47
2.5 Recommended Reading 47
3 Likelihood-Based Approaches t o Modeling the Neural Code
Jonathan Pillow 53
3.1 The Neural Coding Problem 53
3.2 Model Fitting with Maximum Likelihood 55
3.3 Model Validation 64
3.4 Summary 68
 Contents

4 Combining Order Statistics w i t h Bayes Theorem for

Millisecond-by-Millisecond Decoding of Spike Trains
Barry J. Richmond and Matthew C. Wiener 71
4.1 Introduction 71
4.2 An Approach to Decoding 72
4.3 Simplifying the Order Statistic Model 82
4.4 Discussion 84
5 Bayesian Treatments of Neuroimaging Data
Will Penny and Karl Friston 93
5.1 Introduction 93
5.2 Attention to Visual Motion 94
5.3 The General Linear Model 95
5.4 Parameter Estimation 98
5.5 Posterior Probability Mapping 101
5.6 Dynamic Causal Modeling 104
5.7 Discussion 109

I11 Making Sense of the World 113

6 Population Codes
Alexandre Pouget and Richard S. Zemel 115
6.1 Introduction 115
6.2 Coding and Decoding 116
6.3 Representing Uncertainty with Population Codes 120
6.4 Conclusion 127
7 Computing w i t h Population Codes
Peter Latham and Alexandre Pouget 131
7.1 Computing, Invariance, and Throwing Away Information 131
7.2 Computing Functions with Networks of Neurons: A General
Algorithm 132
7.3 Efficient Computing; Qualitative Analysis 136
7.4 Efficient Computing; Quantitative Analysis 137
7.5 Summary 142
8 Efficient Coding of Visual Scenes by Grouping and Segmentation
Tai Sing Lee and Alan L. Yuille 145
8.1 Introduction 145
8.2 Computational Theories for Scene Segmentation 148
8.3 A Computational Algorithm for the Weak-Membrane
Model 152
8.4 Generalizations of the Weak-Membrane Model 156
8.5 Biological Evidence 161
8.6 Summary and Discussion 180
9 Bayesian Models of Sensory Cue Integration
David C . Knill 189
9.1 Introduction 189
9.2 Psychophysical Tests of Bayesian Cue Integration 191
9.3 Psychophysical Tests of Bayesian Priors 195
9.4 Mixture models, Priors, and Cue Integration 199
9.5 Conclusion 204

IV Making Decisions and Movements 207

10 The Speed and Accuracy of a Simple Perceptual Decision: A Mathematical
Primer
Michael N.Shadlen, Timothy D. Hanks, Anne K. Churchland,
Roozbeh Kiani, and Tianming Yang 209
10.1 Introduction 209
10.2 The Diffusion-to-Bound Framework 210
10.3 Derivation of Choice and Reaction Time Functions 213
10.4 Implementation of Diffusion-to-Bound Framework in the
Brain 226
10.5 Conclusions 233
11 Neural Models of Bayesian Belief Propagation
Rajesh P.N.Rao 239
11.1 Introduction 239
11.2 Bayesian Inference through Belief Propagation 240
11.3 Neural Implementations of Belief Propagation 244
11.4 Results 248
11.5 Discussion 258
12 Optimal Control Theoy
Emanuel Todorov 269
12.1 Discrete Control: Bellman Equations 270
12.2 Continuous Control: Hamilton-Jacobi-Bellman Equations 273
12.3 Deterministic Control: Pontryagin's Maximum Principle 277
12.4 Linear-Quadratic-Gaussian Control: Riccati Equations 283
12.5 Optimal Estimation: Kalman Filter 287
12.6 Duality of Optimal Control and Optimal Estimation 290
12.7 Optimal Control as a Theory of Biological Movement 294
 ...
vzll Contents

13 Bayesian Statistics and Utility Functions in Sensorimotor Control

Konvad I? Kording and Daniel M. Wolpert 299
13.1 Introduction 299
13.2 Motor Decisions 301
13.3 Utility: The Cost of Using our Muscles 308
13.4 Neurobiology 314
13.5 Discussion 316
Contributors 321

Index 324
Series Foreword

Computational neuroscience is an approach to understanding the information

content of neural signals by modeling the nervous system at many different
structural scales, including the biophysical, the circuit, and the systems levels.
Computer simulations of neurons and neural networks are complementary to
traditional techniques in neuroscience. This book series welcomes contribu-
tions that link theoretical studies with experimental approaches to understand-
ing information processing in the nervous system. Areas and topics of par-
ticular interest include biophysical mechanisms for computation in neurons,
computer simulations of neural circuits, models of learning, representation of
sensory information in neural networks, systems models of sensory-motor in-
tegration, and computational analysis of problems in biological sensing, motor
control, and perception.

Terrence J. Sejnowski
Tomaso Poggio
Preface

When we perceive the physical world, make a decision, and take an action, a
critical issue that our brains must deal with is uncertainty: there is uncertainty
associated with the sensory system, the motor apparatus, one's own knowl-
edge, and the world itself. The Bayesian framework of statistical estimation
provides a coherent way of dealing with these uncertainties. Bayesian meth-
ods are becoming increasingly popular not only in building artificial systems
that can handle uncertainty but also in efforts to develop a theory of how the
brain works in the face of uncertainty.
At the core of the Bayesian way of thinking is the Bayes theorem, which
maintains, in its simplest interpretation, that one's belief about the world should
be updated according to the product of what one believed in before and what
evidence has come to light since. The strength of the Bayesian approach comes
from the fact that it offers a mathematically rigorous computational mechanism
for combining prior knowledge with incoming evidence.
A classical example of Bayesian inference is the Kalman filter, which has
been extensively used in engineering, communication, and control over the
past few decades. The Kalman filter utilizes knowledge about the noise in
sensory observations and the dynamics of the observed system to keep track
of the best estimate of the system's current state and its variance. Although
Kalman filters assume linear dynamics and Gaussian noise, recent Bayesian
filters such as particle filters have extended the basic idea to nonlinear, non-
Gaussian systems. However, despite much progress in signal processing and
pattern recognition, no artificial system can yet match the brain's capabilities
in tasks such as speech and natural scene recognition. Understanding how
the brain solves such tasks could offer considerable insights into engineering
artificial systems for similar tasks.
A Bayesian approach can contribute to an understanding of the brain at mul-
tiple levels. First, it can make normative predictions about how an ideal per-
ceptual system combines prior knowledge with sensory observations, enabling
principled interpretations of data from behavioral and psychophysical exper-
iments. Second, algorithms for Bayesian estimation can provide mechanistic
interpretations of neural circuits in the brain. Third, Bayesian methods can
be used to optimally decode neural data such as spike trains. Lastly, a better
understanding the brain's computational mechanisms should have a synergis-
tic impact on the development of new algorithms for Bayesian computation,
leading to new applications and technologies.

About This Book

This book is based on lectures given at the First Okinawa Computational Neu-
roscience Course, held in November 2004 at Bankoku Shinryokan, Okinawa,
Japan. The intention of the course was to bring together both experimental and
theoretical neuroscientists employing the principles of Bayesian estimation to
understand the brain mechanisms of perception, decision, and control.
The organization of the book is as follows. In the Introduction, Doya and
Ishii give the mathematical preliminaries, including the Bayes theorem, that
are essential for understanding the remaining chapters of the book. The second
part of the book, Reading Neural Codes, introduces readers to Bayesian con-
cepts that can be used for interpretation of neurobiological data. The chapters
by Fairhall, Pillow, and Richmond and Wiener explore methods for characteriz-
ing what a neuron encodes based on its spike trains. The chapter by Penny and
Friston describes how Bayesian theory can be used for processing and model-
ing functional brain imaging data. The third part, entitled Making Sense of
the World, assembles chapters on models of sensory processing. Pouget and
Zemel review ideas about how information about the external world can be
coded within populations of neurons. Latham and Pouget explore the use of
such codes for neural computation. Lee and Yuille consider how top-down and
bottom-up information can be combined in visual processing, while Knill uses
Bayesian models to investigate optimal integration of multiple sensory cues.
The final part, Making Decisions and Movements, explores models of the dy-
namic processes governing actions and behaviors. The chapter by Shadlen,
Hanks, Churchland, Kiani, and Yang focuses on neurons in higher visual cor-
tex that accumulate evidence for perceptual decisions over time. Rao discusses
a model of how cortical circuits can implement "belief propagation," a general
method for Bayesian estimation. Todorov reviews optimal control theory from
the viewpoint of Bayesian estimation. Finally, Kording and Wolpert utilize
Bayesian decision theory to understand how humans make decisions about
movements.
It is our hope that this book will stimulate further research into Bayesian
models of brain function, leading to a deeper, mathematically rigorous under-
standing of the neural processes underlying perception, decision, and action.

Acknowledgments
We wish to thank The Cabinet Office, Japanese Neural Network Society, Tama-
gawa University, and Kyushu Institute of Technology for sponsoring Okinawa
Computational Neuroscience Course 2004, and Sydney Brenner, President of
Okinawa Institute of Science and Technology, for advocating the course.
 ...
Pveface xzzz

The course and the subsequent book publication would have been impossi-
ble without all the secretarial work by Izumi Nagano. Emiko Asato and Ya-
suhiro Inamine also helped us in manuscript preparation. Our thanks go to
MIT Press editors Barbara Murphy and Kate Blakinger and to Terry Sejnowski
for supporting our book proposal.
AP would like to thank the National Science Foundation (NSF) for their
support. RPNR would like to acknowledge the support of the National Sci-
ence Foundation (NSF),the ONR Adaptive Neural Systems program, the Sloan
Foundation, and the Packard Foundation.
P A R TI

Introduction
1 A Probability Primer
Kenji Doya and Shin Ishii

1.1 What Is Probability?

The subtitle of this book is "Probabilistic Approaches to Neural Coding," so, to
start with, we have to be clear about what is probability [I].
A classical notion of probability is the so-called frequentist view. If you toss a
coin or roll a die infinitely many times, the ratio of having a particular outcome
among all possible outcomes would converge to a certain number between
zero and one, and that is the probability. An alternative idea of probability is
the "Bayesian" view [2], which regards probability as a measure of belief about
the predicted outcome of an event.
There has been a long debate between the two camps; the frequentists refuse
to include a subjective notion like "belief" into mathematical theory. Bayesians
say it is OK, as long as the way a belief should be updated is given objectively
131. The Bayesian notion of probability fits well with applied scientists' and
engineers' needs of mathematical underpinnings for measurements and deci-
sions. As we will see in this book, the Bayesian notion turns out to be quite
useful also in understanding how the brain processes sensory inputs and takes
actions.
Despite the differences in the interpretation of probability, most of the math-
ematical derivation goes without any disputes. For example, the Bayes theo-
rem, at the core of the Bayesian theory of inference, is just a straightforward
fact derived from the relationship between joint probability and conditional
probability, as you will see below.

1.1.1 Probability Distribution and Density

We consider a random variable X,which can take either one of discrete values
21,..., x~ or continuous values, for example, x E Rn. We denote by P(X =
x), or just P(x)for short, the probability of the random variable X taking a
particular value x.
 4 1 A Probability Primer

For discrete random variables, P ( X ) is called the probability distribution func-

tion. The basic constraint for probability distribution function is non-negativity
and unity, i.e.,

If X takes a continuous value, its probability of taking a particular value

is usually zero, so we should consider a probability of X falling in a finite
interval P ( X E [xl,221). Here P ( X ) gives a probability density function, whose
constraint is given by

Here the integral is taken over the whole range of the random variable X .
Despite these differences, we often use the same notation P ( X ) for both
probability distribution and density functions, and call them just probability for
convenience. This is because many of the mathematical formulas and deriva-
tions are valid for both discrete and continuous cases.

1.1.2 Expectation and Statistics

There are a number of useful quantities, called statistics, that characterize a
random variable. The most basic operation is to take an expectation of a function
f ( X ) of a random variable X following a distribution P ( X )

or a density P ( X ) as

We often use shorthand notations Ex [ ] or even E [ ] when the distribution

or density that we are considering is apparent. Table 1.1is a list of the most
popular statistics.

1.1.3 Joint and Conditional Probability

If there are two or more random variables, say X and Y, we can consider their
joint probability of taking a particular pair of values, P ( X = x;Y = y). We can
also consider a conditional probability of X under the condition that Y takes a
particular value y, P ( X = x Y = y ) .
 1.1 What Is Probability?

Table 1.1 Most popular statistics

name 1 notation definition
mean < X >, PX E[X]
variance V a r [XI,0% E [ ( X- E = E [ X 2 ]- E [XI2
covariance Cov [ X ,Y ] E [ ( X - E [ X I )(Y - E [ Y ] )=] E [ X Y ]- E [XIE [Y]
Cov[X,Y]
correlation Cor [ X .Y ]

The joint and conditional probabilities have a natural relationship

When we start from the joint probability P ( X ,Y ) ,P ( X ) and P ( Y ) are derived

by summing or integrating the two-dimensional function toward the margin
of the X or Y axis, i.e.
N
P ( X ) = )P ( X ,Y = Y , ) , (1.6)
i=l

so they are often called marginal probablity.

1.1.4 Independence and Correlation

When the joint probability is just a product of two probabilities, i.e.,

the variables X and Y are said to be independent. In this case we have

Otherwise we say X and Y are dependent.

A related but different concept is courelation. We say two variables are uncor-
related if
.
E [ X Y ]= E [XIE [Y] (1.9)
In this case the covariance and correlation are zero.
If two variables are independent, they are uncorrelated, but the reverse is
not true. Why? Let's imagine a uniform probability P ( X ,Y )over a rhombus
around the origin of X - Y space. From symmetry, X and Y are obviously
uncorrelated, but the marginal probabilities P ( X ) and P ( Y ) are triangular, so
their product will make a pyramid rather than a flat rhombus, so X and Y are
dependent.
 1 A Probability Primer

1.2 Bayes Theorem

From the two ways of representing the joint probability (1.5),we can relate the
two conditional probabilities by the following equation:

as long as P ( Y ) never becomes exactly zero. This simple formula is famous as

the Bayes theorem [2]. The Bayes theorem is just a way of converting one condi-
tional probability to the other, by reweighting it with the relative probability of
the two variables. How can we be so excited about this?
This is quite insightful when we use this theorem for interpretation of sen-
sory data, for example,

P ( d a t a hypothesis)P(hypothesis)
P(hypothesisdata) =
P(data)

Here, the Baves theorem dictates how we should update our belief of a certain
hypothesis, P(hypothesis)based on how well the acquired data were predicted
from the hypothesis, P ( d a t a hypothesis). In this context, the terms in the Bayes
theorem (1.10)have conventional names: P ( X ) is called the prior probability and
P ( X 1 Y ) is called the posterior probability of X given Y . P ( Y 1 X ) is a generative
model of observing Y under hypothesis X , but after a particular observation is
made it is called the likelihood of hypothesis X given data y.
The marginal probability P ( Y ) serves as a normalizing denominator so that
the sum of P ( X Y ) for all possible hypotheses becomes unity. It appears as if
the marginal distribution is there just-for the sake of bookkeeping, b u t as we
will see later, it sometimes give us insightful information about the quality of
our inference.

1.3 Measuring Information

Neuroscience is about how the brain processes information. But how can we
define "information" in a quantitative manner [4]? Let us consider how infor-
mative is an observation of a particular value x for a random variable X with
probability P ( X ) . If P ( X = x ) is high, it is not so surprising, but if P ( X = x )
is close to zero, it is quite informative. The best way to quantify the information
or "surprise" of an event X = x is to take the logarithm of the inverse of the
probability

Information is zero for a fully predicted outcome x with P ( X = x ) = 1, and

increases as P ( X = x ) becomes smaller. The reason we take the logarithm is
 1.3 Measuring Information 7

that we can measure the information of two independent events x and y, with
joint probability P ( x ,y) = P ( y ) P ( y ) by
, the sum of each event, i.e.
1 1 1 1
log -= log
P ( x ) P ( y ) = log ( x ) + 1% -
P ( x ,y) - P(Y) '

It is often convenient to use a binary logarithm, and in this case the unit of
information is called a bit.

1.3.1 Entropy
By observing repeatedly, x should follow P ( X ) ,so the average information we
have from observing this variable is
H ( X ) = E [- log P ( X ) ]= )-P ( X ) log P ( X ) , (1.12)
X

which is called the entropy of X . Entropy is a measure of randomness or uncer-

tainty of the distribution P ( X ) , since the more random the distribution, the
more information we gather by observing its value. For instance, entropy
takes zero for a deterministic variable (as H ( X ) = O for P ( X = x ) = 1 and
P ( X # x ) = 0), and takes the largest positive value log N for a uniform distri-
bution over N values.

1.3.2 Mutual Information

In sensory processing, it is important to quantify how much information the
sensory input Y has about the world state X . A reasonable way is to ask how
much uncertainty about the world X decreases by observing Y, so we take the
difference in the entropy of P ( X ) and P ( X Y ) ,
I ( X ; Y )= H ( X ) - H ( X I Y ) , (1.13)
where H ( X Y ) is the conditional entropy, given by the entropy of conditional
distribution P ( X I Y = y) averaged over the probability of observation P ( Y =
Y)I

I ( X ;Y )is called the mutual information of X and Y . It is symmetric with respect

to X and Y. This can be confirmed by checking that the entropy of the joint
probability P ( X ,Y ) = P ( Y IX)P ( X ) = P ( X 1 Y )P ( Y ) is given by
H ( X ,Y ) = H ( X ) + H ( Y I X ) = H ( Y )+ H ( X Y ) , (1.15)
and hence the mutual information can be presented in three ways:

+
I ( X ;Y ) = H ( X ) - H ( X 1 Y ) = H ( Y ) - H ( Y X ) = H ( X ) H ( Y ) - H ( X , Y ) .
(1.16)
 1 A Probability Primer

1.3.3 Kullback-Leibler Divergence

We often would like to measure the difference in two probability distributions,
and the right way to do it is by information. When we observe an event x,
its information depends on what probability distribution we assume for the
variable. The difference in information with distributions P ( X ) and Q ( X ) is

1 1 P(x)
log -- log -
p ( x ) = 1% -.
Q( X I Q( X I
If x turns out to follow distribution P ( X ) ,then the average difference is

which is called the Kullback-Leibler (KL) divergence. This is a good measure of

the difference of two distributions, but we cannot call it "distance" because it
does not usually satisfy the symmetry condition, i.e., D(P. Q ) # D ( Q ,P ) .

1.4 Making an Inference

Let us now consider the process of perception in a Bayesian way. The brain
observes sensory input Y and makes an estimate of the state of the world X .

1.4.1 Maximum Likelihood Estimate

The mechanics of the sensory apparatus determines the conditional probability
P ( Y X ) . One way of making an inference about the world is to find the state
X that maximizes the likelihood P ( Y = y X ) of the sensory input y. This
is called the maximum likelihood ( M L ) estimate. Although the ML estimate is
quite reasonable and convenient, there are two possible drawbacks. First, in
the world, there are more probable and less probable states, so inference just
by the present sensory input may not be the best thing we can do. Second,
using just a single point estimate of X can be dangerous because it neglects
many other states that are nearly likely.

1.4.2 Maximum a Posteriori Estimate

This is why the Bayes theorem can be useful in perceptual inference. If we
express the probability of different world states as a prior probability P ( X ) ,
we can combine the sensory information and this prior information according
to the Bayes therorem:
 1.4 Making an Inference 9

If we put aside the normalizing denominator P ( Y ) , the posterior probability

P ( X 1 Y ) of the world state X given sensory input Y is proportional to the prod-
uct of the likelihood P ( Y X ) and the prior probability P ( X ) . The state X that
maximizes the posterior probability is called the maximum a posterioir (MAP)
estimate.

1.4.3 Bayesian Estimate

The MAP estimate can incorporate our prior knowledge about the world, but
it still is a point estimate. We can instead use the full probability distribution
or density of the posterior P ( X 1 Y ) as our estimate. For example, if we make a
decision or motor action based on the estimated world state X , how sharp or
flat is the posterior distribution gives us the confidence of our estimate. When
the distribution is wide or even has multiple peaks, we can average the corre-
sponding outputs to make a more conservative decision rather than just using
a single point estimate.

1.4.4 Bayes Filtering

A practically important way of using the posterior probability is to use it as the
prior probability in the next step. For example, if we make multiple indepen-
dent sensory observations

the likelihood of a state given the sequence of observations is the product

The posterior is given by

but this can be recursively computed by

Here, P ( X l y l , ..., y t P l ) is the posterior of X given the sensory inputs till time
t - 1 and serves as the prior for further estimation at time t.
So far we assumed that the world state X stays the same, but what occurs
if the state changes while we make sequential observations? If we have the
knowledge about how the world state would change, for example, by a state
transition probability P ( X t I X t - I ) , then we can use the posterior at time t - 1
multiplied by this transition probability as the new prior at t:
 1 A Probability Primer

Table 1.2 Povular vrobabilitv distribution and densitv functions

name definition range mean variance
Binomial x = 0.1, ...; N Na Na(1 - a )
A i l.
>.

\,I = (N-z)!z!
Poisson 1
I
+axe-"
A!
1
I
x = O , l , 2 , ... 1
I
a 1
I
a
-(z-@)~
Gaussian zEe 202
X E R P a2
or normal I I I I
Gamma l b a x a - 1 e -bz x >0 a a

Jr
- -

r(n) b b2
r(a)= xa-l e-x dx
( r ( a )= a! if a is an integer)

Thus the sequence of the Bayesian estimation of the state is given by the fol-
lowing iteration:

This iterative estimation is practically very useful and is in general called the
Bayesfilter. The best known classical example of the Bayes filter is the Kalman
filter, which assumes linear dynamics and Gaussian noise. More recently, a
method called particlefilter has been commonly used for tasks like visual track-
ing and mobile robot localization [ 5 ] .

1.5 Learning from Data

So far we talked about how to use our knowledge about the sensory transfor-
mation P ( Y 1 X ) or state transition P ( X t I X t - 1 ) for estimation of the state from
observation. But how can we know these transformation and transition prob-
abilities? The brain should learn these probabilistic models from experience.
In estimating a probablistic model, it is convenient to use a parameterized
family of distributions or densities. In this case, the process of learning, or
system identification, is regarded as the process of parameter estimation. Table
1.2 is a list of popular parameterized distribution and density functions.
When we make an estimate of the parameter, we can use the same principle
as we did in the world state estimation above. For example, when the obser-
vation Y is a linear function of the state X with Gaussian noise, we have a
parameterized model

where 0 = ( w ,a ) is a parameter vector. From the set of input-output observa-

tions { ( x l ,y l ) ; ..., ( x T .yT)), we can derive a maximum likelihood estimation
 1.5 Lenrningfvom Dnta

by searching for the parameter that maximizes

A convenient way of doing ML estimation is to maximize the log-likelihood:

l o g P ( ~ i..., ; Y T I X I ,
T
... X T , ~ )= z l o g p ( y t x t , O ) =
;

t=l
z
T

t=l
-
(yt - w x t I 2
202
-T log 60.

From this, we can see that finding the ML estimate of the linear weight w is the
same as finding the least mean-squared error (LMSE) estimate that minimizes the
mean-squared error

1.5.1 Fisher Information

After doing estimation, how can we be certain about an estimated parameter
Q? If the likelihood P ( Y I Q )is flat with respect to the parameter 8, it would be
difficult to make a precise estimate. The Fisher information is a measure of the
steepness or curvature of the likelihood:

Ip(0) = Ey [( d log P ( Y 8 )
) 2] = El [- "l o ~ ~ ~ Y H ) ]
A theorem called Crame'r-Rao inequality gives a limit of how small the variance
of an unbiased estimate 8 can be, namely,

For example, after some calculation we can see that the Fisher information
matrix for a Gaussian distribution with parameters 0 = ( p ;a 2 ) is

If data Y = ( y l ;...,y ~ are

) given by repeated measures of the same distribution,
from log P ( Y 0 ) = c:=, log P ( y t 0 ) , the Fisher information is T times that of a
single observation. Thus Cramkr-Rao inequality tells us how good estimate of
the mean p we can get from the observed data Y depends on the variance and
number of observations, $.
 1 A Probability Primer

1.5.2 Bayesian Learning

We can of course use not only ML, but MAP or Bayesian estimation for learning
parameter 8, for example, for the sensory mapping model P ( Y X , 8) by

In the above linear example, if we have a prior knowledge that the slope is
not so steep, we can assume a Gaussian prior of w,

Then the log posterior probability is

so maximizing it with respect to w is the same as minimizing the least mean-

squared error with a penalty term

Such estimation with additional regularization terms is used to avoid extreme

solutions, often in an adhoc manner, but the Bayesian framework provides a
principled way of how to design them 161.

1.5.3 Marginal Likelihood

The normalizing denominator P ( Y 1 X ) of the posterior distribution (1.25) is
given by integrating the numerator over the entire range of the paremeter

which is often called marginalization. This is a hard job in a high-dimensional

parameter space, so if we are just interested in finding a MAP estimate, it is
neglected.
However, this marginal probability of observation Y given X, or marginal
likelihood, conveys an important message about the choice of our prior P ( 8 ) .
If the prior distribution is narrowly peaked, it would have little overlap with
the likelihood P ( Y X , 8), so the expectation of the product will be small. On
the other hand, if the prior distribution is very flat and wide, its value is in-
versely proportional to the width, so the marginal will again be small. Thus
the marginal probability P ( Y X ) is a good criterion to see whether the prior is
 1.6 Graphical Models and Other Bayesian Algorithms 13

consistent with the observed data, so it is also called evidence. A parameter like
a, of the prior probability for a parameter w is called a hyperparameter, and the
evidence is used for selection of prior probability, or hyperparameter tuning.
The same mechanism can also be used for selecting one of discrete candi-
dates of probabilistic models n'i, .... In this case the marginal probability
for a model P(A)&)can be used for model selection, then called Bayesian criterion
for model selection.

1.6 Graphical Models and Other Bayesian Algorithms

So far we dealt with just two or three random variables, but in real life there
are many states, observations, and parameters, some of which are directly or
indirectly related. To make such dependency clear, graphical representations
of random variables are useful. They are called graphical models, and the Bayes
rule is used for estimation of the latent variables and parameters. Especially
when a graphical model is represented by a directed acyclic graph (DAG), or
equivalently, for an n-dim. variable vector X,

where Pai denotes the parent variables of the variable X i in the DAG, such
a model is called a Bayesian network. For estimation of any missing variable
in a Bayesian network, various belief propagation algorithms, the most famous
one being the message passing algorithm, have been devised in recent years, and
there are excellent textbooks to refer to when it becomes necessary for us to use
one.

References
[I] Papoulis A (1991) Random Variables,and Stochastic Process. New York: McGraw-Hill.
[2] Bayes T (1763) An essay towards solving a problem in the doctrine of chances. Philo-
sophical Transactions of Royal Society, 53,370-418.
[3] Cox RT (1946) Probability, frequency and reasonable expectation. American Journal of
Physics, 14, 1-13.
[4] Shanon CE (1948) A mathematical theory of communication. Bell System Technical JOLLY-
nal, 27,379-423,623-656.
[5] Doucet A, de Freitas ND, Gordon N, eds. (2001) Sequential Monte Carlo Methods in Prac-
tice. New York: Springer-Verlag.
[6] MacKay DJC (2003) Inforrnation Theory, Inference, and Learning Algorithms. Cambridge,
UK: Cambridge University Press.
P A R T I1

Reading Neural Codes

2 Spike Coding
Adrienne Fairhall

Spikes: What Kind of Code?

Most neurons in the brain convey information using action potentials, or spikes.
This chapter will examine the coding properties of the spike train outputs
recorded from individual neurons. What kind of code do these spike trains
represent? How do spikes encode complex inputs; how can they be decoded?
Spike trains are the primary means of communication in the nervous system.
All current approaches to neural coding assume that the arrival times of spikes
comprise all that needs to be known about them. While variations in spike
height and width occur as a result of stimulus and prior spiking history [19], it
is generally assumed that these variations disappear in transmission through
the axon and downstream synapses and thus are incapable of conveying differ-
entiated information about the stimulus. This picture is a great simplification,
allowing us to think about the spike code in terms of a time series of all-or-
none events. However, there are still many possible ways in which these events
might communicate information through the nervous system. Does the exact
timing of spikes matter, and if so, at what accuracy? Perhaps precise timing
is irrelevant, and all that matters is the total number of spikes produced over
some time window, or in aggregate by a population of neurons. If spike tim-
ing is important, what are the elementary symbols of the code: single spikes
or patterns of spikes such as bursts? What do these symbols represent in the
stimulus? How do groups of neurons work together to transmit information:
independently or through intricate correlations capable of representing com-
plex stimulus patterns?
In figure 2.1, we show rasters of spike trains recorded from retinal ganglion
cells during the repeated viewing of a natural movie. Responses occur reliably
at fixed times, with a precision on the order of one to several milliseconds. Dif-
ferent neurons in the ganglion cell array respond at different times during the
stimulus presentation, signaling distinct features of the visual input. This is
the form in which visual information is transmitted from our eye to the rest of
the brain. In this chapter, we will concentrate on spike trains from single neu-
 18 2 Spike Coding

Figure 2.1 Simultaneously recorded spike trains from twenty retinal ganglion cells
firing in response to a repeated natural movie. Figure courtesy of J. Puchalla and M.
Berry.

rons to show how stimuli are encoded in the arrival times of single spikes, and
conversely, how one may go about decoding the meaning of single spikes as a
representation of stimulus features. We will show how information theory has
been applied to address questions of precision and reliability in spike trains,
and to evaluate the role of multiple-spike symbols.

2.1.1 From Rate to Spikes

The measurement of a tuning curve is a classic method for establishing and
quantifying the correlation of neural activity with an external signal or a be-
havior generated by the animal. As an external parameter varies systemati-
cally, the response of the neuron is measured in terms of the mean number of
spikes per second, counted in a time bin often of hundreds of milliseconds in
length. Stimuli are typically well separated in time so that responses unam-
biguously belong to a single stimulus category. While such characterizations
are valuable, information relevant for behavior must often be conveyed over
shorter time scales [58], bringing into question whether the integration of spike
count over long time scales is an appropriate representation. Researchers have
therefore been motivated to study responses to dynamically varying stimuli
[14,40,18,20], both in order to take contextual effects into account and to char-
 2.1 Spikes: Whnt Kind of Code? 19

acterize coding under more naturalistic conditions.

Despite internal noise sources, many neuron types are capable of firing in re-
sponse to a repeated direct current input with reproducible responses of high
temporal precision [38]. Even when stimuli are transmitted through a neural
circuit, the firing of single neurons can still be remarkably precise. While tem-
poral precision has been documented most extensively in invertebrate identi-
fied neurons [21] and in the sensory periphery [43,10], time-locked spiking to
dynamic stimuli has been observed under anesthesia in the lateral geniculate
nucleus (LGN) [16, 57, 761, and in somatosensory [50], auditory [24, 231 and
visual cortex [81, 601. Recent evidence suggests that temporal locking is also
observed in cortex in awake, behaving animals [29].

2.1.2 Timing and Information

The ability of a code to transmit information is limited by the fidelity with
which a given symbol of the code represents a given input. The quality of a
code can be quantified using information theory [66, 67, 151. When consider-
ing neural firing as a code, information theory allows us to evaluate the power
of different representations for encoding a stimulus. For example, imagine
that one is given a series of spikes recorded in response to a variety of stim-
uli. What is the appropriate time bin at which to measure the response? We
can answer this question by computing how much information the response
contains about the stimulus when represented using different time bins. Very
coarse time bins may lose information by averaging away temporal structure
in the spike train; yet using overly fine time bins will simply proliferate the
number of interchangeable output symbols corresponding to a given input,
due to spike timing jitter. Computing information can reveal the appropriate
time bin for representing the response. Later we will show an example of the
application of this procedure to neural responses in the LGN.
The Shannon information [66, 671 of a random variable X,distributed ac-
cording to P(X), is defined as - log, P(X). This quantity is the number of bits,
or the number of yes/no questions, required to establish the value of the ran-
dom variable: for example, if something is located somewhere in a 4 x 4 grid
with equal probability P(X)= 1/16, it would take - log,(l/l6) = 4 yes/no
questions to locate it (think of performing a binary search!). This definition is
clear for a discrete variable. For a continuous variable, characterized by a prob-
ability density function p(X), the probability of any particular value X = x is
zero, hence the information is infinite: no number of yes/no questions will es-
tablish the value to infinite precision. A sensible answer is obtained only by
integrating the probability density over an interval of x. In real systems, the
presence of noise imposes a precision limit, or an effective natural discretiza-
tion scale.
The overall variability or uncertainty of X is measured by its entropy,

H[P(X)]
=- P(x)log, P(x).
xEX
 20 2 Spike Coding

This is simply an average over the Shannon information, or the average num-
ber of bits required to specify the variable. Here we will mostly be concerned
with the mutual information, the amount by which knowing about one random
variable reduces uncertainty about another. For our purposes, one of these
variables will be the stimulus, S, and the other, the response, R. The response
R must clearly be variable (that is, have nonzero entropy) in order to encode
a variable stimulus. However, the variability observed in the response might
reflect variability in the input, or it might be due to noise. Mutual information
quantifies the amount of useful variability, the amount of entropy that is asso-
ciated with changes in the stimulus rather than changes that are not correlated
with the stimulus- i.e. noise.
The mutual information is defined as

I ( S ;R ) = ): P ( s ,r ) log, P ( s ,r )
SES.TER P(s)P(r)

This representation shows that the mutual information is simply the Kullback-
Leibler divergence between the joint distribution of S and R and their marginal
distributions, and therefore measures how far S and R are from independence.
Another way to write the mutual information is

In this form, it is clear that the mutual information is the difference between the
total entropy of the response and the entropy of the response to a fixed input,
averaged over all inputs. This averaged entropy is called the noise entropy,
and quantifies the "blurring" of the responses due to noise. We emphasize
that "noise" here is not necessarily noise in the physical sense; it is simply any
nonreproducibility in the mapping from S to R. This "noise" may in fact be
informative about some other variable not included in our definition of S or
R. Mutual information is symmetric with respect to S and R. It represents the
amount of information about S encoded in R, or equivalently, the amount of
information about R that can be predicted from a known S.

2.1.3 Information in Single Spikes

To compute the information in a spike train, we will first discuss a method
[13] to compute the information conveyed about the stimulus by the arrival of
a single spike. This method has the advantage that it makes no assumptions
about the nature of the encoding or decoding process. By symmetry, the in-
formation gained about the stimulus given that one observes a single spike is
the same as the information gained about the arrival time of a spike, given that
one knows the stimulus. The key idea is that this mutual information is the re-
duction in entropy between the prior distribution, the spike time distribution
when the stimulus is unknown, and the spike time distribution when the stim-
ulus is known. Let us consider a single time bin of size At, where At is small
2.1 Spikes: Whnt Kind of Code? 21

enough to contain a maximum of one spike only. Knowing nothing about the
stimulus, the probability to observe either a spike, r = 1, or no spike, r = 0, in
this time bin is

where r is the mean firing rate.

When the neuron is given a particular random, dynamic stimulus sequence,
s ( t ) , the result is a particular pattern of output. Averaged across many repeti-
tions, one obtains a temporally modulated firing pattern, r ( t ) ,such as would
be obtained by averaging the rasters in figure 2.1. Depending on the system,
the modulations in r ( t ) may be very sharp and well-isolated, as in figure 2.1,
or there may be relatively small modulations around some mean firing rate.
Intuitively, in the former case the stimulus and spikes are highly correlated; in
the latter, less so. Now we can compute the noise entropy from the conditional
distributions:

Denoting p = ? A t and p ( t ) = r ( t ) A t ,the information is given by

Here a time average, ~ , d t has , been substituted for the average over the
ensemble of stimuli, J d s P ( s ) , or its discrete equivalent appearing in equation
(2.3). This is valid if the random sequence s ( t ) is sufficiently long. Assuming
T
that p << 1, we may expand log(1 - p) p and use $ J, dt p ( t ) + p for T + cx,
to obtain

To obtain information per spike rather than information per second, we di-
vide by the mean number of spikes per second, r a t , and truncate to first order:

While p << 1, p(t) may be large, and one might worry that one is discarding
an important component of the information, this truncation amounts to com-
puting only information in spikes, and neglecting the information contributed
by the lack of spikes, or silences. For salamander retinal ganglion cells, this
contribution turns out to be very small: we found it to be less than 1% of the
total information (A. L. Fairhall and M. J. Berry 11, unpublished observations).
This result may break down for neurons with higher firing rates.
 2 Spike Coding

2.1.4 Information in Spike Sequences

While this formulation is simple and elegant, it is limited to the computation

of information in single spikes. A more general method was introduced by
Strong et al. [75]. Here, the neural response is represented by discretizing the
spike train into bins of size At.For simplicity, let us again take At to be shorter
than the refractory period of the neuron, so that bins will contain either a sin-
gle spike and be given the value 1, or no spikes, given the value 0. Sequences
of N bins are then N-letter binary words, w (figure 2.2). (One could of course
use larger bins and generalize to ternary or quaternary words, to dubious ad-
vantage.) We need now to compute the total entropy and the noise entropy.
The total entropy will be given by the entropy H[P(w)] of all words observed
in response to a long random stimulus. The noise entropy can be found by
repeating a segment of a random stimulus sequence, s(t), many times. In this
case, at every time t, the same stimulus was presented, and the word distribu-
tion at time t provides a conditional distribution P(wls(t)). Taken from LGN
data, figure 2.3a shows an example of the full distribution P(w)generated from
the long random stimulus. Figure 2.3b shows one of the conditional distribu-
tions P(wls(t)) sampled at a particular time in the repeated sequence. We now
need, according to equation (2.3),to average the entropy of P(wls(t)) over the
conditioning variable, the stimulus s. As above, if the repeated segment is
long enough, a time average samples the ensemble of s so is equivalent to an
average over P(s) .
There are two parameters in this choice of representation of the spike train:
the temporal precision, or the bin width At,and the total duration of the word,
L = NAt.To ensure that all correlations in the spiking output have been taken
into account, one would like to examine the limit where N + oo. In practice
this is not possible as the number of possibilities for w increases as 2 N , so that
P(w)and P(w1 s) rapidly become impossible to sample. In general, the issue
of finite sampling poses something of a problem for information-theoretic ap-
proaches and has accordingly been an active area of study. This topic deserves
a chapter on its own and so we will simply point the reader in the direction
of a few recent papers addressing finite size biases in information estimates,
with particular application to neuroscience [75, 49, 77, 45, 44, 34, 801. Within
sampling limits, Reinagel and Reid [57] examined the information content of
spike words from responses of LGN neurons. Figure 2.4 shows the informa-
tion rate (bits per second) about the stimulus conveyed by the spike train as
a function of the temporal precision of the spike binning At and the inverse
word duration 1/L. For this neuron, the information rate is maximal at higher
precisions, up to around 1ms, for all word lengths. The drop in information for
decreasing l / L is a result of sampling. For a fixed At,the information varies
approximately linearly with 1/L. The information value plotted in the narrow
bar 111 + 0 is the value obtained by extrapolating the total and noise entropies
as a function of l / L from their approximately linear regimes to the origin at
infinite word length [75].
Information has also used to assess the importance of particular complex
2.1 Spikes: What Kind of Code?

time (s)

Figure 2.2 (a). A spike train and its representationin terms of binary "letters." A word
consists of some number of sequentialbinary letters. @). A randomly varying Gaussian
stimulus (here, a velocity stimulus) and the spike-trainresponses from fly visual neuron
H1 for many repetitions. Adapted from Strong et a1.1731

symbols. Let us consider the symbol defined by the joint occurrence of some
pair of output events, El and E2. The synergy [13] is defined as the difference
between the mutual information between output and stimulus obtained from
the joint event compared with that obtained if the two events were observed
independently,

A positive value for Syn(E1,E2;s) indicates that El and E2 encode the stimu-
lus synergistically; a negative value implies that the two events are redundant.
This and related quantities [27, 48, 571 have been applied to multiple-spike
outputs to assess the contribution to stimulus encoding of timing relationships
among spikes in the fly visual neuron H1 [13]and the LGN [57]. Petersen et al.
[47,52]found that, for responses in rat barrel cortical neurons, the first spike of
a burst conveyed most of the information about the stimulus.
 2 Spike Coding

Figure 2.3 (a). The probability distribution P(w)of all words w generated by a long
random stimulus. (b).The word distribution P(wls(t))generated at a particular time t
by a given repeated stimulus sequence s ( t ) .

Another type of complex symbol is that produced by simultaneousfiring re-

sponses of multiple cells. The retina provides a perfect system for the study of
population coding, since it is easy to stimulate with dynamical stimuli and its
output, the planar layer of retinal ganglion cells, is accessible to multielectrode
array recording (figure 2.1). These neurons form a mosaic with overlapping
receptive fields [41, 55, 641. It has been shown that pairs of neurons convey
largely redundant information [55]. Recently, information-theoretictools have
been developed to assess the information gain resulting from simultaneous
observation of groups of N neurons due to successive higher-order correla-
tions [63]. Application of these tools to retinal ganglion cell recordings demon-
strates that painvise correlations dominate the additional information encoded
in the population [62]. Similar questions have also been addressed in cortex,
e.g. [42,51].
The question "what kind of code?" has different answers in different sys-
 2.2 Encoding and Decoding 25

Figure 2.4 Information rate in spikes per second as a function of parameters of the
spike-train representation, the bin width At, and the inverse total word length 1/L.
Reproduced with permission from Reinagel and Reid,[55] (see color insert).

tems, and for neural populations, we are only beginning to have the available
experimental and theoretical methods to address it. Here we have seen that
information-theoretic tools allow us to address the question in a quantitative
way.

2.2 Encoding and Decoding

How is the stimulus encoded by the spike train? In other words, given the
stimulus, can one predict the timing of a spike? For a single neuron, the con-
version from current or conductance stimulus to spiking response is a function
of the detailed biophysics of the neuron. For an external stimulus, the entire
network between the environment and the recorded neuron contributes to the
transformation. One might also "take the organism's point of view" [58], or the
inference problem that downstream neurons are in some sense solving: what
is the stimulus given observation of the spike train?
Exploring the Variety of Random
Documents with Different Content
passed me in the race in which I had had more than four
years the start. My father was fond of manly exercises, and
his sons inherited his taste. After our parent's death, Faiz ul
Din and I continued to pursue the sports of hunting and
shooting.

"On one too memorable occasion, we made a hunting

expedition into a wild part of the country. After a very
difficult and prolonged chase, I succeeded in killing a small
deer; Faiz ul Din, more fortunate, brought back in triumph
the skin of a tiger, slain by his own hand. We both
presented the spoils of the chase to our mother. The deer
skin was thrown aside; but my mother had the tiger's skin
made into a handsome rug, with the head stuffed and
jewels put in the place of eyes! In a jealous fit, I struck out
these eyes with my dagger, and contemptuously kicked the
rug into the verandah. Faiz ul Din came in at the moment,
and flew at me as if possessed by the tiger's spirit. There
was a struggle between us; though the younger, he was the
stronger. Enough—I slew him with the dagger, which was
still in my hand."

Robin uttered an exclamation of horror, and intuitively

drew himself a little farther away from the murderer of a
brother. There had been a ghastly skeleton indeed, at the
bottom of the dark pool.

A painful, oppressive silence followed; broken at last by

Robin's inquiry, "Did you ever see your poor mother again?"

"No," was Ali's reply; "I fled from the palace as soon as
I had secured about me a large sum in gold, and some of
my more portable treasures. Hassan, who followed me a
day or two afterwards, brought me many more things of
value. I made it worth his while to keep silent, and began a
series of journeys in various parts of the world, partly to
carry on trade in horses and jewels, partly—as I once said
to you before—to flee from myself."

Robin could understand the latter reason better than

the first. It was to him inexplicable that a man with such a
burden of guilt on his soul should care to make money by
trading. But Robin was not an Oriental.

"Did your mother know who did the terrible deed?"

asked the lad.

"She knew all; the dagger which I left behind, and my

sudden flight, were sufficient evidence against me," replied
Ali. "My mother cursed me in the presence of her servants!
I can never, never meet her again; she is now childless
indeed."

"And you can never return to Persia?"

"I do not think that I should incur personal risk by

going," replied Ali, in a more indifferent tone; "these things
are not looked upon in our land as they are in yours. My
countrymen think little of blood being shed in a hasty
quarrel, and I have that which would make my peace. But I
should hate to return to Persia, bearing with me the weight
of a brothers blood and a mother's curse."

Young Hartley felt sickened with horror. He could hardly

endure to remain in the presence of one who had
committed so terrible a crime. Robin was not sufficiently
well read in Oriental history to know how fearfully common
fratricide has been amongst Asiatics of the highest rank;
nor did he make sufficient allowance for the lowering of the
moral standard caused by following a religion that in some
cases not only palliates murder, but raises it into a merit.
Robin was more given to feel acutely than to calculate
deeply. He had not acquired the callousness in regard to sin
which often follows familiarity with its loathsome details,
like the insensibility to vitiated air which comes from
perpetually breathing it. The emotion in the breast of our
young Knight of St. John after hearing Ali's story might be
well expressed by one most forcible line from Shakespeare
—

"Oh! I am sickened with this smell of sin!"

Ali saw the impression made by his words, he had

noticed the slight shrinking back from his person, he felt
that the only human being whom he had sought to make
his friend was lost to him for ever. Tenfold bitterness
returned to his spirit. With the haughty air of one who is
offended, rather than conscious of having given offence, the
Persian rose from his reclining position, and, standing erect,
said to young Hartley, who had covered his eyes with his
hand, "Enough—you have my secret; can I trust to your
honour not to betray it?"

"I will never divulge it; but would that I had never
heard it!" was Robin's reply.

The Amir strode out of the tent, heedless of the heat

and the glare. His attendants, after taking their noonday
repast of fruit, under what shelter the camels and piled
luggage afforded, were indulging in a siesta. Ali was the
only being awake.

"So it is gone, and never to return, that glimpse of

brightness which beguiled me into idle hope!" muttered the
Amir to himself. "The very boy who owes his life to me,
whom I have watched over, nursed, almost loved, regards
me with unconcealed loathing! Well, I will soon liberate him
from the presence which he hates; I will keep my promise
to take him to Djauf, and then we part, to see each other no
more in this world—or the next!"

Ali was not far from the tent, and in the midst of his
gloomy reflections, his ear caught low sounds of distress
issuing from it. He went nearer and listened. The Persian
heard Robin pouring out the anguish of his young loving
heart in tones that Ali had never before heard bursting from
human lips. The words were uttered between broken sobs,
for Robin was too weak to restrain his emotions, and he
thought himself quite alone. Ali could distinguish such
sentences as these:

"O Lord! Remember Thine own Word; is it not

written that, if any man see his brother sin a sin
which is not unto death, he shall ask, and Thou
wilt give him life, for them that sin not unto
death. I know not whether this sin is such, but,
oh most pitying, most loving Saviour! Have
mercy—have mercy on my poor guilty brother!
Save him, for Thou only canst save. Thou dost
hate sin, but, oh Thou dust love the sinner! Let
not my brother perish; give Ali eternal life.
Didst Thou not die for him as well as for me?"

Robin's tears were falling fast; his were not the only
tears that fell. Ali's eyes, that had never wept since the
days of his childhood, were moistened now; the knee that
had never been bent in real supplication for mercy was now
on the earth, the hard heart was throbbing, and what had
been but stern remorse was softening into repentance.
 "The Feringhee is pleading for me, God will hear him!
The boy calls me brother, the name which he denied to me
before, he gives me now! If the disciple think me yet within
reach of mercy, will the Master cast me out?"

CHAPTER XIX.
A BITTER CUP.

IT is now time to return to Harold, and those whom

circumstances had placed under his care. Harold's was a
strong, firm spirit, but it could hardly bear up under the
accumulated afflictions which had so suddenly been heaped
upon him. All, indeed, would have been calmly endured, but
for the last crushing weight of anguish caused by the loss of
his brother. It was terrible to make a calculation as to how
long Robin's youth, fine constitution, and brave spirit would
be able to maintain a lingering struggle against famine,
heat, and thirst—how long it would take to transform the
suffering mortal into the rejoicing saint. To have known that
Robin had actually died would have been a kind of relief, for
Harold was of a less hopeful temperament than his brother.
The elder brother did not look for miraculous deliverance
from the fiery furnace, but rather for strength to endure the
flames.

In his state of deep depression, Miss Petty's thoughtless

tongue inflicted perpetual wounds on Harold, as one with a
limb crushed under a fragment of rock might yet be
sensible to the petty annoyance of an insect's buzz and
sting.

"Where do you think that they are dragging us, Harold?

Is not this a round-about way to India? Won't the Queen
send an army to free us? Do you think we'll be sold as
slaves? Are there cannibals in this horrid Arabia?" These,
and many other such questions, repeated again and again
till a brief reply was extracted, tried sorely the patience of
Harold. Whilst, with weary limbs and blistered feet, the
young missionary paced the desert way behind the camel,
he had to endure this infliction.

After hours of walking, Harold was relieved from his

bonds and suffered to mount a camel. This was chiefly an
alleviation to misery, because, for a while it relieved him
from the necessity of closely following his talkative
companion. There was no halt until night, for oases were
few and far between; but the excessive length of the stage,
which had cost the life of one camel, and the exhausted
state of the other unfortunate beasts of burden, compelled
a longer rest than usual.

The description of the evening meal need not be

repeated. There was little variety in the halts. There was
here, however, no sheep to slaughter and no tent to
accommodate poor Miss Petty. She and her Lammikin had
to bivouac on the bare ground, under the sky. Harold, who
was not far from his charges, was startled in the night by
shriek upon shriek. Were the Arabs murdering their
unfortunate captives? Harold hurried to the spot in time to
set his heel on one of the small dark reddish scorpions
found in the desert, which had crawled on Shelah's dress.

"Horrible creature! And there's another! I declare that I

can't and won't sleep on the sand!" cried Miss Petty,
furiously shaking her clothes lest one of the hideous reptiles
should be concealed in some fold. But there was nothing
else on which to sleep!

Harold returned to his own place near Tewfik, the

Bedouin who had first seized him, and who consequently
seemed to regard the captive as his own special property.
Weariness might have enabled the missionary to find some
relief from sorrow in sleep, had he not been kept awake by
the loud talking of the Arabs near him. From the few words
which he made out, Harold felt assured that he himself was
the subject of conversation. What Harold could not
understand in the following dialogue, his imagination
tolerably well supplied, though gaps of ignorance remained
to perplex the mind of the hapless captive.

"We have won poor spoil this time," said the chief of the
Shararat band. "These kafirs had hardly a piastre amongst
them, the jewels are tinsel and glass; two of the party are
dead already, and two of those left are not worth a handful
of date-stones."

"Kismat" (fate), was Tewfik's characteristic reply.

"What shall we do with the tall Feringhee?" inquired the

chief, glancing towards the spot where Harold was lying.

"Sell him, if we can get a purchaser. It will be strange if

no one in Djauf be in want of a slave."

"But he is white; slaves in Arabia are usually curly-

headed blacks from the African coast."

"He'll be a choice rarity then, like a white camel," was

the laughing reply. Bedouins are fond of a joke.
 "The worth of a camel is certainly not in its colour but in
its power of bearing burdens," said the chief.

"The Feringhee has plenty of bone and muscle, and

spirit too," observed the Bedouin robber; "I have felt the
strength of his arm. I should say that he could lift three
maund." *

* A maund is 80 lbs.

"No, he's slight—not two," said the chief.

"He's worth forty tomauns of any one's money," cried

Tewfik.

"I say thirty; we'll be lucky if we get them," rejoined the

other.

There was a little squabbling over this matter of

Harold's price. The voices became louder, the manner of the
Arabs more excited, especially hot grew the dispute when
the subject was how far Tewfik was entitled to the purchase
money of his captive, or whether the coveted tomauns
should not be divided amongst the band.

"Son of an ass!" exclaimed the chief angrily. "Three

such as you could not have mastered that Feringhee had
not we been near to aid!"

"A slave! A price to be given for me—an Englishman!

This is the last drop in my bitter, bitter cup!" thought
Harold. "Was it for this that I left my country and devoted
myself to work for souls? Could I not have been spared
such misery, such humiliation as this? But I see before me
the footprints of One who drank of a cup yet more bitter,
who submitted to degradation yet deeper. It was through
anguish that the Master passed to glory, shall the disciple
shrink back? But is it cowardice to hope that the misery
may be short, that I may in mercy be soon permitted to
rejoin my dear lost brother!"

The wrangling amongst the Bedouins ended in

compromise; Tewfik was to keep a third of the money paid
for his unhappy slave, and the Arab's good temper being
restored, he laughingly told the chief that he would freely
throw the old woman and her child into the bargain.

"I would not give a lame ass for the two," quoth the
chief.

CHAPTER XX.
DESERT DANGERS.

IF poor Miss Petty had been wretched when she had at

last stretched herself on her hard sandy bed, to be pursued
even in her dreams by scorpions, she awoke to a joyful
surprise.

"Look, look, Shelah!" exclaimed Theresa, rousing her

tired little companion. "We've got to the end of horrid
Arabia at last! No more brackish wells, with water not fit for
a pig to bathe in; no more barren sand! See these shady
trees before us—no doubt they are laden with fruit,—see
the clear delicious water! It was horrid to have only a rub of
hot dry sand in the place of good soap and water!"

Shelah rubbed her sleepy eyes, then jumped up,

clapped her hands with delight, and shouted for joy. No
wonder that the poor wanderers over barren wastes, under
a blazing, scorching sun, with scarcely any vegetation
visible but a few stunted trees, or prickly bushes in
favoured spots, should be transported at the beautiful view
which now met their delighted eyes. Great was the joy of
beholding a large clear lake, dotted with verdant isles,
which looked as if they must be the homes of bright-
plumaged birds, and butterflies without number!

"It's India—I know it's India!" cried Shelah. "For papa

told me that Bombay is an island, and that he would feast
me on lots of mangoes and plantains. I'll just go on eating
fruit from morning till night, and stuff some under my pillow
to take as soon as I open my eyes!"

"And we'll be rowed about in gay boats," began Miss

Petty, then, interrupting herself as she caught sight of
Harold approaching, she exclaimed, "Oh! Is not this a sight
to make one dance with joy!"

"A deceitful joy," said Harold sadly; "you are looking on

a mirage. What appears like water is only sand."

"I don't believe it!" cried Miss Petty. "Can't I trust my

own eyes?"

"I'm sure that's water!" exclaimed little Shelah.

Harold was at the moment called away by Tewfik, who

wanted to make trial of his captive's strength and skill in
loading a camel. Before obeying the call, Hartley repeated
to Miss Petty his assertion that the supposed island-studded
lake was but an optical delusion common in desert lands.

"Harold is perked up with his book-learning," observed

Miss Petty, "but he's not so much wiser than his elders.
Don't I know a lake when I see one!"

"I'm going to have a dip, a jolly good dip!" cried Shelah,

whose spirits rose like an india-rubber ball when pressure is
removed. Off she rushed, impelled by charming hopes of
splashing about in the water, followed by Miss Petty, who
half forgot weariness and misery in her eagerness to reach
—what did not exist!

Poor Theresa! That search after the supposed lake was

an emblem of what her whole life had been; impelled by
vanity, worldliness, selfishness, her hair had grown grey,
her years had been wasted in the pursuit of the world's
deceitful mirage.

In the meantime, Harold joined the group of Arabs who

were standing in a semi-circle round a collection of
mashales, filled almost to bursting with a supply of water
which was to last the whole party for three long days. Each
of these brown water-bags was made of the entire skin of a
sheep, the head and legs excepted, the place where the
neck had been, serving, when unfastened, as a channel
through which the water could flow.
 The Arabs laughed to see the Feringhee take up in his arms
what ought to be borne on the back.

"Lift that!" said Tewfik to Harold, in a tone of command.

The Englishman's pride rose in arms; he was no slave of

a dirty ignorant Bedouin, to do for him the work of a
bihiste! But common-sense showed Harold that such pride
was worse than folly; he was not told to do anything wrong,
and he had no power to resist with success. The stately
form was bowed, and Harold raised the heavy weight by an
effort of sheer strength, for he had not the professional skill
of a water-carrier. The Arabs laughed to see the Feringhee
take up in his arms, as he would have done a child, what
ought to be borne on the back.

"Put another mashale upon him, where a mashale

should be!" shouted Tewfik.
 As Harold was about to drop the first heavy skin, the
Bedouin bade him forbear. "You shall carry a double load!"
exclaimed the Arab. "One in your own way and one in mine.
Bend your proud back to receive it."

"It is beyond my strength," said Harold, in what Arabic

he could command.

"We will soon see if such be the case!" cried Tewfik,

raising a staff which he had in his hand, as if with intention
to strike.

But the stick did not descend, nor was the double
burden lifted by the pale-faced captive.

A sudden exclamation from the chief caused all eyes to

be suddenly turned towards the south, from which came a
gust of wind so oppressively hot, that it seemed as if it had
come direct from a roaring furnace. Every Arab, as if by
instinct, muffled his face in his mantle, and then threw
himself on the ground; the camels, which had been
kneeling, stretched themselves out, and lay with their long
necks extended, and their noses resting on the sand. Not a
word was spoken save the exclamation, "The simoom! Allah
save us!" which burst from the chief, as he placed himself
so that his camel should be between him and the poisonous
blast which was sweeping towards the encampment. The
sky had almost suddenly become terribly dark, with a livid
tint of purple towards the south. Harold dropped the
mashale, and crouched behind it, resting his brow against
the moist skin.

Then swept the deadly simoom of the desert upon the

party, almost suffocating them with the burning sand which,
it has been said, sometimes not only kills, but so effectually
buries its victims that no traces remain to tell where they
lie! To Harold the scorching blast felt like the breath of the
angel of death, and he was tempted to pray that to him it
might be such indeed. But life was strong within the young
Englishman still: the rushing simoom came and passed over
the prostrate men and beasts, as the heaviest trials
sometimes come, and pass away.

The cloud of hot sand went sweeping on, and—though

with garments clagged with what it had left behind—the
Arabs were able to rise from the ground, uttering
ejaculations which—at least from Harold's lips—took the
form of thanksgiving. Yes, the poor captive could thank
God, he scarcely knew why, that his life was prolonged;
perhaps there was some undefined hope that it had been
spared for some gracious purpose, if for suffering, still for
service. Some blows might yet be struck in the good cause
by the Knight of St. John.

But the simoom of the Arabian desert had had its

message for one who had indeed suffered but never served.
Theresa Petty, lured by the mirage, had wandered from the
encampment, and had been overtaken by the poisonous
blast. Being utterly unprepared for it, the unhappy woman
had been smitten down, as if laid low by a scythe. The
accident, as it seemed, of her lying half over Shelah O'More,
and so forming a kind of screen to the terrified child, had
been the means of preserving the poor little girl.

It was Shelah's bitter cry which guided the Arabs to the

spot, as they were passing on their way towards Djauf.
They had indeed missed their captives from the party, but
Harold could not persuade the Bedouins to make any search
for those whom they deemed of little value. Hartley, who
was on foot, went up to the place where Shelah sat crying
in helpless distress.
 "Where is Miss Petty?" he hastily inquired of the child.

"She's there," said Shelah, pointing to what looked like

a low, a very low mound of sand.

Harold hastily removed some of the sand, uncovering

enough to ascertain that life was quite extinct.

"Dead!" he said in an undertone, but it caught the ear

of Shelah.

"Dead!" repeated Shelah in turn. "The good lady is

dead, and Robin, and now she is dead—I think it will be my
turn next!"

"I hope not," said Harold gently.

"Would you mind?" asked Shelah.

The artless question touched Harold's heart. "Yes, I

should mind very much, Shelah," he said.

The poor child, sobbing, threw herself into his arms,

and clung to the only being near who cared whether she
lived or died.

Harold had not a minute even to utter a prayer by Miss

Petty's corpse. The Arabs, who had been already delayed in
their journey by the simoom, insisted on his instantly
joining the march, and, had Harold lingered, would have
used force to compel submission. Gently young Hartley
raised Shelah, so that, without dismounting, an Arab could
place her before him on his camel. Harold himself had to go
on foot.

The caravan moved slowly on, leaving the corpse of

Miss Petty behind. There was a strange similarity between
the fate of Grace Evendale and that of Theresa, both dying
in an Arabian desert with but a single human being near,
both left in unknown, unmarked graves. And yet the
difference between them was as that between the convict
and the conqueror; one going into endless exile, the other
departing to receive a crown. The comparison suggests less
of similarity than of contrast.

CHAPTER XXI.
ONLY ONE LAMB.

THERE is a beautiful story, with which many are familiar,

of a good missionary who, when too aged to go on with the
work which he loved, was found meekly teaching the
alphabet to a little child, thankful that he had still power to
perform this humble labour for God. Harold was reminded
of this anecdote by the position in which he found himself in
relation to poor little Shelah.

The child, desolate and helpless in a land of strangers,

where the name of Christian was scarcely known, had no
one to whom to look for kindness and protection but Harold.
He had regarded her as unlovely and unloveable; Shelah, in
her merrier days, had excited no sympathy in his mind; but
Christian pity now touched a chord, and that chord wakened
something like music in young Hartley's desolate spirit. As
he marched on painfully in the heat, keeping as near as he
could to the camel on which poor Shelah was perched,
Harold thought much of the future fate of the young Irish
girl. She was of good family, her father a distinguished
officer in the army, and Shelah was his only child. When the
news of her having been carried off by Arabs should reach
India, efforts, and strenuous ones, would doubtless be
made for her deliverance. But Arabia was a large country in
which to search, without newspapers for advertisements, or
postal system for letters, or wires to flash messages with
lightning speed.

"Were I to be separated from Shelah, which is likely

enough," thought Harold, "or were anything to happen to
me, all trace of the child might be utterly lost. Shelah would
be buried in some Mahomedan zenana, and childish and
thoughtless as she is, would probably soon forget
everything about her family and her language. I doubt
whether the poor girl would remember her own name for a
month. I wish that I had some means of stamping it—either
on her form or her memory."

Harold glanced up at the little girl, who still wore her

cardinal's hat, though its colour had almost entirely faded.
The motion of the camel made Shelah appear as if being
rocked on waves; she was clinging to the large bundles
strapped on the camel, in order to feel the motion less.
Harold raised his voice that it might reach the child.

"What is your name?" he asked, to see how far she was

able to identify herself with the daughter of Sir Patrick
O'More.

"Lammikin," cried Shelah, looking down from her perch.

"Tell me your other name," said Harold.

"I don't want another name; I'm just Lammikin; that is

what Robin used to call me."
 "This will never do," thought Harold. Again he raised his
voice:

"Do you know the name of your father?"

"Papa," was the ready reply, and Harold could draw no

other.

"Do you know, my child, where he lives?"

"In some island; but I don't like islands—they are

nothing but sand."

"And like sand is your memory," thought Harold,

realising how short a time it would take to obliterate almost
everything from a mind such as Shelah O'More's. The young
man compassionated the misery to be endured, perhaps for
many long years, by loving parents making a wearisome,
never-ending, useless search in these wild regions after an
only child, hope growing fainter and fainter, and at last
dying away in despair.

A thought occurred to the missionary's mind.

"Shelah, you love singing," he said; "shall I make a little

song for you to sing as you travel along?"

"It's hard to sing with the big beast bumping me up and

down like this," replied Shelah. "But I do like songs, most of
all if they're funny."

Harold, to an easy, popular air, which he had often

heard the child humming, gave the following jingling rhyme.
How strange it was to find himself singing:
 "Shelah O'More; I'm Shelah O'More;
Take me to India's bright, beautiful shore."

The little device had instant success. Shelah for a few

moments loosened her clinging hands in order to clap them.

"I like that song!" she exclaimed, and instantly began to

sing it. Then she paused to ask a question.

"Shall I find the good woman and Robin on India's

bright, beautiful shore?" said the child.

"No," replied Harold, with a quivering lip; "they have

gone to heaven's shore, which is more bright and beautiful
by far."

"Then I'll change the song!" cried Shelah, and she

instantly sang out:

"Take me to heaven's bright, beautiful shore."

Harold took the hint unconsciously given. He who had

hoped to gather in a Christian flock from amongst the
heathen, had here his charge confined to that of one child,
a single lamb to feed for the Master.

"I want you to try something besides singing, poor

Lammikin," he said. "I want to teach you a little prayer to
be said night and morning. It will, I hope, help you to reach
the beautiful place."
 Shelah again loosened her grasp, and clasped her little
sunburnt hands together.

"Say—'Please, Lord, make Shelah a good child, for

Christ's sake,'" said Harold, choosing the simplest petition
which rose to his mind.

"I know a better prayer than that," said Shelah.

"'O God, teach me to love Thee, for the sake of

the Lord Jesus.'

"The kind lady taught me to say that, and Robin gave

me a verse:

"'God is love.'"

"Keep those two precious remembrances of them!"

exclaimed Harold, his dry, heated eyes relieved by
unwonted moisture. "Sing them daily, say them again and
again, till we all meet on the beautiful shore."

Harold himself was no longer utterly wretched. That

calm spirit of submission had come over his mind, which
has been compared to the bending down of the ripe, golden
corn, the sign that the harvest time is near.

So onward proceeded Hartley with the Arab banditti

towards Djauf; whilst Robin, with the Persians, was from
another quarter impatiently pressing on in the same
direction. But the little delay which had been occasioned by
Hassan's flight on Firdosi had prevented the two
movements from coinciding in point of time. In the city of
Djauf the two young Knights of St. John were never to
meet.

CHAPTER XXII.
SLAVERY.

THE pen of an eloquent traveller has thus described the

city which Harold and the Shararat Bedouins entered after
their painful journey through the desert.

"A broad deep valley, descending ledge after ledge, till

its innermost depths are hidden from sight amid far-
reaching shelves of reddish rock; below, everywhere
studded with tufts of palm-groves, and clustering fruit-trees
in dark green patches down to the farthest end of its
windings; a large brown mass of irregular masonry
crowning a central hill; beyond, a tall and solitary tower
overlooking the opposite bank of the hollow, and farther
down small round turrets and flat house-tops half buried
amid the garden foliage."

"Is this India, bright beautiful India at last!" exclaimed

Shelah, looking on the lovely scene with delight. To her, at
least, the sight of houses and fruit-trees gave unmingled
pleasure; the child, enjoying the present, neither took
thought for the future, nor felt regret for the past.

Djauf presented an unusually gay appearance on the

morning when it was entered by Harold and the Bedouin
band. It was the day closing the grand festivities with which
were celebrated the marriage of the Arabian Governor's
eldest daughter. The bazaars were crowded with people in
the gayest of Oriental costumes, and noisy with drums and
other instruments unpleasing to European ears, with
vociferous shouting and gabbling in half-a-dozen different
tongues. The inhabitants of the city were easily
distinguished from the wild sons of the desert, being taller
in stature, lighter in complexion, and franker in manner,
with long curling black locks; the Djaufites showed to
advantage beside the suspicious-looking Bedouins.

Here Persians went prancing by on their high-mottled

steeds, there Arabs, wearing red cotton vests with large
hanging sleeves, their heads enwrapped in kerchiefs striped
red and yellow, lounged along or chattered at the numerous
stalls piled with sweetmeats for which Djauf is famed.
Bihistes, bending under their burdens were with difficulty
making their way through the crowds, stopping frequently
to impart "the gift of God" to the thirsty. Camels, donkeys,
cattle, helped to block up the roads, but no one seemed to
be in a hurry. The day was one intended for pleasure, and
Shelah enjoyed the bright changing scene and the noise, as
if all the tamasha had been got up for her special
amusement.

The centre of all the excitement and gaiety is the castle

in which the governor dwells, and from which the bridal
procession is in a short time to emerge. This castle is a
large mass of irregular masonry, with a thick tower in the
centre, suggestive rather of strength than of beauty. We will
enter through the arched gate, and cross the large paved
court, which is crowded with the bridegroom's followers and
the governor's armed retainers. A hundred sabres flash in
the sun, intermingled with guns, and weapons of ruder
construction. Turbans of various hues, high caps, the fez,
the kerchief twisted round the head, embroidered cloaks
bordered with silver and gold, here a red mantle, there a
costly shawl, with glitter of sparkling jewels which, in the
East, are by no means left to the exclusive use of women,
make the scene suggestive of one read of in the "Arabian
Nights."

An inner court brings us into the Governor's large

reception room called the Khawah, where the potentate of
Djauf sits in state, propped on his gold-striped cushions, to
receive the congratulations of his numerous guests. The
bride is not visible; we must imagine her dressed in red and
gold, and almost weighed down with jewels, the central
point of interest in the zenana, which is as densely crowded
with chattering women as the court and banqueting room
are with men.

But in the midst of the brilliant scene, a cloud is on the

Governor's face. He had promised to his son-in-law the gift
of a favourite Nubian slave, skilled in music, perfect in the
art of preparing coffee, something of a jester withal, and
behold! On the very day of the departure of the wedded
pair, Barahat has fallen down and broken his leg, after—oh!
shameful sound to Mahomedan ears!—too free indulgence
in the forbidden!

"Let not his Highness's mind be disturbed," said a

courtier, whose head was encircled with a kerchief adorned
with a broad band of camels' hair, skilfully entwined with
bright coloured silk. "If the Nubian fell, it was kismat (fate),
the loss of a slave is more easily supplied than that of a
good horse. Some Shararat Bedouins came into the city at
daybreak, bringing with them a handsome slave, of the
complexion of a Circassian and the mien of a prince, and a
white child with hair red as the beard of the Prophet. The
slaves are both for sale."
 "Of what race? Where found? What price do the robbers
demand?" asked the ruler of Djauf.

"They come from some Wiliyati (European) land," said

the Arab; "no robber tells where he found his spoil, these
slaves may have been taken from some wreck on the coast.
Sixty gold tomauns are asked for the young man, and
twenty for the girl."

"What can they do?" asked the Governor, after for a

brief space turning over the subject in his mind, whilst
leisurely sipping his coffee.

The courtier gave a list of accomplishments to which

Harold certainly laid no claim. The white slave was a poet, a
musician; the girl who accompanied him danced to his
playing.

The Arab would not have dared to have declared all this
had he not thought that, the bridal party being on the point
of starting for a place distant hundreds of miles from Djauf,
there was no danger of detection. The sinfulness of fraud
and falsehood never troubled the conscience of the Arab, for
he could not be said to possess one. He had been nurtured
on lies, and felt rather pride than shame at success in
cheating his employer.

After obtaining from the governor the eighty pieces of

gold, the courtier hurried off to make his purchases from
the Shararat Arabs. It brought the hot blood to Harold's
pale cheek when, standing silently by, he heard the
wrangling, the eager bargaining, the noisy asseverations,
the blasphemous appeals to heaven, over the sale of an
Englishman. It was humiliating to have his price beaten
down, as if he had been some mere beast of burden.
 "What are they saying? Why are they so angry? What
are they quarrelling about?" asked Shelah. "And why are
they looking so hard at me?" Harold could not give
utterance to a reply to the questions asked by the poor little
slave.

"After all," thought Harold, "I am not the first one of the
Lord's people to have to endure the humiliation of having a
price put upon me." Harold remembered Joseph; he
remembered One far more exalted than Israel's son, for
whose sacred person pieces of silver had been counted
down. It is only in sin that there is shame.

The courtier was skilful in the art of bargaining, and,

after at least half-an-hour given to noisy disputing, he paid
down forty tomauns for Harold, Shelah being thrown in as a
make-weight by Tewfik, who considered the baronet's child
as a thing of no value at all.

The first result of a change of masters was a very

welcome one to the slaves. Harold had been unable to
change his garments since the day when he had fallen into
Bedouin hands; and this, with the impossibility of bathing,
had been to the English gentleman one of the most
unsupportable of his trials. But, having become a gift from
the Governor of Djauf to his high-born son-in-law, the slave
must appear in befitting guise, with not a grain of dust upon
him. Hartley had at once the luxury of a bath, and then was
clothed from head to foot in spotless white, a muslin turban
was wound around his head, and around his waist was
twisted a kamarband of crimson and gold.

Given over to the charge of some Arab women, Shelah

also underwent a transformation. Greatly enchanted with
her finery, Shelah met Harold about an hour afterwards.
The Lammikin was attired in yellow gauze, spangled with
silver, her red locks hidden under a large veil of the same
gaudy material.

"Am I not grand?—Like a queen!" exclaimed Shelah.

"And are not these people kind to dress me like this! But
oh, Mr. Hartley!" added the Lammikin, as she looked up
with wondering admiration at Harold in his Oriental
costume. "You are quite beautiful! You look like one of the
angels in the book of Bible pictures! You want nothing but
white wings! Do you think that they will grow?" asked the
child.

The faintest of smiles rose to Harold's lips at the artless

question. He thought, with a sigh, of the verse:

"'Oh, that I had wings like a dove, then would I

flee away and be at rest!'"

CHAPTER XXIII.
A PROMISE.

"How beautiful the hour of early dawn,

When the first rays glance up the Eastern sky,
When the bright fingers of the fresh'ning morn
Draw back the veil of dark obscurity,
And give all Nature's beauties to the eye,
Her fairest scenes unfolding to the view;
The lark with buoyant pinion mounts on high,
 And on the emerald lawn the pure soft dew
Sparkles with every beam which breaks the bright
clouds
through.

"Thus on the night of ignorance and sin

The radiant morning of Conversion breaks,
A beam from heaven seems to shine within;
And, as the lark his earthly nest forsakes
And upward soars towards the source of light,—
From bonds of sin the soul enraptured breaks,
And—winged by Faith—springs on her upward flight
Till that clear day when Faith itself is lost in sight!"

A CHANGE, something like that described above, had

come over the spirit of Ali, the Persian. The Amir had never
been an enthusiastic follower of the False Prophet, and what
Ali had heard and seen during his travels in various lands
had extinguished any respect that he had felt for the
Mahomedan faith. He had long suspected the Koran to be a
tissue of lies palmed upon Arabian credulity by an impostor,
a book unworthy of comparison with the Bible, which Ali
had sometimes read in a cursory manner. But to leave hold
of a false religion is a very different thing from grasping a
true one. To extinguish smoky lamps is not a means of
calling in the radiant day.

Ali, till he met a simple, true-hearted Christian, was an

unbeliever as regarded the power of any faith to change the
life. The Amir had been unfortunate in meeting with several
nominal Christians, had shrewdly compared their conduct
with their creed, and rejected the latter because
inconsistent with the former. Ali had, as many do, found a
refuge against the shafts of conscience in carping criticism
of others; he was not worse, so he thought, than many who
believe themselves certain of heaven through the merits of
One whose example they do not follow, whose commands
they do not obey.

But Ali's eyes were now opened; he looked on himself

as stained with sin, and saw in Christianity, such as the
Hartleys had embraced, the only means of being saved from
eternal condemnation. No longer the Persian listened to
Robin's recitals from Scripture in the spirit of a critic; for Ali
was thirsting for the water of life, and could not pause to
comment on the form of the cup which held it. Robin was
delighted, but not surprised, to find that his prayers had
been heard, for had he not pleaded with One whom
Scripture describes as the Hearer of prayer?

It was at night, during the last halt made before Djauf

would be reached, that Ali confessed to Robin his own
desire to become a Christian.

Robin's eyes sparkled with joy.

"I will accompany you and your brother to India as soon

as it is possible to do so," said the Amir, "study your
Scriptures thoroughly, and then receive baptism without
delay."

The expressive face of Robin was suddenly shaded, as if

by a doubt.

"How, do you not desire me to become a Christian?"

asked the Persian quickly.

"I wish it intensely!" cried Robin.

"And have I not already given myself to the Saviour, has

Christ not entered my heart?"
 "Have you given yourself to Him out and out?" asked
the youth. "If Christ have entered your heart are you ready
to do His will in all things?"

"My future conduct will show it."

"But what of the past?" said the younger Hartley,

looking on the ground as he spoke, for he felt pain in giving
pain.

"The past cannot be recalled—you have said that all is

forgiven."

"Yes—as far as regards God; but we must make what

amends we can to man also. When Christ came to
Zacchaeus, the publican received free salvation, but still he
said, 'Lord, the half of my goods I give to the poor, and if I
have done wrong to any man I restore him fourfold.'"

"I do not understand you, boy!" said Ali, and very deep
grew the furrow on his brow. "I have taken one life, and I
cannot restore it; God does not require an impossibility."

Robin was silent, he knew not how to express what was

on his mind: but Ali was resolved to have an explanation.

"If you were in my place what would you do?" asked the
Amir.

"I do not know what I should do, but I know what I

should feel that I ought to do," replied Robin, with some
reluctance.

"What might that be?" asked Ali, looking the young

Englishman full in the face with his keen, piercing eyes.
 Robin met the gaze as he made reply, "Go to my
mother, entreat her forgiveness, and then give myself up to
justice."

This was so contrary to any idea which had ever been

entertained by the Oriental, that his first emotion was that
of astonishment at the childish simplicity which could make
so absurd a suggestion. However, Robin was evidently in
earnest, the warm blood was mantling even to his brow,
and he intuitively clenched his hand as if realising what an
effort it would cost him, what courage he felt that it would
require to do what he deemed to be right in so terrible a
case.

Ali did not lose his temper, but his voice sounded harsh
as, after a pause of some minutes, he expressed himself as
follows:

"There is no justice—I mean according to English ideas

—in Persia. If I became my own accuser, I should but be
regarded as a fool. I should not be injured in life or limb,
but every hanger on at a corrupt court would seize on the
opportunity of robbing me of every piastre that I possess. I
should be stripped of all that I have inherited, all that I
have made by skilful speculations in jewels and horses since
leaving Persia. I should simply be reduced to a penniless
beggar; unless, indeed, by speaking out my opinion
regarding Mahomet, I should be promoted to the rank of a
martyr."

"But surely you should visit your mother?"

"It would be more tolerable to me to own myself a

murderer in the palace of the Shah, than to face her whom
I have bereaved of her favourite son!" exclaimed Ali. "I
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebookfinal.com