Bayesian Network Structure Ensemble Learning 1st Edition by Feng Liu, Fengzhan Tian, Qiliang Zhu 9783540738701 instant download
Uploaded by
pdiwzzt8350Bayesian Network Structure Ensemble Learning 1st Edition by Feng Liu, Fengzhan Tian, Qiliang Zhu 9783540738701 instant download
Uploaded by
pdiwzzt8350Bayesian Network Structure Ensemble Learning 1st
Edition by Feng Liu, Fengzhan Tian, Qiliang Zhu
9783540738701 download
https://ebookball.com/product/bayesian-network-structure-
ensemble-learning-1st-edition-by-feng-liu-fengzhan-tian-qiliang-
zhu-9783540738701-10416/
Download more ebook instantly today at https://ebookball.com
Get Your Digital Files Instantly: PDF, ePub, MOBI and More
Quick Digital Downloads: PDF, ePub, MOBI and Other Formats
A Novel Greedy Bayesian Network Structure Learning Algorithm for
Limited Data 1st Edition by Feng Liu, Fengzhan Tian, Qiliang Zhu
9783540738701
https://ebookball.com/product/a-novel-greedy-bayesian-network-
structure-learning-algorithm-for-limited-data-1st-edition-by-
feng-liu-fengzhan-tian-qiliang-zhu-9783540738701-10414/
Prediction of Protein Subcellular Locations by Combining K Local
Hyperplane Distance Nearest Neighbor 1st Edition by Hong Liu, Haodi
Feng, Daming Zhu 9783540738701
https://ebookball.com/product/prediction-of-protein-subcellular-
locations-by-combining-k-local-hyperplane-distance-nearest-
neighbor-1st-edition-by-hong-liu-haodi-feng-daming-
zhu-9783540738701-9288/
Science of Cyber Security 1st edition by Wenlian Lu, Kun Sun, Moti
Yun, Feng Liu 3030891372 9783030891374
https://ebookball.com/product/science-of-cyber-security-1st-
edition-by-wenlian-lu-kun-sun-moti-yun-feng-
liu-3030891372-9783030891374-19994/
learning bayesian networks 1st edition by Richard Neapolitan ISBN
0130125342 978-0130125347
https://ebookball.com/product/learning-bayesian-networks-1st-
edition-by-richard-neapolitan-
isbn-0130125342-978-0130125347-19568/
Nuclear Power Plants Innovative Technologies for Instrumentation and
Control Systems 1st Edition by Yang Xu, Yongbin Sun, Yanyang Liu,
Feng Gao, Pengfei Gu, Zheming Liu ISBN 9811634580 9789811634581
https://ebookball.com/product/nuclear-power-plants-innovative-
technologies-for-instrumentation-and-control-systems-1st-edition-
by-yang-xu-yongbin-sun-yanyang-liu-feng-gao-pengfei-gu-zheming-
liu-isbn-9811634580-9789811634581-17016/
Dental Digital Photography From Dental Clinical Photography to Digital
Smile Design 1st Edition by Feng Liu ISBN 981131621X 9789811316210
https://ebookball.com/product/dental-digital-photography-from-
dental-clinical-photography-to-digital-smile-design-1st-edition-
by-feng-liu-isbn-981131621x-9789811316210-5472/
A Direct Measure for the Efficacy of Bayesian Network Structures
Learned from Data 1st Edition by Gary Holness 9783540734987
https://ebookball.com/product/a-direct-measure-for-the-efficacy-
of-bayesian-network-structures-learned-from-data-1st-edition-by-
gary-holness-9783540734987-10082/
Making the Key Agreement Protocol in Mobile ad hoc Network More
Efficient 1st edition by Gang Yao, Kui Ren, Feng Bao, Robert Deng,
Dengguo Feng ISBN 3540202080 9783540202080
https://ebookball.com/product/making-the-key-agreement-protocol-
in-mobile-ad-hoc-network-more-efficient-1st-edition-by-gang-yao-
kui-ren-feng-bao-robert-deng-dengguo-feng-
isbn-3540202080-9783540202080-10110/
Approxamation methods for efficient learning of Bayesian networks 1st
Edition by Carsten Riggelsen ISBN 1586038214 9781586038212
https://ebookball.com/product/approxamation-methods-for-
efficient-learning-of-bayesian-networks-1st-edition-by-carsten-
riggelsen-isbn-1586038214-9781586038212-19724/
Bayesian Network Structure Ensemble Learning
Feng Liu1 , Fengzhan Tian2 , and Qiliang Zhu1
1
Department of Computer Science, Beijing University of Posts and
Telecommunications,
Xitu Cheng Lu 10, 100876 Beijing, China
[email protected]
2
Department of Computer Science, Beijing Jiaotong University,
Shangyuan Cun 3, 100044 Beijing, China
Abstract. Bayesian networks (BNs) have been widely used for learning
model structures of a domain in the area of data mining and knowledge
discovery. This paper incorporates ensemble learning into BN structure
learning algorithms and presents a novel ensemble BN structure learning
approach. Based on the Markov condition and the faithfulness condition
of BN structure learning, our ensemble approach proposes a novel sam-
ple decomposition technique and a components integration technique.
The experimental results reveal that our ensemble BN structure learn-
ing approach can achieve an improved result compared with individual
BN structure learning approach in terms of accuracy.
1 Introduction
Bayesian network is an efficient tool to represent a joint probability distribu-
tion and causal independence relationships among a set of variables. Therefore,
there has been great interest in automatically inducing Bayesian networks from
datasets. [6] During the last two decades, two kinds of BN learning approaches
have emerged. The first is the search & score method [2],[9], which uses heuristic
search methods to find the Bayesian network that maximizes some given score
function. Score function is usually defined as a measure of fitness between the
graph and the data. The second approach, which is called the constraint-based
approach, estimates from the data whether certain conditional independences
hold among the variables. Typically, this estimation is performed using statisti-
cal or information theoretical measures [1],[11].
Although encouraging results have been reported, both of the approaches done
so far suffer some computational difficulties in accuracy and cannot overcome
the local maxima problem. A statistical or information theoretical measure may
become unreliable on small sample datasets. At the same time, the computa-
tion of selected score function may also be unreliable on small sample datasets.
Moreover, the CI-testing space and structure-searching space are so vast that
heuristic methods have to be used. So, the two approaches are usually limited
to find a local maxima.
To further enhance the accuracy and to try to overcome the local maxima
problem in BN leaning, this paper proposes an ensemble BN structure learning
R. Alhajj et al. (Eds.): ADMA 2007, LNAI 4632, pp. 454–465, 2007.
c Springer-Verlag Berlin Heidelberg 2007
Bayesian Network Structure Ensemble Learning 455
approach that aims to achieve a better result in BN induction. In Section 2,
we briefly introduces Bayesian network. In Section 3, we propose the overall
process of our learning method. We present the details of our learning approach
in sections 4, 5 and 6. In section 7, experimental results are compared and
analyzed. Finally, we conclude our work in section 8.
2 Bayesian Network
A Bayesian network is defined as a pair B = {G, Θ}, where G is a directed acyclic
graph G = {V (G), A(G)}, with a set of nodes V (G) = {V1 , . . . , Vn } representing
a set of random variables and a set of arcs A(G) ⊆ V (G) × V (G) representing
causal independence/dependence relationships that exist among the variables.
Θ represents the set of parameters that quantifies the network. It contains a
parameter θvi |πi = P (vi | πi ) for each possible value vi of Vi , and πi of Πi .
Here Πi denotes the set of parents of Vi in G and πi is a particular instantiation
of Πi .
For example, Fig.1 shows the Bayesian network called World and the param-
eter table ΘH|{E,F } of the node H. In the World network, the nodes A,B and
E are root nodes which have not inarcs in the World.
A B E F H P(H|E,F) T x
i Si
0 0 0 0.9
D E 0 0 1 0.1
C
1 0 0 0.7
1 0 1 0.3
F 0 1 0 0.8
0 1 1 0.2
1 1 0 0.1
G H 1 1 1 0.9
Fig. 1. An example of Bayesian network (World )
3 Ensemble BN Learning Overview
The overall process of our ensemble BN structure learning approach is shown in
Fig.2.
Our approach belongs to the category of ensemble methods, ”sub-sampling
the training dataset”.[3] Given the original training dataset D, our algorithm ap-
plies sample decomposition technique to generate several training sub-datasets
Di . From each generated training sub-dataset Di , the component learner learns a
component (Bayesian network) BNi . Then, using components integration tech-
nique, these learned components (BNs) are combined into a result Bayesian
network.
456 F. Liu, F. Tian, and Q. Zhu
Sample the Original Dataset
Decomposition
Technique
D1 D2 DS-1 DS
BN Learner
BN1 BN2 BNS-1 BNS
BNs
Integration
Technique the Result BN
Fig. 2. Process of ensemble BN structure learning
4 Root Nodes Based Sample Decomposition Method
BN structure learning is to estimate conditional independences and dependences
among the variables in the training dataset. The two most important sufficient
conditions for BN structure learning are the Markov condition and the faithful-
ness condition. The joint probability distribution P over the training dataset and
the true BN G satisfy the Markov condition if and only if under the distribution
P , a node is independent of its non-descendant nodes given its parent nodes in
the true BN G. The faithfulness condition means that all and only the condi-
tional independence relations true in P are entailed by the Markov condition
applied to G. [10]
Bootstrap sampling used by Bagging methods is a powerful tool for model se-
lection. When learning the structure of a graphical model from limited datasets,
such as the gene-expression datasets, Bagging methods [3] which use the boot-
strap [4] sampling have been applied to explore the model structure [14],[13],[15],
[16]. However, Bagging methods have several disadvantages over BN structure
learning. On the one hand, the distributions generated from the sub-datasets ob-
tained using Bagging methods and the true BN G may be unsatisfied with the
Markov condition and the faithfulness condition. On the other hand, although
Bagging methods may asymptotically converge to the true BN by re-sampling a
large number of times, they require some convergence conditions. For example,
the bagging method using non-parametric bootstrap sampling requires uniform
convergence in the distribution of the bootstrap statistic as well as a continuity
condition in the parameters.[14], [5]
To solve the above problems, we propose a novel sample decomposition
method, which sub-samples the training dataset according to the values of root
nodes in the true BN, for ensemble BN structure learning. The sample decom-
position method is called Root Nodes based Sample Decomposition(RNSD).
4.1 Root Nodes Based Sample Decomposition
The detail of RNSD method is shown in Fig.3. The idea behind this decompo-
sition method is based on the following 3 facts:
Bayesian Network Structure Ensemble Learning 457
1. Learning BN from the sub-dataset sampled from the original training dataset
by limitting the range of values for some root node to be a part of all the pos-
sible values for the root node, we expect to get all the Markov independences
implied by the true BN.
2. Any arc, which is not connected with some root node in the true BN, can
be learned on at least one sub-dataset sampled from the original training
dataset by specifying values for the root node. So, learning on sampled sub-
datasets that all the possible values of the specified root node are considered,
we expect to learn all the “dependences” (arcs) of the true BN.
3. The joint marginal probability distributions of some nodes vary with the
different sub-datasets sampled from the original training dataset by limitting
the different ranges of the values for some root node.
Therefore, RNSD method can guarantee that all the Markov independences
implied by the true BN can be learned from the sampled sub-datasets. Moreover,
RNSD method can also hold the diversity in marginal probability distributions
during components (BNs) learning.
1) Search root nodes Hi (i=1,...,L) on the original dataset D (Note: L is the number of found root nodes);
2) Compute marginal probability tables of every found root node {P(Hi)|1i L};
3) Construct probability tables Ts for all pairs and triples of the found root nodes {P(Hi,Hk), P(Hi,Hj,Hk) | 1 i, j, k L},
where P(Hi,Hk) = P(Hi) * P(Hk), P(Hi,Hj,Hk) = P(Hi) * P(Hj) * P(Hk);
4) For each probability table Ts, sort the probability values by ascendant order;
5) For each probability table Ts, obtain the group Gr(Hi,...,Hk) of the sub-datasets D{H i z hi ! H k z hk } ,..., by
sub-sampling the dataset D given {Hihiġ...ġHkhk}, where (hi,...,hk )ęTs;
6) For each group Gr(Hi,...,Hk) of the sub-datasets, prune the sub-datasets which sampling rate is smaller than ı ;
7) Prune the groups of sub-datasets which have few sub-datasets (such as the groups which only have no more
than 2 sub-datasets)
Fig. 3. Root Nodes based Sample Decomposition method
Take the World network in Fig.1 as an example. Let σ = 0.8. Assume that
the algorithm found the root nodes A, B and E in step 1. The sorted probability
tables after step 4 are shown in Fig.4. In step 5, the groups of the sampled sub-
datasets were generated. Finally, 4 groups of sub-datasets obtained after pruning
in steps 6 and 7 are shown in Fig.5.
4.2 Correctness Proof
Definition 1. Given a joint probability distribution P, X and Y are condi-
tional independent given Z, denoted as Ind(X,Y |Z), if and only if the fol-
lowing statement holds: P (x | y, z) = P (x | z), ∀x, y, z such that P (yz) > 0,
where x, y and z denote an instantiation of the subsets of variables X, Y and Z,
respectively.[11]
Definition 2. Given a joint probability distribution P, X and Y are condi-
tional dependent given Z, denoted as Dep(X,Y |Z), if and only if the fol-
lowing statement holds: P (x | y, z) = P (x | z), ∃x, y, z such that P (yz) > 0,
458 F. Liu, F. Tian, and Q. Zhu
Sampling
Gr(A, B)
rate
D{A0ġB1} 0.98
D{A1ġB1} 0.92
D{A0ġB0} 0.82
E P(E) Sampling
Gr(A, E) Sampling
0 0.2 rate Gr(A, B, E)
A P(A) B P(B) rate
0 0.2 1 0.1 1 0.2 D{A0ġE0} 0.96
D{ A0ġB1ġE0} 0.996
1 0.8 0 0.9 2 0.6 D{A0ġE1} 0.96
D{ A0ġB1ġE1} 0.996
A B P(A,B) A B E P(A,B,E) D{A0ġE2} 0.88
0 1 0 0.004 D{ A0ġB1ġE2} 0.988
0 1 0.02 D{A1ġE0} 0.84
1 1 0.08 0 1 1 0.004 D{ A1ġB1ġE0} 0.984
0 1 2 0.012 D{A1ġE1} 0.84
0 0 0.18 D{ A1ġB1ġE1} 0.984
1 0 0.72 1 1 0 0.016 Sampling
Gr(B, E) D{ A0ġB0ġE0} 0.964
1 1 1 0.016 rate
B E P(B,E) A E P(A,E) 0 0 0 0.036 D{B1ġE0} 0.98 D{ A0ġB0ġE1} 0.964
1 0 0.02 0 0 0.04 0 0 1 0.036
1 1 0.02 0 1 0.04 D{B1ġE1} 0.98 D{ A1ġB1ġE2} 0.952
1 1 2 0.048
1 2 0.06 0 2 0.12 0 0 2 0.108 D{B1ġE2} 0.94 D{ A0ġB0ġE2} 0.892
0 0 0.18 1 0 0.16 1 0 0 0.144
D{B0ġE0} 0.82 D{ A1ġB0ġE0} 0.856
0 1 0.18 1 1 0.16 1 0 1 0.144
0 2 0.54 1 2 0.48 1 0 2 0.432 D{B0ġE1} 0.82 D{ A1ġB0ġE1} 0.856
Fig. 4. Sorted probability tables Fig. 5. Groups of sampled sub-datasets
(σ = 0.8)
where x, y and z denote an instantiation of the subsets of variables X, Y and Z,
respectively.[11]
Assume that the original training dataset D is data faithful to the true BN G.[11]
We take the World network in Fig.1 as an example to prove the correctness.
Proposition 1. Learning a BN from the sub-dataset sampled from the original
training dataset by limitting the range of the values for some root node to be
a part of all the possible values for the root node, we expect to obtain all the
Markov independences implied by the true BN from the learned BN.
Proof. Assume that we obtain the sub-dataset DE=0 from the original training
dataset D by limitting the range of the values for the root node E to be E =
0 ⇔ {(E = 1) ∪ (E = 2)}.
Assume that P denotes the distribution faithful to the true BN and P denotes
the distribution over the sub-dataset DE=0 .
We take 2 cases to consider whether there exists the Markov independences
implied by the true BN in the distribution P over the sub-dataset DE=0 .
1. For the nodes of which the parents set contains the root node E, for example
the node D, there exists the Markov independence Ind(D, C | A, B, E) in
the true BN.
According to the definition of conditional independence, there exists:
P (d | c, a, b, E = 1) = P (d | a, b, E = 1)
P (d | c, a, b, E = 2) = P (d | a, b, E = 2)
Bayesian Network Structure Ensemble Learning 459
We can infer P (d | c, a, b, E = 2) = P (d | a, b, E = 2).
According to the definition of conditional independence, we can infer that
there exists Ind(D, C | A, B, E) in the distribution P over the sub-dataset
DE=0 .
2. For the nodes of which the parents set does not contain the root node E,
for example the node G, there exists the Markov independences Ind(G, C |
F, H) and Ind(G, E | F, H) in the true BN.
According to the definition of conditional independence, there exists:
P (g | c, f, h, E = 1) = Ngcf h1 /Ncf h1 = P (g | f, h)
P (g | c, f, h, E = 2) = Ngcf h2 /Ncf h2 = P (g | f, h)
P (g | f, h, E = 1) = Ngf h1 /Nf h1 = P (g | f, h)
P (g | f, h, E = 2) = Ngf h2 /Nf h2 = P (g | f, h)
We can infer that P (g | c, f, h) = P (g | f, h).
According to the definition of conditional independence, we can infer that
there exists Ind(G, C | F, H) in the distribution P over the sub-dataset
DE=0 .
According to case 1, 2, we infer that Proposition 1 is correct.
Lemma 1. Learning a BN from the sub-dataset sampled using RNSD method,
we can obtain all the Markov independences implied by the true BN from the
learned BN.
Proof. Using the same way as Proposition 1, we can infer it.
Proposition 2. Learning BNs from the sub-datasets which are sampled by by
limitting the range of values for some root node to be a part of all the possible
values for the root node, if all the possible values of the root node can be included
in the sub-datasets, then we can obtain all the edges of the true BN from the
learned BNs.
Proof. There is an edge between the node C and the node F in Fig.1. We can get
Dep(C, F | A) and Dep(C, F | A, E). According to the definition of conditional
dependence, we can infer the following formula:
Dep(C, F | A, E) ⇔ ∃e ∈ E, Dep(C, F | A, E = e) (1)
According to Dep(C, F | A)DE=0 ⇔ Dep(C, F | A, E = 0)D and the formula
(1), we can infer the proposition.
According the above inferences, we can conclude that the BNs learned from
the sub-datasets, which are sampled using our RNSD method, include all the
Markov independences and edges implied by the true BN.
460 F. Liu, F. Tian, and Q. Zhu
4.3 Root Nodes Search Method
The search method sees Fig.6.
1) Conduct order-0 CI tests for each pair nodes, then build the undirected graph UG0;
2) Conduct order-1 CI tests for any three nodes X, Y and Z that in the undirected graph UG0, X
and Y, and Y and Z, are directly connected; and X and Z are not directly connected, if Dep(X,
Z |Y), then direct the edges XĺY and ZĺY;
3) Find the maximal cliques {Ga1,...,Gak} consisting of the nodes which have no inarcs, that every
clique is undirected complete graph and has at least one outarc;
4) Order the maximal cliques by the number of nodes in ascendant order and prune the cliques
Gai that ||Gai|| > G ;
5) For every maximal clique, use the exhaustive search & Bayesian score function method to
learn the root node in the maximal clique;
6) Detect and delete pseudo root nodes.
Fig. 6. Root nodes search method
The idea behind the search root nodes method is based on the following
assumption, which is correct in most situations both for synthetic datasets and
for real-life datasets:
Assumption 1. If there exists a directed path X −→ Y between node X and
node Y in a Bayesian network, then Dep(X, Y | N U LL).
Under the above assumption, we can obviously infer that every clique obtained
after the 4th step of the method has one and only one root node.
In most cases, Assumption 1 is satisfied. However, some exceptions may occur
when there are many nodes (normally, the number of nodes including the two
nodes on the path 5) on the directed path between two nodes, that is, the two
nodes may be independent conditional on NULL. Moreover, even if Assumption
1 is satisfied in any situation, some results after step 5 in Fig.6 may be pseudo
root nodes on limited datasets.
We take one step to solve the pseudo root nodes problem. The step is to detect
pseudo root nodes and delete them (see step 6 in Fig.6). The detection for pseudo
root nodes is based on 2 kinds of independences. One kind of independences is
the Markov independences given the obtained root node of other nodes in the
maximal clique. The other kind of independences is the independences among
root nodes. Firstly, if the first kind of independences given some found root node
is not satisfied, then the found root node is pseudo root node and prune the
pseudo root node. Secondly, if the second kind of independences is not satisfied
by the two found root nodes, then at least one found root node is pseudo root
node, and we prune the two root nodes.
For example, during the World network learning, the running result for every
step of our root nodes searching method sees Fig.7.
Our root nodes search method does not have to find all the root nodes in the
true BN, it is enough to find several root nodes in the true BN for our ensemble
BN learning in terms of accuracy. We can also use other methods to search root
nodes, such as the RAI algorithm [12].
Bayesian Network Structure Ensemble Learning 461
A B A B
B
B E A
D E D E E
C C
C A
F F
G H G H
(Step 3 & (Step 5
(Step 1) (Step 2) 4) & 6)
Fig. 7. Search root nodes in the World network
5 Bayesian Network Learner
Bayesian network learner is an individual BN learning algorithm and a build-
ing block of ensemble BN structure learning algorithm. Normally, it needs to
be computationally efficient. Therefore, we selected OR algorithm [9], TPDA
algorithm [1], and other algorithms using fast heuristic search (such as Greedy
search) methods [8] as our BN learners. In our implementations for these algo-
rithms, we applied partial nodes order information which was acquired by our
root nodes search method.
6 Bayesian Networks Integration Method
Our integration method includes 2 parts: the integration of the Bayesian net-
works in the same group; the integration of the intergroup undirected networks.
6.1 Intragroup Bayesian Networks Integration
The method for intragroup Bayesian network integration is shown in Fig.8.
For any edge e, (
e is undirected edge of the arc e) in the BNs, consider the
quantity:
1
L
P (
e) = e ∈ BNi }
1{
L i=1
If P ( e ), then it is more probable that e exists in the true BN than e
e) > P (
does. Furthermore, if P ( e) > 1/2, we classify edge e as “true”. Therefore, we
can obtain the most probable BNs represented in the form of undirected network
U Gi which every edge e has a probability table g(e) of {→, ←, ↔}.
462 F. Liu, F. Tian, and Q. Zhu
In the group Gri, assume we have learned L Bayesian networks BN1 ,! , BN L :
1) For the Bayesian Networks BN1 ,! , BN L , according to the principle of simple
voting, compute the probability of every edge e without considering
orientation in the Bayesian Networks
2) Prune the edges whose probabilities are less than 1/2, and create a undirected
network UGi, which every edge e has a probability table g(e)
about ^o, m, l`
Fig. 8. Intra-group Bayesian Networks Integration
6.2 Intergroup Undirected Networks Integration
Assume there are m groups of sub-datasets sampled using RNSD method. After
intragroup Bayesian networks integration for every group Gri (i = 1, . . . , m),
we obtained m undirected networks U Gi . For any possible arc e, consider the
quantity:
1
m
P (e) = e ∈ U Gi , i = 1, . . . , m}
p(e)1{
m i=1
Note: Weight p(e) is the probability value of {→, ←}, where p(→) = g(→)+g(↔)
and p(←) = g(←) + g(↔).
Finally, we take search method and score function to generate the result
Bayesian network of our ensemble BN structure learning method. The process
sees Fig.9.
Assume we have learned m undirected networks UGi (i=1,...,m):
1) For the undirected networks UGi (i=1,...,m), according to the principle of
weighted voting, compute the probability value of every possible arc which
exists in the undirected networks
2) Order arcs by the probability values in ascendant order, and apply exhaustive
or heuristic search method and Bayesian score function to learn the maximal
score Bayesian network by ‘adding arc’, ‘deleting arc’ and ‘reversing arc’
operators
Fig. 9. Inter-group Bayesian Networks Integration
7 Experimental Results
We implemented OR algorithm, OR-BWV algorithm, OR-HNSD algorithm,
TPDA algorithm, TPDA-BWV algorithm, and TPDA-HNSD algorithm. OR-
BWV and TPDA-BWV algorithms use Bagging sampling method and weighted
voting integration method. Tests were run on a PC with Pentium4 1.5GHz and
1GB RAM. The operating system was Windows 2000. 4 Bayesian networks were
used. From these networks, we performed experiments with 500, 1000, 5000
Bayesian Network Structure Ensemble Learning 463
Table 1. Bayesian Networks
BN Nodes Num Arcs Num Roots Num Max In/Out-Degree Domain Range
Alarm 37 46 12 4/5 2-4
Barley 48 84 10 4/5 2-67
Insur 27 52 2 3/7 2-5
Table 2. Average BDe(Alarm-OR) Table 3. Average BDe(Alarm-TPDA)
SIZE OR OR-BSW OR-RNSD SIZE TPDA TPDA-BSW TPDA-RNSD
500 -15.1000 -14.9235 -13.8901 500 -18.0973 -17.8045 -17.3650
1000 -14.8191 -14.1917 -13.7610 1000 -15.4286 -15.0012 -14.2455
5000 -13.9041 -13.8941 -13.1002 5000 -14.4960 -14.4731 -13.2682
Table 4. Average BDe(Barley-OR) Table 5. Average BDe(Barley-TPDA)
SIZE OR OR-BSW OR-RNSD SIZE TPDA TPDA-BSW TPDA-RNSD
500 -82.3448 -81.4104 -80.1943 500 -103.7931 -117.5443 -110.5973
1000 -78.9655 -78.2544 -76.2538 1000 -111.0690 -112.2151 -103.5482
5000 -76.7081 -75.2273 -73.3371 5000 -116.8919 -106.7328 -99.8341
Table 6. Average BDe(Insur-OR) Table 7. Average BDe(Insur-TPDA)
SIZE OR OR-BSW OR-RNSD SIZE TPDA TPDA-BSW TPDA-RNSD
500 -24.0167 -23.3812 -24.0010 500 -28.7857 -28.3471 -28.5172
1000 -22.6077 -21.5445 -22.5077 1000 -25.1111 -24.8157 -25.0111
5000 -19.4286 -18.9523 -19.2286 5000 -20.8571 -20.7916 -20.5571
training cases each. For each network and sample size, we sampled 10 original
datasets and record the average results by each algorithm. Moreover, we applied
Bagging sampling with 200 times in OR-BWV and TPDA-BWV algorithms. Let
σ = 0.8 in Fig.3 and δ = 5 in Fig.6.
We compared the accuracy of Bayesian networks learned by these algorithms
according to the average BDeu score. The BDeu score corresponds to the poste-
riori probability of the structure learned.[7] The BDeu scores in our experiments
were calculated on a seperate testing dataset sampled from the true BN contain-
ing 50000 samples. Tables 2-7 report the results.
There are several noticeable trends in these results. Firstly, as expected, as the
number of instances grow, the quality of learned Bayesian network improves, ex-
cept to TPDA for Barley network (500, 1000, 5000). It is due to that constraint-
based method is unstable for limited datasets (500, 1000, 5000) relative to Barley
network. At the same time, we can see that TPDA-BWV algorithm and TPDA-
RNSD algorithm improve the stability of TPDA, that is, the quality of learned
Bayesian networks by TPDA-BWV and TPDA-RNSD improves with the in-
crease of sample size. Secondly, our RNSD based ensemble algorithms OR-RNSD
and TPDA-RNSD are almost better than or at least equal to the individual
Bayesian network learning algorithms in terms of accuracy on limited datasets.
464 F. Liu, F. Tian, and Q. Zhu
Thirdly, in most cases, our ensemble algorithms have better performance than
BWV ensemble algorithms. Finally, for Bayesian networks with few root node
(such as Insur network), our ensemble algorithms have little improvement on
learning accuracy. So, they are ineffective for these Bayesian networks.
8 Conclusion
We proposed a novel sampling technique and a components integration technique
to incorporate ensemble learning into BN structure learning. Our results are
encouraging in that they indicate that the our method achieved a more accurate
result BN than individual BN learning algorithms.
References
1. Cheng, J., Greiner, R., Kelly, J., Bell, D., Liu, W.: Learning Belief Networks form
Data: An Information Theory Based Approach. Artificial Intelligence 137(1-2),
43–90 (2002)
2. Cooper, G., Herskovits, E.: A Bayesian Method for Constructing Bayesian Belief
Networks from Databases. In: Ambrosio, B., Smets, P. (eds.) UAI ’91. Proceedings
of the Seventh Annual Conference on Uncertainty in Artificial Intelligence, pp.
86–94. Morgan Kaufmann, San Francisco (1991)
3. Dietterich, T.G.: Machine Learning Research: Four Current Directions. AI Maga-
zine 18(4), 745–770 (1997)
4. Davison, A.C., Hinkley, D.V.: Bootstrap Methods and Their Application, 1st edn.
Cambridge Press, New York (1997)
5. Efron, B., Tibshirani, R.J.: An Introduction to the Bootstrap, 1st edn. Chapman
Hall, New York (1993)
6. Heckerman, D.: Bayesian Networks for Data Mining, 1st edn. Microsoft Press,
Redmond (1997)
7. Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian Networks: the
Combination of Knowledge and Statistical Data. Machine Learning 20(3), 197–243
(1995)
8. Chickering, D.M., Heckerman, D., Geiger, D.: Learning Bayesian Network: Search
Methods and Experimental Results. In: Fisher, D., Lenz, H. (eds.): Learning from
Data: Artificial Intelligence and Statistics 5, Lecture Notes in Statistics 112, 143–
153. Springer, New York (1996)
9. Moore, A.W., Wong, W.K.: Optimal Reinsertion: A New Search Operator for Ac-
celerated and More Accurate Bayesian Network Structure Learning. In: Fawcett,
T., Mishra, N. (eds.) ICML 2003 – Machine Learning. Proceedings of the Twentieth
International Conference, pp. 552–559. AAAI Press, Washington DC (2003)
10. Pearl, J.: Causality: Models, Reasoning, and Inference, 1st edn. Cambridge Press,
London (2000)
11. Spirtes, P., Glymour, C., Scheines, R.: Causation, Prediction and Search, 2nd edn.
MIT Press, Massachusetts (2000)
12. Yehezkel, R., Lerner, B.: Recursive Autonomy Identification for Bayesian Network
Structure Learning. In: Cowell, R.G., Ghahramani, Z. (eds.) AISTATS05. Proceed-
ings of the Tenth International Workshop on Artificial Intelligence and Statistics,
pp. 429–436. Society for Artificial Intelligence and Statistics, London (2005)
Bayesian Network Structure Ensemble Learning 465
13. Friedman, N., Goldszmidt, M., Wyner, A.: On the Application of the Bootstrap
for Computing Confidence Measures on Features of Induced Bayesian Networks.
In: Heckerma, D., Whittaker, J. (eds.) Learning from Data: Artificial Intelligence
and Statistics VII. Proceedings of the Seventh International Workshop on Artificial
Intelligence and Statistics, pp. 197–202. Morgan Kaufmann, San Francisco (1999)
14. Friedman, N., Goldszmidt, M., Wyner, A.: Data Analysis with Bayesian Networks:
A Bootstrap Approach. In: Laskey, K.B. (ed.) UAI ’99. Proceedings of the Fif-
teenth Conference on Uncertainty in Artificial Intelligence, pp. 196–205. Morgan
Kaufmann, San Francisco (1999)
15. Friedman, F., Linial, M., Nachman, I., Pe’er, D.: Using Bayesian Networks to
Analyze Expression Data. Journal of Computational Biology 7, 601–620 (2000)
16. Pe’er, D., Regev, A., Elidan, G., Friedman, N.: Inferring Subnetworks from Per-
turbed Expression Profiles. Bioinformatics 17(Suppl. 1), 1–9 (2001)
Other documents randomly have
different content
a dainty little phaeton, in which the general was to drive Cora and
Miss Wilford, drawn by two of the sleekest, roundest, and sauciest
little ponies that ever came out of Ultima Thule.
I was to drive the drag to the meet; and, after the hunt,
Berkeley was to meet us at a certain point on the Cupar Road, and
drive the vehicle home, if I felt disposed to yield the ribbons to him,
which I had quite resolved to do.
Of the noise and excitement, the spurring, yelping, and
hallooing, sounding of horns, and cracking of whips; the greetings of
rough and boisterous country friends; the criticisms that ensued on
dogs, horses, and harness; of how the cover was drawn, and the fox
broke away; how huntsmen and hounds followed "owre bank, bush,
and scaur," as if the devil had got loose, and life depended on his
instant re-capture, and of all the incidents of the hunt, I need give
no relation here.
The afternoon was well-nigh spent before we saw the last of my
uncle's companions; and to the luncheon provided by Mr. Binns we
had done full justice, the roof of the drag being covered by a white
cloth, and improvised as a dining-table, whereon was spread a
déjeûner service of splendid Wedgwood ware, the champagne
sparkling in the sun, and the long glasses of potash and Beaujolais
foaming up for the thirsty; and Largo Law, a green and conical hill,
verdant to its summit a thousand feet above the waters of the bay,
was throwing its shadow to the eastward, when we made
arrangements for our return; and, thanks to dear Cora's tact and
management, rather than my own—for timidity and doubt
embarrassed me—I contrived to get Lady Louisa into the tandem.
After which, by giving a hint to Willie Pitblado, he managed to set
the horses kicking and plunging in such an alarming fashion that it
was necessary to give them their heads for a little way, as if to
soothe their ruffled tempers, just as he adroitly had got into the
back seat.
Lady Chillingham, the M.P., the Misses Spittal, and
Rammerscales were all bundled into the drag; others were on the
roof, great-coated or well-shawled, for a cool drive home, and the
whole party set out for the Glen, viâ Clatto and Collessie, a twenty-
five miles' drive.
It was past the hour of three before all was packed up and we
were all ready to leave Largo. The grave old butler, Binns, looked at
his watch, and said—
"Mr. Newton, you know the route we go by."
"Yes; round by Dunnikier Law."
"That is the road Sir Nigel wished us to drive; but you'll require
to use your whip if we are to be home before dark."
"Never fear for that, Binns," said I, while leading the way in the
tandem with Lady Louisa beside me, and no attendant or other
companion, save Willie Pitblado, who had or had not ears and eyes
just as occasion required, Mamma Chillingham believing the while
that she was with other ladies in the close carriage.
"Keep a tight hand on the leader, sir," whispered Pitblado; "she's
a blood mare, rather fresh from the stall, and overcorned a bit."
"She is hard-mouthed," said I, "and pulls like the devil."
"As for the wheeler, I think the splinter-bar is too low, and she
kicks and shies at it; but the breeching is as short as we could make
it. Keep a sharp look out on both, sir," said he, warningly, and then
relapsed into apparent immobility.
For the first time since our introduction had I been alone with
Lady Louisa—I say alone, for I did not count on my servant, who
seemed wholly intent on looking anywhere but at us, and chiefly
behind, as if to see how soon we could distance the four-in-hand
drag and the rest of our party.
The vehicle we occupied was a hybrid affair, which my uncle
frequently used, half gig and half dog-cart, four-wheeled, with
Collinge's patent axles, lever drag, and silver lamps, smart, strong,
light, and decidedly "bang up."
We went along at a spanking pace. My fair companion was
chatty and delightfully gay; her dark eyes were unusually bright, for
the whole events of the day, and the lunch al fresco, had all tended
to exhilaration of spirits.
She forgot what her rigid, aristocratic, and match-making
mamma might think of her being alone thus with a young subaltern
of lancers; but though her white ermine boa was not paler than her
complexion usually was, she had now a tinge, almost a flush, on her
soft, rounded cheek that made her radiantly beautiful, and I felt that
now or never was the time to address her in the language of love.
I knew that the crisis had come; but how was I to approach it?
CHAPTER XI.
The rocky guardians of the clime
Frown on me, as they menaced death;
While echoing still in measured time
The gallop of my courser's hoof,
They hoarsely bid me stand aloof.
Where goest thou, madman? Where no shade
Of tree or tent shall screen thy head.
Still on—still on; I turn my eyes—
The cliffs no longer mock the skies:
The peaks shrink back, and hide their brow,
Each other's lofty peaks below.
FROM THE POETRY OF MICKIEWICZ.
As if inspired by fortune, or my good genius, Lady Louisa began
thus, in a low voice—
"By the way, Mr. Norcliff, you were to have shown me the house
in which Alexander Selkirk—or Robinson Crusoe—was born in 1676, I
think you said?"
"Oh; it is only a cottage, consisting of one storey and a garret;
but the next time we come to Largo, I shall show you his flip-can,
musket, and a lock of his hair."
"Ah, that reminds me, Mr. Norcliff, that you must return to me
the lock of hair which you obtained when inspired with romance by
Miss Calderwood's legend last night."
"Lady Louisa, I implore your permission to retain it," said I, in a
low voice.
"To what end, or for what reason?" she asked, with a furtive
smile.
"I am going far, far away, and it will serve as a memento of
many happy days, and of one whom I shall never cease to
remember, but with——"
"Why, you don't mean to say that—that you are serious?" she
asked, in a voice that betrayed emotion, while my heart rose to my
trembling lips, and I turned to gaze upon her with an unmistakable
expression of love and tenderness, which made her colour come and
go visibly.
Reassuring herself, she began to smile.
"Perhaps your creed is a soldier's one?" said she, with a little
convulsive laugh, as she tied her veil under her chin.
"A soldier's! I hope so; but in what sense do you mean?"
"'To love all that is lovely, and all that you can,' as the song has
it."
I laid a hand lightly on her soft arm, and was about to say
something there could be no misconstruing, while a film seemed to
pass over my eyes, and my soul rose to my lips; but Pitblado, who,
whether he was listening or not, had a sharp eye on the cattle, now
said—
"Beg your pardon, sir, but I don't like the look of that leader."
"The blood mare with the white star on her forehead," said I,
touching her lightly on the flank with the whip, and making her
curvet; "she is usually very quiet."
"Perhaps so, sir; but she's always clapping her ears close down
—throwing her eyes backward, and showing the whites. She's up to
mischief, I'm certain."
"Jump down, then," said I, "shorten the curb, and lengthen the
traces by a hole or two."
This was done in a trice; Willie sprang into his seat like a
harlequin, and away we went from the Kirktoun of Largo at a
rasping pace.
"She's a lovely animal, with pasterns like a girl's ankles; but
she's clapping her tail a little too close in for my taste, sir, and she's
up to some devilry," persisted Pitblado, and ere long his surmises
proved correct.
"We've left the drag behind; distanced it clean already," said I.
"It's a heavier drag than the regimental one at head-quarters,
sir," said Willie, taking the hint to look back now; but the sound of
hoofs or wheels could no longer be detected in the still evening air
behind.
Full of blood and ill-natured, over-corned, and anxious to get
back to their stables, the speed of the animals increased to a pace
that soon became alarming, and the light vehicle to which they were
harnessed, as I have said, a tandem, swept along like a toy at their
heels, while we flew eastward by Halhill; and, ere we reached the
woods of Balcarris, where the road turns due north, and round by
the base of Dunnikier Law, it was evident that they were fairly and
undoubtedly off!
The leader had got the bit between her teeth, and, when
descending a hill-side, the splinter-bar goaded the wheeler to
madness. All my strength, together with Pitblado's, failed to arrest
their mad career, and, while imploring Lady Louisa, who clung to me,
"to hold fast, to sit still," and so forth, I bent all my energies rather
to guide them along, and avoid collisions, than to attempt to stop
them; and, to add to our troubles, the patent drag gave way.
Luckily, the road was smooth, and free from all obstruction.
"To the left, sir—to the left," shouted Pitblado, as we came to a
place where two roads branched off; "that is Drumhead. Our way
lies due west."
Pitblado might as well have shouted to the wind; the infuriated
brutes took their own way, and tore at an awful pace due north.
Horses pasturing by the wayside trotted to the rear, and sheep
browsing in the fields fled at our approach; cattle kicked up their
heels, and scampered away in herds. House-dogs barked, terriers
yelled, and pursued us open-mouthed; children, ducks, cocks, and
hens fled from the village gutters; peasants, at their cottage doors,
held up their hands, with shouts of fear, while broad fields and lines
of leafless trees, turf dykes, and hedges, drains, and thatched
dwellings seemed all to fly past with railway speed, or to be
revolving in a circle round us.
A shriek of commiseration burst from my affrighted companion,
when, just as we swept past the base of Drumcarra Craig, in the
cold, bleak, and elevated district of Cameron, poor Willie Pitblado,
who had risen to give me the assistance of his hands in bearing on
the reins, or for the last time to try and let down the faulty drag, fell
out behind, and vanished in a moment. And now before us spread
Magus Muir, where the graves of Archbishop Sharpe's murderers lie
in a field that has never been ploughed even unto this day.
Twilight had come on, and a brilliant aurora, forming great
pillars of variegated light, that shot upward and downward from the
horizon to the dome of heaven, filled all the northern quarter of the
sky with singular but many masses of streamers. Thus, the brilliance
of the atmosphere cast forward in strong and black outline the range
of hills that bound the Howe of Fife, and terminate the valley
through which the Ceres flows to join the Eden; and all this, I think,
conduced to add to the terror of the horses.
Pitblado's fate greatly alarmed and concerned me, for he was a
brave, handsome, and faithful fellow, and an old acquaintance; but I
had another—a nearer, dearer—and more intense source of anxiety.
If she who sat beside me, clinging to me, and embracing my left arm
with all her energy—she whom I loved so deeply, and whom I had
lured into the tandem, when she might have been safely in the drag
or carriage, should lose her life that night, of what value would my
future existence be, embittered with such a terrible reflection?
"If a linchpin comes loose, or a trace gives way," thought I, "all
will be over with us both."
"Oh, Mr. Norcliff, Mr. Norcliff!" she exclaimed, while the tears,
which she had no means of wiping away, streamed over her pale
and beautiful face, and while her head half-reclined on my shoulder.
"Heaven help us, this is terrible—most terrible! We shall certainly be
killed!"
"Then I hope it shall be together," I exclaimed. "Lady Loftus—
dear Lady Loftus—dearest Louisa (here was a jump) trust to me,
and me only! (what stuff men will talk; who else could she trust to?)
and if it is in the power of humanity to save you, you shall be saved,
or I shall die with you. Louisa, oh, Louisa, hear me. I would not—I
could not survive you; but—but sit still, sit close, grasp me and hold
on for Heaven's sake. (D—n that leader!) Oh, Louisa, I love you, love
you dearly and devotedly. You must believe me when I say it at a
time like this; when death, perhaps, is staring us face to face. Speak
to me, dearest!"
I felt that the day, the hour, the moment of destiny had come;
that time of joy or sorrow forever, and casting all upon it, committing
the reins to my right hand, I threw my left arm round her, and
pressing her to my breast, told her again and again how fondly I
loved her, while still our mad steeds tore on.
"I know that you love me, Mr. Norcliff," she said, in a low and
agitated voice, as her constitutional self-possession returned. "I have
long seen it—felt it."
"My adorable Louisa!"
"And I will not—will not——"
She paused, painfully.
"What? Oh, speak."
"Deny that I love you in return."
"Heaven bless you, my darling, for saying so; for lifting a load of
anxiety from my heart, and for making me so happy," I whispered,
making an effectual effort to kiss her forehead.
"But then, Mr. Norcliff——"
"Alas! yes; but what?"
"There is mamma; you know, perhaps, her views concerning me
—ambitious views; but we must take another time, if Heaven spares,
to talk of that matter."
"What time so good as this?" I exclaimed impetuously, as we
tore along, and Magus Muir, the Bishop's Wood, and Gullane's
gravestone were left behind. "Poor me, a lieutenant of the lancers;
and the earl, your father."
"Oh, dear papa—good, easy man—I don't think he troubles his
head much in the affair; but if mamma knew all this, such a violation
of her standing orders, heaven help us!"
She could almost have laughed but for the peril on which we
were rushing, and a shrill little cry escaped her, as the leader
suddenly quitted the hard highway, and, followed by the wheeler,
passed throughan open field gate, and continued at the same
frightful speed across a large space of pasture land that sloped
steeply down to where my forebodings told me the Eden lay, and
there, sure enough, in less than a minute, we could see the river
rolling among the copsewood, with its waters swollen by the snows
that had recently melted among the Lomond hills.
Though a placid stream usually, and having a pretty level
course, in that quarter the banks were rugged, and the bed full of
fallen larches and large boulder stones. If the vehicle overturned,
what might be the fate of her who had just acknowledged that she
loved me?
A prayer—almost a solemn invocation—rose to my lips, when,
with the rapidity of light, the thought occurred to me of heading the
leader towards a little stone bridge that spanned the stream. It was
a mere narrow footway for shepherds, sheep, and cattle, and not of
sufficient breadth to permit the passage of a four-wheeled gig; but I
knew that if the latter could be successfully jammed between the
walls, the course of the runaways would be arrested.
There was no alternative between attempting this and risking
death from drowning or mutilation in the rugged bed of the swollen
stream.
Down the steep grassy slope our foam-covered cattle rushed
straight for the narrow bridge; I grasped the rail of the seat with one
hand and arm; the other was round Louisa, lest the coming shock
might throw us off. In an instant we felt it, and she clung to me,
half-fainting, as there was a terrible crash, a ripping and splitting
sound, as wood was smashed and harness rent. Our course was
arrested—the wheels and axle of the fore-carriage wedged between
the stone walls of the narrow bridge, the wheeler kicking furiously at
the splinter-bar and splash-board, and the leader, the blood mare,
the source of all the mischief, hanging over the parapet in the
stream, snorting, half-swimming, and for ought I cared, wholly
hanging.
My first thought was my companion. We both trembled in every
limb as I lifted her gently to the ground, and placed the seat-
cushions on a stone, where she might sit and compose herself till I
considered what we should do next, and where we were.
She was greatly agitated, but passively permitted me to encircle
her with my arms, to assure her that she was safe, to press her
hands, and to wipe away her tears caressingly. I forgot all about
poor Pitblado, "spilt" on the road, all about my uncle's best blood
mare hanging in the traces, and all about the half-ruined gig.
In short, I felt only the most exquisite joy that I had gained, as
it were, life and Louisa together. It was that moment of intense
rapture, when, combined with the natural revulsion of feeling
consequent to escape from a deadly peril, I enjoyed that emotion
which a man feels once, and once only, in a lifetime, when the first
woman he loves confesses to a mutual regard; and, half-kneeling, I
stooped over her, kissing her again and again, assuring her—of I
know not what.
From one of her fingers I transferred to mine a ring of small
value—a pearl set in blue enamel, leaving in its place a rose
diamond. It was a beautiful stone, of the purest water, which I had
found when our troops sacked the great pagoda at Rangoon, and I
had it set at Calcutta by a jeweller, who assured me that it was
worth nine hundred rupees, or ninety pounds, and I only regretted
now that it was not worth ten times as much, to be truly worthy of
the slender finger on which I placed it.
She regarded me with a loving smile on her pale face, and in
the quiet depths of her soft dark eyes, as she reclined in my arms. I
gazed on her with emotions of the purest rapture. She was now
humbled, gentle and loving—this brilliant beauty, this proud earl's
daughter—mine, indeed—all that a man could dream of as
perfection in a woman or as a wife; at least, I thought so then; and I
was not a little proud of the idea of what our mess would say—the
colonel, Studhome, Scriven, Wilford, Berkeley, and the rest—of a
marriage that would certainly be creditable to the regiment, though
we had titles and honourables enough in the lancers; and already, in
fancy, I saw myself "tooling" into Maidstone barrack-square in a
dashing phaeton, with a pair of cream-coloured ponies, with Norcliff
and Loftus quartered on the panels, and silver harness, and Louisa
by my side, in one of the most perfect of morning toilettes and of
marriage bonnets that London millinery could produce.
Poor devil! with only two hundred per annum besides my pay,
and the war before me, I was thus acquiring castles in Airshire, and
estates in the Isle of Sky.
Oblivious of time, while the woods and hills of Dairsie were
darkening against the sky, while the murmuring Eden flowed past
towards the Tay, and the ever-changing spears and streamers of the
northern aurora were growing brighter and more bright, I remained
by the side of Louisa, wholly entranced, and only half-conscious that
something should be done to enable us to return home; for night
was coming on—the early night of the last days of January, when
the sober sun must set at half-past four—and I knew not how far we
were from Calderwood Glen.
Suddenly a shout startled us; the hoofs of horses were heard
coming rapidly along the highway, and then three mounted men
wheeled into the field and rode straight towards us. To my great
satisfaction, one proved to be my faithful fellow, Willie Pitblado, who,
not a wit the worse for his capsize on the road, had procured horses
and assistance at the place called Drumhead, and tracked us to
where we lay, wrecked by the old bridge of the Eden.
"Poor Willie," said Louisa, "I thought you were killed."
"No, my lady," said he, touching his hat; "it's lang or the de'il
dees by the dykeside."
Of this answer she could make nothing.
The gig was now released and run back, and though scratched,
splintered, and started in many places by the shock to which it had
been subjected, it was still quite serviceable. The wheeler was
traced to it again, the leader, her ardour completely cooled now, was
fished out of the stream, and harnessed again, and in less than half
an hour, so able had been the assistance rendered us, we were
bowling along the highway towards my uncle's house.
An hour's rapid driving soon brought us in sight of the long
avenue, the lighted windows, and quaint façade of the old mansion,
at the door of which I drew up; and as I threw the whip and reins to
Willie Pitblado, and, fearless now even of Mamma Chillingham,
handed my companion down, tenderly and caressingly, I found
myself an engaged man, and the fiancé of one of the fairest women
in Britain—the brilliant Louisa Loftus!
CHAPTER XII.
It passed—and never marble looked more pale
Than Lucy, while she listened to his tale.
He marked her not; his eye was cold and clear,
Fixed on a bed of withering roses there;
He marked her not, for different thoughts possessed
His anxious mind, and laboured in his breast.
ELLIS.
Notwithstanding all that had passed, and that we had been carried
so far in the wrong direction, we were not long behind the rest of
our party in reaching Calderwood, where the history of our disaster
fully eclipsed for the evening all the exciting details of the fox-
hunters, though many gentlemen in scarlet, with spattered tops and
tights, whom Sir Nigel had brought, made the drawing-room look
unusually gay.
Lady Louisa remained long in her own apartment; the time
seemed an age to me; yet I was happy—supremely happy. I had a
vague idea of the new emotions that served, perhaps, to detain her
there; but an air of cold reserve and unmistakable displeasure
hovered on the forehead of her haughty mother.
When Louisa joined us, she had perfectly recovered her usual
equanimity and presence of mind—her calm, pale, and placid aspect.
She was somewhat silent and reserved; this passed for her natural
terror of the late accident, and though we remained some distance
apart, her fine dark eyes sought mine, ever and anon, and were full
of intelligent glances, that made my heart leap with joy.
Cora, who shrewdly suspected that there had been more in the
affair than what Berkeley called "a doocid spill," regarded us with
interest, and with a tearful earnestness that surprised us, after our
return, and during the explanation which we were pleased to make.
But whatever tales my face told, Louisa's was unfathomable, so from
its expression suspicious little Cora could gather nothing; though,
had she carried her scrutiny a little further, she might have detected
my famous Rangoon diamond sparkling on the engaged finger of her
friend's left hand.
Cora was on this night, to me, an enigma!
What had gone wrong with her? When she smiled, it seemed to
several—to me especially—that the kind little heart from whence
these smiles were wrung was sick. Why was this, and what or who
was the source of her taciturnity and secret sorrow?—not Berkeley,
surely—they had come home in the drag together—she could never
love such an ass as Berkeley; and if the fellow dared to trifle with
her—but I thrust the thought aside, and resolved to trust the affair
to her friend and gossip, the Lady Loftus.
A few more days glided swiftly and joyously past at Calderwood
Glen; we had no more riding and driving; but, as the weather was
singularly open and balmy for the season, we actually had more
than one picnic in the leafless woods, and I betook me to the study
of botany and arboriculture with the ladies.
I enjoyed all the delicious charm of a successful first love! The
last thought on going to repose; the first on waking in the morning;
and the source of many a soft and happy dream between.
The peculiarity, or partial disparity, of our positions in life caused
secrecy. Denied, by the presence of others, the pleasure of openly
conversing of our love, at times we had recourse to furtive glances,
or a secret and thrilling pressure of the hand or arm was all we
could achieve.
Then there were sighs the deeper for suppression,
And stolen glances sweeter for the theft;
And burning blushes, though for no transgression,
Tremblings when met, and restlessness when
left.
Small and trivial though these may seem, they proved the sum of
our existence, and even of mighty interest, lighting up the eye and
causing the pulses of the heart to quicken.
We became full of petty and lover-like stratagems, and of
enigmatical phrases, all the result of the difficulties that surrounded
our intercourse when others were present—especially Lady
Chillingham, who was by nature cold, haughty, and suspicious, with,
I think, a natural born antipathy to subalterns of cavalry in particular.
Cora saw through our little artifices, and Berkeley, that Anglo-Scotch
snob of the nineteenth century, had ever his eyes remarkably wide
open to all that was going on around him, and thus the perils of
discovery and instant separation were great, while our happy love
was in the flush.
This danger gave us a common sympathy, a united object, a
delicious union of thought and impulse. Nor was romance wanting to
add zest to the secrecy of our passion. Ah, were I to live a thousand
years, never should I forget the days of happiness I spent in
Calderwood Glen with Louisa Loftus.
Our interviews had all the mystery of a conspiracy, though, save
Cora, none as yet suspected our love; and there was a part of the
garden, between two old yew hedges—so old that they had seen the
Calderwoods of past ages cooing and billing, in powdered wigs and
coats of mail, with dames in Scottish farthingales and red-heeled
shoes—where, at certain hours, by a tacit understanding, we were
sure of meeting; but with all the appearance of chance, though
occasionally for a time so brief, that we could but exchange a
pressure of the hand, or snatch a caress, perhaps a kiss, and then
separate in opposite directions.
Those were blessed and joyous interviews; memories to
treasure and brood over with delight when alone. In the society of
our friends, my heart throbbed wildly, when by a glance, a smile, a
stolen touch of the hand, Louisa reminded me of what none else
could perceive, the secret understanding that existed between us.
And yet all this happiness was clouded by a sense of its brevity,
and by our fears for the future; the obstacles that rank and great
fortune on her side, the lack of both on mine, raised between us;
and then there was the certain prospect of a long and dangerous—
alas! it might prove, a final separation.
"They who love," writes an anonymous author, "must ever drink
deeply of the cup of trembling; but, at times, there will arise in their
hearts a nameless terror, a sickening anxiety for the future, whose
brightness all depends upon this one cherished treasure, which often
proves a foreboding of some real anguish looming in the distant
hours."
"Where is all this to end?" I asked of myself, as the conviction
that something must be done forced itself upon me, for the happy
days were passing, and my short leave of absence was drawing to a
close.
One day, by the absence of some of our friends, and by the
occupation of others, we found ourselves alone, and permitted to
have a longer interview than usual, in our yew-hedge walk, and we
were conversing of the future.
"I have two hundred a year besides my pay, Louisa." (She
smiled sadly at this, and the smile went doubly to my heart.) "The
money has been lodged for my troop with Cox and Co., and my good
uncle means well concerning me; yet, I feel all these as being so
small, that were I to address the Earl of Chillingham on the subject
of our engagement, it would seem that I had little to offer, and little
to urge, save that which is, perhaps, valueless in his aristocratic eyes
——"
"And that is?"
"My love for you."
"Don't think of addressing him," said she, weeping on my
shoulder; "he has already views for me in another quarter."
"Views, Louisa!"
"Yes; pardon me for paining you, dearest, by saying so; but it is
nevertheless true."
"And these views?" I asked, impetuously.
"Are an offer made for my hand by Lord Slubber de Gullion."
My heart died within me on hearing this name, which, as I once
before stated, comes as near the original as possible.
"Hence you see, dearest Newton," she resumed, in a mournful
and sweetly-modulated voice, "were you to address my father, it
would only rouse mamma, and have the effect of interrupting our
correspondence for ever."
"Good heavens! what then are we to do?'
"Wait in hope."
"How long?"
"Alas! I know not; but for the present at least our engagement,
like our meetings and our letters, if we can correspond, must be
secret—secret all. Were the earl, my father, to know that I loved
you, Newton (how sweetly those words sounded), he and mamma
would urge on Lord Slubber's suit, and, on finding that I refused,
there would be no bounds to mamma's wrath. You remember Cora's
story of the 'Clenched Hand;' you remember the 'Bride of
Lammermoor,' and must see what a determined mother and long
domestic tyranny may do."
I clasped my hands, for my heart was wrung; but she regarded
me kindly and lovingly.
"On your return home, as colonel of your regiment, perhaps, we
shall then, at all hazards, bring the matter before him, and treat
Slubber's offer with contempt, as the senile folly of an old man in his
dotage. You, at least, shall propose for me in form——"
"And if Lord Chillingham refuse?"
"Though we English people can't make Scotch marriages now, I
shall be yours, dearest Newton, as I am now, only that it shall be
irrevocably and for ever."
A close and mute embrace followed, and then I left her in a
paroxysm of grief, while my head whirled with the combined effects
of love and joy, and of sorrow, not unmixed with anger.
"I wonder what the subjects are that lovers talk of in their tête-
à-têtes," says my brother of the pen and sword, W. H. Maxwell, and
the same surmise frequently occurred to myself, before I met or
knew Louisa Loftus.
We never lacked a subject now. The peculiarities of our relative
positions, our caution for the present, and our natural anxieties for
the future, afforded us full topics for conversation or surmise; but
the few remaining days of my leave "between returns" glided away
at Calderwood Glen; the time for my departure drew nigh; already
had Pitblado divided a sixpence with my lady's soubrette, and
packed up all my superfluous traps, and within six and thirty hours
Berkeley and I would have to report ourselves in uniform at head-
quarters, or be returned absent without leave.
It was in the evening, when I had gone as usual to meet Louisa
at the seat where the close-clipped yew hedges formed a pleasant
screen, that, to my surprise, and by the merest chance, I found it
occupied by my cousin Cora.
The January sunset was beautiful; the purple flush of evening
covered all the western sky, and bathed in warm tints the slopes of
the Lomond hills. The air was still, and we heard only the cawing of
the venerable rooks that perched among the woods of the old
manor, or swung to and fro on its many gilt vanes.
Cora was somewhat silent, and I, being thoroughly disappointed
by finding her there in lieu of Louisa Loftus, was somewhat taciturn,
if not almost sulky.
Somehow—but how, I know not—Cora led me to talk insensibly
of our early days, and as we did so, I could perceive that she
regarded me earnestly from time to time, after I simply remarked
that ere long I should be far, far away from her, and among other
scenes. Her dovelike, dark eye became suffused, and the tinge on
her rounded cheek died away when I laughingly referred to the days
when we had been little lovers, and when Fred Wilford and I—he
was now a captain of ours—used to punch each other's heads in
pure spite and jealousy about her; but this youthful jealousy once
took a more dangerous turn.
Among the rocks in the glen an adder of vast size took up its
residence, and had bitten several persons. It had been seen by some
to leap more than seven yards high, and was a source of such terror
to the whole parish, that my uncle, and even the provost of
Dunfermline, had offered rewards for its destruction.
On this I boldly dared my boy-rival to face it; but Fred Wilford,
who was on a visit to us from Rugby, had more prudence, or less
love for little Cora, and so declined the attempt.
Flushed with boyish pride and recklessness, I climbed the steep
face of the rock, stirred up the adder with a long stick, flung it to the
ground, and killed it by repeated blows of an axe, a feat of which my
uncle never grew tired of telling, and the reptile was now in the
library, sealed up in a glass case, being deemed a family trophy, and,
as Binns said, always kept in the best of spirits.
I sat with Cora's white and slender hand in mine, gazing at her
soft and piquant features, her pouting lips and dimpled chin, and the
dark hair so smoothly braided under her little hat, and over each
pretty and delicate ear. Cora was very gentle and very charming; she
had ever been to me a kind little playmate, a loving sister, and she
sighed deeply when I spoke of my approaching departure.
"You go by sea?" she asked.
"If we go to Turkey—of course."
"Embarking at Southampton?"
"Embarking at Southampton—exactly, and sailing directly for the
East, I suppose," said I, while leisurely lighting a cigar; "I shall soon
learn all the details and probabilities at head-quarters; but the route
may not come for two months yet, as red-tape goes."
"You will think of us sometimes, Newton, in those strange and
dangerous lands? Of your poor uncle, who loves you so well, and—
and of me?"
"Of course, and of Louisa Loftus. Don't you think her very
handsome?"
"I think her lovely."
"My cigar annoys you?"
"Not at all, Newton."
"But it makes you turn your face away."
"You met often, I believe, before you came here?"
"Oh, very often. I used to see her at the cathedral every Sunday
in Canterbury; at the balls at Rochester and Maidstone——"
"And in London?"
"Repeatedly! I saw her at her first presentation at Court, when
the colonel presented me, on obtaining my lieutenancy, and
returning from foreign service. She created quite a sensation!"
I spoke in such glowing terms of my admiration for Louisa
Loftus, that some time elapsed before I detected the extreme pallor
of Cora's cheek, and a peculiar quivering of her under lip.
"Good heavens, my dear girl, you are ill! It is this confounded
cigar—one of a box that Willie got me in Dunfermline," I exclaimed,
throwing it away. "Your hand is trembling, too."
"Is it? Oh, no! Stay! I am only a little faint," she murmured.
"Faint! Why the deuce should you be faint, Cora?"
"This bower of yew hedges is close; the atmosphere is still, or
chill, or something," she said, in a low voice, while pressing a lovely
little hand on her bosom; "and it seems to me that I felt a pang
here."
"A pang, Cora?"
"Yes, I feel it sometimes."
"You, one of the best waltzers in the county! You have no
affection of the heart, or any of that sort of thing?"
She smiled sadly, even bitterly, and rose, saying—
"Here comes Lady Louisa. Say nothing of this."
Her dark eyes were swimming; but not a tear fell from their
long, black, silky lashes, that lent such softness to her sweet and
feminine face. She abruptly withdrew her tremulous hands from
mine, and just as Louisa approached, hurriedly left me.
What did all this emotion mean? What did it display or conceal?
I was thoroughly bewildered.
A sudden light began to break upon me.
"What is this?" thought I. "Can Cora be in love with me herself?
Oh, nonsense! she has known me from boyhood. The idea is absurd!
Yet her manner——. This will never do. I must avoid her, and to-
morrow I leave for England!"
Louisa sat beside me, and, save her, Cora and all the world were
alike forgotten.
CHAPTER XIII.
Forget thee? If to dream by night, and muse on thee
by day;
If all the worship, deep and wild, a poet's heart can
pay;
If prayers in absence, breathed for thee to Heaven's
protecting power;
If winged thoughts that flit to thee, a thousand in an
hour;
If busy Fancy, blending thee with all my future lot;
If this thou call'st forgetting, thou, indeed, shall be
forgot.
MOULTRIE.
I had but one, only one, meeting more with Lady Louisa, and it was,
indeed, a sad one. We could but hope to meet again—near
Canterbury, perhaps—at some vague period before my regiment
marched; and prior to that I was to write to her, on some polite
pretence, under cover to Cora.
This was certainly somewhat undefined and unsatisfactory for
two engaged lovers, especially for two so ardent as we were, and in
the first flush of a grand passion; but we had no other arrangement
to make; and never shall I forget our last, long, mute embrace on
the last evening, when, scared by footsteps on the garden walk, we
literally tore ourselves away, and separated to meet at the dinner-
table, and act as those who were almost strangers to each other,
and to perform the mere formalities, the politenesses, and cold
ceremonies of well-bred life.
I could not help telling my good uncle of my success; but under
a solemn promise of secrecy, for a time at least.
"All right, boy," said he, clapping me on the shoulder. "Keep her
well in hand, and I'll back you against the field to any amount that is
possible; but that gouty old peer, my Lord Slubber, is richer than I
am; and then Lady Chillingham has the pride of Lucifer. Draw on me
whenever you want money, Newton. Since Archie died at college,
and poor Nigel at the battle of Goojerat, I have no boy to look after
but you."
The last hour came inexorably. We shook hands with all. When
that solemn snob, my brother officer, Mr. de Warr Berkeley, and I
entered the carriage which was to take us to the nearest railway
station, there were symptoms of considerable emotion in the faces
of the kind circle we were leaving, for the clouds of war had
darkened fast in the East during the month we had spent so
pleasantly; and the ladies—the poor girls especially—half viewed us
as foredoomed men.
Louisa was as pale as death; she trembled with suppressed
emotion, and her eyes were full of tears. Even her cold and stately
mother kissed me lightly on the cheek; and at that moment, for
Louisa's sake, I felt my heart swell with sudden emotion of regard
for her.
My uncle's hard but manly, hand gave mine a hearty pressure,
and he kindly shook the hand of Willie Pitblado, who was bidding
adieu to his father, the old keeper, and slipped a couple of
sovereigns into it.
Sir Nigel's voice was quite broken; but there was no tear in the
hot, dry eyes of poor Cora. Her charming face was very pale, and
she bit her pouting nether lip, to conceal, or to prevent, its nervous
quivering.
"An odd girl," thought I, as I kissed her twice, whispering, "Give
the last one to Louisa."
But, ah! how little could I read the secret of the dear little heart
of Cora, which was beating wildly and convulsively beneath that
apparently calm and unmoved exterior! But a time came when I was
to learn it all.
"Good-bye to Calderwood Glen," cried I, leaping into the
carriage. "A good-bye to all, and hey for pipeclay again!"
"Pipeclay and gunpowder too, lad," said my uncle. "Every ten
years or so the atmosphere of Europe requires to be fumigated with
it somewhere. Adieu, Mr. Berkeley. God bless you, Newton!"
"Crack went the whip, round went the wheels;" the group of
pale and tearful faces, the ivy-clad porch, and the turreted façade of
the old house vanished, and then the trees of the avenue appeared
to be careering past the carriage windows in the twilight, as we sped
along at a rapid trot.
For mental worry or depression there is no more certain and
rapid cure than quick travelling and transition from place to place;
and assuredly that luxury is fully afforded by the locomotive
appliances of the present age.
Within an hour after leaving Calderwood, we occupied a first-
class carriage, and were flying by the night express, en route for
London, muffled to the eyes in warm railway-rugs and border plaids,
and each puffing a cigar in silence, gazing listlessly out of the
windows, or doing his best to court sleep, to wile the dreary hours
away.
Pitblado was fraternising with the guard in the luggage-van,
doubtless enjoying a quiet "weed" the while.
Berkeley soon slept; but I prayed for the celebrated "forty
winks" in vain; and thus, wakeful and full of exciting thoughts, I
pictured in reverie all that had occurred during the past month.
Gradually the unwilling, but startling, conviction forced itself
upon my mind that my cousin. Cora loved me! This dear and
affectionate girl, from whom I had parted with such a frigid salute as
that which Sir Charles Grandison gave Miss Byron at the end of their
dreary seven years' courtship, loved me; and yet, blinded by my
absorbing passion for the brilliant Louisa Loftus, I had neither
known, seen, or felt it.
Her frequent coldnesses to me, and her ill-concealed irritation at
the cool insolence of Berkeley's languid bearing, on more than one
occasion, were all explained to me now.
Dear, affectionate, and single-hearted Cora! A hundred
instances of her self-denial now crowded on my memory. I
remembered now, at the meet of the Fifeshire fox-hounds at Largo,
that it was she who, by a little delicate tact and foresight, contrived
to give me that which she knew I so greatly coveted—the drive
home in the tandem with Lady Louisa.
What must that act of self-sacrifice have cost her heart, if
indeed she loved me? I could not write to her on such a subject, or
even approach an idea that might, after all, be based on supposition,
if not on vanity. More than this—I felt that the suspicion of having
excited this secret passion must preclude my writing to Louisa under
cover to Cora. Common delicacy and kindness suggested that I
should not, by doing so, further lacerate a good little heart that
loved me well.
But the next thought was how to communicate with Louisa,
Cora being our only medium. Nor could I forget that when I was up
the Rangoon river, and when my dear mother died at Calderwood,
that it was Cora's kiss that was last upon her cold forehead, and
Cora's little hand that closed her eyes for me.
Swiftly sped the express train while these thoughts passed
through my mind, and agitated me greatly. To sleep was impossible,
and ere midnight I heard the bells of Berwick-upon-Tweed announce
that we had left the stout old kingdom of Scotland far behind us,
and were flying at the rate of fifty miles an hour by Bedford,
Alnwick, and Morpeth, towards the Tyne, and the land of coal and
fire.
Every instant bore me farther from Louisa; and I had but one
comfort, that ere long she would be pursuing the same route—
perhaps seated in the same carriage—as she sped to her home in
the south of England.
I dearly loved this proud and beautiful girl; and if human
language has a meaning, and if the human eye has an expression,
she loved me truly in return; but though the conviction of this made
my heart brim with happiness, it was a happiness not untinged with
fears—fears that her love was, perhaps, the fancy of the hour,
developed by propinquity and the social circle of a quiet country
house; fears that my joy and success were too bright to last; and
that, after a time, she might see her engagement with a nameless
subaltern of cavalry in the light of a mésalliance, and be dazzled by
some more brilliant offer, for the heiress and only child of the Earl of
Chillingham could command many.
War and separation were before us; and if I survived to return,
would she love me still, and still indeed be mine?
Her father's consent was yet to be obtained. In my impatience
to know the best or the worst, I frequently resolved to break the
matter by letter to his lordship; but, remembering the tears and
entreaties of Louisa, I shrank from the grave responsibility of
tampering with our mutual happiness.
At other times I thought of confiding the management of the
affair entirely to my uncle; but abandoned the idea almost as soon
as I conceived it: knowing that the fox-hunting old baronet was
more hot-headed, proud, and abrupt than politic. In conclusion, I
thought it might be better done by a letter from the East, when the
earl might politely half entertain an engagement which a bullet
might dissolve; or, should I leave the affair over till I returned?
Oh! might I ever return—and if so, how mutilated? And if I died
before the enemy, in imagination I saw, in the long, long years that
were to follow, myself perhaps forgotten, and Louisa, my affianced
bride, the wife of—another.
CHAPTER XIV.
And why not death, rather than live in torment?
To die is to be banished from myself;
And Sylvia is myself: banished from her
Is self from self; a deadly banishment!
What light is light, if Sylvia be not seen?
What joy is joy, if Sylvia be not by?
Unless it be to think that she is by,
And feed upon the shadow of perfection.
SHAKSPEARE.
While yet half-slept, and wholly unrefreshed, after our long and rapid
journey by train, we donned our uniforms, with sword-belt and
sabretashe, duly reported ourselves to the colonel, who welcomed
us back, and within an hour I found myself established in my old
quarters, and once more falling into the every-day routine of
barrack-life, just as if I had never left Maidstone, and as if my visit to
Calderwood and my engagement with Louisa were all a dream. But I
Welcome to Our Bookstore - The Ultimate Destination for Book Lovers
Are you passionate about books and eager to explore new worlds of
knowledge? At our website, we offer a vast collection of books that
cater to every interest and age group. From classic literature to
specialized publications, self-help books, and children’s stories, we
have it all! Each book is a gateway to new adventures, helping you
expand your knowledge and nourish your soul
Experience Convenient and Enjoyable Book Shopping Our website is more
than just an online bookstore—it’s a bridge connecting readers to the
timeless values of culture and wisdom. With a sleek and user-friendly
interface and a smart search system, you can find your favorite books
quickly and easily. Enjoy special promotions, fast home delivery, and
a seamless shopping experience that saves you time and enhances your
love for reading.
Let us accompany you on the journey of exploring knowledge and
personal growth!
ebookball.com
