Combinatorial Search From Algorithms To Systems

Youssef Hamadi download

https://ebookbell.com/product/combinatorial-search-from-
algorithms-to-systems-youssef-hamadi-36521020

Explore and download more ebooks at ebookbell.com

Here are some recommended products that we believe you will be
interested in. You can click the link to download.

Local Search In Combinatorial Optimization Emile Aarts Editor Jan

Karel Lenstra Editor

https://ebookbell.com/product/local-search-in-combinatorial-
optimization-emile-aarts-editor-jan-karel-lenstra-editor-50731276

Combinatorial Algorithms Generation Enumeration And Search 1st Donald

L Kreher

https://ebookbell.com/product/combinatorial-algorithms-generation-
enumeration-and-search-1st-donald-l-kreher-20668760

Information Theory Combinatorics And Search Theory In Memory Of Rudolf

Ahlswede 1st Edition Christian Heup Auth

https://ebookbell.com/product/information-theory-combinatorics-and-
search-theory-in-memory-of-rudolf-ahlswede-1st-edition-christian-heup-
auth-4242050

Combinatorial Game Theory Richard J Nowakowski Bruce M Landman

https://ebookbell.com/product/combinatorial-game-theory-richard-j-
nowakowski-bruce-m-landman-45150316
Combinatorial Optimization 7th International Symposium Isco 2022
Virtual Event May 1820 2022 Revised Selected Papers Ivana Ljubi

https://ebookbell.com/product/combinatorial-optimization-7th-
international-symposium-isco-2022-virtual-event-may-1820-2022-revised-
selected-papers-ivana-ljubi-48671898

Combinatorial Mathematics Douglas B West

https://ebookbell.com/product/combinatorial-mathematics-douglas-b-
west-48866824

Combinatorial Reciprocity Theorems An Invitation To Enumerative

Geometric Combinatorics Graduate Studies In Mathematics Matthias Beck

https://ebookbell.com/product/combinatorial-reciprocity-theorems-an-
invitation-to-enumerative-geometric-combinatorics-graduate-studies-in-
mathematics-matthias-beck-48950642

Combinatorial Commutative Algebra 2nd Edition Ezra Miller Bernd

Sturmfels

https://ebookbell.com/product/combinatorial-commutative-algebra-2nd-
edition-ezra-miller-bernd-sturmfels-49055580

Combinatorial Optimization Under Uncertainty Reallife Scenarios In

Allocation Problems Ritu Arora

https://ebookbell.com/product/combinatorial-optimization-under-
uncertainty-reallife-scenarios-in-allocation-problems-ritu-
arora-50418864
Youssef Hamadi

Combinatorial
Search: From
Algorithms to
Systems
Combinatorial Search: From Algorithms to Systems
Youssef Hamadi

Combinatorial Search:
From Algorithms to Systems
Youssef Hamadi
Microsoft Research Cambridge
Cambridge, UK

ISBN 978-3-642-41481-7 ISBN 978-3-642-41482-4 (eBook)

DOI 10.1007/978-3-642-41482-4
Springer Heidelberg New York Dordrecht London

Library of Congress Control Number: 2013953692

© Springer-Verlag Berlin Heidelberg 2013

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology
now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’s location, in its current version, and permission for use must always be obtained from Springer.
Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations
are liable to prosecution under the respective Copyright Law.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
While the advice and information in this book are believed to be true and accurate at the date of pub-
lication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any
errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect
to the material contained herein.

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)

Foreword

To solve a problem as efficiently as possible, a user selects a type of solver (MIP, CP,
SAT), then defines a model and selects a method of resolution. The model expresses
the problem in a way understandable for the solver. The method of resolution can be
complete (one is certain not to miss solutions) or incomplete (it uses a heuristic, i.e.,
a method that favors the chances of finding a solution but offers no completeness
guarantee).
Since solvers exist, researchers try to simplify the task of the end user, helping
her in these keys steps: the creation of the model, and the finding of a method of
resolution. In this book, Youssef Hamadi helps the user on the second point by
presenting ways to automatically select and adjust resolution strategies.
This book proposes several methods for both SAT and CP solvers. Firstly, the au-
thor demonstrates the benefit of parallelism through the duplication of search strate-
gies. In the best case, this can provide super linear speed up in the resolution process.
In most cases, this results in a more robust resolution method, to the point that such
a solver is never beaten by a solver using the best method. The solver ManySAT,
co-developed by Mr. Hamadi, is based on this idea and has won numerous prizes in
SAT competitions. Its fame goes far beyond the SAT solving domain and this line
of work is now a reference for the domain.
Any resolution method must be guided by the user through the definition of a
resolution strategy which typically defines the next decision to be made, i.e., which
variable must be assigned to which value? This book considers the automatic learn-
ing of the parameters of resolution strategies. It shows how to extract knowledge
from the information available during search. The difficulty is to determine the rel-
evant information and decide how they can be exploited. A particularly novel ap-
proach is proposed. It considers the successive resolutions of similar problems to
gradually build an efficient strategy.
This is followed by the presentation of Autonomous Search, a major contribution
of the book. In that formalism, the solver determines itself the best way to find solu-
tions. This is a very important topic, which has often been approached too quickly,
and which is finally well defined in this book. Many researchers should benefit from
this contribution.

v
vi Foreword

This book is fun to follow and the reader can understand the continuity of the
proposed approaches. Youssef Hamadi is able to convey his passion and conviction.
It is a pleasure to follow him on his quest for a fully automated resolution procedure.
The problem is gradually understood and better resolved through the book.
The quality, diversity and originality of the proposed methods should satisfy
many readers and this book will certainly become a reference in the field. I highly
recommend its reading.
Nice, France Jean-Charles Régin
September 2013
Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
2 Boosting Distributed Constraint Networks . . . . . . . . . . . . . . 5
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Previous Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 Technical Background . . . . . . . . . . . . . . . . . . . . . . . 8
2.3.1 Distributed Constraint Satisfaction Problems . . . . . . . 8
2.3.2 DisCSP Algorithms . . . . . . . . . . . . . . . . . . . . 9
2.3.3 Performance of DisCSP Algorithms . . . . . . . . . . . . 10
2.4 Risks in Search . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.4.1 Randomization Risk . . . . . . . . . . . . . . . . . . . . 11
2.4.2 Selection Risk . . . . . . . . . . . . . . . . . . . . . . . 13
2.5 Boosting Distributed Constraint Satisfaction . . . . . . . . . . . 14
2.5.1 Utilizing Competition with Portfolios . . . . . . . . . . . 15
2.5.2 Utilizing Cooperation with Aggregation . . . . . . . . . 16
2.5.3 Categories of Knowledge . . . . . . . . . . . . . . . . . 16
2.5.4 Interpretation of Knowledge . . . . . . . . . . . . . . . . 17
2.5.5 Implementation of the Knowledge Sharing Policies . . . 17
2.5.6 Complexity . . . . . . . . . . . . . . . . . . . . . . . . 18
2.6 Empirical Evaluation . . . . . . . . . . . . . . . . . . . . . . . 20
2.6.1 Basic Performance . . . . . . . . . . . . . . . . . . . . . 20
2.6.2 Randomization Risk . . . . . . . . . . . . . . . . . . . . 21
2.6.3 Selection Risk . . . . . . . . . . . . . . . . . . . . . . . 22
2.6.4 Performance with Aggregation . . . . . . . . . . . . . . 23
2.6.5 Scalability . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.6.6 Idle Time . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3 Parallel Tree Search for Satisfiability . . . . . . . . . . . . . . . . . 27
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.2 Previous Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

vii
viii Contents

3.3 Technical Background . . . . . . . . . . . . . . . . . . . . . . . 29

3.3.1 DPLL Search . . . . . . . . . . . . . . . . . . . . . . . 30
3.3.2 Modern SAT Solvers . . . . . . . . . . . . . . . . . . . . 30
3.3.3 Multicore Architectures . . . . . . . . . . . . . . . . . . 31
3.3.4 AIMD Feedback Control-Based Algorithm . . . . . . . . 31
3.4 ManySAT: A Parallel SAT Solver . . . . . . . . . . . . . . . . . 32
3.4.1 Restart Policies . . . . . . . . . . . . . . . . . . . . . . 32
3.4.2 Heuristic . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.4.3 Polarity . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.4.4 Learning . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.4.5 Clause Sharing . . . . . . . . . . . . . . . . . . . . . . . 37
3.4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.5 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.5.1 Performance Against a Sequential Algorithm . . . . . . . 39
3.5.2 Performance Against Other Parallel SAT Solvers . . . . . 41
3.6 Control-Based Clause Sharing . . . . . . . . . . . . . . . . . . . 42
3.6.1 Throughput and Quality-Based Control Policies . . . . . 43
3.6.2 Experiments . . . . . . . . . . . . . . . . . . . . . . . . 45
3.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4 Parallel Local Search for Satisfiability . . . . . . . . . . . . . . . . 49
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.2 Previous Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
4.2.1 Incomplete Methods for Parallel SAT . . . . . . . . . . . 50
4.2.2 Cooperative Sequential Algorithms . . . . . . . . . . . . 50
4.3 Technical Background . . . . . . . . . . . . . . . . . . . . . . . 51
4.3.1 Local Search for SAT . . . . . . . . . . . . . . . . . . . 51
4.3.2 Refinements . . . . . . . . . . . . . . . . . . . . . . . . 52
4.4 Knowledge Sharing in Parallel Local Search for SAT . . . . . . . 55
4.4.1 Using Best Known Configurations . . . . . . . . . . . . 55
4.4.2 Weighting Best Known Configurations . . . . . . . . . . 55
4.4.3 Restart Policy . . . . . . . . . . . . . . . . . . . . . . . 56
4.5 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.5.1 Experimental Settings . . . . . . . . . . . . . . . . . . . 57
4.5.2 Practical Performances with Four Cores . . . . . . . . . 58
4.5.3 Practical Performances with Eight Cores . . . . . . . . . 62
4.5.4 Analysis of the Diversification/Intensification Trade-off . 63
4.5.5 Analysis of the Limitations of the Hardware . . . . . . . 68
4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
5 Learning Variable Dependencies . . . . . . . . . . . . . . . . . . . 71
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
5.2 Previous Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
5.3 Technical Background . . . . . . . . . . . . . . . . . . . . . . . 72
5.4 Exploiting Weak Dependencies in Tree-Based Search . . . . . . 74
5.4.1 Weak Dependencies . . . . . . . . . . . . . . . . . . . . 74
Contents ix

5.4.2 Example . . . . . . . . . . . . . . . . . . . . . . . . . . 74
5.4.3 Computing Weak Dependencies . . . . . . . . . . . . . . 75
5.4.4 The domFD Dynamic Variable Ordering . . . . . . . . . 75
5.4.5 Complexities of domFD . . . . . . . . . . . . . . . . . . 76
5.5 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
5.5.1 The Problems . . . . . . . . . . . . . . . . . . . . . . . 78
5.5.2 Searching for All Solutions or for an Optimal Solution . . 78
5.5.3 Searching for a Solution with a Classical
Branch-and-Prune Strategy . . . . . . . . . . . . . . . . 79
5.5.4 Searching for a Solution with a Restart-Based
Branch-and-Prune Strategy . . . . . . . . . . . . . . . . 80
5.5.5 Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . 81
5.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
6 Continuous Search . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
6.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
6.3 Technical Background . . . . . . . . . . . . . . . . . . . . . . . 85
6.3.1 Constraint Satisfaction Problems . . . . . . . . . . . . . 85
6.3.2 Supervised Machine Learning . . . . . . . . . . . . . . . 86
6.4 Continuous Search in Constraint Programming . . . . . . . . . . 87
6.5 Dynamic Continuous Search . . . . . . . . . . . . . . . . . . . . 88
6.5.1 Representing Instances: Feature Definition . . . . . . . . 88
6.5.2 Feature Pre-processing . . . . . . . . . . . . . . . . . . . 90
6.5.3 Learning and Using the Heuristics Model . . . . . . . . . 90
6.5.4 Generating Examples in Exploration Mode . . . . . . . . 91
6.5.5 Imbalanced Examples . . . . . . . . . . . . . . . . . . . 91
6.6 Experimental Validation . . . . . . . . . . . . . . . . . . . . . . 92
6.6.1 Experimental Settings . . . . . . . . . . . . . . . . . . . 92
6.6.2 Practical Performances . . . . . . . . . . . . . . . . . . 93
6.6.3 The Power of Adaptation . . . . . . . . . . . . . . . . . 96
6.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
7 Autonomous Search . . . . . . . . . . . . . . . . . . . . . . . . . . 99
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
7.2 Solver Architecture . . . . . . . . . . . . . . . . . . . . . . . . 101
7.2.1 Problem Modeling/Encoding . . . . . . . . . . . . . . . 102
7.2.2 The Evaluation Function . . . . . . . . . . . . . . . . . . 103
7.2.3 The Solving Algorithm . . . . . . . . . . . . . . . . . . 103
7.2.4 Configuration of the Solver: The Parameters . . . . . . . 104
7.2.5 Control . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
7.2.6 Existing Classifications and Taxonomies . . . . . . . . . 105
7.3 Architecture of Autonomous Solvers . . . . . . . . . . . . . . . 107
7.3.1 Control by Self-adaptation . . . . . . . . . . . . . . . . . 107
7.3.2 Control by Supervised Adaptation . . . . . . . . . . . . . 108
7.3.3 Searching for a Solution vs. Solutions for Searching . . . 108
x Contents

7.3.4 A Rule-Based Characterization of Solvers . . . . . . . . 109

7.4 Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
7.4.1 Tuning Before Solving . . . . . . . . . . . . . . . . . . . 115
7.4.2 Control During Solving . . . . . . . . . . . . . . . . . . 117
7.4.3 Control During Solving in Parallel Search . . . . . . . . 121
7.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
8 Conclusion and Perspectives . . . . . . . . . . . . . . . . . . . . . . 123
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
List of Figures

Fig. 2.1 Heavy-tail behavior of IDIBT and ABT . . . . . . . . . . . . . . 12

Fig. 2.2 DisCSP (left) and agent topologies implied by the variable
orderings max-degree (middle) and min-degree (right) . . . . . . 14
Fig. 2.3 Two contexts for the agent hosting X4 from Fig. 2.2 resulting
from two variable orderings . . . . . . . . . . . . . . . . . . . . 15
Fig. 2.4 Communication and runtime in M-portfolios . . . . . . . . . . . 21
Fig. 2.5 Randomization risk emerging from message delays and thread
activation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Fig. 2.6 No heavy-tails with M-ABT and M-IDIBT . . . . . . . . . . . . 22
Fig. 2.7 S-risk (standard-dev of the parallel runtime) including the R-risk
emerging from distribution . . . . . . . . . . . . . . . . . . . . . 23
Fig. 3.1 Restart strategies . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Fig. 3.2 Implication graph/extended implication graph . . . . . . . . . . . 36
Fig. 3.3 SAT Race 2008: different limits for clause sharing . . . . . . . . 38
Fig. 3.4 SAT Competition 2007: different limits for clause sharing . . . . 38
Fig. 3.5 SAT Race 2008: ManySAT e = 8, m = 1 . . . 4 against Minisat 2.02 41
Fig. 3.6 Throughput-based control policy . . . . . . . . . . . . . . . . . . 44
Fig. 4.1 Performance using four cores in a given amount of time . . . . . 59
Fig. 4.2 Runtime comparison; each point indicates the runtime to solve
a given instance using 4cores-Prob (y-axis) and 4cores-No
Sharing (x-axis) . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Fig. 4.3 Best configuration cost comparison on unsolved instances. Each
point indicates the best configuration (median) cost of a given
instance using 4cores-Prob (y-axis) and 4cores-No Sharing
(x-axis) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Fig. 4.4 Performance using eight cores in a given amount of time . . . . . 62
Fig. 4.5 Pairwise average Hamming distance (x-axis) vs. Number of flips
every 106 steps (y-axis) to solve
the unif-k3-r4.2-v16000-c67200-S2082290699-014.cnf instance . 65
Fig. 4.6 Individual algorithm performance to solve
the unif-k3-r4.2-v16000-c67200-S2082290699-014.cnf instance . 67

xi
xii List of Figures

Fig. 4.7 Runtime comparison using parallel local search portfolios made
of respectively one, four, and eight identical copies of PAWS
(same random seed and no cooperation). Black diamonds indicate
the performance of four cores vs. one core. Red triangles indicate
the performance of eight cores vs. one core, points above the blue
line indicate that one core is faster . . . . . . . . . . . . . . . . . 68
Fig. 5.1 Classic propagation engine . . . . . . . . . . . . . . . . . . . . . 73
Fig. 5.2 Variables and propagators . . . . . . . . . . . . . . . . . . . . . 75
Fig. 5.3 Schedule(Queue Q, Propagator p, Variable Xi ) . . . . . . . . . . 75
Fig. 6.1 Continuous search scenario . . . . . . . . . . . . . . . . . . . . 87
Fig. 6.2 dyn-CS: selecting the best heuristic at each restart point . . . . . 88
Fig. 6.3 Langford number (lfn): Number of instances solved in less than 5
minutes with dyn-CS, wdeg, and dom-wdeg. Dashed lines
illustrate the performance of dyn-CS for a particular instance
ordering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
Fig. 6.4 Number of instances solved in less than 5 minutes . . . . . . . . 95
Fig. 7.1 The general architecture of a solver . . . . . . . . . . . . . . . . 102
Fig. 7.2 Control taxonomy proposed by Eiben et al. [EHM99] . . . . . . . 105
Fig. 7.3 Classification of hyper-heuristics by Burke et al. [BHK+09b] . . . 106
Fig. 7.4 The global architecture of an Autonomous Search System . . . . 108
Fig. 7.5 The solver and its action with respect to different spaces . . . . . 109
List of Tables

Table 2.1 Methods of aggregation . . . . . . . . . . . . . . . . . . . . . . 17

Table 2.2 Performance of aggregation methods for M-IDIBT . . . . . . . 23
Table 2.3 Median parallel runtime (pt) and instances solved (out of 20)
of quasigroup completion problems with 42 % pre-assigned
values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Table 2.4 Idle times of agents in DisCSP . . . . . . . . . . . . . . . . . . 25
Table 3.1 ManySAT: different strategies . . . . . . . . . . . . . . . . . . 40
Table 3.2 SAT-Race 2008: comparative performance (number of problems
solved) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Table 3.3 SAT Race 2008: parallel solvers against the best sequential
solver (Minisat 2.1) . . . . . . . . . . . . . . . . . . . . . . . . 42
Table 3.4 SAT Race 2008: runtime variation of parallel solvers . . . . . . 42
Table 3.5 SAT Race 2008: industrial problems . . . . . . . . . . . . . . . 47
Table 4.1 Overall evaluation using four cores. Each cell summarizes
the results of medium-size instances (top), large-size instances
(middle), and the total overall instances (bottom). The best
strategy for each column is highlighted in bold . . . . . . . . . 61
Table 4.2 Overall evaluation using eight cores. Each cell summarizes
the results of medium-size instances (top), large-size instances
(middle), and the total overall instances (bottom). The best
strategy for each column is highlighted in bold . . . . . . . . . 64
Table 4.3 Diversification-Intensification analysis using four cores over
the whole set of benchmarks . . . . . . . . . . . . . . . . . . . 66
Table 4.4 Diversification-Intensification analysis using eight cores over
the whole set of benchmarks . . . . . . . . . . . . . . . . . . . 66
Table 5.1 All solutions and optimal solution . . . . . . . . . . . . . . . . 79
Table 5.2 First solution, branch-and-prune strategy . . . . . . . . . . . . 79
Table 5.3 First solution, restart-based strategy . . . . . . . . . . . . . . . 81
Table 5.4 Synthesis of the experiments . . . . . . . . . . . . . . . . . . . 82
Table 6.1 Total solved instances . . . . . . . . . . . . . . . . . . . . . . . 95
Table 6.2 Predictive accuracy of the heuristics model . . . . . . . . . . . 96
Table 6.3 Total solved instances . . . . . . . . . . . . . . . . . . . . . . . 96

xiii
Chapter 1
Introduction

Combinatorial search algorithms are typically concerned with the solving of NP-
hard problems. Such problems are not believed to be solvable in general. In other
words there is no known algorithm that efficiently solves all instances of NP-hard
problems. However, tractability results from complexity theory along decades of
experimental analysis suggest that instances coming from practical application do-
mains can often be efficiently solved. Combinatorial search algorithms are devised
to efficiently explore the usually large solution space of these instances. They rely
on several techniques able to reduce the search space to feasible regions and use
heuristics to efficiently explore these regions.
Combinatorial search problems can be cast into general mathematical definitions.
This involves finding a finite set of homogeneous objects or variables whose state
must satisfy a finite set of constraints and preferences. Variables have a domain of
potential values, and constraints or preferences are used to either restrict or order
combinations of values between variables. Dedicated algorithms are able to effi-
ciently enumerate combinations or potential solutions over these definitions.
There are several mathematical formalisms used to express and tackle combina-
torial problems. This book will consider the Constraint Satisfaction Problem (CSP)
and the Propositional Satisfiability problem (SAT), two successful formalisms at the
intersection of Artificial Intelligence, Operations Research, and Propositional Cal-
culus. Despite the fact that these formalisms can express exactly the same set of
problems, as proved by complexity theory, they can be differentiated by their prac-
tical degree of expressiveness. CSP is able to exploit more general combinations
of values and more general constraints; SAT on the other hand focuses on Boolean
variables, and on one class of constraints. These degrees of expressiveness offer dif-
ferent algorithmic trade-offs. SAT can rely on more specialized and finely tuned data
structures and heuristics. On the other hand, algorithms operating on CSP modeling
have to trigger different classes of constraints and variables and therefore have to
deal with the associated overhead. These algorithms or constraint solvers, if differ-
ent, are based on the same principles. They apply search space reduction through
inference techniques, use activity-based heuristics to guide their exploration, diver-
sify their search through frequent restarts, and often learn from their mistakes.

Y. Hamadi, Combinatorial Search: From Algorithms to Systems, 1

DOI 10.1007/978-3-642-41482-4_1, © Springer-Verlag Berlin Heidelberg 2013
2 1 Introduction

This book focuses on Knowledge Sharing in combinatorial search, the capacity

to generate and exploit meaningful information during search. Meaningful informa-
tion is made of redundant constraints, heuristic hints, and performance measures. It
can be used at different levels to drastically improve the performance of a constraint
solver. Typically, information can be shared between multiple constraint solvers
simultaneously working on the same instance, or information can help solvers to
achieve good performance while solving a large set of related instances.
In the first case, multiple constraint solvers are working on the same instance,
and information sharing has to be performed at the expense of the underlying search
effort, since a solver has to stop its main effort to prepare and communicate the
information to other solvers. On the other hand, not sharing information can incur
a cost for the whole system by having solvers potentially exploring the unfeasible
spaces discovered by other solvers.
In the second case, sharing performance measures can be done with little over-
head, and the goal is to be able to acutely tune a constraint solver in relation to
the characteristics of a new instance. This corresponds to the selection of the most
suitable algorithm for solving a given instance [Ric75].
The book contains two main parts. In Chaps. 2, 3, and 4, portfolios of distributed
and parallel algorithms are presented. The reading of Chap. 2 is essential to un-
derstand the notions of selection and randomization risks in combinatorial search.
These risks explain and motivate parallel portfolio solvers. Chapters 5 and 6 present
the benefit of using learning mechanisms during or between search efforts. They can
be read independently. Finally, Chap. 7 unifies the previous chapters into the new
Autonomous Search framework.
Chapter 2 presents portfolios of distributed CSP algorithms [YDIK92] which
demonstrate that competition and cooperation through knowledge sharing can im-
prove the performance of existing distributed search techniques by several orders of
magnitude. We show that a portfolio approach makes better use of computational re-
sources by reducing the idle time of agents. It allows search agents to simultaneously
work at different tree search levels and provides a solution to the classical work im-
balance problem of distributed backtracking. This is achieved through the selective
sharing of heuristic hints and decisions. It also shows the value of knowledge shar-
ing to significantly speed up search and provide portfolios whose performance is
better than any constituent algorithm.
The previous notions are then applied to the important problem of parallel propo-
sitional satisfiability in Chap. 3. This chapter presents the knowledge sharing aspects
of ManySAT, the first parallel SAT portfolio built on lessons learned from portfo-
lios of distributed CSP algorithms. In ManySAT different modern SAT solvers are
organized around a cooperative framework to quickly solve a given instance. They
exchange redundant constraints through advanced control mechanisms which adjust
the level of cooperation in relation with the perceived relevance of the information
exchanged.
Chapter 4 considers parallel local search algorithms for the problem of propo-
sitional satisfiability. This work is motivated by the demonstrated importance of
Another Random Scribd Document
with Unrelated Content
Yosemite and Big Tree
Groves
VIA

Mariposa and Clark's or

Coulterville.

Thus a person can leave Sacramento at

noon, or San Francisco at 4 p. m. by the
C. P. R. R., remain over night at the junction
of the Rail and Stage roads, the second night
at White & Hatch's and arrive in the Valley of
the Yosemite the next evening; or those who
prefer can remain that night at Clark's and
ride leisurely into the Valley early the next
day. The latter course might be preferable to
the majority of tourists, who would desire to
visit en route the Mariposa Grove of Big
Trees, which is but five miles from Clark's.
The trail from Clark's leads through Alder
Creek, Empire Camp, Sentinel Dome, Glacier
Point, and the far-famed "Inspiration Point."
From the latter is obtained the first grand
view of this wonderful Valley, lying four
thousand feet below the "Point."

LAKE TAHOE,
Via Stage fourteen miles from TRUCKEE

DONNER LAKE,
 Three miles from either TRUCKEE or
SUMMIT.

CALAVERAS BIG TREES,

Via Stage, sixty-five miles from GALT, or
sixty-two miles from MOKELUMNE

THROUGH TICKETS:
422 CALIFORNIA
C. P. R. R.: OFFICE,
STREET.
— OAKLAND
C. P. R. R.: OFFICE,
WHARF.
C. & N. W. Ry. 445 CALIFORNIA
OFFICE, STREET.
C. B. & M. R. R. 214 MONTGOMERY
OFFICE, STREET.
C. R. I. & P. R. R. 208 MONTGOMERY
OFFICE, STREET.
K. C. St. J. &
306 MONT. ST.
C. B. R. R. OFFICE,
 *** END OF THE PROJECT GUTENBERG EBOOK BANCROFT'S
TOURIST'S GUIDE. YOSEMITE. SAN FRANCISCO AND AROUND THE
BAY, (SOUTH.) ***

Updated editions will replace the previous one—the old editions

will be renamed.

Creating the works from print editions not protected by U.S.

copyright law means that no one owns a United States
copyright in these works, so the Foundation (and you!) can copy
and distribute it in the United States without permission and
without paying copyright royalties. Special rules, set forth in the
General Terms of Use part of this license, apply to copying and
distributing Project Gutenberg™ electronic works to protect the
PROJECT GUTENBERG™ concept and trademark. Project
Gutenberg is a registered trademark, and may not be used if
you charge for an eBook, except by following the terms of the
trademark license, including paying royalties for use of the
Project Gutenberg trademark. If you do not charge anything for
copies of this eBook, complying with the trademark license is
very easy. You may use this eBook for nearly any purpose such
as creation of derivative works, reports, performances and
research. Project Gutenberg eBooks may be modified and
printed and given away—you may do practically ANYTHING in
the United States with eBooks not protected by U.S. copyright
law. Redistribution is subject to the trademark license, especially
commercial redistribution.

START: FULL LICENSE

THE FULL PROJECT GUTENBERG LICENSE
 PLEASE READ THIS BEFORE YOU DISTRIBUTE OR USE THIS WORK

To protect the Project Gutenberg™ mission of promoting the

free distribution of electronic works, by using or distributing this
work (or any other work associated in any way with the phrase
“Project Gutenberg”), you agree to comply with all the terms of
the Full Project Gutenberg™ License available with this file or
online at www.gutenberg.org/license.

Section 1. General Terms of Use and

Redistributing Project Gutenberg™
electronic works
1.A. By reading or using any part of this Project Gutenberg™
electronic work, you indicate that you have read, understand,
agree to and accept all the terms of this license and intellectual
property (trademark/copyright) agreement. If you do not agree
to abide by all the terms of this agreement, you must cease
using and return or destroy all copies of Project Gutenberg™
electronic works in your possession. If you paid a fee for
obtaining a copy of or access to a Project Gutenberg™
electronic work and you do not agree to be bound by the terms
of this agreement, you may obtain a refund from the person or
entity to whom you paid the fee as set forth in paragraph 1.E.8.

1.B. “Project Gutenberg” is a registered trademark. It may only

be used on or associated in any way with an electronic work by
people who agree to be bound by the terms of this agreement.
There are a few things that you can do with most Project
Gutenberg™ electronic works even without complying with the
full terms of this agreement. See paragraph 1.C below. There
are a lot of things you can do with Project Gutenberg™
electronic works if you follow the terms of this agreement and
help preserve free future access to Project Gutenberg™
electronic works. See paragraph 1.E below.
1.C. The Project Gutenberg Literary Archive Foundation (“the
Foundation” or PGLAF), owns a compilation copyright in the
collection of Project Gutenberg™ electronic works. Nearly all the
individual works in the collection are in the public domain in the
United States. If an individual work is unprotected by copyright
law in the United States and you are located in the United
States, we do not claim a right to prevent you from copying,
distributing, performing, displaying or creating derivative works
based on the work as long as all references to Project
Gutenberg are removed. Of course, we hope that you will
support the Project Gutenberg™ mission of promoting free
access to electronic works by freely sharing Project Gutenberg™
works in compliance with the terms of this agreement for
keeping the Project Gutenberg™ name associated with the
work. You can easily comply with the terms of this agreement
by keeping this work in the same format with its attached full
Project Gutenberg™ License when you share it without charge
with others.

1.D. The copyright laws of the place where you are located also
govern what you can do with this work. Copyright laws in most
countries are in a constant state of change. If you are outside
the United States, check the laws of your country in addition to
the terms of this agreement before downloading, copying,
displaying, performing, distributing or creating derivative works
based on this work or any other Project Gutenberg™ work. The
Foundation makes no representations concerning the copyright
status of any work in any country other than the United States.

1.E. Unless you have removed all references to Project

Gutenberg:

1.E.1. The following sentence, with active links to, or other

immediate access to, the full Project Gutenberg™ License must
appear prominently whenever any copy of a Project
Gutenberg™ work (any work on which the phrase “Project
Gutenberg” appears, or with which the phrase “Project
Gutenberg” is associated) is accessed, displayed, performed,
viewed, copied or distributed:

This eBook is for the use of anyone anywhere in the United

States and most other parts of the world at no cost and
with almost no restrictions whatsoever. You may copy it,
give it away or re-use it under the terms of the Project
Gutenberg License included with this eBook or online at
www.gutenberg.org. If you are not located in the United
States, you will have to check the laws of the country
where you are located before using this eBook.

1.E.2. If an individual Project Gutenberg™ electronic work is

derived from texts not protected by U.S. copyright law (does not
contain a notice indicating that it is posted with permission of
the copyright holder), the work can be copied and distributed to
anyone in the United States without paying any fees or charges.
If you are redistributing or providing access to a work with the
phrase “Project Gutenberg” associated with or appearing on the
work, you must comply either with the requirements of
paragraphs 1.E.1 through 1.E.7 or obtain permission for the use
of the work and the Project Gutenberg™ trademark as set forth
in paragraphs 1.E.8 or 1.E.9.

1.E.3. If an individual Project Gutenberg™ electronic work is

posted with the permission of the copyright holder, your use and
distribution must comply with both paragraphs 1.E.1 through
1.E.7 and any additional terms imposed by the copyright holder.
Additional terms will be linked to the Project Gutenberg™
License for all works posted with the permission of the copyright
holder found at the beginning of this work.

1.E.4. Do not unlink or detach or remove the full Project

Gutenberg™ License terms from this work, or any files
 containing a part of this work or any other work associated with
Project Gutenberg™.

1.E.5. Do not copy, display, perform, distribute or redistribute

this electronic work, or any part of this electronic work, without
prominently displaying the sentence set forth in paragraph 1.E.1
with active links or immediate access to the full terms of the
Project Gutenberg™ License.

1.E.6. You may convert to and distribute this work in any binary,
compressed, marked up, nonproprietary or proprietary form,
including any word processing or hypertext form. However, if
you provide access to or distribute copies of a Project
Gutenberg™ work in a format other than “Plain Vanilla ASCII” or
other format used in the official version posted on the official
Project Gutenberg™ website (www.gutenberg.org), you must,
at no additional cost, fee or expense to the user, provide a copy,
a means of exporting a copy, or a means of obtaining a copy
upon request, of the work in its original “Plain Vanilla ASCII” or
other form. Any alternate format must include the full Project
Gutenberg™ License as specified in paragraph 1.E.1.

1.E.7. Do not charge a fee for access to, viewing, displaying,

performing, copying or distributing any Project Gutenberg™
works unless you comply with paragraph 1.E.8 or 1.E.9.

1.E.8. You may charge a reasonable fee for copies of or

providing access to or distributing Project Gutenberg™
electronic works provided that:

• You pay a royalty fee of 20% of the gross profits you derive
from the use of Project Gutenberg™ works calculated using the
method you already use to calculate your applicable taxes. The
fee is owed to the owner of the Project Gutenberg™ trademark,
but he has agreed to donate royalties under this paragraph to
the Project Gutenberg Literary Archive Foundation. Royalty
 payments must be paid within 60 days following each date on
which you prepare (or are legally required to prepare) your
periodic tax returns. Royalty payments should be clearly marked
as such and sent to the Project Gutenberg Literary Archive
Foundation at the address specified in Section 4, “Information
about donations to the Project Gutenberg Literary Archive
Foundation.”

• You provide a full refund of any money paid by a user who

notifies you in writing (or by e-mail) within 30 days of receipt
that s/he does not agree to the terms of the full Project
Gutenberg™ License. You must require such a user to return or
destroy all copies of the works possessed in a physical medium
and discontinue all use of and all access to other copies of
Project Gutenberg™ works.

• You provide, in accordance with paragraph 1.F.3, a full refund of

any money paid for a work or a replacement copy, if a defect in
the electronic work is discovered and reported to you within 90
days of receipt of the work.

• You comply with all other terms of this agreement for free
distribution of Project Gutenberg™ works.

1.E.9. If you wish to charge a fee or distribute a Project

Gutenberg™ electronic work or group of works on different
terms than are set forth in this agreement, you must obtain
permission in writing from the Project Gutenberg Literary
Archive Foundation, the manager of the Project Gutenberg™
trademark. Contact the Foundation as set forth in Section 3
below.

1.F.

1.F.1. Project Gutenberg volunteers and employees expend

considerable effort to identify, do copyright research on,
transcribe and proofread works not protected by U.S. copyright
law in creating the Project Gutenberg™ collection. Despite these
efforts, Project Gutenberg™ electronic works, and the medium
on which they may be stored, may contain “Defects,” such as,
but not limited to, incomplete, inaccurate or corrupt data,
transcription errors, a copyright or other intellectual property
infringement, a defective or damaged disk or other medium, a
computer virus, or computer codes that damage or cannot be
read by your equipment.

1.F.2. LIMITED WARRANTY, DISCLAIMER OF DAMAGES - Except

for the “Right of Replacement or Refund” described in
paragraph 1.F.3, the Project Gutenberg Literary Archive
Foundation, the owner of the Project Gutenberg™ trademark,
and any other party distributing a Project Gutenberg™ electronic
work under this agreement, disclaim all liability to you for
damages, costs and expenses, including legal fees. YOU AGREE
THAT YOU HAVE NO REMEDIES FOR NEGLIGENCE, STRICT
LIABILITY, BREACH OF WARRANTY OR BREACH OF CONTRACT
EXCEPT THOSE PROVIDED IN PARAGRAPH 1.F.3. YOU AGREE
THAT THE FOUNDATION, THE TRADEMARK OWNER, AND ANY
DISTRIBUTOR UNDER THIS AGREEMENT WILL NOT BE LIABLE
TO YOU FOR ACTUAL, DIRECT, INDIRECT, CONSEQUENTIAL,
PUNITIVE OR INCIDENTAL DAMAGES EVEN IF YOU GIVE
NOTICE OF THE POSSIBILITY OF SUCH DAMAGE.

1.F.3. LIMITED RIGHT OF REPLACEMENT OR REFUND - If you

discover a defect in this electronic work within 90 days of
receiving it, you can receive a refund of the money (if any) you
paid for it by sending a written explanation to the person you
received the work from. If you received the work on a physical
medium, you must return the medium with your written
explanation. The person or entity that provided you with the
defective work may elect to provide a replacement copy in lieu
of a refund. If you received the work electronically, the person
or entity providing it to you may choose to give you a second
opportunity to receive the work electronically in lieu of a refund.
If the second copy is also defective, you may demand a refund
in writing without further opportunities to fix the problem.

1.F.4. Except for the limited right of replacement or refund set

forth in paragraph 1.F.3, this work is provided to you ‘AS-IS’,
WITH NO OTHER WARRANTIES OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR ANY PURPOSE.

1.F.5. Some states do not allow disclaimers of certain implied

warranties or the exclusion or limitation of certain types of
damages. If any disclaimer or limitation set forth in this
agreement violates the law of the state applicable to this
agreement, the agreement shall be interpreted to make the
maximum disclaimer or limitation permitted by the applicable
state law. The invalidity or unenforceability of any provision of
this agreement shall not void the remaining provisions.

1.F.6. INDEMNITY - You agree to indemnify and hold the

Foundation, the trademark owner, any agent or employee of the
Foundation, anyone providing copies of Project Gutenberg™
electronic works in accordance with this agreement, and any
volunteers associated with the production, promotion and
distribution of Project Gutenberg™ electronic works, harmless
from all liability, costs and expenses, including legal fees, that
arise directly or indirectly from any of the following which you
do or cause to occur: (a) distribution of this or any Project
Gutenberg™ work, (b) alteration, modification, or additions or
deletions to any Project Gutenberg™ work, and (c) any Defect
you cause.

Section 2. Information about the Mission

of Project Gutenberg™
Project Gutenberg™ is synonymous with the free distribution of
electronic works in formats readable by the widest variety of
computers including obsolete, old, middle-aged and new
computers. It exists because of the efforts of hundreds of
volunteers and donations from people in all walks of life.

Volunteers and financial support to provide volunteers with the

assistance they need are critical to reaching Project
Gutenberg™’s goals and ensuring that the Project Gutenberg™
collection will remain freely available for generations to come. In
2001, the Project Gutenberg Literary Archive Foundation was
created to provide a secure and permanent future for Project
Gutenberg™ and future generations. To learn more about the
Project Gutenberg Literary Archive Foundation and how your
efforts and donations can help, see Sections 3 and 4 and the
Foundation information page at www.gutenberg.org.

Section 3. Information about the Project

Gutenberg Literary Archive Foundation
The Project Gutenberg Literary Archive Foundation is a non-
profit 501(c)(3) educational corporation organized under the
laws of the state of Mississippi and granted tax exempt status
by the Internal Revenue Service. The Foundation’s EIN or
federal tax identification number is 64-6221541. Contributions
to the Project Gutenberg Literary Archive Foundation are tax
deductible to the full extent permitted by U.S. federal laws and
your state’s laws.

The Foundation’s business office is located at 809 North 1500

West, Salt Lake City, UT 84116, (801) 596-1887. Email contact
links and up to date contact information can be found at the
Foundation’s website and official page at
www.gutenberg.org/contact
 Section 4. Information about Donations to
the Project Gutenberg Literary Archive
Foundation
Project Gutenberg™ depends upon and cannot survive without
widespread public support and donations to carry out its mission
of increasing the number of public domain and licensed works
that can be freely distributed in machine-readable form
accessible by the widest array of equipment including outdated
equipment. Many small donations ($1 to $5,000) are particularly
important to maintaining tax exempt status with the IRS.

The Foundation is committed to complying with the laws

regulating charities and charitable donations in all 50 states of
the United States. Compliance requirements are not uniform
and it takes a considerable effort, much paperwork and many
fees to meet and keep up with these requirements. We do not
solicit donations in locations where we have not received written
confirmation of compliance. To SEND DONATIONS or determine
the status of compliance for any particular state visit
www.gutenberg.org/donate.

While we cannot and do not solicit contributions from states

where we have not met the solicitation requirements, we know
of no prohibition against accepting unsolicited donations from
donors in such states who approach us with offers to donate.

International donations are gratefully accepted, but we cannot

make any statements concerning tax treatment of donations
received from outside the United States. U.S. laws alone swamp
our small staff.

Please check the Project Gutenberg web pages for current

donation methods and addresses. Donations are accepted in a
number of other ways including checks, online payments and
credit card donations. To donate, please visit:
www.gutenberg.org/donate.

Section 5. General Information About

Project Gutenberg™ electronic works
Professor Michael S. Hart was the originator of the Project
Gutenberg™ concept of a library of electronic works that could
be freely shared with anyone. For forty years, he produced and
distributed Project Gutenberg™ eBooks with only a loose
network of volunteer support.

Project Gutenberg™ eBooks are often created from several

printed editions, all of which are confirmed as not protected by
copyright in the U.S. unless a copyright notice is included. Thus,
we do not necessarily keep eBooks in compliance with any
particular paper edition.

Most people start at our website which has the main PG search
facility: www.gutenberg.org.

This website includes information about Project Gutenberg™,

including how to make donations to the Project Gutenberg
Literary Archive Foundation, how to help produce our new
eBooks, and how to subscribe to our email newsletter to hear
about new eBooks.
Welcome to our website – the perfect destination for book lovers and
knowledge seekers. We believe that every book holds a new world,
offering opportunities for learning, discovery, and personal growth.
That’s why we are dedicated to bringing you a diverse collection of
books, ranging from classic literature and specialized publications to
self-development guides and children's books.

More than just a book-buying platform, we strive to be a bridge

connecting you with timeless cultural and intellectual values. With an
elegant, user-friendly interface and a smart search system, you can
quickly find the books that best suit your interests. Additionally,
our special promotions and home delivery services help you save time
and fully enjoy the joy of reading.

Join us on a journey of knowledge exploration, passion nurturing, and

personal growth every day!

ebookbell.com