Cognitive Technologies

Iliano Cervesato

The
Deductive
Spreadsheet
Cognitive Technologies

Managing Editors: D.M. Gabbay J. Siekmann

Editorial Board: A. Bundy J.G. Carbonell

M. Pinkal H. Uszkoreit M. Veloso W. Wahlster
M.J. Wooldridge

For further volumes:

http://www.springer.com/series/5216
Iliano Cervesato

The Deductive Spreadsheet

123
Iliano Cervesato
Carnegie Mellon University
School of Computer Science
Doha, Qatar

Managing Editors
Dov M. Gabbay Jörg Siekmann
Augustus De Morgan Professor of Logic Forschungsbereich Deduktions- und
Department of Computer Science Multiagentensysteme
King’s College London DFKI
London, UK Saarbrücken, Germany

ISSN 1611-2482 Cognitive Technologies

ISBN 978-3-642-37746-4 ISBN 978-3-642-37747-1 (eBook)
DOI 10.1007/978-3-642-37747-1
Springer Heidelberg New York Dordrecht London

Library of Congress Control Number: 2013943471

ACM Computing Classification (1998): H.2, I.2, H.1, D.1

c Springer-Verlag Berlin Heidelberg 2013

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology
now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’s location, in its current version, and permission for use must always be obtained from Springer.
Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations
are liable to prosecution under the respective Copyright Law.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
While the advice and information in this book are believed to be true and accurate at the date of
publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for
any errors or omissions that may be made. The publisher makes no warranty, express or implied, with
respect to the material contained herein.

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)

Grazie di dut, papá
[1932–2012]
Preface

Nowadays, data are more easily accessible than ever, yet support for deriving
interesting consequences from base data is often unavailable, too expensive, or too
technical for many users. For example, a student may have access to prerequisite
listings and expected offering dates of courses but have no way to sieve through
possible course sequences unless the college provides a dedicated tool. Similarly,
an investor may know the instruments held in his or her mutual fund portfolio but
have no easy way to unravel them and reveal his exposure to a specific industry or
company. As an additional example, a secretary in a midsize company will typically
need to juggle clients and transportation schedules by hand into an itinerary for her
boss’s forthcoming business trip. In all cases, manually inferring useful information
from raw data is time-consuming and error-prone, a situation that often results in
bad decisions, suboptimal plans, or missed opportunities.
The problem is not the lack of automated tools: databases are all about relating
data, financial service firms will happily use their software to dissect one’s portfolio
in any desired way, and a plethora of scheduling and workflow applications are
available for purchase. Yet individuals, small and medium enterprises, and even
small departments in large companies rarely avail themselves of such products: even
when the purchase price is affordable, there is an associated financial and cognitive
training cost, often quite large. Indeed, the secretary may be unknowingly querying
databases all day long, but setting one up may go well beyond her training. There is
no simple and general application that empowers users to compute useful inferences
on raw data.
This book addresses this problem by drawing inspiration from a type of
automated data inference that is immensely popular: the spreadsheet. Applications
such as Microsoft Excel and Google Spreadsheet are readily available and allow
users to routinely perform complex custom calculations on numerical data. The
spreadsheet’s clever interface makes it easy to use productively with little or no
training and its gentle learning curve allows users to reinforce their skills and
acquire new ones in a learning-by-doing fashion. However, none of the above data
manipulation problems is expressible in today’s spreadsheets. The approach investi-
gated here remedies this situation by extending the spreadsheet paradigm to enable

vii
viii Preface

users to define useful forms of inference among their data. An implementation

of this idea will allow the student, for example, to download data about course
prerequisites and offerings into his favorite spreadsheet and write a “formula” that
will calculate all possible course sequences. The investor will similarly be able to
see the individual stocks in his portfolio and determine his actual exposure, and the
secretary will have the means to generate workable itineraries and easily choose the
least expensive or most convenient. Differently from related efforts, our approach
focuses on retaining the cognitive simplicity of the traditional spreadsheet while
making these extended functionalities available: like charts in Excel, for example,
they should be unobtrusive when not needed and should be intuitively usable when
needed. Unlike charts, the proposed extension touches many of the fundamental
mechanisms underlying the spreadsheet architecture.
We call this extended paradigm the deductive spreadsheet, which we contrast
with the traditional spreadsheets in use today. This book describes an abstract design
for the deductive spreadsheet. As such, it is not tied to any specific commercial
application, although we draw examples and motivations from several, and it has
not yet been implemented. We adopt an interdisciplinary approach to developing
this idea. The design of the deductive support of this application, the part that sits
under the hood and that the user rarely thinks about, weaves together techniques
and results from several fields within computer science and computational logic,
in particular programming language theory, logic programming, databases, proof
theory, and model theory. The parallel design of the user interface, the aspect that
the user does see and will use to access these extended functionalities, combines
research from cognitive psychology and some of the more experimental areas of
computer science, in particular human-computer interaction. The result is not only
the blueprint for a useful productivity tool but may be more importantly a balanced
integration of methodologies from fields that rarely intersect in academic research
(although they do in industry).
Expanding somewhat on the design, the internal extension underlying the
deductive spreadsheet centers around interpreting a group of columns and rows as
a relation, something that users of a traditional spreadsheet commonly do already
each time they organize data in tabular form. The key idea is then to provide
support for manipulating relations as first-class objects, in particular, to allow us
to write formulas that compute to relations and to extend the core mechanisms
for evaluation, update, and explanation to these new entities. As a language
for relational formulas, we design a variant of the logic programming language
Datalog. Datalog has been studied for many years in the context of deductive
databases and is now applied in far broader domains, it has excellent computational
properties and good expressiveness, and it comes with a number of algorithms
that are compatible with a spreadsheet context. We extend its syntax to support
the embedding of traditional spreadsheet formulas and to make it usable in a wide
range of practical circumstances. These logical formulas are written in cell ranges
and, upon evaluation, fill this range with the set of records that satisfy them. In
order to do so, they interpret other areas of the spreadsheet as relations and combine
portions of those records into their own result. The addition of Datalog-like formulas
Preface ix

significantly extends the expressive power of the traditional spreadsheet, enabling

useful classes of new problems to be solved in a spreadsheet environment—in
particular all the examples mentioned above.
On the outward-facing side, the user interface of the deductive spreadsheet is
designed as a conservative extension of the mostly excellent interface of typical
commercial spreadsheets. Indeed, the traditional spreadsheet owes much of its
success to a user interface that has evolved to provide intuitive access to the
underlying functionalities. In particular, it offers consistent interaction modalities
across features and a gentle learning curve that allows users to progress through
exposure and simple experimentation. By carefully leveraging recent methodologies
from cognitive science, specifically the attention investment model and the cognitive
dimensions of notation, we engineered a user interface for the deductive spreadsheet
that retains the very same level of usability of current tools. The deductive
functionalities are indeed barely noticeable for users who have no need for them
and yet accessible in a natural way for those who need their added expressiveness.
The new deductive components are made available through intuitive extensions of
all common gestures and other standard interaction methodologies. Furthermore,
the deductive infrastructure is seamlessly integrated into the traditional spreadsheet
so that users not only still have access to the usual functionalities but are able to use
them as part of logical inferences.
Looking forward, a future implementation of the design in this book could
act as an automated assistant for the daily reasoning and decision-making needs
of computer users, in the same way as the traditional spreadsheet assists them
every day with calculations simple and complex. It will enable users without
formal training in logic or computer science to interactively define logical rules
in the same way as they define numerical formulas in today’s spreadsheet. This
deductive spreadsheet application targets the same segment of computer users as the
traditional spreadsheet, estimated at over 50 million users. It has the same potential
to boost productivity and help people with their daily tasks that made the traditional
spreadsheet so successful.
The book is divided into two main parts: the first half focuses on the deductive
engine that underlies this application, the foundations that users do not see. After
giving a mathematical model of traditional spreadsheet applications, it extends
them with operators to perform a number of relational tasks, akin to the user
view of a database but in a spreadsheet context. Expressing this extension in a
logic programming framework is a natural stepping stone toward giving it powerful
deductive capabilities. The second half of the book deals with the user interface, the
part of the application that the user actually interacts with. It reviews the elements
of the graphical user interface of traditional spreadsheet applications and describes
practical methodologies for designing user interfaces from the field of cognitive
psychology. It then proposes a design that conservatively integrates mechanisms for
a user to take advantage of the new deductive capabilities. This is followed by the
results of some preliminary usability experiments.
The book is expected to appeal to researchers and practitioners in the various
areas underlying this work. Researchers will not only find interesting new
x Preface

developments in most of these areas but also learn how to reach out to adjoining
and distant fields to achieve a multidisciplinary focus. Practitioners will find fully
developed solutions to numerous problems that are not solvable using a traditional
spreadsheet application or not easily solvable with them. They will then be able to
either enter them in a forthcoming deductive spreadsheet application or to use the
provided insight to program them.

Doha, Qatar Iliano Cervesato

August 2012
Acknowledgments

We are grateful to numerous individuals with whom we discussed some of the

ideas contained in this book and who provided precious feedback and pointed us
to essential references in the literature. These include Stefano Ceri, Cédric Fournet,
Burkhard Freitag, Georg Gottlob, Mike Kassof, Dusko Pavlovic, Frank Pfenning,
Alberto Pravato, Alessandro Roncato, Letizia Tanca, and David Walker. We are
especially indebted to the users who volunteered some of their time to provide us
with valuable comments on functionalities and user interface design.
The research that led to this book was supported by DARPA under contract
W31P4Q-05-C-R040 and QNRF under grant NPRP 4-341-1-059.

xi
Contents

1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1
1.1 Deductive Decision-Making . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 3
1.2 Target Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 4
1.3 Drawbacks of Existing Solutions .. . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5
1.3.1 Spreadsheets . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5
1.3.2 Logic Programming . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 6
1.3.3 Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 9
1.3.4 Task-Specific Tools . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 10
1.4 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 10
1.4.1 The Deductive Engine .. . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 11
1.4.2 The User Interface .. . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 13
1.5 A More Elaborate Example .. . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 14
1.6 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 17
1.6.1 Logics and Spreadsheets . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 18
1.6.2 Beyond Logical Spreadsheets .. . . . . . . . .. . . . . . . . . . . . . . . . . . . . 20
1.7 Work Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 22
2 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 25
2.1 Functional Requirements . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 25
2.2 Cognitive Requirements . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 28
2.3 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 30

Part I Deductive Engine

3 The Traditional Spreadsheet.. . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 33

3.1 Spreadsheet Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 33
3.2 Evaluation .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 37
3.2.1 Naı̈ve Strategy .. . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 37
3.2.2 Elements of Fixpoint Theory . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 39
3.2.3 Dependency Graph .. . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 43
3.3 Updates .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 49

xiii
xiv Contents

3.4 Explanation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 53
3.5 Summary .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 53
4 The Relational Spreadsheet .. . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 55
4.1 Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 55
4.1.1 Generic Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 57
4.1.2 Spreadsheet Arrays . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 58
4.1.3 Array Operators . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 59
4.1.4 Array Formulas . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 60
4.1.5 Component Expressions . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 60
4.1.6 Spreadsheets with Arrays .. . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 62
4.1.7 Arrays Versus Scalars . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 64
4.2 Relational Support in the Traditional Spreadsheet . . . . . . . . . . . . . . . . . . . 64
4.3 Extending Spreadsheets with First-Class Relations .. . . . . . . . . . . . . . . . . 66
4.3.1 Formalizing Relations . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 67
4.3.2 Relational Operators and Expressions . .. . . . . . . . . . . . . . . . . . . . 71
4.3.3 Relational Spreadsheets.. . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 85
4.3.4 Relational Spreadsheets Versus Relational Databases . . . . . 91
4.4 Evaluation .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 95
4.4.1 Cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 95
4.4.2 Naı̈ve Evaluation .. . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 98
4.4.3 Dependency Graph .. . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 102
4.4.4 Semi-naı̈ve Evaluation .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 106
4.5 Updates .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 108
4.5.1 Relation-Level Updates .. . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 108
4.5.2 Incremental Updates of Relations . . . . . . .. . . . . . . . . . . . . . . . . . . . 110
4.5.3 Recalculation Policies . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 114
4.6 Explanation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 115
4.7 Summary .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 116
5 The Logical Spreadsheet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 119
5.1 A First-Order Logic Refresher . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 120
5.1.1 Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 120
5.1.2 Model Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 122
5.1.3 Proof Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 122
5.2 The Logic of the Relational Spreadsheet . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 123
5.2.1 Relational Substrate .. . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 124
5.2.2 Relational Algebra as Logic .. . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 127
5.2.3 Stratification .. . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 135
5.3 Inference in a Definite Spreadsheet Theory . . . . . .. . . . . . . . . . . . . . . . . . . . 139
5.3.1 Non-recursive Definite Theories . . . . . . . .. . . . . . . . . . . . . . . . . . . . 139
5.3.2 Herbrand Models. . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 143
5.3.3 Fixpoint Characterization .. . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 144
5.3.4 Computing Fixpoints . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 145
5.3.5 Proof Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 148
Contents xv

5.4 Inference in a Normal Spreadsheet Theory .. . . . . .. . . . . . . . . . . . . . . . . . . . 154

5.4.1 Non-recursive Normal Theories .. . . . . . . .. . . . . . . . . . . . . . . . . . . . 154
5.4.2 Model Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 157
5.4.3 Proof Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 160
5.5 Spreadsheet Evaluation Revisited . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 163
5.5.1 Naı̈ve Evaluation .. . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 163
5.5.2 Localized Semi-naı̈ve Evaluation . . . . . . .. . . . . . . . . . . . . . . . . . . . 166
5.6 Updates Revisited. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 168
5.6.1 Positive Update of Positive Literals . . . . .. . . . . . . . . . . . . . . . . . . . 168
5.6.2 Negative Update of Positive Literals . . . .. . . . . . . . . . . . . . . . . . . . 172
5.6.3 Updates to Negative Literals . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 174
5.6.4 Update Propagation in a Normal Theory . . . . . . . . . . . . . . . . . . . 176
5.7 Explanation Revisited . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 177
5.7.1 Positive Explanation . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 178
5.7.2 Negative Explanation . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 179
5.7.3 Integrated Explanation .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 180
5.8 Summary .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 180
6 The Deductive Spreadsheet . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 183
6.1 Spreadsheets with Recursive Definitions . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 184
6.1.1 Transitive Closure . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 184
6.1.2 Expressing Extensional Universal Quantification . . . . . . . . . . 186
6.1.3 Combining Deductive and Functional Computation . . . . . . . 189
6.1.4 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 191
6.2 Recursive Spreadsheet Theories . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 192
6.2.1 Syntax and Basic Definitions . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 192
6.2.2 Fixpoint Semantics and Evaluation . . . . .. . . . . . . . . . . . . . . . . . . . 198
6.2.3 Bottom-Up Derivation and Updates.. . . .. . . . . . . . . . . . . . . . . . . . 206
6.2.4 Top-Down Derivation and Explanation .. . . . . . . . . . . . . . . . . . . . 208
6.2.5 Historical Perspective .. . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 211
6.2.6 Relational Counterpart and Relative Expressiveness .. . . . . . 213
6.3 Bounded Termination.. . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 217
6.3.1 Head Constraints in Recursive Clauses .. . . . . . . . . . . . . . . . . . . . 217
6.3.2 Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 221
6.3.3 Irrelevant Terms .. . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 229
6.4 Additional Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 230
6.4.1 Abbreviated Forms .. . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 231
6.4.2 More Transitive Closure Problems .. . . . .. . . . . . . . . . . . . . . . . . . . 234
6.4.3 Bill of Materials Problem .. . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 236
6.4.4 Workflow Problem . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 237
6.4.5 Scheduling Problem . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 239
6.4.6 Anti-trust Control Problem . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 240
6.5 Extensions.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 241
6.5.1 Embedded Implication .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 242
6.5.2 Complex Objects . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 249
xvi Contents

6.5.3 Typing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 250

6.6 Summary .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 250

Part II User Interface

7 The Traditional Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 255

7.1 Layout .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 260
7.1.1 Cells, Worksheets, and Workbooks . . . . .. . . . . . . . . . . . . . . . . . . . 260
7.1.2 Formatting Within a Cell . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 263
7.1.3 Formatting Within a Worksheet . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 264
7.2 Formulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 265
7.2.1 Scalar Formulas . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 266
7.2.2 Array Formulas . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 268
7.2.3 Graphical Construction of Formulas . . . .. . . . . . . . . . . . . . . . . . . . 269
7.2.4 Copy and Paste of Formulas .. . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 272
7.3 Controlling Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 274
7.3.1 Updates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 274
7.3.2 Explanation .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 275
7.4 Productivity Facilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 277
7.5 Summary .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 277
8 Cognitive Interface Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 279
8.1 Interface Usability Evaluation . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 280
8.1.1 User Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 280
8.1.2 Predictive User Models . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 281
8.1.3 Lightweight Approximate Methods .. . . .. . . . . . . . . . . . . . . . . . . . 283
8.2 Cognitive Dimensions of Notation .. . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 283
8.2.1 Discussion Tools for Visual Languages .. . . . . . . . . . . . . . . . . . . . 284
8.2.2 Principal Cognitive Dimensions.. . . . . . . .. . . . . . . . . . . . . . . . . . . . 285
8.3 Attention Investment Model . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 293
8.3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 294
8.3.2 Attention Investment Analysis of the Deductive
Spreadsheet . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 294
8.3.3 Integration with the Cognitive Dimensions Model .. . . . . . . . 296
8.4 Target Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 296
8.5 Summary .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 298
9 Toward a Deductive Interface . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 299
9.1 Deductive Layout .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 300
9.1.1 Defining Relations . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 300
9.1.2 The Formula Input Textbox . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 306
9.1.3 Further Considerations.. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 307
9.2 Textual Input of Clausal Theories.. . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 309
9.2.1 Base Relations and Predicate Names . . .. . . . . . . . . . . . . . . . . . . . 310
9.2.2 Logical Syntax of Clauses . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 312
9.2.3 Relational Syntax of Clauses . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 319
Contents xvii

9.2.4 Handling and Preventing Errors .. . . . . . . .. . . . . . . . . . . . . . . . . . . . 322

9.2.5 Integrating Logical Formulas and Traditional
Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 324
9.3 Graphical Construction of Clausal Theories . . . . .. . . . . . . . . . . . . . . . . . . . 325
9.3.1 Mouse-Assisted Clause Definition . . . . . .. . . . . . . . . . . . . . . . . . . . 326
9.3.2 Dialog-Assisted Clause Definition . . . . . .. . . . . . . . . . . . . . . . . . . . 330
9.3.3 Copy and Paste . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 333
9.4 Updates .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 335
9.4.1 Altering the Geometry of a Relation . . . .. . . . . . . . . . . . . . . . . . . . 335
9.4.2 Cut, Copy and Paste Across Clausal Definitions . . . . . . . . . . . 338
9.4.3 Controlling Evaluation .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 338
9.5 Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 340
9.6 Productivity Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 343
9.7 Getting Help . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 344
9.8 Summary .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 345
10 Preliminary Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 347
10.1 Outline of the Experiment . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 347
10.1.1 User Background . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 348
10.1.2 Illustration of the Deductive Spreadsheet . . . . . . . . . . . . . . . . . 349
10.1.3 Exposure to the User Interface .. . . . . . . .. . . . . . . . . . . . . . . . . . . . 353
10.2 Tested Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 353
10.3 Results of the Experiments . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 355
10.3.1 Advanced Users . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 355
10.3.2 Intermediate Users . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 356
10.3.3 Beginners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 356
10.4 Summary.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 357
11 Future Developments .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 365
11.1 Prototyping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 365
11.1.1 Strategies .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 366
11.1.2 Pre-prototyping . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 367
11.2 Experimental Assessment . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 369
11.3 Functional Extensions .. . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 370
11.3.1 Technical Challenges .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 371
11.3.2 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 371
11.4 Interface Extensions .. . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 372
11.4.1 Cognitive Challenges .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 372
11.4.2 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 374

Annotated Bibliography .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 375

Symbols in Use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 395

Index . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 401
List of Figures

Fig. 1.1 Example of base relations . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 13

Fig. 1.2 Example of deduced relations . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 14
Fig. 1.3 The investor’s spreadsheet . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 15
Fig. 3.1 A simple timecard spreadsheet . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 34
Fig. 3.2 Iterations in the evaluation of the timecard example .. . . . . . . . . . . . . . 39
Fig. 3.3 Least upper bound and greatest lower bound of a set S 0 . . . . . . . . . . . 41
Fig. 3.4 The dependency graph of the timecard example .. . . . . . . . . . . . . . . . . . 44
Fig. 3.5 The dependency graph of the timecard example
annotated with expressions . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 45
Fig. 3.6 Evaluation of the timecard example on its dependency
graph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 46
Fig. 3.7 Environment partitioning for semi-naı̈ve evaluation . . . . . . . . . . . . . . . 48
Fig. 3.8 Two-step update of the timecard spreadsheet . .. . . . . . . . . . . . . . . . . . . . 49
Fig. 3.9 The dependency graph of the updated timecard
example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 51
Fig. 3.10 Evaluation of the first update in the timecard example . . . . . . . . . . . . 52
Fig. 3.11 Evaluation of the second update in the timecard
example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 52
Fig. 3.12 Model of a spreadsheet . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 54
Fig. 3.13 Model of spreadsheet evaluation . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 54
Fig. 4.1 Rewriting the timecard spreadsheet with array
operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 56
Fig. 4.2 Example of scalar and array operations in commercial
spreadsheets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 60
Fig. 4.3 A simplified registrar’s spreadsheet . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 68
Fig. 4.4 Support for scalar, array and relational operations
in the relational spreadsheet . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 72
Fig. 4.5 Core operators of relational algebra (plus join) . . . . . . . . . . . . . . . . . . . . 74
Fig. 4.6 Relational expressions in the registrar’s spreadsheet . . . . . . . . . . . . . . 80

xix
xx List of Figures

Fig. 4.7 Extended relational expressions in the registrar’s

spreadsheet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 82
Fig. 4.8 Pseudo-relational expressions in the registrar’s
spreadsheet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 84
Fig. 4.9 Choices for the notion of subexpression
in the relational spreadsheet . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 86
Fig. 4.10 Differences between a relational database
and a relational spreadsheet . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 92
Fig. 4.11 Cost upper bounds of relational operations . . . .. . . . . . . . . . . . . . . . . . . . 96
Fig. 4.12 Three views of an attribute-dependency subgraph
in the registrar’s spreadsheet . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 103
Fig. 4.13 Expression-dependency subgraph in the registrar’s
spreadsheet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 105
Fig. 4.14 A relational-dependency subgraph in the registrar’s
spreadsheet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 106
Fig. 4.15 A mixed dependency subgraph in the registrar’s
spreadsheet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 106
Fig. 4.16 Environment partitioning for semi-naı̈ve evaluation . . . . . . . . . . . . . . . 107
Fig. 4.17 Abstract syntax of the relational spreadsheet . .. . . . . . . . . . . . . . . . . . . . 116
Fig. 5.1 Relational substrate of the registrar’s spreadsheet .. . . . . . . . . . . . . . . . . 126
Fig. 5.2 Logical expression of selected registrar’s relations .. . . . . . . . . . . . . . . . 135
Fig. 5.3 Logical rendering of the registrar’s spreadsheet . . . . . . . . . . . . . . . . . . . . 136
Fig. 5.4 Stratified call graph in the registrar’s spreadsheet . . . . . . . . . . . . . . . . . . 137
Fig. 5.5 Normal theories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 181
Fig. 6.1 Transitive course requirements in the registrar’s spreadsheet .. . . . . 185
Fig. 6.2 Course evaluation in the registrar’s spreadsheet . . . . . . . . . . . . . . . . . . . . 187
Fig. 6.3 Recursive clauses and formulas in the registrar’s spreadsheet . . . . . 190
Fig. 6.4 Stratified call graph in the recursive part
of the registrar’s spreadsheet.. . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 194
Fig. 6.5 Dataflow graph and head constraint restriction .. . . . . . . . . . . . . . . . . . . . 197
Fig. 6.6 Effective top-down derivability .. . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 208
Fig. 6.7 The language of the deductive spreadsheet within logic
programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 213
Fig. 6.8 Relational definitions for the registrar’s example .. . . . . . . . . . . . . . . . . . 214
Fig. 6.9 Relative expressive power of the languages examined . . . . . . . . . . . . . 215
Fig. 6.10 Computing with lists in the registrar’s spreadsheet .. . . . . . . . . . . . . . . . 228
Fig. 6.11 Abbreviations for some common constructions . . . . . . . . . . . . . . . . . . . . 232
Fig. 6.12 Some interpretations of transitive closure .. . . . . .. . . . . . . . . . . . . . . . . . . . 235
Fig. 7.1 Initial screenshot of eight commercial spreadsheets.. . . . . . . . . . . . . . . 256
Fig. 7.2 Layout of Microsoft Excel 2003.. . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 261
Fig. 7.3 Assigning symbolic names to a multiple selection . . . . . . . . . . . . . . . . . 263
Fig. 7.4 A data list and a data form for it . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 264
Fig. 7.5 The formula bar and related functions . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 267
List of Figures xxi

Fig. 7.6 Textual formula input . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 267

Fig. 7.7 Textual input of an array formula . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 268
Fig. 7.8 Building a formula using the mouse . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 270
Fig. 7.9 Constructing a formula using a dialog . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 271
Fig. 7.10 Cut and paste involving formulas . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 273
Fig. 7.11 Color-coded visualization of the references embedded
in a formula .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 275
Fig. 7.12 Tracing the dependents of a cell using the audit tool .. . . . . . . . . . . . . . 276
Fig. 8.1 Schematic effect of changing a system along a dimension .. . . . . . . . 285
Fig. 8.2 Major cognitive dimensions . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 286
Fig. 8.3 Selected trade-offs among dimensions .. . . . . . . . .. . . . . . . . . . . . . . . . . . . . 287
Fig. 9.1 Right-click menu access to the relation definition dialog . . . . . . . . . . 302
Fig. 9.2 Relation definition dialog .. . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 303
Fig. 9.3 “path” as defined by the “Define Relation: : :” dialog . . . . . . . . . . . . 304
Fig. 9.4 Multiline formula input textbox . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 308
Fig. 9.5 Clausal theories with abbreviated forms . . . . . . . .. . . . . . . . . . . . . . . . . . . . 313
Fig. 9.6 Error handling in Microsoft Excel.. . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 323
Fig. 9.7 Error handling in the deductive spreadsheet . . . .. . . . . . . . . . . . . . . . . . . . 324
Fig. 9.8 Mouse-assisted predicate selection .. . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 327
Fig. 9.9 Mouse-assisted predicate selection (continued) . . . . . . . . . . . . . . . . . . . . 328
Fig. 9.10 Mouse-assisted insertion of a constraint . . . . . . . .. . . . . . . . . . . . . . . . . . . . 329
Fig. 9.11 Dialog-assisted clausal entry . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 332
Fig. 9.12 Entering an abbreviated form using a dialog.. . .. . . . . . . . . . . . . . . . . . . . 334
Fig. 9.13 Invoking the “Redefine Relation” dialog .. . . . . . .. . . . . . . . . . . . . . . . . . . . 336
Fig. 9.14 Global recalculation preferences for the deductive spreadsheet . . . 340
Fig. 9.15 Asking for an explanation . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 341
Fig. 9.16 Deductive explanation dialog .. . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 342
Fig. 9.17 A connection graph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 344
Fig. 10.1 Specification of the direct flights between airports . . . . . . . . . . . . . . . . . 350
Fig. 10.2 Questionnaire .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 359
Fig. 10.3 Illustrated clausal definitions . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 360
Fig. 10.4 Interface screenshots used in experiments (1) . .. . . . . . . . . . . . . . . . . . . . 361
Fig. 10.5 Interface screenshots used in experiments (2) . .. . . . . . . . . . . . . . . . . . . . 362
Fig. 10.6 Interface screenshots used in experiments (3) . .. . . . . . . . . . . . . . . . . . . . 363
Fig. 10.7 Interface screenshots used in experiments (4) . .. . . . . . . . . . . . . . . . . . . . 364
Chapter 1
Introduction

Envisioned in the early 1960s, born in the late 1970s and popularized during the
1980s with the personal computer [Pow03], the spreadsheet is one of the most
ubiquitous computing applications (it has only in recent years been surpassed by
email programs and web browsers). Over 50 million people—secretaries, accoun-
tants, teachers, engineers, officers, managers and many others—rely on it every
day for everything from grocery lists to multimillion dollar transactions [BABC 00].
The vast majority of these users have no or little formal computer science training,
and yet the intuitive interface and clever design of the spreadsheet let them put
together complex computations with a few mouse clicks, instantly turning them
into unaware programmers. This aspect cannot be overemphasized: in almost no
time can a novice user perform simple tasks, and just a few hours of exposure
are sufficient to acquire the skills to perform non-trivial computations over large
amounts of data.1 De facto, the spreadsheet is the tool of choice for non-scientific
numerical computations and decision-making based on numerical data.
However, a large percentage, if not the majority, of the decisions we make every
day depends on information that is not numerical, or not only numerical. Consider
the following simple example from the academic world:
Is a student who has taken ‘Operating Systems’ and ‘Theory of Computation’ qualified to
take ‘Advanced Networking’? If not, what does she need to take first? Would that allow
her to take ‘Advanced Networking’ next semester? If not, what is the earliest date she can
take it?

Answering each of these questions requires logical reasoning. The last also has a
numerical component. Students and courses are symbolic data and they participate

1
I cannot pass on the following anecdote: A friend of mine, a physician by trade, discovered the
personal computer and Excel a few years ago. Within a month, he had entered several years of
financial data for all the stocks he owned, together with various formulas to predict their trend.
He never found the silver bullet that would allow him to beat the market, but he eventually turned
his self-taught Excel skills to medicine and built a sophisticated application to track the cardiac
history of his patients. It is believed that several lives were saved in that way.

I. Cervesato, The Deductive Spreadsheet, Cognitive Technologies, 1

DOI 10.1007/978-3-642-37747-1 1, © Springer-Verlag Berlin Heidelberg 2013
2 1 Introduction

in relations that describe what courses a student has taken, what the prerequisites of
a course are, and when courses are offered. Inference rules (not shown here) describe
the reasoning processes that need to be performed to produce the answers. Similar
setups and reasoning tasks routinely take place in many professional and personal
settings: diagnosis of diseases, combat readiness of troops, legal argumentation,
even fashionable color coordination require reasoning patterns akin to the above
example.
For all their merits, spreadsheets as we know them are pretty much useless when
it comes to these forms of symbolic inference: they offer a few logical operators,
but these are designed for simple conditional tests, not deductive reasoning.
Worse, the computer applications that do support these forms of reasoning, logic
programming languages, some databases, and many domain-specific tools, require
a level of sophistication that even many trained programmers do not have. Indeed,
these applications are seldom installed in an end-user’s computing environment.
This leaves a prospective problem-solver to either undergo lengthy training or give
up on automation. This last option often leads to arbitrary, suboptimal, or incorrect
decisions.
Based on these considerations, it is clear that an application supporting sophis-
ticated logical reasoning with the same degree of usability as a spreadsheet would
fill a significant void in the end-user’s computing landscape. Following [Gun04],
we call this tool a deductive spreadsheet, a tool that (quoting)
would allow users to define logic statements and inference rules for symbolic reasoning
in the same way that current spreadsheets allow users to define mathematical formulae for
numerical calculations.

It should be emphasized that this deductive spreadsheet has the same target audience
as what we will call from now on the traditional spreadsheet: those 50C million
managers, officers, engineers, teachers, accountants, secretaries, etc., who do not
have a formal training in either computer science or logic, and yet would welcome
unintimidating automated assistance with their daily reasoning tasks. We speculate
that this enormous market represents a significant business opportunity, but most
importantly the deductive spreadsheet has the same potential to boost productivity
and help people with their daily tasks that made the traditional spreadsheet so
successful.
In this book, we propose an approach to engineering a deductive spreadsheet
that not only meets the reasoning and usability requirements outlined above,
but integrates them seamlessly into the existing spreadsheet paradigm. We postulate
that a deductive extension to the spreadsheet is superior to a separate application for
two reasons: first, many decisions have symbolic and numerical aspects; second, the
distinction between the two may be lost on an entry-level or even intermediate user,
who may never use a separate deductive spreadsheet application for this reason.
This introductory chapter motivates the deductive spreadsheet and outlines our
approach. In Sect. 1.1 we identify three useful classes of reasoning problems that
the deductive spreadsheet should support. In Sect. 1.2, we characterize the target
audience of this tool. In Sect. 1.3 we review existing deductive technology, pointing
1.1 Deductive Decision-Making 3

out drawbacks and taking note of useful features in the spreadsheet context.
We sketch our approach in Sect. 1.4 and demonstrate it on an extended example
in Sect. 1.5. We discuss related work in Sect. 1.6. The concluding Sect. 1.7 outlines
the organization of the remainder of this book.

1.1 Deductive Decision-Making

There are several ways in which logical reasoning can participate in the process of
making a decision. If we think of an automated decision-assistant as an infallible
logical reasoner, here are three of the most useful questions with which a human
decision-maker may consult it:
Yes or no? Given a problem described in terms of facts established to be true
and inference rules that permit making logical deductions from them, determine
whether a given objective is realizable or not. Consider the following example,
adapted from above:
Assume that Sue has taken ‘Operating Systems’ and ‘Theory of Computation’. Assume
also that there is a rule that states that sitting in ‘Advanced Networking’ requires having
taken ‘Operating Systems’, ‘Computer Architecture’ and ‘Basic Networking’. When
asked the question Can Sue take ‘Advanced Networking’?, an automated reasoner will
answer ‘No’.

While this example hardly requires automated assistance in itself, it will typically
be part of an environment comprising hundreds of students, courses and rules,
where establishing course dependencies may require long chains of inference,
and where rules may have exceptions. In this broader context, an automated
decision maker may be not just useful, but necessary.
For which? In the same setting, determine what values make a given objective
realizable. Continuing with this example, when asked the question
What courses does Sue need to take before sitting in ‘Advanced Networking’?

an automated reasoner shall answer ‘Computer Architecture’ and ‘Basic

Networking’.
This form of reasoning, in which logical inference does not just decide
on an objective but computes the values that realize it, is one of the most
powerful forms of deductive assistance a human decision-maker can hope for,
and therefore no automated decision tool can be complete, or even practical,
without it. Our design for a deductive spreadsheet will definitely include it.
It shall be noted that the yes/no question above is a special case of this reasoning
pattern in which we do not ask for any value to be returned. Moreover, this
flexible schema specializes to many advanced forms of logical inference such
as hypothetical reasoning (“what if” analysis), reasoning in changing situations,
and transformations of symbolic data from one form to another.
4 1 Introduction

Why? Finally, a user may not be satisfied with just an answer, but may want an
explanation of how that answer was reached:
Here Sue is asking Why do I need to take ‘Computer Architecture’ and ‘Basic
Networking’ before sitting in ‘Advanced Networking’?, and the automated reasoner will
produce the above rule.

Explanation, i.e., the ability to show a representation of the proof of a logical

statement, is seldom a prominent component of a logic-based deductive tool.
We believe however that it plays a crucial role in an end-user application such
as the deductive spreadsheet. Explanation, especially if carefully integrated in
the user interface, provides confidence in the validity of the deduction and its
results, in the same way as tracking cell references helps the user of a traditional
spreadsheet gain confidence in the correctness of the computed values. If he can
understand the logical process, a decision-maker is likely to use the answer of
the automated assistant rather than give up on it, again as in the spreadsheet
world. Moreover, user-friendly explanation is a powerful debugging tool that an
end-user will need in order to find and correct inevitable errors. Finally, this
feature is indispensable whenever a decision needs to be audited.
The design proposed in this book for a deductive spreadsheet supports all of
these forms of logical reasoning, within the bounds of computability theory. While
these bounds impose certain formats on acceptable logical statements, they do not
otherwise impact the target audience of the deductive spreadsheet.

1.2 Target Users

As mentioned above, the intended audience of the deductive spreadsheet is the

same that already uses traditional spreadsheets on a daily basis. The largest group
consists of computer users without formal training in computer science or logic.
Users comfortable with basic spreadsheet tasks such as entering formulas and using
a variety of built-in functions should have no difficulty performing simple logical
inferences in a deductive spreadsheet. Like in Excel for example, we expect a novice
user to take 10–20 hours (not doing data entry) to reach that level. As in Excel, skills
will grow with time and experimentation. An experienced or expert user will be able
to perform significantly more complex reasoning tasks.
The deductive spreadsheet will be particularly useful in all environments where
individuals are required to routinely make complex decisions. Note that we do not
envision a deductive spreadsheet as a replacement for the human decision maker,
but as a tool that provides automation in view of assisting him or her in this task.
Quoting again [Gun04]:
Such a deductive spreadsheet tool could be used by military planners to capture standard
operating procedures, rules of engagement, logistics procedures, and a wide range of
decision rules. It could be used by business managers to capture business rules, operating
procedures, and decision rules.
1.3 Drawbacks of Existing Solutions 5

Additionally, lawyers, doctors, students, etc., would use it in their daily decisions,
and so would many people for day-to-day activities outside of their profession.
We again agree with [Gun04] in that
If successfully developed, this tool could become a ubiquitous part of everyone’s computing
environment.

A second target audience consists of professional spreadsheet programmers

who develop customized layouts or functionalities, typically in industrial
settings—their spreadsheet programs often perform restricted forms of data
inference. We anticipate that this group will embrace the deductive spreadsheet,
a tool that can significantly simplify parts of their job by enabling them to express
very sophisticated data inferences easily.

1.3 Drawbacks of Existing Solutions

We now briefly review existing technologies that embody some of the functionalities
of a deductive spreadsheet, as outlined above. They each have significant inade-
quacies in view of supporting end-users with their logical decision-making needs.
However, some also embed ideas and techniques that are clearly useful in the
deductive spreadsheet. We will point them out as we go, anticipating some of the
aspects of the design in Sect. 1.4.

1.3.1 Spreadsheets

To most people, a spreadsheet is a collection of cells which can contain either

a value (e.g., “42”, “12.99”, “Total”, or the default blank value) or a formula
(e.g., “(A3 – 32) * 5/9”). From a programming language point of view, a spreadsheet
is just a simple functional language without recursion—its expressive power is very
limited. Spreadsheet applications provide three core functionalities:
Evaluation: When loading a spreadsheet file, a spreadsheet application computes
the formula contained in the cells and displays the corresponding value.
This yields the evaluated spreadsheet displayed on the user’s screen.
Update: Once a spreadsheet is loaded (and evaluated), every time the user
modifies the contents of a cell, changes are propagated to all parts of the
spreadsheet that depend on this location and the value of the affected cells is
redisplayed.
Explanation: The ability to answer questions such as “Why does this cell
report this value?” is an essential functionality as it has been shown that the
vast majority of spreadsheets are riddled with errors [EB02, Pan98, BG87].
Commercial spreadsheets support explanation by offering tools that visualize
dependencies among cells in an evaluated spreadsheet, although new approaches
have been proposed (e.g., [CC00]).
6 1 Introduction

These functionalities and the very constituents of the traditional spreadsheet will be
examined in much greater detail in Chap. 3.
Formulas can also contain cell ranges (also known as arrays) such as “A2:A31”
in “SUM(A2:A31)”. Most commercial spreadsheets also provide a little-known
mechanism by which a formula can evaluate to an array, i.e., produce a value not just
for a single cell, but for multiple cells at once. These array formulas reside in a cell
range: groups of cells can indeed hold a formula. From a programming language
perspective, arrays introduce convenience, but no extra expressive power. Arrays
and array formulas will be examined in more detail in Chap. 4.
The traditional spreadsheet was born as an electronic incarnation of accounting
books, and as such it provides outstanding support for numerical calculation.
However, the closest it has to offer to logical reasoning are the common Boolean
operators AND, OR, NOT, which are intended to be used in the conditions of IF
statements (and even there, their prenex syntax makes for unnecessarily complex
expressions). One can hardly perform any serious reasoning with those.
Because its formulas are drawn from a simple functional language without
recursion, the traditional spreadsheet does not have the computational power to
support the forms of reasoning we are interested in. Note that this limitation is also
what makes spreadsheet calculations always terminate. As we extend the traditional
spreadsheet with support for symbolic reasoning, we shall strive to maintain this
characteristic.
Finally, a formula in a traditional spreadsheet computes a value for the cell it
resides in. We will see that in the case of the deductive spreadsheet, it is more
convenient to associate a logical statement with one or more columns, in general.
In summary, there are a number of reasons why traditional spreadsheets cannot
adequately support the kind of symbolic reasoning envisioned here. On the other
hand, many of the interface design decisions, the powerful support for numerical
calculation, and guaranteed termination are all desirable features for the deductive
spreadsheet.

1.3.2 Logic Programming

Logic programming uses a fragment of first-order logic to write computer programs.

The best known instance is the programming language Prolog [SS94].
At first sight, logic programming seems perfectly adequate for the task at hand,
at least if we initially ignore user interface issues. Sue’s example can be expressed
in many ways in Prolog. Here is one of them that will turn useful in what follows:
student(sue).
course(os). course(tc). course(ca). course(bn).

hasTaken(sue, os). hasTaken(sue, tc).

requires(an, os). requires(an, ca). requires(an, bn).

1.3 Drawbacks of Existing Solutions 7

canTake(S, C) :- student(S), course(C), course (RC),

not missingReq(S, C, RC).

missingReq(S, C, RC) :- student(S), requires(C, RC),

not hasTaken(S, RC).
The first two lines list the students and the courses (abbreviated) we are reasoning
about—there may be many more of each. The third line represents the courses that
Sue has taken, and the fourth describes course prerequisites. The last two lines
are inference rules that describe when a generic student S can take a course C :
IF (written :-) S is not missing any course RC required by C . The last line defines
missingReq (the comma is read AND). Another two lines would be needed to
account for transitive course requirements.
The query ?-canTake(sue,an) will produce the expected negative answer
(No in Prolog), while the query ?-missingReq(sue,an,RC) will return two
values for the variable RC : ca and bn, again as expected.
Now, in spite of this success, there are a number of reasons why Prolog is an
unlikely foundation for the deductive spreadsheet, even if we were to devise a nice
graphical user interface for it:
• Unless carefully coded by an experience programmer, Prolog specifications are
extremely prone to non-termination. Something as simple as issuing a query with
a variable instead of a constant can cause an infinite loop. Writing the conditions
of an inference rule in the “wrong” order can have the same effect (this is as if
the spreadsheet formula A1 C A2 computed 5, but A2 C A1 hung the computer).
This is clearly undesirable in the hands of an end-user: absurdly unpredictable
behaviors of this kind will cause frustration, leading him to give up before he has
acquired the skills needed to avoid these pitfalls.
• Even if these problems were fixed (and there have been attempts),
non-termination and slow-termination would still be an issue as Prolog has
very high computational power. Differently from a spreadsheet, it would be
impossible to guarantee that a task written in full Prolog will terminate, or
terminate in a timely manner.
• Finally, the operational model of Prolog does not blend well into a
spreadsheet-like application. Prolog operates top-down, that is it starts from
a user query and looks at the inference rules that may have entailed it. It then
tries to prove each of their conditions by looking at the rules that may have
entailed them, and so on until all these subgoals have been satisfactorily proved.
The variable instantiation is reported to the user as a final value. If the user wants
another value, he shall ask Prolog to search for another proof, and so on until all
different ways to solve the query have been examined.
Top-down evaluation works wonders when a query yields only one value
(or only the first value is of interest). In our view, the deductive spreadsheet
should always report all the values that satisfy a query, as in the example above.
Then, top-down evaluation is less than ideal as, unless tamed by an expert
programmer, it is prone to infinite loops and duplicated answers.
: : : and Prolog user interfaces do not look a bit like Excel.
8 1 Introduction

In summary, Prolog embeds the expressive power we would consider for the
deductive spreadsheet, but it requires too much sophistication for a user to make
effective use of it, and its operational model is not fully compatible with a
spreadsheet environment. Making some top-down reasoning available as an option
for the advanced user is a possibility that requires further evaluation.
However, there is more to logic programming than just Prolog. In this book, we
will pay particular attention to a syntactic fragment of Prolog called Datalog [Ull88].
The example at the beginning of this section is actually written in this language.
In a way, Datalog turns Prolog on its head. Its evaluation strategy is bottom-up:
it starts with the true facts of a logical specification (the relations student,
course, hasTaken and requires in the above example) and uses them to
compute all their immediate consequences according to the given inference rules
(the relation missingReq here). It then uses these derived facts and the inference
rules to compute a second round of consequences (here canTake), and then a
third, and so on until all the logical consequences of the specification have been
discovered. The syntactic restrictions ensure termination, even in the presence of
recursive rules.
Bottom-up evaluation is a good fit for the deductive spreadsheet for several
reasons:
• Termination is guaranteed. This is a desirable property that blends well into the
spreadsheet model.
• It reports all the solutions to a query, without duplicates. It will be convenient
to display them in as many spreadsheet columns as there are variables that
participate in the query.
• Because evaluation always starts from the true facts and not from the query,
Datalog supports evaluating multiple queries at once. Bottom-up evaluation
will compute the answers to all of them simultaneously. This aspect is again
reminiscent of a spreadsheet, where, in general, numerous cells contain formulas
that will be evaluated simultaneously.
• A change in any part of the specification, either a fact or a rule, will cause
immediate re-evaluation and update of all the answers.
For these reasons, we will embrace the Datalog model, bottom-up evaluation in
particular, as one of the pillars of the deductive spreadsheet. An appropriate flavor
of Datalog (indeed, there is a whole family of Datalog languages [RU95]) for this
purpose is investigated in this book.
Datalog has been extensively studied as a foundation for deductive databases
(see next section) and is nowadays used in a broad range of applications [LM03,
GW10, LCGC 06, ARLGC 09, CARGC 12]. We are unaware of any commercial
implementation outside of this context. However, several industrial-strength deduc-
tive databases are available and will contribute to some aspects of the design of the
deductive spreadsheet.
1.3 Drawbacks of Existing Solutions 9

1.3.3 Databases

If we ignore for a moment the possibility of recursive rules, the Datalog specification
we have just described is a very simple relational database [Cod70]: the base facts
(student, course, hasTaken and requires) are tables, with one record for
each instance of the relation; the rules implement derived tables (views), or just
queries if one is interested in evaluating them only once.
The tables for our example are:

student course hasTaken requires

Name Initial Student Course Course Required
sue os sue os an os
ca sue tc an ca
bn an bn
an

The second line gives a name (starting with an uppercase) to each attribute. It can
be observed that this tabular visualization is compatible with the layout of a typical
spreadsheet.
The view corresponding to the rule defining missingReq is given by the
following SQL statement:
CREATE VIEW missingReq(Student,Course,Required) AS
SELECT *
FROM student, requires
WHERE NOT EXISTS (SELECT *
FROM hasTaken
WHERE student.Name = hasTaken.Student
AND requires.Required = hasTaken.Course)

The statement for canTake is similar. Their values could be visualized in a tabular
form, which brings us even closer to the spreadsheet world. We may not expect an
end-user to enter, or even understand, such expressions. However, some database
management systems such as Microsoft Access provide intuitive graphical methods
to construct simple views and queries.
A deductive database extends a relational model with the possibility of defining
views recursively, which corresponds to admitting the recursive inference rules of
Datalog [RU95, Zan99]. Recursion is now part of SQL, the standard database query
language.
There is clearly much inspiration that the design of the deductive spreadsheet can
take from databases, and in particular deductive databases. It would be incorrect,
however, to think that deductive databases already implement all the requirements
of the deductive spreadsheet. Indeed, a database management system is primarily
meant to provide efficient access to large amounts of data that necessarily reside
on secondary storage. This focus forces a certain mind-set on the user of a
10 1 Introduction

database, who must think in terms of keys, indices and relationships: fine-tuning
a database is not trivial. This also imposes limitations on the database management
software itself: for example it is not realistic to update views in real-time in the same
way as changing the value of a cell propagates instantly in a spreadsheet.
In summary, operating a database requires a background and a level of sophistica-
tion that cannot be expected from the target user of the deductive spreadsheet. Some
implementations, such as Microsoft Access, do provide a graphical interface that
simplifies managing and querying the database. This is only a small step toward the
occasional end-user as menus and dialogs still refer to complex database concepts.
On the positive side, deductive databases embed techniques that can be used as a
foundation for the deductive spreadsheet. Major re-engineering is however required
to eliminate unnecessary database-specific features and to implement functionalities
expected of a spreadsheet.

1.3.4 Task-Specific Tools

Finally, there is a vast number of tools specialized to reasoning in certain domains

and of certain forms: work-flow programs, theorem provers, expert systems, etc.
These tools lack generality for our purposes, and often require considerable user
sophistication. We will not discuss them further.

1.4 Approach

We propose an architecture for the deductive spreadsheet as a powerful yet

unintimidating aid to complex decision making for end-users without formal logic
or computer science training. In spite of this clear potential, we are doubtful about
its acceptance and ultimate success as a stand-alone application: entry-level users
would not grasp the difference with a traditional spreadsheet (in the same way as
few end-users know what to do with Microsoft Access within the Office suite)
while more advanced users will find the separation stands in the way of productivity.
Instead, we believe that the deductive paradigm needs to be seamlessly integrated
into traditional spreadsheets as a key to user acceptance. Only in this way will novice
users be able to “discover” these new logical extensions when they need them,
and only in this way will power-users be able to combine the traditional functional
paradigm and the deductive support to construct fast and elegant solutions to their
problems.
Therefore, this book designs the deductive spreadsheet as an extension of a
traditional spreadsheet such as Microsoft Excel, Google Spreadsheet, or OpenOffice
and LibreOffice Calc with powerful deductive capabilities that can assist an end-user
with the process of making decisions and manipulating both symbolic and numerical
data.
1.4 Approach 11

• Internally, this extension takes the form of a logic programming language with a
bottom-up evaluation strategy. The deductive and traditional functionalities are
smoothly integrated, so that logical reasoning can apply to cells with calculated
values, and formulas can refer to cells produced by logical inference.
• Externally, an end-user sees an interface nearly identical to that of a tradi-
tional spreadsheet. The proposed extension is conservative, unobtrusive and yet
accessible. Indeed, a user who wants to use the tool as a traditional spreadsheet
is not penalized by the added deductive component. On the other hand, using the
logical extension is natural, even in conjunction with traditional functionalities.
We now refine this preview by anticipating salient aspects of the architecture for the
deductive spreadsheet as developed in the rest of this book.

1.4.1 The Deductive Engine

In Sect. 1.3, we have suggested designing the deductive spreadsheet around a logic
programming environment, and more precisely using a Datalog-like language with
bottom-up evaluation. We now further elaborate on the exact choice of the language
and its execution model.

Language

In its most basic form, Datalog is a very simple fragment of first-order logic
where:
• All values are atomic symbols (e.g., sue and 12 as opposed to sister(sue)
or 12+5).2
• True facts are relations among atomic symbols (e.g., hasTaken(sue,os)).
• Rules have the form “IF a1 AND . . . AND an THEN b”, which states that b can
be deduced to be true whenever all the ai have been shown to be true. Rules can
mention variables for atomic symbols and can be recursive.
Evaluating pure Datalog always terminates, which is why this simple language
played such a prominent role in the research on deductive databases. However, pure
Datalog is so minimal that many common reasoning tasks take unintuitive forms.
For this reason, several extensions have been proposed [RU95]:

2
The language underlying Prolog differs from Datalog by removing all restrictions on the
structure of values: this enables very powerful forms of reasoning, but also the possibility of
non-termination.
12 1 Introduction

• Allow a restricted but natural form of negation in the IF part of the rules. We have
already relied on this in our example.
• Allow values to be not only atomic symbols but restricted sets of atomic symbols
(aggregates).
• Allow arithmetic and other functional operations to appear in a rule.
The first two extensions maintain termination, even when combined. The third does
not in general, but useful terminating classes have been isolated. Altogether, these
extensions are far from the deductive power of full first-order logic, but they add
natural support for several useful forms of reasoning.
In the context of the deductive spreadsheet, guaranteed termination of the
reasoning process is a necessary feature. Indeed, this behavior is consistent with the
calculation of formulas in a traditional spreadsheet, which is very important from
a cognitive point of view. Expert Prolog programmers know how to debug infinite
loops, but end-users experience non-termination only when an application hangs,
an event often followed by a reboot. For this reason, we believe that a (terminating)
idiom of Datalog should constitute the foundation of a basic interaction with the
deductive spreadsheet. Which idiom of Datalog?
We made use of negation in Sue’s example. Adopting a flavor of Datalog
that admits negation (and possibly aggregation) is technically adequate, but can
be cognitively taxing. Going back to our example, navigating through various
negations is unnatural and can be intellectually challenging for users unfamiliar with
logical reasoning. For this reason, we also look at alternative linguistic extensions
to Datalog that support more direct ways of expressing recurrent rules as in
this example. We take inspiration to some recent progress in the theory of logic
programming for this. A key factor in making this choice is the need for a transparent
and intuitive support for explanation.
Embedding numerical formulas within rules may be seen as functionally
redundant since such calculations can always be performed using the traditional
functionalities of a spreadsheet. We believe however that it is cognitively useful as
it allows a closer mapping between the deductive intent of a user and the language
of inference rules.
The benign coexistence and even integration of the traditional formula-based
and the new deduction-based languages in the same application is achieved by
carefully designing the logical extension of the spreadsheet. In our approach, this
is a special case of the problem of combining the paradigms of logic programming
and functional computation, which is the subject of a research area known as
functional-logic programming [Han94].

Execution Model

As mentioned earlier, we favor bottom-up computation: it always terminates with

Datalog programs, it computes all solutions at once, it does not return duplicate
answers, and it supports simultaneous queries. Moreover, efficient implementation
1.4 Approach 13

Fig. 1.1 Example of base relations

techniques have been devised, some of which support incremental computation,

an essential feature when deduced values need to be recomputed on the fly in
response to a change in the input [RSS92]. All these reasons make bottom-up
strategies particularly appealing in the context of the deductive spreadsheet.
We adapt existing algorithms to the demands of the deductive context.

1.4.2 The User Interface

As we design the deductive spreadsheet, we must never forget that the purpose
of this technology is to assist an end-user with limited computer science and
logic training in his daily reasoning tasks. Therefore, translating these sophisticated
reasoning mechanisms into an intuitive and usable interface is paramount.
We argued above in favor of a conservative and unobtrusive extension of the
excellent interface of a traditional spreadsheet. Moreover, the added functionalities
shall be consistent with existing modalities, and accessing them shall be natural.
The major challenge is to devise elegant graphical modalities for the end-user
to comfortably define and review inference rules. Another important aspect is to
make the integration between the traditional and the deductive components of the
application as seamless as possible.
Figure 1.1 shows the base relations for an extension of Sue’s example as they
appear to a user of the deductive spreadsheet.3 Figure 1.2 simulates instead the
values resulting from executing the shown inferences in a bottom-up fashion.4 Here,
we show relations side by side, but this is not a constraint: they may reside anywhere
on the spreadsheet, even on different sheets. Requiring a relation to consist of
adjacent columns is not necessary either, although probably easier for a user to
work with.

3
This mockup screenshot was produced by inputting the displayed values in Microsoft Excel 2010.
We will base illustrations throughout the book on this and other versions of Excel, and occasionally
of other spreadsheet applications.
4
The inference rules are displayed using Prolog syntax as Excel comments. In Chap. 9, we will
develop a more polished user interface and a concrete syntax for the deductive spreadsheet.
14 1 Introduction

Fig. 1.2 Example of deduced relations

As we design the user interface for the deductive spreadsheet, we follow tested
methodologies to assess proposed solutions against the cognitive requirements
of the user. Techniques stemming from the theory of cognitive dimensions of
notations [GP96] and the attention investment model [Bla02] have been applied with
great success to evaluate the traditional spreadsheet interface and design extensions
of this paradigm [BADC 01, PBB03].
While we propose a design for a user interface of the deductive spreadsheet,
rigorous experimental validation is delegated to future work.

1.5 A More Elaborate Example

In this section, we give a brief tour of the deductive spreadsheet and the kind of
problems it can solve by means of a small case study. Chapter 6 will expand on both
aspects. This tour will also give a preview of the Datalog-inspired concrete syntax
we will present in Chap. 9.
Consider the following problem: an investor is trying to analyze his portfolio
and plan a new round of investments. He sets up a spreadsheet where he lists each
instrument he owns (stock, bonds, mutual fund, etc.) and how much each is worth
in columns A and B (several Internet tools make it easy to track this data and
even to download it to a spreadsheet). This tabular data is naturally interpreted as
a relation—call it portfolio (commercial spreadsheets allow giving mnemonic
names to cells, ranges and other spreadsheet entities). He also downloads informa-
tion about immediate instruments, specifically their name, type (stocks or bonds)
and sector in columns D, E, F , calling the resulting table immediate. He does
1.5 A More Elaborate Example 15

Fig. 1.3 The investor’s spreadsheet

the same for his mutual funds, for which he records the fund name, its holdings and
the fund’s percentage of each holding in columns H , I , J , calling this third table
funds. The left part of Fig. 1.3 displays how sample data for this example could
appear in an Excel-inspired rendering of the deductive spreadsheet.
One simple thing the investor may want to know is which of his stocks are in
health care companies and how much they are worth, for example. This information
is easily retrievable from his data, yet no commercial spreadsheet supports such a
simple operation. Had he loaded his data in a database instead, a simple relational
query would return the answer, but setting up and querying a database is not as easy
as using a spreadsheet. The deductive spreadsheet gives him the means to compute
this answer directly in his spreadsheet: one way to do so is to select a two-columns
data range, say L3WM11—let us bind it to the mnemonic name healthStocks,
and enter the following Datalog clause (the syntax is an adaptation of Datalog
discussed in Chap. 9).
healthStocks(N,W) IF portfolio(N,W)
AND immediate(N,"stock","health").

Here, portfolio(N,W) denotes an arbitrary record in the relation

portfolio (recall this is just a mnemonic abbreviation for a cell range such
as A3WB30), referring to the name part of the record as N and to the value as W .
Similarly, immediate(N,stock,health) identifies a record in the relation
immediate with name N , type stock and sector health. The overall clause
can be read as follows: a record .N; W / is in healthStocks if .N; W / is in
portfolio and .N; stock; health/ is in immediate, for the same name N .
The records determined in this way are copied to the range healthStocks.
16 1 Introduction

Now, how much his health stocks are worth can be computed with the standard
SUM function on the right column of this range—the deductive spreadsheet blends
seamlessly the new logical formulas and the traditional spreadsheet formulas.
The right portion of Fig. 1.3 shows a crude simulation—a more sophisticated
interface is examined in Chap. 9.
A more interesting exercise for this investor is to pick apart his mutual funds and
list all industry sectors he carries in his portfolio. Now, many mutual funds hold just
stocks and bonds as instruments, but some hold shares of other mutual funds. In the
deductive spreadsheet, it is easy to write recursive Datalog clauses that unravel all
funds to their immediate holdings:
fundSector(F,S) IF fund(F,N,_)
AND immediate(N,_,S).
fundSector(F,S) IF fund(F,F’,_)
AND fundSector(F’,S).
Read aloud, fundSector contains the record .F; S / if fund contains .F; N; /
for some N and immediate contains .N; ; S / for the same N , or if fund
contains .F; F 0 ; / and fundSector contains .F 0 ; S / recursively. Here, we have
written “ ” for a field whose value is irrelevant and can therefore be arbitrary in a
record. Situations such as the above, where an entity is made of parts, those parts of
subparts, and so on up to elementary constituents is very common in a number of
industries, from banking to manufacturing, and unraveling all basic constituents is
referred to as the bill of materials problem (see also Chap. 6). Now, the investor can
find his own exposure by sector by selecting just the instruments in his portfolio,
which is achieved through the following clause:
sectorExposure(F,S) IF portfolio(F,_)
AND fundSector(F,S).
It is tempting to modify the above clauses to compute not just the sectors represented
in each fund, but also the percentage of each sector in the fund:
sectPc(F,S,P) IF fund(F,N,P)
AND immediate(N,_,S).
sectPc(F,S,P) IF fund(F,F’,P’)
AND sectPc(F’,S,P’’)
AND P = P’*P’’.
However, by doing so, we have just escaped Datalog, which does not allow
calculated values (here P = P’*P’’) in deduced relations because they under-
mine its most appealing property: necessary termination. The deductive spreadsheet
partially lifts this restriction of Datalog, thereby allowing the above clause, by stop-
ping the recursion after a user-defined bound. To find out the percentage of
his portfolio invested in each sector, the investor would modify the clause for
1.6 Related Work 17

sectorExposure as follows (total is a mnemonic name for the cell containing

the total of his investment):
sectorPercent(F,S,P) IF portfolio(F,W)
AND sectPc(F,S,P’)
AND P=P’*W/total.
and then add the percentages in each sector using an aggregating variant of the SUM
function.
This relational infrastructure also allows for easy planning using what-if
scenarios. For example, this investor can anticipate the effect of buying a new
fund by just adding a candidate entry in the portfolio table and watching how
the change propagates to all the calculated relations. He can experiment with various
ways of rebalancing his portfolio by just changing the amount he has invested in
each fund: this allows him to make informed decisions before making any actual
change to his positions. He may also want to scrutinize some new fund he has
never used before, for example by making sure that this fund does not hold any
of its own shares, directly or indirectly (this would be troubling, to say the least).
The following clause relates each fund to each of its holdings:
fundInFund(F,F’) IF fund(F,F’,_).
fundInFund(F,F’’) IF fundInFund(F,F’)
AND fund(F’,F’’).
These clauses compute the transitive closure of the relation fund. It is another
classic problem that deductive databases can handle but relational databases cannot,
and that is totally out of reach of today’s spreadsheets (more on transitive closures
in Chap. 6). Now funds that hold their own shares are identified as follows:
suspiciousFund(F) IF fundInFund(F,F’)
AND F=F’.
Needless to say, the calculations in this example are simplistic by the standards of
any investor, but they can easily be developed into more useful forms of analysis by
uploading richer data and by expressing more sophisticated inferences: the point is
that none of these calculations is expressible in a traditional spreadsheet, while the
deductive spreadsheet can very easily accommodate them.

1.6 Related Work

In this section, we outline recent and not-so-recent work related to the concept of
a deductive spreadsheet proposed in this book. We then quickly examine trends in
areas that feed into our design, specifically spreadsheets, logic programming and
databases.
18 1 Introduction

1.6.1 Logics and Spreadsheets

We begin by reviewing some of the early attempts at integrating reasoning

capabilities into spreadsheets, which interestingly took place as the very idea of
a spreadsheet was being shaped. As spreadsheets were taking off with consumers,
this early interest waned however, which resulted in over a decade of limited
activity. With the beginning of the current millennium, logic-based spreadsheets
have however attracted renewed interest as a number researchers have been busy
exploring a variety of approaches to enhancing traditional spreadsheet applications
with forms of logical reasoning.

Early Attempts

The idea of combining logic programming and spreadsheets started circulating in

the early 1980s, at a time when both technologies were relatively new, and the
personal computer had just begun trickling off the production lines. The first and
maybe most comprehensive early proposal was put forth by Frank Kriwaczek in
his 1982 Master’s thesis [Kri82], a revised version of which was later published
for a wider audience as [Kri88]. The LogiCalc system he describes was meant
to “reconstruct logically, in micro-Prolog, a typical spreadsheet program”. It not
only captured most of the functionalities of early spreadsheets, but also provided
powerful extensions such as relational views, integrity constraints, bidirectional
variables, symbolic manipulations, and complex objects (e.g., lists, graphics, and
spreadsheets themselves) inside cells. The input and output was based on the limited
teletype interface of early computers as modern graphical user interfaces had not
been invented at the time.
A few years later, Marteen van Emden proposed using the concept of incremental
query, by which a standard Prolog query can be refined interactively, for solving
spreadsheet-like problems [vEOT86]. The proof-of-concept Prolog implementation
realized core spreadsheet functionalities as well as exploratory logic programming,
but made use of the (still non-graphical) matrix display of the spreadsheet only
for output, one solution at a time. This idea was later refined [CvEL88] to report
all solutions to a Prolog query in a tabular fashion using a windowed graphical
interface.
The last attempt of the decade at integrating logic and spreadsheets was Michael
Spenke and Christian Beilken’s PERPLEX system [SB89], which relied on logical
formulas to express bidirectional integrity constraints within a fully graphical
spreadsheet. The resulting system was very powerful but unlikely to be usable to
an entry level user. Bidirectional integrity constraints are interdependencies between
two or more cells so that entering a value in any of them computes a result for all the
others, a useful feature that has attracted renewed interest recently [GA00, KG07],
but that is orthogonal to the extension considered in this book.
1.6 Related Work 19

Surprisingly, this thread of research dried out in the late 1980s in spite of the
gigantic advances in user interfaces and the coming of age of logic programming in
the 1990s. The lone exception is Knowledgesheet [GA00], a system that explored
an alternative way of embedding logical constraints within a spreadsheet.

Recent Developments

With 15 years of intervening research and the availability of powerful yet

inexpensive hardware, there has been a recent resurgence of interest in revisiting the
intuitive idea that users can benefit from carefully integrating symbolic reasoning
support into spreadsheet applications. This is an indication of the growing impor-
tance of the underlying problem.5 Besides the work on bidirectional constraints just
mentioned [GA00, KG07], several approaches for doing so have emerged. We will
examine three of them and relate them to our own design. In all cases, the work
has focused on the functional aspects of extending the spreadsheet paradigm, with
limited effort spent on the cognitive aspects of the parallel user interface extension.
A distinguishing aspect of our design is its emphasis on the user interface, usability
and user testing, which occupy the entire second half of this book.
The LESS system [VvBCE07] integrates PowerLoom [Mac94], a highly
sophisticated logic-based knowledge representation and reasoning engine, within
Microsoft Excel. Through PowerLoom, LESS supports reasoning with arbitrary
first-order logic formulas, subsumption, classifications, higher-order constructs and
much more. Well aware that this expressive power may overwhelm all but the
most experienced users, the designers of LESS have layered its architecture into
four “tiers”, from tier 1 which provides full programmatic access to PowerLoom
within Excel formulas to tier 4 which offers prepackaged templates for common
functionalities. A partial prototype of LESS was implemented as an Excel add-on
which delegates knowledge representation and reasoning tasks to a PowerLoom
library [VvBCE07]. LESS is a lot more expressive than the extensions presented
in this book, which only supports a terminating subset of logic programming.
This expressive power comes however at a high cognitive cost for the majority of
users, a cost that the layering into tiers mitigates only partially: users not only need
to be familiar with logic, especially if they need to use lower-tier functionalities,
but some aspects of LESS deviate from the common way spreadsheets are used
(e.g., entering rules separately from the cells they calculate). By contrast, our
design pays extreme attention to retaining the cognitive simplicity of spreadsheets
while making useful deductive capabilities available to the users.

5
This interest was evident in the Workshop on Logical Spreadsheets (WOLS’05) held at Stanford
University in the summer of 2005. The presented systems fell into two clear classes: proposals
which allowed the Prolog-style computation of new values from existing values, and systems which
provided support for bidirectional constraints. The design discussed in this book falls into the first
class.
20 1 Introduction

The unnamed system in [TB08, TWB07] extends Excel with mechanisms to

flag a range or a cell as part of a relation. These relations are automatically
copied into a deductive logic engine implementing the OWL C SWRL ontology
language [HPSBC 04], a World Wide Web Consortium standard. The extension also
contains mechanisms to populate a range with values deduced by evaluating logic
rules akin to our use of clauses (recursion does not seem to be supported however).
These input and output relations can feed into each other and interact freely
with traditional spreadsheet formulas. Logical formulas are entered in separate
“property definition worksheets” using a format reminiscent of Excel’s data list
filters (see Sect. 4.2). The expressive power of this proposal appears similar to ours,
although the apparent lack of recursion limits the class of representable problems
(e.g., the relation coursesForSkill illustrated in Fig. 1.2 cannot be expressed).
The authors clearly dedicated a lot of thought to usability issues and the resulting
systems seem to be more than adequate for what appears to be its target audience,
knowledge engineers and people working with ontologies (accessing large external
ontologies is supported, for example). By contrast, our proposal is geared toward
a less specialized population of spreadsheet users and therefore it is engineered to
have an even lower cognitive footprint (for example, clauses are entered in the cells
that will display the result of their evaluation, just like current arithmetic formulas).
The system closest to ours is XcelLog [RRW06]. It too extends the spreadsheet
paradigm with Datalog clauses and has therefore the exact same expressive power.
The way this extension is provided to the user is however very different: relations
appear to live in dedicated worksheets, with the first column holding the unique
“key” of each record (the rows) and the following columns corresponding to
attributes. Unusually, the cell at row r and column c contains the set of all values
for that key and this attribute: a cell can indeed contain multiple values, which is
a significant departure from traditional spreadsheets. By contrast, a relation in our
proposed extension is a range anywhere in a spreadsheet and, as usual, at most one
value appears in each cell. A consequence of this design is that XcelLog clauses are
entered in individual cells and, most remarkably, without the user having to name
variables, at least in the simplest cases. We are not aware of any usability study for
this system. An XcelLog prototype [RRW07] has been implemented as an Excel
add-on that uses XSB [War98], a high-performance memoizing logic programming
system, as its deductive engine.

1.6.2 Beyond Logical Spreadsheets

We conclude this section on related work with a brief review of recent developments
in three areas that are most relevant to our work: spreadsheets, logic programming,
and (deductive) databases.
1.6 Related Work 21

Spreadsheets

The spreadsheet was imagined in the early 1960s and first implemented in the late
1970s. It acquired popularity in the 1980s with the personal computer [Pow03] and
for a while it was the most commonly used application on a PC (only recently
surpassed by web browsers and email clients). It is still an indispensable tool for
the vast majority of the users of any computer system [BABC 00]. Recent releases
of commercial spreadsheets have however included relatively little innovation,
which can be seen as the sign of a mature technology. For example, the main
differences between Microsoft Excel 2003 and the previous edition was an improved
coordination within the Office suite, the addition of a few statistical functions
and, sadly, the fix of a large number of security holes. The main updates in the
next version, Excel 2007, were the adoption of an XML-based file format and the
replacement of the formula bar with a context-sensitive “ribbon”, a rather disruptive
change to a user interface that many users had become quite comfortable with.
Around the same time, Google introduced Google Docs, a cloud-based office suite
accessible through any browser. Multiple users could collaborate on a document,
in particular a spreadsheet, from any Internet-enabled computer and in real time,
without having to ever bother emailing files or with software updates. Microsoft
incorporated some of this technology in its latest version, Excel 2010, with further
cloud-integration announced for the next edition, Excel 2013.
The possibility of extending Excel and other spreadsheet applications by means
of custom-made modules called add-ons has fueled a cottage industry of sorts and
numerous extensions are now available, mainly new mathematical functions and
formatting utilities. We are not aware of any such add-on providing the extended
deductive functionalities discussed in this book.
In the shadow of this limited commercial innovation, a lively research community
is busy analyzing the way people use and misuse spreadsheets, and suggesting
improvements (see http://www.eusprig.org for the latest developments). Academic
research has mainly concentrated on measuring and improving the productivity
and usability of spreadsheets as a human-computer interaction problem. A par-
ticular focus has been the high error rate found in spreadsheets [Pan98] and
countermeasures such as auditing [Saj00].
A few researchers have investigated functional extensions to the spreadsheet
paradigm. The inclusion of user-defined functions (as opposed to programmer-coded
add-ons) is studied in great detail in [PBB03]. Other extensions, such as recursion,
floating cells and graphical values are included in the Forms/3 language [BADC 01].

Logic Programming

The main logic programming ideas underlying the proposed design for the deductive
spreadsheet took shape in the golden years of logic programming, from the late
1970s to the mid-1990s. That era brought us the theoretical research underlying
22 1 Introduction

bottom-up evaluation and Datalog, and their practical realization in the deduc-
tive database [RU95, RSS92]. Prolog, top-down evaluation and negation received
considerable attention during this period [SS94].
To a large extent, recent research has focused on broadening the definition of
logic programming [Han94, MNPS91] and then identifying expressive computable
fragments of logics different from the traditional first-order logic of Prolog and
Datalog [CP02,WCPW03]. The results of this research have just started percolating
into the Datalog model [BDM01] and may be useful in future refinements of the
deductive spreadsheet.
In the last 10 years, Datalog has had a resurgence in a variety of domains other
than databases. Indeed, it has been used to specify and enforce fine-grained user
authentication [LM03], to implement network protocols [GW10, LCGC 06], and to
program cyber-physical systems [ARLGC 09], just to cite a few. Differently from
the relatively static databases of the 1980s, these applications are characterized by a
highly dynamic fact base, which requires efficient algorithms to propagate updates
resulting from the assertion of new facts and the retraction of stale facts [ARLGC 09,
CARGC 12, GMS93, LCGC 06] and a logical understanding of the principles they
espouse [LC12, NJLS11].

Deductive Databases

Defined around 1980, Datalog and the idea of a deductive database were for a
long time little more than an academic curiosity [RU95, Zan99]. The development
of industrial-strength prototypes such as CORAL [RSSS94] and LDL [AOTC 03]
turned them into a mainstream technology. Recursive views, the trademark of
deductive databases, have been incorporated into the SQL3 standard, and later into
SQL4.

1.7 Work Plan

This book explores a design for the innovative concept of a deductive spreadsheet.
Chapter 2 lays out the requirements for both the underlying deductive apparatus and
the supporting user interface. Each of these two aspects is analyzed in a dedicated
group of chapters which occupy much of this book.
Part I gradually introduces the logical language underlying the deductive spread-
sheet. It starts with a model of the traditional spreadsheet and isolates the closest
these applications have to offer to symbolic reasoning. It then introduces an
extension that includes relational algebra as a bridge to the properly deductive
extension. Pure Datalog and its execution model are then integrated into the
spreadsheet model. The remaining chapters of this part explore useful extensions to
Datalog in the context of a spreadsheet: traditional operations within rules, negation,
1.7 Work Plan 23

aggregation, and native support for hypothetical reasoning. This part of the book is
mathematical in nature and brings together a number of key developments that have
taken place in the last 50 years in various areas of logic and of computer science.
Part II analyzes the alterations of the user interface of a spreadsheet to
accommodate our deductive extension. We begin with a review of the salient
aspects of the user interface of a traditional spreadsheet and immediately propose
an incremental design supporting the proposed deductive functionalities. We then
recall established cognitive methodologies for evaluating interfaces and apply them
in support of our design. We conclude this part with preliminary experiments
aimed at evaluating the reaction of a small group of users to a simulation of our
proposed interface design. This second part of the book is rooted in cognitive
psychology, and is given a less formal treatment than the first. It brings together
several threads of research in the areas of human-computer interaction, interface
design, and evaluation.
The concluding Chap. 11 outlines future development in each of these aspects
and discusses approaches to implementing an application based on this work. It is
followed by an annotated bibliography.
Chapter 2
Requirements

In this chapter, we state the principal requirements for the deductive spreadsheet.
We separate them into functional, regarding the operations supported by the tool,
and cognitive, concerning the user interactions with the tool. We will rely on these
requirements to guide the design of the deductive spreadsheet. They will also be
key in a future implementation of a deductive spreadsheet based on this design.
The former class will be primarily used in Part I where we design the inferential
engine of the tool. The latter will play an important role in Part II, where we extend
the interface of a traditional spreadsheet to provide access to the new deductive
facilities.
We give the functional requirements of the deductive spreadsheet in Sect. 2.1, and
present its cognitive requirements in Sect. 2.2. How to evaluate these requirements
is discussed in Sect. 2.3.

2.1 Functional Requirements

This class of requirements is concerned with the functionalities that the design
makes available. They concentrate on the supported operations and how they interact
among each other. They also assess abstract issues such as expressiveness and
computational cost. These requirements do not encompass the manner in which
functionalities are presented to the user. In particular, design decisions such as
concrete syntax and visual feedback are not discussed here, nor is a method for
evaluating them.
The principal functional requirements for the deductive spreadsheet are described
next.

I. Cervesato, The Deductive Spreadsheet, Cognitive Technologies, 25

DOI 10.1007/978-3-642-37747-1 2, © Springer-Verlag Berlin Heidelberg 2013
26 2 Requirements

F1. Functional Conservativity

Every traditional spreadsheet functionality shall still be available in the deductive

spreadsheet. In particular, each of the following notions and the associated classes
of operations shall remain supported:
1. Cells, scalar values and scalar formulas.
2. Arrays and array formulas.
3. Relations (known as data lists in Microsoft Excel, for example) and their
supporting operations, such as filtering and sorting.
4. Explanation and auditing.
The support for relations will be naturally enhanced thanks to the deductive
extension.

F2. Expressiveness

The deductive spreadsheet shall embed symbolic reasoning capabilities at least as

expressive as the Datalog fragment of first-order logic programming. In particular,
the following notions shall be supported:
1. Logical formulas in the Horn clause fragment of first-order logic with a term
language deprived of function symbols. In particular, support will be available
for recursive clauses.
2. Stratified negation in the body of a clause.
3. Interpreted comparison predicates, in particular, equality.
4. A fragment of a functional language, appropriately restricted so as not to violate
any other functional requirement.

F3. Supported Inferences

The deductive spreadsheet will support yes/no queries, queries returning values, and
explanation. Specifically:
1. The user shall be able to ask for all the values that satisfy a given logical formula.
This encompasses yes/no queries as a special case.
2. Logical inference will return all solutions to these queries without duplicates.
3. The user shall be able to ask why a certain value was returned and the system will
trace the inference back to established facts, both by unfolding the reasoning one
step at a time, and by showing all inferences at once.
4. The user shall similarly be able to ask why a certain value was not returned.
2.1 Functional Requirements 27

F4. Termination

Logical inference shall produce all results in a finite time. In particular:

1. The logical query language of the deductive spreadsheet shall be decidable.
Datalog satisfies this property. The proposed extensions of Datalog shall retain
it.
2. The decision procedure implementing the reasoning capabilities shall not allow
infinite loops. Bottom-up evaluation achieves this for Datalog, and will be
retained in the proposed extensions.
3. The decision procedure shall report all results to a query without duplicates.
Again this is a characteristic of bottom-up evaluation in Datalog that shall be
preserved.
4. The computational expense of processing a query shall be comparable to or not
overwhelmingly larger than the cost of evaluating a traditional spreadsheet.

F5. Updates

Changes in cell contents (including formulas) or inference rules shall be

immediately reflected in any deduced value, unless automatic re-evaluation has
been disabled by the user. Namely:
1. Modifying a value contained in a cell shall propagate to the functional part of the
spreadsheet as well as to any deduced value that relies on it, even indirectly.
2. Modifying a scalar or array formula shall similarly propagate the resulting
recalculated values to all parts of the spreadsheet that depend on it, whether in
traditional formulas (functional dependency) or in calculated relations (deductive
dependency).
3. Modifying a logical formula shall instantly lead to its re-evaluation and changes
in the resulting computed or deduced values shall propagate to all parts of the
spreadsheet that depend on it, either functionally or deductively.
4. The user shall have the ability to disable automatic re-evaluation if desired.

F6. Functional Integration

The deductive and traditional components shall be integrated, so that values

computed deductively can be part of formulas and values calculated from formulas
can participate in deductive tasks.
1. A logical formula shall be able to draw its input from cells whose value has been
calculated through a scalar or an array formula.
28 2 Requirements

2. A scalar or array formula shall be able to draw its input from cells whose value
has been deduced by means of a logical inference.
3. A logical formula may embed a scalar or array formula, subject to the constraint
that no other requirement is violated.
4. A scalar or array formula may embed a logical formula, subject to the constraint
that no other requirement is violated.

2.2 Cognitive Requirements

This class of requirements has to do with the general experience of the user during
his or her daily interactions with a tool. Therefore, issues that are addressed include
the manner in which the graphical interface provides access to the underlying
functionalities, the familiarity with respect to similar applications, and the effort
involved in using the tool and discovering new features.
The principal cognitive requirements of the deductive spreadsheet are listed next.

C1. Cognitive Conservativity

All traditional functionalities shall be accessible to the user in the exact same way as
in a traditional spreadsheet. The deductive interface shall therefore be conservative
and unobtrusive. In particular:
1. All traditional methodologies for entering data and formulas in a spreadsheet
must be supported by the deductive extension. In particular, values and formulas
can be entered in a cell as text using a keyboard. Formulas can also be entered
using menus and “wizards” in conjunction with cell range selection using the
mouse. The intelligent cut-and-paste functionality shall be available as in the
traditional spreadsheet to iterate formulas to new regions of the spreadsheet.
2. All traditional methodologies to display calculated values and other feedback
shall be supported. This includes the standard tabulated display of calculated
cells, and visual rendering using charts and other graphical methods.
3. The user shall not be penalized by the deductive extension whenever he or she is
not using it. In particular, no time overhead should be incurred when using only
the traditional functionalities of the deductive spreadsheet.

C2. Consistency

The commands supporting the deductive extensions shall be consistent with the
existing modalities of interaction and with the user’s expectation. Therefore:
2.2 Cognitive Requirements 29

1. The user shall be able to enter logical formulas as text using the keyboard.
2. He or she shall also be able to make use of the mouse to select ranges and specific
operations.
3. Menu items and “wizards” shall also be available consistently with the traditional
functionalities.
4. The intelligent cut-and-paste functionality shall be supported and operate on
logical expressions in the same way as for scalar formulas.
5. These commands shall be intuitive and easy to use.

C3. Cognitive Integration

Using the deductive capabilities shall be natural and intuitive, including when used
in conjunction with the traditional functionalities. Specifically:
1. The syntax supporting the deductive functionalities shall be chosen so as to have
an intuitive meaning even for users unfamiliar with logic.
2. The textual and graphical syntax available to access the deductive functionalities
shall be consistent with the input methods for traditional formulas, and similarly
for output.
3. There should be as few barriers as possible preventing a user from embedding
traditional formulas in a deductive expression and vice versa.

C4. Performance

For common forms of data inference, an implementation of the deductive spread-

sheet should have a performance that is subjectively comparable to spreadsheet
operations of intermediate complexity. Specifically:
1. Update propagation should feel instantaneous, both for traditional and deductive
formulas.
2. Evaluation from scratch should not take longer than loading a midsize traditional
spreadsheet.
3. A deductive spreadsheet that does not use deductive formulas should have the
same subjective performance as a traditional spreadsheet.

C5. Discovery

The deductive extension should have a gentle learning curve, so that users can rein-
force and extend their skills by simply using the tool and minimal experimentation.
In particular:
30 2 Requirements

1. Pull-down menus, buttons, right-mouse-button menus and “wizards” should

unobtrusively make deductive operations and concepts available to the user.
2. Once exposed to the basic deductive task of entering a simple conjunctive
formula, entering more complex formulas, e.g., disjunction or negation, should
quickly become natural.
3. The user should easily grasp the possibility of using the output of a logical query
as the input of a scalar or array formula, and vice versa.
4. The syntax for embedding traditional and logical formulas within each other
should be intuitive.
5. Error messages should be helpful in guiding the user toward proper use, not just
away from unavailable options.
6. Supporting material in the form of an integrated help, a user manual and possibly
a step-by-step tutorial should be available and easily accessible whenever needed.
7. In the long run, Internet support, both from the implementers and a user
community, is expected.

2.3 Evaluation

Functional requirements are assessed by checking that the design of the deductive
apparatus meets them. This can be done by inspection for a number of minor
requirements especially when they are not interdependent. Some more theoretical
requirements, such as decidability, are often established in the literature or by
extending arguments found in the literature. Verifying functional requirements that
heavily depend on each other generally involves substantial analysis.
Cognitive requirements need to be assessed empirically and experimentally.
Methodologies to do so abound in the literature. We chose to base our design on two
techniques, the theory of cognitive dimensions of notations [GP96] and the attention
investment model [Bla02], both of which had been applied with good results in other
extensions of the spreadsheet [BADC 01, PBB03]. These techniques are intended to
assist a designer who is not an human-computer interaction expert in the task of
creating a plausible preliminary user interface. An actual evaluation of our design
needs to rely on rigorous empirical assessment methodologies based on predictive
modeling and user testing.
 Part I
Deductive Engine

Outline

In the first part of this work, we review a succession of languages and adapt them
to the context of the deductive spreadsheet. Our analysis closely tracks the develop-
ment of Datalog and deductive databases. At the same time, it points to aspects
that do not apply in a spreadsheet and emphasize issues that become crucially
important. For example, the notion of query plays a minor role in the deductive
spreadsheet, which makes the huge body of literature on query optimization largely
irrelevant. On the other hand, real-time update propagation acquires tremendous
importance.
In Chap. 3, we develop a simple model of the traditional spreadsheet. We outline
the structure and operational semantics of this useful tool, with particular emphasis
on the notion of update. This allows us to distill a methodology and a number of
guiding principles for the deductive spreadsheet.
In Chap. 4, we recall basic concepts of relational algebra as used in relational
databases and import them into the spreadsheet model, obtaining what we call
the relational spreadsheet. This relational extension already highlights some of the
differences between databases and spreadsheets, differences that acquire importance
in more complex languages. It also allows us to settle the issue of seamlessly
integrating our extension in the traditional paradigm.
Chapter 5 provides a logical interpretation of the resulting relational spreadsheet
as the non-recursive fragment of a suitable flavor of Datalog enriched with negation
and constraints. The theory of this language is developed into a model that
emphasizes the unique requirements of a spreadsheet. In particular, it provides
a logical account of key spreadsheet functionalities such as evaluation, update
and explanation. A spreadsheet based on this interpretation can already be called
“deductive”, for a very weak notion of deduction.
Chapter 6 defines the deductive spreadsheet proper by adding recursion to this
logical interpretation, hence turning it into a constrained form of Datalog with
negation. This step provides a substantial boost in expressiveness while requiring
32 I Deductive Engine

only small theoretical adjustments. It also enables further extensions, notably the
ability to weave simple lists into the language. The added deductive power, which
includes inferential closure, hypothetical reasoning and some form of aggregation,
is illustrated by way of examples.
Chapter 3
The Traditional Spreadsheet

This chapter presents a simple mathematical model of the traditional spreadsheet.

While nothing deductive is to be found here, it will give us a solid foundation on
which to develop a series of deductive extensions in the remaining chapters of Part I.
This model will be a constant yardstick on which to gauge our proposals. It will
highlight the important facets of a spreadsheet, from a functional point of view,
thereby providing us with a development methodology and a number of guiding
principles for the deductive spreadsheet.
In Sect. 3.1, we define our abstraction of a spreadsheet. In Sect. 3.2, we char-
acterize the evaluation process that replaces formulas with their calculated value.
In Sect. 3.3, we look at updates and how they are propagated throughout a
spreadsheet. In Sect. 3.4, we discuss the explanation mechanisms supported by a
generic spreadsheet and comment on what is provided in commercial products.
Section 3.5 summarizes this chapter.

3.1 Spreadsheet Model

In this section, we give a simple mathematical model of the traditional spreadsheet

and its constituent entities. Commercial products add many extra features, but their
fundamental mode of operation does not stray far from this basic abstraction.
As we develop our model, it will be useful to refer to the simple example
spreadsheet in Fig. 3.1. The top part of the figure shows the contents of each cell
as entered by the user: in particular, it displays the formulas of calculated cells
rather than their result. We use the syntax commonly available in applications
such as Microsoft Excel. The bottom part shows the spreadsheet as it appears
on the user’s screen, after all the formulas have been evaluated. This typical
spreadsheet records the hours worked weekly by a part-time employee and performs
some simple statistics and accounting. Row 1 provides captions for each column.
Rows 5 and greater are blank. The first two columns display the beginning and end
date of each work week. The value in cell A2 is an actual date (the date of hire,

I. Cervesato, The Deductive Spreadsheet, Cognitive Technologies, 33

DOI 10.1007/978-3-642-37747-1 3, © Springer-Verlag Berlin Heidelberg 2013
34 3 The Traditional Spreadsheet

A B C D E
1 Monday Friday Hours To date Weekly average
2 11/29/04 =A2 +4 8 =C2 =D2/((A2 -A2)/7 + 1)
3 =A2+7 =A3 +4 17 =D2 +C3 =D3/((A3 -A2)/7 + 1)
4 =A3 +7 =A4 +4 25 =D3 +C4 =D4/((A4-A2)/7 + 1)
5
(Entered spreadsheet)

A B C D E
1 Monday Friday Hours To date Weekly average
2 11/29/04 12/03/04 8 8 8.00
3 12/06/04 12/10/04 17 25 12.50
4 12/13/04 12/17/04 25 50 16.67
5
(Evaluated spreadsheet)

Fig. 3.1 A simple timecard spreadsheet

maybe), while the remaining values are given by the shown formulas. Column C
records the hours worked each week. Column D contains formulas to display the
cumulative hours worked by the employee. Finally, column E shows the average
number of hours worked weekly. Like many spreadsheets, this example is expected
to grow with time, typically by adding one row each week until employment ends.
We start with a set V of values whose elements we will denote by the letter v,
possibly sub- or superscripted. Values represent what is displayed in a spreadsheet:
strings, various types of numbers (integers, floating point, dates, etc.), error marks,
and more. Formulas are not values, but they produce a value when calculated.
In Fig. 3.1, “Monday”, “11/29/04”, and “8” are examples of values. Within V,
we distinguish the special value [, corresponding to the contents of a blank cell,
and the error value . This model could be extended with additional distinguished
values (undefined, etc.).
The cells of a spreadsheet are referenced by means of a system of locations drawn
from a set L. In Fig. 3.1, as well as in early spreadsheets, locations are simply the
column-row coordinates of a cell (e.g., A2). Nowadays, multiple sheets and external
references add new dimensions and flexibility. We write l, variously decorated,
for locations.
Operators such as “C” and “/” in Fig. 3.1 are used to calculate new values of
interest from the information in other parts of the spreadsheet. We assume a fixed
set Op of operator symbols: this is consistent with all commercial implementa-
tions, none of which provides simple means for a user to define new operators.1

1
To be more precise, a programmer can implement new operators and make them available by
means of add-on modules, something for which an end-user often lacks the know-how. There have
also been recent proposals (e.g., [PBB03]) for operators definable directly by an end-user, which
do not require programming skills, but no such extension is part of any commercial product as
of yet.
3.1 Spreadsheet Model 35

An operator op takes a fixed or variable number of arguments (this number is

called the arity of this symbol). For example, “C” has arity 2, while “SUM” can
take arbitrarily many arguments (usually rendered using a range such as “A2W A19”,
which we do not model in this chapter).2 We liberally use infix notation and standard
precedence rules. Each operator op of arity n is associated with an operation,
denoted by op,b that maps n argument values to a result value. This operation is
therefore a function opb W V n ! V. For example, the addition symbol “C” is
associated with the addition operation, which returns the sum of its two arguments,
so that 8 C O 17 is 25. Operations applied to the wrong number of arguments
return . An operation also returns an error if some of its arguments belong to
the wrong syntactic class (e.g., when trying to add two strings—although many
implementations use defaults values for ill-typed arguments, for example identifying
a non-numerical string with 0).
Values, locations and operators allow writing expressions. They are defined by
the following grammar:

Expressions: e WWD v j l j op.e1 ; : : : ; en /

Again, n may be different from op’s arity: an expression will evaluate to  in that
case. We denote the set of all expressions by E. Since every value is an expression,
we have that V  E. We write e v e 0 for the subexpression relation: it holds if e
occurs within e 0 . For example, “A4  A2” v “D4=..A4  A2/=7 C 1/” in cell E4 of
our example. We write e @ e 0 if e v e 0 and e ¤ e 0 . An expression that is not a value
is called a formula. In most commercial spreadsheets and in the example in Fig. 3.1,
formulas are introduced by the symbol “D” (which is not part of the formula).
A spreadsheet has an expression in every location. It is then natural to view it as a
function s W L ! E. However, not all such functions correspond to spreadsheets as
we commonly understand them. We need to introduce some definitions to precisely
characterize well-formed spreadsheets.
Let s W L ! E be a function from the set L of locations to the set E of
expressions and let   E 2 be any binary relation over expressions. The referential
closure of  over s is the binary relation s  E 2 defined as follows:
• s.l/ s l for all l 2 L;
• e s e 0 if e  e 0 ;
• e s e 00 if e s e 0 and e 0 s e 00 .
Then, e @s e 0 simply extends the definition of e @ e 0 to encompass not just the
subexpressions of e 0 , but also the expressions contained in any location l mentioned
in e 0 , and their subexpressions, and the expressions contained in any location they
mention, and so on. In particular, l @s l 0 holds if and only if the value at location l 0

2
More precisely, Excel’s SUM takes up to 30 arguments, each of which can be a range. We ignore
the difference and assume the number of arguments to be arbitrary.
36 3 The Traditional Spreadsheet

depends on the value in cell l. We say that s contains a circular dependency if there
is any location l that depends on itself, i.e., l @s l for some l.
Finally, a well-formed spreadsheet s W L ! E is a spreadsheet subject to two
conditions:
1. Finitely many cells in the domain of s are non-blank.3
2. s does not contain any circular dependency.
The first constraint implies that only finitely many cells need to be accessed during
evaluation. The second restriction makes sure that a cell needs to be visited a finite
number of times when calculating the spreadsheet. Without it, evaluation could enter
infinite loops. Commercial products issue a warning if a user accidentally includes
a circular dependency.4
Together, these two restrictions guarantee that evaluation will always terminate.
We shall take great care not to violate them as we introduce deductive extensions
to this paradigm: the first will hold in its current form, although the evaluation of a
(logical) formula will write a potentially unbounded (but always finite) number of
cells. We will relax the second by allowing controlled forms of circular references
in formulas (by means of recursion), but in doing so we will always make sure that
the finiteness of evaluation is maintained.
All the spreadsheets we will consider will be well-formed. We will therefore drop
this qualifier. We will often write a spreadsheet s W L ! E as s D fl 7! eg.
We will enrich this model with arrays and array operations in Chap. 4 as a
jumping board to dive in the deductive spreadsheet.

3
In our model, we implicitly lift any restriction on the number of cells in a spreadsheet. All
commercial applications force spreadsheets to be finite: for example, Microsoft Excel has “IV”
as its last column (for a total of up to 256), and 65,536 rows. Other commercial products have
similar bounds.
4
More precisely, although a warning is always issued, most commercial applications allow the
advanced user to enable a restricted form of circular references: assume for example that cell A1
contains the value 600, A2 contains the formula D A3  0:2 and A3 contains D A1 C A2. There
is a circular dependency between A2 and A3. Cell A2 is initialized with the value 0, which gives
600 for A3. At this point, A2 is re-evaluated on the basis of this new value for A3, yielding 120
which is used to refine the value of A3 to 480. The values of cells A2 and A3 eventually converge
to 100 and 500, respectively. The maximum number of iterations is user-defined (100 by default
in Microsoft Excel) and the notion of convergence is taken to mean that a cell’s value is within
another user-defined constant of the value in the previous iteration (the default is 0.001 in Excel).
Note that this amounts to numerically solving the equation 600  0:2x D x.
While the model we are developing can easily accommodate such bounded iterations, we
refrain from doing so for several reasons. First, this would add a layer of complication that would
only have the effect of obscuring our deductive extensions. Second, the user base of this esoteric
technology is very small. Indeed, few users are even aware of its existence and even fewer can
competently handle it. Third, circular evaluation is disabled by default in all commercial products.
Indeed, end-users have a hard time writing correct spreadsheets in the first place (see [Pan98] for
some disturbing statistics) even without this error-prone option.
3.2 Evaluation 37

3.2 Evaluation

The evaluation of a spreadsheet is the process of calculating a value for every

formula in it and displaying this value in the corresponding cell. This is also called
calculation. Spreadsheet evaluation can be described in many ways. In this section,
we take an approach that is at the same time very general and closely related to some
of the deductive extensions we will introduce in the next chapters.
Full evaluation is a rare event: it typically occurs only when opening a file
in a spreadsheet application such as Microsoft Excel, i.e., once in every session.
Furthermore, all commercial implementations provide a way to force it at any time,
in the very rare case something goes awry: in Excel, this is achieved by pressing
“CTRL-SHIFT-ALT-F9”, for example. In contrast, recalculation occurs every time
the user modifies the contents of a cell: this frequency can be influenced by changing
policy settings, something a user seldom bothers doing. We will study recalculation
and the general implications of updates in Sect. 3.3.
This section is organized as follows: we present our approach to evaluation in
Sect. 3.2.1, but wait until Sect. 3.2.2 to provide the mathematical insight to explain
it. In Sect. 3.2.3 we re-express our notion of evaluation in a more structured way,
which we will repeatedly use later.

3.2.1 Naı̈ve Strategy

In the previous section, we defined a spreadsheet s as a mapping from locations to

expressions: s D fl 7! eg. In order to represent the evaluation process, we will
introduce a parallel structure that associates values, rather than expressions, to cells.
Before doing so, a few auxiliary notions are needed. Let ? be a symbol not
occurring in V. We will use it as the uncalculated value: a placeholder for a value
not yet computed. We write V? for the augmented set V [ f?g. We extend each
b W V n ! V to an operation op
operation op b ? W V?n
! V? as follows:
8
<? if vi D ? for some i
b ? .v1 ; : : : ; vn / D
op
:op.v
b 1 ; : : : ; vn / otherwise

In words, op b when applied to actual values, but returns ?

b ? operates exactly like op
if any of its arguments is ?.
Given these preliminaries, an environment  is a mapping from locations to
extended values:  W L ! V? . Let Env be the set of all environments. One
environment that will play a prominent role in this section is the initial environment
in which every value is uncalculated: fl 7! ?g. For conciseness, we overload
symbols and denote it by ?.
38 3 The Traditional Spreadsheet

An arbitrary expression e can now be evaluated against an environment , an

action we denote by e @ . Every location in e is replaced with the value recorded
in  (possibly ?) and a value is extracted by applying the extended operation op
b?
in correspondence with every operator op in e. More formally, e @  is defined as
follows:

v@ D v
l @  D .l/
op.e1 ; : : : ; en / @  D op
b ? .e1 @ ; : : : ; en @ /

Note that e @  evaluates to ? only if e mentions a location l that  maps to ?.

It is a simple conceptual step to extend this notion to the parallel evaluation of an
entire spreadsheet s D fl 7! eg against an environment :

ER fl7!eg ./ D fl 7! e @ g

It is convenient to interpret ER as an environment transformer, parametrized by a

spreadsheet s. The function ER s is called the naı̈ve evaluation step, for reasons that
will become apparent shortly. For every location l, the functional ER s re-binds l from
.l/ to e @ , where e is the contents of l in the spreadsheet s. In other words, ER s ./
evaluates every cell in s, drawing the value of each location mentioned in a formula
from . In particular, recall that if e is a value v, then e @  D v.
At this point, we have all the tools we need to define the evaluation of a
spreadsheet s. The calculated spreadsheet displayed to the user is an environment
 that we will construct in a number of iterations. As the user loads s from
a file,  is created and every cell in it is marked as containing no value: this
corresponds to the initial environment ?. The first iteration loads in  all the values
present in s (as well as the value of any formula that does not depend on any
location—e.g., “D 0:64  12”). The second iteration produces an actual value for
only the formulas that refer to locations containing those values. The third iteration
evaluates all the formulas that refer to cells that have received a value in the first
and second iterations. The process repeats until all locations in  contain an actual
value, i.e., not ?. This is the evaluated spreadsheet, displayed to the user.
We formalize this intuition as follows. We start with the initial environment ?.
Each of the above intuitive iterations corresponds to applying ER s to the last
environment obtained in this way. This is repeated until an environment that does
not mention ? is produced. In symbols, we are calculating

ER sn .?/

for some n. This approach is called the naı̈ve evaluation strategy.

Figure 3.2 demonstrates this process on the timecard spreadsheet in Fig. 3.1
(call it s). The initial environment is displayed above the label ER s0 : every cell
contains ?. After one application of ER s , we obtain an environment ER s1 containing an
3.2 Evaluation 39

A B C D E A B C D E
1 ⊥ ⊥ ⊥ ⊥ ⊥ 1 Mo... Fr... Ho... To... We...
2 ⊥ ⊥ ⊥ ⊥ ⊥ 2 11/29 ⊥ 8 ⊥ ⊥
3 ⊥ ⊥ ⊥ ⊥ ⊥ 3 ⊥ ⊥ 17 ⊥ ⊥
4 ⊥ ⊥ ⊥ ⊥ ⊥ 4 ⊥ ⊥ 25 ⊥ ⊥
5 ⊥ ⊥ ⊥ ⊥ ⊥ 5

Ës0 Ës1

A B C D E A B C D E
1 Mo... Fr... Ho... To... We... 1 Mo... Fr... Ho... To... We...
2 11/29 12/03 8 8 ⊥ 2 11/29 12/03 8 8 8.00
3 12/06 ⊥ 17 ⊥ ⊥ 3 12/06 12/10 17 25 ⊥
4 ⊥ ⊥ 25 ⊥ ⊥ 4 12/13 ⊥ 25 ⊥ ⊥
5 5

Ës2 Ës3

A B C D E A B C D E
1 Mo... Fr... Ho... To... We... 1 Mo... Fr... Ho... To... We...
2 11/29 12/03 8 8 8.00 2 11/29 12/03 8 8 8.00
3 12/06 12/10 17 25 12.50 3 12/06 12/10 17 25 12.50
4 12/13 12/17 25 50 ⊥ 4 12/13 12/17 25 50 16.67
5 5

Ës4 Ës5

Fig. 3.2 Iterations in the evaluation of the timecard example

actual value in every cell in s that holds a value (including all the [ cells—rows 5 and
below) and ? for every formula in s. For typographic reasons, we elided the year of
every date (in cell A2 so far) and truncated the strings in row 1. At the next round,
ER s2 , the environment has grown. All the non-? values of ER s1 are retained, and a few
are added: every cell in s containing a formula that only refers to locations that have
a value in ER s1 sheds its ? mark in favor of an actual value. This trend continues for
another three iterations (ER s3 , ER s4 , and ER s5 ). This last environment does not contain
any occurrence of ?; hence it represents the final value of the spreadsheet, which
we can compare to the lower part of Fig. 3.1.

3.2.2 Elements of Fixpoint Theory

In order to better understand how and why this technique works, we will now
recall some basic elements of the theory of fixpoints, which goes back to Alfred
Tarski [Tar55]. While this is somewhat heavy machinery for the simple language we
have developed so far, it will become an indispensable tool as soon as we introduce
recursion in the deductive spreadsheet (in Chap. 6). Therefore, we believe it is
advantageous to present these notions now and provide a uniform treatment to all
40 3 The Traditional Spreadsheet

the elements of the language we are developing. Many of the definitions below are
adapted from [Llo87]. We will illustrate abstract definitions on our current setting
as well as on examples that will become important later.
A binary relation  on a set S is a partial order if it satisfies the following
properties:
Reflexivity: x  x for each x 2 S .
Antisymmetry: x  y and y  x implies x D y for each x; y 2 S .
Transitivity: x  y and y  z implies x  z for each x; y; z 2 S .
We write < for the strict version of , where x < y iff x  y and x ¤ y. A partially
ordered set, or poset, is a set S together with a partial order . This is denoted
by .S; /.
Example 3.1 (Environments). The set Env of all environments is a poset when
considered together with the relation <Env defined as follows:
(
.l/ D 0 .l/ or
 <Env 0 iff for each location l
.l/ D ? and  .l/ ¤ ?
0

In words,  <Env 0 if  is identical to 0 except for some actual values replaced by

?. The relation Env is <Env modulo equality:  Env 0 iff  <Env 0 or  D 0 .
Example 3.2 (Powersets). An important example of poset is the powerset P X of
any set X under the inclusion relation .
A poset .S; </ is called a totally ordered set if the order relation < is total:
every element in S is comparable—in symbols, for any x; y 2 S , either x < y or
y < x or x D y. Given a poset .S; </, a subset S 0 of S is an ascending chain with
respect to < if .S 0 ; </ is totally ordered, where .S 0 ; </ is the ordered set obtained
by restricting the order < of S to S 0 .
Example 3.3 (Environments). The environments ER s0 , ER s1 , ER s2 , ER s3 , ER s4 and ER s5 form
an ascending chain in .Env; <Env /.
Example 3.4 (Powersets). Any sets X1 ; X2 ; : : : ; Xn  X such that X1  X2
 : : :  Xn are an ascending chain in .P X ; /.
Given a subset S 0 of S , an element x 2 S is an upper bound of S 0 if x 0  x for
every x 0 2 S 0 . If ub S 0 is the set of all upper bounds of S 0 , an element xN 2 ub S 0 is
the least upper bound of S 0 if xN  x 0 for every x 0 2 ub S 0 . Note that S 0 may not
have any upper bound, and even if it does, there may not be a least upper bound.
However, this object is unique if it exists. In that case, we indicate it as lub S 0 .
The notion of lower bound and greatest lower bound (glb) are defined dually.
A poset .S; / such that glb S 0 exists for every S 0  S is called a lower
semi-lattice. In particular, glb S exists, and is often denoted by ? and called the
bottom element of S . If both lub S 0 and glb S 0 exist for every S 0  S , then
3.2 Evaluation 41

Fig. 3.3 Least upper bound

and greatest lower bound of a S
ub S 
set S 0

• lub S 

S ≤

• glb S 

lb S 

.S; / is a complete lattice. In that case, lub S is denoted by > and called the top
element of S . This is illustrated in Fig. 3.3.
Example 3.5 (Environments). The set Env of all environments is a lower
semi-lattice. Indeed, the glb of any set of environments Env0  Env is the
environment that associates the value v to any location l such that 0 .l/ D v
for every 0 2 Env0 , and maps all other locations to ?. The initial environment ?
is glb Env, the bottom element of Env. However, a set Env0 of environments may
not have an upper bound, and therefore no lub. Thus, .Env; Env/ is not a complete
lattice in general.
Example 3.6 (Powersets). .P X ; / is a complete lattice. For any subset S 0 of P X ,
glb S 0 and lub S 0 are the intersection
T and the union of all the
S elements of S 0 ,
respectively. In symbols, glb S D X 0 2S 0 X and lub S D X 0 2S 0 X . The top
0 0 0 0

and bottom elements of P X are X and ;, respectively.

We now look at functions over posets and how they interact with the notions just
introduced. We will retain .Env; Env/ as our source of examples but temporarily
abandon powersets: we will show how the next few definitions apply in an important
example in Chap. 6.
A function F W S ! S over .S; / is monotonic if for every x; y 2 S , whenever
x  y, it is the case that F .x/  F .y/.
Example 3.7 (Environments). Given any spreadsheet s, it is easy to show that the
functional ER s is monotonic: since  <Env 0 means that 0 extends  with additional
values, every non-? value in ER s ./ will be present also in ER s .0 /, but some of
the locations mapped to ? in ER s ./ may contain an actual value in ER s .0 /. Hence
ER s ./ Env ER s .0 /.
An element x 2 S is a fixpoint of F if F .x/ D x. Clearly, the set of fixpoints
of F is itself a poset with respect to . We will be particularly interested in
42 3 The Traditional Spreadsheet

situations where this set is not empty and has a smallest and/or greatest element.
If these exist, they will be called the least fixpoint and greatest fixpoint of F ,
respectively. We denote them by lfp.F / and gfp.F /, respectively.
Example 3.8 (Environments). The environment ER s5 in Fig. 3.2 is a fixpoint for ER s :
indeed, applying ER s to it produces an identical environment. We will see that the
evaluation procedure in Sect. 3.2.1 always calculates a fixpoint for ER s , and that this
fixpoint is unique.
The Knaster-Tarski theorem [Tar55] (see also [LNS82]) affirms that if .S; /
is a lattice and F is monotonic, then both lfp.F / and gfp.F / exist (actually, the
set of all fixpoints of F forms a lattice). Furthermore, lfp.F / D glb fx W F .x/ D
xg D glb fx W F .x/  xg and gfp.F / D lub fx W F .x/ D xg D lub fx W x 
F .x/g. If .S; / is a lower semi-lattice, only lfp.F / is guaranteed to exist, and it is
characterized in the same way.
A function F W S ! S is continuous if for every ascending sequence S 0  S
such that lub S 0 exists, then

F .lub S 0 / D lub fF .x/ W x 2 S 0 g

We denote this latter set by lub F .S 0 /. Observe that a continuous function is

necessarily monotonic (take S 0 D fx; yg).
Example 3.9 (Environments). Within Env, an ascending sequence Env0 starts with
an environment containing many occurrences of ? and progressively replaces them
with actual values. Since such a sequence is totally ordered, its lub always exists.
Proving that lub.ER s .Env0 // Env ER s .lub Env0 / is nearly trivial. The reverse direction
requires unfolding the definition of ER s .
The least upper bound of a continuous function F can be given a more
computational characterization. We define the ordinal powers of F as follows5 :

F0 D ?
F i C1 D F .F i /
F ! D lub fF i W i 2 Ng

Note that, in order for this notion to be well defined, .S; / needs to be at least
a lower semi-lattice. Indeed, it is easy to verify that F i  F i C1  F ! for every
i 2 N.
Specializing an earlier result due to Tarski, Kleene has proved that whenever
.S; / is a lower semi-lattice and F is continuous, then F o  lfp.F / for any ordinal
o in N [ f!g. Moreover, there exists an ordinal oN such that for any o > o, N it is the
case that F o D lfp.F /. This is known as Kleene’s theorem.

5
A more accurate definition of ordinal power would have cases for all limit ordinals, not just !.
However, we will not need any ordinal bigger than ! thanks to our continuity hypothesis.
3.2 Evaluation 43

This result suggests a semi-algorithmic approach to computing lfp.F /: starting

with ?, repeatedly apply F obtaining the ascending sequence ?; F 1 ; F 2 ; : : : until
an element repeats. When this occurs, we have reached lfp.F /. Note that this may
not occur in finite time since the above property does not rule out that lfp.F / D F ! .
Example 3.10 (Environments). Having ascertained above that .Env; Env/ is a
lower semi-lattice and that the calculation functional ER s for a spreadsheet s is
continuous over it, the above result tells us that the evaluation methodology in
Sect. 3.2.1 simply consisted in calculating the least fixpoint of ER s .
Because of the constraints on the dependencies in a spreadsheet, the calculation
of ER s always terminates, i.e., the ordinal oN above is finite (we will characterize it
precisely in the next section). Indeed, since there are only finitely many non-blank
cells, there is only a finite number of ? to replace after the first iteration (which
handles all values in s, including blank cells). The requirement that there are no
circular dependencies makes ER s monotonic.
As we conclude this section, it is useful to broaden our approach to evaluation
in the light of the theory we just developed. The naı̈ve functional ER s is just one
way to evaluate a spreadsheet: it starts from the values in s and calculates formulas
in layers of dependency (more on this in Sect. 3.2.3). Other strategies are equally
viable: for example, in the timecard spreadsheet in Fig. 3.1, we could fully evaluate
column A, then column B followed by D and finally E; or we could proceed by rows;
or we could jump around. As long as a strategy respects dependencies (i.e., does
not evaluate a formula before a value for each location it refers to is available),
computed values are stable (i.e., the contents of a cell is initially ?, is later replaced
by an actual value, and is never changed after that), and is exhaustive (i.e., all cells
are eventually assigned a non-? value); this strategy corresponds to a continuous
function over .Env; Env / and therefore all the results we have obtained for ER s apply
to it as well.

3.2.3 Dependency Graph

Although the naı̈ve evaluation procedure presented in Sect. 3.2.1 soundly captures
the notion of evaluation of a spreadsheet on the basis of the theory in Sect. 3.2.2,
we will now re-express it in a more structured way. The basic methodology
will remain the same, but the added structure will make it easier to assess
evaluation costs, discuss optimizations, and analyze updates in Sect. 3.3. A similar
structure will also play a crucial role when introducing deductive extensions to the
spreadsheet in the next few chapters.
Given a spreadsheet s, we define the dependency graph of s, denoted by DGs , as
the oriented graph with nodes and edges defined as follows:
• The nodes of DGs is the set of locations L of s.
• There is an edge from location l to location l 0 , written l  l 0 , only if l 0 contains
a formula that mentions l, i.e., iff l v s.l 0 /.
44 3 The Traditional Spreadsheet

Fig. 3.4 The dependency E4

graph of the timecard
example

B4 E3 D4

B3 A4 E2 D3 C4

B2 A3 D2 C3

A2 C2

We will write l 2 DGs and l  l 0 2 DGs for a node and an edge in DGs ,
respectively.
When displaying a dependency graph, we will generally omit nodes without any
incoming or outgoing edge. They correspond to cells that contain values that are
never used in any other cell. In particular, this includes the vast majority of blank
cells. We will generally represent cells containing a formula, especially when it
refers to other locations. The dependency graph of our timecard example is shown
in Fig. 3.4, where we have rearranged the columns for clarity.
A path from l0 to ln in DGs is a series of locations fli gi D0::n with edges li  li C1
for all i D 0; : : : ; n  1. We write l0  ln as an abbreviation for this path, and
denote its length by jl0  ln j, which here is n. A path of length 0 is called trivial.
We write jDGs j for the length of the longest path in DGs .
As its name implies, the dependency graph of a spreadsheet s provides a clear
picture of the dependencies among locations as defined in Sect. 3.1. Indeed, there
is a non-trivial path from l 0 to l in DGs if and only if l depends on l 0 in s, a fact
we wrote as l 0 @s l above. The requirement that a spreadsheet contains no circular
references implies that a dependency graph will always be acyclic: for no l 2 DGs
is there a non-trivial path l  l 2 DGs .
The dependency graph of a spreadsheet serves as a useful skeleton to display
other information in the form of node annotations. In particular, we can compose
it with any function f W L ! X with domain L to display the value f .l/ 2 X
associated with each node l. In Fig. 3.5, we compose the dependency graph of our
timecard example with the spreadsheet itself (recall that s W L ! E) to display the
entered content of every cell and their dependencies.
Even more useful is composing the dependency graph with an environment so
that the nodes display the intermediate values of an evaluation. Figure 3.6 shows
the result for each step ER s0 ; : : : ; ER s5 of the evaluation trace in Fig. 3.2. For reference,
we also display the locations in ER s0 . We have enclosed each occurrence of ? in a
3.2 Evaluation 45

=D2/((A2−A2)/7+1)

=A4 + 4 =D3/((A3−A2)/7+1) =D3 + C4

=A3 + 4 =A3 + 7 =D4/((A4−A2)/7+1) =D2 + C3 25

=A2 + 4 =A2 + 7 =C2 17

11/29/04 8

Fig. 3.5 The dependency graph of the timecard example annotated with expressions

dotted circle while using a solid oval for actual values. Dotted arrows exit ? nodes:
they represent missing data for the evaluation of the target node. Solid arrows
indicate ready information: it either has been used to calculate the value of a
formula, or it can be used at the next iteration. We will call this variant of DG
the evaluation graph of a spreadsheet and we will spend the rest of this section on
its numerous applications. We write EGs ./ for the evaluation graph of spreadsheet
s at environment . Again, EGs ./ is a particular way of rendering .

Cost

The evaluation graph can be conveniently used to assess the cost of calculating a
spreadsheet. Our measure of cost will be the number of times a value is assigned
to a cell during evaluation. In the case of an iterative procedure, such as the naı̈ve
strategy outlined in Sect. 3.2.1, we simply add the number of locations evaluated at
each iteration:
X
cost D evaluated cells.i /
i 2iterations

In the case of the naı̈ve evaluation, we can obtain a simpler expression: the function
ER s updates all the cells in the spreadsheet at every iteration. In practice, it is
sufficient to consider only the finite number of locations that either contain a
formula, or are referenced by one. Our example contains 20 such interesting cells,
but any over-approximation will do. Call this number cellss . The above formula
simplifies to:

cost D iterations  cellss

46 3 The Traditional Spreadsheet

⊥ E4 ⊥

B4 ⊥ ⊥ E3 ⊥ D4 ⊥ ⊥ ⊥

B3 ⊥ A4 ⊥ ⊥ E2 ⊥ D3 ⊥ C4 ⊥ ⊥ ⊥ ⊥ 25

B2 ⊥ A3 ⊥ ⊥ D2 ⊥ C3 ⊥ ⊥ ⊥ 17

A2 ⊥ ⊥ C2 11/29/04 8

Ës0 Ës1

⊥ ⊥

⊥ ⊥ ⊥ ⊥ ⊥ ⊥

⊥ ⊥ ⊥ ⊥ 25 12/10/04 12/13/04 8.00 25 25

12/03/04 12/06/04 8 17 12/03/04 12/06/04 8 17

11/29/04 8 11/29/04 8

Ës2 Ës3

⊥ 16.67

12/17/04 12.50 50 12/17/04 12.50 50

12/10/04 12/13/04 8.00 25 25 12/10/04 12/13/04 8.00 25 25

12/03/04 12/06/04 8 17 12/03/04 12/06/04 8 17

11/29/04 8 11/29/04 8

Ës4 Ës5

Fig. 3.6 Evaluation of the timecard example on its dependency graph

The number of iterations is given by the length of the longest path in DGs plus
one: jDGs j C 1. Indeed, evaluation starts with a graph containing only ? (ER s0 in
Fig. 3.6); with the first iteration ER s1 , it assigns an actual value to all the nodes without
incoming arrows; in the following iterations, ER si C1 , it follows edges in the graph as
it inserts an actual value in all nodes all of whose predecessors contain actual values
3.2 Evaluation 47

(edges on the longest path are always followed). Since there are five iterations in
our example, evaluating it with ER s has cost 20  5 D 100. The maximum number
of iterations is then equal to cellss . This leaves the naı̈ve evaluation with an upper
bound of

cost D O.cells2s /

The cost can be substantial for a large spreadsheet. It should however be noticed
that ER s is very inefficient:
• It recalculates the value of every non-? location at every iteration: since the
actual value of a cell does not change once evaluated, this is wasted work, and
repeatedly so.
• It attempts to calculate the value of locations even when they depend on cells
currently valued at ?. Again this is useless work.
We will repeatedly encounter evaluation functions that behave in this way in later
chapters. A direct implementation yields the naı̈ve evaluation procedure: a great
mathematical tool, but far from optimal from a computational viewpoint.

Semi-naı̈ve Evaluation

The above analysis suggests an obvious optimization that goes by the name of
semi-naı̈ve evaluation in deductive database circles (more on this in Chap. 6): rather
than evaluating the entire spreadsheet at each iteration, calculate just the cells that
depend only on locations that have changed since the last iteration, and only if they
do not depend on any ?-valued cell. Here is how this would work on our example
in Fig. 3.6:
1. Starting from the initial environment ER s0 , calculate the value of just the
cells without an incoming arrow. If we set our interesting perimeter to the
non-blank cells of s, these are A1WA2, B1, C1WC 4 (using the range notation for
conciseness). We obtain ER s1 in this way.
2. The formulas that depend only on the cells updated in the previous iteration are
in A3, B2, and D2: we calculate their value, obtaining ER s2 .
3. The formulas depending only on cells updated in iterations 1 and 2 are in A4,
B3, D3 and E2. Evaluating them yields ER s3 .
4. We iterate on cells B4, D4 and E3 obtaining ER s4 .
5. This last iteration leaves only E4 to be evaluated, on which no other location
depends.
We will now give a mathematical representation of the semi-naı̈ve evaluation and
then assess its cost. Given a spreadsheet s and an environment , we partition  into
two partial functions:
48 3 The Traditional Spreadsheet

Fig. 3.7 Environment

partitioning for semi-naı̈ve ∇s ´ ´
evaluation e

e ⊥ ⊥
Δs ´

⊥
v
v
∇s ´ v v v

• The restriction s  of  to just the ?-valued cells that do not depend on any
currently unevaluated location. In symbols,

s  D fl 7! .l/ such that .l/ D ? and ? 62 .fl 0 v s.l/g/g

• The rest of : rs  D  n s ,
An intuitive representation of this partitioning is displayed in Fig. 3.7. Each step
in the semi-naı̈ve evaluation is then described by the function EP s defined as
follows:
 
l 7! e @  W l 2 dom.s /
EP fl7!eg ./ D
l 7! .l/ W l 2 dom.rs /

Notice that only the case corresponding to locations in s  contributes to the

number of cells evaluated during any given application of EP s : the remaining
cells are known not to change and therefore no evaluation takes place on them.
The semi-naı̈ve evaluation iterates this basic step a number of times equal to the
length of the longest path in DGs plus 1:

EP sjDGs jC1 .?/

During the semi-naı̈ve evaluation, each cell in the spreadsheet is evaluated

exactly once. Hence, the cost of this procedure is given by

cost D cellss

Note that this cost is optimal: it is impossible to do better given our measure of
cost (additional efficiency gains can be obtained, for example, by keeping track of
subexpressions that appear in more than one cell and evaluating them once—clearly,
this is effective only if the bookkeeping overhead does not dominate the savings).
3.3 Updates 49

A B C D E
1 Monday Friday Hours To date Weekly average
2 11/29/04 =A2 +4 8 =C2 =D2/((A2 -A2)/7 + 1)
3 =A2 +7 =A3 +4 17 =D2 +C3 =D3/((A3 -A2)/7 + 1)
4 =A3 +7 =A4 +4 25 =D3 +C4 =D4/((A4 -A2)/7 + 1)
5 =A4 +7 =A5 +4 25 =D4 +C5 =D5/((A5 -A2)/7 + 1)

A B C D E
1 Monday Friday Hours To date Weekly average
2 11/29/04 =A2 +4 8 =C2 =D2/((A2 -A2)/7 + 1)
3 =A2 +7 =A3 +4 17 =D2 +C3 =D3/((A3 -A2)/7 + 1)
4 =A3 +7 =A4 +4 25 =D3 +C4 =D4/((A4 -A2)/7 + 1)
5 =A4 +7 =A5 +4 19 =D4 +C5 =D5/((A5 -A2)/7 + 1)

Fig. 3.8 Two-step update of the timecard spreadsheet

3.3 Updates

As mentioned at the beginning of Sect. 3.2, full evaluation is a very rare event that
takes place only when loading a spreadsheet from file. Much more common is
updating a spreadsheet, something that can take place hundred of times in each
session. Figure 3.8 demonstrates this on our timecard example: at the end of a
new week, the hours worked by the employee are recorded. We realize this in two
steps:
1. At the top of the figure, we simply copy the contents of row 4 into row 5.
The enhanced paste functionality of modern spreadsheet applications takes care
of adapting the locations mentioned in the formulas to the new row (the exception
is cell A2 in column E, which is treated as an absolute reference—for the sake of
readability, we did not display the usual syntax to indicate this).
2. In the bottom of the figure, we replace the contents of cell C5 (copied from row 4
in the first step) with the correct number of hours worked this week, 19 in this
case.
This modality of extending a spreadsheet is typical. In a highly professional setting,
an administrator may have prepared a spreadsheet for the entire year so that the user
only needs to insert “19” in the appropriate location. However, most end-users left
to their own devices will take the two-step approach just outlined, possibly with a
few more steps added to make and repair mistakes.
An update is a change in the contents of one or more cells in an already evaluated
spreadsheet. The above example shows two updates: the first involves five cells,
the second only one. After each update, the spreadsheet needs to be recalculated.
Recalculation or re-evaluation is the process of propagating the changes to all parts
of the spreadsheet that depend on the modified locations by updating the displayed
50 3 The Traditional Spreadsheet

values of the affected cells. We will spend the remainder of this section modeling
updates and analyzing re-evaluation.
Given a spreadsheet s D fl 7! eg, an update on s is a partial function s D
fl * e 0 g such that dom s is the set of updated locations. In the top part of Fig. 3.8,
s D fA5 * A4 C 7; B5 * A5 C 4; C 5 * 25; D5 * D4 C C 5; E5 *
D5=..A5  A2/=7 C 1/g. The second update is more localized: s 0 D fC 5 * 19g.
Applying an update s to a spreadsheet s simply replaces the modified locations
of s with the contents in s. This notion is defined as follows:
(
s.l/ if l 2 dom s
.s ˇ s/.l/ D
s.l/ otherwise

We will often write s 0 for s ˇ s. The results of applying the first update above to
the timecard spreadsheet in Fig. 3.1 is shown in the top part of Fig. 3.8. Applying
the second update to it produces the bottom spreadsheet in this figure.
Next, we study how updates alter the dependency graph of a spreadsheet s. First,
observe that an update does not modify the nodes of DGs : an update never creates
or destroys cells, but simply changes their contents. Instead, an update often has an
effect on the edges of DGs :
• If s.l/ is a value (possibly [) and s.l/ is a formula e referencing some
cell l 0 , the dependency graph of s will have to be modified with an added
C C
edge l 0  l. We denote this addition by l 0  l. Formally, l 0  l iff l 2
dom s and l v s.l/.
0

• If s.l/ is a formula e referencing some cell l 00 and s.l/ is a value, DGs will
have to be modified by removing the edge l 00  l. We denote this alteration by
 
l 00  l. Formally, l 00  l iff l 2 dom s and l 00 v s.l/.
• If both s.l/ and s.l/ contain formulas, then DGs will generally be modified
with both the addition and removal of edges.
• If s.l/ and s.l/ contain values, then DGs is left unchanged with respect to
location l.
This is summarized in the following table:

Formula in Δs(l) Value in Δs(l)

+ − −
Formula in s(l) l l ; l l l l
 +
Value in s(l) l l

Notice that deleting a cell amounts to replacing its contents with the blank value [.
Figure 3.9 illustrates this process on the timecard example. The dotted nodes rep-
resent the locations altered after the first update. We refrained from displaying them
in Fig. 3.4 since they contained blank cells, without any incoming or outgoing edge.
The dotted arrows denote the modified edges of the original dependency graph:
3.3 Updates 51

Fig. 3.9 The dependency E5

graph of the updated timecard
+
example
+
B5 E4 D5
+ +
+

B4 A5 E3 D4 C5

+ +

B3 A4 E2 D3 C4

B2 A3 D2 C3

A2 C2

all of them are additions and all point to updated cells. The second update does not
further alter the structure of this graph.
We can define the corresponding application of an update s to the dependency
graph of s next. The updated dependency graph DGs ˇ s is the graph such that:
• Nodes.DGs ˇ s/ D Nodes.DGs /.
 C
• Edges.DGs ˇ s/ D Edges.DGs / n fl 00  l W l 00  lg [ fl 0  l W l 0  lg
It is easy to verify that this is the dependency graph of the updated spreadsheet
s ˇ s, in symbols: DGs ˇ s D DGsˇs .
We now turn to the notion of re-evaluation. A simple-minded approach to
displaying the changes resulting from an update is to evaluate the entire updated
spreadsheet from scratch. This is clearly inefficient in most cases since a minority
of cells is affected by the changes, in general. We will now study approaches that
evaluate only the locations that are directly affected by the update: the updated cells
themselves and all cells that depend on them.
Given the environment  corresponding to the final value of the original
spreadsheet s, we start with a pre-evaluation environment s that differs from 
simply by resetting all cells affected by the update s to ?. As mentioned above,
these locations—call them cellss —are the updated cells themselves and all cells
that depend on them, i.e., all the cells that sit on a path starting on an updated cell
of DGs 0 . In symbols,

cellss D fl 0 W l  l 0 2 DGsˇs and l 2 dom sg

At this point, s is simply defined as follows:

(
? if l 2 cellss
s .l/ D
.l/ otherwise
52 3 The Traditional Spreadsheet

⊥ 25.00

⊥ 16.67 ⊥ 12/24/04 16.67 75

12/17/04 ⊥ 12.50 50 ⊥ 12/17/04 12/20/04 12.50 50 25

12/10/04 12/13/04 8.00 25 25 12/10/04 12/13/04 8.00 25 25

12/03/04 12/06/04 8 17 12/03/04 12/06/04 8 17

11/29/04 8 11/29/04 8

Fig. 3.10 Evaluation of the first update in the timecard example

⊥ 23.00

12/24/04 16.67 ⊥ 12/24/04 16.67 69

12/17/04 12/20/04 12.50 50 ⊥ 12/17/04 12/20/04 12.50 50 19

12/10/04 12/13/04 8.00 25 25 12/10/04 12/13/04 8.00 25 25

12/03/04 12/06/04 8 17 12/03/04 12/06/04 8 17

11/29/04 8 11/29/04 8

Fig. 3.11 Evaluation of the second update in the timecard example

The evaluation graph corresponding to this environment in the case of the first
update to the timecard example is displayed on the left-hand side of Fig. 3.10.
Recalculation simply applies any chosen evaluation strategy starting from s
rather than ?. If the semi-naı̈ve procedure is used, the number of iterations before
all occurrences of ? have been given an actual value is given by the length of the
longest path in DGsˇs starting from a node in dom s. The total cost of the update
is jcellss j, which, in general, is substantially smaller than the total number of cells
of interest of s (cellss ).
The result of the recalculation of the first update in the timecard spreadsheet
is shown as the right evaluation graph in Fig. 3.10. The cost of this update is 5.
The pre-evaluation graph for the second update and the result of recalculation are
shown in Fig. 3.11, with a cost of 3.
3.5 Summary 53

3.4 Explanation

A spreadsheet is a powerful aid to making decisions based on numerical data.

However, the wisdom of these decisions strongly depends on the accuracy of the
data they are based on. Raymond Panko has provided striking evidence that as
many as a third of all spreadsheets are riddled with errors of various types [Pan98].
Formulas are a common source of errors, especially when they reference other cells.
In this section, we draw our attention to the support for the explanation available
in a spreadsheet. The question we want answered is “why does this location report
this value?”. This is a question an end-user often asks herself as she examines the
effect of a formula. Some errors are caught and corrected right away, others can pass
unnoticed for a long time.
The evaluation graph corresponding to an evaluated spreadsheet is the prime tool
for providing explanations. Indeed, whenever a user wants to know why a location l
has the shown value, following the edges of EGs ./ backwards will point her to the
cells this location depends on. She can then inspect them, and maybe ask the same
question of some of them. For example, the question “Where does the value 8.00 in
cell E3 come from?” is answered by following the edges entering cell E3 backwards
(see Fig. 3.11 for example): its direct predecessors are cells A2, A3 and D3, going
one level back gives cells C3 and D2, and one more level back points to cell C2.
The evaluation graph can as easily list the cells that depend on any given location
by following the edges in the forward direction. This is not as useful an explanation
mechanism as tracing dependencies backwards, but if an error is found, it permits
easily identifying the affected cells.
Commercial spreadsheets implement a variety of explanation mechanisms, all
based on the above approach. When inspecting a formula contained in a cell, the
locations of the referenced cells as well as the cells themselves are color-coded
for easy visual inspection. This allows easy debugging of formulas, especially
when working with ranges. More powerful is a tool variously called “Auditing”
(in Microsoft Excel) or “Detective” (in OpenOffice.org Calc). It visually implements
the tracing of references along the dependency graph. For example, when invoking
Excel’s “Trace precedent” functionality from any given cell, it displays colored
arrows to this cell’s immediate predecessors in the evaluation graph. Invoking it
a second time displays the arrows from their predecessors, and so on until all
dependents are covered.
We will draw inspiration from these explanation mechanisms when studying
deductive extensions of the spreadsheet in the chapters to come. However, we will
see that the range of interesting questions will be wider.

3.5 Summary

In this section, we summarize the work done and highlight important lessons for
the design of the deductive spreadsheet. Figure 3.12 summarizes our model of a
spreadsheet.
54 3 The Traditional Spreadsheet

Values V v [numbers, strings, etc.]

Locations L l [cell references]
Operators Op op [of fixed or variable arity]
Expressions E e ::= v | l | op(e1,...,en) [formulas]
Spreadsheets S s : L→E [finite, no circular dependencies]

Fig. 3.12 Model of a spreadsheet

Extended Values V⊥ V⊥ = V ∪ {⊥} [ Values augmented with ⊥]

Environment Env ´ : L → V⊥

Fig. 3.13 Model of spreadsheet evaluation

The basic definitions underlying the concept of evaluation are expressed in

Fig. 3.13. The evaluation of an expression e in an environment  is denoted by e @ .
An evaluation strategy starts from an initially unevaluated environment ? and
progressively instantiates its cells with values. We have examined two iterative
evaluation strategies:
Naı̈ve Strategy: Defines the function ER s from environments to environments as
the parallel application of e @  to all the cells in s. Starting from ?, it iterates
the application of ER s until all occurrences of ? have been replaced with actual
values.
Semi-Naı̈ve Strategy: Defines a function EP s that only operates on the locations
where applying ER s replaces a ? with an actual value.
These procedures imply that the calculation of s is a fixpoint of both ER s and EP s
with respect to the ordering Env on environments. Since .Env; Env / is a lower
semi-lattice and both functions are monotonic and continuous over it, Kleene’s
theorem ensures us that this fixpoint is indeed calculated starting from ? and
iterating ER s or EP s . Earlier restrictions force these iterations to be finite.
The dependency graph DGs displays information concerning s using a structure
that highlights the dependencies among cells. It permits assessing the cost of calcu-
lation, quadratic with the naı̈ve strategy, but linear with the semi-naı̈ve evaluation.
It also provides support for explaining the results of calculation. The semi-naı̈ve
strategy has optimal cost.
Efficient procedures for the recalculation of s as a result of an update are critical
as updates occur more frequently than full calculations. Both the naı̈ve and the
semi-naı̈ve strategies are applicable, but the latter is again preferable.
A few points shall be kept in mind as we develop a deductive extension of the
spreadsheet:
• It shall be blend smoothly into this model in order to support traditional
functionalities.
• Efficient algorithms for re-evaluation are critical, more so than efficient
procedures for full calculation.
• Explanation facilities shall be maintained and expanded.
Chapter 4
The Relational Spreadsheet

In this chapter, we implement a first major extension of the spreadsheet paradigm

in the direction of symbolic reasoning. As we do so, we will not look at logic for
inspiration, but at databases. Indeed, we will import into the spreadsheet model
a large fragment of relational algebra, the query language of relational databases.
A group of columns, for example, can naturally be interpreted as a relation. It is
then a simple conceptual step to define the contents of other groups of columns
as a transformation on such relations: the traditional operators of relational algebra
achieve precisely this. The deductive relevance of this extension derives from the
fact that there exists a well-known interpretation of relational algebra into a simple
fragment of logic, which will be described in Chap. 5. The resulting deductions are
very limited, but we will extend them in Chap. 6. Our main challenge in this chapter
consists in designing this relational extension so that it blends smoothly into the
model of a spreadsheet outlined in Chap. 3.
This chapter is organized as follows. In Sect. 4.1, we look at the closest a
traditional spreadsheet comes to offering relational expressions: array formulas.
We incorporate these objects into our model from Chap. 3. Commercial products
do offer a limited notion of relations and a few commands to manipulate them:
we go over them in Sect. 4.2. In Sect. 4.3, we formalize relations in our spreadsheet
model and introduce the operators of relational algebra. We show in Sect. 4.4
that the evaluation infrastructure outlined in the previous chapter can be extended
to support the calculation of relational expressions. We devise efficient update
strategies in Sect. 4.5. In Sect. 4.6, we revisit the explanation mechanism and adapt
it to encompass relational values. Section 4.7 summarizes the work done.

4.1 Arrays

Arrays, or cell ranges, are a familiar concept in a spreadsheet. For example,

the simplest way to add all the values in column A from row 2 to 31 is to
enter the formula D SUM.A2WA31/: here, A2WA31 is an array. Arrays can span

I. Cervesato, The Deductive Spreadsheet, Cognitive Technologies, 55

DOI 10.1007/978-3-642-37747-1 4, © Springer-Verlag Berlin Heidelberg 2013
56 4 The Relational Spreadsheet

A B C D E
1 Monday Friday Hours To date Weekly average
2 11/29/04 8 =C2
3 =A2 +7 =A2:A4 + 4 17 =D2 +C3 =D2:D4/((A2:A4 −A2)/7 + 1)
4 =A3 +7 25 =D3 +C4
5

Fig. 4.1 Rewriting the timecard spreadsheet with array operations

several columns, as in D SUM.A2WC 31/. In Chap. 3, we treated an array as

an abbreviation for all the cells it encompasses; we will now promote arrays to
first-class objects.
Not as well known is the fact that most commercial spreadsheets make available a
mechanism by which a formula can evaluate to an array, i.e., produce a value not just
for the cell it resides in, but for a whole array of cells around this location. These
are called array formulas. Not many users are aware of this possibility because
they are unlikely to stumble onto it without reading the manual—something few
users do. For example, just entering an array formula in Microsoft Excel requires
an unintuitive sequence of actions sealed by the special “CTRL-SHIFT-ENTER”
key combination. Moreover, as if to discourage even the advanced user, modifying
an array formula can be rather cumbersome, in particular since the clever cut-and-
paste functionalities one is used to do not work within array formulas. It is our
belief that all these limitations stem from an immature extension of the otherwise
excellent user interface. As a by-product of Part II, we will propose a more natural
design for discovering, defining and updating the array formulas of commercial
spreadsheets.
Let us illustrate the notion of array formula by rewriting our familiar timecard
example using these expressions. Figure 4.1 shows one of the possible ways to do so.
It is useful to compare it to our original timecard example in the top part of Fig. 3.1.
A glimpse reveals that two of the formulas are associated to ranges of cells rather
than individual locations. We examine the simpler one in column B first. While in
Fig. 3.1 each of the cells Bi (for i D 2 : : : 4) contained the formula D Ai C 4, now
the whole cell range B2WB4 contains one formula: D A2WA4 C 4.1 Note that one
of the operands of C is now an array. Each location in the output range B2WB4 is
evaluated by adding 4 to the corresponding location in the input range A2WA4.2 The
more complicated formula in the range E2WE4 is interpreted essentially in the same

1
For a reason we do not completely understand, Microsoft Excel surrounds array formulas with
braces, therefore writing our D A2WA4 C 4 as fD A2WA4 C 4g.
2
To be more precise, 4 is implicitly promoted to the three-element array .4; 4; 4/ and C is
interpreted as an array version of traditional addition: it adds its arguments component-wise and
writes the result in the cell with the same relative index. All the operands are expected to have the
same dimensions, although arrays that are not long or wide enough are automatically extended by
default values (often blank) while results that are too big for the target range are truncated.
4.1 Arrays 57

way: the subformula A2WA4 A2 subtracts the value in A2 from each cell in A2WA4.
The result is divided by 7 component-wise and then incremented by 1. The cells in
this intermediate array are finally used to divide the corresponding values in D2WD4.
Evaluating this variant of our original spreadsheet displays the exact same results
already seen at the bottom of Fig. 3.1. In a commercial spreadsheet application such
as Excel, we could have made an even more aggressive use of array formulas by
writing D A2WA3 C 7 in A3WA4, and D D2WD3 C C 3WC 4 in the range D3WD4.
Differently from the instances shown in Fig. 4.1, in both of these cases the input
array overlaps the output array. At first sight, this may appear to be a circularity.
However, individual cells never rely on their own value, either directly or indirectly.
Hence there is not circularity at the level of the elements of the array. Because
such overlaps are admitted in Excel, evaluation cannot proceed in an array-at-a-time
fashion (since the contents of A4 depend on A3, which is part of the output array),
but rather cell-at-a-time as in Chap. 3.
With our upgraded example as a guideline, we will now extend our spreadsheet
model from Chap. 3 to encompass arrays and array formulas. Some general
definitions first.

4.1.1 Generic Arrays

Each element in an array is identified through an index that we will generically

denote by , variously adorned. The admissible indices of an array are given as
an index set, denoted by I . We call a set of admissible index sets an index space,
written I. Therefore, an index  is an element of an index set I , itself an element of
an index space I.
Spreadsheet arrays are invariably based on the index space I D fŒ1::j   Œ1::k W
j; k 2 NC g, whose index sets are all the rectangular intervals Œ1::j   Œ1::k for each
positive natural number j and k. The numbers j and k define the geometry of the
index set Œ1::j   Œ1::k.3 Note that neither j nor k can be zero, which would lead
to degenerate arrays with no elements.
Given an index set I 2 I and some set X , an I -array xE over X is a mapping

xE W I ! X

that associates an object in X to each element  2 I . We write xE Π2 X for the

element of xE at index . For succinctness, we will refer to a .Œ1::j   Œ1::k/-array

3
In the rest of this book, we will assume the bidimensional array structure commonly found in
spreadsheets. This work can however be generalized to higher dimensions, with index sets defined
as hyperintervals Œ1::j1   : : :  Œ1::jd  for some dimension d . Then, the geometry of these sets
would be given by the dimension d together with the tuple .j1 ; : : : ; jd /. Further generalizations
are possible.
58 4 The Relational Spreadsheet

as a j k-array. We will generally omit this prefix altogether and speak of an array
over X when the index set is unimportant or easily reconstructible from the context.
In that case, we recover the index set of xE as IxE . We denote the set of all I -arrays
on X by X I . We denote the set of all arrays on X by X I , i.e.,
[ [
XI D XI D fxE W I ! X g:
I 2I I 2I

We call the elements of X scalars.

Three geometries are particularly important:
• A j 1-array is called a horizontal or row vector. It will occasionally be
convenient to write a j 1 row vector xE as the ordered list of its constituents,
E 1 D xi for i D 1::j . Then, j will be called the
i.e., as .x1 ; : : : ; xj / where xŒi;
length of xE and denoted by jxj. E Given two row vectors xE and xE 0 of length j and
j 0 respectively, the concatenation of xE and xE 0 , denoted by .x; E xE 0 /, is the vector of
length j C j 0 whose first j elements are contributed by xE while the rest is taken
from xE 0 . Formally, .x; E xE 0 / D .x1 ; : : : ; xj Cj 0 / where xi D xE Œi  if 1  i  j and
xi D xE 0 Œi  j  if j C 1  i  j C j 0 .
• A 1k-array is similarly called a vertical or column vector. All the above
definitions for row vectors have a counterpart for column vectors. We use
the same list symbology and the same notation for concatenation. We will
drop the row or column qualification when the orientation of a vector is clear
from the context or unimportant.
• Besides being both horizontal and vertical, 11-arrays, of the form Œ1::1 
Œ1::1 ! X , are especially important. Indeed, X 11 is isomorphic to X since
the index set Œ1::1  Œ1::1 contains a single element .1; 1/. This observation will
allow us to implicitly identify a scalar with the corresponding 1-element array,
when convenient.
Given an I -array xE and a subset I 0  I of its index set I (note that it may be the
case that I 0 62 I), the image of I 0 according to xE , written xŒI
E 0  D fxŒ
E W  2 I 0 g,
is in general just a set of elements of X . It is however a subarray of xE whenever I 0
can be mapped into an index set tr.I 0 / 2 I by means of an appropriate translation
function tr. In our case, subarrays will be rectangular subintervals of I , and the
translation function will realign their first coordinates to .1; 1/. In Fig. 4.1, the
range A3WA4 is a subarray of A2WA4. We will be particularly interested in vectorial
subarrays.

4.1.2 Spreadsheet Arrays

These general definitions immediately give us a notion of array of values V I ,

E
with elements Ev, and array references (or array of locations) LI , with elements l.
4.1 Arrays 59

Locations and (now scalar) values are defined as in Chap. 3. The notation A2WA4
in Fig. 4.1 is a commonly found abbreviation for the 13-array of locations
f.1; 1/ 7! A2; .1; 2/ 7! A3; .1; 3/ 7! A4g. Similarly, location array C 2WC 4
contains the array of values f.1; 1/ 7! 8; .1; 2/ 7! 17; .1; 3/ 7! 25g.

4.1.3 Array Operators

Operators can now take arrays as arguments: the classical example is SUM, which
can have up to 30 arguments, each of which can be either an array or a scalar (which
we uniformly interpret as a 11-array). Operators can also return an array: examples
are “C”, “=” and “” in Fig. 4.1. We upgrade our definition of operator from Chap. 3
to take a fixed or variable number of arrays as arguments (its arity) and return
an array as a result. We denote a generic operator by op E (we still write op when
the returned object is a scalar). Again, we liberally make use of infix notation and
common precedence rules.
Similarly to the scalar case, each operator op E of arity n is associated with
an operation opb
E that maps n argument values (possibly arrays) to a result value
(possibly an array). Note that this definition does not impose geometry constraints
on the values accepted and returned by an array operation. Indeed, in commercial
spreadsheets many operators such as “C” accept arguments of varying geometry
and therefore the corresponding operation shall return appropriate results for each
of these geometries. For example, although the operation C O is intuitively expected
to take two arrays of j  k values each and returns a j k-array of values obtained
by adding the input arrays component-wise, C O is much more liberal in actuality: the
arrays given as arguments do not need to have the same geometry, nor shall they
have anything to do with the expected geometry of the result: the input arrays are
truncated or extended with default values to match the latter. Operators that are not
as foolproof return the error value, , possibly in an array form E.
Many array operators are mere component-wise extensions of familiar scalar
operators: “C” is an example. Indeed, any scalar operator op W E n ! E taking
purely scalar arguments can be promoted to an array operator op E W .I ! E/n !
.I ! E/ defined by op. E xE1 ; : : : ; xEn /Œ D op.xE1 Œ; : : : ; xEn Œ/ for each  2 I . Many
array operators do not fit into this mold however. Indeed, some scalar operators
natively work on arrays: SUM is one of them. Furthermore, some operators
returning arrays cannot be reduced to the component-wise extension of a scalar
operation: for example, Excel’s TRANSPOSE is a pure array-valued operation.
Excel also provides a seldom used way to build an array extensionally, i.e., directly
from scalars. Example operators using all possible combinations of arrays as input
to and output of operators are shown in Fig. 4.2.
60 4 The Relational Spreadsheet

Output
Input
Scalars Arrays

Scalars +, IF, . . . (extensional arrays)

Arrays SUM, COUNT, . . . +, TRANSPOSE, . . .

Fig. 4.2 Example of scalar and array operations in commercial spreadsheets

4.1.4 Array Formulas

As in the scalar setting of Chap. 3, arrays of values, arrays of locations and array
operators are the basic ingredients for writing array expressions. These objects are
therefore defined just as in the scalar case:

Array expressions: eE WWD Ev j lE j op.E

E e1 ; : : : ; eEn /

Note that this subsumes scalar expressions once we identify scalars with 11-arrays.
By a conceptual extension of our conventions, we denote the -th component of an
expression eE by eEŒ. We will soon see that any such eEŒ can be simulated by a
scalar-valued expression e .
The subexpression relation is defined as in Chap. 3, with one added case: an
E v xE for every  2 I ,
array element is a subexpression of an array. Formally, xŒ
where xE is an I -array. For example, A2 is a subexpression of A2WA4. We still write
this relation as v and @ for its strict version. Both relations naturally extend to
subarrays. For example, A2WA3 is also a subexpression of A2WA4.

4.1.5 Component Expressions

In Chap. 3, the cell dependency in a spreadsheet s was defined on the basis of

the referential closure @s of the subexpression relation @. The same will apply in
the presence of arrays, but for an upgraded definition of referential closure. A few
considerations are in order as we prepare to formalize this concept.
A purely syntactic notion of dependency, similar to what we used in Chap. 3,
implies that cell B2 in Fig. 4.1 depends on A3 because B2 co-hosts the array
formula D A2WA4 C 4 which has A3 as a subexpression. However, the contents of
A3 cannot influence the value of B2. As we saw at the beginning of this section,
commercial spreadsheets rely on a finer notion that tracks dependencies at the
component level rather than array-wide: indeed, the value at cell B3 in Fig. 4.1 only
depends on the value of location A3 and the number 4, even if the array formula
4.1 Arrays 61

that contributes to it also mentions A2 and A4. In general, other operators will force
other dependencies; for example, if B3 contained SUM.A2WA4; 4/, its value would
depend on all the cells mentioned in it. Therefore the notion of dependency at the
component level becomes semantic rather than syntactic as when considering scalar
expressions (or array formulas as a whole): each operator follows its own set of
rules.
Let us formalize these ideas. Given an n-ary operator op, E we write .i /  op
E to
E i -th argument (for i D 1::n) can influence the value calculated
indicate that op’s
by this operator: i is then a dependent argument of op. E An operator invariably
depends on all of its arguments.4 This changes when lowering this notion to the
individual components of an argument and/or the result. We then write .i / 0 op E
to indicate that the -th component of argument i can influence the value of the
0 -th component of op.E This relation is not always true: we have seen in the case
of addition that .1/ 0 C and .2/ 0 C iff  D 0 , that is, the -th component of
an array addition depends only on the -th component of its arguments. Generally,
the returned component at index  of an I -valued operator op E will depend only on a
subset of its input components, independently from the actual input values.
In a commercial spreadsheet, the use of array formulas can always be simulated
by scalar-valued operators. Indeed, for every array operator op E and every index  in
the output geometry of op, E there exists a scalar operator, which we indicate as opŒ,
E
that emulates the behavior of op E at index . The arguments of opŒ
E will be all the 0
components of the each argument .i / of op E such that .i / 0 op.
E We have already
seen that the behavior of array addition at index  is simply the scalar addition of
the -th component of its arguments. We can now lift this notion to the level of
expressions, obtaining a scalar formula eEΠthat emulates the behavior of eE at each
index  2 IeE:

E
lΠD E
lΠ(the -th component of l) E
EvΠD EvΠ(the -th component of Ev)
E e1 ; : : : ; eEn /Œ
op.E D E
opŒ.e 1 ; : : : ; em / where eij D eEi Œj  and .i / j  op
E

This notion is natively defined in the case of array of locations and array of values.
In passing, we can lift the component-level dependency to the level of
expressions: e is a dependent subexpression of eE at , written e eE, if the following
holds:

e lE iff E
e D lŒ
e Ev iff e D EvŒ
(
e D eEi Œ0  and .i / 0 op
E
E e1 ; : : : ; eEn /
e op.E iff
or e eE0 and eE0  op.E
E e1 ; : : : ; eEn /

4
While the laws of algebra certainly do not require this, it is a common and convenient practice:
it makes no sense to force a user to write arguments that never influence the result of an expression.
62 4 The Relational Spreadsheet

Again looking at Fig. 4.1, A3 Œ1;2 .A2WA4 C 4/—the second component returned
by D A2WA4 C 4, in location B3—but A4 6 Œ1;2 .A2WA4 C 4/. Notice that in the
scalar world, l v e implied l  e. Now, although l v eE, it may not be the case that
l  eE for all .

4.1.6 Spreadsheets with Arrays

Given these lengthy premises, we can now redefine a spreadsheet as a mapping

between arrays of locations and array expressions, with scalars being a particular
case, with some dependency restrictions outlined below. Each location in the
spreadsheet must participate in exactly one such location array and consequently
host at most one expression. More formally, a spreadsheet consists of two
components:
E each with its
1. A partition L of the set L of all locations into disjoint arrays flg,
own geometry IlE. This is formally defined as a function L W L ! LI that maps
each location l to an array lE such that l 2 l.
E We will often write L extensionally
E
as flg.
2. A map s W flgE ! E I that associates an array expression eE to every element lE of
this partition.
We will generally refer to a spreadsheet .flg; E s/ by simply mentioning the second
component, leaving the reference to the partition flg E implicit. We will often write s
as flE 7! eEg. It is expected that the geometry of each array of locations lE will be equal
to the geometry of the corresponding array expression eE. As we said, the geometry
of eE is automatically adjusted to the geometry of lE in practice.
Since the structure underlying a spreadsheet has changed, we need to update the
associated notion of referential closure of a relation , which we used to define
the concept of dependency in Chap. 3. Because of the new form, we have two
candidates:
Array-wide referential closure: Given .L; s/, this first variant is the relation 
Es
given by:
• s.L.l//  E s l for all l 2 L;
• eE 
E s eE0 if eE  eE0 ;
• eE 
E s eE00 if eE 
E s eE0 and eE0 
E s eE00 .
The first line relates a location l to the entire expression s.L.l// it co-hosts.
There is a dependency from l to l 0 according to  E s if l 
E s l 0 . In particular, s has
a circular dependency with respect to E s if l E s l for some l 2 L.
Applying this definition to the subexpression relation @, has the effect of chasing
the subexpressions of an array formula through the locations they mention.
For example, in Fig. 4.1 we have that A2 @ E s B2, but also A3 @ E s B2 because
4.1 Arrays 63

A2WA4 @ E s B2, and A2 @ A2WA4 and also A3 @ A2WA4. While a plausible basis
for defining the notion of dependency, it does not correspond to what commercial
spreadsheets implement.
Component-wise referential closure: In the same context, the second variant is
the relation s given by:
• s.L.l//Œ s l for all l 2 L, where l D L.l/Œ;
• eE s eE0 if eE  eE0 ;
• eE s eE00 if eE s eE0 and eE0 s eE00 .
Differently from above, the first line relates a location l only to the portion of the
expression s.L.l// that influences the value of l. The notion of cell dependency,
l s l 0 , and circular dependency, l s l for some l 2 L, is defined as in the
previous case.
This definition is much more discriminating than the previous array-oriented
version: upon encountering a location l, it pursues the search just in that cell
rather than in the whole location array it participates in, i.e., s.L.l//. Therefore,
it will follow the scalar projection eEΠof any expression eE appearing in that range.
Since eEΠmentions only the dependent subexpressions of eE at , this will result
in a much more focused set of dependents.
At this point, a well-formed spreadsheet is a spreadsheet .L; s/ subject to the
same two properties as scalar spreadsheets:
1. Finitely many cells in the domain of s are non-blank.
2. s does not contain any circular dependency with respect to @s (although there
may be circularities along @E s ).
All the spreadsheets we will consider shall be well-formed. Therefore we will drop
this qualifier.
Let us now take the point of view of the individual cells. Saying that a spreadsheet
flE 7! eEg associates the partition element lE with the array expression eE simply means
E in lE is associated with the -th component of eE. We have
that every location lŒ
seen that there exists a scalar formula, eEŒ, that is equivalent to eE at index .
Viewing flE 7! eEg as a map that associates each location lΠE to the expression
component eEΠhas the effect of recasting our upgraded notion of spreadsheet back
into the mold of our original definition from Chap. 3: fl 7! eg. This means that
we can reuse with minimal modification the entire evaluation infrastructure set up
in that chapter. The naı̈ve and semi-naı̈ve strategies apply without any alteration.
The changes to dependency graphs and related notions are limited to simply using
the component-wise referential closure of @ in its definition. As a result, the purely
scalar timecard spreadsheet in Fig. 3.1 and its re-expression using arrays in Fig. 4.1
share the same dependency graph, which we showed in Fig. 3.4. When equipping it
with the component subexpression at each location, we obtain precisely the graph
in Fig. 3.5, which means that stepwise evaluation proceeds in the exact same way
as with the original spreadsheet. This trivially explains why both versions of the
timecard spreadsheet produce the same output.
64 4 The Relational Spreadsheet

4.1.7 Arrays Versus Scalars

As we just saw, the use of array formulas can always be simulated by scalar-valued
operators: in most cases, their effect is achieved by copying a scalar formula over
a number of rows or columns. Indeed, the example in Fig. 4.1 is equivalent to
our original spreadsheet in Fig. 3.1. More rarely, as when using TRANSPOSE,
the transformation needs to be manual. Therefore, array formulas are essentially an
abbreviation for scalar formulas. Using array formulas has a minimal computational
advantage, mostly at the time of parsing, since each expression appears exactly
once rather than being replicated multiple times. This becomes an issue only with
very large spreadsheets, or very slow computers. Array formulas currently impose
a penalty in terms of usability: entering an array is somewhat unintuitive, but more
importantly, modifying an array is as hard as deleting it and re-entering it from
scratch. This is acceptable for archival work, but not for the typical spreadsheet that
needs to be updated frequently. We do not see any reason for this overhead and
attribute it to a design that has not reached maturity. We will propose an alternative
design in Part II.
Given this analysis of arrays, Chap. 3 characterizes rather exhaustively the
traditional spreadsheet from a programming language perspective. At its core,
a spreadsheet embeds a simple functional language without recursion.5 What is
evaluated is the individual cell, even in the presence of array formulas, and this
value is a function of the surrounding locations. A cell is indeed analogous to a
pocket calculator with nearly unlimited memories.
In spite of this humble characterization, commercial spreadsheets are one the
most useful and pervasive computing applications. They are used daily across our
society and can hold anything from a grocery list to a multi-million dollar budget.
The key to their success is the coupling of the basic framework above with a
nice user interface, a number of productivity tools, and a rich set of operators.
A particularly important aspect of the interface is its ability to display a multitude of
values at once, hence allowing extensive visual feedback and simplifying decision
making.

4.2 Relational Support in the Traditional Spreadsheet

The tabular layout of a traditional spreadsheet naturally lends itself to interpreting

groups of columns as relations. A row is then seen as a record of related
values, and the values in each participating column are understood as having
the same meaning. Take our timecard example in Fig. 3.1 one last time:
.12/06/04; 12/10/04; 17; 25; 12:50/ on row 3 relates the date of the Monday

5
More precisely, a bounded form of recursion, disabled by default, is supported.
4.2 Relational Support in the Traditional Spreadsheet 65

(12/06/04) and the Friday (12/10/04) of a week, the number of hours worked that
week by an employee (17), the cumulative number of hours worked as of that Friday
(25), and the weekly average as of the end of that week (12.50). All the rows in
the timecard example (except row 1 that contains captions) share this interpretation.
Notice that some of these fields are input by the user (e.g., 17) while others are
calculated on the basis of other fields.
Commercial products provide a limited toolkit for supporting this relational
interpretation of a region of the spreadsheet. Microsoft Excel, for example,
recognizes this intent when the user enters textual captions in the top row of a
new worksheet and then inputs the first record. Excel then interprets this group
of columns as a “data list” and makes available a number of commands to
manipulate it:
• The user can bring up a form for inputting new records or modifying old ones.
This form shows the caption of each column in the data list next to an input box,
which either contains a preexisting value, or is blank—an example can be seen
in Fig. 7.4. This device aids the user by abstracting away all but the record he is
working on. Any change in the structure of the data list (e.g., adding, deleting or
moving a column) is automatically reflected in the form.
• The user can filter a data list in place by instructing Excel to hide from view
all records that do not satisfy a given condition. In our timecard example, a
user could ask to see only the records where the employee worked less than
8 hours, for instance. Filters are set up using a variety of methods, depending on
the complexity of the condition and the sophistication of the user. One of these
options allows saving the filtered data in a new area of the spreadsheet rather than
just hiding the tuples that do not satisfy the condition.
• The data list can be lexicographically sorted in place.
• A number of “database functions” are available to conditionally perform opera-
tions such as SUM, on a data list. For example, DSUM takes three arguments:
a relation, the column whose values should be added, and a criterion which
specifies which records should be taken into account. This last field is expressed
as a reference to a cell range containing the specification of the criterion itself.
• A set of columns can be imported from an external application, either a database
or another spreadsheet. This is the only other way of creating a new relation.
It picks columns from the external application, arranges them in some specified
order and optionally filters the resulting records. This is rather cumbersome in
that importing a relation requires defining it, saving it into a temporary index file,
and finally loading it into the spreadsheet.
Other commercial spreadsheet applications offer similar concepts and commands.
For example, relations are called “database ranges” in OpenOffice.org’s Calc, and
variants of the above actions are available to manipulate them.
Relations are an important concept in the context of the deductive spreadsheet
because they can naturally be interpreted as logical entities: simplifying our
example, the presence of the record .12/06/04; 17/ in the timecard relation can
indeed be understood as indicating that it is a true fact that the employee worked
66 4 The Relational Spreadsheet

17 hours during the week of 12/06/04. Records not appearing in the relation are
facts that are not true. Symbolic reasoning establishes new true facts on the basis of
facts that are known to be valid. This corresponds to constructing new relations
on the basis of existing relations. These new facts can themselves participate
in further deductions, i.e., contribute to further relations. We will formalize this
correspondence in Chap. 5 and extend it in Chap. 6.
As we just saw, commercial spreadsheets do provide a selection of tools to work
with relations. Among them, the support for defining new relations is very limited,
however: a relation can be built manually (possibly using a form), or it can be
the result of filtering existing data, or it can laboriously be imported from another
application. All of these options are strictly interactive: a user must initiate them.
More importantly for our purposes, neither of the two methods that create a relation
out of existing relations has the ability to propagate updates automatically: adding
or modifying the source data in an external application has no effect on previously
imported records. Similarly, changes to the records of a data list that has been used
to produce a relation by filtering are not reflected in the latter. Updates must be
propagated manually.
This is in stark contrast with the spreadsheet formulas we have considered so far.
A formula is bound to a cell or cell array even after it has been evaluated, so that
any change in a value it depends on is immediately reflected. Relational commands
are not bound to the relations they calculate: once the output has been produced,
they are forgotten. It is indeed on purpose that we referred to them as “commands”
rather than “operations”. Operations, as they appear in formulas, are first-class
entities: they are an inherent part of a spreadsheet, both during normal usage and
when saved on disk. The relational actions are instead second-class in Excel and
other commercial products: they act on a spreadsheet as would “Print” or “Save”,
but are not part of it. A consequence of this fact is that relational commands do not
participate in the evaluation process of a spreadsheet in the same way as formula
operators do. In particular, they do not fit in the evaluation model we introduced in
Chap. 3.

4.3 Extending Spreadsheets with First-Class Relations

We will now propose an extension to the spreadsheet paradigm that supports a

first-class notion of relation. This will allow us to write expressions that when
evaluated produce a relation, which will be automatically updated any time this
expression’s input changes. This section extends the model of the traditional
spreadsheet defined in Chap. 3 (and amended with arrays in Sect. 4.1) to support
these concepts. We delay discussing the evaluation of relational expressions until
Sect. 4.4.
We formalize relations into our spreadsheet model in Sect. 4.3.1. We define the
notion of relational expression in Sect. 4.3.2: we first examine the interplay of
arrays and relations, then import operators from relational algebra as the foundation
4.3 Extending Spreadsheets with First-Class Relations 67

of relational expressions, and finally define array variants for some of them.
Altogether, this substantially extends the range of manipulations currently available
in commercial spreadsheets with useful new tools. These concepts are brought
together in Sect. 4.3.3 where we define the relational extension of the spreadsheet.
Having paralleled our work on the traditional spreadsheet, Sect. 4.3.4 stops and
reflects on the differences between this relational extension and the main application
of relational algebra: database query languages.

4.3.1 Formalizing Relations

In this section, we formalize the notion of relation discussed above. As we do so,

it will be useful to refer to a new example to clarify concepts as we introduce them.
For this purpose, Fig. 4.3 shows a spreadsheet containing data that are relevant to
the registrar of a fictitious university. For typographical reasons, we have broken it
into four parts, each of which will be interpreted as a separate relation.
• The first few columns, displayed at the top of the figure, contain data about the
students: columns A and B contain their first and last names, while column C
contains an improbable student identifier, given by concatenating the first and
last name of each student with a space in between (“&” is indeed Excel’s syntax
for string concatenation). For illustration purposes, we rely on an array formula
to perform this calculation. We have extended the traditional syntax by using the
underscore (“ ”) to denote the last row in a column. Therefore A3WA denotes all
the cells in column A with the exclusion of A1 and A2, and similarly for B3WB .
• Columns H , I and J in the central portion of Fig. 4.3 contain information about
the courses offered, more specifically the name of the course, an abbreviation,
and the name of the instructor. A more realistic setup would place this informa-
tion on a different sheet, or even a different spreadsheet altogether. For simplicity,
we adopt the more rudimentary approach in the figure.
• The next set of columns, at the bottom left of the figure, records the courses taken
by each student together with the semesters they were taken and the resulting
grades. For the sake of illustration, we have used both literal values and formulas
to refer to the students in column P .
• Finally, the bottom right portion of Fig. 4.3 associates each grade with a number
that could be used, for example, to compute a student’s GPA.
Notice that there is nothing new in this example: this spreadsheet uses only values,
scalar expressions, and array formulas. We could have written it in Excel or Google
Spreadsheets. Each group of columns above is naturally interpreted as a relation,
which is again compatible with commercial spreadsheets.
We will now introduce relations within our formalization of a spreadsheet from
Chap. 3. This will not affect the way the example in Fig. 4.3 is modeled, but will
enable us to extend it in sections to come. The example itself will help explain some
of the definitions.
68 4 The Relational Spreadsheet

A B C ···
1 Students
2 First Last Student
3 Sue Smith
4 Alan Bond
5 Nick Vitter =A3:A & ” ” & B3:B
6 Sue Lee
7 ··· ···

··· H I J ···
1 Courses
2 Name Abbrv Instructor
3 Operating Systems os Mary Vaught
4 Basic Networking bn Emil Wyre
5 Advanced Networking an Judy Less
6 Computer Architecture ca Emil Wyre
7 Theory of Computation tc Mary Vaught
··· ··· ···

··· P Q R S ··· ··· Y Z

1 HasTaken Grades
2 Student Course Semester Grade Letter Number
3 Sue Smith os S03 A- A 4.00
4 =C3 tc F04 B+ A- =Z3 −0.25
5 Nick Vitter os F03 A- B+ =Z4 −0.25
6 Alan Bond ca F03 A- B =Z5 −0.25
7 Alan Bond an S04 C+ B- =Z6 −0.25
8 =C5 bn S04 B C+ =Z7 −0.25
9 Sue Smith an F03 B- C =Z8 −0.25
··· ··· ··· ··· ··· ···

Fig. 4.3 A simplified registrar’s spreadsheet

Generic Relations

As in the case of arrays, we start with a few general definitions that we will later
instantiate to appropriate spreadsheet entities. Given a vectorial index set I (i.e., an
index set of the form Œ1::j   Œ1::1 or Œ1::1  Œ1::k), an I -relation xN over X is a
subset of X I , the set of all I -vectors over X :

xN  X I

We will refer to xN simply as a relation when the index set I it is based on

is unimportant or easily derivable from the context. Consistently with common
terminology, we call the common length of the vectors in xN the arity of x.
N Each of
the shown areas in the spreadsheet in Fig. 4.3 is a relation over horizontal vectors.
The first two are ternary, the third is 4-ary, and the last binary. We will not be
4.3 Extending Spreadsheets with First-Class Relations 69

interested in nullary relations. We will however admit the empty relation which
contains no element. We denote it by ;, or ;I when the particular vectorial index
set I is important.
We write XN I for the set of all I -relations over X : therefore XN I is the powerset
of X I . The set of all relations over X , for any vectorial index set I 2 I, is then
denoted by XN I , or simply XN when I is fixed, as when working with bidimensional
spreadsheets.
We call an element xE 2 xN a record, or sometimes a tuple. For clarity, we will
alternatively indicate a record with the letter r, possibly subscripted. Each of these
objects is a vector in X I . For practical reasons that will become apparent shortly,
we will not consider a vector of blank values, E[, a record. We will frequently make
use of the list notation .x1 ; : : : ; xjEx j / introduced in Sect. 4.1. For example, .A; 4:00/
is a record in the bottom right relation in Fig. 4.3. Each of the indices i D 1; : : : ; jxj E
is called an attribute and is used to positionally identify the element xi inside a
tuple. The objects occurring at the same index throughout a relation generally have a
coherent interpretation which in a sense is the meaning of the attribute. For example,
the values in column A are first names in Fig. 4.3, while the values in column R are
semester acronyms. We use the letter a, variously adorned, to denote an attribute.
Having defined an attribute as a vectorial index a in the index set of a relation
N the extensional attribute of xN at a, denoted by xŒa,
x, N is the set of all xE Œa for all
xE 2 x,
N symbolically xŒa
N D fxŒaE W xE 2 xg. N We will drop the qualifier “extensional”
when clear from the context. The (extensional) attributes of a relation xN is then the
set of all extensional attributes of x, N i.e., Attr.x/
N D fxŒ
N W  2 IxN g.
A relation naturally lends itself to a tabular representation. This is clearly
demonstrated in the four relations in the spreadsheet shown in Fig. 4.3 once fully
evaluated. Each record occupies a separate row, with attributes corresponding to
columns. Our definition also supports the reverse orientation, i.e., relations with
vertical records and attributes along the rows, an option unavailable in commercial
products. Note that in commercial products the commonly used tabular topology of
a relation is often just a stepping stone for a more convenient rendering: numerical
relations are already displayable using a variety of charts. Additional forms of
representation are naturally associated with symbolic data: for example, a relation
describing course requirements can conveniently be displayed as a workflow graph.
We will come back to these issues in Part II of this book.

Spreadsheet Relations

The general definition of relation immediately specializes to the notion of relational

N where V was the set of all scalar values
value: a relational value vN is an element of V,
in a spreadsheet. The central portion of Fig. 4.3 is a relational value. Five records are
shown, although the ellipses suggest that there may be more. Evaluating a relation
will always produce a relational value.
70 4 The Relational Spreadsheet

Unlike with arrays, we will not need to define a notion of relational reference.
Indeed, array references served the purpose of denoting (rectangular) collections
of cells, both within formulas and as formula containers in the definition of
spreadsheets themselves. Because of the tabular interpretation of relations, this
very same mechanism can be used to denote an area of the spreadsheet to which
we want to give a relational interpretation. Indeed, we will refer to the topmost
relation in Fig. 4.3 as A3WC both within a relational expression (defined in the next
section) and for the purpose of formalizing the notion of the relational spreadsheet
(in Sect. 4.3.3). Similarly, we may refer to the second attribute of this relation as
B3WB .6
It is common practice to assign symbolic names to relations rather than refer
to them through array references. We have done so in the first row of Fig. 4.3 by
merging the appropriate number of cells for each relation (a feature supported by
all commercial spreadsheets) and writing a relevant identifier into it. In this way,
we can conveniently refer to the second relation in that figure as “Courses” rather
than through the obscure array reference H 3WJ . While this certainly makes for a
leaner discussion, it also enables us to use these symbolic names within relational
expression thanks to the existing mechanism of range names, by which a cell or cell
range can be assigned a mnemonic name which can take its place in any formula.
We will make an intuitive use of range names for the time being, but come back
to them in Part II. We rely on the same device to associate symbolic names to
attributes: this allows us to refer to the first attribute of Students in column A
as “First” rather than 1 or A3WA . The second row in the spreadsheet in Fig. 4.3
provides symbolic names for each attribute. Note that distinct attributes can have the
same name. In our example, Student is an attribute of both Students and HasTaken.
Whenever needed, we will disambiguate them by prefixing the name of the relation
to the attribute separated by a dot: Students:Student and HasTaken:Student. These
are called qualified names.

Relations Versus Arrays

A relation is substantially different from a bidimensional array although both map

to similar regions of a spreadsheet. Consider the case of relations with horizontal
records and columnar attributes (the arguments below also apply to the dual
orientation).
The crucial observation is that the rows (and columns) in an array are ordered,
while a relation is a set of horizontal records. The fact that a relation is a set
has two important consequences: first, the order of its elements is unimportant;
second, duplicate records are treated as one. Neither characteristic is particularly

6
Properly speaking, A3WC is a reference to the relation contained in this group of cells, and
similarly B3WB is a reference to its second attribute. While it is important to keep in mind the
distinction, we will often blur it in actual discussion.
4.3 Extending Spreadsheets with First-Class Relations 71

meaningful in the examples in Fig. 4.3, but their consequences will become evident
when considering relational expressions based upon them: indeed, the order of the
records in the calculated relation will be unpredictable in general, and there will not
be duplicated records.
Another difference between relations and arrays concerns the number of objects
they contain. Arrays are sequences of a fixed finite number of rows (or dually
columns): their extension is determined at the time they are defined and is very
hard to change after that. Instead, we will think of a relation as containing a variable
but generally unbounded number of records. The extension of a relation will often
be expressed in terms of the last row of the spreadsheet (as in A3WC above) and
therefore appear unlimited for all but the most expansive computations. In general,
only an initial segment of a relation will contain actual data—the rest being blank.
For this reason, we required that a record always contain at least one non-[ element.
While unbounded relations are particularly common, nothing will prevent us from
fixing a priori the maximum number of records.
Maybe the most striking difference between relations and arrays is that the former
will be treated as a unit during evaluation. As we saw in Sect. 4.1, an array is atomic
for editing, but each component is evaluated individually. It will not be possible,
in general, to reduce the evaluation of a relational expression to the evaluation of
individual records.

4.3.2 Relational Operators and Expressions

Following the methodology we developed in the case of scalars and arrays, our next
step shall consist in introducing a notion of relational operator that can be used to
build relational expressions. As we do so, we shall carefully engineer the interface
between this new representational paradigm and the familiar infrastructure in order
to obtain a language that is not only consistent but also seamlessly integrated. The
resulting classes of operators and their interactions can be previewed in Fig. 4.4.
This section will be dedicated to explaining this diagram and defining the entities
that appear in it.
In Chap. 3, we adopted arithmetic functions and other operators that when
evaluated produced a scalar (e.g., C and &). In Sect. 4.1, we similarly made
available operators drawn from an algebra of arrays (e.g., component-wise addition
and TRANSPOSE). We also drew bridges to the scalar world through operators
such as SUM and COUNT. This was summarized in Fig. 4.2 and is reported in the
upper-left two-thirds of Fig. 4.4 (save for the second line of the central cell).
In order to work with relations, we draw our operators from relational algebra,
which however covers only the lower-right corner of Fig. 4.4. The remaining
quadrants of the figure, and a good part of this chapter, are occupied by mechanisms
that take a relation as input and return a scalar or an array, and dually that build
a relation from scalars and arrays. While it is conceivable to leave the set of such
bridging operators open-ended (in the same way as a potentially unbounded number
of SUM-like operators can be defined to go from arrays to scalars), we adopt a more
72 4 The Relational Spreadsheet

Output
Input
Scalars Arrays Relations

Scalars +, &, . . . (extensional arrays) ⇐ (via extensional arrays)

+, TRANSPOSE, ...
Arrays SUM, . . . (implicit coercion)
SORT, ∪V , ∪H , ¼a , ¾C
⇑
Relations (via arrays) (implicit choice) ∪, −, ×, ¼æ , ¾C

Fig. 4.4 Support for scalar, array and relational operations in the relational spreadsheet

disciplined approach. The passage from arrays to relations will take the form of
an implicit coercion, hence allowing relational expressions to operate on arrays.
In the reverse direction, we will define two methods for promoting a relation to
a bidimensional array. This will have the effect of making all the current array
operators available for computing with relations. Last, we will draw inspiration from
relational algebra to define a handful of new and useful array operators, displayed
on the second line of the central square of Fig. 4.4.

Interface Between Arrays and Relations

One of the requirements in Chap. 2 called for the seamless integration of inferential
extensions and traditional functionalities in the deductive spreadsheet. While we
have not introduced any apparatus for symbolic reasoning yet, the operations that
interface arrays and relations constitute the foundations of this integration. This
interface is schematically described by the next diagram. We will spend the next
few paragraphs formalizing the meaning of the labels.

(implicit coercion)

Arrays Relations

Non-deterministic ordering
(SORT)

In the previous section, we mentioned that we will rely on array references to

denote areas of a spreadsheet that we want to interpret as relations and feed into rela-
tional expressions. This is a particular case of a more general phenomenon. Consider
4.3 Extending Spreadsheets with First-Class Relations 73

a j k-array x.E Each of its rows is therefore a j 1 vector, ordered from 1 to k.

Since an ordered set of j 1 vectors is a set of j 1 vectors with an extra property
(the ordering), we can interpret xE as a j 1-relation whose records are just the
row vectors of this array. By a similar argument, an array can also be viewed as a
relation of vertical records. These considerations enable us to use an array wherever
a relation is expected. Therefore, the first production for our grammar of relational
expressions simply admits any array expression:

Relational expressions: eN WWD hE

ei j : : :

In this production, we use the angled brackets to clearly mark the interface between
arrays and relations, but we will generally omit them in practical examples. We call
hE
ei a base relation. We will complete the grammar of relational expressions in the
rest of this section.
Clearly, the geometry of the provided array and the expected relation need to
match. If they do not, two courses of actions can be taken: we can either return an
error relation (N ) of the appropriate geometry, or we can use default rules to reshape
the input or the output. Which alternative is preferable, in which circumstances,
and what are appropriate default rules, are all questions that needs to be assessed
empirically.
In order to fully integrate relations into a spreadsheet, it is important to provide a
means to pipe the result of a relational expression into an array formula. We will see
concrete examples shortly. Simply reverting the previous argument and interpreting
any j 1-relation xN as a j k-array xE for an appropriate k (and similarly for the dual
orientation) leaves one issue unresolved: how should the records of xN be ordered as
rows in the array xE ? There are two possible answers:
• We can leave the issue unresolved and freeze the records of xN in whatever order
relational evaluation returns them. The resulting ordering is non-deterministic
because there is no guarantee, in general, that two evaluations of the same
relational expression will return the records in the same order. Therefore,
any subsequent computation that depends on this order will run the risk
of returning incorrect results. On the other hand, many operators are either
commutative—SUM for example—or do not rely on the relative order of their
arguments, e.g., component-wise increment.
• Alternatively, we can resolve the issue by sorting the records of xN according to
some criterion. A common approach is to specify a lexicographic order based
on an arrangement of attributes. We denote this last operator by SORTaE Œe, N
where aE lists the attributes in eN according to the desired priority. A partial list
is automatically completed according to a fixed default policy.
This second option can be reduced to the first by introducing SORTaE as an
array operator. This has a number of advantages over making it available as an
interface function. First, SORT can then be used to sort not just relations, but also
generic arrays. Commercial spreadsheets make this functionality available as
a command, as we saw in Sect. 4.2, but not as an operator. Second, a purely
74 4 The Relational Spreadsheet

x̄1 ∪ x̄2 : x̄1 − x̄2 : x̄1 × x̄2 :

⎧ ⎧ ⎧
⎪
⎪ ⎪
⎪ ⎨
⎨ ⎨ x̄2
⎫ ⎫ x̄1
x̄1 x̄1 ⎩
⎪
⎪ ⎬ ⎪
⎪ ⎬ ⎧
⎩ ⎩ x̄2
x̄2 x̄2 ⎨
⎭ ⎭
x̄1
⎩ x̄2

¾C x̄ : ¼j x̄ : x̄1 /C x̄2 :

/
⎫ ⎧
⎪
⎪ ⎨
⎪
⎪ x̄2
⎪
⎪ x̄1
⎪
⎬ ⎩
x̄ ⎧ x̄2
⎪
⎪ ⎨
⎪
⎪
⎪
⎪ x̄1
⎪
⎭
 ⎩ x̄2
x̄

Fig. 4.5 Core operators of relational algebra (plus join)

relational SORT would differ from all other relational operations in that sorting
does not have a proper logical meaning.
These considerations allow us to extend the grammar of array expressions from
Sect. 4.1 as follows:

Array expressions: eE WWD e j Ev j lE j op.E

E e ; : : : ; eE/ j Œe
N

N non-deterministically freezes the ordering of e.

where Œe N We will generally omit the
brackets when they can be extrapolated from the context. For convenience, we have
included a redundant production for scalar expressions. Note that SORTaE ŒeN is just
E
a particular operation op.

Pure Relational Algebra

Where arithmetic computes with numbers, relational algebra computes with

relations. Therefore, a generic relational operator op N will map a certain number
of relations .xN 1 ; : : : ; xN n /, given as arguments, to a relation xN that will be its result:

N W
op XN  : : :  XN ! XN
.xN 1 ; : : : ; xN n / 7! xN

Given the fact that relations are sets of records, a computation on generic relations
can be reduced to the composition of a small number of operations that work
either on the set structure of its arguments, or on the structure and quality of
the records appearing in them. Specifically, relational algebra traditionally reckons
the following five core operators, whose intuitive semantics is sketched in Fig. 4.5
(please ignore ‰ for the moment):
4.3 Extending Spreadsheets with First-Class Relations 75

Union: Given two relations xN 1 and xN 2 based on the same index set I , the union
of xN 1 and xN 2 , denoted by xN 1 [ xN 2 , returns the records appearing in either relation.
This is simply set-theoretic union. As such, records that appear both in xN 1 and xN 2
are collapsed within xN 1 [ xN 2 .
Consider our registrar example in Fig. 4.3. Assume that columns D through F
contain the first and last names of the instructors in our university, together with
their concatenation as an identifier. Call the resulting relation Instructors and
notice that it has the same structure as Students. Then, the relational expression
Students [ Instructors has a record for every person who is either a student or an
instructor, maybe identifying the members of the university who are allowed in
a classroom. Individuals who are both students and instructors appear only once.
Difference: Given again I -relations xN 1 and xN 2 , the difference of xN 1 and xN 2 ,
denoted by xN 1  xN 2 , returns the tuples occurring in xN 1 but not in xN 2 . This is
set-theoretic difference.
In the setting above, Instructors  Students identifies the core faculty, i.e., the
instructors who are not students.
Cartesian product: Given a j1 -ary relation xN 1 and a j2 -ary relation xN 2 , the
Cartesian product of xN 1 and xN 2 , denoted by xN 1  xN 2 , is the .j1 C j2 /-ary relation
whose records are the concatenation .r1 ; r2 / of every record r1 2 xN 1 with every
record r2 2 xN 2 . Recall that neither r1 nor r2 can be entirely blank. This operation
is not set-theoretic but purely relational since it refers to the structure of records.
It is always a challenge to provide a convincing example involving Cartesian
product alone because it is almost always used together with other operators,
as we will see. Therefore, without even attempting to motivate its usefulness,
Students  Students would produce 6-attribute records that pair up each student
with every other student in our registrar example.
Projection: Given an I -relation xN with records of length j and a list aE D
.a1 ; : : : ; ah / of attributes in the range Œ1; : : : ; j , the projection of xN onto aE ,
written aE x, N is the relation whose records consist of only the aE components
of any record in x. N The order of the attributes in the result is specified by aE . Note
that the attributes in aE are not required to be distinct. Therefore, h may be smaller,
equal, or even larger than j .
In Fig. 4.3, the expression First Students returns a unary relation consisting of
the first names of every student in the school. Duplicate entries, such as the two
“Sue”’s, would appear only once. Note how we used the attribute identifier First
rather than the index 1.
Selection: Given a relation xN and a condition C that assigns the value true or
false to any record in x, N the selection of xN according to C , denoted by C x, N
is the relation consisting of only the records of xN that satisfy C . Conditions
typically compare an attribute of a record with either a value or another attribute.
Differently from all the above operators, selection is not purely relational in that
it must refer to the domain of the relation through the condition, both for values
and for meaningful notions of comparison.
For example, the expression FirstD“Sue” Students would return a relation
consisting of a record for only the students whose first name is “Sue”.
76 4 The Relational Spreadsheet

Comparing this list with the commands available in a commercial spreadsheet, such
as Microsoft Excel, to manipulate relations reveals one exact match: filtering, which
corresponds to selection. A rudimentary form of projection is made available when
importing a data list from an external application. None of the other operators
is present. Therefore, making them available in the context of a spreadsheet
substantially enhances the expressiveness of this tool. We will further extend this
basic pool of operators shortly.
We now formalize the intuition behind each relational operator op N above by
defining the behavior of the corresponding operation opb
N when applied to relational
values of the appropriate form:

vN 1 [ O vN 2 D fEv W Ev 2 vN 1 or Ev 2 vN 2 g D vN 1 [ vN 2
vN 1  O vN 2 D fEv W Ev 2 vN 1 and Ev 62 vN 2 g D vN 1  vN 2
vN 1 O vN 2 D f.Ev1 ; Ev2 / W Ev1 2 vN 1 and Ev2 2 vN 2 g
O a1 ;:::;ah vN D f.EvŒa1 ; : : : ; EvŒah / W Ev 2 vN g  fE[g
O C vN D fEv 2 vN W CO .Ev/ is trueg

The central column extensionally defines each operation in the left column by
describing how each of the resulting relations is constructed. The first two
implement set-theoretic union and difference, respectively, shown as such in the
rightmost column. As we mentioned earlier, a list consisting exclusively of blank
values is not considered a record. We will define the meaning of CO .Ev/ as soon as we
have formalized conditions.
In the same way as scalar operators allow building scalar expressions based on
values and locations, we use the above relational operators to construct relational
expressions, written e,N possibly subscripted, starting from base relations. These
entities are formally defined by the following grammar:

Relational expressions: eN WWD hE

ei j eN [ eN j eN  eN j eN  eN
j aE eN j C eN

The first production, the order-forgetting coercion of an array into a relation, was
introduced earlier. It provides the base case for building relational expressions.
Such a base relation will typically be an array reference lE, but, by allowing generic
array expressions eE, we implicitly allow extensionally specifying relations using the
existing mechanism for arrays. We do not expect this feature to be commonly used
in practice.
Note that, differently from scalar and array formulas, relational expressions are
based on a fixed and rather small set of constructors. Indeed, the operation op b
N
associated to each operator opN is fixed by the laws of relational algebra, as we saw.
We will expand this collection with additional operators that are naturally lifted from
the spreadsheet world later in this section. We will extend it in a different direction
in the next few chapters with powerful deductive capabilities. We denote the set of
all relational expressions by R.
4.3 Extending Spreadsheets with First-Class Relations 77

The selection conditions C appearing in the above grammar are defined as

follows at the macroscopic level:

Selection conditions: C WWD c j C ^ C j C _ C j :C j true j false

They are simply Boolean combinations of base conditions, c, defined next:

Base selection conditions: c WWD ae j aa

Here, a stands for an attribute and  for an appropriate comparison operator

on the domain of the relations (ultimately values). Therefore a base condition
either compares an attribute to a domain value, or with another attribute. Different
comparison operators will be available depending on the actual values being
compared (e.g., < makes sense for numbers, but not for strings). One operator
that shall invariably be available is equality, D. We assume that each comparison
operator  is realized by a comparison relation that we will indicate as . O We will
shortly extend the notion of base condition so that each side of  is an arbitrary
scalar expression with zero or more attributes embedded in it.
As suggested in defining the selection operation O C vN above, conditions are used
to select certain records in a relation. In particular, we associate with each condition
C a function CO that maps an I -record to the Boolean values true and false:

CO W V I ! ftrue; falseg

where V I is the set of vector values of geometry I . Records that CO maps to true will
be selected, while those that evaluate to false will not. The formal definition of the
application CO .Ev/ is given next:

2
.a  v/.Ev/ is true $ EvŒa 
O v is true
3.a  a /.Ev/ is true $ vEŒa 
0
O EvŒa0  is true
3 ^ C /.Ev/ is true c1 .Ev/ is true and C
$ C c2 .Ev/ is true
3
.C 1 2
.C _ C /.Ev/ is true c c
$ C1 .Ev/ is true or C2 .Ev/ is true
.1
1 2
:C /.Ev/ is true $ CO .Ev/ is not true
tb
b v/ is false
rue.Ev/ is true
false.E

We defined an attribute as a vectorial index. Therefore, when writing a vN ,

we meant the a-th component of each record in vN . When the argument of  is a
complex relational expression e, N it will be more convenient to let a be a reference
N Indeed,
to an attribute of the base relations appearing in e.

Instructor;Grade AbbrvDCourse .Courses  HasTaken/

78 4 The Relational Spreadsheet

is more clear than 3;7 2D6 .Courses  HasTaken/. There is an easy way to
reconstruct the indices of a top-level relational expression from the indices of its
base relations. In the rest of this chapter, we will use one method or the other to
refer to attributes. When relying on base relation attributes in a definition, we will
often express them as hE e i:.
A selection or projection operator can refer to attributes that do not exist in the
relation given them as input. Indeed, we can write 4 Students in our example in
Fig. 4.3 although the relation Students has only three attributes. We already observed
this problem when discussing the interface between arrays and relations, but did not
settle on the best approach to addressing it (using default completions or returning
an error relation). A similar problem derives from the fact that we can take the
union or difference of relations whose records do not have the same length, as in
Students [ Grades for example.

Derived Relational Operators

A number of useful derived operators can be defined on the basis of the five core
operators of relational algebra introduced earlier. Two particularly useful instances
are join and intersection:
Join: The join of two relations xN 1 and xN 2 according to a condition C , written
xN 1 ‰C xN 2 , is obtained by filtering the Cartesian product of xN 1 and xN 2 through C
and projecting away the attributes that C equates:

xN 1 ‰C xN 2 D ¤C .C .xN 1  xN 2 //

Join is maybe the most commonly used relational operator. It is able to construct
records with values contributed by two relations. Differently from , the
condition ensures that only records deemed interesting are retained. As a final
touch, projecting away duplicated attributes retains only essential information.
This behavior is intuitively displayed in the bottom right corner of Fig. 4.5. Note
that the Cartesian product can be defined on the basis of a primitive notion of
join: xN 1  xN 2 D xN 1 ‰true xN 2 .
The form of join whose condition only equates attributes (i.e., uses only D as
a comparison operator and only ^ as Boolean glue) is particularly common.
It is called natural join and the condition is generally omitted when it can
be unambiguously reconstructed. This is the case when attributes are specified
using symbolic names, and different relations rely on the same name to refer to
homologous attributes.
As an example, assume we are interested in correlating the grades students
received in a class with the instructor who taught it. The following relational
expression will produce records including the name of an instructor and every
grade he has ever given:

Instructor;Grade .Courses ‰AbbrvDCourse HasTaken/

4.3 Extending Spreadsheets with First-Class Relations 79

We saw the corresponding core expression earlier. If we wanted to restrict the

search to just the year 2004, we would refine this expression as follows:

Instructor;Grade SemesterD“S 04”_SemesterD“F 04” .Courses ‰AbbrvDCourse HasTaken/

The next expression creates a relation that differs from HasTaken only by the fact
that all grades have been replaced by their numerical equivalent.

Student;Course;Semester;Number HasTaken ‰GradeDLetter Grades

We will repeatedly rely on this calculated relation below. Therefore, it is

convenient to assign it a name: HasTakenNum.
The next expression retrieves the first and last name of every A student, i.e.,
those students who have received a grade numerically greater than 3:50 (an A or
an A) in any course they have taken:

First;Last Number>3:50 .Students ‰ HasTaken ‰GradeDLetter Grades/

Notice that the condition on the first join is implicit as it equates the Student
attribute of relations Students and HasTaken.
The relation featuring the complementary condition contributes to an expression
that returns the students who have never received a grade lower then A:

First;Last Students
 First;Last Number3:50 .Students ‰ HasTaken ‰GradeDLetter Grades/

Intersection: Relational intersection, written xN 1 \ xN 2 , is defined by xN 1 \ xN 2 D

xN 1 .xN 1 xN 2 /, as in the case of sets. It can also be defined using natural join. In our
extended example, Students \ Instructors would return the records of every
member of the university that is both a student and an instructor.
Figure 4.6 continues our example by showing additional parts of the spreadsheet
in Fig. 4.3. We have included the most interesting expressions encountered so far in
the top and central parts. The bottom part shows two simple examples that highlight
the interplay of array and relational operations. The relational expression at the
bottom left of the figure operates on a proper relation, Student Students, and the
result of the array formula First & “ ” & Last, which concatenates corresponding
elements of the arrays First and Last with a space in between. The example next to
it demonstrates the dual behavior. The relational expression Number Grades isolates
the numerical column of Grades, which is converted to a non-deterministically
ordered array and then component-wise incremented by 1. We will generally omit
the conversion brackets when easily inferable.
80 4 The Relational Spreadsheet

AA
AB AC ··· ··· AE AF ···
1 Members StudentsWithAs
2 First Last Name First Last
3 = ¼ First ,Last ¾ Number >3.50
= Students ∪ Instructors ( Students / HasTaken
/
4
5
// Grade = Letter Grades )

··· AK AL AM AN ···
1 HasTakenNum
2 Student Course Semester Number
3
= ¼ Student , Course , Semester , Number
HasTaken / Grade = Letter Grades )
4 /
5

··· AS ··· ··· AU ···

1

2 Names GradeInc
3

4 =First & “ ” & Last ∪ ¼ Student Student =[ ¼ Number Grades ] + 1

5

Fig. 4.6 Relational expressions in the registrar’s spreadsheet

Extended Relational Algebra

A number of extensions of the five core relational operators have been proposed to
gain in succinctness or expressive power when querying a database. Some of these
extensions are particularly relevant in the context of a spreadsheet since they tighten
the integration between the functional and the relational paradigms. We examine
two of them:
• The base conditions of the selection operation compare an attribute to either a
constant or another attribute. It is natural to generalize it to the comparison of two
“attribute expressions”, entities that differ from scalar expressions by allowing
attributes to appear in them.
For example, the following relational expression makes use of this extension
to return the name of all the students whose grades dropped (or rose) by more
than one point over the course of their studies:

Student.1/  HasTakenNum.1/ ‰Student.1/ DStudent.2/ HasTakenNum.2/

abs.Num.1/ Num.2/ />1:0

where the operator abs returns the absolute value of a number, and the
superscripts allow distinguishing the two instances of HasTakenNum and their
attributes. Notice that the (highlighted) selection condition is now a complex
expression involving attributes.
4.3 Extending Spreadsheets with First-Class Relations 81

• The traditional projection operation simply returns some of the attributes of

its input relation. It is again natural to consider an extension that allows it to
return something more interesting than a mere copy of attributes. Using attribute
expressions instead of projection attributes allows us to achieve this level of
expressiveness.
For example, the following expression creates a relation that expresses the
numerical value of each grade on the scale Œ0 : : : 5 instead of the original Œ0 : : : 4
of Grades:

Letter; 5Number=4 Grades

It should be observed that both of the above behaviors could have been realized
using only the core relational operators and array expressions, but not as a
single formula: intermediate tables are needed where relational and functional
computations meet. Consider the first expression above:
1. Compute the relational expression

Student.1/ ;Num.1/ ;Num.2/ HasTakenNum.1/ ‰Student.1/ DStudent.2/ HasTakenNum.2/

and save it on three adjacent columns. Call them Student, Num1 and Num2,
respectively.
2. Insert the following array formula on the next column,

abs.Num1  Num2/

Call it Diff .
3. Now, interpret Student, Num1, Num2, and Diff as a relation Aux and feed it into
the following relational expression:

Student Diff >1:0 Aux

These three formulas achieve the same effect as the extended relational expression
shown above. Four additional columns are needed, however, and the rationale
behind it is fragmented. This is clearly displayed in Fig. 4.7, whose upper part relies
on the above extended relational expression while the central part implements the
approach just described. The bottom part of this figure shows the second extended
relational expression above and its emulation using only core relational algebra and
arrays. The difference is not as marked in this case.
Both extensions rely on the notion of attribute expression, which we denote by
the non-terminal æ, possibly subscripted, and we formalize as follows:

Attribute expressions: æ WWD a j e j op.æ1 ; : : : ; æn /

An attribute expression is indeed just a scalar expression which can contain

occurrences of attributes.
82 4 The Relational Spreadsheet

··· BA ···
1 BigGapStudents
2 Student
3

4 = ¼ Student (1) ¾ abs( Num (1) −Num (2) )>1.0

HasTakenNum (1) / Student (1) = Student (2) HasTakenNum (2)
/
5

··· BC BD BE BF BG ···
1 Aux BigGapStudents’
2 Student Num1 Num2 Diff Student
3

4 = ¼ Student (1) ,Num (1) ,Num (2) =abs( Num1 − Num2 ) = ¼ Student ¾ Diff >1.0 Aux
5 HasTakenNum (1)
6
// Student (1) = Student (2)
(2)
7 HasTakenNum
8

··· BL BM ··· ··· BP BQ ···

1 Grades5 Grades5’
2 Letter Num5 Letter Num5
3

4 = ¼ Letter ,(5∗ Number /4) Grades = ¼ Letter Grades =5 ∗ Num/ 4

5

Fig. 4.7 Extended relational expressions in the registrar’s spreadsheet

Relational expressions and subordinate definitions are then updated as follows,

where we have highlighted the changes by placing them in boxes:

Base selection conditions: c WWD ææ

Selection conditions: C WWD (unchanged)
Extended relational expressions: eN WWD hE
ei j eN [ eN j eN  eN j eN  eN
j æ E eN j C eN

We now need to extend the relational semantics introduced above to account for
these changes. So far, the operation corresponding to each operator was defined on
values of the appropriate syntactic category, the only exceptions being selection
and projection, which also mentioned attributes as place-holders for evaluated
record elements. Attribute expressions can contain attributes deeply buried within
layers of operators. The operations corresponding to the extended projection and
selection will accept an evaluated form for attribute expressions which may still
have attributes as arguments of scalar operators, but all of whose scalar expressions
will have been reduced to values. In particular, an evaluated attribute expression,
which we denote by æ, O does not contain locations:

Evaluated attribute expressions: æO WWD a j v j op.æO 1 ; : : : ; æO n /

4.3 Extending Spreadsheets with First-Class Relations 83

Then, the semantics of selection does not change, at least superficially, while the
line for projection is significantly altered:

O æO 1 ;:::;æO h vN D f.æO 1 .Ev/; : : : ; æO h .Ev// W vE 2 vN g

O C vN D fEv 2 vN W CO .Ev/ is trueg

O it is convenient to propagate the changes to base

Before formalizing the notation æ,
selection conditions, which make use of the same mechanism:

3
.æO  æO 0 /.Ev/ is true $ O æO 0 .Ev/ is true
O v/ 
æ.E

O v/ indicates the evaluation of the location-free attribute

The notation æ.E
expression æO on the record of values Ev. It simply replaces each attribute with
the corresponding value in Ev, and applies the operations until a value is returned:

O v/
a.E D EvŒa
vO .Ev/ D v
7
op.æ1 ; : : : ; æn /.Ev/ D op.
b æO 1 .Ev/; : : : ; æO n .Ev//

Pseudo-relational Operators

A useful piece of information in the context of a registrar’s office is the GPA of a

student, “Nick Vitter”, for example. The following formula may appear to compute
precisely this number:

SUM.Number StudentD“Nick Vitter” HasTakenNum/ =

COUNT.Course StudentD“Nick Vitter” HasTakenNum/

It is however incorrect: since relations are sets, relational expressions collapse

identical records. Therefore, if “Nick Vitter” has received the same grade in more
than one course, the above expression would count that number just once. Similar
problems would emerge if he had taken the same course more than once, e.g., after
failing it.
Staging this computation over a number of steps solves these problems:
we can save the records resulting from the relational expression StudentD“Nick Vitter”
HasTakenNum to a temporary relation, and then divide the SUM of the Number
column by the COUNT of the Course column. This approach has a major
drawback, besides the annoyance of splitting the computation: it cannot be
applied systematically to automatically compute the GPA of all students. Indeed,
the user would need to set up a relation such as the above for each student
individually.
84 4 The Relational Spreadsheet

··· CA CB ···
1 StudentsWithGPA
2 Stude nt GPA
3
SUM( ¼ Number [¾ Student =CA3 :CA HasTakenNum]) /
= ¼ Student Student =
4
COUNT ( ¼ Course [¾ Student =CA3 :CA HasTakenNum])
5

Fig. 4.8 Pseudo-relational expressions in the registrar’s spreadsheet

An alternative way of performing this computation relies on the “database

functions” of commercial spreadsheets. This could be written as follows in
Microsoft Excel7 :
DSUM.HasTakenNum; Number; Student D “Nick Vitter”/ =
DCOUNT.HasTakenNum; Course; Student D “Nick Vitter”/

An array version of this formula can then be used to automatically compute the GPA
of all students.
This formula computes the desired result. In the light of the work in this section,
it is worth observing that the same effect is obtained by providing array equivalents
of the relational operators. Consider for the time being the array projection E aE eE
that differs from relational projection by the fact that it does not collapse duplicated
records (and that the ordering of the output array is deterministic, a property we will
not use). Then, the computation of “Nick Vitter”’s GPA is given by the following
expression:

SUM.E Number ŒStudentD“Nick Vitter” HasTakenNum/ =

COUNT.E Course ŒStudentD“Nick Vitter” HasTakenNum/

Again, this can easily be adapted to compute the GPA of every student automatically.
Figure 4.8 shows how to do precisely this.
We now go through the operators of relational algebra and comment on the
corresponding array operators.
Union: The natural array counterpart of relational union is the append operation.
Given two arrays xE1 and xE2 , the expression xE1 [ E xE2 makes a copy of xE1 and
inserts a copy of xE2 after it. Of course, there are two ways of doing so: xE2
could be appended below xE1 , or to its right. Therefore, we shall annotate [ E
with the orientation in which the fusion should take place. We obtain xE1 [ E V xE2
(for vertical append) and xE1 [E H xE2 (for horizontal).
Projection: We have already encountered E aE eE as the multiplicity and
order-conscious adaptation of relational projection. Differently from concate-
nation, the attributes aE will specify the orientation of the operation. Note that

7
More precisely, the third argument would be a reference to a range of cells containing syntax
specifying the condition.
4.3 Extending Spreadsheets with First-Class Relations 85

the extension of relational projection with attribute expressions can similarly be

adapted to the array world.
Selection: Array selection, EC eE, is the counterpart of (extended) relational
selection which preserves duplicates and order. We could have used it in the
above example instead of writing StudentD“Nick Vitter” HasTakenNum.
An array version of the Cartesian product is not substantially different from the
relational original. Hence we refrain from introducing it. Several notions of array
difference could instead be defined: we do not explore them until needed in an
example.
No commercial spreadsheet makes any of these operations available, although all
provide a choice of “database functions”, each of which specializes one or more of
the above operations. Notice however that each of these database functions can be
constructed on the basis of EC and the corresponding array operator.

4.3.3 Relational Spreadsheets

Given the notions of relation and relational expression defined in the previous
section, we are ready to formalize the relational extension of a spreadsheet. Each
location in our example spreadsheet either contains a scalar expression, or is the
target of an array expression or of a relational expression. Therefore, following our
steps from Sect. 4.1, we parcel out the set L of available locations and assign an
expression to each partition. Thus, a relational spreadsheet consists of the following
two components:
1. A partition L of the set L of all locations into disjoint arrays, i.e., a function
L W L ! LI that maps each location l to an array lE such that l 2 L.l/. Each
array lE has its own geometry IlE.
2. A function s W flgE ! E I [ ENI that associates either an array expression eE 2 E I
or a relational expression eN 2 ENI to each element lE in this partition.
The following schema may help visualize this definition:

⎫
L LI ⎪
⎪
⎬
s
⎪
⎪
⎭
E EI E¯I

where the case for scalars (E) is once more implicitly accounted for as 11-arrays
in the definition itself.
86 4 The Relational Spreadsheet

To
From
Arrays Relations
r
æ →
Arrays ∪ a
→
r r
→
Relations a a
→

Fig. 4.9 Choices for the notion of subexpression in the relational spreadsheet

E s/ as s, and retrieve
As with arrays alone, we generally refer to a spreadsheet .flg;
E
the partition flg from the domain of s when needed. The geometry IlE of lE must
match the geometry of the expression s.l/ E assigned to this range of cells. As in
the case of arrays, the geometry of this expression is expected to be automatically
E in practice. Similarly to traditional spreadsheets, a large number of
adjusted to fit l,
cells in L will typically be blank, i.e., contain the blank value [.
The above definition differs from the corresponding notion for arrays in
Sect. 4.1 only by the added option of having an array reference contain a relational
expression. Not surprisingly, the definitions of referential closure of a relation 
with respect to s remain unchanged. For convenience, we report from Sect. 4.1 the
component-wise version of this notion, s , which is the one we will use in defining
well-formed relational spreadsheets.
The component-wise referential closure of a relation  with respect to s is the
relation s given by:
• s.L.l//Œ s l for all l 2 L, where l D L.l/Œ;
• eE s eE0 if eE  eE0 ;
• eE s eE00 if eE s eE0 and eE0 s eE00 .
We need to complement this definition by specifying the meaning of s.L.l//Πwhen
L.l/ contains a relational expression eN and when L.l/ contains an array formula of
N This last case falls back on the first by setting ŒeŒ
the form Œe. N D eŒ.
N Now, let a
be the attribute of eN of which  is a subindex. Then, we simply set eŒ
N D e:a.N
Our next task shall consist in devising an appropriate notion of subexpression,
whose referential closure will allow us to define the concept of dependency,
which will again be an important ingredient in defining well-formed spreadsheets.
The definition of subexpression for arrays, written v in Sect. 4.1, will change
minimally. Instead, there are at least two meaningful choices for its relational
counterpart: we can either follow entire relations, or we can track just their attributes
(tracking records makes little sense since a relation is a set of records—we will
however revisit this statement in Sect. 4.5 when examining updates). Figure 4.9
summarizes this situation and also accounts for the interface between the two
sublanguages. We begin by defining the two candidate notions of relation-level
subexpression and then adapt the array-level counterpart.
4.3 Extending Spreadsheets with First-Class Relations 87

Relational Subexpressions

The most intuitive notion of sub-expression for relations simply looks at the
arguments of the operators. Given two relational expression eN and eN 0 , it takes the
form eN vr eN 0 and is defined as follows:

eN vr eN
eN vr eN1 ? eN2 if eN vr eN1 or eN vr eN2 where ? 2 f[; ; g
eN vr C eN 0 if eN vr eN 0
eN vr æ E eN
0
if eN vr eN 0

We write @r for the strict variant of this relation, which we call relational
subexpression.
This purely syntactic form of subexpression will be very useful below. It should
be noted, however, that it is very coarse. Indeed, it may the case that eN vr eN 0 ,
but what eN contains has no effect on eN 0 . Consider for example the expression
Student Students  Grades. Clearly, Grades vr Student Students  Grades.
However, all attributes of Grades are projected away and therefore no record in
this relation can influence the result of this expression. The attribute-level notion of
subexpression will address precisely this problem.
Before introducing it, we present the natural extension of vr to the relation-array
interface. A relation eN is a subexpression of an array eE, written eN @r eE, if it
syntactically occurs within eE. Formally:

eN @r ŒeN 0  if eN vr eN 0
eN @r op.E
E e1 ; : : : ; eEn / if eN @r eEi for some i D 1::n

As a final touch, we very coarsely define the notion of a relation depending on an

attribute, eN vr eN 0 :a, as

eN vr eN 0 :a if eN vr eN 0

We will need this for the referential closure of @r to be well defined.

Dependent Attributes

Our second notion of subexpression overcomes the deficiencies just outlined by

N The key observation is that,
focusing on the attributes of a relational expression e.
in general, only a subset of the attributes of the base relations occurring in eN will
N the rest will be projected away. This subset
play a role in the calculated value of e:
will be even smaller if we inquire about the attributes contributing to any given
N This notion is semantic since it depends on the behavior of each
attribute a of e.
individual relational operator. This leads to a set of definitions that is substantially
more detailed than in the previous case.
88 4 The Relational Spreadsheet

N 0 of a relation eN depends on attribute hE

An attribute e:a ei:a of a base relation hE
ei,
written hE
ei:a v e:a
a
N 0 , if:

hE
e i:a va hE
e i:a
e i:a v .eN1 ? eN2 /:a0
hE a
if hE
e i:a va eNi :a0 for i D 1::2 where ? 2 f[; g
(
e i:a va eN1 :a0
hE if 1  a0  jeN1 j
hE
e i:a va .eN1  eN2 /:a0 if
e i:a va eN2 :.a0  jeN1 j/ if jeN1 jC1a0  jeN1 jCjeN2 j
hE
hE N 0
e i:a va .C e/:a if hE
e i:a va e:a
N 0
or hEei:a 2 Attr.C /
hE
e i:a va .æ1 ;:::;æh e/:a
N 0 if hE
e i:a 2 Attr.æa0 /
or hE N
ei:a 2 Attr .e/

In the third line, dealing with , we translate a0 to an appropriate attribute of either

eN1 or eN2 . We took the liberty of considering a0 purely numeric. The last two lines
deserve further explanation. Consider the case of selection first: the attribute hEei:a
can contribute to .C e/:aN 0 in two ways: it can either influence the attribute e:aN 0
N or it can appear in the selection condition C . The second option is formally
of e,
expressed as hE ei:a 2 Attr.C /, where the function Attr.C / is defined as follows by
induction on the structure of C :

Attr.C1 ? C2 / D Attr.C1 / [ Attr.C2 / for ? 2 f^; _g

Attr.:C / D Attr.C /
Attr.æ1  æ2 / D Attr.æ1 / [ Attr.æ2 /

The rationale for projection is similar: observing that a0 must be one of the
projection indices 1::h, we can simply check whether hE
e i:a is mentioned anywhere
in the attribute expression æa0 , expressed as hE
ei:a 2 Attr.æa0 /, or we can look
for hE
ei:a in any selection condition within e. N We first formalize the function
N
Attr .e/:

Attr .hEe i/ D ;
Attr .eN1 ? eN2 / D Attr .eN1 / [ Attr .eN2 / where ? 2 f[; ; g
Attr .C e/ N D N [ Attr.C /
Attr .e/
Attr .æ E e/N D N
Attr .e/

Both Attr.C / and Attr .e/N rely on the function Attr.æ/, which collects the
attributes appearing in an attribute expression æ:

Attr.e/ D ;
ei:a/
Attr.hE D fhE
ei:ag
S
Attr.op.æ1 ; : : : ; æn // D i D1::n Attr.æi /
4.3 Extending Spreadsheets with First-Class Relations 89

Relational attribute dependency hEe i:a va e:a

N 0 is naturally lifted to arrays eE0 on
the right by looking for subexpressions of eE whose a0 attribute depends on hE
0
e i:a:

hE N 0
ei:a @a Œe:a if hE
ei:a @a e:a
N 0
hE
ei:a @a .op.E
E e1 ; : : : ; eEn //:a0 if hE
ei:a @a eEi :a0 for some i D 1::n

Finally, we introduce the notion of a relation eN depending on a base attribute

hE e i:a va eN if hE
ei:a as hE e i:a va e:a
N 0 for some attribute a0 2 IeN .

Functional Subexpression

The notion of subexpression for array and scalars is defined exactly as in Sect. 4.1:

eE v eE
E e1 ; : : : ; eEn /
eE v op.E if eE v eEi for some i D 1::n

with @ being its strict version.

At this point, we only need to formalize what it means for an array to be
a subexpression of a relational entity to complete the map outlined in Fig. 4.9.
Consider the simpler attribute-oriented case first. An array eE depending on an
N of a relation e,
attribute e:a N written eE @a e:a,
N is defined as follows:

N
eE va e:a if hE
e 0 i:a0 @a e:a
N and eE v eE0 Œa0  for some a0

where the notion of attribute eEŒa of the array formula underlying a base relation hE
ei
was defined earlier.
Simply replacing va with vr in the above definition is not sufficient to obtain
an adequate notion of dependency of a relation eN on an array eE. Indeed, eE can not
N base relations, but it can also be mentioned in one of its
only appear in one of e’s
attribute expressions. Therefore, the relation-level variant of this notion, which we
write eE @r e,
N shall be defined as follows.

eE @r eN if eE v eE0 and hE
e 0 i vr eN
or eE @ eN
æ

where the relation eE @æ eN formalizes the fact that eE occurs in an attribute expression
N
of e:

eE @æ eN1 ? eN2 if eE @æ eN1 or eE @æ eN2 and where ? 2 f[; ; g

eE @æ C eN if eE @æ C or eE @æ eN
eE @æ æ E eN if eE væ æE or eE @æ eN
90 4 The Relational Spreadsheet

This definition relies on the similar concept for selection conditions, eE @æ C :

eE @æ C1 ? C2 if eE @æ C1 or eE @æ C2 for ? 2 f^; _g
eE @æ :C if eE @æ C
eE @æ æ1  æ2 if eE væ æ1 or eE væ æ2

In turn, this definition relies on the similar notion for attribute expressions, eE væ æ:

eE væ e if eE v e
eE væ op.e1 ; : : : ; en / if eE v ei for some i D 1::n

Well-Formed Spreadsheets

When chasing subexpressions across references, we have the choice of using either
vr or va to cope with relations:
• In the first case, we rely on eE v eE0 when comparing array expressions, on eN vr
eN 0 for relations, and on eN @r eE and eE @r eN at the boundary between the two
sublanguages. Let v+ r be the resulting relation. Formally but with a slight abuse
of notation, v+ r D .v [ vr /. We denote the strict version of this relation by @+ r .
As an example, consider the relation eN D Instructor;Grade .Courses
‰AbbrvDCourse HasTaken/, where the relations Courses and HasTaken occupied
cells H 3WJ and P 3WS , respectively, with the attributes Instructor and Grade
on columns J and S in Fig. 4.3. Then, H 3 v+ r e. N
• In the second case, we similarly use the attribute-centric variants of these
relations, eE v eE0 , e:a
N va eN 0 :a0 , e:a
N @a eE and eE @a e:a, N to define the relation
v as v D .v [ @ /. Again, @ is the strict variant of v+ a .
+ a + a a + a

In the setup above, it is not the case that H 3 v+ a eN because cell H 3 lies on an
attribute, Name, that is projected away in e. N
It is clear that v+ a is finer than v+ r where they are both defined: in particular,
if eE @+ a eE0 , then eE @+ r eE0 .
We can now define the referential closures @+ rs and @+ as of @+ r and @+ a , respectively,
relative to a spreadsheet s. As usual, @+ rs propagates @+ r across the cells of s appearing
in an expression, and similarly for @+ as . Again, if eE @+ as eE0 , then eE @+ rs eE0 . We say that
a location l depends on a location l 0 in s according to @+ r (or according to @+ a ) if
l @+ rs l 0 (or if l @+ as l 0 , respectively).
As an illustration, assume cells DA3WDA of the registrar’s spreadsheet contain
the relational expression Student Number>3:50 HasTakenNum. This is a variant of the
relation StudentWithAs in Fig. 4.7. Consider location DA3. Then, each location l in
4.3 Extending Spreadsheets with First-Class Relations 91

 AK3WAN (relation HasTakenNum),

 P 3WS (relation HasTaken),
 Y 3WZ (relation Grades),
 C 5, A5, B5,
 C 3, A3, B3.
is such that l @+ rs DA3. Instead, only locations l 0 among
 AK3WAK , AN 3WAN (only attributes Student; Number of HasTakenNum),
 P 3WP , S 3WS (only attributes Student; Grade of HasTaken),
 Y 3WZ (all attributes of Grades),
 C 5, A5, B5,
 C 3, A3, B3.
are such that l 0 @+ as DA3.
Finally, a spreadsheet s contains a circular reference with respect to @+ r (or
with respect to @+ a ) if there is a location l 2 L such that l @+ rs l (or such that
l @+ as l, respectively). Since @+ a is finer than @+ r , a spreadsheet s containing a
circular dependency with respect to @+ a will have one with respect to @+ r , but not
necessarily vice versa. Consider the following counterexample:

··· EA EB EC ED ···
1 Rel1 Rel2
2 A1 B1 A2 B2
3

4 = π A2 Rel2 = πB1 Rel1

5

Then, it is easy to ascertain that EA3 @+ r EA3, but EA3 6@+ a EA3.
We conclude this section by defining a well-formed relational spreadsheet
as a spreadsheet .L; s/ subject to the same two properties as scalar and array
spreadsheets:
1. Finitely many cells in the domain of s are non-blank.
2. s does not contain any circular dependency with respect to @+ as .
The second condition bases the notion of circular dependency on @+ as because, as we
just saw, it is finer than @+ r . All the spreadsheets we will consider in this chapter will
be well-formed. Thus, we will generally drop the qualifier “well-formed”.

4.3.4 Relational Spreadsheets Versus Relational Databases

Relations are the distinguishing aspect of the data model underlying relational
databases with relational algebra as their archetypal query language [Cod70,
O’N94]. Having just imported both in the spreadsheet paradigm, it is then important
92 4 The Relational Spreadsheet

Relational Database Relational Spreadsheet

Data usage Queries Relational expressions
Results Ephemeral Persistent
Propagates updates? No Yes
Lifetime Use once Infrequently changed
Occurrences Many Few
User expertise Minimal Minimal
Setup expertise Very high Minimal
Data volume Very large Small to medium
Physical medium Disk Main memory
Topology Often distributed Always local
Designer Expert administrator User
Design Scientific Ad hoc
Fine tuning Keys, indices, etc. None
Access Multiple users Single user

Fig. 4.10 Differences between a relational database and a relational spreadsheet

to have a clear understanding of what sets the relational spreadsheet apart from a
relational database.
Taking a step back, the purpose of a database is to provide efficient access
to large amounts of data so that queries about these data can be answered with
acceptably small latency. Relational databases isolate the physical layout of the data
into abstract relations. This enables using relational algebra as a powerful query
language.
In contrast, the primary focus of the relational spreadsheet is to provide an aver-
age user with the means to rearrange tabular information in ways she deems useful
or interesting. The user interprets areas of the spreadsheet as relations and writes
relational expressions to combine them and persistently display derived information.
These short descriptions hint at distinct philosophies and applicative targets.
Indeed, the differences are numerous and touch facets related to engineering,
practice and theory. We will articulate them along three axes: the usage of data,
the operator’s required expertise, and the volume of the data. These differences are
summarized in Fig. 4.10.

Data Usage

A user interacts with a relational database by issuing queries written in a gentrified

form of relational algebra, often the SQL language [O’N94]. A short time later,
an answer is returned and she can use it for whatever purpose she needed it.
The same or a different user will occasionally update the database by adding,
removing or modifying entries. A new query needs to be issued to witness the effect
of these changes.
4.3 Extending Spreadsheets with First-Class Relations 93

The relational spreadsheet does not have a notion of query. Instead, relational
expressions (which will be given a textual form in Part II) are used to process
information present in other regions of the spreadsheet (its base relations) and
display it in a fixed cell range. Any update, by insertion, modification or removal,
to a base relation is immediately reflected in the result.
Relational expressions in a spreadsheet are therefore persistent, in the sense that
they remain there until explicitly changed by the user. The values they display is kept
up-to-date with respect to the records present in their base relations. By contrast,
a database query is ephemeral: once an answer has been returned, it takes another
query to display any intervening change. Queries are therefore akin to commands
rather than expressions. A relational spreadsheet will generally contain a small
number of relational expressions, a few dozen at most. Instead, even a small
relational database will be the target of thousands of queries, often issued by
different users.

Needed Expertise

Traditional spreadsheets owe their popularity in large part to the fact that they
enable users with very little computer literacy to perform quite complex tasks. This
key characteristic shall be maintained in the relational and soon in the deductive
spreadsheet.
It takes very little sophistication to issue queries and enter updates in a
relational database. However, the setup and the maintenance of the database itself
require substantial expertise. A database administrator is often a professional with
numerous years of training, a deep knowledge of computer science in general, and
working familiarity with the particular database management software. We will
elaborate on his tasks shortly.

Data Volume and Its Consequences in a Relational Database

Relational databases are carefully engineered to store very large quantities of data
and efficiently retrieve any portion of them. Nowadays, a database with just a few
gigabytes of data is considered small. Indeed, terabyte-size databases and sometimes
relations are commonly found in large and not-so-large organizations. Database
hardware with enough main memory to allow direct access to such volumes of data
is conceivable, especially with the widespread availability of 64-bit architectures,
but would be prohibitively expensive. For this reason, a database management
system keeps data on secondary storage and copies blocks in main memory as
needed to efficiently answer queries. For efficiency, fault-tolerance and privacy, data
are often distributed on several disks and increasingly over several servers.
Because each disk access takes several milliseconds, quite some ingenuity is
required to ensure that queries will be answered within a time acceptable to a user.
This is realized, in part, by implementing efficient algorithms and data structures in
94 4 The Relational Spreadsheet

the database software. The most critical aspect in minimizing disk access, however,
is the layout of the database itself, which is the responsibility of the database
administrator. This begins when the database is first set up with the identification of
the relevant attributes and their allocation to relations on the basis of their functional
dependencies, i.e., the expected relationships among the information they represent.
This results in optimally sized records, which can be uniquely identified by just
a few attributes (often just one) known as a key. This permits defining a system
of indices that enables retrieving records of interest with a minimum number of
disk accesses, often just one if a key is given. Relations and indices are then
stored on disk in such a way that objects that will likely be accessed together
reside on the same block or on contiguous blocks. Attribute relationships and other
properties are often retained in the form of integrity constraints which are later used
to validate updates to the relation: this permits both ensuring the logical consistency
of the data, and making sure that the effectiveness of indices and the physical layout
are retained.
The administrator’s role continues after the initial setup phase by fine-tuning
the performance of the database, and reflecting changes to the data being modeled
(for example by adding attributes). One useful tool, especially for backward
compatibility, is the view, i.e., a virtual relation implemented as a relational
expression. Because it takes so much effort to design a database, these applications
are generally multi-user.

Data Volume and Its Consequences in a Relational Spreadsheet

The physical characteristics of a typical spreadsheet are quite different.

A spreadsheet is meant to reside in the main memory of an average personal
computer.8 A spreadsheet that is too big for the available memory cannot be loaded.
In practice, most spreadsheets are rather small, with their size often measured in
kilobytes, occasionally reaching a few megabytes. Although there is a growing
trend toward collaborative support, spreadsheets are single-user applications.
In particular, an individual spreadsheet invariably corresponds to a file in the user’s
machine or a network, although it may refer to other spreadsheets.
The relational spreadsheet is primarily a spreadsheet. Therefore, it shall be
engineered to abide by the very same constraints that bound traditional spreadsheets.
In particular, it resides in main memory and is principally single-user. We expect the

8
Up to around 2005, personal computers had rarely more than 512 MB of main memory, that
virtual memory and swapping extended to 4 GB (on a then-common 32-bit architecture), at the
price of performance. This is also the maximum size of a sheet in current commercial spreadsheet
applications: indeed, sheets are limited to 65,536 rows and 256 columns, or about 16 million cells,
and each cell can contain a maximum of 256 characters, which means that a sheet full to the seams
would be about 4 GB. Nowadays, 64-bit architectures remove this limitation to all practical effects,
while commodity computers with RAM in excess of 4 GB allow keeping large spreadsheets in main
memory.
4.4 Evaluation 95

average disk size of a relational (and soon deductive) spreadsheet to be comparable

to that of a traditional spreadsheet (in a typical home-office environment, the user is
indeed likely to perform the data-entry herself). Its memory footprint is expected
to be larger, in general, because of the ease with which relational expressions
allow computing information that spans multiple cells. While array formulas already
permit this, there is evidence that few users take full advantage of them.
Because retrieving data from main memory is much faster than disk access
(about 100,000 times faster, in fact) and since relations are not nearly as big as in
a database, relational expressions can be computed quickly in a spreadsheet even if
the layout of the base relations is suboptimal or even poor. Indeed, these five orders
of magnitude are enough leeway to do away entirely with the services of an expert
administrator and the tools of his trade: keys, indices, views, etc. Instead, defining
a relational expression shall be as simple as selecting the areas of the spreadsheet
corresponding to the base expressions and specifying the relational operators that
should combine them. The user shall not be expected to define or even know
about keys and indices. This does not mean that an implementation of the relational
spreadsheet will not make internal use of indices and other bookkeeping information
to speed up computations, but only that the user will not need to have anything to do
with them. Indeed, the relational spreadsheet should perform reasonably well even
in the presence of poorly designed relations.

4.4 Evaluation

Having defined what the relational extension of a spreadsheet is, we now turn
to describing how to evaluate it. Before presenting any procedure, Sect. 4.4.1
comments on the cost of evaluating a relational expression in comparison with
traditional formulas. Following our footprints from Chap. 3, we begin in Sect. 4.4.2
with the routine extension of the naı̈ve evaluation strategy to account for relations
and their interface operators to scalars and arrays. We upgrade the definitions of
dependency graph and related concepts in Sect. 4.4.3, and revisit the semi-naı̈ve
strategy of Chap. 3 in Sect. 4.4.4. The most interesting aspect of computing with
relations has to do with devising efficient strategies for propagating updates,
an aspect we address in Sect. 4.5.

4.4.1 Cost

Since scalar operations take a fixed number of arguments, the cost of evaluating
them does not depend on the input they are given: they have constant cost, O.1/.
Things change as soon as we bring arrays into the picture. Consider component-wise
addition as a simple example: assuming that its arguments have size n, it will
perform n scalar additions, for a linear cost O.n/. As it does so, it also writes n cells,
which entails that the cost per cell of output is still constant. Reasoning similarly,
96 4 The Relational Spreadsheet

Operation Overal cost Output size Cost per output record

SO RT a x O (n log n) O(n) O (log n)
x̄ 1 ∪ x̄ 2 O(n log n) O(n) O (log n)
x̄ 1 − x̄ 2 O(n log n) O(n) O(n log n)
x̄ 1 × x̄ 2 O(n 2 ) O(n 2 ) O (1)
x̄ 1 C x̄ 2 O(n 2 ) O(n 2 ) O(n 2 )
¾ C x̄ O(n) O(n) O(n)
¼ æ x̄ O (n log n) O(n) O(n log n)
ē(x̄1 ; : : : ; x̄ m ) O(n m ) O(n m ) O(n m )

Fig. 4.11 Cost upper bounds of relational operations

SUM has cost O.n/, and since its result is a scalar, the cost per cell is now O.n/.
These upper bounds are typical of scalar and array operations.
Because they manipulate entire relations, it may be expected that the operations
of relational algebra are extravagantly expensive. Figure 4.11 assesses their
worst-case cost along the same three dimensions used in the case of scalars and
arrays: the maximum number of elementary operations, the maximum size of their
output, and the maximum unitary cost. Although not relational, we have included
SORT since we championed it in Sect. 4.3.2 and because sorting plays a role in
determining the cost of most purely relational operations. In the figure, the unit of
data is taken to be the record rather than the individual cell. Argument relations
are assumed to have length n. For SORT, n is the number of items to be sorted.
It is assumed that the size of the condition C of a join or a selection is insignificant
compared to n, and similarly for the attribute expressions of a projection. Finally,
the input relations and the result of evaluation are required not to contain duplicates.
The rationale behind the upper bounds in Fig. 4.11 is as follows:
• Sorting is well known to have complexity O.n log n/. Since its output has always
length n, the unitary cost is therefore O.log n/.
• Implementing union as a simple append would have linear cost, but the result
may contain duplicates. Finding and removing duplicates is equivalent to sorting
either the input relations or the result of an append. Sorting has cost O.n log n/.
The result relation clearly contains O.n/ records, which yields a unitary cost of
O.log n/, just like sorting.
• The difference of two relations can be computed by sorting them, which again
has cost O.n log n/, and then performing a parallel linear scan on the two sorted
relations. Again, the maximum length of the output is O.n/. The maximum cost
per output record is however O.n log n/ because the output relations may contain
just one record independently from the value of n (it may indeed be empty,
a situation we exclude from this computation).
• Cartesian product is inherently quadratic, as both overall cost and number of
records produced. The unit cost is however constant.
4.4 Evaluation 97

• Because a join with condition true simulates a Cartesian product, join has
quadratic worst-case cost and output size. However, just like difference, its output
can also be very small, which yields a worst-case unit cost of O.n2 /.
• A selection can be implemented as a linear scan of its input relation, without any
need to sort it first, and the size of the result is clearly linear as well. Because
again all this work can retain a single record, the unitary cost can be as high
as O.n/.
• While a projection can be implemented linearly, ensuring that the output relation
is free from duplicates again costs O.n log n/. The other bounds of this case are
obtained as in the case of selection.
These bounds are higher than for the scalar and array examples at the beginning
of this section. It is important to keep in mind, however, that these costs reflect
worst-case scenarios. Average figures will indeed be substantially lower for many
of these operators. For example, join conditions will generally narrow the result
down to a sublinear number of records. Sorting the arguments of a natural join over
the common attribute will often be the dominant factor of the actual cost of this
operation.
Because relational operations are seldom used in isolation, it is useful to extend
this investigation to relational expressions. In the worst case, a relational expression
eN mentioning m base relations will have a cost that is polynomial in m, as shown in
Fig. 4.11. In practice, these costs are much lower: the output of eN will typically have
length comparable to n and the overall cost will rarely be more than quadratic.
Because of these potentially high costs, relational databases include a number
of mechanisms aimed at mitigating them. We have already encountered indices,
which provide constant-time access to specific records (note that these are disk
accesses). A complementary approach involves the gathering of numerous statistics
about attribute values, which can often help decide in which order to perform the
operations prescribed in a query. This is called query optimization. It should be
noted that, although base relations are stored on disk, it rarely takes more than a few
seconds for a database to answer a query.
Because a relational spreadsheet contains few relational expressions and because
they change frequently, the role of query optimization is not as prominent as in
a database. In fact, it may be counterproductive to collect elaborate statistics if
they will be used only once. Inexpensive optimizations, for example relational
expression normalization, are still quite welcome. Which optimizations should be
kept in the relational spreadsheet requires further analysis. The availability of fewer
optimizations is balanced by the fact that a relational spreadsheet resides in main
memory and that its relations will generally be small compared to a database.
For this reason, we expect actual evaluation time to be no worse than a typical
database query time, and in fact much better (recall that memory access is 100,000
times faster than disk access).
As we will see shortly, relational expressions need to be evaluated in full
only when loading the spreadsheet. Therefore, the overhead of full evaluation is
a one-time event. As in traditional spreadsheets, recalculation is expected to be
98 4 The Relational Spreadsheet

much more frequent. Since the addition, modification or removal of a record in a

base relation needs to propagate immediately to all relational expressions where it
appears, dedicated data structures and optimizations should be in place for speeding
up this process. This aspect will be discussed in further detail in Sect. 4.5, which
addresses updates.

4.4.2 Naı̈ve Evaluation

As in the scalar world of Chap. 3, evaluation according to a strategy E will begin

in the initial environment ? where each location shows the uncalculated value ?.
We will then iterate the application of E until it produces an environment where
no location contains ?. We saw that if E is continuous, this environment is the
least fixpoint of E and that it will be obtained after a finite number of iterations.
The simplest such strategy, naı̈ve evaluation, recalculates the entire spreadsheet
at each iteration, upgrading a ?-valued cell l with a proper value only when no
location l depends on contains ?. In this section, we formalize naı̈ve evaluation in
the context of the relational spreadsheet.
The notion of environment does not change from the scalar world: an environ-
ment  is still a mapping  W L ! V? from the set of locations L to the set
V? D V [ f?g of values extended with the uncalculated object ?. In particular,
the initial environment ? maps each location to ?.
Next, we extend every operation op e W VQ n ! V,
b Q whether scalar, vectorial or
b Q Q
e ? W V? ! V? that behaves just like op
relational, to the operation op n b
e when applied
Q if any of its arguments is ?.
to proper values, but returns ? Q The notation ?
Q subsumes
the scalar and relational uncalculated values of the appropriate geometry, ? and
N respectively. This definition assumes the following pattern in the case of scalar,
?,
array, and relational operations:
8
<? if Evi ΠD ? for some i D 1::n and  2 IEvi
obp? .Ev1 ; : : : ; Evn / D
:obp.Ev ; : : : ; Ev / otherwise
1 n

bE ? .Ev1 ; : : : ; Evn /
op D b
f 7! op
E ? .Ev1 ; : : : ; Evn /ΠW  2 Iop
Eg

8
<?N if r 2 vN i and rΠD ? for some i D 1::n and  2 INvi
bN ? .Nv1 ; : : : ; vN n /
op D
:op.N
bN v1 ; : : : ; vN n / otherwise

E is evaluated component-wise. Therefore, the array

Recall that an array operation op
E ? may contain proper values for some indices and ? for others. This
returned by op
N works
is expressed by the second line of this definition. A relational operation op
instead in a relation-at-a-time fashion. Therefore, its entire output will consist of
4.4 Evaluation 99

uncalculated values if any cell it depends on contains ?. This is captured by the

third line, where ?N represents the uncalculated relation with the same geometry as
N The scalar case is a simple extension of the definition in Chap. 3:
the output of op.
? is returned whenever one of the arguments contains ?.
We now define the evaluation of a scalar, array or relational expression eQ with
respect to an environment , written eQ @ . We begin by reexamining our original
definition of this notion from Chap. 3 to support scalar expressions that take arrays
as arguments (e.g., SUM):

e@ v@ D v
l @ D .l/
e1 ; : : : ; eEn / @  D
op.E b ? .E
op e1 @ ; : : : ; eEn @ /

As in Chap. 3, a proper value evaluates to itself, the value of a location is extracted

from the environment, and the value of a complex expression with op as its top
symbol is recursively reduced to the application of the extended operation op? to
the values returned in each of its arguments. Intermediate vectorial values are treated
as if each component were an individual argument.
The evaluation of a generic array-valued operation op E is performed
component-wise. Therefore, the first line of the definition below reduces the
calculation of a generic array expression eE to the evaluation of the scalar expression
eEΠthat simulates eE in correspondence with each index  in its domain. Therefore,
some of the components of eE @  may contain proper values, while others will host
the uncalculated value ?. Note that this and the previous three cases define the
notion of naı̈ve evaluation for spreadsheets with arrays of Sect. 4.1.

eE @  eE @  D f 7! eEΠ@  W  2 IeEg
N @  D ŒeN @ 
Œe

The last case in this definition deals with the interface from relations to arrays. If eN
evaluates to the undefined relation ? N in , then its unsorted coercion into an array
will also contain ? at every index.
The evaluation of a base relation hE ei in an environment  is simply the value
of this expression evaluated as an array. The evaluation of complex expressions
relies on the application of each extended relational operation to the evaluation of
its arguments.

eN @  hE
ei @  D eE @ 
N eN1 ; : : : ; eNn / @  D
op. b
opN ? .eN1 @ ; : : : ; eNn @ /

It is worth expanding this last line for each of the five core operators of relational
algebra, or more precisely their extension as of Sect. 4.3.2:
100 4 The Relational Spreadsheet

.eN1 [ eN2 / @  D eN1 @  [ O? eN2 @ 

.eN1  eN2 / @  D eN1 @   O? eN2 @ 
.eN1  eN2 / @  D eN1 @   O? eN2 @ 
.æ N @
E e/ D O ?.æE @ / .eN @ /
.C e/N @ D O ?.C @ / .eN @ /

The evaluation of an attribute expression evaluates all contained proper

expressions in the given environment , but does not touch the attributes: they
are placeholders that will be instantiated and evaluated when supplied with an
appropriate record. A complex attribute expression op.æ1 ; : : : ; æn / will be mapped
to ? if any of its arguments is uncalculated in . Otherwise, the operator op is left
in place if any of its arguments contains an attribute.

æ@ e@ D e@

a@ D a
8
ˆ?
< if æi @  D ?
op.æ1 ; : : : ; æn / @  D for some i D 1::n
:̂
op.æ1 @ ; : : : ; æn @ / otherwise

Note that æ @  produces an evaluated attribute expression, as defined in Sect. 4.3.2.

Finally, we define the evaluation of a condition C with respect to an environment
 by simply lifting our last definition to the structure of these objects:

C @ .æ1  æ2 / @  D æ1 @  ? æ2 @ 
.C1 ^ C2 / @  D C1 @  ^? C2 @ 
.C1 _ C2 / @  D C1 @  _? C2 @ 
.:C / @  D :? C @ 
true @  D true
false @  D false

The ?-extension of each of the comparison symbols  and Boolean operators ^, _

and : returns ? if any of its arguments is ?, and otherwise leaves the structure of
the condition unchanged.
With all these concepts in place, we are ready to define the parallel evaluation
of an entire spreadsheet s with respect to an environment . We must distinguish
two cases, depending on whether s associates a cell partition lE in its domain to
an array formula or to a relational expression. In the former case, we evaluate the
array expression, isolate the value of each component, and report it pointwise in the
corresponding location. In the second case, we evaluate the relational expression as
a monolithic entity and enter the resulting value in the partition lE as a block.
4.4 Evaluation 101

8
E 7! .s.lE/ @ /ΠW  2 IEg if s.l/
[ < flΠE 2 EI
ER s ./ D
l
: flE 7! s.lE/ @ g E 2 EN
if s.l/
E
l2dom s

Notice that in the relational case, there is no fixed mapping between subvector in lE
and the records returned by the evaluation. Indeed, where a record will be written in
lE is unpredictable in general, and may vary between two rounds of evaluation.
In spite of the numerous additional cases, the substance of the definition of
ER s does not change from the purely scalar setting of Chap. 3: it refines its input
environment by replacing ? with a proper value whenever all the arguments of
the expression associated with a cell are fully evaluated, and leaves ? unchanged
whenever this is not the case. In particular, it is a simple exercise to verify that this
functional is still monotonic and continuous with respect to the partially ordered
set .Env; Env / of all environments under the ?-eliminating ordering Env —indeed
.Env; Env/ did not change from Chap. 3, and therefore is still a lower semi-lattice.
Thus, by Kleene’s theorem, the least fixpoint of ER s is obtained by iterating this
functional starting from the bottom of this lattice: ?. Since each application of ER s
necessarily removes some occurrence of ? and a spreadsheet contains finitely many
non-blank cells, this iterative process will terminate with a spreadsheet containing
a proper value in each of its cells. Therefore, the naı̈ve evaluation of a relational
spreadsheet s is once more given by the following formula:

evaluation.s/ D lfp ER s

It is interesting to consider the cost of such an evaluation. As in the purely scalar

case, each application of ER s computes a value (possibly ?) for every cell in the
non-blank portion of a spreadsheet, a number we called cellss in Chap. 3. Since
each iteration must change at least one of these cells from ? to a proper value, this
leaves us with an upper bound of

cost D O.cells2s /

By observing that the number of iterations cannot be larger than the length of the
longest chain of dependencies in s (call it ds ), we can refine this result to the exact
formula

cost D ds cellss

where the cost is given in terms of the number of cells that are written in a full
evaluation. This figure is still quadratic in cellss , in the general case. As for scalar
spreadsheets, semi-naı̈ve evaluation refines ER s into a better strategy with linear cost
(which is optimal).
102 4 The Relational Spreadsheet

4.4.3 Dependency Graph

As in the scalar setting, the dependency graph of a relational spreadsheet and the
notions derived from it are a powerful aid for the advanced analysis of these objects.
Similarly to Chap. 3, we will use them to visualize various types of spreadsheet
information (from the raw dependencies of cells to the evaluation process), to
precisely calculate the cost of evaluation, and to refine the evaluation algorithm
outlined in the previous section into the optimal semi-naı̈ve strategy. Unfortunately,
the large number of cells participating in a typical relational expression will force
us to limit the number of examples we will present and to restrict them to fragments
of the registrar spreadsheet in Figs. 4.3 and 4.6–4.8.
The dependency graph DGs of a scalar spreadsheet s contained an edge from
a location l1 to a location l2 precisely when l1 was a subexpression of s.l2 /. With
two relational notions of subexpressions, attribute dependency @a and relational
subexpression @r , we can use either one as the basis for a relational dependency
graph.

Attribute-Dependency Graph

We begin by defining the attribute-dependency graph of a relational spreadsheet s,

which we denote by DGas . It is simply given by:
• Nodes.DGas / D L
• Edges.DGas / D fl1  l2 W l1 @+ a s.L.l2 //g
A path from node l0 to node ln is a sequence of locations fli gi D0::n with edges
li  li C1 in DGas for all i D 0; : : : ; n  1. We write l0  ln as an abbreviation
for this path, and denote its length by jl0  ln j, which here is n. The requirement
that s does not contain circular references implies that DGas is necessarily acyclic.
We can then write jDGas j for the length of the longest path in DGas . The length ds
of the longest chain of dependencies in a spreadsheet s, introduced in Sect. 4.4.2 to
describe the cost of a naı̈ve evaluation, can then be expressed as ds D jDGas j C 1,
similarly to the scalar case in Chap. 3.
The attribute dependency graph of the registrar’s spreadsheet is very large, even
if the base relations were limited to the tuples shown in Fig. 4.3. The top part of
Fig. 4.12 shows the subgraph that deals only with relations Grades and Grades50 ,
defined in Figs. 4.3 and 4.7, respectively. Notice that there is an edge from every
node in the Y row to every node in the BP row. This is due to the fact that cells
BP 3WBP host the relational formula Letter Grades: every location in this range
depends on every location in attribute Grades:Letter (located on Y 3WY ) according
to @+ as .
While DGas supports many of the tasks for which we used scalar dependency
graphs, it is preferable to work with an abstraction that groups together nodes
belonging to the same attribute. The central portion of Fig. 4.12 takes this step:
4.4 Evaluation 103

⎧
⎪
⎪ BQ3 BQ4 BQ5 ···
⎪
⎨
Grades5
⎪
⎪
⎪
⎩
BP 3 BP 4 BP 5 ···

⎧
⎪
⎪ Y3 Y4 Y5 ···
⎪
⎨
Grades
⎪
⎪
⎪
⎩ Z3 Z4 Z5 ···

Grades5.Num5
BQ3 BQ4 BQ5 ···

Grades5 .Letter
BP 3 BP 4 BP 5 ··· ···
= BP3 BP

Grades.Letter
Y3 Y4 Y5 ··· ···

Grades.Number
Z3 Z4 Z5 ···

Grades5 .Letter Grades5 .Num5 = BP3:BP

Grades.Letter Grades.Number

Fig. 4.12 Three views of an attribute-dependency subgraph in the registrar’s spreadsheet

dotted line circumscribe cells belonging to the same attribute, either in a base
relation or in a relational formula. Additionally, it replaces the field of edges from
row Y to row BP with a single dotted edge between attributes Grades:Letter and
BP 3WBP (or equivalently Grades50 :Letter).
We formalize this representation as the two-level attribute dependency graph of
a relational spreadsheet s, written DGAs . It is defined as follows:
• The nodes of DGAs consist of the set LI of location arrays:

Nodes.DGAs / D LI
104 4 The Relational Spreadsheet

The nodes of interest are any non-blank individual location mentioned in either
the domain or the range of a spreadsheet s, plus all location arrays that identify
an attribute of a base relation in any relational expression eN in s (that is, flE W
lE D eEŒa and hE
e i:a va eN for eN 2 range sg), plus any array reference hosting
E W a 2 Attr.e/
an attribute of a calculated relation eN in s (that is, flŒa N and eN 2
N is the set of attributes of relational expression e).
range sg, where Attr.e/ N
• DGAs contains an edge n1  n2 from a node n1 to a node n2 whenever n1
occurs dependently within s.n2 /, where this notion is understood as an attribute
projection whenever n2 refers to a subrange of an array in dom s. Additionally,
we have containment edges l ! lE whenever l 2 lE and lE ! l if the reverse
2 3

relation holds. Formally, Edges.DGAs / is the set of edges n1  n2 given by any

of the following cases:
1. l1  l2 , where l1 ; l2 2 L, and s.L.l2 // 2 E I , and l1 va s.L.l2 //;
2. lE1  l2 , where lE1 2 LI ; l2 2 L, and s.L.l2 // 2 E I , and lE1 D eEŒa, and
hEei:a va s.L.l2 //;
3. l1  lE2 , where l1 ; l 2 L; lE2 2 LI , and s.L.l// 2 E,
N and lE2 D L.l/:a and l1 va
s.L.l//:a;
4. lE1  lE2 , where l 2 L; lE1 ; lE2 2 LI , and s.L.l// 2 E,N and lE2 D L.l/:a0 and
El1 D eEŒa, and hE
ei:a v s.L.l//:a ;
a 0

5. l1 ! l2 where l1 2 L; lE2 2 LI , and l1 D lE2 Œ;

E
2

6. lE1 ! l2 where lE1 2 LI ; l2 2 L, and l2 D lE1 Œ.

3

2
In the central part of Fig. 4.12, we have represented the containment arrows !
3
and ! by putting one node inside the other. Also, we have used a dotted edge
for attribute-level dependency.
The notion of path is defined for DGAs in the same way as for DGas . It can easily
be checked that any path na1  na2 in DGas corresponds to a unique path nA1  nA2
in DGAs . This second path contains one regular edge for every edge in na1  na2 ,
plus a number of containment edges. Therefore, when computing the length of a
path in DGAs , we will ignore any containment edge present in it. This implies that
the length jDGAs j of the longest path in DGAs is equal to jDGas j.
The bottom part of Fig. 4.12 further summarizes this information by collapsing
the dotted areas, and merging the solid arrows from Grades:Number to
Grades50 :Num5. We will occasionally use this representation at an intuitive level,
but we refrain from formalizing it.
In Chap. 3, we used the notion of dependency graph (written DGs ) as a jumping
board for displaying other useful types of information. In particular, when composed
with the underlying spreadsheet, the nodes of DGs showed each expression with the
dependencies highlighted by the edges. When composing DGs with an environment,
we obtained a graphical account of the evaluation process and could see how
values propagate along dependency paths. The same holds at the relational level.
We can compose DGAs with the spreadsheet s to visualize the dependencies among
4.4 Evaluation 105

Grades5 .Letter Grades5 .Num5

¼Letter Grades 5 ∗ Grades.Number/4

Grades.Letter Grades.Number
A A- B + ::: 4.00 Z 3 − 0.25 Z4 − 0.25 :::

Fig. 4.13 Expression-dependency subgraph in the registrar’s spreadsheet

all contained expressions. Figure 4.13 demonstrates this on the fragment of the
spreadsheet examined in Fig. 4.12. The evaluation graph EGAs ./ of s relative to
an environment  is similarly obtained by composing DGAs with .

Relation-Dependency Graph

All of the definitions obtained so far were based on the attribute-dependency relation
@+ a . We can follow the same recipe starting from the notion of relational dependency
@+ r , and define similar concepts, but at the coarser granularity of relations.
The relation-dependency graph of a spreadsheet s, denoted by DGrs , is therefore
given by:
• Nodes.DGrs / D L
• Edges.DGrs / D fl1  l2 W l1 @+ r l2 g
Observe that, differently from DGas , DGrs may not be acyclic for a well-formed
spreadsheet s, as in the counterexample at the end of Sect. 4.3.3.
Instead, the abstract relation-dependency graph of s, DGR s , is given by:
I
• Nodes.DGR s / D L , where the interesting portion consists of L and the set of
array references hosting a relation in s, and the array references used as base
relations in a relational expression.
• The edges of DGR A
s are defined similarly to those of DGs , but without any
reference to attributes. Formally, Edges.DGs / is the set of edges n1  n2 given
R

by any of the following cases:

1. l1  l2 , where l1 ; l2 2 L, and s.L.l2 // 2 E I , and l1 vr s.L.l2 //;
2. lE1  l2 , where lE1 2 LI ; l2 2 L, and s.L.l2 // 2 E I , and hlE1 i vr s.L.l2 //;
3. l1  lE2 , where l1 2 L; lE2 2 LI , and lE2 2 dom s, and s.lE2 / 2 E,N and l1 vr s.lE2 /;
4. lE1  lE2 , where lE1 ; lE2 2 LI , and lE2 2 dom s, and s.lE2 / 2 E,
N and hlE1 i vr s.lE2 /;
! lE2 where l1 2 L; lE2 2 L , and l1 D lE2 Œ;
2 I
5. l1
lE1 ! l2 where lE1 2 LI ; l2 2 L, and l2 D lE1 Œ.
3
6.
An abridged subgraph of DGR s is shown in Fig. 4.14. This representation will be
particularly useful below since it succinctly shows relation-level dependencies.
As for DGAs , we can compose DGR s with the spreadsheet or an environment to
display information other than just locations.
106 4 The Relational Spreadsheet

StudentsWithAs BP3:BP

Students HasTaken Grades

Fig. 4.14 A relational-dependency subgraph in the registrar’s spreadsheet

Grades5
BQ3 BQ4 BQ5 ···

BP3 :BP
BP 3 BP 4 BP 5 ··· ···

Grades
Letter
Y3 Y4 Y5 ···

Z3 Z4 Z5 ···

Fig. 4.15 A mixed dependency subgraph in the registrar’s spreadsheet

Note that it is possible to combine DGAs and DGR

s to display mixed information
as shown in the example in Fig. 4.15.

4.4.4 Semi-naı̈ve Evaluation

As in the scalar case, the naı̈ve evaluation strategy is highly inefficient because, at
every iteration, ER s recalculates every expression in the entire spreadsheet, including
those held in locations that already have a value (hence redoing previous work) and
those in cells that depend on ?-valued locations (thus, wasting effort). As in Chap. 3,
the semi-naı̈ve evaluation functional, that we denote by EP s , eliminates this redun-
dant computation by updating only ?-valued cells containing an expression all of
whose dependent locations contain proper values: no other expression is evaluated.
The formalization of EP s is very similar to the scalar case. Given a spreadsheet s
and an environment , we partition  into two partial functions:
4.4 Evaluation 107

Fig. 4.16 Environment

partitioning for semi-naı̈ve ∇s ´ ´
evaluation ⎧ e
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎨ ē
¯
⊥, ⊥ ¯
⊥
Δs ´
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪ ⊥
⎩ v
v̄
∇s ´ v v, v̄ v̄

• The restriction s  of  to just the ?-valued cells that do not depend on any
currently unevaluated location. In symbols,

s  D fl 7! .l/ such that .l/ D ? and ? 62 .fl 0 va s.L.l//g/ g

• The rest of : rs  D  n s .
This is intuitively rendered in Fig. 4.16. Each step in the semi-naı̈ve evaluation is
then described by the functional EP s defined as follows:
8 9
ˆ
ˆ E 7! .s.l/
flΠE @ /ΠW  2 IE and s.l/
E 2 E I g=
ˆ
ˆ l
if lE  dom.s /
[ < ;
EP s ./ D flE 7! s.l/ E 2 ENI g
E @  W s.l/
ˆ
ˆ
E
l 2dom s ˆ
:̂ E
fl 7! .l/ W l 2 lg if lE  dom.rs /

The first two lines, where lE  dom.s /, coincide with the behavior of the naı̈ve
evaluation. The last line leaves every cell in dom.rs / untouched. Only the former
case contributes to the number of cells evaluated during any given application of
EP s . The semi-naı̈ve evaluation iterates this basic step a number of times equal to the
length of the longest path in DGas plus 1:
a
evaluation.s/ D EP s s .?/
jDG jC1

During semi-naı̈ve evaluation, each cell in the spreadsheet is evaluated exactly

once. Hence, the cost of this procedure is given by

cost D cellss

As in the scalar case, this cost is optimal.

108 4 The Relational Spreadsheet

4.5 Updates

It is an integral part of the spreadsheet philosophy that any modification to

the contents of a cell should propagate to the rest of a relational spreadsheet
in a timely fashion. As in the scalar setting of Chap. 3, these updates and the
subsequent recalculation are expected to occur very frequently, sometimes several
times per minute, in comparison to loading a spreadsheet from file and consequently
evaluating it from scratch, an event that may take place just a few times in a day.
Therefore, it is important to devise update processing procedures whose cost is in
general much smaller than full evaluation.
In a scalar spreadsheet, we achieved efficient updates by isolating the cells cellss
that depended on any modified cell and invoking semi-naı̈ve evaluation from an
environment s that differed from the evaluated spreadsheet by assigning ? to
every location in cellss . We saw that this procedure was optimal. Since array
expressions are evaluated component-wise, this approach immediately extends to
that upgraded model, examined in Sect. 4.1.
The same procedure is applicable in a relational spreadsheet. Optimality is
maintained at the level of the units of evaluation, which continue to be cells for scalar
or array expressions, but are rather large blocks of cells for relational expressions.
This may not be optimal, however, if we consider relational values as sets of records
rather than monolithic relations. Indeed, an update in a base relation of an expression
eN often affects just a small fraction of the tuples eN evaluates to.
We extend our original model of updates from Chap. 3 to account for relational
and array expressions in Sect. 4.5.1. We then look for efficient record-level
procedures to streamline the propagation of changes throughout relational
expressions in Sect. 4.5.2. Finally, recalculation policies are discussed in Sect. 4.5.3.

4.5.1 Relation-Level Updates

E ! E I [ E,
Given a relational spreadsheet .L; s/, with s W flg N an update consists of
two parts, in general:
• A repartition L of the set of locations L, i.e., a partial function L W L * LI
that modifies the geometry of zero or more blocks of cells, and
• An expression update s W .range L ! E I [ E/ N [ .range L  range L *
I N
E [ E/ that describes changes in the contents of location arrays. s will
contain an entry for every new item in L, and may contain additional updates
corresponding to changes in the contents of cells that were not repartitioned.
It will be convenient to extend L with all the mappings of L that contribute to s
even if they do not properly change the partitioning of L. Therefore, from now on,
we will assume that range L D dom s.
4.5 Updates 109

Applying an update .L; s/ to a spreadsheet .L; s/ simply replaces the

modified partitions and expressions with the new contents:
( (
L.l/ if l 2 dom L s.l/E if lE 2 dom s
.LˇL/.l/ D E D
.sˇs/.l/
L.l/ otherwise E
s.l/ otherwise

As in the scalar case, an update induces changes in the dependency graphs of a

spreadsheet. We examine the attribute-dependency graph, DGas , and later comment
on how the described notions extend to the other types of dependency graphs
introduced in Sect. 4.4.3. Since the nodes of DGas are all the locations L in the
spreadsheet, an update affects only the edges of this graph. Let l 2 dom L be
a modified location.
• If s.L.l// is a value and s.L.l// is an expression eQ referencing some cell l 0
(i.e., l 0 v+ a e),
Q DGas will have to be modified with an added edge l 0  l. As in the
C
scalar case, we denote this addition by l 0  l.
• If s.L.l// is a formula eQ and l 00 is a cell such that l 00 v+ a eQ and moreover
s.L.l// is a value, DGas is modified by removing the edge l 00  l. We denote

this alteration by l 00  l.
• If both s.L.l// and s.L.l// contain formulas, then DGas will generally be
modified with both the addition and removal of some edges.
• If s.L.l// and s.L.l// contain values, then DGas is left unchanged with
respect to location l.
This is summarized in the following table, adapted from Chap. 3:

Formula in Δs(ΔL(l)) Value in Δs(ΔL(l))

 +  − −
Formula in s(L(l)) l l; l l l l
+
Value in s(L(l)) l l

The application of an update .L; s/ to the attribute-dependency graph DGas of s

is then given by:
• Nodes.DGas ˇ s/ D Nodes.DGas /.
 C
• Edges.DGas ˇ s/ D Edges.DGas / n fl 00  l W l 00  lg [ fl 0  l W l 0  lg.
It is again easy to verify that this is the dependency graph of the updated spreadsheet
s ˇ s, in symbols: DGas ˇ s D DGasˇs .
Working with DGas will be sufficient in this section. A more structured but as
precise representation of the effect of an update can be gained by tracking it on
the two-level dependency graph DGAs of s. A formal definition is somewhat more
involved since the repartition L may lead to the inclusion of some nodes and the
removal of others, in addition to changes in the edges. An update can similarly
be propagated to the relation-dependency graphs DGrs and DGR s of s. Since these
110 4 The Relational Spreadsheet

graphs do not track dependencies as precisely as their attribute counterparts, more

edges (and nodes) will be marked during the update process.
Given the environment  corresponding to the evaluation of s, we define the
environment s that differs from  by resetting to ? every location cellss
depending on an updated cell. Formally,

cellss D fl 0 W l  l 0 2 DGasˇs and l 2 dom Lg

and
(
? if l 2 cellss
s .l/ D
.l/ otherwise

Then, we invoke semi-naı̈ve evaluation (or any other strategy) from s to
recalculate the updated spreadsheet. If semi-naı̈ve evaluation is used, exactly
jcellss j cells will be written. This cost is minimal in the sense that it corresponds
to the number of cells affected by the update.

4.5.2 Incremental Updates of Relations

The procedure just outlined retains optimality at the level of units of evaluation:
formulas for scalar or array valued cells, and relational expressions for
relation-valued blocks of cells. If we consider a relational value as a set of records
rather than an entire relation, it is apparent that large amounts of computation may
be wasted. Indeed, an update in a base relation of an expression eN often affects only
N leaving the rest unchanged. Consider for example
a few tuples in the evaluation of e,
the relational expression

First;Last Number>3:50 .Students ‰ HasTaken ‰GradeDLetter Grades/

introduced in Sect. 4.3.2 (recall that the first join is over the students’ names).
Adding a student at the beginning of the year will have no effect on the evaluated
relation since she will not have taken any course yet. Adding a course record in the
HasTaken relation will add at most one record to the calculated relation, and only if
the obtained grade is A or A and the student never received such a grade before.
While the evaluation of the above relational expression may contain hundreds of
records, the change will impact at most one of them. The effects of removing or
modifying an existing record are similarly focused.
Since recalculating a relational expression often involves a non-insignificant cost,
it is desirable to devise approaches that propagate relational updates at the record
level. Let e.hE N e1 i; : : : ; hE
en i/ denote a relational expression eN built on base relations
hE
e1 i; : : : ; hE
en i. If each hE ei i evaluates to a relational value vN i , for i D 1::n, then
4.5 Updates 111

N v1 ; : : : ; vN n / is the evaluation unit of the naı̈ve or semi-naı̈ve strategies. Let now

e.N
vN i ˇ Nvi denote the update to the relational value vN i for each i D 1::n, with Nvi
representing the change. We then want to devise a method for expressing e.N N v1 ˇ
Nv1 ; : : : ; vN n ˇ Nvn / as

N v1 ˇ Nv1 ; : : : ; vN n ˇ Nvn / D e.N

e.N N v1 ; : : : ; vN n / ˇ e.N
N v1 ; : : : ; Nvn /

The factor e.N N v1 ; : : : ; vN n / is given to us from previous evaluation: only

N v1 ; : : : ; Nvn / needs to be computed. Clearly, we are looking for procedures
e.N
that limit the size of e.N N v1 ; : : : ; Nvn / to a minimum.
In our quest for such a factorization, we start by looking at each individual
operation in the core relational language. For example, we will be trying to reduce
.Ev ˇ Ev/ [ Ev0 to a function of Ev [ Ev0 and a component that depends mostly on Ev.
For the sake of simplicity, we will initially assume that Ev contains a single record
r so that Ev ˇ Ev is either Ev ˚ r to denote the addition of r to Ev or Ev
r to indicate
its removal. In the first case, it is assumed that r is not a member of Ev, while r
is required to occur in Ev in the second case. We omit the braces for conciseness.
The manner in which each core operator of relational algebra propagates the change
is then described in the following table:

=⊕ =
  
(v̄  r) ∪ v̄  (v̄ ∪ v̄ ) ⊕ r if r ∈ v̄ ∪ v̄ (v̄ ∪ v̄ ) r if r ∈ v̄ ∪ v̄ 
v̄  ∪ (v̄  r) (v̄ ∪ v̄  ) otherwise (v̄ ∪ v̄  ) otherwise
(v̄ − v̄  ) ⊕ r if r ∈ v̄ − v̄ 
(v̄ − v̄  ) r if r ∈ v̄ − v̄
(v̄  r) − v̄ 
(v̄ − v̄  ) otherwise (v̄ − v̄  ) otherwise
(v̄ − v̄  ) r if r ∈ v̄ − v̄  (v̄ − v̄  ) ⊕ r if r ∈ v̄ − v̄ 
v̄ − (v̄   r)
(v̄ − v̄  ) otherwise (v̄ − v̄  ) otherwise
(v̄  r) × v̄  (v̄ × v̄ )  (r × v̄  )


v̄ × (v̄   r) (v̄ × v̄  )  (v̄ × r)

(σC v̄)  r 
if C(r) is true
σC (v̄  r)
(σC v̄) otherwise
(πæ
 v̄) ⊕ πæ
 r if πæ
 r ∈ πæ v̄
πæ
 (v̄  r) πæ (v̄ − r)
πæ
 v̄ otherwise

A number of observations are in order.

• An update to either argument of a union or to the first argument of a difference
simply propagates through unless the added or removed tuple r is already
accounted for in the other argument.
• A change to the second argument of a difference follows the same principle, but
reverses the polarity of the update.
112 4 The Relational Spreadsheet

• Cartesian product always propagates the change by concatenating it with

every tuple in the other argument. Notice that this will generally force several
simultaneous tuples to be considered at the next level up.
• Selection simply checks whether r satisfies the associated condition.
An update to the arguments of a join combines the effect of Cartesian product
and selection (and possibly projection) into a single operation that will generally
propagate the changes much more efficiently than the chaining of the individual
procedures as per this table.
• Projection propagates updates in a manner substantially different from the other
operations.
– Calculating the projection æ E .N v ˚ r/ of the extension of vN with a record r
simply reduces to checking whether the projected record æ E r already belongs
to the calculated relation æE N
v . It is inserted otherwise.
– Calculating the projection æ E .N v
r/ of the smaller relation vN
r is
unfortunately not as straightforward. Indeed, several records in vN may have the
projected form æ E r. Therefore, it would not be sound, in general, to simply
remove this tuple from æ EvN . Hence, this is the one case in which we are bound
to calculate æE .N
v  r/ from scratch, in general.
Our inability to devise a factorization for the negative update of a projection is
somewhat mitigated when tracking updates on DGas rather than DGrs in the case
where the modified attribute of the tuple r is projected away. Indeed, the overall
expression where this occurs would not be marked for update at all in DGas .
The drawback of recalculating an entire relation can be eliminated if we are
prepared to employ sufficient bookkeeping information, and incur the overhead of
maintaining it. This is easily achieved for relational expressions where  appears
only at the top level, a common by-product of optimization: we simply associate a
counter with each calculated tuple to keep track the number of underlying records
that project onto it. Then, æ v
r/ is computed by decrementing the counter
E .N
associated with æ E r: this tuple is removed from æ N only if it reaches zero.
E v
A more cumbersome approach tracks the dependencies of each record obtained
by evaluating a relational expression e.N N v1 ; : : : ; vN n / to the records in the base relations
vN 1 ; : : : ; vN n that directly cause it to appear in the evaluated relation. While very
onerous, this method has advantages when a user asks for an explanation for a tuple
occurring in the evaluated relation.
The above procedure is easily extended to situations where Ev describes the
addition or the removal of more than one record. The general case, where it contains
mixed tuples, is handled by partitioning Ev into the sets of added and removed
records ˚ Ev and  Ev, respectively, and applying the two procedures in parallel to
their respective data sets.
The update methods for individual relational operators can often be chained
to obtain efficient update procedures for complex expressions. In some occasions,
it may however be necessary to cache intermediate results in order to apply them.
Either case yields procedures for incrementally updating the value of a relational
expression in response to a change in its base relations. They are generally much
4.5 Updates 113

more efficient than a complete re-evaluation of this expression, at least in terms

of the number of records that are altered. Because of the bookkeeping overhead,
there may be circumstances where such an incremental update may take longer
than re-evaluation. The existence of fast decision methods for determining which
alternative is preferable, although promising, is not investigated in this work.
Promising work, e.g., [LC12, NJLS11], has examined the logical presentation
discussed in Chaps. 5 and 6.
Intuitively, integrating this incremental evaluation of relational updates into our
overall scheme is rather simple: the handling of array and scalar formulas does not
change, but whenever a relational expression is marked for re-evaluation we apply
the above procedure. This however has some drastic implications in the manner
updates are propagated throughout a spreadsheet:
• First, notice that a spreadsheet as a whole is still updated by an iterative procedure
guided by the dependency graph DGasˇs . The number of iterations cannot be
larger than in the relation-at-a-time approach.
• When performing an update at the level of relations at the beginning of the
section, it was possible to statically determine which cells in the spreadsheet
needed updating: we identified them as cellss and reset them to ? in s .
In particular, cellss contained either all cells in a calculated relation or none.
In the incremental case, recalculating a relational expression will leave many
cells unchanged and only the locations hosting modified records shall be marked
as updated for the next iteration. This implies that the determination of cellss is
now dynamic: a location depending on an unchanged record does not need to be
updated. In particular, an iteration of updates now alternates with the calculation
of the cells that will be updated at the next iteration. We can view the old cellss
as a worst-case scenario that is refined at each iteration by dropping update
directives that proved overly pessimistic.
• Since some evaluated records may be left unchanged, the update procedure will
need not only s but also the original evaluated environment  to operate. Since
cellss is generated dynamically, it actually needs to be given just  as input,
and will calculate s on the go.
• Because recalculation will leave many records unchanged in general, the order
of the tuples in the re-evaluated relation may not be as unpredictable as
we anticipated earlier in this chapter. This is rather desirable from a user’s
perspective. Note however that some reordering, or at least compacting, will be
needed to cope with the removal of records.
We will revisit the issue of efficient update propagation in Chap. 5, where the logical
interpretation of a relational spreadsheet will open the door to new techniques for
doing so.
114 4 The Relational Spreadsheet

4.5.3 Recalculation Policies

The default policy in a traditional spreadsheet invokes recalculation as soon as

the user has modified a cell. This may not be a best practice in a relational
(and soon deductive) setting since, even with the above optimizations in place,
propagating changes through relational expressions can take considerably more time
than re-evaluating an equal number of scalar formulas. In a traditional spreadsheet,
where only scalar and array expressions are present, modifying an individual cell
often coincides with an atomic update, a change to a chunk of data that the user
sees as a unit. Note that, even there, this is not always the case: a logical update
sometimes corresponds to an entire row, as in the example in Chap. 3.
In a relational setting, it is natural to identify an atomic update with a record,
especially during data entry. Indeed, launching re-evaluation after the user has
entered each individual attribute is likely to be a waste of effort in this case.
Therefore, whenever a cell which is part of a record is updated, it seems reasonable
to delay recalculation until the rest of the record has been updated as well. This shall
be done with care, however, as to avoid non-recalculation when the user wanted to
modify just one attribute of a record. For this reason, recalculation should be forced
after a certain amount of time t has elapsed, or if the user moves to a cell that is
not expected to be part of this record.
In summary, a plausible default recalculation policy in a relational spreadsheet
may adhere to the following script whenever an update is made by directly writing
into a cell:
• If the modified cell is not within the scope of a record, launch recalculation as
soon as the user has selected a different cell.
• If the modified cell is part of a record, proceed as follows when the user selects a
different cell:
– If the new cell is not part of the record, invoke recalculation;
– If it is part of the record and the user does not update it within t milliseconds,
launch recalculation;
– Otherwise, repeat the procedure for this new cell.
A similar policy is viable also when a new record is added by copying a preexisting
record and then altering it. Finally, an update may take place by invoking a wizard
(as in Microsoft Excel’s “data lists”), in which case re-evaluation is invoked as soon
as the user commits the record.
The effectiveness of this policy needs to be established experimentally. Similarly,
appropriate values of its parameters, especially t , will need to be adjusted on the
basis of user observation. We currently expect a value of about one second to be
adequate. This parameter may be too low level to allow the user to set it within a
preferences menu.
4.6 Explanation 115

4.6 Explanation

In the scalar case, the dependency graph of a spreadsheet contains all the necessary
information to trace the value of an expression back to the value of the cells it
mentions, to the value of the cells that cater to them, and so on. This constitutes
a powerful explanation mechanism, and as such it is implemented under various
names in all commercial spreadsheets.
The same mechanisms can be extended to support array formulas. Notice that
there is now a choice as to how to proceed: we can either trace the computed array
value as a unit, or trace its individual components. In the first case, we point the
user to the locations that appear in the array formula as a whole. In particular, array
references are always identified as blocks of cells. In the second case, we only need
to trace the dependent locations of the highlighted cell. Because an array formula
can always be simulated by a collection of scalar formulas, this alternative is realized
exactly as in the purely scalar case, by using the dependency graph. Needless to say,
it allows for much more precise explanations. It is all the more surprising, then, that
commercial spreadsheets only make the first option available to the user. Although
array-level explanation can occasionally be useful, component-level tracing is likely
to be much more useful to a user. For this reason, we recommend it be implemented.
A similar analysis applies in the case of relational expressions. We have
developed two types of dependency graphs for the relational spreadsheet: DGrs traces
dependent base relations as a whole, while the finer DGas works at the level of
attributes (DGR A
s and DGs are more structured variants). While they can point to
all the locations that contribute to a relational expression, they do so only at the
level of attributes, if not of entire relations. This is not the type of question a user is
likely to ask.
Instead, a user will want to know where a particular record showing in the
output comes from. A proper answer will point her to the cells that contain records
(or individual values, e.g., within a selection condition) that cause this record to be
produced, and to the records they derive from, and so on. Another likely explanation
request is about why an expected record is not in the output of a relational
expression. Again, this requires tracing back the evaluation of the expression up
to a point where it is apparent that the requested record could not be produced
(often a selection or a base relation). A variant may inquire about the absence of any
record with a specified value for some attribute. We call the first question positive
explanation and the second negative explanation.
Our dependency graphs are powerless vis-à-vis of these inquiries because
they do not track dependencies at the record level. Some of the data structures
suggested for the purpose of achieving efficient updates incorporate promising
ideas, but developing a reliable infrastructure to answer these requests is likely to be
a demanding task. Luckily, we will not need to do so. In fact, the interpretation
of relational algebra into logic in Chap. 5 will provide us for free with general
mechanisms for answering precisely these questions, and many more.
116 4 The Relational Spreadsheet

Scalar expressions: e ::= v | l | op(e ,..., e)

Array expressions: e ::= e | v | l | op(e ,..., e) | [ē]
Relational expressions: ē ::= e ē ∪ ē | ē − ē | ē × ē | ¼æ ē | ¾C ē
¯
op(ē,...,ē)
Evaluated attribute expressions: æ ::= a | e | op(æ ,.. ., æ)
Selection conditions: C ::= c | C ∧ C | C ∨ C | ¬C | true | false
Base selection conditions: c ::= æ ∼ æ

Fig. 4.17 Abstract syntax of the relational spreadsheet

4.7 Summary

Traditional spreadsheets make available arrays and operators that take arrays
as input and return arrays as output. We have seen that not only can they be
transformed into equivalent scalar expressions, but that dependencies are tracked
at the component level, consistently with this model. Traditional spreadsheets offer
a notion of relation and rudimentary commands to operate on them.
We have designed a relational extension of the traditional database that not
only models scalars and arrays, but also integrates a first-class notion of relation
and operations (rather than commands) on them. Operators are borrowed from
relational algebra, the mathematical formalism underlying relational database query
languages. This allows constructing relational expressions that manipulate relations
in the same way as traditional formulas operate on scalars and arrays. Functional
(scalar and array) and relational operators coexist in the same expression through
carefully crafted interface operators that allow interpreting an array eE as a relation
hE
ei and a relation eN as an array Œe.
N Figure 4.17 summarizes the syntax of relational
spreadsheet expressions.
A relational spreadsheet is a pair consisting of a partition flg E of the set of
available locations into array references and a function s that assigns each array
reference lE to either an array formula eE or a relational expression e.
N A well-formed
spreadsheet is finite and does not contain circular dependencies among locations,
where the appropriate notion of dependency tracks relational expressions at the
attribute level.
The evaluation of the relational spreadsheet is a simple extension of the
scalar case. Scalar formulas and relational expressions are units of evaluation.
The conceptually simple naı̈ve strategy evaluates them en masse through successive
iterations from the totally unevaluated environment. The semi-naı̈ve strategy
takes dependencies into account to achieve optimal evaluation cost. A number of
dependency graphs facilitate evaluation and support other tasks, such as explanation.
Processing relational updates is substantially different from the functional case
because small changes in a base relation of an expression generally result in very
limited changes in the calculated records. It is generally much more efficient to have
these changes percolate to the output than re-evaluate the whole expressions.
4.7 Summary 117

The explanation facilities of the relational spreadsheet are nearly identical to

those of the traditional spreadsheet: scalar mechanisms are lifted to relations or
possibly attributes. This is useful to gain a bird’s eye view of the relational structure
of a given spreadsheet, but insufficient for day-to-day operation, where a user may
be more interested in tracking individual records.
Already at this early stage of the development of the deductive extension of the
spreadsheet, we can assess our progress relative to the points raised at the end of
Chap. 3:
• The relational extension of the traditional spreadsheet integrates relational and
functional operators to the fullest extent possible. Indeed, the interface between
arrays and relations allows mixing them completely freely. Moreover, attribute
expressions make arbitrary scalar expressions available in selection conditions
and projection attributes.
• We have devised efficient evaluation and recalculation procedures for the
relational spreadsheet. Their actual performance as well as the most effective
optimizations will have to be assessed experimentally.
• The relational extension has maintained the explanation facilities of the
traditional spreadsheet, but has not adequately expanded them. The logical
interpretation of a relational spreadsheet will address this issue.
Chapter 5
The Logical Spreadsheet

This chapter takes a transversal step toward the deductive spreadsheet by providing a
logical counterpart of the relational spreadsheet we just defined. By adapting a well-
known translation of relational algebra into first-order logic, our database-inspired
extension of the traditional spreadsheet acquires a distinct logic programming
flavor. Indeed, relational expressions are mapped to a recursion-free variant of
the popular Datalog language augmented with constraints. In this incarnation, the
spreadsheet safely mixes traditional constructs with simple logical formulas rather
than relational expressions. The expressive power of the resulting language is
unchanged, but carefully allowing recursive definitions will significantly extend
its expressiveness at nearly no cost in Chap. 6. This logical interpretation offers a
powerful change of perspective in how we view the relational spreadsheet. This
allows us not only to get a better understanding of the general characteristics of
this extension, but also provides us with a new set of tools for realizing the core
spreadsheet functionalities. Indeed, evaluation, update and explanation are each
supported by one or more fundamental logical concepts, hence further validating our
design with the long history of this field. Furthermore, it sheds additional light on
the interaction of our extensions with the functional core of a spreadsheet. Finally,
the abundant literature on logic programming and computational logic in general is
an essential resource where to start looking for answers to many of the problems
that implementing the deductive spreadsheet will inevitably expose.
This chapter is organized as follows. Section 5.1 recalls some basic notions about
first-order logic, in particular its syntax, model theory and proof theory. In Sect. 5.2,
we describe the logical interpretation of the relational fragment of our current
spreadsheet model, while accounting for its scalar and array components. The
logical language emerging from this translation is examined in the next two sections.
We begin in Sect. 5.3 by concentrating on the purely positive logic programming
language obtained when relational difference is not present in the input spreadsheet.
In particular, we develop the computational aspects of its model theory and proof
theory. Section 5.4 similarly deals with negation as introduced by the logical
translation of full relational algebra. Section 5.5 applies these results to obtain
effective logic-based evaluation techniques for an entire spreadsheet. Section 5.6

I. Cervesato, The Deductive Spreadsheet, Cognitive Technologies, 119

DOI 10.1007/978-3-642-37747-1 5, © Springer-Verlag Berlin Heidelberg 2013
120 5 The Logical Spreadsheet

revisits updates in this logical light. Section 5.7 recasts explanation, the third major
functionality of a spreadsheet, in logical terms. Finally, Sect. 5.8 summarizes the
work done.

5.1 A First-Order Logic Refresher

In this section, we recall some fundamental definitions concerning first-order

logic. Needless to say, this short discussion will barely scratch the surface of this
fascinating discipline, one of the oldest in the history of science. It will however
clear the path for exploring the fragment of this language and the associated methods
that endow relational algebra with a logical interpretation. It is in this light that
we will extend the expressive power of the relational spreadsheet with powerful
deductive capabilities in Chap. 6.
We first recall some basic definitions about the syntax of first-order logic, and
then discuss two classical approaches to making logical inferences. The reader
interested in a broader overview of logic or in more details is referred to classical
texts such as [Men97, Kle67], or [Pfe] for a modern approach.

5.1.1 Syntax

First-order logic is a flexible language for expressing aspects of a domain of interest,

describing intrinsic properties of this domain, and asking whether other properties
follow from them.
The objects in this domain are expressed using terms. What terms look like is
regulated by the underlying term language, which depends on the domain being
modeled. For the logic to be truly first-order, the term language shall admit logical
variables as terms and generally within terms. Besides this, it can be arbitrary.1 In
this book, we will be considering a term language consisting of just constants such
as “Sue”, “A-” and “4.00”, in addition to variables. In a few specific places, we will
also use terms drawn from the language of scalar spreadsheet expressions. We will
write t, variously decorated, for terms, and use capital letters, generally from the
end of the alphabet, to denote variables. A term that does not contain any logical
variable is called ground.
With terms describing objects in the domain of interest, basic relationships
between objects are expressed using predicates over the corresponding terms.

1
Many authors fix the term language of first-order logic to be the free algebra generated by a set
of constants, an infinite set of variables, and a set of constructors. Although we will rely on a
simple instance of precisely this scheme, there is no reason not to allow more liberal languages,
for example, featuring types, equational rewriting, or binders.
5.1 A First-Order Logic Refresher 121

A predicate has the form p.t1 ; : : : ; tn /, where the predicate symbol p identifies
which of the many relationships among objects t1 ; : : : ; tn we are referring to. For
example, in a domain of natural numbers, 3  5 and 3 D 5 describe two
relationships between the terms 3 and 5, where we have used the standard infix
notation for the predicate symbols  and D. Notice that predicates describe both
relationships that hold (such as 3  5) and relationships that do not hold (such as
3 D 5), where the meaning of “hold” will be discussed later.
Properties of the domain of interest are described using logical formulas obtained
by combining predicates by means of the standard constructors of first-order logic.
These are the logical constants true and false, the connectives ^ for conjunction,
_ for disjunction,  for implication, and : for negation, and the two quantifiers
“for all X ” and “exists X ”, written 8X and 9X . We write ' for a logical formula.
The quantifiers bridge the logical layer of predicates and logical constructors,
and the term layer. A variable X in a formula ' is bound if it occurs within the
scope of the quantifier 8X or 9X . It is free otherwise. If ' contains a free variable
X , we can substitute every occurrence of X (that is not in the scope of a quantifier
8X or 9X ) in ' with a term t, an operation that we denote by Œt=X '. A formula
that does not contain any free variable is closed.
Properties are used either to describe the characteristics of the domain of interest,
or to represent consequences of these characteristics. Both types are expressed using
formulas. The formulas corresponding to the descriptive properties of a domain
are variously called a theory, a set of assumptions or hypotheses, a program, a
context, etc. We will generally use the first of these terms and collectively denote
them by .
Before proceeding further, we summarize the various syntactic categories intro-
duced so far in the following grammar:

Terms t WWD X j : : :
Predicate P WWD p.t; : : : ; t/
Formulas ' WWD P j true j false j :' j ' ^ ' j ' _ '
j ' ! ' j 8X:' j 9X:'
Contexts WWD j ; '

Naturally, one of the key questions in logic has to do with establishing whether
a formula ' is a consequence of a set of assumptions . We will now review two
major approaches to doing so for a particular flavor of logic known as classical first-
order logic. The first checks whether ' is a valid property in all the domains in which
each property in is valid.2 The second defines mechanical rules for establishing
logical inferences on the sole basis of the form of ' and .

2
The correspondence between a domain of interest and a logical representation is an entirely
different question.
122 5 The Logical Spreadsheet

5.1.2 Model Theory

The first approach to formalizing logical inference comes from the subfield of
mathematical logic known as model theory. It goes back to the logician Alfred
Tarski. A formula ' is a logical consequence of a set of assumptions , which we
will write ˆ ', if any domain that satisfies every formula in also satisfies '.
A domain is some set D. An interpretation, I , associates every ground term to an
object in D and every n-ary predicate to a subset of Dn . I maps a term with variables
X1 ; : : : ; Xm to a function that when given m domain objects returns a domain object.
An assignment ˛ associates every variable X with some object in D. Together, they
allow mapping any term, possibly with free variables, to an object in D. We write I ˛
for their composition. Then, the validity of a formula ' with respect to interpretation
I and variable assignment ˛, denoted by I; ˛ ˆ ', is inductively defined as follows:

I; ˛ ˆ true always
I; ˛ ˆ false never
I; ˛ ˆ p.t1 ; : : : ; tn / iff .I ˛ .t1 /; : : : ; I ˛ .tn // 2 I.p/
I; ˛ ˆ :' iff it is not the case that I; ˛ ˆ '
I; ˛ ˆ '1 ^ '2 iff I; ˛ ˆ '1 and I; ˛ ˆ '2
I; ˛ ˆ '1 _ '2 iff I; ˛ ˆ '1 or I; ˛ ˆ '2
I; ˛ ˆ '1  '2 iff I; ˛ ˆ '1 implies I; ˛ ˆ '2
I; ˛ ˆ 8X:' iff for all d 2 D it is the case that I; ˛X 7!d ˆ '
I; ˛ ˆ 9X:' iff there is d 2 D such that I; ˛X 7!d ˆ '

where ˛X 7!d is the assignment that differs from ˛ by mapping X to the domain
object d . An interpretation and variable assignment pair .I; ˛/ is a model for a
formula ' if I; ˛ ˆ '. Note that the choice of ˛ is irrelevant if ' is closed.
A formula ' is a logical consequence of formulas '1 ; : : : ; 'n , written
'1 ; : : : ; 'n ˆ ', if for every interpretation I and variable assignment ˛ such
that I; ˛ ˆ 'i for all i D 1::n, it is the case that I; ˛ ˆ '.

5.1.3 Proof Theory

The second approach to determining whether a formula ' follows from a set of
assumptions ignores any domain that may be a model of ' and . Instead, it
only looks at the symbols that occur in ' and . It was pioneered by David Hilbert
and Gerhard Gentzen. The interpretation of every logical constructor is given by
a set of purely syntactic inference rules, while predicates and terms are generally
not interpreted. The notion corresponding to model-theoretic logical consequence is
now called derivability: “' is derivable from ” is written ` '. It is established
by chaining a finite number of inference rules into a derivation or proof of ` '.
5.2 The Logic of the Relational Spreadsheet 123

Here are two examples of inference rules, taken from a rule set known as the sequent
calculus:

` '1 ` '2
. ; '; / ` '
0
` '1 ^ '2

The first states that ' is derivable from any context containing '. The second says
that if both '1 and '2 are derivable from , then '1 ^ '2 is derivable from .
The form ` ' above is called a judgment and it expresses a relationship
between various syntactic entities, here and '. This particular type of judgment
is called a sequent. The derivability of a judgment is established by means of
inference rules such as the above. The judgment underneath the horizontal line is
called the rule’s conclusion. A rule also contains zero or more judgments above the
line: they are its premises. Given derivations of instances of all judgments in the
premises, the rule integrates them into a derivation for the corresponding instance
of its conclusion.
A set of inference rules aimed at establishing derivability in first-order logic is
called a proof system. Well-designed proof systems satisfy the correctness theorem,
which requires that ` ' iff ˆ ' for every ' and . There are several such
proof systems. We will not discuss them further in the general context of first-order
logic. We will however present a proof system for the fragment of this logic that is
needed to interpret relational expressions.
Proof-theoretic derivability may appear more readily mechanizable than model-
theoretic logical consequence since the former refers to a finite set of rules while
the latter makes uses of arbitrary domains and interpretations. Not necessarily so.
Establishing derivability is undecidable for full first-order logic, which puts the
apparent mechanizability of proof theory into jeopardy, in general. On the other
hand, some fragments of first-order logic are such that it is sufficient to examine a
single domain and use very specific interpretations. This will be the case below.

5.2 The Logic of the Relational Spreadsheet

In theory, an entire spreadsheet can be seen as a logical formula. Our interpretation

will however be much more restricted and concentrate on the purely relational
aspects since the array and scalar components are already handled quite well
by standard approaches, such as the one outlined in Chap. 3.3 Viewing relational
concepts in logical terms has several advantages. First, it provides an alternative
way of thinking about the entities modeled in a relational spreadsheet. This is likely
to be beneficial to a user who is more inclined to viewing a problem logically rather

3
See footnote 4 for a brief description of how to incorporate scalar and array formulas in the present
discussion.
124 5 The Logical Spreadsheet

than relationally. Second, it will generalize and somewhat simplify the evaluation
methods presented so far, revisiting them within a branch of knowledge with a
much longer history and far more depth than our embryonic study of relational
spreadsheets. Third, it will pave the road for the truly deductive extension of the
spreadsheet in Chap. 6.
We identify the portion of the relational spreadsheet to which we will give a
logical interpretation in Sect. 5.2.1. We then define the translation of relations and
relational expressions in Sect. 5.2.2. Finally, we account for non-relational aspects
in Sect. 5.2.3.

5.2.1 Relational Substrate

The portion of a relational spreadsheet s that is amenable to a logical interpretation

consists of all the relations mentioned in s. We will call it the relational substrate
of s, denoted by RS N s . The relations contributing to it can be collected from three
places:
• First, we have the set of all base relations hEei mentioned anywhere in s. Formally,
they are given by the set

N sbase D fhEei W hEei vr s.El / for El 2 dom sg

RS

Notice the use of vr since we are tracking entire relations rather than individual
attributes.
• Second, we shall consider every relational expression eN D s.El/ hosted by any
array reference El in s. It will be convenient to identify them not through the
expression eN itself but by the reference El, from which eN can be recovered. It is
formally defined by the set:

N sref D fhEli W El 2 dom s and s.El/ 2 Eg

RS N

We place each of these array references in angled brackets because they are often
used as base relations in other relational expressions. This set is therefore not
disjoint from the previous one, in general.
• Finally, we must consider all relational expressions ŒeN embedded within any
array formula in s. This fragment is formally given by the following set:

N semb D fŒe
RS N vr s.El / for El 2 dom sg
N W Œe

An example is the subexpression StudentDCA3WCA HasTakenNum in Fig. 4.8.

N s D RS
Then, RS Nsbase
N s [ RS
[ RS
ref
Ns emb
.
5.2 The Logic of the Relational Spreadsheet 125

It will be convenient to use a symbolic name to identify each element of RS N s.

E
Whenever the array reference l has an associated range name, we will use it as an
identifier; for example Students in our case study. If it does not, we will simply use
El itself, as in BP 3WBP . For elements of the form hEe i (where Ee is not an array of
N we simply invent some unique identifier, for example we may
locations) and Œe,
identify the embedded relational expression above as Fig 4.8. In the following, we
will often rely on these symbolic names when referring to the element of RS N s they
identify.
The relational substrate of s can be conveniently organized into a structure that
highlights the dependencies among its elements. This structure will be a graph RSs
with much in common with the relation-dependency graphs DGrs and in particular
DGR s . The main difference will be that, while these graphs track dependencies
at the level of locations, RSs instead focuses on dependencies in the relational
substrate of s, some of whose elements are not associated with locations.
We will be mostly interested in those dependencies between elements of RS N s that
are purely relational, i.e., not mediated by array or scalar formulas. For example,
all relations in the dependency graph in Fig. 4.14 are related in this way. Indeed,
relation StudentsWithAs uses relations Students, HasTaken and Grades as its base
relations (see Fig. 4.6 for the definition of StudentsWithAs). Similarly, the location
array BP 3WBP contains a relational expression that references relation Grades.
Although relation Grades50 (in Fig. 4.7) depends on Grades in DGR s , their
dependency is not purely relational since Grades50 does not contain an expres-
sion mentioning Grades (although its Letter attribute does). We will record this
functional dependency in RSs as a different type of edge from the purely relational
dependencies above.
The relational substrate graph of s is a bi-graph RSs with nodes among (the
N s and two types of edges: n1 ! n2 will denote a purely relational
identifiers of) RS
dependency of n2 on n1 , while n1 Ü n2 will indicate a functional dependency.
Formally, RSs is defined as follows:
• Nodes.RSs / D RS N s.
• Edges.RSs / is the set of edges n1 ! n2 and n1 Ü n2 defined as follows:

hEei ! hEli iff hEe i vr s.El /

N iff hEe i vr eN
hEei ! Œe
n1 Ü n2 iff n1 @+ rs n2 but n1 ! n2 62 Edges.RSs /

The relational substrate graph of the portion of the registrar’s example shown in
Figs. 4.3, and 4.6–4.8 is displayed in Fig. 5.1. The added relations TAs and AStudents
will illustrate an important concept in a later section.
We write n1 Ü n2 for a path that may traverse either kind of edge. In contrast,
we write n1 ! n2 for a path from n1 to n2 that traverses only ! edges in RSs . Note
that embedded relations of the form ŒeN occur at most as the last element within such
126 5 The Logical Spreadsheet

F&L Names

TAs

Instructors Members StudentsWithGPAs

CA3:CA F ig 4.8

Students HasTakenNum BigGapStudents

HasTaken StudentWithAs BC3:BE Aux BigGapStudents

Grades AStudents

Courses GradeInc

Grades5

BP 3:BP Grades5

Fig. 5.1 Relational substrate of the registrar’s spreadsheet

a path. For most of this section, we will be primarily interested in the subgraph of
!
RSs consisting uniquely of its ! edges. We denote it by RSs .

!
Because a spreadsheet cannot contain any circular dependency, RSs is acyclic.
Note however that RSs may contain cycles through Ü edges because of its foun-
dation on vr (see Sect. 4.3.3 for a counterexample). There is a simple embedding of
RSs into the abstract relational dependency graph DGR r
s as well as in the simpler DGs .
0 R
A node n in RSs corresponds to the node n in DGs that hosts the relational object
labeling n. An edge in RSs is mapped to a path in DGR s . This path is empty if its
end-nodes are identified in DGR s : this may happen for example if an array reference
El contains a formula with hEei appearing within a subexpression Œe. N

!
We will be particularly interested in the nodes of RSs without any incoming edge.
We will call them primitive relations. They correspond to objects in the spreadsheet
that are used exclusively as base relations. In particular, they are not obtained by
evaluating a monolithic relational expression. Examples in the registrar spreadsheet
include Students, Grades, but also Grades50 and StudentsWithGPAs.
5.2 The Logic of the Relational Spreadsheet 127

5.2.2 Relational Algebra as Logic

We will now describe a well-known interpretation of relational algebra as a

particularly favorable fragment of first-order logic as far as mechanizability is
concerned [CGT90]. Intuitively, each base relation and relational expression in
a spreadsheet s will be translated into logical formulas whose predicates will
correspond to relations in RSs . We dedicate this section to motivating this embedding
on the basis of our ongoing example and to formalizing it. The particular flavor
of first-order logic that emerges from this translation will be studied in Sects. 5.3
and 5.4. We will then relate spreadsheet evaluation and logical inference. In Chap. 6,
we will extend this embedding to gain in deductive power without losing any
computational property.

Primitive Relations

!
Let p identify an n-ary primitive relation in RSs . Assume that it has been evaluated
to vN (a preliminary step we will not undertake until Sect. 5.5). We can then define
a theory in first-order logic that describes p as vN in s. We take the term language
of this logic to be the set V of all scalar values, in addition to logical variables.
We then consider an n-ary predicate symbol, which we also call p, and define the
logical theory …p associated with p as follows:

…p D fp.Ev/ W Ev 2 vN g

It is then intuitively clear that, for any v1 ; : : : ; vn 2 V, the formula p.v1 ; : : : ; vn / is

a logical consequence of …p if and only if .v1 ; : : : ; vn / 2 vN :

…p ˆ p.v1 ; : : : ; vn / iff .v1 ; : : : ; vn / 2 vN

We will substantiate this intuition in Sect. 5.3. Notice that each attribute of the
relation p corresponds to a specific argument of the predicate p.
We can try this out on the Students relation in Fig. 4.3, which is primitive
according to the relational substrate in Fig. 5.1. Then …Students is given by:

Students.“Sue”; “Smith”; “Sue Smith”/

Students.“Alan”; “Bond”; “Alan Bond”/
Students.“Nick”; “Vitter”; “Nick Vitter”/
Students.“Sue”; “Lee”; “Sue Lee”/

Then, …Students ˆ Students.“Sue”; “Smith”; “Sue Smith”/, but …Students 6ˆ

Students.“Sue”; “Bond”; “Sue Bond”/, unless such a record can be found in the
omitted part of the relation Students.
128 5 The Logical Spreadsheet

A ground predicate p.Ev/ in the above logical interpretation of a primitive relation

p is called a fact and therefore the resulting collection of facts is a factual theory.
We will write …p for the factual theory of primitive predicate p, and …s for the
union of the factual theories of all primitive predicates in RSs :
[
…s D …p
p 2 RSs
p primitive

Non-primitive Relations

Verifying whether …Students ˆ Students.Ev/ holds hardly requires any logical

inference since it simply boils down to checking whether Students.Ev/ occurs in
…Students . More complicated formulas on the right-hand side of ˆ bring us more
interesting results. Consider for example:

…Students ˆ Students.v1 ; v2 ; v3 / ^ v1 D “Sue”:

Then it is clear that this judgment will hold for precisely those triples .v1 ; v2 ; v3 / 2
Students such that v1 D “Sue”. Let us collect them in a relation that we
call StudentsSue. We have seen a relational expression that computes precisely
StudentsSue in Chap. 4:

FirstD“Sue” Students

Indeed, we used it to explain the selection operator in Sect. 4.3.2. In order to

interpret this relational expression into logic, we introduce the new predicate symbol
StudentsSue that will correspond to the homonymous relation StudentsSue, and
construct a logical theory such that ˆ StudentsSue.v1 ; v2 ; v3 / if and only if
.v1 ; v2 ; v3 / 2 StudentsSue. The above logical consequence and our interpretation of
the triples .v1 ; v2 ; v3 / it mentions suggest a way to do so: if Students.v1 ; v2 ; v3 / is true
and v1 D “Sue”, then StudentsSue.v1 ; v2 ; v3 / is true. Since this sentence holds for
arbitrary values of v1 ; v2 and v3 , we can abstract them away by replacing them with
universally quantified variables F , L, and I (we have chosen these variable names
in an attempt to recall the meaning of the corresponding attributes: first names, last
names, and identifiers, respectively). This yields the formula:

8F:8L:8I: Student.F; L; I / ^ F D “Sue”  StudentsSue.F; L; I /

Let StudentsSue be the theory that contains just this formula; then, again appealing to
the intuition, we have that

…Students ; StudentsSue ˆ StudentsSue.v1 ; v2 ; v3 /

iff .v1 ; v2 ; v3 / 2 Students and v1 D “Sue”
iff .v1 ; v2 ; v3 / 2 StudentsSue
5.2 The Logic of the Relational Spreadsheet 129

Therefore, the theory we were looking for is simply the union .…Students ;
StudentsSue / of …Students and StudentsSue .
The above implicational formula defines the predicate StudentsSue in the same
sense as the shown relational expression defines the relation StudentsSue. We can
put them next to each other to compare their structures:

8F:8L:8I:
Student.F; L; I / ^ F D “Sue” FirstD“Sue” Students
 StudentsSue.F; L; I / DW StudentsSue

where the last line on the right informally indicates that the expression above
it defines StudentsSue. First, observe that each variable in the formula captures
an attribute in the expression: F , L, and I positionally identify the attributes
First, Last, and Student of each record of the relation Students, and similarly for
StudentsSue; moreover, F is compared to “Sue” in the formula in the same way as
First is compared with this value in the expression. Second, notice that the formula
defines the predicate StudentsSue tuple by tuple while the expression operates in a
relation-at-a-time fashion. This difference in granularity is mediated by universally
quantifying the variables F , L, and I over the whole logical formula. Finally,
observe that the behavior of relational selection is emulated in the logic by taking the
conjunction of its condition with the formula that represents the relational argument,
Student.F; L; I / here.
The observation that logical variables correspond to attributes suggests a simple
way to logically emulate the projection operation of core relational algebra: simply
omit projected-away variables from the arguments of defined predicates. Indeed,
the following logical and relational expressions both describe the set of tuples of
StudentsSue deprived of its redundant first component (which invariably holds the
value “Sue”):

8F:8L:8I:
Student.F; L; I / ^ F D “Sue” Last;Student FirstD“Sue” Students
 StudentsSue0 .L; I / DW StudentsSue0

The formula differs from the definition of StudentsSue by the fact that we
have dropped F as an argument to the defined predicate. Note that the
original relational expression for StudentsSue can be equivalently rewritten as
First;Last;Student FirstD“Sue” Students, which brings us even closer to the corresponding
logical formula.
This exercise can easily be continued for all the other operators of core
relational algebra as well as their extensions in Sect. 4.3.2: union then corresponds
to disjunction, difference becomes a conjunction with a negated subformula, and
Cartesian product reduces to the conjunction of two formulas that do not share any
variable. We will now formalize this interpretation by defining a translation function
from relational expressions to logical formulas.
130 5 The Logical Spreadsheet

Let p be a non-primitive node in the relational substrate RSs of a spreadsheet s.

By definition, p has either the form hEli with s.El/ D e,
N or the form Œe.
N In both cases,
eN defines p. We logically capture this association by means of the formula

E ; YE : peq.
8X N X/E  p.X
E/

where peqN represents the logical interpretation of the relational expression e,

N and
E ; YE denotes the logical variables corresponding to the attributes mentioned in the
X
N with YE being projected away. The logical interpretation peq.
base relations of e, N X E/
N
of the relational expression is defined by induction on the structure of e:

E hEe i / D phEei .X
phEeiq.X E hEei /
pR1 [ R2 q.X/E D pR1 q.X E / _ pR2 q.X
E/
E E
pR1  R2 q.X/ D pR1 q.X / ^ :pR2 q.X E/
E ; YE / D pR1 q.X
pR1  R2 q.X E / ^ pR2 q.YE /
pæE Rq.YE / D YE D æ E / ^ pRq.X
E .X E/
pC Rq.X/ E D C.X/E ^ pRq.X E/

The first line handles base relations hEe i identified in RSs by the symbol phEe i , which
we reuse as a predicate symbol representing hEei. The attributes of this relation are
mapped to the tuple of distinct variables X E hEe i , which is given as the argument of phEe i
and recorded on the left-hand side as the variables to be exported to higher levels.
The next two lines describe the rather simple translation of union and difference.
Note that the two subformulas are required to use the same variables. The treatment
of Cartesian product on the next line is similar but the variable exported by the
interpretation of the two subrelations shall be disjoint (otherwise there would be a
dependency between them) and both sets contribute to the variables exported by the
overall formula. The next line defines the translation of the generalized projection
of a relational expression eN onto a sequence æ E of attribute expressions. For the time
being, we shall assume that æ E does not mention any cell reference (or that they have
been replaced by values in advance). The notation æi .X E / indicates the term obtained
from æi by replacing each attribute it mentions by the corresponding variable in
XE . The projection as a whole assigns each term æi .X E / to a new distinct variable
E
Yi . The notation Y D æ E
E .X / is meant to indicate the conjunction of all equalities
Yi D æi .X E /. Notice that it is the YE ’s that are passed on by the overall formula, not
the XE ’s. The final line handles selection: the notation C.X E / indicates the natural
interpretation of the condition C as a logical formula with attributes replaced with
the corresponding variable in X E . Again, we assume that the embedded attribute
values do not mention cell references.
5.2 The Logic of the Relational Spreadsheet 131

The class of first-order formulas that emerges is described by the following

grammar:

Clauses Cl WWD B  H j 8X:Cl

Heads H WWD P
Bodies B WWD P j æ  æ j B ^ B j B _ B j :B
Predicates P WWD p.X; : : : ; X /

Each implicational formula is called a clause and each logical variable occurring in
it is universally quantified on its outside. A clause is always closed. The consequent,
or head, of the implication is the predicate being defined (such as StudentsSue
above). Its antecedent, or body, is the formula defining it. It can mention predicates
(such as Students), conditions of the general form æ  æ with variables for
attribute names, and they can be combined by means of conjunction, disjunction
and negation. Within this logical interpretation, we will refer to conditions æ  æ
as constraints. In particular, the form X D æ such that X appears in the head will
be a head constraint.4
This fragment of first-order logic casts our formalism as a member of the
family of logic programming languages [Llo87, MNPS91], whose most successful
offspring is Prolog [CM87, O’K90, SS94]. The presence of constraints further
defines it as part of the constraint logic programming branch [JM94], while the
particular type of terms it uses flags it as a flavor of Datalog [CGT90], which,
not surprisingly, underlies most work on deductive databases. We will give a more
precise characterization in Chap. 6, since it will be lacking one important aspect of
logic programming languages until then.

4
This translation can easily be extended [vEOT86] to express the scalar part of a spreadsheet by
associating each cell l with a unary predicate l.vl / containing the value of l. Then, every scalar
expression e is interpreted into a pair e  D .æ; B/ as follows:

l D .Ll ; l.Ll // where Ll is a variable uniquely assigned to cell l

v D .v; true/
op.e1 ; : : : ; en / D .op.æ1 ; : : : ; æn /; B1 ^ : : : ^ Bn / where ei D .æi ; Bi / for i D 1::n

Finally, every binding l 7! e in a spreadsheet s is modeled by inserting the formula

X D æ ^ B  l.X/

into cell l, where e  D .æ; B/.

While this approach extends to array formulas, it does not take into consideration partial
interactions with calculated relational expressions, such as in Fig. 4.7 where a scalar expression
draws its input from a relational result. Although this could be addressed using the techniques
described in Sect. 5.2.3, we are more inclined to take the more conservative path of relying on
tested approaches.
132 5 The Logical Spreadsheet

The correspondence between relational attributes and logical variables imposes

some restrictions on the occurrences of the latter. Specifically:
• Because projection operates only on the attributes of the embedded base rela-
tions, a variable X occurring in the head of a clause also must occur
– Either in a predicate p.XE / in the body of the clause,
– Or as the left-hand side of a head constraint X D æ.
• Similarly, since selection conditions can only operate on attributes of base
relations, any variable X occurring within a conjunct æ1  æ2 , or on the right-
hand side of a head constraint Y D æ, must also occur in a predicate p.X E / in the
body of the clause.
• Because union operates on relations that have the same attributes, whenever
a variable X occurs in the head of a clause and in the subformula Bi of a
disjunction B1 _ B2 in its body, it shall also appear in the other argument B3i .
Making this requirement conditional on X occurring in the head accounts for the
possibility of projecting away attributes before taking their union.
• Since relational difference too operates on relations with the same attributes,
whenever a variable X occurs in the head of a clause and within a negated
subformula :B in its body, it shall also occur in a body formula that is not within
the scope of a negation.
These restrictions are typical of Datalog [CGT90].

!
The graph RSs , which includes only the ! edges of the relational substrate of s,
has a simple interpretation in this language: there is an edge from node q to node p

!
in RSs if and only if predicate q appears in the clause defining p, as per the above
translation. We will take this as the definition of the call graph of any collection
of clauses , independently of whether it was obtained from a spreadsheet. We will
denote it by CG .
Before demonstrating the above translation on more examples from the regis-
trar’s spreadsheet, we introduce a number of convenient and commonly accepted
conventions to improve on readability. They are inspired by standard practices in
logic programming [Llo87, O’K90, SS94]. Specifically,
• Since all quantifiers are universal and all appear at the periphery of a clause, we
can safely drop them, under the understanding that Cl.X E / stands for the closed
E E
clause 8X :Cl.X /. In order for this convention to be usable, we must assume that
we can always tell a variable apart from a constant value.
• Every clause has an implication as its top-level connective and its consequent
is always the predicate that is being defined. It is common practice in logic
programming to reverse the direction of the implication (as well as its argument)
to emphasize the definitional aspect of a clause. Our example above is then
written

StudentsSue.F; L; I / 
Student.F; L; I / ^ F D “Sue”
5.2 The Logic of the Relational Spreadsheet 133

It is read ‘StudentsSue.F; L; I / if Student.F; L; I / and F D “Sue” ’.

• A subformula of the form X D Y , where both X and Y are variable and do not
both occur in the head, is generally dropped by replacing every occurrence of Y
with X (or vice versa) in the clause where it occurs. Note that we have already
done this in the clause defining StudentsSue0 . This simplification has the effect
of turning the translation of every natural join into a conjunction of subformulas
sharing variables.
• Assume that a clause Cl contains the subformula B1 ^ :B2 and that B2 is a
composite formula, i.e., not a predicate. It is convenient to factor out B2 by
introducing a new predicate symbol of the appropriate arity—call it aux—and
replacing Cl with two clauses:
E /,
– A clause Cl0 that differs from Cl by replacing B1 ^ :B2 with B1 ^ :aux.X
and
– The clause aux.XE /  B2 .

where X E are the variables occurring in B2 that are used in other parts of
Cl. This process can be iterated if B2 contains negative subformulas. If every
comparison symbol  has a negated form 6 (for example < and ), the
systematic application of this transformation results in clauses where negation
only precedes predicates. A negated predicate :P in the body of such a clause
is called a negative literal, which suggests the appellation positive literal for a
body predicate P that is not preceded by a negation.
• In the context of a relational spreadsheet s, assume that a clause Cl contains
E / and phEe i .YE / where Ee 2 depends on Ee 1 , i.e., Ee 1 @+ rs Ee 2 .
distinct predicates phEe1 i .X 2
This functional dependency is conveniently handled by replacing any occurrence
E / in Cl with aux.X
of phEe 1 i .X E /, where aux is a new predicate symbol, and by
E
adding the clause aux.X /  phEe1 i .X E /.
• By distributing disjunction over conjunction, we can obtain clauses of the form
p.XE /  B1 _ : : : _ Bn where each Bi is a conjunction of predicates, negated
predicates, or base conditions, for i D 1::n. Then, we can replace this clause
with the n clauses:

E /  B1
p.X

E
p.X /  Bn

Overall, these simplifications (all of which have a logical justification) have two
major effects. First, a non-primitive node p in the relational substrate RSs is now
associated not to a single clause defining p, but to a set of clauses, which we collect
into a clausal theory that we will call p . On the other hand, the body of each of
these clauses is much simpler than in the general translation. We will come back to
the exact structure of these clauses in Sects. 5.3 and 5.4, where we will study them
in great detail. We collect the clauses contributed in this way by every non-primitive
134 5 The Logical Spreadsheet

predicate in RSs in the clausal theory of the spreadsheet s, which we denote by s .

Formally,
[
s D p
p 2 RSs
p non-primitive

At this point, we can combine the factual theory …s of s and its clausal theory
s into a single theory that we denote by the symbol … s :
0 1 0 1
B C B C
B [ C B [ C
… s D …s [ s B
D B …p C B p C
C[B C
@ p 2 RS A @ p 2 RSs
A
s
p primitive p non-primitive

Summarizing, the factual subtheory …p contributed by a primitive node p is the list

of all the predicates p.Ev/ such that Ev 2 vN , where vN is the relational value of p. Such a
p is sometimes called an extensional predicate and the set …p is then its extension.
The clausal subtheory p of a non-primitive node p defining a relational expression
eN contains all the clauses derived from 8X E ; YE :peq.
N X E /  p.X E / using the above
simplifications. Such a p is occasionally called an intensional predicate.
The main effect of the above simplifications is the factorization of complex
negative body subformulas as separate clauses. Notice that this very same effect
could have been achieved directly in the relational spreadsheet by defining auxiliary
relations for the corresponding formulas, all of which occur as the second argument
of the difference operator. If sN is the result of this transformation on a relational

!
spreadsheet s, then it is easy to see that its relational substrate graph, RSsN , is
isomorphic to the call graph CG… s of the simplified theory corresponding to s.

!
Compared to RSs , it may contain added nodes corresponding to the auxiliary
predicates introduced to factor out complex negative subformulas and dependent

!
base relations, and the relative edges. However, as in RSs , the primitive predicates

!
of RSsN are identified in CG… s by precisely those nodes without any incoming !
edge. Notice that RSsN and RSs contains Ü edges between the very same nodes. In
particular, the added nodes of RSsN are never the source or the target of any such edge.
Figure 5.2 shows the theories resulting from translating the majority of the
relational expressions displayed in Figs. 4.3 and 4.6–4.8, as well as in the text
of Chap. 4, for the registrar’s spreadsheet. The original relational expressions are
reported on the left-hand side. As a further aid to reading the resulting formulas, we
have replaced every variable occurring exactly once in a clause with the symbol “ ”.
We have used the same symbol in the last row, to avoid giving a name to predicates
corresponding to relations ŒeN embedded in array or scalar formulas (we called this
particular instance Fig 4.8 earlier). Figure 5.3 shows the spreadsheet fragments in
Figs. 4.6 and 4.7 with logical formulas instead of relational expressions.
5.2 The Logic of the Relational Spreadsheet 135

Fig. 5.2 Logical expression of selected registrar’s relations

5.2.3 Stratification

We need to take into account the fact that there may be functional dependencies
between nodes of the relational substrate RSs of s (or equivalently in RSsN ), which
are not accounted for in … s . Indeed, the relation Grades50 defined in Fig. 4.7 is
primitive in the relational substrate graph in Fig. 5.1 (also in Fig. 5.4). However,
its values will not be available until the relation corresponding to node BP 3WBP
has been fully calculated. Any use of Grades50 in determining logical consequences
136 5 The Logical Spreadsheet

Fig. 5.3 Logical rendering of the registrar’s spreadsheet

should not be allowed until then. Similar observations hold for relations Aux and
StudentsWithGPAs.
In order to handle this issue, we will partition the call graph of … s into
functionally dependent strata that are internally free from functional dependencies.
No logical consequence in one stratum should be attempted until all the logical
consequences of the strata it depends on have been made available. In order to do
so, we shall operate on RSsN , which simply imports the functional dependencies of
RSs , given by its Ü edges, into CG… s .
5.2 The Logic of the Relational Spreadsheet 137

Fig. 5.4 Stratified call graph in the registrar’s spreadsheet

For reasons that will be explained shortly, it is convenient to view any negated
predicate :q occurring in … s as a node :q separate from q but functionally
dependent on it. Therefore, we extend RSsN with the node :q for every predicate
symbol q such that :q occurs in … s , as if it were a primitive predicate itself. If :q
occurs in the definition of a predicate p, then the updated graph will contain the edge
:q ! p rather than q ! p (unless p also depends positively on q, in which case
both edges will be present). Finally, we add the functional dependency q Ü :q
for every such q. We call the graph obtained by this transformation the stratified
relational substrate graph, or more simply the stratified call graph, of s, and denote
fSs . Note that its nodes are literals rather than predicates. The stratified call
it by R
graph corresponding to the relational substrate in Fig. 5.1 is displayed in Fig. 5.4.
Please ignore the dotted line for the time being.
A set & of predicates in … s constitutes a stratum if no path in RfSs between any
two predicates p; p 0 2 & contains a functional dependency (i.e., an Ü edge). In
particular, if p 2 & and :q occurs in a clause defining p, then q 62 &. Let R fSs&
be the subgraph of R fSs whose nodes are the predicates in & plus all the literals
occurring in any of their defining clauses, and whose edges are the restriction of the
edges of RfSs to just those nodes. Note that RfSs& may contain functional dependency
edges. However, if n Ü n occurs in R
0 fSs& (for example if s contains the clause
138 5 The Logical Spreadsheet

p.X; Z/  q.X; Y / ^ :q.Y; Z/ and p 2 &), then n; n0 62 &: in particular, neither

& & &
can be the target of a ! edge. A stratum & identifies the subtheory … s D .…s ; s /
of … s containing all the clauses or facts defining the predicates in &. Formally,
&
[
s D p
p2&

&
[ [ [
…s D …p [ …q [ …q
p2& q 62 & q 62 &
P B^q.XE / 2 s& P B^:q.X E / 2 s&

&
The clausal part s is simply given by the set of clauses defining the non-primitive
&
predicates among &. Its factual component …s collects the theories of every
primitive predicate in &, as well as the theory of every predicate q that does not
belong to & but that is mentioned positively or negatively in any defining clause of a
predicate in this stratum. These are treated as if their predicates were primitive. We
will reconcile this aspect with their possible clausal nature in Sect. 5.4.
Figure 5.4 shows two strata, &0 and &1 , whose predicates are the positive literals
on each side of the dotted line. This figure also identifies the subgraphs R fSs&0 and
fSs 1 (the latter also contains the origin of every ! edge crossing into it).
R
&

A stratification is a partition &E D &0 ; : : : ; &n of the set of all predicates occurring
in … s such that &i is a stratum for i D 0::n and if pi Ü pj , for pi 2 &i and
pj 2 &j , then i < j . The functional dependencies among predicates in different
strata induce a functional dependency graph FDs .E& / among strata: its nodes are the
strata of the stratification, and it contains an edge & Ü & 0 if there are p 2 & and
p 0 2 & 0 such that p Ü p 0 in R fSs . By definition this graph is acyclic.
The theory … s is stratifiable if it has a stratification. It is easy to verify that this
is the case exactly when the relational substrate graph RSs is acyclic, which in turn
means that the relational dependency graph DGrs of s is acyclic. In this book, we will
concentrate on stratifiable theories and the spreadsheets that generate them. This
excludes pathological situations such as the one discussed at the end of Sect. 4.3.3,
although our model could be refined to accommodate them.
A stratifiable theory … s may have several stratifications, in general: in the
extreme, each predicate can belong to its own stratum. We will be interested in
stratifications with a minimal number of strata. A stratification &0 ; : : : ; &n is linear
if &i Ü &i C1 for i D 0::n  1. A theory … s has always at least one linear
stratification. A generic stratification can indeed be linearized by merging strata, as
long as functional independence is preserved. Among all the linear stratifications of
s, we will be interested in the one where each predicate occurs in the stratum with the
lowest possible index, short of violating the functional dependency constraint. This
is the least stratification of s. In Fig. 5.4, .&0 ; &1 / constitute a linear stratification
of the set of predicates in the call graph of the registrar’s example. It is its least
stratification.
Stratification is a standard approach for harnessing negation within logic pro-
gramming [CH85, ABW87, Llo87]. It is particularly effective in the context of
deductive databases [CGT90]. It is noteworthy that this very same mechanism
5.3 Inference in a Definite Spreadsheet Theory 139

provides a simple and elegant method to modeling the functional dependencies

appearing in a spreadsheet.
Given any linear stratification &E D &0 ; : : : ; &n , the first stratum &0 has several
&
interesting properties. First, any predicate mentioned in any clause of s 0 necessar-
ily belongs to &0 . Second, it does not contain negative literals in the body of any of
fSs&0 contains no Ü edges. Clauses with this property are called
its clauses. Third, R
&
definite, and therefore s 0 is a definite theory. Note that, while the bottom stratum of
every stratification is always definite, additional definite strata may be present, even
when considering the least linear stratification. Indeed, functional dependencies may
cause a set of definite clauses to be stratified.
Definite theories have received a lot of attention, especially as a foundation
for logic programming [Llo87] and its numerous applications. We will dedicate
Sect. 5.3 to adapting those results to the particular notion of definite theory that
emerged from the relational spreadsheet. In particular, its model theory will provide
us with a logical way of evaluating (the relational part of) a spreadsheet, and its
proof theory will be the key to a flexible explanation mechanism. We will bring
negation back into the picture in Sect. 5.4.

5.3 Inference in a Definite Spreadsheet Theory

In this section and the next, we will study those first-order theories stemming from
the logical interpretation of the relational spreadsheet. We first analyze definite
theories, which characterize the least stratum of any stratification. This fragment
of first-order logic, especially its constraint-free sublanguage, is at the very core of
logic programming [Llo87], and as such, it has been the subject of intense scrutiny
in the past 50 years [Rob65, Kow79, MNPS91]. Although the specific language
we are considering does not seem to have received much attention, it is at the
confluence of two important research streams: constraint logic programming [JM94]
and Datalog [CGT90].
In Sect. 5.3.1, we summarize and formalize their syntax and describe two impor-
tant types of graphs associated with them. In Sects. 5.3.2–5.3.4, we examine their
model theory by first describing sufficient interpretations for definite theories, then
characterizing models as the fixpoints of a certain operator, and finally discussing
termination issues. Section 5.3.5 investigates the proof theory of this fragment of
first-order logic.

5.3.1 Non-recursive Definite Theories

We now summarize the syntax of definite theories and formalize the notion of call
graph, which tracks dependencies at the level of predicates. We also introduce
another useful type of graph which highlights dependencies at the predicate
argument level.
140 5 The Logical Spreadsheet

Syntax

The logical interpretation of a primitive relation identified a very simple fragment

of first-order logic whose theories are described by the following grammar:

Factual Theories … WWD j …; p.Ev/

We called a ground predicate p.Ev/ a fact, and therefore referred to the above sets as
factual theories. We wrote …p for the factual theory of a primitive predicate p.
A relational expression that did not make use of the difference operator was
instead mapped to a set of clauses whose body was a conjunction of predicates
and constraints. A clause whose body does not contain any negative literals is called
a definite clause, and a collection of only definite clauses is called a definite theory
or definite program. They are defined by the following grammar:

Definite Clausal Theories WWD j ; Cl

Definite Clauses Cl WWD H B
Heads H WWD p.X; : : : ; X /
Definite Bodies B WWD P j ææ j B ^B
Predicates P WWD p.X; : : : ; X /

A definite clause has therefore the following expanded form:

^ ^
P  Pi ^ æi i æ0i
i D1::m i D1::o

Recall that any subformula æ  æ0 is called a constraint. It is a head constraint if

it has the form X D æ0 where X occurs among the variables in the head P of this
clause.
We shall impose a number of safety restrictions on the variable occurrence in a
clause. Specifically:
1. A variable X occurring in the head of a clause also must occur:
(a) either in a predicate p.XE / in the body of the clause,
(b) or as the left-hand side of a head constraint X D æ.
2. Any variable X occurring within a constraint æ1  æ2 , or on the right-hand side
of a head constraint Y D æ, must also occur in a predicate p.XE / in the body of
the clause.
A definite clause satisfying these restrictions is called safe, and similarly for a
definite theory. We saw in Sect. 5.2.2 that the logical interpretation of a difference-
free relational expression naturally restricted the use of variables: it can easily be
ascertained that these restrictions correspond exactly to the above safety require-
ment. Without these constraints, we could consider clauses that do not correspond
5.3 Inference in a Definite Spreadsheet Theory 141

to any relational expression. More serious yet is the fact that the deduction methods
described below would fail if they are not enforced. For this reason, we shall
always restrict our attention to safe clauses and theories. Note that the second safety
restriction could be relaxed transitively so that the variables it constrains ultimately
get their values in a predicate in the body of the clause.
The clauses obtained by interpreting a relational spreadsheet in first-order logic
are never recursive: it is never the case that the definition of a predicate p refers
to p itself, either directly by appearing in the body of one of its clauses, or
indirectly by contributing to the definition of any of its body predicates. The main
deductive extension in Chap. 6 will stem from carefully relaxing this restriction. The
study of non-recursive theories in the present chapter will introduce us to many of
the technicalities needed for the general case, but without the additional layer of
complication brought in by recursion.
At this point, a definite theory is simply given by considering a factual theory …
together with a definite clausal theory . Formally,

Definite Theories … WWD …;

We will use the symbol … , variously decorated, to denote a definite theory. We will
often factorize … as the pair (…, ) with the same decorations to indicate its factual
and clausal components, respectively.5

Call Graph

The call graph, CG… , of a theory … captures how predicates participate in the
definition of other predicates. More precisely, CG… is formalized as follows:
• Nodes.CG… / is the set of all predicate symbols appearing in … .
• Edges.CG… / is the set of all edges q ! p such that q appears in a clause
defining p:

E /  q.YE / ^ B/ 2 … g
Edges.CG… / D fq ! p W .p.X

We observed that the call graph CG… s of a spreadsheet theory … s has much in
common with the relational substrate graph RSs of the spreadsheet s, which itself is
intimately related to the relational dependency graph DGrs . In particular, it is acyclic

5 E/X
It should be observed that every fact p.Ev/ is logically equivalent to the clause p.X E D Ev.
Furthermore, the transformation in Sect. 5.2.2 may itself produce clauses in this form. For these
reasons, many presentations in the literature, especially those concerned with general logic
programming [Llo87] more than Datalog, dispense with factual theories altogether by bundling
facts in the clausal component . In our context, keeping the distinction between facts derived
from primitive predicates and clauses obtained by translating complex relational expressions will
prove advantageous, both conceptually and in terms of presentation.
142 5 The Logical Spreadsheet

and it outlines the dependencies that a theory establishes among the predicates it
mentions.
It is easy to ascertain that, if … s is the theory obtained from the relational

!
substrate of a spreadsheet s that never uses relational difference, then RSs and CG… s
are the same graph.

Dataflow Graph

The dataflow graph DF… of a theory … tracks the dependencies within a definition
not at the level of predicates, but by chasing variables. Therefore, its nodes are the
individual arguments of each predicate in … and its edges relate an argument in the
head of a clause to all the variables that can influence it. It is defined as follows:
• Nodes.DF… / D fpji W p 2 Nodes.CG… / and p has arity n and 1  i  ng.
• Edges.DF … / contains two types of edges: qjj ! pji indicates that the i -th
head argument of a defining clause for p contains the same logical variable as
the j -th argument of a predicate q in its body, and qjj ) pji means that the i -th
argument Xi of p is the target of a head constraint Xi D æ.Z/ E in some clause
for p where the j -th argument of q occurs among the Z’s.E In symbols:

Edges.DF… / D E /  q.YE / ^ B/ 2 … and Xi D Yj g

fqjj ! pji W .p.X
E /  Xi D æ.Z/
[ fqjj ) pji W .p.X E ^ q.YE / ^ B/ 2 …
and Zk D Yj g

This graph is acyclic for any theory … .

The dataflow graph of a theory … is clearly much more precise than its call
graph. Whenever … is obtained from a relational spreadsheet s, the resulting
dataflow graph is related to the attribute dependency graph DGas in the same way
as its call graph is related to s’s relational dependency graph DGrs . In particular, it
tracks dependencies at the level of variables (i.e., attributes under the interpretation
as a relational spreadsheet) rather than the locations they occupy. As in the case of
!
CGs (or RSs ), DF s can be extended with functional dependencies and related to the
attribute dependency graph DGas through a mapping of nodes and edges.
We now turn to the problem of establishing logical inferences in the fragment of
first-order logic we just obtained. As we do so, we will track the development of
logic programming at large, although in a rather superficial manner. The interested
reader is referred to [CGT90, Llo87], for a general overview, and to some of the
papers cited below for details. It should be noted that the particular setting in which
we are operating, spreadsheets, led us to adapting many of these results to the
idiosyncrasies of this specific context.
We will first look at the model theory of this sublanguage, which is considerably
simpler than in the general case. It is so simple that we will be able to use it directly
to compute the set of all predicates p.Ev/ that are a logical consequence of a theory
… s , hence providing an alternative, logical, way to evaluate (the relational part of)
a spreadsheet s.
5.3 Inference in a Definite Spreadsheet Theory 143

5.3.2 Herbrand Models

The model theory of definite programs is remarkably simple. Indeed, specializing

a previous general result by Leopold Löwenheim and Thoralf Skolem, the logician
Jacques Herbrand showed that it is sufficient to consider a single domain in order to
characterize an important class of their logical consequences. This class consists of
E : B of any body formula. They are called goal formulas.
the existential closure 9X
With a slight abuse of notation, we will omit the existential quantifiers and therefore
identify them with clause bodies. We will be particularly interested in the logical
consequences that are facts, or atomic logical consequences. A fact p.Ev/ is indeed a
goal formula once we express it as 9X E : p.X
E/^X E D Ev.
The sufficient domain for any such investigation is the set of all ground terms
in the term language, which for us is the set V of all scalar values. It is known as
the Herbrand universe. Moreover, only a very restricted class of interpretations over
this domain needs to be examined. In our language, an interpretation I over V is a
Herbrand interpretation if
• I.v/ D v for each v 2 V.
• I.æ/ D æ O for every ground attribute expression æ. Recall that we disallowed
references in attribute expressions (for the time being); therefore any ground æ is
built out of just operators and values. Thus, æO is simply the value resulting from
calculating æ.
• I.v1  v2 / D v1  O v2 , where 
O is the comparison operation corresponding to
the operator , as per Sect. 4.3.2. It reduces v1  v2 to either true or false for any
pair of values v1 ; v2 2 V.
• I.p/  V n , for every n-ary predicate symbol p.
A Herbrand interpretation maps every ground term that can appear in a predicate
to itself. Ground attribute expressions occurring within conditions are reduced
to their value according to standard evaluation. Constraints are interpreted to
themselves. Therefore, two Herbrand interpretations I and I 0 can only differ
on the interpretation given to the predicate symbols. Consequently, a Herbrand
interpretation is completely characterized by a subset of the Herbrand base HBV D
fp.Ev/ W p is n-ary and Ev 2 V n g.
Given a definite theory … , a Herbrand model of … is any Herbrand interpreta-
tion I  HBV that is a model of each formula in … , which we denote by I ˆ … .
No variable assignment is needed since these formulas are closed by definition. Not
every Herbrand interpretation I is a model of … : for example, if … contains the
fact p.Ev/, but I does not, then I cannot model it. On the other hand, every definite
theory has at least one model: HBV itself. A model can never contain too many
facts.
One important property of Herbrand models is the model intersection property:
if I1 ; I2  HBV are Herbrand models of a definite theory … , then I1 \ I2 is a
Herbrand model of … . Its simple proof proceeds by induction on the definition
of validity given in Sect. 5.1, and can also be found in numerous references in
144 5 The Logical Spreadsheet

the literature, such as [CGT90, LNS82, JM94, Llo87]. One key consequence of
this property is that every definite theory … has a least Herbrand model, I… ,
obtained by taking the intersection of all the Herbrand models of … . Its existence
is guaranteed by the fact that HBV is always a model of … .
The least Herbrand model of a definite theory captures the intended use of clauses
to define a predicate. Indeed, there are Herbrand models of the registrar’s example
containing Students.4:00; “bn”; “S04”/ even if this tuple does not correspond to any
student. Only the least Herbrand model contains facts Students.v1 ; v2 ; v3 / for exactly
those values of v1 ; v2 ; v3 corresponding to students. Similar considerations hold for
predicate symbols that are not primitive.
The definition of logical consequence in Sect. 5.1 implied that, for a definite
theory … and a goal formula B, the judgment … ˆ B holds if every model of
… is also a model of B. For definite goals, this reduces to showing that B is valid
in the least Herbrand model of … , i.e., I… ˆ B. Thus, taking B to be a predicate,
it follows that I… is the set of all atomic logical consequences of … .
If & is a definite stratum in the theory … s of a relational spreadsheet s, this
last observation entails that one way to logically emulate the evaluation of all the
relational expressions corresponding to a predicate in & is to produce the least
&
Herbrand model of … s . This was elegantly achieved by Marteen van Emden and
Robert Kowalski, who characterized the least Herbrand model of a definite theory
as the fixpoint of a useful operator on interpretations [EK76].

5.3.3 Fixpoint Characterization

Knowing that I… ˆ … by construction, we can unfold this notation, obtaining

that I… is a model of every clause 8X E ; YE : p.X
E /  B.X E ; YE / in … . In symbols:
I… ˆ 8X E ; YE : p.X
E /  B.X E ; YE /. By the definition of validity, this entails that for
every variable assignment ˛, it holds that I… ; ˛ ˆ p.X E /  B.X E ; YE /. Since ˛
assigns a value to each variable, this is equivalent to saying that I… is a model of
every ground instance p.Ev/  B of p.X E /  B.X E ; YE /. By the definition of validity
again, this implies that I… ˆ p.Ev/ only if I… ˆ B. Therefore, p.Ev/ 2 I… requires
that I… be a model of the corresponding instance of the body of the original clause
8X E ; YE : p.XE /  B.X E ; YE /.
Therefore, I… contains the head of every ground instance of a clause whose body
is modeled by I… . Since the call graph CG… of a definite theory … is acyclic, a
head predicate cannot influence the atomic logical consequences of the predicates
appearing in its body. This suggests a staged approach to obtaining I… : determine
the atomic logical consequences of every primitive predicate p0 in … (their body
does not refer to any other predicate symbol, by definition). Then, use the above
approach to obtain the atomic logical consequences of every non-primitive predicate
p1 that mentions only primitive predicates in the body of its defining clauses. Repeat
this step for every predicate that uses only p0 and p1 predicates in its defining
5.3 Inference in a Definite Spreadsheet Theory 145

clauses, and so on until all the atomic logical consequences of every predicates in
… have been obtained. Notice that this corresponds to following the edges of CG…
upwards from its primitive nodes.
In order to formalize this intuition, we define InstV . / to be the set of all ground
instances of any clause in … , where … and are the factual and clausal parts
of … . We then define the following operator T…; over Herbrand interpretations
of … :
V
T…; .I / D I [ … [ fp.Ev/ W p.Ev/  i D1::m pi .Evi /
V
^ i D1::o vi i v0i 2 InstV . /
and pi .Evi / 2 I [ … for all i D 1::m
and vi O i v0i is true for all i D 1::o g

While T…; operates on generic Herbrand interpretations, it has a characteristic

behavior on any model I of … : indeed, it is easy to prove that I ˆ … iff T…; .I / 
I . (We have informally shown the forward direction in motivating this operator; the
reverse direction follows from the definition of T…; .) See [Llo87, EK76] for a pre-
packaged proof.
Since a Herbrand interpretation is any subset of the Herbrand base HBV , the
set of all Herbrand interpretations is just the powerset P HBV of HBV . Now, we
have seen in Chap. 3 that for any set X , the structure .P X ; / is a complete lattice
with ; and X as its bottom and top elements. Therefore, .P HBV ; / is a complete
lattice.
T…; is clearly monotonic over .P HBV ; /. It is a simple exercise to ascertain
that it is actually continuous [Llo87]. Therefore, by Kleene’s theorem, T…; has
a least fixpoint which is obtained by taking successive iterations starting from the
bottom element of this lattice:

lfp.T…; / D T…;
!
.;/

At this point, recall that I is a model of … if and only if T…; .I /  I . Since

this operator is monotonic, for every interpretation I , we have that I  T…; .I /.
Therefore, the set of all models of … coincides with the set of the fixpoints of T…; :
indeed I ˆ … iff I D T…; .I /. This implies that the least fixpoint lfp.T…; / of
this operator is also the least Herbrand model I… we were looking for.

5.3.4 Computing Fixpoints

We will now show that the above fixpoint characterization of the least Herbrand
model of a definite theory … can be used to effectively compute I… (after some
minor adjustments). We begin by observing that lfp.T…; / is obtained after finitely
many iterations of T…; , whose exact number is a simple function of the length of
longest path in the call graph CG… of … .
146 5 The Logical Spreadsheet

Termination

Let the level of a predicate p in CG… be the length of the longest path from a
primitive node to p. Then, define the set CG… Œi  to be the set of predicates in CG…
whose level is i . Then, we have that:
j
If p 2 CG… Œi  and … ˆ p.Ev/, then p.Ev/ 2 T…; .;/ iff j  i C 1.
The easy proof proceeds by induction on i .
Since every predicate belongs to a finite level, a consequence of this result is that
jCG… jC1
I… D T…; .;/, where jCG… j is the length of the longest path in CG… .
The required number of iterations is only one of the possible sources of infinity
in the above characterization of I… as lfp.T…; /. Another potential problem derives
from the fact that the set InstV . / of all the instances of any clause in … is itself
infinite. We can trace this issue back to the fact that the Herbrand universe V should
abstractly be considered infinite, even if unavoidable memory limitations will bound
it in every spreadsheet implementation.
We will now show that it is sufficient to restrict the Herbrand universe to a finite
subset V… of V which is derived from … D .…; /. The key observation is that
every ground term contributing to I… must necessarily be obtained either from a
fact in … or from head constraints of the form Y D æ occurring in the body of
clauses in . Therefore, starting from the primitive predicates of … , we can walk
up the dataflow graph DF … making a note of all the values we find. This idea is
formalized in the following definitions:
S
V… D pji 2 … Vpji
8
<S
p.:::;vi ;:::/ 2 …p fvi g if p is primitive
Vpji D S
: E VXi jB if p is not primitive
.p.X/B/ 2 p
8T
< E 62 B 0
Xi DYik Vq k jik if .Xi D æ.Z//
V V k Ek D k

„ ƒ‚ … :fæ.v
Xi j k q . Y / ^ B 0 E 2 B0
O 1 ; : : : ; vn / W vj 2 VZj jB g if .Xi D æ.Z//
B

The set V… collects the values that can appear in I… . It is assembled by taking
the union of the sets Vpji for all predicate-argument pairs pji appearing in DF… .
If p is primitive, we simply gather all the values occurring in its i -th argument.
Otherwise, we take the union of the values contributed by each of its defining
clauses p.X E /  B to its i -th argument Xi . This is represented by the set VXi jB . How
this set is calculated depends on whether Xi is directly contributed by one or more
predicates q k in B or if it is obtained by means of a head constraint Xi D æi .Z/ E
in B. In the first case, Xi can only assume values made available by all of these
q k in the appropriate position, say ik : hence we take the intersection of all the Vq k jik .
In the second case, we must calculate values for æi . Each variable Zji occurring
in it is instantiated with values in Vq k jjk made available by all body predicates q k
5.3 Inference in a Definite Spreadsheet Theory 147

where Zji occurs (at position jk ). Notice that VXi jB , and therefore Vpji and V… ,
are over-approximations in general since they do not take into consideration generic
constraints of the form æ  æ0 possibly occurring in B.
A simple induction on the level of a predicate p suffices to prove that Vpji is
finite. Therefore, since there are only finitely many predicates, V… is finite as well.
Consequently, the set InstV… . / of all instances of clauses of … relative to V… is
finite. We can then rewrite the definition of T…; as follows:
V
T…; .I / D I [ … [ fp.Ev/ W p.Ev/  i D1::m pi .Evi /
V
^ i D1::o vi i v0i 2 InstV… . /
and pi .Evi / 2 I [ … for all i D 1::m
and vi O i vi is true for all i D 1::o g

This updated definition and the above bound on the number of iterations make the
least fixpoint of T…; an effective characterization of the least Herbrand model I…
of … . Another important consequence of the finiteness of V… is that I… itself is
finite.

Practical Fixpoint Computation

The set V… may be a wild over-approximation of the set of values that can appear in
I… . This can make our latest characterization of T…; unusable in practice, although
it is theoretically effective.
A more efficient method stems from the observation that the sole reason for
considering ground instances of the clauses in … in the definition of T…; is
that body predicates can be in the right form for testing their membership in the
input Herbrand interpretation. An immediate improvement suggests itself: use this
interpretation to construct the instances of the body of the clause that it satisfies and
propagate the instantiation to the head. This is implemented in the following variant
of T…; [EK76, CGT90], which resurrects variable assignments from the definition
of validity rather than rely on ground instances of clauses:
V V
T…; .I / D I [ … [ f˛.P / W P  i D1::m Pi ^ i D1::o æi i æ0i 2
and ˛.Pi / 2 I [ … for all i D 1::m
and ˛.æi / 
O i ˛.æi / is true for all i D 1::o g

Observe that the first line picks clauses from rather than from the set of its ground
instances. The variable assignment ˛ is computed from the conjuncts in this clause
and the input interpretation I as well as the factual theory …. We emphasize this
computational interpretation by calling it a substitution. Its computation proceeds
as follows:
• The literals Pi in the selected clause are matched in parallel against I [ …, hence
producing a set f˛j g of candidate substitutions for the clause.
• Each of these substitution ˛ 2 f˛j g is tested against all the generic constraints
E  i æ 0 .X
æi .X/ E /. Only the substitutions that validate all constraints are retained.
i
148 5 The Logical Spreadsheet

• Each remaining substitution ˛ is extended to every variable Y occurring in the

head of the clause but not in any of its body predicates by evaluating the attribute
expression æ.X E / of the corresponding head constraint Y D æ.X E / against ˛.

Note that there may be a multitude of substitutions ˛ validating a clause for a given
input interpretation.6

5.3.5 Proof Theory

As mentioned in Sect. 5.1, proof theory offers an alternative stance on defining what
it means for a formula ' to follow from a theory . It differs from model-theoretic
approaches by relying only on the symbols mentioned in and '. We will now
develop the proof theory of definite programs, which is as simple as its model theory.
It is adapted from classical logic programming approaches such as [Llo87] to the
context of the deductive spreadsheet, and also follows a more modern understanding
of logic programming [MNPS91, Pfe].
Let … D .…; / a definite theory and p.Ev/ a ground fact. The derivability of
p.Ev/ from … , denoted by … ` p.Ev/, is defined by means of the following three
rules:
V
.P  niD1 Bi / 2 p.Ev/ D ˛.P / …; ` ˛.B1 / …; ` ˛.Bn /
np
…; ` p.Ev/

p.Ev/ 2 … O æO 0 is true
æO 
pr gc
…; ` p.Ev/ … ` æ  æ0

Rule pr simply asserts that p.Ev/ is derivable from .…; / whenever it occurs in
…. This takes care of the simple case where p is a primitive predicate. Rule np
handles the case where p is not primitive: p.Ev/ is derivable from .…; / if there is
a substitution ˛ and a clause P  B in such that p.Ev/ is precisely ˛.P / and the
(ground) instance ˛.Bi / of each conjunct Bi in B is itself derivable from .…; /.

6
This expression will be our final definition for T…; for definite programs. As mentioned in
footnote 5, it is uncommon to draw a separation between facts and clauses in a definite theory since
a fact can be seen as a particular case of a clause. The above operator assumes a slightly different
form in that case, which is commonly seen in the logic programming literature [Llo87, EK76]:
V V
T… .I / D I [ f˛.P / W P  iD1::m Pi ^ iD1::o æi i æ0i 2 …
and ˛.Pi / 2 I for all i D 1::m
and ˛.æi / O i ˛.æi / is true for all i D 1::o g

The two formulations can easily be shown to be equivalent.

5.3 Inference in a Definite Spreadsheet Theory 149

If Bi is a predicate, then its derivability is simply assessed by using rules pr or np

again. If instead it is a (ground) constraint æ  æ0 , then its derivability corresponds
to its validity with respect to the true=false-valued relation .O This is captured by
rule gc.
Because only Herbrand interpretations need to be considered when studying the
model theory of definite programs, it can be immediately proved [Llo87,JM94] that

… ˆ p.Ev/ iff … ` p.Ev/

There is a clear correspondence between those three rules and our last expression
for the T…; operator. Indeed, it is easy to show that … ` p.Ev/ iff p.Ev/ 2 I… .
A derivation of … ` p.Ev/ is a tree with this judgment as its root, inner
nodes given by appropriate instances of the above rules, and leaves in the form
of the premises of rules pr or gc. Such a tree constitutes a proof that p.Ev/ is
derivable from … . How this tree, and in general the proof theory of definite
programs, is used depends on the modality by which it is constructed. The
computational interpretations of the two main modalities are radically different:
starting from the leaves and using (variants of) the above rules to produce derivable
facts yields an alternative way of constructing the least Herbrand model I… ;
instead, attempting to build a derivation given only the root judgment corresponds
to answering a query. Proof-theoretically, their difference is accounted for by
the concept of polarization [NM10, LM09] within the context of focused proof
search [And92, How98, Cha06, Sim12]. Both techniques have an application in a
spreadsheet environment. We will now examine them in closer detail.

Bottom-Up Construction

Let us start by presenting an alternative formulation of the above inference rules.

Rule pr remains the same (yet we rename it pr# for uniformity), but we merge np
and gc into a new rule that we call np#. This rule also explicitly makes a distinction
between the way head and generic constraints are processed:

… ` ˛.P1 / b
˛.Y1 / D ˛.æ/1 1
˛ .æ0 /1  1
O 1 ˛.æ00 /1
.P  B/ 2 :: :: ::
b 1 1
: : :
… ` ˛.Pm / ˛.Yo / D ˛.æ/o ˛ .æ0 /o0 
O o0 ˛.æ00 /o0
np#
…; ` ˛.P /
V Vo Vo0
where B D m i D1 Pi ^ i D1 .Yi D æi / ^ i D1 .æi i æi /. For readability and
0 00

typographic reasons, we have omitted the “is true” part of the constraint validation
premises.
150 5 The Logical Spreadsheet

The bottom-up construction starts with an empty set I of facts and repeatedly
uses rules pr# and np# to add new facts to it until I coincides with the least
Herbrand model I… of a given definite theory … D .…; /:
1. Because of the form of rule pr#, … can be loaded into I right away.
2. Rule np# is then repeatedly used as follows:
(a) Pick a clause P  B in ;
(b) Select facts q1 .Ev1 /; : : : qm .Evm / in I that match its body predicates
Q1 ; : : : ; Qm ;
(c) Apply the resulting substitution ˛ to its generic constraints and verify their
validity;
(d) Extend ˛ to all head variables by means of the available head constraints;
(e) Add ˛.P / to I unless it was already there.
This procedure stops when no choice of P  B or q1 .Ev1 /; : : : qm .Evm / adds any
new fact into I . It is clear that I is then equal to I… : indeed rules pr# and
np# are nothing but a proof-theoretic presentation of the last T…; operator of
Sect. 5.3.4. The main difference is that, while that operator worked in a set-at-a-
time fashion, the procedure we just outlined operates in a fact-at-a-time manner. A
number of (deductive) database query optimizations are based on variants of this
method [CGT90]. In the context of the relational and soon deductive spreadsheet, it
is expected to efficiently handle updates (see Sect. 5.6 for details).

Top-Down Construction

A natural question to ask is whether a fact p.Ev/, or in general a goal formula B, is

derivable from a definite theory … . Theorem proving is all about answering such
questions, as has been most research in logic programming. They also correspond to
queries in a database. The literature on the subject is indeed more than abundant. The
importance of this question is noticeably redimensioned in the context of the deduc-
tive spreadsheet. Indeed, normal operation is concerned with the set of all atomic
logical consequences of a theory, which the fixpoint approach can compute very
effectively as we will see in Sect. 5.5. Traditional methods for proving specific facts
or formulas are not as useful for this purpose. We will however see in Sect. 5.7 that
they play a key role as part of the explanation facilities of the deductive spreadsheet.
The notion of derivation tree helped define derivability, but does not inherently
suggest any procedure for building a tree that supports … ` B. Bottom-up
construction does provide an answer, but it does so in the most unfocused manner:
by enumerating every fact that is derivable from … . We will now present a
second variant of the rules at the beginning of this section that allows efficiently
constructing a derivation tree for any given goal B and definite theory … .
We begin by modifying the structure of our main judgment: instead of … ` B
for a closed B, it will be convenient to rely on the form … ` B 7! ˛ where B
may contain variables and ˛ is a substitution for them. This judgment will have a
derivation precisely when … ` ˛.B/ has one. Separating ˛ out will allow us to
5.3 Inference in a Definite Spreadsheet Theory 151

interpret the substitution as the output of the construction while … and B will be
its inputs. Indeed, properly managing substitutions is a critical aspect of the tree
construction. The rules defining this judgment are given next:

˛.P / 2 … .P 0  B/ 2 ˛.P / D ˛.P 0 / …; ` ˛.B/ 7! ˛B

pr" np"
…; ` P 7! ˛ …; ` P 7! ˛; ˛B

… ` B1 7! ˛1 … ` ˛1 .B2 / 7! ˛2 … ` ˛1 ; : : : ; ˛n1 .Bn / 7! ˛n

V bd"
… ` niD1 Bi 7! ˛1 ; : : : ; ˛n

… ` B 7! ˛; ˛XE
æ O æO 0 is true
O  gl"
gc" hc"
… ` æ  æ0 7! ¿ … ` X D æ 7! fX 7! æg
O E : B 7! ˛
… ` 9X

Rule pr" is obtained from pr: a (primitive) predicate P that has an instance ˛.P /
in the factual theory … is derivable, and ˛ is returned to indicate which instance of
P this tree is a derivation of. Note that P may be partially instantiated. Rule np",
a clear offspring of np, handles non-primitive predicates: in order to verify whether
(some instance of) P is derivable from … , this rule looks for a clause P 0  B
in whose head P 0 is unifiable with P , i.e., such that there is a substitution ˛
with ˛.P / D ˛.P 0 /. If so, we attempt to construct a derivation for ˛.B/. Note
indeed that B may contain variables that do not occur in P 0 , and that ˛ may not
resolve all head variables if P itself mentions variables. Then, ˛ is extended with
the substitution ˛B returned by this subgoal, and reported on the right-hand side
of 7!. Rule bd" reduces the derivability of a body to the derivability of each
conjunct B1 ; : : : ; Bn appearing in it. Notice again that B2 may contain variables
that do not occur in B1 . Therefore, we shall apply the substitution ˛1 reported by
B1 to B2 before attempting this goal. Solving ˛1 .B2 / will produce a substitution ˛2
that will be applied to B3 , and so on. All these partial substitutions contribute to the
substitution returned by the rule. Rule gc" resolves generic constraints similarly
to rule gc above: it is applicable only if its arguments are ground (by the
safety requirements, all variables occurring in a generic constraint also appear in
a body predicate—it can therefore be delayed); therefore it always produces the
empty substitution. A head constraint X D æ may be handled differently: if the
variable X has been instantiated, then rule gc" applies. Otherwise, X is still a
variable and the returned substitution is fX 7! æg. O Note that, again by safety, æ
shall be ground. Rule hc" takes care of this second possibility; it is a special case of
our earlier rule gc. Finally, rule gl" extends the range of possibilities envisioned so
far by describing how to prove a goal 9X E : B. It simply tries to prove B but throws
away the part ˛XE of the returned substitution concerned with any variables among
XE . In practice, ˛ E is often a value of interest. Indeed, the main reason for asking
X
whether … ` 9X E : B has a derivation is often just to obtain ˛ E . This is achieved by
X
simply asking for a derivation of … ` B 7! ˛. Therefore, rule gl" can be ignored
for all practical purposes.
152 5 The Logical Spreadsheet

While the above rules describe very precisely how to manage substitutions, a
number of questions must be resolved before obtaining an effective procedure.
Note that the field of logic programming has been grappling with these issues for
years [Llo87, CGT90, MNPS91, Pfe]. We will now discuss the best practices in the
context of the deductive spreadsheet:
• How to manage choice points? With the exception of hc", all rules above can
fail to be applicable, hence halting the construction of the proof tree. In general,
this does not mean that the goal at hand is not provable, but simply that we tried
a possibility that did not live up to its expectations, and we shall try another
one. Indeed, rules pr" and np" are often choice points: multiple facts or clauses
may be applicable, each with a different substitution ˛. This issue is traditionally
approached by choosing one of the alternatives, according to some strategy, but
maintaining sufficient state to remember which other possibilities remain to be
considered. The order in which to consider alternatives has a non-insignificant
impact on performance.7
• How to solve subgoals? Traditional logic programming [Llo87] solves subgoals
in a depth-first manner. Rule bd" supports this strategy: B1 is completely solved,
hence reporting the substitution ˛1 ; only then is ˛1 .B2 / attempted, and so on—we
will see shortly that the order in which body conjuncts are attempted is a factor
as well (in particular rule gc" requires constraints to be ground). Depth-first
resolution is extremely efficient in most cases and does not have any drawbacks
for non-recursive theories. When recursion is thrown into the mix, as it will be in
Chap. 6, then a simple-minded application of this technique can needlessly cause
infinite loops.
The alternative breadth-first strategy pursues the solution of all subgoals at
the same time: once one step has been made toward solving subgoal Bi , it will
not be reconsidered before all other subgoals have had a chance to make a step
forward. This strategy always terminates for the type of definite programs we are
considering in this book. However, it can require maintaining exponentially more
state than depth-first resolution. This can have a negative impact on perceived
performance.
Breadth-first resolution is supported by the following variant of rule bd":

… ` P1 7! ˛1 1
˛ .æ0 /1  1
O 1 ˛.æ00 /1
:: Lm ::
˛D
1 1
: i D1 ˛i :
… ` Pm 7! ˛m ˛ .æ0 /o0 
O o0 ˛.æ00 /o0

… ` B 7! ˛; fY1 b ;:::;Y
7! ˛.æ/ 1 o
bg
7! ˛.æ/ o
bd0 "

7
Another approach, embraced by a certain flavor of parallel logic programming, considers all the
choices at once. We will not investigate it further because commercial implementations still use
the more traditional sequential model and there is a substantial overhead in managing parallelism.
5.3 Inference in a Definite Spreadsheet Theory 153

V Vo Vo0
where B D m i D1 Pi ^ i D1 .Yi D æi / ^ i D1 .æi i æi / and ˛ ˚ ˛ fuses
0 00 0

substitutions with overlapping variables, requiring the overlap to coincide and

accumulating the other portions.
In an application such as the deductive spreadsheet, it is essential that users
do not experience infinite loops. It is also important that evaluation be efficient.
The negative aspect of depth-first can be nearly eliminated by automatically
transforming the theory … prior to attempting the goal B [CGT90]. In general,
this involves rewriting the clauses defining a predicate as to ensure termination.
However, it is often sufficient to reorder subgoals so as to force key substitutions
to be produced early.
• In what order to solve subgoals? Because rule bd" has multiple premises, more
than one goal will generally be pending evaluation at any instant. How to choose
the next subgoal to solve has an impact on overall performance. The following
high-level strategy has been shown to produce good results:
1. Early failure detection clearly leads to fewer failed proof-tree construc-
tions, and therefore better overall performance. Therefore, ground constraints
should always be validated as early as possible by invoking rule gc".
2. Early binding of variables will generally lead to smaller failed proof-tree
constructions. Therefore, subgoals for primitive predicates should be given
priority. If rule gc" is not applicable, then goals that make use of rule pr"
should be attempted.
3. Subgoals for non-primitive predicates are attempted next, hence triggering
rule np".
4. Since head constraints only calculate values for head variables, there is no
point in processing them until a substitution has been produced for the rest of
the body. Therefore, rule hc" should be used last.
Steps (2) and (3) can further be refined if we maintain statistics about each
predicate, especially the number of facts that satisfy it, and possibly an indexing
of the values taken by some or all of its arguments. Note that these statistics are
easy to gather in a relational spreadsheet because the extension of a predicate is
the value of the corresponding relational expression.
• How to recover from failure? The traditional way to recover from a failed subgoal
is to backtrack to the last choice point and attempt one of the yet uncharted
possibilities. If there are none, then the previous choice point shall be examined,
and so on. If all options have been exhausted, a failure is reported for the top-
level goal. Backtracking generally involves reversing the effect of intervening
substitutions. This requires again maintaining state, although it can be done
relatively efficiently.
Prolog [Llo87, O’K90, SS94], the most prominent logic programming language,
relies on a fixed depth-first evaluation strategy which processes subgoals left-to-
right and tries alternative clauses in the textual order in which they are specified in
a program (first-to-last). This is known as SLD-resolution. While applicable in our
currently non-recursive language, the possibility of non-termination will force us
away from it in Chap. 6.
154 5 The Logical Spreadsheet

5.4 Inference in a Normal Spreadsheet Theory

In this section, we examine the full fragment of first-order logic that emerges
from simplifying the logical interpretation of a relational expression as described
in Sect. 5.2.2. The resulting theories, which are called normal, differ from definite
programs by the presence of negated predicates in the body of clauses. This
alters significantly the model and proof theory developed in Sect. 5.3. However,
stratification and the restriction to a natural interpretation of negation allows
building on most of that work.
This closely tracks major developments in logic programming, where substantial
amounts of research have been invested toward the goal of extending the elegant
solutions devised for definite theories to tame negation and the substantial increase
in expressive power that comes with it. This long-standing challenge is still open in
some respects. In this section, we specialize to the spreadsheet context some of the
solutions that proved most successful for deductive databases. The reader is invited
to consult [CGT90] for an overview of alternative approaches to importing negation
into logic programming.
We formalize (non-recursive) normal theories and related concepts in Sect. 5.4.1.
We examine their model theory in Sect. 5.4.2 and their proof theory in Sect. 5.4.3.

5.4.1 Non-recursive Normal Theories

We now define the syntax of normal theories, extend the definitions of call and
dataflow graphs to this richer language, and adapt the notion of stratification within
a normal theory.

Syntax

In Sect. 5.2.2, set difference in the relational spreadsheet was interpreted into a for-
mula containing negation. We also showed that the product of this translation could
be simplified so that this operator would prefix at most predicates. The resulting
language defines formulas that differ from definite clauses by the possibility of
negated predicates appearing in their body. They are called normal clauses. Theories
consisting of just normal clauses are consequently normal theories. These notions
are formalized as follows:

Normal Clausal Theories WWD j ; Cl

Normal Clauses Cl WWD H B
Heads H WWD p.X; : : : ; X /
Normal Bodies B WWD P j :P j æ  æ j B ^ B
Predicates P WWD p.X; : : : ; X /
5.4 Inference in a Normal Spreadsheet Theory 155

Expanding from this grammar, normal clauses have the following form:
^ ^ ^
P  Pi ^ :Qi ^ æi i æ0i
i D1::m i D1::n i D1::o

where some of the constraints on the right could be head constraints Xi D æ0i . A
body component p.X E / or :p.X
E / is called a literal: the first form is a positive literal,
the second a negative literal. For the time being, we will continue to require that a
theory be non-recursive.
Negative literals are similar to generic constraints in that they limit the values
that can be propagated to the head of a clause. Therefore, we shall extend the safety
restrictions on definite programs with the stipulation that variables in a negative
literal appear in some positive body predicate. The safety requirement for normal
clauses is therefore defined as follows:
1. A variable X occurring in the head of a clause also must occur:
E / in the body of the clause,
(a) Either in a positive literal p.X
(b) Or as the left-hand side of a head constraint X D æ.
2. Any variable X occurring within a generic constraint æ1  æ2 , or on the right-
E/
hand side of a head constraint Y D æ, must also occur in a positive literal p.X
in the body of the clause.
3. Any variable X occurring in a negative literal :Q must also occur in a positive
E / in the body of the clause.
literal p.X
The last entry is new. We will restrict our investigation to safe clauses and
theories satisfying these restrictions: as in the definite case, the methods we will
be discussing do not work for unsafe theories. Moreover, unsafe theories do not
correspond to any relational expression. As with definite clauses, condition (2) can
be relaxed transitively.
A normal theory … is a pair .…; / consisting of a factual theory … and a normal
clausal theory . We will omit the qualifiers “normal” and “definite” when the type
of theory under discussion is clear from the context.

Call Graph

The call graph, CG… , of a normal theory … D .…; / is again intended to describe
how predicates participate in each other’s definitions. Because negation will require
a special treatment, it is convenient to define its nodes not just as the predicates
occurring in … , but also to include the negated predicates found in its clauses. As
already with the relational substrate graph of Sect. 5.2.1, we relate predicate symbols
and their negations by a special type of edge. CG… is then formalized as follows:
• Nodes.CG… / is the set of all predicate symbols and negated predicate symbols
appearing in … .
156 5 The Logical Spreadsheet

• Edges.CG… / is the set of all edges .:/q ! p such that .:/q appears in a clause
defining p, and all edges p Ü :p for p; :p in Nodes.CG… /:

Edges.CG… / D E /  q.YE / ^ B 2 g
fq ! p W p.X
[ f:q ! p W p.XE /  :q.YE / ^ B 2 g

[ fp Ü :p W p; :p 2 Nodes.CG… /g

As for definite theories, this definition parallels the notion of relational substrate
graph RSs of the spreadsheet s. The call graph CG… s of the theory … s obtained
from s differs from RSs at most by the absence of functional dependencies.
As in Sect. 5.2.1, the Ü edges provide a mechanism to incorporate extra-logical
dependencies in a normal theory. The functional dependencies of a spreadsheet are
a prime example. Therefore, we will silently allow call graphs to contain extra Ü
edges, except where explicitly noted. The graph CG… in the above definition will
then be called the strict call graph of … . We will occasionally use the phrase
enriched call graph for its extended form.

Stratification

The notions of stratification and related concepts are readily adapted to generic
normal theories from the definitions in Sect. 5.2.3. A stratum is a set & of predicates
in … such that no path in CG… between two member predicates p and q contains
a Ü edge. This defines the following subtheory … & D .…& ; & / of … :
[
& D p
p2&
[ [ [
…& D …p [ …q [ …q
p2& q 62 & q 62 &
P B^q.XE / 2 & P B^:q.X E / 2 &

Its clausal part collects the definition of each non-primitive predicate in &. Its factual
part includes the factual theory of every primitive predicate in &, but also the factual
theories of every predicate q occurring in the body of a clause in & . Note that …q
is empty for any q that is not primitive in … . We call & a definite stratum if … & is
a definite theory, and a normal stratum if it is a normal theory.
A stratification is a partition &E D &0 ; : : : ; &n of the set of all predicates occurring
in … such that &i is a stratum for i D 0::n and if pi Ü pj in CG… for pi 2 &i and
pj 2 &j , then i < j . The Ü edges of the call graph CG… induce what we called
the functional dependency graph FD… .E& / among strata: its nodes are the strata of
the stratification, and it contains an edge & Ü & 0 if there are p 2 & and p 0 2 & 0
such that p Ü p 0 in CG… . Because we have disallowed recursive clauses, this
graph is necessarily acyclic and therefore every (non-recursive) normal theory is
5.4 Inference in a Normal Spreadsheet Theory 157

stratifiable. A stratification &0 ; : : : ; &n is linear if &i Ü &i C1 for i D 0::n  1. A

generic stratification can always be linearized by merging strata that are not on the
same Ü path. Because we did not admit recursion, the partition of the predicates
of … into singletons is a stratification and therefore every (non-recursive) normal
theory has at least one linear stratification. The least stratification of … is the linear
stratification where each predicate occurs in the stratum with the lowest possible
index, short of violating the constraint on Ü edges.
These definitions naturally extend to any enriched call graph CGs based on
CG… . Note however that in this case, stratifiability is not guaranteed since the
corresponding functional dependency graph may contain cycles. We will only be
interested in enriched call graphs that are stratifiable.
Given a linear stratification &E D &0 ; : : : ; &n , notice that … &0 ; : : : ; … &n defines a
partition &0 ; : : : ; &n of the clausal theory of … , but not of its factual theories, in
general. Observe also that, in the absence of cycles through Ü edges, &0 is always
a definite stratum.
As observed in Sect. 5.2.3, stratification and related concepts have a long history
within logic programming. Originally proposed in [CH85] and thoroughly analyzed
in [ABW87], they are used mostly for the purpose of taming negation [CGT90,
Llo87]. We have adapted them to also capture the functional dependencies present
in the spreadsheet setting.

Dataflow Graph

The definition of the dataflow graph DF… of a normal theory … does not change
from the definite case. As observed earlier, negative literals have a filtering effect on
the data in a clause, in the same way generic constraints impose limitations on the
values that are propagated. Generic constraints were not included in the definition
of dataflow graph in Sect. 5.3.1 precisely because they never inject values. For the
same reason, negative literals shall be excluded.

5.4.2 Model Theory

The model theory of normal clauses is not as clean as for definite clauses. Thanks
to our safety requirements, validity can still be established by solely considering
Herbrand interpretations. However, most of the other pleasant properties of definite
programs do not hold for normal theories. In particular, a normal theory may not
have a least Herbrand model, but could have several distinct Herbrand models
each of which is minimal. This dramatically complicates the practical determination
of atomic logical consequences. A particularly natural attitude toward negation in
stratified theories allows however electing one such model as the perfect model, and
use it exactly like the least Herbrand model in Sect. 5.3.
158 5 The Logical Spreadsheet

Perfect Model

The notion of least Herbrand model is ill defined in the case of normal theories.
Consider for example a theory consisting of the normal clause

p.X /  :q.X / ^ r.X /

and the single fact r.a/. Then, by unfolding the definition of validity, it can be
shown that this theory has two Herbrand models that are both minimal in the sense
that removing any object from either of them jeopardizes validity: fp.a/; r.a/g and
fq.a/; r.a/g.8 Since there is more than one such model, neither is the “least” one.
Note that the model intersection property does not hold for normal programs: fr.a/g
is not a model in this example.
Of these two models, fp.a/; r.a/g is particularly natural under the interpretation
of the above clause as defining the predicate p on the basis of q and r: because
r.a/ is a true fact and there is no fact q.a/ to veto the propagation of a to the head
variable X , the fact p.a/ ought to be true. The other model, fq.a/; r.a/g, although
logically correct, is at odds with intuition: r.a/ is clearly there but there is no trace
of any q.a/. We call fp.a/; r.a/g the perfect model of the above theory, and treat
it as if it were its single least model, therefore ignoring fq.a/; r.a/g. This practice
is logically unsound, but it is effective, predictable, and corresponds to an intuitive
understanding of negation. However, it works only for stratified theories [CGT90,
Llo87].9
The general principle supporting this interpretation is known as the Closed World
Assumption (CWA) [Rei78]. It infers :p.Ev/ whenever the ground predicate p.Ev/ is
not a logical consequence of a definite theory … . When applied to normal theories,
it immediately leads to inconsistency: in the above example, it would infer both
:q.a/ and :p.a/, which is clearly inconsistent with the above clause. Stratification
allows applying the CWA in stages: the bottom stratum &0 of this example, which
defines the predicates q and r, has fr.a/g as its least Herbrand model; when
evaluating the next stratum &1 , which defines p by means of the above clause, the
CWA is applied solely with respect to the predicates in &0 , hence producing p.a/.
Stratification circumvents inconsistency. In this context, the model it produces is the
perfect model.
This immediately suggests an adaptation of the fixpoint computation of the least
Herbrand model of a definite theory to calculate the perfect model of a stratifiable
normal theory: extend the T…; operator to an individual stratum using the CWA,

8
In classical logic, this clause is logically equivalent to q.X/  :p.X/ ^ r.X/, which underlies
a further justification for the second model, and explains the symmetry between them.
9
A more general notion, local stratification [Prz87, CGT90], admits a larger class of theories by
taking into consideration the ground predicates of a theory rather than just the predicate symbols
that appear in a clause. The added class of theories is however likely to be inconsequential in the
spreadsheet context. Moreover, local stratifiability is undecidable.
5.4 Inference in a Normal Spreadsheet Theory 159

and then apply this operator one stratum at a time, using the models obtained for
lower strata as if they were factual theories of the current one.

Fixpoint Characterization of a Normal Stratum

Let & be a stratum in a (linear) stratification of a normal theory … and let … & D
.…& ; & / be the corresponding subtheory. Recall that & may mention negative
literals, but only if the corresponding predicates are primitive in …& . Then, our
last expression for the T…; operator can naturally be adapted to handle this new
situation:
V V
T…& ; & .I / D I [ …& [ f˛.P / W P  i D1::m Pi ^ i D1::n :Qi
V
^ i D1::o æi i æ0i 2 &
and ˛.Pi / 2 I [ …& for all i D 1::m
and ˛.Qi / 62 …& for all i D 1::n
and ˛.æi / O i ˛.æi / is true for all i D 1::o g

By the safety constraints on variable occurrences, the negative literals :Qi do not
contribute to the calculation of potential substitutions ˛, but, similarly to generic
constraints æi i æ0i , they are used to eliminate candidates. Note that this expression
is equivalent to our last version in the case of definite theories.
Since all predicate symbols used negatively are primitive by assumption, T…& ; &
is monotonic and continuous relative to the usual lattice of Herbrand interpreta-
tions [Llo87]. Therefore, it has a least fixpoint that is computed via successive
iterations from the empty set: lfp.T…& ; & / D T…!& ; & .;/. By the same argument as
for definite theories, an interpretation I is a Herbrand model of … & D .…& ; & / if
and only if T…& ; & .I /  I . Therefore, this fixpoint is a minimal model. We call it
the perfect model of … & and denote it by I… & .
Observe that these conclusions would be incorrect in general if any predicate
q occurring negatively in & were defined clausally, i.e., if q were not primitive.
Indeed, the condition ˛.Qi / 62 … would have the effect of ignoring any calculated
instance of q, which would prevent the negative literal from acting as a filter. If we
were to relax this condition so that it takes the input interpretation into account,
hence writing it ˛.Qi / 62 I [ …, monotonicity would clearly be lost.
The effectiveness of the fixpoint characterization of the perfect model I… & as
!
T…& ; & .;/ is proved exactly in the same way as in Sect. 5.3.4. In particular, the
number of needed iterations is equal to jCG… & j C 1, where jCG… & j is the length of
the longest ! path in the call graph CG… & of … & . Moreover, since we are relying
on substitutions, we do not need to worry about issues such as the finiteness of
the Herbrand universe. However, that argument can be used to prove that I… & is
finite.
160 5 The Logical Spreadsheet

Fixpoint Characterization of a Normal Theory

We can use this extended fixpoint characterization to obtain the perfect model of a
stratifiable normal theory … as a whole. Given a linear stratification &E D &0 ; : : : ; &n
of its predicates, we use the above operator to calculate the perfect model of each
successive subtheory … &i , for i D 1::n. The set of clauses & fed to the T…& ; &
operator is simply the clausal part &i of … &i . The set of facts will be …&i extended
with the perfect model obtained for stratum i  1. This is formalized as follows:
8
< I… &0 D lfp.T…&0 ; &0 /
:I D lfp.T.…&i C1 [ I… &i /; &i C1 /
… &i C1

Then, I… D I… &n is the perfect model of … .

Since a stratification contains finitely many strata and the fixpoint computation
terminates within each strata, this defines an effective procedure for computing the
perfect model of … . Note however that the overall number of iterations will in
general be larger than jCG… j C 1. It can be shown that the perfect model of a
normal theory can be computed in this way starting from any stratification, linear or
not.

5.4.3 Proof Theory

The basic notion of derivability in a normal theory can be defined by adding a single
rule to the corresponding system for definite programs:
Vn
.P  i D1 Bi / 2 p.Ev/ D ˛.P / …; ` ˛.B1 / …; ` ˛.Bn /
np
…; ` p.Ev/

p.Ev/ 2 … æO  O 0 is true
O æ … 6` q.Ev/
pr gc nl
…; ` p.Ev/ … ` æ  æ 0
… ` :q.Ev/

Rules np, pr and gc do not change from the definite case. Instead, the highlighted
rule nl defines the provability of a negative literal by reducing the derivability of
… ` :q.Ev/ to the absence of a derivation for the judgment … ` q.Ev/, a state
of affairs which we indicate with … 6` q.Ev/. Operationally, this means that every
attempt at constructing a derivation for … ` q.Ev/ is unsuccessful. This approach,
known as negation-as-failure [Cla78,Llo87], is clearly based on the CWA and is the
standard treatment of negation in logic programming languages, especially Prolog.
5.4 Inference in a Normal Spreadsheet Theory 161

It does not soundly capture the logical semantics of negation, but is a sufficient
approximation for most applications.
As in the definite case, we will now specialize this system for the bottom-up
and the top-down construction of a derivation. The former can be used to logically
handle updates, the latter as an explanation facility. Note that … 6` q.Ev/ is not a
judgment in rule nl, but a meta-theoretic requirement. As we specialize the above
system, we will bring this notion to the front stage by describing effective means to
verify it.

Bottom-Up Construction

As for definite theories, the bottom-up construction of the derivation judgment … `

P mimics the work of the fixpoint operator at the level of individual facts. In a
stratifiable normal theory, we will apply this technique one stratum at a time, and
therefore the judgment it defines has the form … & ` P . It is described by the
following two rules, in which the differences with respect to the definite case (other
than stratum annotations) have been highlighted:

p.Ev/ 2 …&
pr#
…& ; & ` p.Ev/

… & ` ˛.P1 / ˛.Q1 / 62 …& b/ O ˛1

b ˛.æ
˛.Y1 / D ˛ .æ/1 0
.æ / 1 1
00
1
:: :: :: ::
.P  B/ 2 &

b b 1
: : : :
… & ` ˛.Pm / ˛.Qn / 62 …& ˛.Yo / D ˛ .æ/o ˛.æ0 /o0 
O o0 ˛ .æ00 /o0
np#
…& ; & ` ˛.P /

V Vn Vo Vo0
where B D m i D1 Pi ^ i D1 :Qi ^ i D1 .Yi D æi / ^ i D1 .æi i æi / in the
0 00

second rule.
Given a normal theory … D .…; / and a linear stratification &E D &0 ; : : : ; &n
for it, facts p.Ev/ derivable from … are computed one stratum at a time starting at
0
&0 . Let, …&i the set of facts derivable from … &0 ; : : : ; … &i . Formally,
0
…&0 D fp.Ev/ W …&0 ; &0 ` p.Ev/g
0 0
…&i C1 D fp.Ev/ W .…&i [ …&i C1 /; &i C1 ` p.Ev/g
0
It is then clear that each set …&i is the perfect model I… &i of … &i which we obtained
0
earlier. Consequently, I… D …&n . The derivability … ` p.Ev/ of a fact p.Ev/ from a
normal theory … with respect to a stratification &1 ; : : : ; &n then simply reduces to
0
p.Ev/ 2 …&n . Note that there is always a smallest index i such that:
0
… ` p.Ev/ iff p.Ev/ 2 …&j for all i  j  n
162 5 The Logical Spreadsheet

Top-Down Construction

In principle, extending the top-down derivation of a definite judgment … ` B 7! ˛

to a normal theory … can be reduced to simply adding a single negation-as-failure
rule to appropriately handle negative body literals. This is the highlighted rule nl"
below. Note that by the safety requirements, we can delay the resolution of this
goal until it is ground, which implies that the returned substitution is vacuous.
The premise stipulates that the corresponding positive literal is not derivable. In
the context of a top-down construction, a meta-theoretic interpretation of non-
derivability is not fully satisfactory. For this reason, we shall treat … 6` Q as
a first-class judgment in the style pioneered by James Harland [Har93, Har94],
by providing inference rules that define it. They are highlighted in the second box
below. In the judgment … 6` Q, the predicate Q will always be ground.
˛.P / 2 … .P 0  B/ 2 ˛.P / D ˛.P 0 / …; ` ˛.B/ 7! ˛B
pr" np"
…; ` P 7! ˛ …; ` P 7! ˛; ˛B

… ` B1 7! ˛1 … ` ˛1 .B2 / 7! ˛2 … ` ˛1 ; : : : ; ˛n1 .Bn / 7! ˛n

V bd"
… ` niD1 Bi 7! ˛1 ; : : : ; ˛n

0
O 
æ O æ
O is true
gc" hc"
… ` æ  æ0 7! ¿ … ` X D æ 7! fX 7! æg
O

… ` B 7! ˛; ˛XE … 6` Q
gl" nl"
E B!
… ` 9X: 7 ˛ … ` :Q 7! ¿

P 62 … For all .P 0  B/ 2 if P D ˛.P 0 / then …; 6` ˛.B/

np"
…; 6` P

… ` B1 7! ˛1 … ` ˛1 .B2 / 7! ˛2 … 6` ˛1 ; : : : ; ˛i 1 .Bi / for 0  i  n

Vn bd"
… 6` i D1 Bi

O æO 0 is false
æO  … ` q.Ev/ 7! ¿
gc" ng"
… 6` æ  æ0 … 6` :q.Ev/

Rule np" defines the non-provability of a ground positive literal P . First of all, P
shall not be a fact in … (otherwise, P would be immediately provable). Second, if
the head P 0 of any clause P 0  B in is unifiable with P , then the corresponding
instance of its body shall not be provable. Rule bd" defines what it means for a body
5.5 Spreadsheet Evaluation Revisited 163

B not to be provable relative to a depth-first strategy (other strategies are handled

similarly): for all orderings B1 ; : : : ; Bn of the elementary formulas of B, some initial
segment B1 ; : : : ; Bi 1 may have a derivation, but the associated instance of Bi does
not. A ground constraint is unprovable if it evaluates to false, as formalized in rule
gc". A ground negative literal :q.Ev/ is unprovable if there is a derivation of its
positive counterpart q.Ev/, which will clearly return the empty substitution. This is
formalized in rule ng".
Observe that the top-down construction just presented does not make use of
stratification. Indeed, … in the left-hand side of its judgments is an arbitrary normal
theory, not necessarily an individual stratum.

5.5 Spreadsheet Evaluation Revisited

We will now start applying the techniques devised in the last two sections as logical
alternatives to implementing the various functionalities of a relational spreadsheet.
We begin with evaluation, which was analyzed at length in Sect. 4.4 of the previous
chapter.
Whenever a relational expression eN appeared in a spreadsheet s, we simply relied
on the semantics of relational algebra to evaluate it, as long as it did not refer to
any still uncalculated location. We will now parallel that development, but rely
exclusively on the fixpoint semantics of the normal theory associated with s. We will
initially do so in a very coarse way, but later refine this approach so that it better suits
semi-naı̈ve evaluation. Logic-based calculation will reveal its advantages in Chap. 6
where we will allow recursive predicates, hence making our spreadsheet model truly
deductive.

5.5.1 Naı̈ve Evaluation

Let s be a relational spreadsheet and … s D .…s ; s / the normal theory that

interprets its relational expressions. In Sects. 5.3 and 5.4, we conveniently assumed
that … s did not mention any location. While this dramatically simplified the
development of its meta-theory, the result of our translation from Sect. 5.2.2 will
generally embed cell references within constraints (see for example the last row
in Fig. 5.2). Moreover, since primitive predicates stem from base relations, their
concrete definitions will generally mention locations, as in Fig. 4.3 in Chap. 4.
We shall address this issue in a way similar to the relational case: we rely
on an environment  to assign a value to these references before performing any
evaluation. Indeed, we will exercise the techniques developed in Sects. 5.3 and 5.4
on the location-free theory … s @ , and not on … s . Since some locations in 
may carry the uncalculated value ?, we must be prepared to deal with clauses
and predicates that embed it. To this end, whenever ? appears in any clause for
a predicate p, we replace the entire definition of p with the uncalculated fact
164 5 The Logical Spreadsheet

E If p is primitive, for example of the form hEei, and it depends on a locations

p.?/.
E With this understanding, … s @ 
containing ?, we similarly downgrade it to p.?/.
and the notions it depends on are defined as follows:

[ [
.…s ; s / @  D .…p @ / [ . p @ /
…p 2 …s p 2 s
(
E
fp.?/g if p D phEei and ? 2 Ee @ 
…p @  D
hEe @ i otherwise
(
E
fp.?/g if Cl @  D ? for any Cl 2 p
p @  D
fCl @  W Cl 2 p g otherwise
(
Vn ? if Bi @  D ? for some i D 1::n
.P  i D1 Bi / @  D Vn
P  i D1 .Bi @ / otherwise

P @ D P
.:Q/ @  D :Q
(
? if æ @  D ? or æ0 @  D ?
.æ  æ0 / @  D
.æ @ /  .æ0 @ / otherwise

Note that the literals in a clause body are never replaced with ? since they do not
embed cell references. With these definitions in place, we need to alter the form
of our fixpoint operator so that ? does not accidentally participate in a logical
E may lead to the partial instantiation of a
inference. Otherwise, a positive literal p.?/
head predicate with ?, and a negative literal :q.?/E may fail to veto the propagation
of some values. Therefore, we define the following ?-conscious variant T…?& ; & of
the stratum-level operator T…& ; & introduced in Sect. 5.4.3:

T…?& ; & .I / D I [ …&

8
ˆ
ˆ E
fp.?/g E /  q.YE / ^ B 2 p&
if there is p.X
ˆ
ˆ
ˆ
ˆ E 2 I [ …&
such that q.?/
[ ˆ
<
[ E
fp.?/g E /  :q.Z/
if there is p.X E ^ B 2 p &
ˆ
p non-primitive ˆ
ˆ
ˆ E 2…
ˆ
ˆ such that q.?/ &
:̂
T…& ; p& .I / otherwise

It behaves exactly like T…& ; & for those predicates that do not depend on a pseudo-
E Indeed, it directly calls this operator in that case. It only
fact of the form p.?/.
5.5 Spreadsheet Evaluation Revisited 165

propagates predicates in uncalculated form otherwise. The operator T…?& ; & shares
the properties of T…& ; & . In particular, it has a least fixpoint lfp.T…?& ; & / obtained
by finite iteration from the empty Herbrand interpretation. It is a partial version
of the perfect model and therefore we denote it by I…? & . We can then iterate this
construction over a stratified normal theory … , obtaining an overall partial perfect
model that we denote by I…? .
If … above is the situated theory … s @  for a relational spreadsheet s and
environment , this last expression is the perfect model of … s with respect to the
environment . If peN is the predicate corresponding to relational expression eN in
s, the value appearing in I…? s @  for peN is the same as the value eN @  obtained
relationally in Chap. 4. Therefore, we can make use of it in every circumstance
where eN @  appeared.
We begin with updating the definition of evaluation of an array formula Ee with
respect to an environment (see Sect. 4.4):

Ee @  Ee @  D f 7! Ee Π@  W  2 IEe g
N @  D ŒfEv W pŒNe .Ev/ 2 I.…
Œe ?
s @ / g

The first line remains unchanged since it does not refer to relational expressions.
Instead, we replace the second (originally Œe N @  D ŒeN @ ) with the set of tuples
reported by I…? s @  for peN .
We can then apply the same technique to update the definition of the naı̈ve
evaluation operator ER s of a relational spreadsheet s:
[
ER s ./ D fElΠ7! .s.El/ @ /ΠW  2 IEl g
El 2 dom s
s.El/ 2 E I
[
[ fEl 7! fEv W phEli .Ev/ 2 I.…
?
s @ / gg
El 2 dom s
s.El/ 2 EN

The first part of this expression, which deals with array formulas, is defined exactly
as in Chap. 4. The second portion relies instead on the perfect model of s at  to
populate the domain of a relational expression. It is easy to show that it computes
the same environment as our original definition for ER s . Therefore, we can use it for
the fixpoint evaluation of a relational spreadsheet s.
This approach to evaluating a relational spreadsheet is as inefficient as the
original naı̈ve procedure described in Chap. 4. Indeed, each iteration of ER s embeds
?
the fixpoint computation I.… s @ / which has the same cost as the relational
evaluation of all the relational expressions in s. A simple-minded adaptation of the
techniques in Chap. 4 to distilling a semi-naı̈ve refinement of ER s does not yield the
?
expected efficiency boost because it would still imply the computation of I.… s @ / at
166 5 The Logical Spreadsheet

each iteration, each time with a slightly more instantiate environment  and therefore
theory … s @ .
?
Clearly, once a predicate p has been calculated in I.… s @ / , it will remain
? 0
calculated in I.… s @ 0 / for every refinement  of , and hold the same value.
The above procedure does not take this property into consideration, and therefore
recalculates p at each iteration of ER s or of the corresponding semi-naı̈ve refinement.

5.5.2 Localized Semi-naı̈ve Evaluation

In order to address this issue, we will now partition … s into a number of func-
tionally dependent theories … si so that at any iteration of the outer operator, which
we will call ER s0 , either all or none of its clausally defined predicates will be fully
si @ / will either contain only ? as a value, or it will not
?
calculated. Therefore, I.…
mention ? at all. Partitioning … s according to a functional dependency conscious
stratification of its predicates, for example its least stratification &E obtained from
the stratified call graph R fSs , is a step in the right direction, but it does not take into
account the fact that some functional dependencies have been artificially inserted to
handle negation. We will then construct … si by collecting the definition of all the
original predicates present in the i -th stratum of &E and all the auxiliary predicates
they have engendered. We now formalize this intuition.
Let s be a relational spreadsheet and R fSs its stratified call graph, defined in
Sect. 5.2.3. Let &E D &1 ; : : : ; &n be the least stratification of R fSs . For i D 1::n,
s
let &i be the subset of &i containing solely predicates corresponding to relational
expressions hosted in cell ranges in s:

&is D fphEli W phEli 2 &i and s.El/ 2 Eg

N

This excludes predicates obtained from embedded relational formulas Œe N and
from base relations hEe i such that Ee is not a cell range. It also excludes all
the auxiliary predicates introduced to massage the raw logical interpretation of
relational expressions into a normal theory. Indeed, if s.El / D e, N recall that we
initially interpreted eN as the formula phEli .XE /  peq.
N X;E YE /. Recall also that the
factorization of complex negative formulas out of peq. N X; E YE / may have produced
nhEli
1
a number of auxiliary predicates p E ; : : : ; p E . The same may happen because of
hli hli
functional dependency inside e. N
We will ignore the first two classes of excluded predicates for the moment, but
s
we shall account for the latter. Therefore, let the stratification &E i be the restriction
of &E to the set of predicates
j
&is [ fp E W phEli 2 &is g
hli
5.5 Spreadsheet Evaluation Revisited 167

s
The first stratum of &E i is &is and every successive stratum contains auxiliary
predicates as they appear in &E .
s
Finally, let … si be the stratified normal theory corresponding to &E i . Notice that
this theory may now mention some predicates of the form phEei where hEe i is not a
cell range. We can subject … si to the same manipulations already applied to … s .
In particular, we can situate it at any given environment  as … si @ , and we can
?
compute its partial perfect model I.… si @ / .
The construction just described can be repeated (generally on a smaller scale)
for every predicate of the form pŒNe in … s , hence obtaining a theory … sŒNe that will
be used to redefine the evaluation of the array formula Œe N @  as ŒfEv W pŒNe .Ev/ 2
?
I.… s
@ / g.
ŒNe

At this point, we can present an alternative version ER s0 of the evaluation operator

ER s , with a decentralized attitude toward relational expressions which relies on the
localized theories … si (and indirectly … sŒNe ) rather than the wholesale … s . It is
defined as follows:
[
ER s0 ./ D fElΠ7! .s.El/ @ /ΠW  2 IEl g
El 2 dom s
s.El/ 2 E I
[
[ fEl 7! fEv W phEli .Ev/ 2 I.…
?
s @ / gg
i
El 2 dom s
phEli 2 &is

The operators ER s0 and ER s are equivalent. In particular, they have the same least
fixpoint: the evaluation of s. They also have the exact same computational cost:
indeed, the various … si ’s are just a partition of … s , and therefore the cost of
computing the collection of the perfect models of the former and the one perfect
model of the latter is the same.
The difference emerges when refining ER s and ER s0 into semi-naı̈ve procedures, EP s
and EP s0 respectively, following our steps from Chap. 4. Indeed, the perfect model
of each … si can be computed independently of the others. Moreover, an attempt
at producing it before all the objects it functionally depends on (either embedded
cells or functionally dependent predicates) have a non-? value will result in each
of predicate p in it being evaluated to p.?/. E Therefore, the semi-naı̈ve version EP s0
R
of Es can be defined so that it calculates a perfect model for … si only once: the
0
?
first time it is applied to an environment  such that the perfect model I.… si @ /
E
contains non-? predicates. The usual analysis of dependencies in Chap. 4 will point
out when this is the case. Note again that this fine-grained approach is not an option
when starting from ER s because the whole … s must be processed at each iteration.
The semi-naı̈ve operator EP s0 is as efficient as the relational semi-naı̈ve procedure
outlined in Chap. 4. It has the added advantage that it can natively handle recursive
predicates, as we will see in Chap. 6.
168 5 The Logical Spreadsheet

5.6 Updates Revisited

At the cost of repeating ourselves, we shall stress again that updates are much more
frequent than full evaluation in a spreadsheet. Therefore, although it is useful that
the initial calculation be fast, it is much more important that the myriad of small
updates resulting from normal use be propagated quickly. In Chap. 4, we have laid
out a general update infrastructure for the relational spreadsheet and observed that
most relational operations supported incremental updates. However, we remained
vague on how operation-level updates propagated to entire expressions: relational
algebra operators could be nested arbitrarily, which made it difficult to describe
a fully general approach. The logical interpretation of relational expressions have
instead a very predictable form: they are clauses whose body consists of literals,
possibly with constraints intermixed. This regularity allows us to discuss updates
and their propagation at a much finer granularity and in greater detail in this section.
It will be useful to present our approach in small incremental steps. We will
initially limit our investigation to the effect that updating a primitive predicate
q has on any clause mentioning q in its body. We start in Sect. 5.6.1 with the
simplest case, in which q has been extended with new facts, and in which it is
used only as a positive literal. In Sect. 5.6.2, we see what are the consequences of
the removal of facts from q. We consider negative and mixed occurrences of q in
a clause in Sect. 5.6.3. Finally, Sect. 5.6.4 brings these various techniques together
and addresses the general problem of propagating updates within a normal theory.

5.6.1 Positive Update of Positive Literals

Let s be a fully calculated relational spreadsheet, and q a predicate in it. Because s is

fully calculated, we can treat q as if it were primitive and collect the calculated pred-
icates of the form q.Ev/ in a set that we will call the extension of q and denote by …q ,
possibly with some abuse of notation if q is clausally defined in s. Assume now that
an update to s results in the addition of a new set …q of facts of the above form.
Therefore, …q is extended to …q [ …q . We can capture this update logically by
introducing predicates q that describe …q and q 0 corresponding to the extended
set of facts …q [ …q . In symbols, we want to define a theory … C q such that:

q ` q.E
… C v/ iff q.Ev/ 2 …q
q ` q .E
… C v/ iff q.Ev/ 2 …q [ …q
0

The first line is achieved by defining the set …q D fq.Ev/ W q.Ev/ 2 …q g. The
second line is then clausally defined by the theory qC given by the following two
clauses:
5.6 Updates Revisited 169

E /  q.X/
q 0 .X E
0 E E
q .X /  q.X/

Then, … C q D …q [ …q [ q . For the sake of conciseness, we will collapse

C

these two clauses in the formula q 0 .XE /  q.X E / _ q.X E /, which is not a clause,
but is logically equivalent to them. Note that since no other clause defines q 0 ,
we can understand this formula as a bi-implication: it then assumes the form
E / $ q.X/
q 0 .X E _ q.X/. E While not logically sound, this transformation, known
as Clark completion [Cla78, Llo87], enforces the interpretation of the set of clauses
with head p as the definition of predicate p, in the sense that these clauses should
be the only possibilities for deriving p. It is the logical justification of the notion of
negation-as-failure, which we relied upon when defining the proof theory of normal
theories in Sect. 5.4.3.
Let now p be a clausally defined predicate that depends positively on q. Then,
p contains a clause Cl of the form p.X E /  q.XE ; YE / ^ B. We will initially assume
that q does not occur in B. Any addition to …q will lead to an extension of the
logical consequences of p. Following the above model, if we can express the change
to the set of logical consequence of p as a predicate p, then the clause p 0 .X E/ 
p.X E / _ p.XE / describes the logical consequences of p after the update. We can
also define p 0 directly by replacing q with q 0 in Cl. By expanding q 0 as q _ q, we
obtain:
 
E /  q.X
p 0 .X E ; YE / _ q.X;E YE / ^ B

By distributing _ over ^, this is equivalent to the two clauses

E /  q.X
p 0 .X E ; YE / ^ B
0 E E YE / ^ B
p .X /  q.X;

Since the body of the first is precisely the original body of Cl, we can take the
second to be the definition of p. It will be convenient to relativize this predicate to
the specific clause Cl, hence writing it as pCl . Therefore,

E /  q.X;
pCl .X E YE / ^ B

This rationale is schematically captured in the following diagram, where the thick-
lined box represents the result of the update to p, the thin line delimits its original
values, and the dotted/dashed lines enclose portions of q and q that are not
validated by B.
170 5 The Logical Spreadsheet

B
⎧ ⎫ ⎫
⎪
⎪ ⎪
⎪ ⎪
⎪
⎪
⎪ ⎬ ⎪
⎪
⎪
⎪ ⎪
⎪
⎪
⎪ p(X) q(X) ⎪
⎪
⎪
⎪ ⎪
⎪ ⎪
⎪
⎪
⎨ ⎭ ⎪
⎬
p (X) .⎫ q (X)
⎪ .⎪ ⎪
⎪
⎪ .. ⎪ ⎪
⎪
⎪
⎪
⎪ .⎬ ⎪
⎪
⎪
⎪
⎪ Δp(X) .. Δq(X) ⎪
⎪
⎪
⎪ .⎪⎪ ⎪
⎪
⎪
⎩ .⎭ ⎪
⎭
.
...........

Before extending this technique any further, two observations are in order:
• First, the set of logical consequences of pCl may not be disjoint from that of
p. Consider for example the clause p.X /  q.Y / ^ r.X /. Note that p does
not depend on q since there is no flow of information from the variable Y to the
variable X . The clause for pCl would have the form pCl .X /  q.Y /^r.X /
in which X and Y are still unrelated so that the set of logical consequences of
pCl is identical to that of p. While updates to q cannot have any effect on p
(as long as q.Ev/ 2 …q for some Ev), the above technique recomputes p in full
with every change to q. Here, q.Y / could easily be optimized away. This is not
always possible, however, as in the following variant of this clause: p.X / 
q.Y / ^ r.X / ^ X > Y . Here, updates to q with small values for Y may still lead
to a full recalculation of p.X / as pCl .X /.
The possibly large overlap between p and pCl is certainly unwelcome news
since it implies that the calculation of pCl may perform more work than strictly
necessary for the purpose of upgrading p to p 0 . While redundant, this work is not
useless: if p.Ev/ was derivable and pCl .Ev/ is produced, this means that updating
q yielded a new way to prove p.Ev/. This enables a very precise accounting of the
derivations of each fact p.Ev/, which turns out to be very useful when considering
negative updates. Indeed, we can associate each fact with a counter that tracks
its number of derivations. A logical way to achieve the same effect would be
to switch to a multiplicity-conscious refinement of the formalism we have used
so far, as done in [LC12]. Linear logic [Gir87], which is amenable to a logic
programming interpretation [HM94, LPPW05] compatible with Datalog, is the
most promising candidate. We will not explore this possibility in the present
book.
• Second, the fact that B appears in the definition of pCl may cause a significant
overhead even for small changes to q. In extreme cases, it may be as efficient to
recompute p from the updated q as to determine pCl . However, it is expected
to be advantageous in the common situation where q and B share a majority of
variables and the size of B is relatively small. Adaptive approaches that decide
5.6 Updates Revisited 171

at run-time whether it is more convenient to recompute p from the updated q or

incrementally as pCl will need to be investigated.
Generalizing from the above scenario, consider now a clause Cl0 for p containing
two occurrences of q: p.X E /  q.X E 2 / ^ B, where X
E 1 / ^ q.X E, XE 1 and X
E 2 may
overlap. Then, by an argument similar to the above, it can be easily seen that pCl0
has the following defining clauses:

E /  q.X
pCl0 .X E 1 / ^ q.XE 2/ ^ B
E /  q.X
pCl0 .X E 1 / ^ q.X
E 2/ ^ B
E /  q.X
pCl0 .X E 1 / ^ q.X
E 2/ ^ B

where the last clause accounts for interactions between the two occurrences of q in
Cl0 over the update. The same idea can be iterated if q has more than two occurrences
in a clause. It also applies to situations where different positive literals have been
updated. This is schematically rendered in the following diagram, where we have
refrained from representing B for convenience:
q (X 2 )

q(X 2 ) Δq(X 2 )
⎧ ⎫ ⎫
⎪
⎪ ⎪ ⎪
⎪
⎪
⎪ ⎪
⎬ ⎪
⎪
⎪
⎪ ⎪
⎪
⎪
⎪ p(X) q(X 1 )⎪
⎪
⎪
⎪ ⎪ ⎪
⎪
⎪
⎨ ⎪
⎭ ⎪
⎬
p (X) . . . . . . . . . . .⎫ q (X 1 )
⎪ .. ⎪ ⎪
⎪
⎪ .. ⎪
⎬ ⎪
⎪
⎪
⎪ ⎪
⎪
⎪
⎪ .. Δp(X) Δq(X 1 ) ⎪
⎪
⎪
⎪ ⎪ ⎪
⎪
⎪
⎪ .. ⎪
⎭ ⎪
⎪
⎩ .. ⎭

Generalizing further, assume that p contains several clauses Cl1 ; : : : ; Cln con-
taining an updated predicate q. Then, it is clear that the overall update p to p is
given by the collection of the updates produced for each individual clause Cli for
i D 1::n:

E /  pCl1 .X/
p.X E

E E/
p.X /  pCln .X
172 5 The Logical Spreadsheet

5.6.2 Negative Update of Positive Literals

We will now reiterate the above scenarios, this time assuming that the update to …q
has had the effect of removing the set of facts ı…q . Let then ıq be the predicate
characterizing ı…q .
Consider a clausally-defined predicate p which depends upon q. We can then
abstractly describe the changes that a negative update to q induces onto p by means
of the clause p 0 .X E /  p.X E / ^ :ıp.X E /, where p 0 collects the values of p after
the update and ıp describes the change. As we search for a clausal definition of ıp,
we must be particularly attentive to the fact that this formula does not allow any
slack: if a fact ıp.Ev/ is derivable and p.Ev/ also has a derivation that does not rely
on q, it will be wrongly excluded from p 0 . Giving a strict definition for ıp so that
this situation does not arise is extremely complex, possibly computationally more
demanding than just recomputing p on the basis of the updated q. We will therefore
take a different approach. We will assume, as discussed in Sect. 5.6.1, that every
derivable p.Ev/ is annotated with the number of derivations it has. We will define ıp
so that it also records the number of ways to prove ıp.Ev/ on the basis of ıq, and we
will tag p 0 .Ev/ with the difference of these two values: a result of zero means that
there is no derivation of p 0 .Ev/.
Let Cl be a clause p.X E /  q.XE ; YE /^B depending on q, and initially assume that
q does not occur in B. The values of p contributed by ı…q within this clause can be
characterized very precisely by using ıq to filter the values of the variables XE and YE
that are processed by B and possibly propagated to the head. These tuples are indeed
characterized by the predicate ıpCl defined by the following clause:

E /  q.X
ıpCl .X E ; YE / ^ B ^ ıq.X
E ; YE /

Since ı…q  …q by definition, we have that q.X E ; YE / ^ ıq.X

E ; YE / is logically equiv-
E ; YE /. Therefore we can simplify this clause into:
alent to ıq.X

E /  ıq.X
ıpCl .X E ; YE / ^ B

As observed in Sect. 5.6.1, the computation of B may limit the efficiency of this
incremental update. However, if Cl is the only clause for p, this drawback can
be completely eliminated by observing that the first expression for ıpCl can be
rewritten as:

E /  p.X
ıpCl .X E / ^ ıq.X
E ; YE /

The computed values of p are directly filtered through ıq. Moreover, it is clear that
ıp is ıpCl . This simplification is not sound if there are other defining clauses for
p since it may incorrectly mark for deletion values not contributed by ıq. These
arguments are captured in the following sketch:
5.6 Updates Revisited 173

⎧ ⎫ ⎫
⎪
⎪ ⎪ ⎪
⎪
⎪
⎪ ⎪
⎬ ⎪
⎪
⎪
⎪ ⎪
⎪
⎪
⎪ p (X) q (X) ⎪
⎪
⎪
⎪ ⎪ ⎪
⎪
⎪
⎨ ⎪
⎭ ⎪
⎬
p(X) ⎫ q(X)
⎪
⎪ ⎪
⎪ ⎪
⎪
⎪
⎪ ⎬ ⎪
⎪
⎪
⎪ ⎪
⎪
⎪
⎪ ±p(X) ±q(X) ⎪
⎪
⎪
⎪ ⎪
⎪ ⎪
⎪
⎪
⎩ ⎭ ⎪
⎭

Following the tracks we laid for positive updates, consider next a clause Cl0 for
p containing two occurrences of q: p.XE /  q.XE 1 / ^ q.X
E 2 / ^ B, where X,
E XE 1 and
E may overlap. By the same rationale, we can characterize ıpCl0 by inserting either
X 2

ıq.XE 1 / or ıq.X
E 2 / as a filter:

E /  q.X
ıpCl0 .X E 1 / ^ q.X
E 2 / ^ B ^ .ıq.X
E 1 / _ ıq.X
E 2 //

which we can streamlined into the two clauses:

E /  q.X
ıpCl0 .X E 1 / ^ q.X
E 2 / ^ ıq.X
E 1/ ^ B
E /  q.X
ıpCl0 .X E / ^ q.X
1 E / ^ ıq.X
2 E 2/ ^ B

As before, if Cl0 is the only clause for p, this can be further simplified, with
substantial efficiency gains, into:

E /  p.X
ıp.X E / ^ ıq.X
E 1/
E /  p.X
ıp.X E / ^ ıq.X
E 2/

These results extend to clauses containing more than two occurrences of q, and to
clauses where different predicates have been updated. This is summarized in the
following diagram:
q(X 2 )

q (X 2 ) ±q(X 2 )
⎧ ⎫ ⎫
⎪
⎪ ⎪ ⎪
⎪
⎪
⎪ ⎪
⎬ ⎪
⎪
⎪
⎪ ⎪
⎪
⎪
⎪ p (X) (X ) ⎪
q1 ⎪
⎪
⎪
⎪
⎪ ⎪
⎪ ⎪
⎪
⎨ ⎭ ⎬
p(X) .. . . . . . . . . . . ⎫ q(X 1 )
⎪
⎪ . ⎪
⎪ ⎪
⎪
⎪
⎪ .. ⎬ ⎪
⎪
⎪
⎪ ⎪
⎪
⎪
⎪ . ±p(X)
. ±q(X 1 ) ⎪
⎪
⎪
⎪ . ⎪
⎪ ⎪
⎪
⎪
⎩ . ⎭ ⎪
⎭
..
174 5 The Logical Spreadsheet

Finally, consider a situation where two clauses Cl1 and Cl2 for p mention q or
some other updated predicate. Then, the overall update to p is given by the updates
contributed by both:

E /  ıpCl1 .X/
ıp.X E
E E
ıp.X /  ıpCl2 .X/

and similarly if p has more than two clauses containing updated predicates.
At this point, if all clauses for p are subject to updates, then we can safely
E / as p.X
compute p 0 .X E / ^ :ıp.X E / in a purely logical fashion. If, however, p
has other clauses, they may independently calculate values p.Ev/ accounted for by
ıp. Then, we must use the counter approach to derive p 0 .X E /, or the result may be
incorrect.

5.6.3 Updates to Negative Literals

We now examine the impact of an update of some predicate q to clauses that rely
negatively on q.
Consider first the situation where …q has been extended to …q [ …q , and let
q be the characteristic predicate of …q . Initially assume that there is a clause Cl
for some other predicate p that uses q negatively: p.X E /  :q.X E ; YE / ^ B (recall
that every variable to :q must also occur in a positive literal in B). Observe that
enlarging …q means that more values are filtered out, which will in general lead to
fewer derivable facts of the form p.Ev/. Therefore, the effect of the update can be
described by a clause p 0 .XE /  p.XE / ^ :ıp.X/,
E as in Sect. 5.6.2. Now, we can
E / directly as
define p 0 .X

E /  :q.X
p 0 .X E ; YE / ^ B ^ :q.X;
E YE /

which suggests defining the contribution ıpCl to ıp as

E /  q.X;
ıpCl .X E YE / ^ B

which is in line with what we obtained in Sect. 5.6.2 for negative updates to positive
literals. This same model applies to rules containing two or more occurrences of
:q, as well as the other situations we examined.
Consider next the situation where …q was shrunk by ı…q , so that the clause Cl
above will not restrict as much the set of values it contributes to p. By an argument
pattern similar to what we used already, it is easy to see that the corresponding
extension pCl to p has the form

E /  ıq.X;
pCl .X E YE / ^ B
5.6 Updates Revisited 175

The development then proceeds as in the case for positive updates to positive literals
in Sect. 5.6.1.
The final situation we need to examine involves mixed clauses Cl of the form
p.XE /  q.X E 1 / ^ :q.X
E 2 / ^ B, where p depends on the updated predicate q both
positively and negatively. Consider the situation where …q has been extended by
…q , which we assume is characterized by a predicate q. The net effect on p will
E 1 / will open
be both positive and negative, in general. Indeed, the positive literal q.X
the gates to new consequences of the form p.Ev/, but the negative literal :q.X E 2 / may
bar some of them from materializing. It may also remove some previously derivable
facts of this form. Pictorially, the situation is rendered as follows:
q (X 2 )

p (X) Δq(X 2 ) q(X 2 )

⎧ ⎫ ⎫
⎪
⎪ ⎪
⎪ ⎪
⎪
⎨ ⎬ ⎪
⎪
⎪
⎪
p(X) ±p(X) q(X 1 ) ⎪
⎪
⎪
⎪ ⎪
⎪ ⎪
⎪
⎩ ⎭ ⎪
⎬
.. .. ⎫ q (X 1 )
.. .. ⎪ ⎪ ⎪
⎪
⎪
.. .. ⎬ 1
⎪
⎪
⎪
Δp(X) .. .. ⎪ Δq(X ) ⎪
⎪
⎪ ⎪
⎪
.. .. ⎭ ⎪
⎭
.....................

The thin solid box represents the logical consequences of Cl before the update, while
the thick box describes the values of p it derives after the update. The intuition
behind the other areas is given by their labels. By using this picture as inspiration,
E  .p.X
it is clear that p 0 can be expressed by the formula p 0 .X/ E / _ pCl .X//
E ^
:ıpCl .X E / for appropriate predicates pCl and ıpCl that we will now define. We can
expand this formula into the following two clauses:

E /  p.X
p 0 .X E / ^ :ıpCl .X
E/
0 E E E/
p .X /  pCl .X / ^ :ıpCl .X

E  q 0 .X
If p 0 .X/ E 1 / ^ :q 0 .X
E 2 / ^ B is to describe clause Cl after the update, we
can expand the occurrences of q 0 in terms of q and q, hence obtaining

E /  .q.X
p 0 .X E 1 / _ q.X
E 1 // ^ :q.X
E 2 / ^ :q.X
E 2/ ^ B

which by distributivity rewrites to

E /  q.X
p 0 .X E 1 / ^ :q.X
E 2 / ^ :q.X
E 2/ ^ B
E /  q.X
p 0 .X E 1 / ^ :q.X
E 2 / ^ :q.X
E 2/ ^ B
176 5 The Logical Spreadsheet

which allows us to define pCl and ıpCl as follows:

E /  q.X
pCl .X E 1 / ^ :q.X
E 2 / ^ :q.X
E 2/ ^ B
E /  q.X
ıpCl .X E 2 / ^ q.X
E 1/ ^ B
E /  q.X
ıpCl .X E / ^ q.X
2 E 1/ ^ B

The effect on p of a negative update …q n ı…q to q via Cl has the following

pictorial representation:
q(X 2 )

p(X) ±q(X 2 ) q (X 2 )
⎧ ⎫ ⎫
⎪ ⎪ ⎪
⎪
⎪
⎨ ⎪
⎬ ⎪
⎪
⎪
⎪
p (X) Δp(X) q (X 1 ) ⎪
⎪
⎪ ⎪ ⎪
⎪
⎪
⎩ ⎪
⎭ ⎪
⎬
.. .. ⎫ q(X 1 )
. .⎪ ⎪
⎪
.. .. ⎪ ⎬ ⎪
⎪
.. ±q(X 1 ) ⎪
⎪
±p(X) .. ⎪
⎪
.. .. ⎪
⎪ ⎪
⎪
⎭ ⎪
⎭
.
. . . . . . . . . . . . . . . . . . . . ..

Following the now familiar development, the result of the update to q given by the
E /  q.X
clause q 0 .X E /^:ıq.X
E /. By expanding its body for q in Cl and distributivity,
we obtain the following two clauses for the representation p 0 of p after the
update:

E /  q.X
p 0 .X E 1 / ^ :ıq.X
E 1 / ^ :q.XE 2/ ^ B
0 E E E E
p .X /  q.X / ^ :ıq.X / ^ ıq.X 2 / ^ B
1 1

from which we can derive the following clauses for pCl and ıpCl :

E  q.X
pCl .X/ E 1 / ^ :ıq.X
E 1 / ^ ıq.X
E 2/ ^ B
E /  ıq.X
ıpCl .X E 1 / ^ :q.X
E 2/ ^ B

5.6.4 Update Propagation in a Normal Theory

In the three previous sections, we have performed a local analysis of updates,

studying how changes in the extension of a body literal propagate to the predicate
defined by its clause. While this is undoubtedly progress with respect to the
5.7 Explanation Revisited 177

operation-level investigation of Chap. 4, it does not yet provide us with a concrete

approach to propagating updates throughout a spreadsheet. This section addresses
this issue. We will initially consider the more modest goal of propagating updates
just in the normal theory associated with a relational spreadsheet.
Because the call graph CGs obtained from interpreting a relational spreadsheet
s as a normal theory is acyclic, the procedures outlined in Sects. 5.6.1–5.6.3 can be
applied one layer at a time, starting from the primitive predicate q that was initially
updated and working our way through the paths of CGs that radiate out of q. We
start with the predicates p1 whose longest path to q has length 1 (i.e., q appears in
the body of a clause for p1 , and no other predicate p1 depends on in turn relies on q).
We can then compute the positive and negative updates p1 and ıp1 , respectively,
to p1 . Next, we look for predicates p2 in CGs whose definition is at most two hops
away from q. We can then compute p2 and ıp2 on the basis of q, q, ıq, p1 , p1
and ıp1 (some of which could be empty). This process is iterated until all predicates
affected by the change to q have been updated. Note that we will have to amend this
procedure in Chap. 6 when allowing predicates to be recursive.
In this description, we have considered only logical dependencies for simplicity.
Clearly, an update may permeate through the interface between the relational
operators and the scalar and array worlds. In that case, the traditional mechanisms
discussed in Chap. 3 would be invoked. This in turn may trigger further logical
updates if a change caused by a scalar expression affects a primitive relation. In this
general case, updates are monitored and propagated along the edges of the attribute
dependency graph DGAs . Recall that it subsumes the relational substrate graph RSs ,
hence taking care of functional dependencies among predicates.
The last item we need to discuss is how exactly updates are propagated. The
procedure we have described computes updates to the head of a clause from changes
to the extension of its body literals, which makes top-down derivation inappropriate.
Therefore, the remaining candidate methods are either the fixpoint computation used
for evaluation, or the bottom-up construction of derivations. At the onset, both are
applicable. Recent work [LC12, NJLS11] suggests however that the latter may offer
slightly better control over the ever changing theories involved during an update. A
more detailed study is however needed in order to establish whether this translates
into actual gains in efficiency.

5.7 Explanation Revisited

In Chap. 4, we observed that relational algebra and the various types of dependency
graphs we designed to support relational expressions in a spreadsheet come short of
providing a satisfactory explanation mechanism at the relational level. Indeed, these
graphs help visualize which relation depends on which—even how attributes truly
influence each other—but they are incapable of answering record-level questions,
such as “why is this record in the result?”. In particular, negative requests such as
“why is this expected record not present?” are totally out of reach.
178 5 The Logical Spreadsheet

The top-down approach to provability provides a general mechanism to answer

precisely these types of questions, and several more. We will now examine how this
can be done. In Sect. 5.7.1, we concentrate on the problem of positive explanation,
illustrated by the first question above. The second problem, or negative explanation,
is studied in Sect. 5.7.2. Note that a typical explanation request may explicitly or
implicitly make use of both as the derivation process may line up positive and
negative subgoals. Moreover, it may require an explanation at the scalar level, or
participate in one. These issues are addressed in Sect. 5.7.3.

5.7.1 Positive Explanation

Let s be a relational spreadsheet and … s its associated normal theory. Whenever

the user sees the unexpected record Ev among the result of a calculated relation p,
the ensuing explanation request can be effectively answered by constructing the
derivation tree of the judgment … s ` p.Ev/ and having the user navigate it until she
successfully resolves the cause of her skepticism. Since Ev was in the output of p, this
judgment is certainly derivable. Note however that there may be several derivations
for it. As described in Sect. 5.4.3, the top-down approach provides a simple, fast and
effective method for constructing all of its derivation trees.
The interaction may proceed as follows:
1. In response to the user’s inquiry about Ev, we invoke the top-down procedure on
the judgment … s ` p.Ev/ 7! ¿ of Sect. 5.4.3. In general, it will report several
derivations for p.Ev/, supported by derivation trees D1 ; : : : ; Dn . Finding these
derivations is always possible in the language examined here, but we will need
to be careful in Chap. 6 as we introduce recursion. Notice that, since Ev was in the
output of p, there is at least one such derivation.
2. If there is more than one derivation, the user is asked to choose the one she wants
to inspect first. Call it D.
3. Replace every judgment … s ` B 7! ˛ in D with … s ` ˛.B/, so that every
goal is fully instantiated.
4. If D simply consists of an instance of rule pr", then p is primitive and
the corresponding record is highlighted. If applicable, scalar explanation may
be triggered to inquire about the reason for the individual values in Ev (see
Sect. 5.7.3).
5. If D starts with an application of rule np" (and therefore p is not primitive), we
shall point the user to the rule p.X E /  B that was used to produce p.Ev/ and
display all of the subgoals of the (ground) instance of B mentioned in this rule.
The user shall be able to require an explanation for each of them, specifically:
0
(a) For a positive literal q.Ev /, we repeat step 4 or 5 on the subtree rooted at
0
… s ` q.Ev /.
5.7 Explanation Revisited 179

0
(b) For a negative literal :q.Ev /, we switch to negative explanation mode (see
Sect. 5.7.2).
(c) For a constraint æ  æ0 , we invoke the scalar explanation mechanism if æ or
æ0 originally contained cell references. Their validity should be self-evident
otherwise.
6. At any moment, the user shall be able to expand the explanation of any other
subgoal. She shall also be able to switch to an alternative derivation tree D0 ,
hence going back to step 2. If D0 has much in common with D, steps that the
user has already undertaken for D could be pre-expanded.
This procedure can be implemented graphically in the same way as scalar explana-
tion mechanisms rely on a graphical method to display and explore the dependency
0
graph stemming from any location. For example, each positive subgoal q.Ev / at
0
step 5 identifies the record Ev in relation q, which could be highlighted and related
to p.Ev/ by means of an arrow. We will examine the graphical interface to the
explanation facility in Part II.
As an aside, observe that the top-down construction could be used for more than
just explanation. Indeed, it is conceivable that we allow the user to issue one-time
queries for an arbitrary goal B. The answer would then be a relational value with
one attribute for every variable appearing in B and as many records as there are
distinct substitutions ˛ for the top-down judgment … s ` B 7! ˛. This result is
ephemeral in the sense that it is not bound to any cell in the spreadsheet, but would
appear only within a query window. It could be made permanent by pasting B in
an appropriate range of cells. A proposal in this sense can be found in [CvEL88].
The explanation mechanism just outlined operates also with respect to one-time
queries.

5.7.2 Negative Explanation

The same general principle can be applied to support negative explanation. When-
ever the user observes that an expected value Ev is not computed for p, an explanation
can be obtained by issuing the top-down query … s 6` p.Ev/. If Ev is not a calculated
value for p, then this judgment is derivable. Notice that, for the most part, it will
have a single, rather large, derivation tree, the only exception being whenever a
subgoal of the form … s 6` :Q is encountered since Q may have several positive
derivations.
Displaying this tree and helping the user navigate it is not as simple as in the case
of positive explanation. Indeed, we cannot effectively pre-instantiate inapplicable
clauses so that she can “see” why a fact is not derivable. For example, assume
that the user wants … s 6` p.a; b/ explained, and … s contains the single clause
p.X; Z/  q.X; Y / ^ r.Y; Z/ for p. Then, the problem reduces to … s 6`
q.a; Y / ^ r.Y; b/: there may be values for Y that satisfy q and others that satisfy
180 5 The Logical Spreadsheet

r, but none satisfy both. A more refined explanation can be obtained by asking the
system for a value c of Y that makes q.a; Y / derivable and showing the user that
r.c; b/ has no derivation, or vice versa. Altogether, there will generally be many
more possibilities to explore than for positive explanation. While a motivated user
will sift through them until she is convinced by the explanation (for example by
discovering an error somewhere), a more lenient user may not be as persevering
unless a simple and intuitive graphical mechanism is available. This aspect will be
discussed more thoroughly in part II.
As in the case of positive derivations, it is conceivable that a user asks about the
non-derivability of an arbitrary ground goal B. This is not as useful as in Sect. 5.7.1
because B may not contain variables.

5.7.3 Integrated Explanation

As we mentioned earlier, a record-level relational explanation mechanism based on

top-down derivability must integrate with the techniques available for the scalar and
array components of a spreadsheet. Thanks to our careful design of the interface
between the two worlds, all interactions are localized in exactly three places:
• In base relations hEe i where a relational expression may refer to records computed
by scalar or array formulas;
• In attribute expressions æ present in selection conditions and extended projec-
tions; and
• In array expressions embedding the coercion operator Œe. N
Although each of these transitions imply a mode shift, as scalar and logical
explanation employ different mechanisms, they can be transparently implemented
so that users experience smooth, imperceptible, integration.
It should be observed that the relation- and attribute-level explanation techniques
discussed in Chap. 4 are still available if the user needs them. Indeed, their logical
counterpart, the (enriched) call graph and the data flow graphs of a theory, interact
smoothly with top-down derivation, hence adding a third integrated explanation
modality. Altogether, these mechanisms provide the user with powerful and flexible
explanation facilities.

5.8 Summary

In this chapter, we have translated the relational features developed in Chap. 4 into
first-order logic. The resulting formulas circumscribed a specific instance of the
logic programming paradigm, at the confluence of Datalog and constraint logic
programming. The resulting language, which does not embed recursion at this point,
is summarized in Fig. 5.5.
5.8 Summary 181

Normal Theories ΠΓ ::= Π, Γ

Factual Theories Π ::= · | Π, p(v)
Normal Clausal Theories Γ ::= · | Γ, Cl
Normal Clauses Cl ::= H B
Heads H ::= p(X ; : : : ; X)
Normal Bodies B ::= P | ¬P | æ ∼ æ | B ∧ B
Predicates P ::= p(X ; : : : ; X)

Fig. 5.5 Normal theories

In particular, relational expressions are realized by a set of clauses of the

following form:
^ ^ ^
P  Pi ^ :Qi ^ æi i æ0i
i D1::m i D1::n i D1::o

This logical interpretation does not alter the expressive power of the relational
spreadsheet. As of this chapter, it only provides an alternative syntax for relational
expressions. In particular, the definition of spreadsheet is upgraded only to the extent
that a cell range can contain a normal theory instead of the equivalent relational
expression. The distinction is purely a matter of syntax, although different users
may prefer the relational or logical interpretation in different circumstances. Issues
relating to concrete syntax will be examined in Part II.
The logical interpretation of a relational spreadsheet makes available nearly
50 years of research in logic programming as an aid to the development of the
deductive spreadsheet. In this chapter, we have scrutinized the most promising
results of this multi-decade effort and adapted them to the idiosyncrasies of the
spreadsheet context. In particular, we have observed that goal-oriented derivation
does not play as prominent a role as in traditional logic programming. Indeed,
this computational interpretation suits well the needs for a record-level explanation
mechanism, but is inappropriate at evaluation and update time. Instead, these two
key functionalities require a reading of logic programming that computes and
maintains all the atomic logical consequences of a theory. The fixpoint semantic of
logic programming, initially developed for foundational reasons and later applied
to answer queries in deductive databases, appears to be ideally suited to the initial
evaluation of a spreadsheet. The crucial operator that realizes it has the form:
V V
T…& ; & .I / D I [ …& [ f˛.P / W P  i D1::m Pi ^ i D1::n :Qi
V
^ i D1::o æi i æ0i 2 &
and ˛.Pi / 2 I [ …& for all i D 1::m
and ˛.Qi / 62 …& for all i D 1::n
and ˛.æi / O i ˛.æi / is true for i D 1::o g
182 5 The Logical Spreadsheet

Updates seem to benefit from the closely related bottom-up derivation strategy for
logic programs. In both cases, the notion of stratification, which was originally
developed as a support to effectively implement negation, provides an immediate
mechanism to handle the functional dependencies that can naturally emerge in a
spreadsheet.
It is time to assess again our progresses against the objectives set forth at the end
of Chap. 3:
• We have shown that the logic-inspired mechanisms do not only work well in
isolation, but interface transparently with the scalar and array functionalities
of a spreadsheet. Therefore, as we are transitioning toward the truly deductive
spreadsheet, we continue to have a perfect integration between our extension and
the traditional support.
• We have also shown that the logical interpretation can realize evaluation and
update as efficiently as the relational view of Chap. 4. Additionally, the vast body
of literature on logic program optimization has the potential of further reducing
the overhead of these functionalities.
• As just mentioned, top-down derivation is a well-understood mechanism that is
ideally suited to supporting record-level explanation. Recall that this was the one
aspect that relational algebra was unable to deliver.
As we read these rather positive assessments, it should be kept in mind that the
logical interpretation considered in the current chapter has the exact same expressive
power as the relational language introduced in Chap. 4. We will now extend its
expressiveness.
Chapter 6
The Deductive Spreadsheet

The present chapter completes the design of the deductive spreadsheet by adding
recursion and some of its consequences into the logical language examined in
Chap. 5. While substantially extending the class of problems that can be modeled,
a simple-minded inclusion of this construct raises the concrete possibility of
divergence. This unwanted behavior can be traced back to the presence of head
constraints within recursive clauses and therefore disallowing them in that position
ensures termination. With this restriction in place, all the techniques and results
obtained for the logical spreadsheet extend with minimal adjustments to theories
enriched with recursion. This approach to curbing divergence proves however
overly zealous as it sacrifices a range of extremely common problems. Most
of these problems can be safely rehabilitated by applying a simple behind-the-
scenes transformation bounding the number of recursions in which those clauses
can participate. The same device preempts divergence in several extensions of
the resulting language. We admit one such linguistic extension in the deductive
spreadsheet, lists, as they considerably extend the expressiveness of our formalism
and yet blend smoothly into a spreadsheet infrastructure.
By the end of this chapter, the logical language of the deductive spreadsheet
will have taken form as a version of stratified Datalog enriched with constraints
and lists. This formalism is extremely powerful and yet its disciplined construction
and logical underpinning exude simplicity and elegance. The same design principles
suggest a few avenues for further improvements. One of them is its extension with
embedded implication as a native support for hypothetical reasoning. A partial
exploration of this option makes it a promising candidate for inclusion in a future
revision of our language.
This chapter is organized as follows. Section 6.1 introduces our recursive
extension by pursuing the registrar’s spreadsheet examples of Chaps. 4 and 5 with
natural inquiries that require recursion. We formalize our language and develop its
theory in Sect. 6.2. We ease the tension between termination and expressiveness in
Sect. 6.3, admitting lists in our language in the process. We present a variety of
additional examples in Sect. 6.4, and comment on several extensions, among them

I. Cervesato, The Deductive Spreadsheet, Cognitive Technologies, 183

DOI 10.1007/978-3-642-37747-1 6, © Springer-Verlag Berlin Heidelberg 2013
184 6 The Deductive Spreadsheet

embedded implication, in Sect. 6.5. Finally, Sect. 6.6 summarizes and reflects upon
the work done.

6.1 Spreadsheets with Recursive Definitions

In Chap. 4, we developed a simple example aimed at helping the registrar of a

fictitious university keep track of students and courses. It motivated the relational
extension of the spreadsheet and demonstrated how its operators could be used
to solve a whole new class of problems in the same simple and natural way that
formulas are used in the traditional spreadsheet. In Chap. 5, we translated these new
expressions into logic, hence providing a different perspective. We will now extend
this same example to demonstrate how the addition of recursion to this language
causes another jump in expressive power. At the same time, we show techniques
to solve typical problems. We also point to some natural problems that cannot be
tackled by recursion in its purest form, but that will be addressed in other sections
of this chapter.

6.1.1 Transitive Closure

The base predicates of the registrar’s example, introduced in Chap. 4, described the
following simple relations:
• Student.F; L; S / collects the first name F , last name L and some identifier S for
a student;
• Course.N; C; I / relates the name N of a course, an abbreviation C for it, and
the instructor I who teaches it;
• HasTaken.S; C; D; G/ describes the fact that student S has obtained grade G for
course C during semester D. Because we will never make use of the last two
parameters, we define the predicate Took that only tracks the first two variables:

Took.S; C /  HasTaken.S; C; D; G/

They were introduced as relations in Fig. 4.3. A few more base relations were
defined in Chap. 4, but we will not be relying on them. Instead, for our new examples
to be significant, we need to introduce a few additional predicates that we shall
consider primitive. The first one is Prereq.C; RC/, which extensionally describes
the fact that a student must have successfully taken course RC in order to take course
C : course C has RC as a prerequisite. A few instances are rendered in columns DA
and DB in Fig. 6.1.
The requirements for a course, for example Graphics (abbreviated “gr”) in
Fig. 6.1, may be determined by its instructor, who would then communicate them
6.1 Spreadsheets with Recursive Definitions 185

··· DA DB DC DD ···
1 Prereq CourseDep
2 Course Course’ Course Course’
ic ca
3
CourseDep (C, C  )
ic tc
4
Prereq (C, C  )
5 ca gr
6 bn an CourseDep (C, C  )
7 ca os Prereq (C, C  )
... ... ∧ CourseDep (C  , C  )
8


. . . or . . .
CourseDep (C, C  )
Prereq + (C, C  )

Fig. 6.1 Transitive course requirements in the registrar’s spreadsheet

to the registrar’s office. Here, the graphics instructor has requested Computer
Architecture (“ca”) as a requirement. Note that she may not be aware that “ca” has
Introduction to Computing (“ic”) as a prerequisite and therefore will not specify
it when communicating with the registrar. Thus, the predicate Prereq describes
individual requirements, but not necessarily all the transitive requirements. Now,
members of the registrar’s office may want to see all the course dependencies, for
example to make sure that no circular requirement was entered by mistake.
Solving specific instances of this problem in the relational (or logical) spread-
sheet is possible but awkward. For example, an employee can describe all the
indirect requirements with a single intervening course by means of the clause
Prereq2.C; C 00 /  Prereq.C; C 0 / ^ Prereq.C 0 ; C 00 /; requirements three hops apart
are then given by Prereq3.C; C 00 /  Prereq.C; C 0 / ^ Prereq2.C 0 ; C 00 /, and so on.
Although feasible in a university setting, where prerequisite chains have length 10
or less, this is quite tedious. This number may rise to tens of thousands in other
similar problems, making this approach totally impractical.
This problem is unsolvable in the general case using only the relational or logical
language analyzed so far. However, it has a simple and elegant solution as soon as
we allow clauses to be recursive. Indeed, the predicate CourseDep, defined next,
captures transitive requirement dependencies in their generality:

CourseDep.C; C 00 /  Prereq.C; C 00 /
CourseDep.C; C 00 /  Prereq.C; C 0 / ^ CourseDep.C 0 ; C 00 /

The first clause states that course C depends on C 00 if C 00 is a prerequisite for it.
The second clause specifies that C also depends on C 00 if it has a prerequisite
C 0 which depends on C 00 . Notice that this second clause is recursive as the head
predicate CourseDep also occurs in the body. We have reported these two clauses
186 6 The Deductive Spreadsheet

in columns DC and DD of Fig. 6.1 (the clause below the brace will be introduced as
an abbreviation in Sect. 6.4.1).
Recursion is a very powerful tool that substantially extends the expressive power
of the language we have been defining. We will formally include it in this language
and develop its theory in Sect. 6.2. Before doing so, we will examine a few more
examples of use.
The clauses for CourseDep are typical of a transitive closure problem: given a
finite binary relation R, determine its transitive closure RC . An alternative and even
more common formulation interprets the relation R as the edges of a graph G, hence
making RC the set of all pairs of nodes in G linked by a path. The pattern of the
above clauses can indeed be reused for this problem:

path.X; Z/  edge.X; Z/
path.X; Z/  edge.X; Y / ^ path.Y; Z/

Here, it is assumed that the edges of G are extensionally represented by the primitive
predicate edge in a factual theory …edge . If we collect the two formulas for path
in the clausal theory path , then each pair of nodes n; n0 linked by a path in G
will be acknowledged by the fact that path.n; n0 / is a logical consequence of
.…edge ; path /.
Note that the graph G may contain cycles. If G is finite, the number of distinct
nodes n; n0 connected by a path in G is finite as well (but the number of paths
themselves is clearly infinite). Because we are skirting infinity so closely, we
shall take extreme care in the design of the methods used to establish logical
consequences and make sure that they terminate in all circumstances. This will be
the subject of Sect. 6.2.
The need to take a transitive closure arises frequently when working with
relations (as we have since Chap. 4). Besides course requirements and paths in
graphs, common examples include genealogies (interpret edge as a “parent” relation
and path as “ancestor”), hierarchies (take edge to be “supervisor” and path to
be “works under”), airline connections (let edge and path be “direct flight” and
“indirect flight”), chains of command, scheduling orders, etc.

6.1.2 Expressing Extensional Universal Quantification

We now continue our main example and show how the predicate CourseDep just
defined can be used to draw some rather advanced inferences. Consider first the
problem of establishing what courses a student S needs to take before being able to
sit in a course C . It is solved by the following simple clause:

CouldTakeIf .S; C; RC/  Student. ; ; S / ^ CourseDep.C; RC/

^ :Took.S; RC/
6.1 Spreadsheets with Recursive Definitions 187

··· DF DG DH DI DJ DK DL ···
1 CouldTakeIf MissingReq CanTake
2 Student Course Course’ Student Course Student Course
3
CouldTakeIf (S, C, RC ) MissingReq(S, C ) CanTake(S, C )
4
Student( , , S) CouldTakeIf (S, C, ) Student( , , S)
5
∧CourseDep(C, RC ) ∧Course( , C, )
6
∧¬Took (S, RC ) ∧¬MissingReq(S, C )
7


. . . or . . .
CanTake(S, C ) Student( , , S ) ∧ Course( , C, ) ∧
FORALL RC : CourseDep(C, RC ) . Took (S, RC )

··· DM DN DO DP ···
1 MaybeHarder Easier
2 Course Course’ Course Course’
3
MaybeHarder (C1, C 2) Easier (C1, C 2)
4
Prereq(C1, ) ∧ ¬Prereq(C2, C  ) ¬MaybeHarder (C2, C 1)
5
∧Course( , C 2, ) ∧ Course( , C  , ) ∧Course( , C 1, )
6
MaybeHarder (C1, C 2) Prereq(C1, C1 ) ∧Course( , C 2, )
7

8
∧Prereq(C2, C2 ) ∧ MaybeHarder (C1 , C 2 )

Fig. 6.2 Course evaluation in the registrar’s spreadsheet

As already in Chap. 5, we write ‘ ’ for a variable with a single occurrence in a clause

and whose name is unimportant. This formula simply says that S does need course
RC in order to sit in C whenever C depends on RC and S has not yet taken RC. The
predicate Student in the body of this clause is needed to satisfy the safety restrictions
encountered in Sect. 5.4, which will be in force also in the presence of recursion.
Without it, bottom-up evaluation would not know what to use for the variable S .
A student S is missing a prerequisite for course C if there is some course
he would need to take in order to sit in C . This is immediately captured by the
following clause, which simply abstracts away the third argument of CouldTakeIf :

MissingReq.S; C /  CouldTakeIf .S; C; /

Finally, consider the non-trivial problem of deciding whether a student is eligible

to take a given course. The definitions presented in this section allow writing another
very simple clause to resolve this question:

CanTake.S; C /  Student. ; ; S / ^ Course. ; C; / ^ :MissingReq.S; C /

Indeed, S can take C if she is not missing any prerequisite for it. The other two
predicates are again here to satisfy the safety restrictions. All three definitions are
reported in the top part of Fig. 6.2 (the clause below the brace is again an abbreviated
form that will be introduced in Sect. 6.4.1).
188 6 The Deductive Spreadsheet

Conceptually, CanTake can be defined much more simply by requesting that S

have taken all the prerequisites of C . A direct attempt at capturing this sentence in
a clause, for example

CanTake0 .S; C /  CourseDep.C; RC/ ^ Took.S; RC/

does not yield the desired behavior: this clause answers whether S has taken any
required course RC for C . The request for all prerequisites of C has incorrectly
been rendered as a request for some prerequisite of C . Extending our language
to a fragment of first-order logic that could express the above sentence within a
single clause would compromise termination. The two negations in our original
specification however forces the use of the correct quantifier. Indeed, it asks whether
it is true that there is no dependent course RC of C that S has not taken. This is more
clear if we inline the predicates CouldTakeIf within the definition of CanTake:

MissingReq.S; C /  Student. ; ; S / ^ CourseDep.C; RC/ ^ :Took.S; RC/

CanTake.S; C /  Student. ; ; S / ^ Course. ; C; / ^ :MissingReq.S; C /

By moving the implicit existential quantification behind the first negation (the
variable RC in the clause for MissingReq), we have effectively implemented the
desired form of universal quantification.
We can use this technique together with recursion as in the following example,
which a lazy student may use to select a course with the least number of
prerequisites:

MaybeHarder.C1; C 2/  Prereq.C1; / ^ :Prereq.C 2; C 0 /

^ Course. ; C 2; / ^ Course. ; C 0 ; /
MaybeHarder.C1; C 2/  Prereq.C1; C10/ ^ Prereq.C 2; C 20 /
^ MaybeHarder.C10 ; C 20 /
Easier.C1; C 2/  :MaybeHarder.C 2; C1/
^ Course. ; C1; / ^ Course. ; C 2; /
The first two clauses define the recursive predicate MaybeHarder.C1; C 2/ which
determines whether course C1 has a longer chain of requirements than C 2. The
second clause simply follows prerequisites until the first clause can determine
that C1 still has requirements while C 2 does not. The clause for Easier.C1; C 2/
establishes that C1 is indeed easier than C 2 because all requirement chains for
C 2 are longer than C1’s. These two definitions are reported in the bottom part of
Fig. 6.2.
As an aside, the form of recursion just used does not fit into the transitive closure
pattern. Indeed, it is closely related to that of the popular “same generation cousin”
problem:
sgc.X; X /  person.X /
sgc.X; Y /  parent.X; X 0 / ^ parent.Y; Y 0 / ^ sgc.X 0 ; Y 0 /
6.1 Spreadsheets with Recursive Definitions 189

which computes the pairs of people who are same generation cousins in a genealogy.
It assumes the unary predicate person as identifying people, and the parent
predicate which describes parent-sibling relations (the constraint X ¤ Y is often
added). Definitions such as sgc are often used as benchmarks for optimization
techniques [CGT90].

6.1.3 Combining Deductive and Functional Computation

Assume now that our university awards various types of degrees subject
to students having taken certain key courses. Let the primitive predicate
CourseForDegree.C; D/ list the core courses C associated to each degree D. It
could state for example that having taken Graphics is mandatory in order to obtain
a Masters in Computer Aided Design. Then, the list of all courses that a student
should take in order to obtain a given degree is defined by the following two
clauses.

NeededCourse.C; D/  CourseForDegree.C; D/
NeededCourse.C; D/  NeededCourse.C 0 ; D/ ^ Prereq.C 0 ; C /

Observe that a more efficient non-recursive definition could be written by relying

on PrereqC (or CourseDep) rather than Prereq. Now, the following clause easily
allows listing the courses that any student S still needs to take in order to qualify for
a given degree D:

CourseToDegree.S; D; C /  Student. ; ; S / ^ NeededCourse.C; D/

^ :Took.S; C /
We have reported these two definitions in the top part of Fig. 6.3. The extension of
the relation CourseForDegree is omitted for conciseness.
Although this detailed information is quite useful, it does not readily answer the
question “How many courses does Sue Smith still need in order to get her Masters
in Computer Aided Design?”. After all these examples, it becomes second nature
to try to devise some recursive definition that allows calculating this number. It is
then somewhat disappointing to learn that this simple problem is not solvable in
the simple language we have developed so far. Indeed, there is no way to clausally
count how many facts of the form CourseForDegree there are for any given degree.
This is the time to remember that the deductive spreadsheet tightly integrates all the
usual spreadsheet functionalities. Indeed, we can rely on the functional sublanguage
of our design to construct a simple solution to this problem. Actually, we have
already solved a very similar problem in Fig. 4.8. We want to define a ternary
relation nCourseToDegree in columns DX-DZ of our example that associates each
student and degree to the number of courses she needs to take to be eligible for
190 6 The Deductive Spreadsheet

··· DS DT DU DV DW ···
1 NeededCourse CourseToDegree
2 Course Degree Student Degree Course
3
NeededCourse ( C, D ) CourseToDegree ( S , D , C )
4
CourseForDegree ( C, D ) Student ( , , S )
5
NeededCourse ( C, D ) ∧NeededCourse ( C, D )
6
NeededCourse (C  , D ) ∧¬Took ( S, C )
7

8
∧Prereq (C  , C )

··· DX DY DZ ···
1 nCourseToDegree
2 Student Degree Number
3
( S, D ) =COUNT([ (C )
4
Student ( , , S ) CourseToDegree ( S, D , C )
5
∧CourseForDegree ( , D ) ∧ S = DX 3: DX
6
∧ D = DY 3: DY ])
7

Fig. 6.3 Recursive clauses and formulas in the registrar’s spreadsheet

it. Let us first insert every student-degree pair in columns DX-DY. The anonymous
clause

.S; D/  Student. ; ; S / ^ CourseForDegree. ; D/

achieves this. The following array formula then computes the number of courses in
CourseToDegree for every such pair:

COUNT.Π.C /  CourseToDegree.S; D; C /
^ S D DX3WDX
^ D D DY3WDY /

We put it in column DZ. For any location DZn (with n > 2), the inner anonymous
clause extracts from CourseToDegree all the courses C that the student in cell DXn
needs to take in order to qualify for the degree in cell DYn. The resulting set of
values is converted into an array by means of the Œ: : : operator and counted using
the traditional function COUNT. The combined process is reported at the bottom of
Fig. 6.3.1

1
It should be noted that this problem can also be solved in a purely functional manner by relying
on the extended array operators  E and E introduced in Chap. 4. A purely clausal solution is also
possible, but requires further extensions to our language. It will be examined in Sect. 6.5.
6.1 Spreadsheets with Recursive Definitions 191

6.1.4 Limitations

Extending clausal theories with recursion adds a considerable amount of expressive

power to the purely relational/logical spreadsheet, not to mention its traditional
incarnations. The adoption of this technique may raise expectations beyond what
it can deliver, however. We have just encountered a simple problem that could not
be solved by relying on the logical infrastructure alone. The tight integration of the
functional subsystem was crucial in overcoming this obstacle.
There are actually numerous natural problems that cannot be solved even in the
integrated framework. We will now describe a few. Simple solutions to each of them
can be found in any logic programming textbook, but they revolve around features
that, if introduced in our formalism, would put termination in jeopardy. In later
sections of this chapter, we will explore meta-logical devices that will generally
provide a solution, but not always.
Assume in our ongoing example that courses C and C 0 cannot be taken in the
same semester if C 0 is a prerequisite to C . Then, the length of longest requirement
chain between any course actually taken by a student S and the core courses for a
degree D will determine how many semesters this student still has to go. We will
see that, in order to safeguard termination, this problem cannot be solved in our
current language, even with the help of the functional fragment. Indeed, while it is
easy to count the total number courses S needs to qualify for D as we saw in the
previous section, computing the number of courses on any individual Prereq chain
is beyond reach. The general problem behind this example is the task of counting
the number of edges on a path between two nodes in a graph. This corresponds to
computing the number of hops to a destination in the airline example. While simple
and useful, these problems fall beyond the scope of our language (we will return to
them in Sect. 6.3.1).
Also unsolvable in our language is the task of producing the list of all nodes on a
path. In the airline example, knowing that Prague is reachable by air from Wichita,
KS is generally only the beginning of a long series of questions; the first is often
how many plane changes are involved (we just examined this problem) and it is
often followed by what are the intermediate airports. The corresponding issue in
our example is listing the courses in a chain to a core degree requirement. A general
approach to solving this class of problems needs to cope both with termination
issues and with fundamental interface design aspects, such as how to incorporate
lists in the spreadsheet model. We will further examine it in Sect. 6.3.2.
As a final limitation, consider the variant of the problem studied in Sect. 6.1.1
which allows any given course to have alternative requirements. While it would
be easy to determine all the courses that are possibly needed in order to sit in
some given course, listing clusters of courses depending on any given alternative
is infeasible. This problem can be solved by collecting actually dependent courses
in explicit lists and reasoning on them, but not within our simple framework. This
raises the same issues about lists just outlined. Again, we will explore tentative
solutions in Sect. 6.3.2.
192 6 The Deductive Spreadsheet

6.2 Recursive Spreadsheet Theories

Given this series of examples as motivation, we will now develop the machinery
underlying recursive spreadsheet theories along the lines of the discussion in
Chap. 5. The good news is that we have already done much of the required
work in that chapter. Indeed, this section will mostly be concerned with minor
adjustments to adequately model and mechanize recursive definitions. We will be
particularly concerned with making sure that all the procedures devised so far
remain terminating even in the presence of recursion.
We begin, in Sect. 6.2.1, by updating the safety requirements on general theories
to ensure termination. In Sect. 6.2.2, we revisit the fixpoint semantics of Chap. 5
as an evaluation procedure for recursive theories and show that it terminates also
in this extended setting. In Sect. 6.2.3, we refine our update methodology to take
recursive clauses into account. In Sect. 6.2.4 we take another look at top-down
derivability as an explanation mechanism in recursive theories. In Sect. 6.2.5, we
situate the fragment of first-order logic underlying the deductive spreadsheet within
the taxonomy of logic programming languages. Finally, in Sect. 6.2.6 we interpret
the obtained language back to relational algebra.

6.2.1 Syntax and Basic Definitions

Judging from the examples in Sect. 6.1, the formulas underlying the deductive
spreadsheet differ from the language examined in Chap. 5 by simply lifting the ban
on recursively defined clauses. This is the case at the macroscopic level and, indeed,
the grammar defining the fragment of first-order logic we will be working with
remains the same:

Theories … WWD …;
Factual Theories … WWD j …; p.Ev/
Clausal Theories WWD j ; Cl
Clauses Cl WWD H B
Heads H WWD p.X; : : : ; X /
Bodies B WWD P j :P j æ  æ j B ^ B
Predicates P WWD p.X; : : : ; X /

Terminology and notation remain unchanged with respect to Chap. 5. In particular,

each ground predicate p.Ev/ in a factual theory is a fact. Similarly, body items of the
form P and :P are positive and negative literals, respectively, and each æ  æ0 is
a constraint: a head constraint if it has the form X D æ0 for X occurring in the
head of this body’s clause, a generic constraint otherwise. We collect all clauses
defining an intensional predicate symbol q in the subtheory q and similarly gather
all facts p.Ev/ of a primitive predicate p in the subtheory …p . As in Chap. 5, a
6.2 Recursive Spreadsheet Theories 193

clause without any negative literals is called definite, otherwise normal, and we omit
these adjectives when unimportant or clear from the context. The same applies for
(sub)theories.
Since the defining grammar has not changed, the general form of a clause remains
the same:
^ ^ ^
P  Pi ^ :Qi ^ æi i æ0i
i D1::m i D1::n i D1::o

This time however, the predicate symbol p of the head P may occur in one or more
body literals Pi . This form of recursion, in which p occurs in the body of a clause
defining it, is called simple. All recursive clauses in Sect. 6.1 were simple. A clause
for p may also rely on a predicate q mentioning p in one of its defining clauses.
Then, p and q are said to be mutually recursive. Needless to say, mutual recursion
can generally involve more than two predicates. No matter whether a predicate p
participates in a simple or a mutual recursion, it is often the case that some of
its defining clauses will not be recursive. Such clauses form the base cases of the
definition of p. Note that in a non-recursive theory, all clauses are base clauses.
At this point of Chap. 5, we introduced a number of safety requirements about
variable occurrences within a clause to ensure mechanizability. We recall them in
preparation of further restrictions:
1. A variable X occurring in the head of a clause also must occur:
E / in the body of the clause,
(a) Either in a positive literal p.X
(b) Or as the left-hand side of a head constraint X D æ.
2. Any variable X occurring within a generic constraint æ1  æ2 , or on the right-
E/
hand side of a head constraint Y D æ, must also occur in a positive literal p.X
in the body of the clause.
3. Any variable X occurring in a negative literal :Q must also occur in a positive
E / in the body of the clause.
literal p.X
While these local requirements still apply, the admission of recursive clauses forces
us to impose further restrictions, this time at the level of groups of clauses. They
do not properly apply to variables per se, but they affect how variables are used.
These new requirements are best formalized in terms of the dataflow graph of a
recursive theory.

Call Graph

Recall from Chap. 5 that the (strict) call graph CG… of a theory … D .…; /
tracks the dependencies among the predicates in … . It distinguishes between the
purely relational dependencies due to a predicate symbol q occurring in a clause
defining p, and dependencies arising from negation, which we labeled functional
(we temporarily ignore extra-logical dependencies arising from the underlying
194 6 The Deductive Spreadsheet

Prereq Student Course CourseForDegree Took

ς0 CourseDep NeededCourse 1

¬Prereq ¬Took

MaybeHarder CourseToDegree CouldTakeIf

ς1 2 MissingReq

¬MaybeHarder ¬MissingReq

ς2 Easier CanTake

Fig. 6.4 Stratified call graph in the recursive part of the registrar’s spreadsheet

spreadsheet). The distinction is formalized by relying on two types of edges: !

and Ü, respectively. The precise definition is reported from Chap. 5:
• Nodes.CG… / is the set of all predicate symbols and negated predicate symbols
appearing in … .
• Edges.CG… / is the set of all edges .:/q ! p such that .:/q appears in a clause
defining p, and all edges p Ü :p for p; :p in Nodes.CG… /:

Edges.CG… / D E /  q.YE / ^ B 2 g
fq ! p W p.X
[ f:q ! p W p.XE /  :q.YE / ^ B 2 g

[ fp Ü :p W p; :p 2 Nodes.CG… /g

The call graph corresponding to the theory accumulated from the examples in
Sect. 6.1 is displayed in Fig. 6.4. If … is distilled from a (deductive) spreadsheet s,
a situation we denoted by … s in Chap. 5, additional functional dependencies arise
when cells in the base relation corresponding to a primitive predicate reference or
overlap cells calculated clausally. The arrow from node 1 to node 2 (which corre-
spond to the two anonymous predicates in Fig. 6.3) in Fig. 6.4 is an instance of this
phenomenon. The graph obtained by incorporating these additional dependencies
into CG… s was called the enriched call graph of s. In Chap. 5, it had the property
that there were no cycles through ! edges. Cycles through a combination of ! and
Ü edges were possible in degenerate situations that we chose to ignore.
6.2 Recursive Spreadsheet Theories 195

Cycles naturally emerge in CG… as soon as we allow the clauses of the theory
… to be recursive. Indeed, there are three cycles in Fig. 6.4. As evidenced in
the examples of Sect. 6.1, circularity is indeed inseparable from recursion, and is
actually quite beneficial in general. We shall however impose some restrictions on
recursive definitions in order to ensure termination. Specifically, we will forbid
loops involving Ü edges through stratification and prevent recursively defined
predicates from relying on head constraints (we will relax this requirement in
Sect. 6.3.1).
Note that the different forms of recursion are particularly easy to assess on the
call graph of a given theory. Indeed, simply-recursive predicates are characterized
by self-loops, i.e., ! edges whose origin and destination coincide. Instead, mutually
recursive predicates participate in cycles involving more than just one node. Such
cycles are the strongly connected components of CG… . A simple inspection of
Fig. 6.4 shows that all recursions in our ongoing example are simple.

Stratification and the Negative Literal Restriction

In Chap. 5, we introduced stratification as a device to delay the evaluation of

a group of clauses until all predicates they functionally depended on had been
fully calculated. This staged execution soundly accounted for the extra-logical
dependencies of a theory … s , and it has been used for decades as a simple and
natural mechanism to support negation in logic programming [Prz87, CGT90].
This notion does not change when moving to a recursive theory … . We recall
its definition for convenience. A stratum is a set & of predicates in … such that no
path in CG… between two member predicates p and q contains a Ü edge. Each
stratum & identifies a subtheory … & D .…& ; & /. A stratification is a partition of
the set of all predicates occurring in … into strata &E D &0 ; : : : ; &n such that for
i; j D 0::n if pi Ü pj in CG… for pi 2 &i and pj 2 &j , then i < j . Quotienting
CG… with respect to &E produces the acyclic functional dependency graph FD… .E& /.
A theory … is stratifiable if it has a stratification. In this case, it has a least linear
stratification &E D &0 ; : : : ; &n , where &i 1 Ü &i in FD… .E& / for each i D 1::n and
each predicate occurs in the lowest possible stratum short of violating functional
dependencies. See Sect. 5.4.1 for more details. The least linear stratification in our
example from Sect. 6.1 is marked by the dotted lines in Fig. 6.4.
As in Chap. 5, we will require that the theory … s associated with a spreadsheet
s be stratifiable. This restriction is now more interesting since we admit recursive
clauses. Indeed, it effectively bans recursion through negative literals. In particular,
no clause defining a predicate p can contain :p.X E / in its body. This restriction
is indeed commonly found within deductive databases and even general logic
programming [Prz87, CGT90] as it supports a simple, efficient, and natural attitude
toward negation. Those same motivations apply to the deductive spreadsheet. It
should be noted however that this restriction effectively curtails the expressive
power of our language by limiting the form of admissible theories, although we
196 6 The Deductive Spreadsheet

expect few end-users to ever feel the necessity of writing non-stratifiable spreadsheet
theories. Other, less satisfactory, approaches to handling negation are discussed
in [CGT90].
As noted above, some artificial but otherwise legitimate (non-recursive) spread-
sheet theories are non-stratifiable. As in Chap. 5, we will ignore them for simplicity,
confident that they can be accommodated by appropriately extending the definition
of dataflow graph.

Dataflow Graph and the Head Constraint Restriction

In Chap. 5, we introduced the dataflow graph DF… of a theory … D .…; / as a

refinement of the call graph CG… that tracks dependencies not at the predicate level
but by examining how variables propagate values from the body to the head of a
clause, therefore expliciting the flow of information throughout . Its nodes were
the individual arguments of each predicates in … and its edges related an argument
in the head of a clause to all the variables that can influence it. We defined it as
follows:
• Nodes.DF… / D fpji W p 2 Nodes.CG… / and p has arity n and 1  i  ng.
• Edges.DF … / contains two types of edges: qjj ! pji indicates that the i -th
head argument of a defining clause for p contains the same logical variable as
the j -th argument of a positive literal q in its body, and qjj ) pji means that
E in some
the i -th argument Xi of p is the target of a head constraint Xi D æ.Z/
E
clause for p where the j -th argument of q occurs among the Z’s. In symbols:

Edges.DF … / D fqjj ! pji W .p.XE /  q.YE / ^ B/ 2 … and Xi D Yj g

E /  Xi D æ.Z/
[ fqjj ) pji W .p.X E ^ q.YE / ^ B/ 2 … and Zk D Yj g

We observed that this graph was acyclic for a non-recursive theory.

The definition of dataflow graph remains unchanged when admitting recursion.
However, in this case, DF… will generally contain cycles, as evidenced in the upper
part of Fig. 6.5, which illustrates this concept on the clauses for path. This is clearly
demonstrated in the examples in Sect. 6.1. Cyclic paths through ! edges are totally
benign. Loops that traverse ) arrows can instead put termination into jeopardy. In
order to understand why this is the case, observe that for qjj ) pji to be part of a
cycle, two conditions must be met:
1. The predicate p is recursively defined, and
2. Some recursive clause defining p receives its i -th argument Xi as a head
constraint Xi D æ.
Consider a variant of the path example that keeps a count of how many edges have
been traversed:
6.2 Recursive Spreadsheet Theories 197

edge|1 edge|2 path(X, Z) edge(X, Z)

path(X, Z) edge(X, Y ) ∧ path(Y , Z)

path|1 path|2
...................................................................................................
edge|1 edge|2 path (X, Z, C) edge(X, Z)
∧C=1
path (X, Z, C) edge(X, Y ) ∧ path (Y , Z, C )
path |1 path |2 path |3 ∧C =C +1

Fig. 6.5 Dataflow graph and head constraint restriction

path0 .X; Z; C /  edge.X; Z/ ^ C D 1

path0 .X; Z; C /  edge.X; Y / ^ path0 .Y; Z; C 0 / ^ C D C 0 C 1

Its dataflow graph is given in the lower part of Fig. 6.5. If the factual theory …edge of
the edge predicate implements a graph that is itself acyclic, then evaluating path0 will
return all its paths with their respective lengths. If, however, …edge represent a graph
that has loops, for example if …edge D fedge.a; a/g, then path0 will have infinitely
many valid instantiations of growing length. In this specific example, path0 .a; a; n/
for any n  1 would be derivable. Since the set of logical consequences of this
simple theory is infinite, its evaluation is non-terminating: an undesirable behavior
in the context of a spreadsheet.
Since recursion is a highly prized feature in the deductive spreadsheet, we
will avoid this type of situation by forbidding clauses participating in a recursion
from containing head constraints. We call this the head constraint restriction and
formalize it by saying that the dataflow graph DF… of an admissible theory …
shall not contain cycles involving ) edges. Note that the base cases of a recursive
definition can rely on head constraints: only recursive clauses are affected.
As we will see shortly, the head constraint restriction is sufficient to ensure that
the set of logical consequences of a recursive theory is finite, and therefore that its
evaluation terminates. Giving up head constraints in recursive definitions is however
a high price to pay since they are often harmless and extremely useful in practice.
Consider the following alteration to the clauses for path0 that only reports paths of
length at most 5:

path00 .X; Z; C /  edge.X; Z/ ^ C D 1

path00 .X; Z; C /  edge.X; Y / ^ path00 .Y; Z; C 0 / ^ C  5 ^ C 0 D C C 1

This theory has finitely many logical consequences no matter what …edge contains.
Moreover, it can be used as a template for useful applications such as calculating
the number of hops of an air journey, computing the total distance of a segmented
trip, reporting the cost of a composite product, etc. It however violates the head
constraint restriction and is therefore not admissible.
198 6 The Deductive Spreadsheet

We will require the head constraint restriction to be in effect as we go through the

formal development of recursive theories in the rest of this section. We will however
discuss an effective and yet safe way to drop it in Sect. 6.3.1, hence allowing theories
such as path0 and path00 .

Deductive Spreadsheet

In Chap. 5, we obtained the logical spreadsheet by replacing every relational

expression in a relational spreadsheet s with the corresponding set of clauses.
The deductive spreadsheet is simply obtained by enriching the syntax of these
clauses to admit recursion, subject to the restrictions described in this section.
Therefore, throughout this chapter, we will be exclusively concerned with the logical
subsystem, referring the reader to the previous chapters for every other aspect.
With the syntax of recursive theories solidly in place, we will now revisit the
various methods introduced in Chap. 5 for determining logical consequences. We
will see that, overall, we do not need to make any change to the core results obtained
there to accommodate recursion: we have already done all the hard theoretical work.
We will however need to demonstrate that admitting recursion does not compromise
termination. In some cases, minor adjustments to key procedures will be required to
ensure this. We begin with the model theory of recursive clauses.

6.2.2 Fixpoint Semantics and Evaluation

The model theory of a set of recursive clauses has the same properties as the
simpler fragment studied in Chap. 5. Indeed, validity in a theory … is completely
characterized by the underlying Herbrand base HBV , which contains all the
Herbrand interpretations of … , and in particular its Herbrand models. By the very
arguments presented in Chap. 5, a (recursive) definite theory has a least Herbrand
model and a stratifiable (recursive) theory has a perfect model. We continue to write
I… for either.
Let … be a stratifiable theory. Then, for any stratum & 2 &E and corresponding
subtheory … & D .…& ; & /, the set of logical consequences of … & is constructed
on the basis of the following operator on Herbrand interpretations, reported from
Sect. 5.4.2:
V V
T…& ; & .I / D I [ …& [ f˛.P / W P  i D1::m Pi ^ i D1::n :Qi
V
^ i D1::o æi i æ0i 2 &
and ˛.Pi / 2 I [ …& for all i D 1::m
and ˛.Qi / 62 …& for all i D 1::n
and ˛.æi / O i ˛.æi / is true for all i D 1::o g

This operator is monotonic and continuous over the lattice structure induced by
the subset ordering on the Herbrand base HBV . Its least fixpoint is lfp.T…& ; & / D
T…!& ; & .;/. It contributes the logical consequences of the predicates defined in … &
6.2 Recursive Spreadsheet Theories 199

to the perfect model of … . If &E D .&0 ; : : : ; &n / is a stratification of … (in particular

its least linear stratification), then, again from Sect. 5.4.2, this set is completed as
follows:
8
< I… &0 D lfp.T…&0 ; &0 /
:I D lfp.T.…&i C1 [ I… &i /; &i C1 /
… &i C1

The perfect model of … is indeed I… D I… &n . If … is definite, it has a

single stratum (unless it contains spreadsheet-induced functional dependencies) and
therefore this construction reduces to the computation of lfp.T…; /.
This characterization of I… was effective for non-recursive theories in that it
implemented a terminating (although inefficient) procedure for computing this set
(the naı̈ve evaluation). Key features were that the number of required fixpoint
iterations was finite, and so was the cardinality of the perfect model I… itself. The
addition of recursive clauses may however have invalidated either or both properties
(and would have, had we not imposed the head constraint restriction). We will now
argue that these properties still hold in the presence of recursion. This means not
only that the above construction still produces the perfect model of a theory in finite
time, but that it can be refined into efficient procedures such as the enhanced semi-
naı̈ve evaluation of Sect. 5.5.

Termination

In Sect. 5.3.4 of the last chapter, we proved that the computation of the least
Herbrand model I… of a non-recursive definite theory … accessed at most a finite
set V… of constants defined as follows (where pji stood for the i -th argument of
predicate p):
S
V… D pji 2 … Vpji
8
<S
p.:::;vi ;:::/ 2 …p fvi g if p is primitive
Vpji D S
: E VXi jB if p is not primitive
.p.X/B/ 2 p
8T
< E 62 B 0
Xi DYik Vq k jik if .Xi D æ.Z//
V V k Ek D k

„ ƒ‚ … :fæ.v
Xi j k q . Y / ^ B 0 E 2 B0
O 1 ; : : : ; vn / W vj 2 VZj jB g if .Xi D æ.Z//
B

An immediate consequence of this fact was that I… was bounded by HBV… , a finite
subset of the Herbrand base HBV . Therefore I… ought to be finite. Furthermore,
only finitely many iterations of T…; were necessary to produce it since the iterative
procedure stopped as soon it had reached a fixpoint. An identical argument applied
in the case of a normal theory.
200 6 The Deductive Spreadsheet

While this construction enjoys the same properties in the recursive case, it
is much more difficult to work with it. Indeed, Vpji may appear within its own
definition for a recursive p. For this reason, we will now downgrade … to a non-
recursive theory … 0 , define an approximation W… 0 (much less precise than V… )
and show that W… 0 is finite and that V…  W… 0 . We begin by defining … 0 .
For every predicate p appearing in … , we define the set rec.p/ as:

rec.p/ D fq W p ! q ! p in CG… g

If rec.p/ ¤ ;, then p is a recursive predicate, and moreover, rec.p/ contains all

the predicates defined by mutual recursion with p (including p itself). In particular,
if q 2 rec.p/, then rec.q/ D rec.p/. Let now rec.p/ be the set of all recursive
clauses that
S define the predicates in rec.p/, leaving out any base clause: clearly,
rec.p/  q2rec.p/ q . Note that every clause Cl in rec.p/ depends on every other
member of this set. In particular, Cl contains at least one element of rec.p/ in its
body and it contains no head constraint. We will now collapse the body of all the
clauses in rec.p/ into a single “body” that we will call Brec.p/ . As we do so, we will
exclude all literals whose predicate symbol are in rec.p/. We also leave out negative
literals and (generic) constraints since they will not play any role in our argument.
Thus, Brec.p/ is defined as follows:
^
Brec.p/ D E / W pi 62 rec.p/g
fpi .X
E
Q^i pi .X/^B 2 rec.p/

We now define one clause for each predicate in rec.p/, and each has Brec.p/ as its
body:
[
0
rec.p/ D E  Brec.p/ g
fq.X/
q 2 rec.p/

Note that none of these clauses is recursive. Finally, we define the clausal theory 0
by removing every recursive clause from and replacing it with p.X E /  Brec.p/
for each recursive predicate p:
[ [
0 D n rec.p/ [ 0
rec.p/
p in p in

Observe that 0 is not recursive. Let … 0 D .…; 0 /. Its set of logical consequence
has little to do with that of … , but the portions of the Herbrand universe they rely
on are related, as we will see shortly.
Given a theory … , we define the set W… as the following over-approximation
of the set of values that can appear in any of its logical consequences:
6.2 Recursive Spreadsheet Theories 201

8 S
ˆ
ˆ W… D Wp
ˆ
ˆ p2…
ˆ
ˆ (
ˆ
ˆ
ˆ
<W fv W v in …p g if p is primitive
p D S
ˆ .P B/2 p WP B
otherwise
ˆ
ˆ
ˆ
ˆ E / in B and Ev 2 .WB /jXE j g
ˆ
ˆ W D WB [ fæ.Ev/ W Y D æ.X
ˆ P B S
:̂
WB D q in B Wq

It is worth comparing V… with W… . At the predicate level, the latter construction

operates on whole predicates rather than on arguments (Wp as opposed to Vpji ).
Within a clause body, it is all inclusive by relying on set union rather than trying to
precisely pick the right values using intersection. Indeed it comes as no surprise
that for any theory … , we have that V…  W… . It is also clear that if … is
non-recursive, then both sets are finite.
We can now bring these two developments together and compare the set V… for
a given (recursive) theory … and the set W… 0 for the (non-recursive) theory … 0
obtained for it. BySa simple inductive argument
S on the acyclic portion of CG… , it is
easy to prove that i VXi jB  WP B and i Vpji  Wp for every predicate p and
clause body B (we need to relate B with Brec.p/ for recursive clauses). An immediate
corollary is then that V…  W… 0 . Since W… 0 is finite, so is V… . Therefore, its
perfect model I… is finite and so is the number of iterations of the T operator
required to compute it.
As a last remark, note that we can derive an upper bound on the cardinality of
W… 0 , and therefore of V… for a theory … D .…; /. Indeed, if con is the number
of values appearing in … or in some head constraint in , hc is the number of head
constraints in , var is the largest number of variables appearing in any clause in ,
and pred is the number of predicates in … , then

jV… j  jW… 0 j  pred .con C hc convar /

In practice, this number often exceeds jV… j by several orders of magnitude.

It should be noted that these properties would not hold if recursive clauses were
allowed to contain head constraints. As we saw, such head constraints could be used
to generate new values at each iteration, effectively preventing termination. The
absence of head constraints in recursive clauses disallows unbounded generation of
values.

Cost

Having established that the fixpoint computation of the perfect model of a recursive
theory terminates, we need to determine how much it costs to produce it. Recall
from Sect. 5.3.4 that in the non-recursive (definite or in-stratum) case, the number
202 6 The Deductive Spreadsheet

of iterations of T…; was bounded by the length jCG… j of the longest path in CG… :
exactly jCG… j C 1 iterations were necessary.
There is no such bound in the presence of recursion. Indeed, the number of
iterations does not depend solely on the clausal structure of a theory … D .…; /
but also on the contents of the factual theory …, in general. Consider the familiar
example of the transitive closure path of a binary relation edge. The form we have
encountered so far (called right-recursive) has the following definition:

path.X; Z/  edge.X; Z/
path.X; Z/  edge.X; Y / ^ path.Y; Z/

Its cost is linear in the length l of the longest path in the object-level graph described
by …edge (excluding loops). Indeed, T…i .;/ will contain all the paths of length up to
i and therefore the computation will stop at T…l .;/. If we interpret as a program
and … as data, as often done for deductive databases, then the number of iterations
depends not only on the program as in the non-recursive case of Chap. 5, but also
on the data. In particular, the seemingly innocuous addition of one edge fact can
significantly alter the number of iterations needed to compute the perfect model of
this theory.
Note that this bound on the cost also depends on the way clauses are entered.
Indeed, the outcome is quite different in the case of the following quadratic (i.e.,
twice-recursive) definition of path, which is logically equivalent to the previous
one:

path.X; Z/  edge.X; Z/
path.X; Z/  path.X; Y / ^ path.Y; Z/

Here, the cost is logarithmic in l. Indeed, T…i .;/ will contain all the paths of length
dlog le
up to 2i and therefore the computation will stop at T… .;/.
Unfortunately, not all recursive definitions can be rearranged to manifest this
property. For example, the same generation problem, defined as

sg.X; X /  node.X /
sg.X; Y /  sg.X 0 ; Y 0 / ^ edge.X 0 ; X / ^ edge.Y 0 ; Y / ^ X ¤ Y

is intrinsically linear. This definition is used as a benchmark in [CGT90] for

comparing the performance of the best known optimization methods for computing
least Herbrand models.
Several approaches have been proposed to determine precisely the asymptotic
run-time complexity of a Datalog program [McA02, GM02]. More recently,
Yanhong Liu and Scott Stoller have outlined a program transformation that
guarantees worst time and space complexity for Datalog programs [LS09].
6.2 Recursive Spreadsheet Theories 203

Semi-naı̈ve Fixpoint Computation for Recursive Clauses

A direct use of the T…& ; & operator described at the beginning of this section
yields a version of the naı̈ve evaluation procedure that computes the perfect
model of a recursive theory … and forms the backbone of the homonymous
spreadsheet evaluation method (discussed below). Like every naı̈ve procedure we
have encountered so far, it is inefficient because each iteration recomputes all the
facts produced by previous iterations, and may add a few more. This drawback is
accentuated in the case of recursive theories because, as we just saw, the number
of iterations needed may be much larger than in the non-recursive case, now a
function of both the factual data and the clausal theory. Therefore, it is more
important than ever to mitigate this overhead. In the past, we refined naı̈ve evaluation
into a semi-naı̈ve method that carefully avoided computing the same values twice
(as well as prematurely trying to compute a value before all of its arguments
were available). This was easily achieved by focusing on the entities (either scalar
formulas, relational expressions, or clauses) all of whose arguments had just become
available. Then, their value was computed in a single step.
This simple approach is suboptimal in the presence of recursion. Indeed, the
evaluation of a recursive predicate p may span several iterations. Within this
interval, our current semi-naı̈ve optimization has no effect: at each such iteration
all previously computed values for p would be recomputed. Note however that
our current approach would still be effective for non-recursive predicates (which
are still evaluated in a single step) and at the boundary of groups of mutually
recursive predicates (the strongly connected components of CG… ). Therefore,
we shall concentrate on recursive predicates and extend our present semi-naı̈ve
procedure with a technique to efficiently calculate their logical consequences. We
will draw inspiration from our previous work on updates and from the abundant
literature on Datalog query optimization [CGT90,GKB87,RSS92], where the notion
of semi-naı̈ve evaluation has been scrutinized for years.2
Let p be a recursively defined predicate in a theory … D .…; / that we will
initially assume to be definite, for simplicity. We define …kp to be the set of logical
consequences of p computed up to the k-th iteration of the T…; operator:

…kp D fp.Ev/ 2 T…;

k
.;/g

Since T…; is monotonic, the next iteration, k C 1, will enlarge it to …kC1

p D …kp [
…kC1
p , where the extension …p
kC1
is called the p-differential of T…; at iteration

2
It should be noted that the appellation semi-naı̈ve evaluation is usually reserved for the technique
we are about to present, which reaps benefits only for recursive definitions, rather than the general
approach we have been developing (which will include this technique as a subroutine). We justify
broadening the use of this terminology by the fact that our previous efforts are in the same spirit as
this better recognized but also more specialized application.
204 6 The Deductive Spreadsheet

k C 1 [CGT90, GKB87]. Therefore, …kC1 p can be efficiently obtained if we have a

kC1
fast way to compute …p —as a matter of fact, we do.
Observe that we have encountered and successfully solved this problem in
Chap. 5 in the general setting of normal theories: indeed, …kC1 p can be viewed
as an update to the previously calculated value of …kp triggered by the addition of
…kp at iteration k. The novelty of recursion does not invalidate any of the results
obtained there. It actually supports a simplified treatment because, by stratification,
:p cannot occur in any clause containing p and therefore this form of update
is strictly positive and applies only to positive literals, a situation exhaustively
analyzed in Sect. 5.6.1.
Adapting the framework introduced there to the present problem, we define
predicates p k and p k to describe …kp and …kp for any k. Then, the above char-
acterization of …kC1
p can be expressed logically as the bi-implication p kC1 .X E/ $
E /_p kC1 .X/.
p k .X E Assuming that the extension …kp of p k has been calculated, we
now want to construct an expression for p kC1 that can be calculated with minimal
effort.
Assume first that p contains a single recursive clause Cl for p and that it
has the form p.X E /  p.YE / ^ B, where B does not contain p or any predicate
mutually recursive with p. Recursive clauses with these characteristics are called
linear. Unfolding the recursion, we pinpoint the role of Cl in …kC1
p by writing it as:

E /  p k .YE / ^ B
p kC1 .X

By instantiating the above characterization to p k , we can rewrite this clause as the

formula
 
E  p k1 .YE / _ p k .YE / ^ B
p kC1 .X/

which can be transformed into the following equivalent clauses:

E /  p k1 .YE / ^ B
p kC1 .X
p .X /  p k .YE / ^ B
kC1 E

The first clause has the same body as Cl and therefore represents the contribution
of this clause to …kC1 k
p . In particular it is already accounted for in …p . The second
clause describes the additions that Cl contributes to …kC1
p , which in general are not
represented in …kp . We can take this as the definition of the predicate p kC1 . We
capture this by the clause

E /  p k .YE / ^ B
p kC1 .X
6.2 Recursive Spreadsheet Theories 205

This reasoning is schematically visualized in the following diagram, adapted from

Sect. 5.6.1:

Πkp pk−1 (X)

pk+1 (X)

ΔΠk+1
p Δpk (X)

E/
If p’s definition relies on a set rec.p/ of several recursive clauses Cli D p.X
p.YE / ^ Bi for i D 1::n, and all are linear, we can repeat the above argument,
obtaining the following characterization for p kC1 :

E /  p k .YE / ^ B1
p kC1 .X
:::
p .X /  p k .YE / ^ Bn
kC1 E

Observe that these clauses are isomorphic to the original recursive clauses of p:
indeed, if we drop the superscripts and the ’s, they are precisely the elements of
rec.p/ . This observation supports an alternate expression of …kC1 p which does not
rely on any extension to the original set of predicates in … . Indeed, …kC1p can be
obtained by a single application of the clauses in rec.p/ to just the objects in …kp .
As we do so, we must be careful to provide all the facts needed by the body of these
clauses. This idea is formalized in the following expression

…kC1
p D T.…k n…kp /; rec.p/ .…kp /

where …k abbreviates T…; k

.;/. The supplied factual theory, …k n …kp , contains all
the facts calculated up to iteration k, except those about p, a selection of which is
supplied as the argument …kp .
The argument developed so far also applies if p participates in a mutual recursion
and all clauses in rec.p/ are linear. Our last characterization is then updated to

rec.p/ D T.…k n…k

…kC1
rec.p/ /; rec.p/
.…krec.p/ /

for the obvious meaning of the symbols.

Linear recursive clauses are extremely common and have the unique characteris-
tic that their differential at any step k C 1 can be calculated by running them on the
206 6 The Deductive Spreadsheet

differential at step k. This supports a very efficient incremental fixpoint computation

of the perfect model of a theory … . However, not all recursive clauses are linear.
This simple method does not apply when two or more occurrences of the head
predicate of a clause occur in its body. The differential can still be computed, but not
as easily and with lesser efficiency gains. The details of how to do so can be adapted
from Sect. 5.6.1. We only report the clauses defining p kC1 for rec.p/ consists of a
single clause Cl of the form p.X E /  p.YE / ^ p.Z/E ^ B:

E  p k .YE / ^ p k .Z/
p kC1 .X/ E ^B
kC1 E k E k E
p .X/  p .Y / ^ p .Z/ ^ B
E  p k .YE / ^ p k .Z/
p kC1 .X/ E ^B

Here the computation of …kC1 p relies not only on …kp but also on …kp ; hence the
decrease in efficiency (note however that it does not amount to recomputing …kp as
the naı̈ve approach would do). Note also that these three clauses cannot be construed
as an instance of Cl.

Evaluation of the Deductive Spreadsheet

As anticipated, we have seen that general theories enjoy the exact same model-
theoretic properties as their non-recursive counterparts examined in Chap. 5. There-
fore, we can follow the path cut-out there in order to turn this foundational work into
viable procedures for evaluating a deductive spreadsheet s. Indeed, simply adapting
the fixpoint computation to operate relative to an environment (which may contain
undefined values) yields the naı̈ve evaluation procedure ER s exactly as in Sect. 5.5.
It can be refined into a semi-naı̈ve procedure EP s using the approach discussed in
Chaps. 3 and 4, extended to handle recursion. In spite of the these efforts, this
results only in a slightly better performance for the reasons discussed in Sect. 5.5. A
significant improvement is however achieved by breaking the theory … s of s into
functionally independent portions and evaluating them separately as needed. This
yields the counterpart of the enhanced naı̈ve and semi-naı̈ve procedures, ER s0 and EP s0 ,
respectively, examined in Sect. 5.5. This last approach can be further enhanced by
enacting the optimizations we just developed.

6.2.3 Bottom-Up Derivation and Updates

As with model theory, the basic tenets of the proof theory of recursive clauses do not
change from what we obtained in Chap. 5. Indeed, the general notion of derivability,
defined in Sect. 5.4.3 through the judgment … ` Bi for a ground body element
Bi remains unaltered. However, adding recursion has subtle implications on the
applicability of proof-theoretic techniques to implement spreadsheet functionalities.
6.2 Recursive Spreadsheet Theories 207

We will now begin to examine them, starting with the bottom-up flavor of derivation
and updates. Top-down derivability and its application to explanation are the subject
of Sect. 6.2.4.

Bottom-Up Derivation

Given a definite theory … D .…; /, bottom-up derivability, which relied on

judgments of the form … ` p.Ev/, emulated the fixpoint computation using proof-
theoretic instruments. It started from the facts in … and repeatedly applied the rules
in to them to produce new derived facts, until saturation, which happened in
a finite time. This approach was easily upgraded to stratified theories. The intro-
duction of recursion does not change the basic mode of operation. The argument
presented in the previous section ensures the termination of this procedure.

Updates in the Deductive Spreadsheet

In Sect. 5.6 of the previous chapter, we described a number of efficient update

procedures that could be realized either by bottom-up derivation or by fixpoint
computation. Recall that, given a clause p.X E /  q.YE / ^ B for which the extension
of the predicate q was available, we defined clauses propagating a positive or
negative update to q, expressed as predicates q and ıq respectively, to p. Positive
updates to p were given by clauses defining the auxiliary predicate p in terms of
q, q and ıq, and similarly for the negative updates which were rendered through
the auxiliary predicate ıp.
As previewed in the last section, these update procedures adapt very smoothly
to the recursive case. Indeed, the arguments underlying recursion-aware semi-naı̈ve
evaluation presented at the end of Sect. 6.2.2 can be applied without any modifi-
cation to handle positive updates to positive literals. Because of the requirements
imposed by stratification, the same can be said about positive updates to negative
literals. It should be noted that, differently from the situation analyzed in Sect. 6.2.2,
a typical update will touch several predicates, possibly not mutually recursive.
Therefore non-linear situations tend to be more frequent than when incrementally
calculating recursive predicates. Note that positive updates have received a lot of
attention in the literature, especially as studies of semi-naı̈ve evaluation strategies:
see [CGT90, GKB87, RSS92] for a summary and further pointers to detailed papers
about each approach.
Negative updates are not nearly as popular, and indeed only recently have
they received any attention: [NJLS11] explores positive and negative updates in a
distributed setting, and [LC12] translates both forms into linear logic. Furthermore,
the procedures described in Sect. 5.6.2 do apply in the presence of recursion. The
practical effectiveness of any of these approaches needs however to be assessed
empirically.
208 6 The Deductive Spreadsheet

Fig. 6.6 Effective top-down derivability

6.2.4 Top-Down Derivation and Explanation

The last of the theoretical aspects studied in Chap. 5 that we need to discuss in
the presence of recursion is top-down derivability. In Sect. 5.4.3, we expressed this
notion by means of two judgments,

… ` B 7! ˛ and … 6` B

The first described the fact that a derivation could be constructed for the query B
in … , with the (possibly partial) substitution ˛ constraining the values of variables
appearing in it. The second, needed for handling negative literals, formalized the fact
that the ground query B was not derivable in … . Inference rules for both judgments
are reported in Fig. 6.6 from Sect. 5.4.3.

Non-terminating Tree Constructions

Extending our clausal language with recursion does not per se affect the theoretical
validity of that development. Indeed, neither the form of these judgments nor the
rules themselves require any change. In practice, the situation is quite different as a
simple-minded application of these rules can result in the attempted construction of
infinite derivation trees. Consider for example the standard right-recursive definition
of the path predicate:

path.X; Z/  edge.X; Z/
path.X; Z/  edge.X; Y / ^ path.Y; Z/
6.2 Recursive Spreadsheet Theories 209

.1/
Let the extension of the edge predicate initially be …edge D fedge.a; b/g and let
… .1/ be the theory consisting of those two clauses and this fact. Then, the rules
in Fig. 6.6 will return a first answer to the query … .1/ ` path.X; Y / 7! ˛ by
using rule np" followed by pr" on the first clause and producing the substitution
˛ D fX 7! a; Y 7! bg. Attempts at constructing additional answers will fail
immediately: an initial use of the second clause will invoke rule bd", whose first
subgoal, edge.X; Y /, will succeed with substitution fX 7! a; Y 7! bg, but which
will then expose the subgoal path.b; Z/, which fails right away.
Here, we adopted the depth-first left-to-right subgoal reduction strategy implicit
in rule bd". The outcome could however have been quite different had we relied on
another strategy. Consider depth-first right-to-left reduction, which can be emulated
with the rules in Fig. 6.6 by reversing the order of the literals in the body of the
second clause for path: then, once the initial answer has been produced, this strategy
would start constructing an infinitely high tree by inconclusively expanding the
subgoal path.Y; Z/, interspersing any new step up with a failed attempt at using
rule pr".
The expensive breadth-first strategy would overcome this impasse. This solution
does not solve the general problem, however: consider the two rules above with the
.2/
factual theory …edge D fedge.a; a/g. Then any of the strategies examined so far will
embark into an infinite tree construction after producing the initial answer.
Here, the blame can be put on the presence of a cycle in the graph represented by
the edge predicate. Similar behaviors are however also manifested in the absence
of cycles. Consider the quadratic definition of the path predicate encountered
earlier,

path.X; Z/  edge.X; Z/
path.X; Z/  path.X; Y / ^ path.Y; Z/

.1/
and the factual theory …edge . The same behavior emerges again: after the initial
answer is produced, any of the above strategies starts building an infinite tree.
All of the above behaviors returned the correct answer before diverging in search
of alternative derivations. This is because we were wise enough to always attempt
using the base rule for path first. Had we given the recursive rule higher priority, no
answer would be reported whatsoever.
Divergence is a problem even if all we are interested in is the first answer to
a query. Indeed, if this query has no solution, then only the divergent behavior is
manifested. This is particularly unsettling in the case of negative queries, which are
often expected not to have any solution. Negative queries are either entered by the
user or result from using rule nl" to assess the derivability of a negative literal.
All of these problems are familiar to Prolog programmers, who have learned
to cope with them by carefully ordering clauses and their bodies, by inserting
appropriate checks, and by relying on extra-logical devices to control execution.
A Prolog programmer would not see any problem with the rules in Fig. 6.6 because
she has the skills to work around potential pitfalls.
210 6 The Deductive Spreadsheet

This is clearly not the case of the expected user of the deductive spreadsheet:
an individual with no formal training in either logic or computer science, in
general. Any form of divergence (and even slow convergence) is unwelcome news.
Therefore, it is crucial to devise top-down derivation strategies that do not diverge in
any circumstance. In particular, finite derivability should be ensured independently
of the contents of the factual base … of the given recursive theory … , independently
of the ordering of clauses in , and independently of the order of the conjuncts in
the body of any clause.

Terminating Tree Constructions

Clearly, this objective must be achievable since the perfect model of any recursive
theory is computable in finite time, as we have seen in Sect. 6.2.2. We will now
briefly examine two methods to ensure the termination of top-down derivability.
The first, pioneered by David Smith, Michael Genesereth and Matthew Ginsberg
in [SGG86], observes that the height of the derivation tree of any answer to a given
query B in a theory … is bounded by a function h.… ; B/ that depends only on
… and B: there may be larger derivations, but their answers would duplicate those
of smaller trees. This observation can immediately be used to fend off the threat of
divergence. Given the query … ` B 7! ˛, we build derivation trees as described in
Sects. 5.4.3 and 5.7, but discard any tree as soon as it grows beyond height h.… ; B/.
Complete trees smaller than this bound are returned. Notice that this approach works
with both the breadth-first and depth-first strategies. Notice also that it handles both
positive and negative queries. The practicality of this approach depends the value of
h.… ; B/. The value pred .conChc convar / obtained in Sect. 6.2.2 can be used as an
upper bound on the number of applications of rule np", and therefore determines the
maximum value of h.… ; B/. As we said, that bound is often astronomical and often
bears little resemblance the actual figure for h.… ; B/. More accurate values can be
found in [SGG86], which also discusses dynamically generated halting conditions
for when to stop expanding a branch of the derivation tree. These approaches can
be easily adapted to the specific language used by the deductive spreadsheet. A
thorough investigation is beyond the scope of this book.
The second approach, proposed by David S. Warren in the XSB system [War98],
is radically different, and indeed embeds aspects of bottom-up derivability. The
basic idea is that each time a new answer for an atomic goal P involving a clausally
defined predicate p is produced, it is cached (tabled in XSB terminology), and each
time P is encountered again, previously computed answers are returned rather than
the subtree being rebuilt from scratch. If p is recursive, the discovery of a new
answer for P may trigger further requests which will eventually lead to new unique
answers. The computation stops when no new answer is produced.
Going into more detail, consider the judgment … ` P 7! ˛, where the predicate
p in P is clausally defined and some arguments may be constant while others
may be variable. Resolving P against the defining clauses of p (by the equivalent
of rule np") will yield a number of subproblems of the form … ` Bi 7! ˛i .
6.2 Recursive Spreadsheet Theories 211

Those Bi ’s consisting uniquely of primitive predicates and constraints can be used

to compute answers to P right away. Applying rule bd" to the others will yield
atomic subproblems … ` Pj 7! ˛j . Some of these will be new, and rather than
extending the derivation for P , we will start new derivation trees for them. Others
will already have been encountered: in this case available answers for them will be
returned to the tree for P . A derivation in this setting consists therefore of a number
of server trees, each with a different partial predicate instantiation Pi as its root,
and most with open hypotheses, or client leaf nodes, about some Pj . Whenever
a server P produces a new answer, it communicates it to every occurrence of P
as a client node in other servers. If P 0 is such a server, this new answer for P
may yield a new answer for P 0 which will be fed in every homonymous client
node. The procedure terminates when the system stabilizes and no server produces
new answers. Termination in our language in ensured by the fact that the number
of answers for any P is theoretically bounded by the fixpoint characterization in
Sect. 6.2.2. In practice, this approach is quite efficient, and therefore an excellent
candidate for implementing top-down derivation in the deductive spreadsheet.

Explanation in the Deductive Spreadsheet

As a conclusion of this section, we recall that top-down derivation was the approach
of choice for supporting explanation in the logical spreadsheet, as discussed in
Sect. 5.7. It remains so in the deductive spreadsheet, which makes termination a
paramount property. The tabled method [War98] just discussed appears to be a
good choice for this purpose. Indeed, it is efficient and can easily be adapted to our
interactive explanation needs. Experimental validation is however necessary. The
first approach above [SGG86] seems to suffer from efficiency problems, but further
investigation is needed in order to make a definitive assessment.

6.2.5 Historical Perspective

The above section concludes the core development of the logical infrastruc-
ture underlying the deductive spreadsheet (we will describe useful extensions in
Sects. 6.3 and 6.5). This is therefore a good point to comment on where the language
we have developed stands in the logical spectrum. We will now collect the many
references that peppered the last two chapters into a coherent account.
The fragment of first-order logic we have been working with is closely related to
a number of languages that altogether have fueled the field of logic programming
for the past 50 years. The computational value of logic was first recognized in
the field of linguistics [Col70, Gre69] in the late 1960s. The enabling technology
was the recent discovery of resolution [Rob65] in the spheres of theorem proving.
The specialized language prototype used in these efforts would soon evolve into
Prolog [CM87, O’K90, SS94], a general-purpose programming language that still
212 6 The Deductive Spreadsheet

dominates the logic programming scene. The logical core of Prolog differs from our
language in three respects:
1. It draws its terms from a freely generated language containing not only constants
and variables, like ours, but also uninterpreted function symbols. Therefore, a
predicate can have the form p.f .X /; g.f .a/; Y // rather than just p.X; a; Y /.
Including this possibility in our language leads to two problems: first and
foremost is non-termination; second, it clashes with the traditional design of
a spreadsheet, in which cells evaluate to atomic values rather than structured
objects. We will come back to this aspect in Sects. 6.3 and 6.5.
2. It does not rely on any safety restrictions.
3. It does not admit constraints. However, actual Prolog implementations do make
available “extra-logical constructs” that are akin to our constraints.
The theory of core logic programming blossomed in the 1970s and 1980s with
precise accounts of its model semantics [EK76, Kow79], its proof-theoretic seman-
tics [AvE82, Llo87, MNPS91], and especially the semantics of negation [Cla78,
Prz87, Har94].
The early 1980s saw the development of Datalog and the notion of deductive
database [GM78, Min87, CGT90, Col98]. It was observed that the fragment of core
logic programming deprived of functional symbols and recursion (and extended
with safety restrictions) had the same expressive power as relational algebra.
Adding recursion back extended this expressiveness without compromising effec-
tive computability. Negation always proved problematic in this picture, although
stratification [CH85, ABW87, CGT90] was eventually recognized by most authors
as the best way to handle it.
Constraints were prominently incorporated as first-class citizens of logic pro-
gramming in the late 1980s with the proposal of constraint logic program-
ming [JL87, JM94]. This popular extension of “traditional” logic programming
yielded a family of language known as CLP(D), where D represents a domain
(for example the real numbers), a language of constraints on them (e.g., linear
inequalities over the reals), and a constraint solver for them (e.g., the simplex
algorithm). Constraint logic programming proved very effective in tackling a
number of practical problems [JM94]. The language we have developed in this book
can be viewed as a form of CLP based on a term language deprived of function
symbols and whose constraints are inherited from the underlying spreadsheets.
Figure 6.7 summarizes the relationships among these flavors of logic program-
ming and our proposal. The fragment of first-order logic carved out in this book
can indeed be seen as an extension of Datalog with constraints, or a function-free
specialization of CLP. Traditional logic programming stands at the antipodes of
these two dimensions. Versions of Datalog enriched with constraints are not new,
and have indeed been extensively studied in relation to the notion of constrained
database [KKR95, KLP00] and more recently trust management [LM03]. Con-
straints in constraint logic programming and constraint Datalog play an important
role in the representation of data and are therefore much more general than the
purely evaluative use we make of them.
6.2 Recursive Spreadsheet Theories 213

Constraint
Logic Programming
CLP (D) [JL87; JM94]

− Constraints − Functions

Traditional Constraint Datalog

Logic Programming This Work
Prolog [SS94; O’K90] Also [KLP00, LM03]

− Functions − Constraints

Datalog
[CGT90]

Fig. 6.7 The language of the deductive spreadsheet within logic programming

6.2.6 Relational Counterpart and Relative Expressiveness

We conclude this section by looking back at relational algebra and describing what
extension could capture the added expressive power of recursive clauses. We also
report on the relative computational expressiveness of the various languages we have
examined.

Recursive Relational Definitions

In Chap. 4, we have introduced relational expressions, eN for example, and used

them to populate some of the reference arrays El in the relational spreadsheet. We
interpreted the result of evaluating eN as a relation p. We chose to take a spreadsheet-
oriented perspective when defining these concepts by emphasizing the binding
between El and e.N Alternatively, we could have stressed the relationship between
eN and p by observing that, to all effects, eN acts as the definition of p. We can express
this correspondence as:

p WD eN

Under this interpretation, a relational spreadsheet s extends the basic scalar and
array infrastructure with a set of relational definitions of the above form, one
for each relational expression appearing in s. Since the relational dependency
graph of s was required to be acyclic in Chap. 4, these definitions could not be
recursive.
Once we describe the relational extension of the spreadsheet in terms of relational
definitions, lifting the ban on recursion is a natural avenue to explore, and one that
214 6 The Deductive Spreadsheet

CourseDep = Prereq ∪ ¼Prereq.1,CourseDep.2 (Prereq 2=1 CourseDep)

CouldTakeIf := ¼Student.3,CourseDep (1) .1,CourseDep (1) .2 CourseDep (1)
CourseDep (1) .2=CourseDep (2) .2
(¼Student.3,CourseDep (2) .2 (Student × CourseDep (2) ) − Took )
MissingReq := ¼CouldTakeIf .1,CouldTakeIf .2 CouldTakeIf
MaybeHarder = (¼Prereq (1) .1,Course (1) .2 Prereq (1) ×
(¼Course (1) .2,Course (2) .2 (Course (1) × Course (2) ) − Prereq (2) )) ∪
(¼Prereq (1) .1,Prereq (2) .1 (Prereq (1) Prereq (1) .2=MaybeHarder .1 MaybeHarder
(2)
MaybeHarder .2=Prereq (2) .2 Prereq ))
Easier := (¼Course (1) .2,Course (2) .2 (Course (1) × Course (2) )) − MaybeHarder
NeededCourse = CourseForDegree ∪
¼Prereq.2,NeededCourse.2 (NeededCourse NeededCourse.1=Prereq.1 Prereq)
CourseToDegree := ¼Student.3,NeededCourse (1) .2,NeededCourse (1) .1
NeededCourse (1) NeededCourse (1) .1=NeededCourse (2) .1
(¼Student.3,NeededCourse (2) .1 (Student × NeededCourse (2) ) − Took )
1 := ¼Student.3,CourseToDegree.2 Student × CourseToDegree
2 := ¼CourseToDegree.3 ¾CourseToDegree.1=DX3:DX ,CourseToDegree.2=DY 3:DY
CourseToDegree

Fig. 6.8 Relational definitions for the registrar’s example

leads to the logical language we have designed so far. Indeed, a predicate p defined
by simple recursion corresponds to a relational equation of the form

p D e.p/
N

and any mutually recursive definition of a set of predicates fp1 ; : : : ; pn g can be

simulated by a system of relational equations of the form

p1 D eN1 .p1 ; : : : ; pn /

pn D eNn .p1 ; : : : ; pn /

The head constraint restriction bans the use of generalized projections within a
recursive definition. As such, it adds to the safety restrictions natively present
in relational algebra, as noted in Chap. 5. Instead, the negative literal restriction
corresponds to preventing a recursively defined relation p from occurring on the
right of a relational difference operator in its own definition. It should be noted that
it is an artifact from a purely algebraic point of view, although it yields the same
computational benefits as in our logical language.
For example, the relational definitions in Fig. 6.8 capture the fragment of our case
study examined in Sect. 6.1. For conciseness, we have used positional projection
and selection indices rather than attribute symbols. Note that the definitions for
CourseDep, MaybeHarder and NeededCourse are recursive.
6.2 Recursive Spreadsheet Theories 215

æ,rec
RA+ RAæ
+ RA+

Logrec,¬ Logrec,¬
æ Log¬
æ

Log æ Log æ,hc Log æ,hc,list

RA RAæ RAæ,rec

Logrec Logrec
æ

RA+ : Positive relational algebra [sec. 4.3.2, p. 61 ]

æ
RA+ : Positive relational algebra with attribute expressions [sec. 4.3.2, p. 66 ]
æ,rec
RA+ : Positive relational algebra with attribute expressions and recursion [sec. 6.2.6, p. 165]
RA : Relational algebra [sec. 4.3.2, p. 61 ]
RAæ : Relational algebra with attribute expressions [sec. 4.3.2, p. 66 ]
RAæ , rec : Relational algebra with attribute expressions and recursion [sec. 6.2.6, p. 165]
Log rec,¬ : Datalog without recursion nor negation [sec. 5.3, p. 111]
æ
Log rec,¬ : Constraint Datalog without recursion nor negation [sec. 5.3, p. 111]
æ
Log ¬ : Constraint Datalog without negation [sec. 6.2, p. 149]
Log rec : Stratified Datalog without recursion [sec. 5.4, p. 122]
æ
Log rec : Stratified constraint Datalog without recursion [sec. 5.4, p. 122]
Log æ : Stratified constraint Datalog [sec. 6.2, p. 149]
Log æ ,hc : Stratified constraint Datalog with head constraints [sec. 6.3.1, p. 169]
æ ,hc,list
Log : Stratified constraint Datalog with head constraints and lists [sec. 6.3.2, p. 172]

Fig. 6.9 Relative expressive power of the languages examined

Expressive Power

Since our initial extension of the traditional spreadsheet with relational expressions,
we have examined a number of languages, both relational and logical. We will now
discuss their relative expressive power, including that of interesting sublanguages.
Our results are summarized in Fig. 6.9. For simplicity, we did not account for the fact
that these languages are meant to be embedded within a spreadsheet in our context:
doing so has the effect of boosting their expressiveness by pairing them up with
traditional functional formulas, thanks to the availability of functional dependencies
(as we have seen in Sect. 6.1.3). This more refined analysis will be the subject of
future work. The spreadsheet-embedded variant of each language is expected to
have more expressive power than this language taken in isolation. It is also expected
that the relative strength of the embedded languages will not vary substantially, at
most merging currently distinct classes.
Each area in the upper part of Fig. 6.9 represents the class of problems that
the language that labels it can solve. Acronyms are explained and referenced in
216 6 The Deductive Spreadsheet

the bottom part of this figure. For example, the box surrounding RA expresses
all the problems that can be stated using traditional relational algebra. Formalism
A is more expressive than formalism B if it can solve more problems, which is
rendered as the area for A enclosing the area for B. Thus relational algebra (RA)
is strictly more expressive than positive relational algebra (RAC —i.e., RA deprived
of the difference operator). Two formalisms, e.g., relational algebra and our logical
formalism deprived of recursion (Logrec ), have the same expressiveness if they can
solve the same problems, which we rendered as an area enclosed by a double line.
Clearly, two languages may have incomparable expressiveness: for example each
of RA and RAæ C , positive relational algebra extended with attribute expressions,
can solve problems that the other cannot. Finally, the areas delimited by dotted
lines indicate formalisms that we have mentioned, but not yet fully examined. For
conciseness, we refer to the variants of our logical formalism relative to Datalog, as
discussed in Sect. 6.2.5.
We will now comment on the relations displayed in Fig. 6.9, most of which are
standard results in the literature (see for example [CGT90]). First notice that for the
most part, each relational language has a corresponding logical formalism with the
same expressive power. Indeed, corresponding features (e.g., attribute expressions,
recursion) yield similar expressiveness. The one exception is negation/difference:
until recursion is considered, relational languages with difference solve the same
classes of problems as the corresponding logical languages with negation. Recursion
changes this state of affairs because of our requirement that normal theories be
stratified. Indeed, there are recursive relational definitions that do not correspond
to any stratified theory. Had we allowed non-stratified theories (and accepted the
performance penalty), the two languages would have the same expressive power.
Note that in the absence of recursion, a normal theory is necessarily stratified
(although functional dependencies may alter this). See [CGT90] for an in-depth
discussion of expressiveness in the presence of negation.
Moving now left to right, it is clear that allowing attribute expressions (con-
straints in the logical setting) furthers expressiveness. Note however that the
underlying functional language of a spreadsheet can completely bridge the gap
in this case, as we saw in Sect. 4.3.2. Recursion does provide a real extension of
expressive power, even when considering the full spreadsheet context. Indeed, we
have seen several examples of this in Sect. 6.1, and more will be presented shortly.
As we saw, the head constraint restriction significantly limits the class of
problems that can be solved in the deductive spreadsheet, but also safeguards
termination. We will discuss in Sect. 6.3.1 an approach to partially lifting this
restriction and yet retaining termination. This will allow us to approach the added
expressiveness of fully permitting head constraints in recursive clauses.3 Finally,

3
To be fully precise, the theoretical expressiveness will remain the same as in our current language,
but in the majority of circumstances it will provide the illusion of a language with unrestricted head
constraints.
6.3 Bounded Termination 217

a further extension of the resulting language with lists will significantly extend the
expressive power. This will be examined shortly.

6.3 Bounded Termination

As we saw in the last two sections, recursion is a powerful mechanism that

substantially extends the expressive power of the logical language of Chap. 5. With
this enhanced expressiveness comes the concrete risk of divergence, which we
preempted by disallowing head constraints in recursive clauses. However, we also
saw that some natural problems, such as determining the length of a path in a graph,
inherently require head constraints and that many useful instances are necessarily
terminating. Problems of this type are extremely common, so common that we
believe our design would be lacking in a practically important aspect if it were not
possible to model them. Re-admitting head constraints in full is clearly out of the
question because some theories would be non-terminating, as we saw in Sect. 6.2.1.
Admitting only clauses that necessarily terminate is theoretically infeasible because
distinguishing them from non-terminating theories is undecidable.
In this section, we will explore a compromise solution that gives the illusion of
allowing recursive clauses to feature arbitrary head constraints, but in fact limits the
number of recursions they can participate in through a user defined bound. This is
the subject of Sect. 6.3.1. The same technology is applied in Sect. 6.3.2 to extend the
expressive power of our language by allowing lists as predicate arguments. Lists, or
any other recursively defined object type, are another source of divergence in logic
programming languages. However, as we saw in Sect. 6.1.4, they are also critical to
modeling another class of useful problems, such as displaying the nodes appearing
in a path. We conclude in Sect. 6.3.3 with a brief outline of a second approach to
support some types of head constraints.

6.3.1 Head Constraints in Recursive Clauses

In Sect. 6.2.1, we saw that an indiscriminate use of head constraints in recursive

clauses immediately leads to non-termination. For the sake of novelty, let us rework
our graph example as the problem of calculating the number of miles flown as part
of an itinerary. If the primitive predicate hop.F; T; D/ lists the distance D between
two airports F and T , then this problem is solved by the following instance of the
transitive closure problem:

trip.F; T; D/  hop.F; T; D/
trip.F; T; D/  hop.F; M; D 0 / ^ trip.M; T; D 00 /
^ D D D 0 C D 00
218 6 The Deductive Spreadsheet

Of course, since airlines have return routes, these two clauses are bound to
divergence as any calculation will eventually start going back and forth between
already visited cities. A crude way to avoid non-termination is to put a bound
on the maximum distance that can be flown: let us place it at 24,900 miles—the
circumference of the earth (we will describe a more sensible solution in Sect. 6.3.2):

trip0 .F; T; D/  hop.F; T; D/

trip0 .F; T; D/  hop.F; M; D 0 / ^ trip0 .M; T; D 00 /
^ D D D 0 C D 00 ^ D  24;900

The second clause contains the head constraint D D D 0 C D 00 but also the expres-
sion D  24;900 which effectively bounds D. The bounding action of this constraint
is evident in this example. This is not always the case. Consider the following variant
of the above definition, which an airline may use for pricing an itinerary:

trip00 .F; T; D/  hop.F; T; D/

trip00 .F; T; D/  hop.F; M; D 0 / ^ trip00 .M; T; D 00 /
^ D D min.D 0 C D 00 ; 24;900/

It considers the actual distance of any trip shorter than 24,900 miles, but assigns the
blanket value 24,900 to any longer trip. Surprising as it may be, computation always
terminate in this theory. Indeed, the maximum value of D appearing in any inferred
fact trip.F; T; D/ is 24,900 and since there are finitely many airports, only a finite
number of facts of this form will be produced.
It is in general impossible to automatically distinguish between terminating and
non-terminating recursive theories with head constraints. Indeed, an algorithmic
solution to this problem would imply a solution of the halting problem, a classical
benchmark of undecidability.
We will go down a different path: we will transform the defining clauses of any
recursive predicate that makes use of head constraints by superimposing on them
a very simple instance of this same problem, but one that is known to always
terminate. This will ensure termination if the original theory … was subject to
divergence, but impose only a negligible penalty otherwise. This transformation will
affect only recursive predicates containing a head constraint in one of their defining
clauses. It will however not touch non-recursive definitions, and recursive predicates
which do not rely on head constraints—therefore all safe theories of Sect. 6.2.1
remain unaffected. This transformation will be transparent to the user, who will
simply specify … and not the transformed theory. The user will however be able
to influence its run-time behavior by changing a single parameter (the value hcmax ,
defined below).
Before describing the transformation in its generality, we consider a simple
instance. Let p be a predicate defined by simple recursion. Assume that p contains
two clauses: the base clause Clb D p.X E /  Bb and the recursive clause Clr D
p.X E /  p.YE / ^ Br . Furthermore, assume that Clr mentions one or more head
constraints within Br and that it is linear in p (i.e., no instance of p occurs in Br ).
6.3 Bounded Termination 219

Then, we transform p into the predicate p  and replace p with the clausal theory
p defined as follows:

Clb W E ; C /  Bb ^ C D 0
p  .X
Clr W E ; C /  Br ^ p  .YE ; C 0 / ^ C D C 0 C 1 ^ C  hcmax
p  .X

where the effects of the transformation have been highlighted using boxes. The
predicate p  differs from p in that it takes one additional argument: the counter C .
The transformed base clause Clb in the top row is identical to Clb except that it sets
C to zero. The recursive clause Clr increments C by 1, but also tests it against
the user-defined global bound hcmax . Therefore, the application of this rule will
fail as soon as C exceeds this value. It is clear that the calculation of the logical
consequences of p  will terminate since the recursion cannot unfold more than
hcmax times along any execution path.
We can extend this technique to handle arbitrary forms of recursion, including
non-linear and mutually recursive definitions. Let … be a recursive theory and p
be a recursive predicate containing one or more head constraints Xi D æi in one of
its defining recursive clauses. Recall that p is the set of clauses of p. Let rec.p/
be the set of predicates defined by mutual recursion with p, and rec.p/ be the set of
all mutually recursive clauses involving predicates in rec.p/. Then the transformed
theory …  is defined as follows:
E /  B, we set:
• For each base clause Cl 2 p n rec.p/ with Cl D p.X

E ; C /  B ^ C D0
Cl D p  .X

Base clauses are massaged as in the above instance by having them set the counter
C to 0.
• For each recursive clause Cl 2 rec.p/ \ p without head constraints, if Cl has
V
E/ Ek
the form p.X qk 2rec.p/ qk .Y / ^ B, we transform it into:
^
E C/ 
Cl D p  .X; qk .YE k ; Ck / ^ B ^ C D max Ck
k
qk 2rec.p/

This is a situation that did not appear in our example above. The safe recursive
clauses of p simply propagate the value of the counter appearing in their body.
If this clause is non-linear, its body will contain more than one instance of p or
of a predicate mutually recursive with p. We then need to bring their counters to
a common denominator, which will be the one containing the maximum value.
Note that there is no point in inserting a test against hcmax .
• For each recursive clause Cl 2 rec.p/ \ p featuring head constraints Xi D æi ,
V V
if Cl D p.X E /  i Xi D æi ^ q 2rec.p/ qk .YE k / ^ B, we have:
k

^ ^
E ;C/ 
Cl D p  .X Xi D æ i ^ qk .YE k ; Ck / ^ B
i qk 2rec.p/
^ C D .maxk Ck / C 1 ^ C  hcmax
220 6 The Deductive Spreadsheet

This situation extends the recursive case in our example with non-linearity
(achieved by taking the maximum of all body counters) and mutual recursion.
The head counter is incremented by 1 to support progress. It is also tested against
the constant hcmax to ensure termination.
• For each such predicate p, add the clause

E /  p  .X
Clp D p.X E; /

which simply reduces the evaluation of p to that of p  after throwing away

the counter C . This step ensures that clauses that depend on p in a non-
recursive manner do not unduly need to be altered. It also insulates and seals the
mechanism we just described over the clauses defining rec.p/, not those using
the auxiliary predicates.
• Any clause Cl for a predicate q whose underlying theory q is either non-
recursive or does not mention head constraints in recursive clauses (even through
mutual recursion) remains unchanged: Cl D Cl.
We need to stress again that the user only specifies … , not …  . Furthermore, the
user never sees the auxiliary predicate p  , but only p. Similarly, non-recursive
clauses depending on p do not need to be aware of the use of p  . Indeed,
this transformation operates at the level of strongly connected components of
recursive predicates with head constraints. Therefore, two such components will
be extended with independent counters. Note also that such counters never cross
strata boundaries.
As a final note, there are common situations where a theory … contains recursive
clauses featuring head predicates that can be trivially recognized as terminating. The
clauses for trip0 at the beginning of this section are a clear example. It is conceivable
that one can construct a test to flag these very simple cases as terminating, hence
allowing us to optimize away our transformation. This approach would not be
appropriate for more complex situations as the overhead of their testing procedures
may not warrant the time saved during evaluation.
We will now demonstrate the extended range of admissible logical theories by
extending our ongoing examples from Sect. 6.1. Consider the problem, introduced
but not solved in Sect. 6.1.4, of calculating the minimum number of semesters worth
of coursework a student has to go through before obtaining a certain degree. We are
assuming that a course can be taken only if all its prerequisites have been met.
We begin by modifying the predicates NeededCourse and CourseToDegree of
Sect. 6.1.3 to carry the number of courses needed to obtain a degree.

nNeededCourse.C; D; N /  CourseForDegree.C; D/ ^ N D 1
nNeededCourse.C; D; N /  nNeededCourse.C 0 ; D; N 0 / ^ Prereq.C 0 ; C /
^ N D N0 C 1
nCourses.S; D; N /  Student. ; ; S / ^ nNeededCourse.C; D; N /
^ :Took.S; C /
6.3 Bounded Termination 221

These clauses differ from the ones in Fig. 6.3 only by the addition of a counter
and the elementary arithmetic to update it. Notice however that the clauses for
nNeededCourse rely on head constraints. During evaluation, it is automatically
transformed as seen earlier to a necessarily terminating theory. Clearly, unless
course requirements are circular (which could happen because of an input error), the
evaluation of this specific theory would terminate. However, a structurally identical
problem in the airline example may not.
The predicate nCourses describes the number N of courses a student S still has
to take to qualify for a degree D, along a single requirement chain. We want to
find the maximum such N over all the chains. In order to do this, we can apply
the techniques described in Sect. 6.1.2 to transform a universal question into an
existential one and two negations (one of them is implicit in the constraint N 0 > N ):

MoreCoursesThan.S; D; N /  nCourses.S; D; N /
^ nCourses.S; D; N 0 / ^ N 0 > N
nSemesters.S; D; N /  nCourses.S; D; N /
^ :MoreCoursesThan.S; D; N /

It should be noted that the technique we just developed is ineffective for the
problem solved in Sect. 6.1.3: counting the total number of courses that a student
still has to take, not just those on a single requirement path.

6.3.2 Lists

The bounding transformation and the consequent admission of head constraints

within recursive clauses open the gates to degrees of expressiveness that go well
beyond the examples we have encountered so far. In this section, we will add a new
dimension to the deductive spreadsheet by proposing lists of values as arguments
to a predicate. Using the standard Prolog notation for the moment [CM87, O’K90,
SS94], a list or sequence S is either the empty list, written Œ, or a list S 0 prefixed by
some value v, which is traditionally denoted by ŒvjS 0 :

Lists S WWD Œ j ŒvjS 

We adopt the usual conventions of abbreviating a list Œv1 jŒv2 jŒ: : : vn jŒ : : : as simply
Œv1 ; v2 ; : : : ; vn . Differently from Prolog, we only admit lists of values, although
more general forms, for example lists of lists, will be explored in Sect. 6.5. Unless
otherwise specified, any use of the word ‘list’ in this book will refer to lists of values
as just defined.
Lists provide natural solutions to new classes of useful problems. For example,
in Sect. 6.1.1, we presented a theory CourseDep that calculated the pairs of courses
.C; C 0 / such that course C 0 was a transitive prerequisite of C . We can now use a
list to collect all the intervening courses that allow CourseDep.C; C 0 / to hold:
222 6 The Deductive Spreadsheet

CourseDepList.L/  Prereq.C; C 0 / ^ L D ŒC; C 0 

CourseDepList.L/  Prereq.C; C 0 / ^ CourseDepList.L0 /
^ L0 D ŒC 0 jL00  ^ L D ŒC jL0 

The first clause assigns the two-element list ŒC; C 0  to the argument of
CourseDepList if Prereq.C; C 0 / is true. The second extends a course dependency
list ŒC 0 jL00  to ŒC; C 0 jL00  whenever this same fact is derivable. Therefore, if
CourseDep.C; C 0 / is provable, so is CourseDepList.L/, where C and C 0 are the
first and the last element of the list L, and the intermediate items are the intervening
course requirements. Displaying the transitive prerequisites between two courses is
infeasible unless our language is augmented with lists or a similar construction.
Introducing lists considerably extends the expressive power of the language we
have considered so far, but it also raises a number of questions. We will now examine
these issues and outline solutions for them.

Bounded List Computation

From a computability point of view, allowing a recursively defined data object as

a predicate argument is a sure recipe for non-termination. Lists of values are no
exception. Consider indeed the following variant of the definition of path, altered
not just to compute the pairs of nodes in a graph linked by a path, but also to collect
all of their incident nodes in a list:

path000 .L/  edge.X; Y / ^ L D ŒX; Y 

path000 .L/  edge.X; Y / ^ path000 .L0 / ^ L0 D ŒY jL00  ^ L D ŒX jL0 

Clearly, CourseDepList is an instance of path000 . This definition suffers from the

same deficiency as the counting version path0 in Sect. 6.2.1: if the underlying graph
contains cycles, for example if …edge D fedge.a; a/g, then the evaluation of path000
will diverge, as it would incrementally construct the infinite list Œa; a; : : : in this
example. This phenomenon, the possibility of creating lists of unbounded length, is
precisely the source of non-termination of an extension of our language with lists.
The bounding transformation introduced in Sect. 6.3.1 can address this problem
without modification. Let us examine the list constructor Πj . Whenever arguments
X and L are given, ŒX jL stands for the operation that returns the list obtained by
prefixing X to L.4 Under this light, Πj  is an operator in the exact same sense as
C is an (arithmetic) operator. Then, L D ŒX jL0  is a head constraint, similarly

4
In the above examples, the syntax Œ j  is also used as a destructor in the conjunct L0 D ŒY jL00 :
it does not construct L0 by combining Y and L00 , but instead checks that the first element of L0 is
Y . While such traditional usage of the list constructor has a simple formalization based on pattern-
matching, an approach more in line with our language would introduce an operation, head, which
returns the first element of its argument and express this constraint as Y D head.L0 /.
6.3 Bounded Termination 223

to C D C 0 C 1 in the definition of path0 in Sect. 6.2.1. Head constraints in a

recursive clause constrain the head variables appearing in them (here L or C ) to
assume values different from the objects in the body of this clause, “bigger” values.
This can cause divergence unless there is some mechanism to restrict the magnitude
of these constructed values. Neither the clauses for path000 nor those for path0 embed
any such restriction: in the first case L can grow arbitrarily long, and in the second
C can become arbitrarily large. The bounding transformation introduces a simple
but effective limiting mechanism behind the scenes.
In summary, the possibility of divergence introduced by admitting lists in our
language can be handled exactly like head constraints in recursive clauses, as seen
in Sect. 6.3.1. In this extended sense, both the clauses for CourseDepList and path000
can be left as they were introduced. The bounding transformation will make sure
that their computation terminates.
While the bounding transformation ensures termination, the result may appear
somewhat unexpected to a novice user. Consider again the clauses for path000
together with the factual theory …edge D fedge.a; a/g. Then, the result of evaluation
will be:

path000 .Œa; a/ path000 .Œa; a; a/ path000 .Œa; a; a; a/ path000 .Œa; : : : ; a/
„ ƒ‚ …
hcmax times

Indeed, the bounding transformation does not stop the recursion from visiting a loop
in the object graph over and over. While this may be the intended behavior, it often
is not: certainly not in our airline example. We will see shortly a simple way to avoid
unintended loops.

An Algebra of Lists

Up to this point, we did our best to avoid the question of what the syntax of
lists should be. A minimal syntax consisting of just Πfor the empty list and the
constructor Œ j , as traditionally found in logic programming [CM87, O’K90, SS94]
and used so far, is inadequate in a spreadsheet environment. We have already
observed in footnote 4 that it forces us to rely on pattern-matching to take a list apart:
this clashes with the typical mechanisms available in a spreadsheet. However, we
also noted that this issue can easily be solved by introducing the classical destructor
operators on lists, head.L/ to return the first element of a (non-empty) list L, and
tail.L/ to strip it off and return the rest of L.
A more serious problem is that the usual high-level operators on generic lists,
such as append or member, cannot be defined in our language because they would
violate the safety restrictions (a head variable must appear in a positive body
literal, and similarly for variables occurring in constraints—see Sect. 6.2.1). These
operators have instead simple recursive definitions not only in logic programming,
but also functional programming. Indeed, neither of the following standard logic
programming definitions is valid (variables violating a safety requirement are
highlighted in bold):
224 6 The Deductive Spreadsheet

append.L; Œ; L/
append.L1; ŒXjL2; ŒXjL3/  append.L1; L2; L3/
member.X; ŒXjL/
member.X; ŒYjL/  member.X; L/ ^ X ¤ Y

(we have folded the head constraints into the head of clauses, as typically done in
Prolog). Although it is possible to emulate specific instances of these operations in
our language, this would result in tedious and inefficient theories. We will take a
different approach.
It is theoretically possible to define natural numbers starting from just the number
0 (zero) and the successor function (succ). This is however highly impractical for
daily arithmetic, not to speak of an application such as the spreadsheet. Instead, we
avail ourselves of a number of operators (e.g., C, , etc.) and comparison operators
(e.g., D, >, etc.) which we interpret as primitive, even if they could be derived from
just 0 and succ. They define an abstract algebra that allows us to write complex
expressions such as 3 C 5 > 2  3 in a natural way.
We will follow this same approach in the case of lists, and build an algebra
collecting the most common functions opList and relations List on these objects.
We first need a proper syntax for lists themselves. A list is an ordered collection
of values of finite but unbounded length. In this, it is not too different from a one-
dimensional array (i.e., vector) in a spreadsheet, as defined in Chap. 4. Indeed, the
distinction is only given by the fact that we intend to use lists as arguments of
predicates. Therefore, it is appealing to define a list as an arbitrary one-dimensional
array, Ee , which we will enclose in brackets as ŒEe  when supplied to a predicate. Note
that the comma (“,”) is syntax to construct a vector out of scalars or formulas, which
entitles us to write an extensional list as Œv1 ; : : : ; vn . Going one step further, we can
see these brackets as an operator that promotes an array expression into a list: this
is the first operation in our algebra of lists.
Lists can then be formalized by the first two lines in the following grammar
within our spreadsheet model from Chap. 4:

Lists S WWD Πj opList .S; : : : ; S; Ee ; : : : ; Ee /

List Operators opList WWD Π j tail j append j reverse j splice j : : :
Scalar Operators op WWD C j  j : : : j head j length j : : :

The first line defines a list S as either the empty list Œ, or the result of applying a
list operator opList to appropriate arguments, which could themselves be lists (as in
append, which takes two lists and returns their concatenation) but may also include
scalars (as in substitute, which takes a list and a two scalar values and returns the
list obtained by replacing the first value with the second) or even arrays (indeed,
ŒEe  is such an operator). The second line shows a few common list operators: we
clearly need Π to get off the ground. We already introduced tail to return a list
deprived of its first element as well as append. We show two other operations as
examples.
6.3 Bounded Termination 225

The introduction of lists in a spreadsheet also requires extending the language of

(scalar) expressions with operators that take lists among their arguments but return
a (scalar) value. We show a selection of such added operators in the third line above:
head returns the first element of a list, and length produces its length.
We also need to introduce a number of comparison and relation symbols List so
that lists can participate in attribute expressions and constraints. We assume equality
(D) and inequality (¤) as universal primitives. Useful relational operators on lists
include membership member and its negation notMember, as well as the emptiness
and non-emptiness tests, isEmpty and isNotEmpty. More such symbols may become
useful. Altogether we have the following grammar:

List Relations List WWD member j notMember j isEmpty j isNotEmpty j : : :

It should be noticed that we left list operators opList and relations List open-ended
as it may be convenient to extend them with additional entries as we gain more
experience with these objects. Note also that some of the above operators, such
as head and tail, are defined only partially as applying them to the empty list is
pointless. We rely on the standard error mechanism of the spreadsheet to handle
these cases.
We conclude this part of our discussion with two examples that make use of this
newly introduced syntax. For convenience, we will abbreviate append as the infix
symbol “j”, and write 2 and 62 instead of member and notMember, respectively.
We begin with yet another version of the path predicate, which we call
path.4/ .L/:

path.4/ .L/  edge.X; Y / ^ L D ŒX; Y 

path.4/ .L/  edge.X; Y / ^ path.4/ .L0 / ^ X 62 L0
^ Y D head.L0 / ^ L D ŒX jL0

It differs in two ways from path000 : first it uses the syntax just introduced. Second
it embeds the check X 62 L0 which prevents the recursion from visiting a
cyclic edge more than once. Therefore, when executed against the factual theory
…edge D fedge.a; a/g, it returns just path.4/ .Œa; a/. This is particularly useful in
an adaptation of our airline example from Sect. 6.3.1, where it makes little sense to
change planes more than once in the same airport as part of an itinerary between
two cities.
As a final example, consider the problem outlined in Sect. 6.1 of calculating
the dependencies among courses, but with the twist that some courses can have
alternative requirements. We saw in Sect. 6.1.4 that this was unsolvable within the
simple language we had at that point. Lists and the termination assurance obtained
in this section permit a simple solution. First, we introduce the primitive predicate
AltPrereq.C; Cs/ which associates each course C with the list Cs of courses it
alternatively depends on. Therefore, a student needs to have taken at least one
among the courses listed in Cs before attempting C . We then update the transitive
226 6 The Deductive Spreadsheet

dependency predicate CourseDep.C; C 0 / to CourseDep0 .C; Cs/, where Cs is one of

the alternative prerequisite chains for C , and similarly for MissingReq and CanTake.
They are defined as follows.

CourseDep0 .C; Cs/  Course. ; C; / ^ AltPrereq.C; Cs0 / ^ Cs0 D Œ ^ Cs D Œ

CourseDep0 .C; Cs/  CourseDep0 .C 0 ; Cs0 / ^ AltPrereq.C; Cs00 /
^ C 0 2 Cs00 ^ Cs D ŒC 0 jCs0
MissingReq0 .S; C; Cs/  Student. ; ; S/ ^ CourseDep0 .C; Cs/ ^ Course. ; C 0 ; /
^ C 0 2 Cs ^ :Took.S; C 0 /
CanTake0 .S; C /  Student. ; ; S/ ^ Course. ; C; / ^ CourseDep0 .C; Cs/
^ :MissingReq0 .S; C; Cs/

The first line says that CourseDep0 .C; Œ/ holds if C has no requirements. The
recursive clause for this predicate extends a requirement chain Cs0 for C 0 with any
course C having C 0 as a prerequisite. This yields a requirement chain for C . Notice
how C 0 is tested to belong to Cs00 , and Cs is constructed from C 0 and Cs0 . At this
point, a student S is missing a requirement among the chain Cs for a course C if C
depends on Cs and S did not take one of the courses among Cs. This is captured by
the clause for MissingReq0 . Finally, S can take course C if there is a dependency
chain for C for which S does not miss any requirement.

Lists Within the Spreadsheet Metaphor

The final issue we need to address is how to represent lists of values in a

spreadsheet. Up to now, a primitive or calculated fact occupied a portion of a
row of the spreadsheet, with columns identifying the predicate. For example, the
fact Prereq.ic; ca/ in Fig. 6.1 lay on cells DA3WDB3, with columns DA and DB
hosting the predicate Prereq. Each argument in a fact (a scalar) corresponded to an
individual cell: there ic occupies location DA3. This is consistent with the definition
of a spreadsheet, where each cell evaluates to a scalar. Once we allow the arguments
of a predicate to be lists, something in this infrastructure has to change.
There are essentially two ways in which lists can be accommodated within the
grid layout of the traditional spreadsheet. We can either allow individual cells to
contain not just a scalar but an entire list, or we shall allow a predicate argument to
span more than one column. We will now examine these two options in closer detail.
Lists inside Cells: There have been numerous proposals in the literature for
allowing individual cells to contain complex objects as opposed to just
scalars [BADC 01,Kri88,SB89]. Some of these proposals, e.g., [BADC 01,Kri88],
support cells embedding arbitrary structures, including lists, graphs and even
spreadsheets themselves. Others, e.g., [SB89], only investigate the limited
extension we are suggesting: lists of scalar values.
While very interesting in principle, none of these approaches has made it into a
commercial product, a clear sign that either they suffer from technical difficulties
6.3 Bounded Termination 227

still to be overcome or that there are doubts about the expected popularity of
this solution among users. This last aspect can be attributed to several factors.
First, cells embedding complex objects constitute a substantial departure from
the traditional layout of a spreadsheet and consequently users may have a hard
time discovering this technology or even adapting to it. Second, the development
of an interface supporting it is likely to be in its infancy and therefore still
subject to intensive cognitive assessments. Third and probably most important,
one of the inherent problems of these proposals is that a user cannot see the full
contents of a cell unless he or she does something to bring it to the foreground.
This works against novice users who are unlikely to know the visualization
commands.
Allowing cells to host lists of values is admittedly a more circumscribed problem.
It can indeed be reduced to the problem of visualizing and manipulating a
tri-dimensional spreadsheet, with lists arranged along the added axis. While
tantalizing, this perspective does little to solve the issues listed above. In
particular, a user still cannot see an embedded list in its entirety unless she issues
an appropriate command.
While we do think that these approaches have potential, the described issues
suggest that this technology has not yet reached maturity. We shall keep it in
mind as a second choice.
Multi-column Predicate Arguments: An alternative approach to visually sup-
porting lists is to relax the one-to-one association between predicate arguments
and spreadsheet columns (or rows, depending on the chosen orientation). Recall
that an attribute in Chap. 4 and later a predicate argument in Chap. 5 were defined
as a one-dimensional cell range such as EA3WEA17, possibly unbounded as in
EA3WEA . The approach we are examining allows allocating a multi-column
range such as EB3WEK17 or EB3WEK to a predicate argument.
This is exemplified in Fig. 6.10 where we have entered the clauses for alternative
requirement chains presented earlier in this section. Columns EA through EK
contain the clauses for predicate CourseDep0 , which took two arguments: a single
course C and a list Cs of courses it transitively depended on. When this predicate
is calculated, the computed instances of the first argument C appear in column
EA, starting on row 3, as usual. We have instead allocated columns EB to EK for
Cs, guessing that a requirement list will contain no more than ten courses. The
course list resulting from a successful calculation of the second argument will
be installed across a row EBiWEKi in this range, starting from the leftmost cell.
If this list has less than ten elements, the unused cells will be set to the blank
value. The lower part of Fig. 6.10 shows the two other predicates participating in
this example fragment. MissingReq0 is handled similarly to CourseDep0 : its first
two arguments are scalars while the third has been assigned to a range spanning
another ten columns. Finally, the purely scalar predicate CanTake0 is displayed
as usual.
This solution raises a number of questions, which we will now attempt to
answer:
228 6 The Deductive Spreadsheet

··· EA EB ··· EK ···

1 CourseDep
2 Course Courses
3
CourseDep (C; Cs) Course( ; C ; )
4
∧ AltPrereq(C; C ) ∧ Cs = [] ∧ Cs = []
5

6
CourseDep (C; Cs) CourseDep (C ; C s )
7
∧ AltPrereq(C; Cs ) ∧ C ∈ Cs ∧ Cs = [C ]|Cs
8

··· EL EM EN ··· EW EX EY ···

1 MissingReq CanTake
2 Student Course Courses Student Course
3
MissingReq (S; C; Cs) Student( ; ; S ) CanTake (S; C) Student( ; ; S)
4
∧ CourseDep (C; Cs) ∧ Course( ; C ; ) ∧ Course( ; C ; )
5
∧ C ∈ Cs ∧ ¬Took (S; RC) ∧ CourseDep (C; Cs)
6
∧ ¬MissingReq (S; C; Cs)
7

Fig. 6.10 Computing with lists in the registrar’s spreadsheet

• How large can a list be as an argument? While there are no theoretical bounds
to how many elements a list can contain, it is limited in practice by how many
cells can be allocated to hold a fact. If predicates and therefore their arguments
are aligned along the columns of a spreadsheet, this bound is given by the
maximum number of columns of the specific implementation. The typical
value in commercial products is 256. Note however that aligning predicates
along the rows substantially extends this number (commercial spreadsheets
typically have 65,536 rows), but at the cost of reducing the maximum number
of facts that can be calculated for this relation (to 256 in this scenario).
• What if a list is too long? Consistently with the treatment of arrays and
relations in Chap. 4, lists that are too long for the allocated space will be
truncated. In this case, we call the excess items the overflow of the list.
An alternative design, which however requires experimental assessment,
would allow scrolling a list whenever it is longer than its allocated range. A
similar approach may be applicable in the case of calculated relations with
more tuples than their assigned range can hold. Note that overflow items
(or tuples) would be available for inspection, but not accessible for cross-
paradigm manipulation: assume in Fig. 6.10 that the list calculated for the
first fact of CourseDep0 has 12 elements, two more than the number of cells
in EB3WEK3. Then, any cell containing the scalar formula COUNT.EB3WEK3/
will evaluate to 10 rather than 12.
• Do lists necessarily have to be the last argument? A list can appear anywhere
among the arguments of a predicate.
• How does a user specify the length of a list argument? While a best practice
still has to emerge from experimentation, we shall adhere to the principle that
a user should not be penalized by features she does not use. Therefore, a user
6.3 Bounded Termination 229

shall never need to specify that the length of a scalar argument is 1: by default
all arguments are scalar and have therefore length 1. Whenever the user has
assigned to a predicate more cells than there are arguments, a natural default
interprets the last argument as a list and allocates the excess range to it. If
this is not what the user has in mind, she can manually rectify the range
allocation.
• Can a predicate have more than one list among its arguments? Certainly. As
per the previous question, the user must then explicitly specify the range of
the first of these arguments, and possibly the second as well.
• Can the arguments of a predicate be split among sheets? While the arguments
of a predicate are generally contiguous, it is conceivable that they are
distributed throughout different areas of a single sheet or over multiple sheets
or spreadsheets. The cognitive repercussions of this possibility have however
to be assessed and carefully evaluated.
At the moment, allowing predicate arguments to span multiple columns (or rows)
to accommodate lists appears to be the most promising solution.
While these two proposals are the strongest candidates for allowing lists in
predicate arguments, other possibilities exist. For example, it may be possible
to design external array operators that implicitly access a derivation tree and
deconstruct it into a list. While theoretically viable, solutions of this type are likely
to put excessive demands on the abilities of a typical user.
Because of the inherently cognitive nature of the question of supporting list
arguments in a predicate, a definite answer to which of these solutions shall
eventually be embedded in a product, if any, will be based on experimental evidence.
We will come back to this problem in Part II.

6.3.3 Irrelevant Terms

Consider again the definition of the predicate trip from Sect. 6.3.1:

trip.F; T; D/  hop.F; T; D/
trip.F; T; D/  hop.F; M; D 0 / ^ trip.M; T; D 00 / ^ D D D 0 C D 00

We saw that, under the common assumption that if an airline has a flight from F
to T (predicate hop) it will also have a flight back from T to F , the evaluation of
these clauses will diverge as larger and larger values will be computed for the total
distance D. This is due to the fact that the naı̈ve (and semi-naı̈ve) strategy will stop
only when no new record is computed at some iteration, but here there will always
be new records as D grows. In the case of trip, we would like the computation to
end when the first such D has been computed. We could do this if the termination
condition ignored the third argument of trip.
230 6 The Deductive Spreadsheet

A term whose actual value does not matter (as long as there is one) for the
purpose of computation is called irrelevant, a concept that has recently received
some attention in the fields of proof and type theory [LP09]. It is then tempting to
mark predicate arguments corresponding to the left-hand side of a head constraint
as irrelevant: when evaluating this predicate, the remaining arguments will be used
to decide termination, so that only the first value of irrelevant arguments is reported
to the user. Observe that this approach would work for all the examples examined
in this section: not only when the head constraint computes a number, as in trip, but
also when it builds a list as in CourseDepList and the various definitions of path.
Although intuitively appealing, additional research is required before considering
this approach as part of the deductive spreadsheet. Here are some questions that this
research would need to answer:
• How to identify irrelevant arguments? In the above example, the irrelevant value
of D was computed on the basis of the third, irrelevant, argument of trip and
of the third argument of hop, which may or may not be irrelevant. In general,
an irrelevant argument in a predicate can depend on either relevant or irrelevant
arguments in other predicates. However, a relevant argument shall clearly depend
only on relevant arguments. Now, determining which arguments are irrelevant
requires exploring the dataflow graph of the theory at hand, starting with the
head constraints present in the recursive clauses. Clearly, a definition should be
rejected if it marks all arguments of a predicate as irrelevant.
• Which irrelevant value to return? In the case of trip, the distance D of interest
is clearly the first value computed by the evaluation (nobody likes doing loops
through the same airport), and the same for CourseDepList. But is this always the
case? What about path? At this stage, we do not know: more evidence needs to be
gathered by encoding additional problems. Furthermore, there may not always be
a single “first” value that can be computed: for example, two base clauses could
seed D with different values, which would then propagate. What should happen
then? This could result in a lot of frustration unless the user can easily predict
the behavior of the system. This suggests checking that predicates are functional
with respect to their relevant argument (for each value of the relevant argument,
the irrelevant arguments can assume at most one value).

6.4 Additional Examples

In this section, we will abandon the registrar’s example that has accompanied us
for the past three chapters. We will however further demonstrate the usefulness of
our deductive extension of the spreadsheet by examining a number of examples in
different fields. We begin in Sect. 6.4.2 by revisiting the transitive closure theory by
describing a few more common problems that fit this pattern. Section 6.4.3 solves
the classical bill of materials problem, while Sect. 6.4.4 implements a workflow
solver. Section 6.4.5 constructs a simple meeting planner. We conclude in Sect. 6.4.6
6.4 Additional Examples 231

with another classic, the anti-trust control problem. Before embarking in this
endeavor, it is convenient to introduce abbreviations for the most common reasoning
forms encountered so far. This is the subject of Sect. 6.4.1.

6.4.1 Abbreviated Forms

Up to now, we have mostly expressed our examples using the base syntax defined in
Sect. 6.2.1 and extended with lists in Sect. 6.3.2. While correct, this occasionally led
to rather clumsy representations of very common problems, such as the extensional
universal quantification of Sect. 6.1.2. This can easily be remedied by making
abbreviated forms available for the most common functionalities. Each such form is
internally expanded to our base syntax without user intervention. This has several
advantages.
• First and most obvious, this saves typing, at the cost of a larger number of
constructions to remember and choose from.
• Second and most important, it reduces the cognitive burden on users by making
pre-programmed macros available for common functionalities. This is particu-
larly useful for a user who is not well versed in logic since some of the base
constructions (for example extensional quantification) are rather intricate and
demand a good mastery of logical reasoning.
• Third, besides ensuring correctness, the target clauses to which these abbrevi-
ations translate can be chosen as to be maximally efficient. We have seen that
there are often many competing clause sets that realize the same predicate, and
that their efficiency varies.
Observe that such abbreviated forms are common in the traditional spreadsheet. For
example, all commercial products make a function AVG.El / available to compute
the average of the values in range El, although it could easily be expressed as
SUM.El/=COUNT.El/ using two other standard functions.
We will now propose abbreviations for a number of recurrent functionalities.
We have already encountered some of these, for example transitive closure and
extensional universal quantification. We will encounter several more in the rest of
this section. Figure 6.11 summarizes some common templates by highlighting the
abbreviated body form on the left-hand side of each row. The body construction
it maps to is displayed below the arrow, with supporting clauses on the right-hand
side. For succinctness, we display only one of the possible translations in our base
language: in general each construction will have a number of possible targets,
chosen on the basis of environmental considerations as to maximize efficiency.
We also limit our presentation to a small sample of recurrent abbreviations and
discuss variants only informally. The actual abbreviations to be made available in
an implementation of the deductive spreadsheet shall be established and refined
empirically.
232 6 The Deductive Spreadsheet

Fig. 6.11 Abbreviations for some common constructions

The top part of the figure shows a number of variations on the transitive closure
theme. The first line makes the transitive closure of a binary relation p.X; Y /
available as p C .X; Y /. With this shortcut in place, the definition for CourseDep
in Sect. 6.1.1 (see also Fig. 6.1) can more succinctly be rewritten as

CourseDep.C; C 00 /  PrereqC .C; C 00 /

6.4 Additional Examples 233

or even more succinctly, PrereqC can be used wherever CourseDep appears. We can
similarly write edgeC for path.
The more general form TRANSITIVE X; Y: p.X; Y; Z/ E defines the transitive
closure of predicates that have more than two arguments, or with arguments in a
non-standard order. The abbreviation p C is clearly a special case.
Next, the form Xs D p ŒC .X; Y / accumulates the intermediate elements of the
transitive closure of p from X to Y in the list Xs, which is returned as if it were
computed using a constraint. With this abbreviation, the definition of path.4/ in
Sect. 6.3.2 can be replaced with

path.4/ .L/  L D edgeŒC .X; Y /

or omitted altogether. Although not shown in Fig. 6.11, this technique can easily
be extended to predicates with more than two arguments, for example by writing
Xs DTRANSLIST X; Y: p.X; Y; Z/. E The next two rows show variants that also
return the length of the computed list, and the sum of a third argument, throughout
the recursion. We can then give the following simplified forms for the predicates
path0 and path00 of Sect. 6.2.1, and trip and trip0 from Sect. 6.3.1:

path0 .X; Z; C /  C D COUNT: edgeŒC .X; Z/

path00 .X; Z; C /  C D COUNT  5: edgeŒC .X; Z/
trip.F; T; D/  D D SUM D 0 : hopŒC .F; T; D 0 /
trip0 .F; T; D/  D D SUM  24;900 D 0 : hopŒC .F; T; D 0 /

Notice that we have liberally omitted any mention of the resulting list (written
Xs D : : : above) when unnecessary, and that we have interspersed explicit bounding
conditions as needed.
The basic form of extensional universal quantification examined in Sect. 6.1.2 is
transparently abbreviated into the body expression FORALL X W p.X; YE /: q.X; YE /:
it ascertains that for all values of X such that p.X; YE / holds, so does q.X; YE / for
given values of YE . The translation in our core language is presented in Fig. 6.11.
Notice that in order for the target theory of the transformation to be safe, the
variables YE must occur in a positive literal in the body of the clause where FORALL
appears. This abbreviation yields the following simplified form for the predicate
CanTake of Sect. 6.1.2:

CanTake.S; C /  Student. ; ; S / ^ Course. ; C; / ^

FORALL RC W CourseDep.C; RC/: Took.S; RC/

This is also reported in Fig. 6.2

Given a predicate p containing a designated numerical argument N , the form
N 0 D FINDMAX N: p.N; YE / computes the maximum value of such N in p, with
234 6 The Deductive Spreadsheet

the variables YE possibly further constraining the instances of p to consider. This

abbreviation allows us to give a succinct definition for the predicate nSemesters of
Sect. 6.3.1 as:

nSemesters.S; D; N /  N D FINDMAX N 0 : nCourses.S; D; N 0 /

This construction can be further enriched with bounds and filters, for example in
the form N 0 D FINDMAX N W q.N; YE /: p.N; YE /, which only considers values of
N such that q.N; YE / holds. Other abbreviated forms operating on numbers, such
as FINDMIN or FINDAVG, can be defined similarly. Using the same template,
the abbreviation Xs D FINDALL X: p.X; YE / collects all the values of X for
which p.X; YE / holds into the list Xs. The variables YE are again subject to safety
constraints.
Given a binary predicate p.Y; Ys/ describing a list Ys of values related to Y , the
form Xs D ACCUMULATE Y: p.Y; Ys/ generalizes the notion of transitive closure
by recursively collecting every object related to Y via p into the list Xs. We will use
and further explain this notation and some variants in the next section.
We anticipate that more abbreviated forms will emerge as more examples are
developed. While these pre-packaged reasoning units do not extend the expres-
siveness of the deductive spreadsheet in any way, they simplify the interaction for
users with limited familiarity with formal logic, in the same way as commonly
used formula patterns are made available as macros and redundant operators in the
traditional spreadsheet. We expect the net effect to be to make applications easier to
write and therefore augment productivity.

6.4.2 More Transitive Closure Problems

In earlier sections of this book, we have encountered transitive closure problems

over and over. The basic version of this problem reports the pair of nodes of a graph
that are linked by a path:

path.X; Z/  edge.X; Z/
path.X; Z/  edge.X; Y / ^ path.Y; Z/

which we have abbreviated as edgeC . A number of common problems are solved

by clauses such as these by simply varying the interpretation of the predicates edge
and edgeC . The table in Fig. 6.12 recalls some of the readings we have used, and
suggests a few additional ones—some are further discussed below. Indeed, any path-
related problem that can be interpreted as a hierarchy, in fact as an arbitrary graph,
falls into the category that can be solved by clauses akin to these. In Sect. 6.3,
we have seen several variants of the transitive closure problem that supported
counting the number of nodes on a path (or any measure associated with them)
6.4 Additional Examples 235

p p+
Course prerequisite Transitive course requirements
Parent-child relation Ancestor relation
Airports with direct flights Multi-hop air itineraries
Immediate supervisor relation Indirect boss-employee relation
Military command relation Military chain-of-command
Immediate geographic enclosure Geographic containment
Department-subdepartment relation Business organization chart
Immediate oversight relation Jurisdiction relation

Fig. 6.12 Some interpretations of transitive closure

and reporting these nodes in a list. These techniques are clearly available for any
of the interpretation of edge and edgeC . Other useful variations of the transitive
closure clauses can be constructed, although they tend to be more specific to the
interpretation domain at hand.
Genealogies are a perfect example of transitive closures. When building one’s
family tree (a graph, really), the base relation is the parent-child relation. Its
transitive closure is then the ancestor relation. To spice up the discussion, consider
the following variant: the Mathematical Genealogy Project5 collects information
about mathematicians and computer scientists such as where they did their dis-
sertation, when they defended it, and who was their advisor. After importing this
data as a couple of relations such as “advisor” and “thesis”, it is easy to use the
deductive spreadsheet to explore one’s academic ancestry (one’s advisor, advisor’s
advisor, etc.) to find one’s oldest academic ancestor, to pinpoint to famous academic
ancestors or cousins (e.g., Gottfried Leibniz or Alan Turing), or to find oddities in
the genealogical data (e.g., a gap of 80 years or more between a person’s dissertation
and his advisor’s). Each of these questions can be answered using clauses similar
to those for path above. The graphical user interface can even support tools to
draw the resulting genealogy tree—see the discussion on connection graphs in
Chap. 9.
Finding routes between a starting location and a destination is another instance of
the transitive closure problem, a problem that Sect. 6.3.1 examined in the case of air
travel—this is an admittedly moot example since nowadays many websites offer far
better tools than what one can build at home using a deductive spreadsheet. Here is
a situation that a travel site will not be of much help with: say one’s hobby has to do
with collecting historical transportation schedules. Loading them into the deductive
spreadsheet allows answering questions such as “Was it possible to go from Lisbon
to Istanbul by train in 1905?” or “Could Phileas Fogg have actually gone around
the world in 80 days by public transportation in 1872, as claimed by Jules Verne?”.

5
See http://genealogy.math.ndsu.nodak.edu/
236 6 The Deductive Spreadsheet

Transitive closure problems abound in personal and professional settings: business

organization charts, military chains of command, legal jurisdiction, and geographic
subdivisions are just a few examples.

6.4.3 Bill of Materials Problem

In this section we will examine a simple instance of the bill of materials problem,
a classical benchmark for deductive databases [CGT90, Col98]. Assume that a
company manufactures a number of products (e.g., a car), each consisting of parts
(e.g., the engine), which in turn may be built out of subparts (e.g., the pistons),
themselves constructed out of smaller parts (e.g., the piston’s head), and so on up to
elementary subparts that cannot be further decomposed (e.g., bolts). In this context,
the simplest instance of the bill of materials problem requires producing the list of
all the elementary parts that are present in a product. Other interesting questions
include finding all the products that contain a given subpart, and computing the
cost of a product knowing the cost of the elementary subparts and the overhead of
assembling components into larger components.
We model the relationship between a component P and its immediate subparts by
means of the predicate Parts.P; Ps; C / where Ps is a list of the parts of P . This list
is empty if P is elementary. In this case, C is the cost of P . If P can be decomposed
into subparts, C is the cost of assembling it.
The following mutually recursive clauses solve the basic bill of materials
problem:

Subparts.P; Ss/  Parts.P; Ps; / ^ Subparts0 .Ps; Ss/

Subparts0 .Ps; Ss/  Ps D Œ ^ Ss D Œ
Subparts0 .Ps; Ss/  Subparts.P; Ss0 / ^ Subparts0 .Ps0 ; Ss00 /
^ Ps D ŒP jPs0 ^ Ss D Ss0 jSs00
^ Parts. ; L/ ^ P 2 L ^ Ps0 D splitTail.L; P /

The predicate Subparts.P; Ss/ in the first clause collects the elementary subparts
of a component P into the list Ss. It does so by looking up the immediate
subparts Ps of P and invoking the auxiliary predicate Subparts0 which calculates
the elementary subparts of the list of components in its first argument. It is defined
by the last two clauses. The base case handles the empty list. The last clause for
Subparts0 .Ps; Ss/ is more interesting: a non-empty Ps will have the form ŒP jPs0
and therefore the problem can be reduced to finding the elementary subparts of P
(by invoking Subparts) and the base components of Ps0 (by relying on a recursive
call to Subparts0 ). The conjuncts on the last line ensure that P and Ps0 are indeed
related by being subparts of a common component (the function splitTail.L; P /
splits list L at some occurrence of P and returns what follows it). Although
logically unnecessary, the presence of these conjuncts results in a more efficient
6.4 Additional Examples 237

specification since otherwise bottom-up evaluation would pair up arbitrary parts

P ’s with arbitrary lists of parts Ps’s.
By taking advantage of the abbreviations introduced in Sect. 6.4.1, the above
three clauses can be replaced with the following simple definition:

Subparts.P; Ss/  Ss D ACCUMULATE P: Parts.P; Ps; /

Were we interested in a list of all the subparts of a product P , not just its
elementary subparts, it would be sufficient to replace the first clause in the original
definition of Subparts with the following:

Subparts.P; Ss/  Parts.P; Ps; / ^ Subparts0 .Ps; S Ps/

^ Ss D PsjS Ps

A form ACCUMULATE could be introduced to express this variant. Note that our
model implicitly takes into account the fact that a component may contain several
occurrences of the same subpart. The above theory can however easily be turned
into a more succinct and efficient model that explicitly accounts for the multiplicity
of subparts.
Given either of the above specifications, the problem of finding which products
contain a given component is solved by the following clause:

OccursInProduct.S; P /  Parts.S; ; / ^ Subparts.P; Ps/ ^ S 2 Ps

Finally, the cumulative cost of a product is calculated by the following clauses,

which follow the pattern of the definition of Subparts above:

Cost.P; C /  Parts.P; Ps; C 0 / ^ Cost0 .Ps; C 00 /

^ C D C 0 C C 00
Cost0 .Ps; C /  Ps D Π^ C D 0
Cost0 .Ps; C /  Cost.P; C 0 / ^ Cost0 .Ps0 ; C 00 /
^ Ps D ŒP jPs0 ^ C D C 0 C C 00
^ Parts. ; L/ ^ P 2 L ^ Ps0 D splitTail.L; P /

Using again the abbreviations in the spirit of Sect. 6.4.1, we can simplify this
definition into:
Cost.P; C /  C D SUM C 0 : ACCUMULATE P: Parts.P; Ps; C 0 /

6.4.4 Workflow Problem

A simple variation of the solution to the bill of material problem allows us to

specify the workflow of an industrial process and calculate the earliest time for its
completion, as well as the completion of any intermediate point. Let the predicates
238 6 The Deductive Spreadsheet

Start.a/ and End.b/ mark the fact that a and b are the start and end points of the
process. Furthermore, let Follows.a; L/ indicate that task a cannot take place before
each of the tasks in the list L have been completed. Finally we associate every task
a with the time t it takes to complete it by means of the predicate Time.a; t/.
The problem is solved by the mutually recursive predicates TimeTo.A; T /, which
calculates the minimum time T required to achieve task A without violating any
timing bound, and MaxTime.As; T /, which performs a similar task with respect to
all the tasks in the list As. Predicate Earliest reports the timing constraints of the
process as a whole.

TimeTo.A; T /  Start.A/ ^ T D 0
TimeTo.A; T /  Follows.A; As/ ^ Time.A; T 0 / ^ MaxTime.As; T 00 /
^ T D T 0 C T 00
MaxTime.As; T /  As D Π^ T D 0
MaxTime.As; T /  Time.A; T 0 / ^ MaxTime.As0 ; T 00 / ^ As D ŒAjAs0
^ T D max.T 0 ; T 00 / ^ Follows. ; L/ ^ A 2 L
^ As0 D splitTail.L; A/
Earliest.A; T /  TimeTo.A; T / ^ End.A/

Note the second defining clause of MaxTime, where we rely again on the list
function splitTail to boost the speed of calculation. We already relied on this
technical device in the bill of materials problem. An abbreviated form for aspects
of this definition could be introduced, were it to be commonly found in practical
examples.
The case study we used throughout this book, a college student assembling
a course sequence on the basis of information about course availability, prereq-
uisites, etc., features several aspects that fit the pattern of a workflow problem.
Consider the following questions this student may want answered: “Can I enroll
in ‘Advanced Networking’ next semester?”, “What courses do I need to complete
before I can enroll in ‘Operating Systems’?”, “When is the earliest date I can take
‘Computer Architecture’?”, “What additional courses do I need to take to satisfy
the requirements of my major?”, “What is the least number of courses I can take to
major in CS and get a Math minor?”. Answering these questions involves solving a
workflow problem: there are a number of tasks (here courses) subject to constraints
(availability, dependencies) that have to be performed to complete a project (degree).
Here, a critical feature of the deductive spreadsheet is its support for negation.
Negation not only allows specifying that the absence of certain records matters
in calculated relations (e.g., completed classes should not be returned when the
student looks for the course he still has to take), but it is also key to expressing
concurrent enabling conditions through extensional universal quantification (e.g.,
all prerequisites of a course must have been completed before a student can enroll in
it). A large number of questions can be framed as workflow problems, especially any
time planning is involved, such as in supply-chain management. Dedicated software
packages are available, but many users, even in large companies, do not know about
6.4 Additional Examples 239

them or do not have time to learn how to use them. The deductive spreadsheet
provides a simple way to develop workflow solvers customized to one’s needs in
many situations that are not very large or complex.

6.4.5 Scheduling Problem

Next, we implement a simple meeting planner, which will sieve through the
schedule of the potential attendees of a meeting and display the dates where at least
N participants are available. We rely on the primitive predicate Available.P; D/ to
describe each date D in which person P is available to meet. The cell symbolically
indicated as minMeetingSize (via a one-cell range name) contains the threshold
value N for the meeting to take place.
This very simple planner is specified by the clauses below. The bulk of the work
is done by the predicate AvailableOnDate0 .D; Ps/ which collects in the list Ps all the
potential participants that have expressed their availability for date D. The auxiliary
predicate Missing.D; Ps/ checks whether any of these people is missing from the
list Ps. Both are used in the predicate AvailableOnDate.D; Ps; L/, which returns
the maximal list Ps of people available on D. The last argument, L, is simply the
length of Ps. Finally PossibleMeetingDates.D; Ps/ returns the dates and parties for
which at least N people can attend the meeting (where again the cell containing N
is mnemonically specified as minMeetingSize).

AvailableOnDate0 .D; Ps/  Available.P; D/ ^ Ps D ŒP 

AvailableOnDate0 .D; Ps/  Available.P; D/ ^ AvailableOnDate0 .D; Ps0 /
^ P 62 Ps0 ^ Ps D ŒP jPs0
Missing.D; Ps/  AvailableOnDate0 .D; Ps/ ^ Available.P; D/
^ P 62 Ps
AvailableOnDate.D; Ps; L/  AvailableOnDate0 .D; Ps/ ^ :Missing.D; Ps/
^ L D length.Ps/
PossibleMeetingDates.D; Ps/  AvailableOnDate.D; Ps; L/
^ L  minMeetingSize

We can once more take advantage of the abbreviated forms introduced in

Sect. 6.4.1 to replace the above five clauses with the following succinct definition:

PossibleMeetingDates.D; Ps/  Ps D FINDALL P: Available.P; D/

^ length.Ps/  minMeetingSize

This example implements a very basic meeting planner. It can however easily be
extended to a more realistic model supporting finer grained meeting times, as well
as constraints on participation (e.g., Tom will not attend unless Bill is there as well,
and certainly not if Carl shows up).
240 6 The Deductive Spreadsheet

This simple meeting planner is an instance of the problem of finding an optimal

schedule for a set of tasks, possibly with added constraints such as transit times.
Scheduling is indeed another common problem that can be surprisingly complex
to solve manually even with just a handful of tasks. Other examples are creating
a schedule of classes and allocating workers with specific skills to cover the
multiple tasks of a project. People use traditional spreadsheets to create schedules
by hand, but no support is available to spot inconsistencies or to optimize a schedule
according to some criteria. By contrast, using the deductive spreadsheet one can take
basic constraints in the form of tables (e.g., the availability of the participants in a
meeting) and propose optimal schedules (e.g., with the minimum amount of travel)
or schedules that satisfy some more complex constraints (e.g., a worker cannot be
in two places at the same time). Again, dedicated software is available and widely
used for large or frequent scheduling problems, yet most schedules are crafted by
hand and many are far from optimal.

6.4.6 Anti-trust Control Problem

As our final example, we adapt a problem reported in [CGT90]. Consider a setting

consisting of a number of companies producing goods of various types and selling
them in various regions, or markets. The quantity of goods of a given type T sold
by company C as a percentage of all goods of type T sold in a given market defines
C ’s direct market quota. C may however control another company C 0 (i.e., own
more than 50 % of its shares) which sells goods of type T in this market, hence
indirectly increasing its quota. Certain markets are regulated so that no company
can have a direct or indirect quota of any good larger than a specified threshold.
The anti-trust control problem aims at verifying that no company violates these
regulations.
Our model for this problem conjures together many of the techniques we have
explored in this chapter. We will now present its solution one piece at a time. First,
assume that the primitive predicate HasShares.C; C 0 ; N / describes the fact that
company C owns N % of company C 0 . Then, the following clauses determine when
a company C controls another company C 0 (if it owns more than 50 % of its shares)
and the list of all the companies controlled by C . The technique applied here is
similar to what was used for the meeting planner problem.

Controls.C; C 0 /  HasShares.C; C 0 ; N / ^ N > 50

Controlled.C; Cs/  Controls.C; C 0 / ^ Cs D ŒC 0 
Controlled.C; Cs/  Controls.C; C 0 / ^ Controlled.C; Cs0 /
^ C 62 Cs0 ^ Cs D ŒC 0 jCs0
Missed.C; Cs/  Controlled.C; Cs/ ^ Controls.C; C 0 / ^ C 0 62 Cs
AllControlled.C; Cs/  Controlled.C; Cs/ ^ :Missed.C; Cs/
6.5 Extensions 241

Next we give clauses that compute the total market quota of a company C . In order
to do so, we rely on the feeder predicate Company.C; M; Q/ that describes the
fact that company C directly owns a quota of market M equal to Q%. Then, the
mutually recursive predicates Market and TotalQuota realize this objective using
the same technique already encountered in Sects. 6.4.3 and 6.4.4.

Market.C; M; T /  Company.C; M; Q/ ^ AllControlled.C; Cs/

^ TotalQuota.Cs; M; T 0 / ^ T D Q C T 0
TotalQuota.Cs; M; T /  Company. ; M; / ^ Cs D Π^ T D 0
TotalQuota.Cs; M; T /  Market.C; M; T 0 / ^ TotalQuota.Cs0 ; M; T 00 /
^ T D T 0 C T 00 ^ Cs D ŒC jCs0
^ AllControlled. ; Cs00 / ^ C 2 Cs00
^ Cs0 D splitTail.C; Cs00 /

Finally, if the primitive predicate Limit.M; H / sets the threshold for market M to
H %, the predicate InViolation.M; C; T / returns the companies C that violate the
regulations of market M , together with the total quota T they actually hold.

InViolation.M; C; T /  Limit.M; H / ^ Market.C; M; T / ^ T > H

Once more, we conclude the section by re-expressing this example using the
abbreviated forms introduced in Sect. 6.4.1. Predicate AllControlled can be defined
using FINDALL, hence eliminating the need for the auxiliary Controlled and
Missed. The predicate Market and its dependents are similarly captured by the SUM
variant of ACCUMULATE, which can be in-lined in the clause for InViolation:

Controls.C; C 0 /  HasShares.C; C 0 ; N / ^ N > 50

AllControlled.C; Cs/  Cs D FINDALL C 0 : Controls.C; C 0 /
InViolation.M; C; T /  T D SUM Q:
ACCUMULATE .AllControlled.C; Cs/
^ Company.C; M; Q// ^ Limit.M; H / ^ T > H

The use of abbreviated forms allows an easy expression of this complex problem.
These three clauses expand to eight definitions in our base syntax.

6.5 Extensions

In this section, we briefly examine a number of possible extensions to the language

of the deductive spreadsheet obtained so far. While some are very promising and
would require only minimal adaptations to benefit our design, others are reviewed
here mostly for completeness as they do not seem to meet the cognitive requirements
of a spreadsheet environment. In Sect. 6.5.1, we discuss embedded implication,
a recent and extremely promising extension to logic programming which could
242 6 The Deductive Spreadsheet

noticeably simplify some of our examples. We look into complex objects, a staple
of logic programming, in Sect. 6.5.2. We conclude with a brief overview of typing
in Sect. 6.5.3.

6.5.1 Embedded Implication

We first examine one of the most successful recent extensions of logic programming:
embedded implications [Mil89, MNPS91, Bon91, Fre92, Bon94]. This proposal
simply allows the body of a clause to contain implication subgoals such as P ! P 0 .
This may appear as a vacuous exercise in the context of the deductive spreadsheet or
even of logic programming in general since this subformula is logically equivalent
to :P _ P 0 , which can easily be recast into our current formalism as seen in
Chap. 5. This is certainly the case in the traditional interpretation of logic, known
as classical logic. This equivalence does not hold, however, in the alternative
intuitionistic interpretation of logic [TvD88]. In intuitionistic logic, solving the
implication P ! P 0 requires providing a mechanical method to transform a proof
of P into a proof of P 0 , which in general is different from showing that either :P
or P 0 is provable. Under this interpretation, embedded implications are inherently
hypothetical: P ! P 0 is provable if P 0 can be proved under the assumption that P
holds. This is tantamount to temporarily extending the theory at hand with P while
proving P 0 . It is this last characterization that has fueled the interest in embedded
implication. Having a logical device to temporarily augment a running program
suddenly provides a declarative justification for advanced programming concepts
such as modules [Mil89] and references to external routines [Fre92].
In the context of the deductive spreadsheet, embedded implication provides a
natural support for hypothetical reasoning. Whenever a user wants to know what
would happen to a calculation or a deduction if a certain cell or group of cells were
to assume certain values, he has no choice but modifying these locations and later
remembering to restore them to their original state. With embedded implication, he
can simply write a few clauses or even just a one-time query that assumes (a fact
specifying) that those cells hold the hypothetical values. This is a clearly useful tool
for many forms of what-if analysis, such as allocating budget items, deciding on
mortgage terms, scheduling task priorities, etc.
From the above discussion, it is apparent that including embedded implication in
our language amounts to much more than simply providing a new linguistic feature:
it requires shifting the entire framework we have constructed from a classical
perspective over to intuitionistic logic. This task would be Herculean in a generic
fragment of first-order logic. It is nearly trivial in our case because the language we
have developed so far has the property that the classical and intuitionistic logical
consequences of any of its theories are the same (negation does not satisfy this
property in general, although it does under our stratified interpretation). Adding
embedded implication breaks this symmetry and therefore forces us to take the fork
of the road that heads toward intuitionism.
6.5 Extensions 243

We will now briefly discuss the impact of supporting embedded implications in

the deductive spreadsheet and in particular with respect to its logical sublanguage.
We begin by modifying the syntax we have used so far to accommodate these
objects. We then describe the required changes to the model theory and the proof
theory of our language. We conclude with an example and further discussion.

Syntax

Syntactically speaking, this extension is achieved by permitting the body of a clause

to contain implications with a fact as their antecedent. Therefore, we simply extend
the grammatical rules for clause bodies from Sect. 6.2.1 with a single production,
which is here highlighted:

Bodies B WWD P j :P j æ  æ j B ^ B j P !B

The added production, P ! B, is called a hypothetical subgoal or embedded

implication. Recall that the predicates P in this definition have the form p.X E /.
Therefore, the antecedent of an embedded implication P ! B will contain
variables. Similarly to negative literals and constraints, we shall impose safety
restrictions on the variables X E that appear in a hypothetical subgoal p.X E / ! B0
within the body B of a clause. Namely, whenever a variable X appears in the
antecedent of an embedded implication, it shall also occur in a non-hypothetical
positive literal or on the left-hand side of a non-hypothetical head constraint in the
body B.6 As with negative literals and constraints, these restrictions ensure that
these variables can be instantiated with only a finite number of values. In particular,
this implies that hypothetical subgoals can always be solved after all the variables
appearing in their antecedent have been instantiated.
A number of logical equivalences allow simplifying the structure of clause
bodies. First, the antecedents of nested implications can be turned into a conjunction
by means of the following equivalence:

P1 ! .P2 ! B/  .P1 ^ P2 / ! B

This
V is know as currying. It allows us to focus on hypothetical subgoals of the form
i Pi ! B where B does not start with an implication.
Another logical equivalence allows us to consider just atomic conjuncts as the
target of an embedded implication:

P ! .B1 ^ B2 /  .P ! B1 / ^ .P ! B2 /

6
This simple but rather stringent restriction could be loosened by performing a form of dataflow
analysis known as mode checking [DW88, Sar10]. In order for a clause to be acceptable, the mode
analysis shall certify that X can always be instantiated before attempting to solve the embedded
implication.
244 6 The Deductive Spreadsheet

Together, these equivalences justifyVrestricting our attention to clause bodies

consisting of conjuncts of the form i Pi ! B where B is either a literal or a
constraint. By the safety requirements
V above, this last case can actually be dropped
since a subgoal of the form i Pi ! æ  æ0 is provable if and only if the constraint
æ  æ0 is provable by itself.
By enacting all these transformations, a generic clause can be reduced to the
following normal form:
^ Vm ^ Vm ^
P  . j D1i
Pij ! Pi / ^ . j D1i
Pij ! :Qi / ^ æi i æ0i
i D1::m i D1::n i D1::o
Vmi
with j D1 Pij D true for mi D 0, which reduces this component to simply Pi .
As we conclude this syntactic discussion, we should point out that standard
developments allow for much richer implication goals than what we have just
presented [MNPS91, Fre92, Bon94]. Indeed, the antecedent of an embedded impli-
cation is generally defined as an arbitrary clause (possibly containing embedded
implications) rather than just a fact. We believe that a similar extension can be
supported in the context of the deductive spreadsheet, although further theoretical
research is needed to be ascertain this. Our main reason not to explore this avenue
more aggressively is that it is not clear how useful this generalization would be in
the context of the deductive spreadsheet. We suspect that the cognitive price to pay
for this added expressiveness is too high: mastering this feature would be overly
complex for even an intermediate user.

Model Theory

In Chap. 5, we expressed the validity of a formula ' by means of the judgment

I; ˛ ˆ ', where I was an interpretation and ˛ was an assignment of ground terms
for the variables occurring in '. We saw that in our clausal context, it was sufficient
for ˛ to range over the set V of admissible values, and to take I to be a subset of
the Herbrand base HBV of the theory at hand, i.e., I 2 P HBV , where this last form
expresses that requirements relative to the powerset of the Herbrand base.
We need to make a few changes to this setting in order to accommodate
embedded implication within an intuitionistic framework. First of all, we shall
change our notion of interpretation from a set of true facts (a subset of the Herbrand
base) to a mapping that describes which facts are considered provable from any set
of facts given as assumptions. As a preliminary step, we define a pre-interpretation
I as a function that associates a subset of the Herbrand base (the assumed facts) to
another subset of the Herbrand base (the provable facts):

I W P HBV ! P HBV

In this context, the subsets of the Herbrand base are known as worlds and denoted by
w. An interpretation is then a monotonic pre-interpretation, i.e., a pre-interpretation
I such that for any w; w0  HBV it is the case that if w  w0 then I.w/  I.w0 /.
6.5 Extensions 245

This corresponds to requiring that adding assumptions to a theory does not put the
provability of any fact into jeopardy. While unsound in general in the presence
of negation, this will prove adequate for stratified theories. This is a particular
case of Kripke structure, a prevailing model semantics of intuitionistic first-order
logic [TvD88].
Second, it is convenient to express the validity judgment by describing how an
interpretation defines validity with respect to each individual world. Therefore, we
update the form of the validity judgment to I; ˛; w jˆ '. It states that interpretation
I validates the formula ' in world w with respect to assignment ˛. As in Chap. 5,
this judgment is inductively defined on the structure of '. A specification of this
definition, partially specialized to our language of interest, is as follows:

I; ˛; w jˆ true always
I; ˛; w jˆ false never
I; ˛; w jˆ P iff ˛.P / 2 I.w/
I; ˛; w jˆ :P iff it is not the case that I; ˛; w jˆ P
I; ˛; w jˆ '1 ^ '2 iff I; ˛; w jˆ '1 and I; ˛; w jˆ '2
I; ˛; w jˆ '1 _ '2 iff I; ˛; w jˆ '1 or I; ˛; w jˆ '2
I; ˛; w jˆ P  ' iff I; ˛; .w [ fP g/ jˆ '
I; ˛; w jˆ 8X:' iff for all v 2 V it is the case that I; ˛X 7!v ; w jˆ '
I; ˛; w jˆ 9X:' iff there is v 2 V such that I; ˛X 7!v ; w jˆ '
The main difference with respect to the classical definition in Chap. 5 is in the rule
for implication. There, I; ˛ ˆ P ! ' was reduced to I; ˛ ˆ ' under the condition
that one could show that I; ˛ ˆ P holds. Here, it is required that I validates ' in the
current world w extended with the antecedent P , in symbols I; ˛; .w [ fP g/ jˆ '.
Note that a truly intuitionistic model would handle negation and universal
quantification differently from what is shown here. In particular, it would rely on
a standard Kripke structure [TvD88]. The characteristics of our language are such
that this more general treatment is equivalent to what we are considering here.
As in Chap. 5, an interpretation I (and variable assignment ˛) is a model of a
formula ' if for every world w it can be shown that I; ˛; w jˆ '. Formula ' is a
logical consequence of '1 ; : : : ; 'n if every model of each of the 'i , for i D 1::n, is
also a model of '.

Fixpoint Semantics and Bottom-Up Evaluation

The fixpoint semantics developed for our language can be adapted to handle
embedded implications in the intuitionistic setting. For simplicity, we begin our
discussion by examining just definite theories, characterized by the absence of
negation. Given a theory … D .…; /, the immediate consequence operator T…;
still transforms interpretations. For convenience, it is customary to define it with
respect to each individual world of an interpretation. At an abstract level, it has the
following form:
246 6 The Deductive Spreadsheet

T…; .I /.w/ D I.w/ [ … [ f˛.P / W P  B 2 and I; ˛; w jˆ Bg

T…; extends the interpretation I.w/ of every world w with all the facts in … and
the instances of the heads of all clauses in whose bodies are valid in I at w.
We can specialize this expression to the form of a definite clause, obtaining:
V V mi
T…; .I /.w/ D I.w/ [ … [ f˛.P / W P  iD1::m . j D1 Pij ! Pi /
V 0
^ iD1::o æi i æi 2
S mi
and ˛.Pi / 2 I.w [ j D1 fPij g/ [ … for all i D 1::m
and ˛.æi / 
O i ˛.æi / is true for all i D 1::o g

Notice
Vmi that the validity of the consequent Pi of every embeddedSmi implication
P
j D1 ij S! P i is assessed against the interpretation I.w [ j D1 fP ij g/ of the
mi
world w [ j D1 fPij g, and …. Constraints are handled as usual.
It is easy to show that this operator is monotonic and continuous over the
lattice of interpretations with ordering I v I 0 defined as I.w/  I 0 .w/ for
all w 2 P HBV [Mil89]. Then, by Kleene’s theorem, T…; has a least fixpoint
obtained by taking successive iterations from the bottom element of this lattice,
the interpretation I? such that I? .w/ D ; for all w  HBV :

lfp.T…; / D T…;
!
.I? /

This yields the least model I…; of …; , which is a function from sets of facts to
sets of facts. The logical consequences of this theory are then given by applying the
least model to the empty set (or equivalently to any subset of …): I…; .;/.
Using variations on the techniques in Sect. 6.2.2, it can be proved that this
computation terminates after finitely many iterations. It is therefore an appropriate
foundation for a naı̈ve evaluation strategy, which can be refined to a more efficient
semi-naı̈ve form.
This approach is easily extended to handle non-hypothetical negative literals
in a stratified normal theory [Fre92, Bon94]. When evaluating a stratum … & D
.…& ; & / of a theory … , we simply make sure that negative literals do not occur
in …& :
V Vmi
T…& ; & .I /.w/ D I.w/[…& [f˛.P / W P  i D1::m . j D1 Pij ! Pi /
V
^ i D1::n :Qi
V 0
^ i D1::o æi i æi 2 &
Smi
and ˛.Pi / 2 I.w [ j D1 fPij g/ [ …& for all i D 1::m
and ˛.Qi / 62 …& for all i D 1::n
and ˛.æi / 
O i ˛.æi / is true for all i D 1::o g

Handling hypothetical
V negative body literals is somewhat more complicated because
the assumptions m j D1 Pj of a negated predicate :Q may modify the theory of the
6.5 Extensions 247

stratum where Q is defined, hence altering the set of logical consequences exported
to stratum &. Therefore, the expression for the T operator must have access to the
least models of all the lower-level strata so that it can take the various Pj into
account in order to assess the non-validity of Q [Fre92, Bon94].

Proof-Theoretic Semantics and Top-Down Evaluation

The proof-theoretic semantics of embedded implication is captured by a simple

inference rule [Men97, MNPS91]:
.…; P /; ` B
ie
…; ` P ! B
This rule reduces the problem of finding a proof for the embedded implication P !
B in a theory … D .…; / to the problem of constructing a derivation for just B
but in a theory enlarged with the fact P . This is consistent with the use of a Kripke
structure in the fixpoint characterization we just analyzed as the extended theory
.…; P / is precisely the world in which B would be evaluated.
This rule can immediately be used as the basis for a proof-theoretic bottom-
up evaluation procedure. It also fits into the top-down framework last described in
Sect. 6.2.4. Indeed, it yields the following two simple rules for positive and negative
derivability.

.…; P /; ` B 7! ˛ .…; P /; 6` B
ie" ie#
…; ` P ! B 7! ˛ …; 6` P ! B

Rules pr" and np# shall undergo minor alterations to account for the possibility
of facts in … containing variables. These variables can be instantiated by the
application of the first of these two rules, and this instantiation shall be propagated
in other parts of the theory. The safety restrictions introduced at the beginning of
this section somewhat mitigate this behavior.
In Chap. 5, we advocated the use of top-down evaluation as an ideally suited
mechanism for explanation, and possibly for executing one-time queries. While the
first objective is still met, the second acquires a particularly interesting dimension as
it supports a new form of hypothetical reasoning. In a spreadsheet, what-if analysis
is traditionally performed by modifying key cells and observing how these changes
are propagated. A query containing an embedded implication allows solving a goal
under the additional assumptions specified in the implication’s antecedent, without
making any change in the spreadsheet itself. For example, a student may check how
taking different courses next semester will affect his graduation date. As another
example, a home-buyer may use hypothetical subgoals to predict mortgage payment
under a number of different interest rates. This form of hypothetical reasoning is
quick, non-intrusive and logically motivated.
248 6 The Deductive Spreadsheet

Examples

As a simple example of the use of embedded implication, consider the problem

of an airline executive trying to decide which among a number of possible route
extensions can yield the maximum time savings to passengers. We have already
encountered the primitive predicate hop.F; T; D/ which describes the fact that there
is a direct flight between airports F and T which are D miles apart. We introduce
the predicate CandidateHop, which has the same arguments, to indicate routes being
evaluated for deployment. By using the technique in Sect. 6.4.4, we can easily define
the predicate ShortestRoute.F; T; D/ which computes the shortest route between
two airports. Then, the following clause solves our problem, under the not fully
realistic assumption that the saving in miles flown is proportional to the saving in
time:

SavesMiles0 .F; T; F 0 ; T 0 ; S /  ShortestRoute.F; T; D/

^ CandidateHop.F 0 ; T 0 ; D 0 /
^ .hop.F 0 ; T 0 ; D 0 / ! ShortestRoute.F; T; D 00 //
^ D 00 < D ^ S D D  D 00

Note the use of embedded implication to add hop.F 0 ; T 0 ; D 0 / as a new fact on which
to base the computation of the second instance of ShortestRoute.

Assessing Embedded Implication

Embedded implication is a useful technology that could fruitfully be included into

the language of the deductive spreadsheet described in this chapter. It effectively
adds to the expressiveness of our language, its usage is simple and intuitive, and
it is not plagued by the cognitive drawbacks of the extensions we will explore in
the remainder of this section. It is our belief, however, that additional scrutiny is
required before giving it the green light and making it available in the deductive
spreadsheet. While most of its fixpoint and proof theory are well understood at this
stage, the analysis of hypothetical negative literals needs to be completed. Moreover,
the cost of providing this linguistic feature has not been studied.
We are confident that embedded implication will pass these theoretical tests and
that it may be incorporated in a forthcoming version of the logical language of
the deductive spreadsheet. Once this is done, it will be interesting to investigate
extensions of the class of formulas that can appear in the antecedent of an embedded
implication. As observed earlier, it has been shown in the literature [MNPS91,Fre92,
Bon94] that clauses themselves could be used in that position while preserving the
good theoretical properties of the language. It is however not clear how useful this
full extension would be to an average user of the deductive spreadsheet. Restricted
extensions could however be welcome.
6.5 Extensions 249

6.5.2 Complex Objects

As mentioned in Sect. 6.2.5 the logical formalism we have defined in the early
part of this chapter was closely related to Datalog, which was clearly manifested
by the absence of uninterpreted function symbols in its term language. We have
walked one step closer to Prolog and other logical languages supporting complex
terms in Sect. 6.3.2 with the admission of lists of values as predicate arguments.
In this section, we will briefly analyze the implications of going all the way down
that path and allowing terms freely generated from a set of constants and function
symbols.
This extension would have the effect of allowing cells to contain complex objects.
For example, an employee of a company could be described by a structured term of
the form

employee.name.Henry; C; Smith/;
contacts.address.2498 College Drive; Albuquerque; NM; 65802/;
phone.515-203-8701/;
position.chief analyst; 2=6=2011;
position.analyst; 7=12=2007;
position.tester; 9=23=2003;
new hire/////

rather than the flat predicate

employee.Henry; C; Smith; 2498 College Drive; Albuquerque; NM; 65802; 515-203-8701;

Œchief analyst; 2=6=2011; analyst; 7=12=2007; tester; 9=23=2003/

The structure of the first version is much easier to read and navigate. Notice that we
have rendered recursively embedded position subterms by means of a list.
From a purely computational point of view, complex objects do not add any
new expressiveness to the language of the deductive spreadsheet (with value
lists). Indeed, as in this example, a complex term can always be flattened into a
combination of scalar values and lists. The practical benefits are however evident.
Complex objects suffer from the same non-terminating behavior as lists of values
in Sect. 6.3.2, which can be remedied by relying once more on the bounding
transformation. Therefore, from a purely linguistic point of view, there is no reason
to exclude them from the deductive spreadsheet.
Unfortunately, complex objects do not seem to blend well into the spreadsheet
metaphor, although various authors have suggested ideas for embedding them within
cells [BADC 01, Kri88]. The problem is once more related to their visualization. In
the case of lists, we could summon a solution that retained the property that cells
contain scalar values only, but this does not seem easily achievable in the case of
freely generated terms. Furthermore, we suspect that complex terms are too far from
250 6 The Deductive Spreadsheet

the type of objects that spreadsheet users are accustomed to, and therefore their
inclusion runs the risk of confusion if not rejection. For these reasons, we shall not
incorporate complex objects (save for lists of values—see Sect. 6.3.2) within the
language of the deductive spreadsheet at this stage.

6.5.3 Typing

Since its inception in the 1960s, types have proved enormously popular in pro-
gramming languages and their application to software engineering. Strongly typed
programs contain fewer flaws than untyped ones, which has a positive effect on
development time and reliability. There have been a number of designs of typed
logic programming languages [Pfe92], including some in which sophisticated type
theories are themselves viewed as logic programming languages [Pfe89, PS99].
Our language could easily be extended to support type annotations, and even a
type system that ensures strong typing. It is conceivable that interface functions be
devised to assign types to data values, expressions and formulas. Typing carries,
however, a cognitive price. While a trained programmer is expected to understand
(and use) types, a typical spreadsheet user is likely to not fully comprehend them,
and even find the rigid safety rules they enforce in the way of getting work done.
The strict typing discipline of modern programming languages, a blessing for
professional programmers, would be a curse for the occasional spreadsheet user,
who may give up on these applications before having sufficiently mastered typing
to reap its benefits. Not surprisingly, no commercial spreadsheet application we are
aware of relies on a typed language. We will follow the same trend and refrain from
forcing the formulas of the deductive spreadsheet to be typed. In future work, we
plan to explore offering strict typing as an option for spreadsheet programmers,
disabled by default for end-users.

6.6 Summary

This chapter concludes the development of the formal language underlying the
deductive spreadsheet. Our main achievement has been the extension of the simple
logical formalism of Chap. 5, itself a linguistic variant of the relational language of
Chap. 4, with recursion. This change, which only affects the form of clauses and not
the functional infrastructure of the spreadsheet, has furthered the expressiveness of
the logical spreadsheet with truly deductive capabilities. Indeed, recursive clauses
were shown to provide a simple solution to new classes of problems, questions that
could not be modeled in the logical language of Chap. 4.
A few syntactic restrictions had to be installed to prevent divergence during the
evaluation of these generalized theories. In particular, recursive clauses had to be
forbidden from mentioning head constraints, which could cause non-termination.
6.6 Summary 251

With this guard in place, we could reuse the model-theoretic infrastructure of

Chap. 4 and even extend it to efficiently support the evaluation of recursive theories.
The traditional depth-first and breadth-first proof-theoretic strategies were shown to
diverge even with those restrictions in place. Some sophisticated alternatives were
explored to ensure termination.
Although necessary in general, the ban on head constraints in recursive
definitions appeared to prevent a large class of safe and useful problems from
being modeled. We relaxed it by means of the bounding transformation, which
allowed arbitrary constraints, but imposed a user-defined maximum on the number
of times recursion could visit them during evaluation. This same transformation,
invisible to the user, also opened the door to extending the language of the deductive
spreadsheet by permitting lists of values as predicate arguments. We demonstrated
the resulting leap in expressiveness by tackling a variety of common but non-trivial
problems.
We concluded this chapter by exploring further possible extensions. We devel-
oped a large portion of the theory of embedded implication, but concluded that
this promising technology requires additional scrutiny before welcoming it into
the language of the deductive spreadsheet. We also reviewed complex objects and
typing, but branded them as misfits in the spreadsheet metaphor.
Having completed the study of the language of the deductive spreadsheet, it is
time to go over the functional requirements stated in Chap. 2. As it turns out, our
design for the deductive spreadsheet meets or exceeds each of these requirements:
Functional conservativity: Every traditional spreadsheet functionality is sup-
ported. Indeed, a user not availing herself of the deductive extension will have
access to the exact same language she sees in a traditional spreadsheet.
Expressiveness: The logical language underlying the deductive spreadsheet is
an extension of Datalog with constraints over the functional language of the
traditional spreadsheet. It supports negation through stratification, and embeds
the full range of operator and comparison relations available in a traditional
spreadsheet. It also supports lists of values and the algebra surrounding them.
A further extension with embedded implications is being assessed.
Supported inferences: Evaluation reports all of the values satisfying a predicate,
without duplicates. Explanation allows asking why values were or were not
returned, possibly for partially instantiated questions. In addition, our design
supports one-time queries to perform reasoning tasks without entering them into
the spreadsheet. All the traditional approaches to what-if analysis are supported,
and could be extended with logically based methods centered around embedded
implication.
Termination: All functionalities in the deductive spreadsheet are terminating.
The theoretical assessment of efficiency is positively encouraging, but it needs
to be confirmed by experimental data—see Chap. 11 for the results of some
preliminary experiments.
252 6 The Deductive Spreadsheet

Updates: The automatic and manual propagation of updates is extended to the

logical components of the deductive spreadsheet. It remains unchanged with
respect to the traditional functional component.
Functional integration: The traditional and deductive functionalities are fully
integrated within one another. Interface functions are provided so that a scalar
or array can appear in a clause and vice versa. Traditional formulas can refer to
cells calculated logically and vice versa.
 Part II
User Interface

Outline

Having completed the abstract design of the linguistic functionalities underlying the
deductive spreadsheet, we now turn to the problem of making them available to the
user in a simple and intuitive way. This is an essential aspect of our development
since the ultimate goal of our overall endeavor is not only to augment the expressive
power of the traditional spreadsheet, but also to retain its characteristic usability.
In this part of the specification, we will propose a conservative extension to the
user interface of a typical spreadsheet application with support for the deductive
infrastructure developed in Part I. While formal logic and the theory of program-
ming languages provide tested guidelines that allowed an objective assessment of
the abstract functionalities of the deductive spreadsheet, the value of user interface
design choices is highly subjective and not as well understood. We take our guiding
principles from recent developments in the area of cognitive psychology and follow
them as we design the visual extension of the spreadsheet. Because of the subjective
nature of this task, only empirical assessment on actual users will be able to validate
(or rebuke) our choices. Indeed, while we believe that the abstract functionalities of
the deductive spreadsheet are relatively stable, we expect our proposal for the user
interface to change substantially as we expand our test base and incorporate user
feedback.
Chapter 7 begins our discussion of the visual presentation of the deductive
spreadsheet with a careful review of the most important concepts of the user
interface of the traditional spreadsheet. It isolates the points where to anchor the
deductive extensions to the existing display layout and methods for performing
tasks, which are relatively uniform among commercially available spreadsheets.
In Chap. 8, we introduce the methodology we follow as we design the user
interface of the deductive spreadsheet. Because meeting the cognitive requirements
of the user is paramount, we combine two recent proposals from cognitive science:
the attention investment model takes an abstract view of the problem solving task
254 II User Interface

by framing it in economic terms; in contrast, the cognitive dimension of notations

proposes a number of criteria on which to evaluate individual design choices.
Chapter 9 proposes a series of alterations to the traditional spreadsheet interface
in order to support our deductive extension. As we do so, we strive to remain
conservative with respect to the traditional spreadsheet and to make the complete
set of deductive functionalities available. We rely on the cognitive tools of Chap. 8
in a prescriptive way to achieve these goals.
In Chap. 10, we report on some preliminary experiments on a few representative
users. Empirical assessments of this sort will be substantially expanded in future
work in order to exhaustively evaluate our design and propose improvements.
Chapter 7
The Traditional Interface

In preparation of weaving the deductive functionalities defined in Part I into the

visual language of the traditional spreadsheet, this chapter reviews the principal
concepts underlying the user interface of modern commercial spreadsheets. A close
inspection of these applications reveals a number of dimensions across which the
user interface is defined. First of all, there is the distinction between the static
organization of the spreadsheet and the dynamic aspects of user interaction: this
is the difference between what a user sees and what she needs to do to trigger a
desired effect. A second dimension revolves around the typical use of a spreadsheet
as a problem solving device, and formulas as the instruments it makes available
for this purpose. Users are exposed to formulas both as individual objects and as
building blocks of the overall solution to their problem. At the former, local, level
of interaction, the user interface provides various methods to enter an individual
formula into a cell: commercial products have taken great care to make formula
input as simple as possible. Globally, a user needs to juggle all the formulas in her
spreadsheet to ensure that they collectively solve her problem: support here is mostly
provided through auditing tools and methods for controlling evaluation. Finally, the
user interface addresses the problem of presenting computed data back to the user
(or to her customers) in an effective way. This goes from their default rendering as
a grid of cells, to minor formatting involving fonts and colors, to their expression as
elaborate charts.
Altogether, these various aspects, and several more that are not relevant to
our extension, constitute a fairly complex system, with many opportunities for
diversification and experimentation. It is therefore all the more surprising that the
most popular commercial spreadsheet applications rely on nearly identical graphical
user interfaces. For example, the initial screen of the eight products shown in Fig. 7.1
differ in inessential details such as the location of toolbars and the name of menus.
This is clearly the sign of a technology that has reached maturity, for the most part.
Because the differences tend to be very minor, we will conduct our investigation
relative to Microsoft Excel, which introduced the graphical user interface of the
spreadsheet—as an Apple Macintosh application—in the 1980s (the grid layout in
textual mode goes back to Daniel Bricklin’s original VisiCalc in the late 1970s) and

I. Cervesato, The Deductive Spreadsheet, Cognitive Technologies, 255

DOI 10.1007/978-3-642-37747-1 7, © Springer-Verlag Berlin Heidelberg 2013
256 7 The Traditional Interface

Google Spreadsheet

Apple Numbers

Fig. 7.1 Initial screenshot of eight commercial spreadsheets

is currently the most commonly found spreadsheet application. The specific version
we rely on is Excel 2003, which is de facto the greatest common denominator
between the more recent versions of Excel and all other spreadsheet applications
7 The Traditional Interface 257

LibreOffice Calc

KDE KSpread

Fig. 7.1 (continued)

on the market (the most common are Google Spreadsheet, Apple’s Numbers, and
OpenOffice and LibreOffice Calc). We refer the reader to [Har03] for a thorough
yet user-friendly treatment of the concepts touched here, and many more.
We examine the general layout of a spreadsheet in Sect. 7.1, where basic format-
ting and the static organization of cells and worksheets is discussed. Section 7.2
258 7 The Traditional Interface

Gnome Gnumeric

Microsoft Excel 2000

Fig. 7.1 (continued)

7 The Traditional Interface 259

Microsoft Excel 2010

VisiCalc

Fig. 7.1 (continued)

describes the available options for entering text and especially formulas in a
cell: we first review the textual language of formulas, and then focus on the
graphical facilities that modern spreadsheets provide to simplify their construction.
Section 7.3 discusses how a user can control updates as well as the tools available to
trace the behavior of a formula. Section 7.4 elaborates on some relevant productivity
facilities, e.g., charts, available in commercial products. Finally, the salient aspects
of the user interface of the traditional spreadsheet are summarized in Sect. 7.5.
Throughout this presentation, we will highlight points that our deductive extension
shall touch, but we will not provide specific solutions until Chap. 9.
260 7 The Traditional Interface

7.1 Layout

In this section, we begin to examine the static aspects of the user interface of
the traditional spreadsheet. The question we strive to answer is “How do things
look?”. We focus on the basic organization of the workspace in Sect. 7.1.1 as
well as visual formatting of these elements, within a cell in Sect. 7.1.2 and across
cells in Sect. 7.1.3. We delay the discussion of the textual appearance of formulas,
technically another form of information that the users sees when opening an existing
spreadsheet: they will be examined in Sect. 7.2.
We rely on the user interface of Microsoft Excel 2003 for illustration—see
also [Har03]. This choice is due to the fact that, although Microsoft has recently
changed the look of Excel, all competing products still rely on a user interface that is
nearly identical to that of Excel 2003 and its predecessors. The graphical user inter-
faces of seven commercial spreadsheets are displayed in Fig. 7.1: going left-to-right
and top-down, we have Google Spreadsheet, Apple Numbers, LibreOffice Calc,
KDE KSpread, Gnome Gnumeric, Microsoft Excel 2000, and Microsoft Excel 2010.
The bottom-right image shows the text-based user interface of VisiCalc [Bri09], the
precursor of all spreadsheets in 1979.

7.1.1 Cells, Worksheets, and Workbooks

Figure 7.2 shows an annotated enlargement of the initial screen displayed by Excel
2003 when opening a blank spreadsheet—it is representative of many current
applications. The top part of the window contains the menu bar, from which the
user can access nearly all the commands available in the system by using the
mouse. Commands performing related functionalities are grouped under the same
menu, with submenus further collecting operations acting on the same or similar
components of the spreadsheet. Clicking on an item in a menu or submenu either
directly performs the command, or opens a dialog that offers further options.
Occasionally, a wizard guides the user through the phases of a complex task. Excel
2007 introduced an alternative presentation of menu commands as a ribbon located
just below the menu bar: when clicking on a menu in the menu bar, the ribbon
gives persistent graphical access to its commands rather than displaying them in
menu form and forcing the user to select a command right away. The contents of the
ribbon are also to some extent context-sensitive. Although sensible in principle, the
ribbon has received mixed reviews from users and no other commercial spreadsheet
application has adopted it.
In the default configuration, one or more toolbars appear just below the menu
bar. Toolbars contain a number of iconic buttons or selection lists that act as
shortcuts for frequently used commands. As such, there are no sub-toolbars. Related
commands are grouped in the same toolbar. Starting with Excel 2000, toolbars can
be configured to add or remove commands, although few users ever bother changing
7.1 Layout 261

Menu bar
Toolbar
Formula bar
Column locator

Active cell

Row locator

Worksheet navigation

Fig. 7.2 Layout of Microsoft Excel 2003

the default settings. Toolbars can also be relocated to other positions besides the
near top of the window. They can indeed be aligned along its left, right, or bottom
margins, and positioned before or after other toolbars. They can also be turned
into floating objects that hover anywhere on the screen where the user may care to
put them. In some other commercial products (e.g., Gnumeric and Calc) and older
versions of Excel, toolbars cannot be moved.
One toolbar that holds a special status in nearly all applications is the formula
bar, which collects a number of essential functionalities related to the currently
selected cell. In particular, it identifies it, it reports on its contents, and it allows
entering text or formulas into it. Applications generally disallow relocating it. We
will examine the formula bar more closely in Sect. 7.2.
The very bottom of the window is usually reserved for displaying information
about the status of the application and about any special mode being activated. This
is the status bar.
The central and largest area of the window displays the data contained in an
individual spreadsheet file (or the user’s newly created work). The familiar grid
characteristic of these applications is called a worksheet. It is divided into horizontal
rows and vertical columns, and expands well beyond the physical frame of the
application window. Scrolling permits exposing invisible parts of the worksheet.
Excel and most commercial applications fix the size of a worksheet to 256 columns
and 65,536 rows. Users of a traditional spreadsheet rarely exhaust all the available
262 7 The Traditional Interface

rows, but complex problems occasionally require more columns than available.
A spreadsheet file can contain more than one worksheet, which can be selected
by means of a worksheet navigation toolbar generally located at the bottom left of
the main window. Menu commands allow visualizing several worksheets at once.
All the worksheets contained in a given spreadsheet file constitute a workbook.
Workbooks are a useful way to organize one’s data, and also overcome the problem
deriving from the limited number of columns in an individual worksheet. Note that
this multiplicity of worksheets does not constitute a third dimension in which to lay
down data, but simply a way to collect multiple sheets in the same file.
The top part of the worksheet contains the column header, which reports
identifiers for the columns, while names for the rows are displayed on the left-hand
side in the row header. They define a system of coordinates that allow addressing
individual cells on the worksheet. The default system of coordinates is known as
A1: it identifies cells by giving letters for its columns (from A to IV) and numbers
for its rows (the top left column has coordinate A1). The alternative R1C1 system
presents the user with a pair of numbers, first the row and then the column (the top
left column is then R1C1, with ‘R’ and ‘C’ acting as mnemonic prefixes for the two
dimensions). Cells are internally addressed in the R1C1 format.
Most of the time, there is a single active cell, on which operations can be
performed. The active cell is marked in various ways: a border frames it, its row and
column headers are highlighted, and its coordinates are reported on the left-hand
side of the formula bar. There are several options for changing which cell is active:
clicking on a different cell (as long as we are not in formula input mode), using
the cursor keys (again unless not in input mode), or entering its coordinate in the
formula bar or a goto dialog. Not only individual cells, but any rectangular area of
the spreadsheet can be selected by highlighting a cell and moving the mouse while
keeping its left button depressed, or by using the cursor keys while pressing the
SHIFT key. One or more rows can be selected by similarly acting on the row header.
The same applies to columns. The entire worksheet can be selected by clicking on
the top left rectangle where the row and column headers meet. It is even possible to
select multiple areas by holding the CTRL key, as shown in Fig. 7.3. A number of
context-dependent commands acting on the selected cell or cells can be invoked by
clicking with the right button of the mouse (on a PC).
An alternative way to refer to cells, rectangular worksheet areas, and actually
even arbitrary regions, is to assign them a symbolic range name as shown in Fig. 7.3.
Range names can be used to find the associated area in the worksheet, but they are
especially useful as a mnemonic identifier within formulas.
Although the deductive spreadsheet constitutes a substantial extension of tradi-
tional applications such as Microsoft Excel, most of the core layout choices just
outlined do not need to be altered. Indeed, data will still be organized in workbooks
containing one or more worksheets, each of which will be a bidimensional grid of
cells, and the basic modality of interaction will remain the same. A few changes
will be visible, such as the addition of a few (sub-)menus and toolbars. Others will
be nearly invisible, such as possible support for more than 256 columns. Indeed,
7.1 Layout 263

Fig. 7.3 Assigning symbolic names to a multiple selection

most of the deductive extension will rest on existing interface features such as range
names and data lists (see Sect. 7.1.3).

7.1.2 Formatting Within a Cell

An individual cell can contain either data or scalar formulas. A rectangular group
of cells can contain an array formula. We will further comment on the contents of
cells in Sect. 7.2. Instead, we now concentrate on the visual aspect of cells and the
options for modifying it.
By default, cells are uniform white rectangles outlined with a thin gray line.
Text entered in them is black. Commercial spreadsheets offer however a large
number of options for customizing their appearance to one’s taste. The contained
values can be formatted using different fonts, sizes, colors, alignments, etc. It is
also possible to instruct the application to display the contents according to some
intended interpretation such as dates, currencies, etc., often including fine detail
such as the number of decimal digits. The cell themselves can be decorated with
borders and a variety of backgrounds. They can also be individually protected
against modification or inspection.
The formatting commands are available from the menus, using the toolbar or
ribbon, by right-clicking on the selected cell(s), and through keyboard shortcuts.
The user interface of our deductive extension will retain all the formatting options
available within a cell. We will not need to alter them nor to introduce any new one.
264 7 The Traditional Interface

Fig. 7.4 A data list and a data form for it

7.1.3 Formatting Within a Worksheet

Many of the formatting options just discussed are applicable to selections spanning
multiple cells: each highlighted cell is then affected by the operation. Some
formatting operations are however specific to groups of cells. It is convenient to
classify them according to the geometry of the cell selection they apply to.
Rectangular areas of the spreadsheet can be treated as a unit when working with
array formulas (see Sect. 7.2 for more details) as well as in order to perform actions
such as selective printing. Most commercial products allow merging adjacent cells
into a single cell. This is particularly useful in order to associate a caption to an area
of the spreadsheet interpreted as a table with multiple columns (see Fig. 7.4 for an
example).
Rows and column as a whole have their own formatting operations, which
include the possibility of modifying their height and width, respectively. It also
possible to hide entire rows or columns (which corresponds to temporarily setting
their height/width to zero) and unhide them when desiring to inspect the data
contained in them. This is very useful for the purpose of managing the always
scarce area visible in a spreadsheet window. Most applications actually support
outlines, which associate a number of contiguous rows or columns with a button
to hide/unhide them as a group.
7.2 Formulas 265

Most traditional spreadsheets provide support for facilitating the interpretation of

a group of contiguous columns as a relational table. In Microsoft Excel, this concept
is called a data list. A data list is defined by simply writing the header of a table on
a row and one or more records below it. The left-hand side of Fig. 7.4 shows this
process on an example from Chap. 4: row 2 contains the headers (more properly field
names) for the four-column table, with a name for the table as a whole in a merged
cell on row 1; rows 3 and 4 show two records. Once a data list has been defined, it
can conveniently be manipulated by means of an automatically generated data form,
such as the one on the right-hand side of Fig. 7.4. The data form associated with a
data list relates each value in a record to the corresponding field name. It allows
displaying, creating, updating, deleting and searching records in the data list. Other
operations become available once a group of columns has been interpreted as a data
list. In particular, records can be sorted along one or more fields, they can be filtered
so that only records that satisfied specified criteria are displayed (the downward-
facing black triangle next to the field names in Fig. 7.4 serve this purpose), and they
can be selectively summarized by means of aggregation functions such as SUM,
AVG, etc.
Finally, there are operations that have a visual impact on an entire worksheet.
Among them is the possibility of decorating a spreadsheet with floating objects such
as charts, pictures, diagrams, etc.
Again, our deductive extension will not interfere with the existing formatting
options, either by limiting them or by adding new ones. Because of the tabular nature
of relations and predicates, it will instead emphasize the use of existing features that
operate on entire regions of a worksheet. The concept of data list will however need
to be streamlined to provide simpler and more immediate support to defining and
working with tabular information in the form of relations and predicates. It will also
need to be generalized to support lists as fields (see Chap. 6) and also to allow table
to be defined along the rows of a worksheet.

7.2 Formulas

Although spreadsheets are occasionally used to format grocery lists, contacts, and
other data, the power of these applications truly emerges when using formulas.
Indeed, formulas allow extracting information that is implicit in the raw data that
populates other cells. The visualization of these derived values, for example a
currency conversion or the total of an expense list, enriches the information available
to the user by expressing it in a more usable way. Indeed, formulas raise the
spreadsheet from a simple data recording application to a powerful problem solving
tool. Calculated values can furthermore be formatted in the very same way as basic
data.
We will dedicate the rest of this chapter to this problem solving support and
especially to the interface functionalities available to the user to take advantage of
it. We begin with the most basic method for constructing a formula: entering it
266 7 The Traditional Interface

as text in a cell. We examine the syntax and structure of scalar formulas as seen
by the user in Sect. 7.2.1, while the textual method for entering array formulas is
reviewed in Sect. 7.2.1. Although many users rely on this textual form of formula
construction, many more find it clumsy if not frustrating, and happily do without
it whenever possible. For them, all commercial spreadsheets make available conve-
nient graphical methods for constructing formulas. This is examined in Sect. 7.2.3.
It should be noted that these methods are just front-ends to the textual language,
and therefore a thorough understanding of this more rudimentary formalism is
essential. Section 7.2.4 describes what is maybe the most commonly used approach
in large spreadsheets: the clever copy-and-paste mechanism characteristic of these
applications.
It is to the problem solving potential of the traditional spreadsheet that our
deductive extension attaches. Indeed, the clausal infrastructure described in Part I
enriches the notion of formula by providing logically motivated mechanisms that
operate on blocks of cells at a time. In Chap. 9, we will extend the textual syntax
of Excel and similar products to accommodate clausal theories. We will also extend
the range of its graphical formula construction methods to these new objects.

7.2.1 Scalar Formulas

In order to enter a scalar formula in a cell, the user simply types an ‘D’ sign and
then the formula she wants to include in that cell. Cell B17 in Fig. 7.6 contains
the simple formula that converts a temperature expressed in Celsius degrees into
Fahrenheit. The formula itself, “D B16  9=5 C 32”, is displayed in the rightmost
textbox of the formula bar while cell B17 shows its result, 68, calculated from the
fact that the reference B16 contains the value 20. In general, this box holds the
contents entered in a cell: with the exception of formulas, this is also what appears
in the cell itself, modulo formatting. The appearance of the formula bar changes as
shown in Fig. 7.5 as soon as the user hints at entering a calculated expression by
pressing the ‘D’ key. Various functionalities that we will discuss in further detail in
this section become available.
We described the abstract syntax of textual formulas in Chap. 3. We will now
comment on the concrete syntax, i.e., what the user actually types into the input
box. For the most part, writing a formula from scratch is a very simple matter:
many arithmetic formulas use the notation we all learned in school, such as “D
20  9=5 C32”: operators are infix and obey the usual precedence rules which can be
overridden by including parentheses. This is pretty much what we would type into
a handheld calculator. This intuitive picture is subject to two minor complications.
First, commercial spreadsheets make available a number of functions that
exceeds by far the arithmetic operators of our youth. For them, an infix notation
is often impractical because they can take more than two arguments (sometimes a
variable number of arguments), or because it would be unrelated to our experience.
In these cases, the notation is borrowed from our calculus classes: the name of the
7.2 Formulas 267

AutoSum

Select formula Cancel Enter Insert function Formula display

Fig. 7.5 The formula bar and related functions

Fig. 7.6 Textual formula input

operator is followed by a pair of parentheses containing its arguments separated by

commas. A classical example is SUM, which adds together up to 30 arguments,
each of which can be either a number or an array of numbers.
Second, formulas can embed cell references, and therefore get some of their
input from other cells. This possibility accounts for a large part of the success of a
spreadsheet: it separates the location where data are stored from the locations where
they are used, hence allowing localized updates to influence faraway results, and
it supports breaking a complex formula into manageable chunks over intermediate
cells. Embedding a reference to a single cell in a formula is achieved by simply
including its A1 coordinates, as in “D B16  9=5 C 32” above. It is also possible
to specify a cell reference in R1C1 notation, but we will not discuss this possibility
until Sect. 7.2.4. Operators like SUM also accept references to cell ranges, which
identify a rectangular region of the spreadsheet by specifying the cells at its top
left and bottom-right corners. For example, the central selection in Fig. 7.3 is
identified as C2:D10. One or more columns can be referenced in their entirety
by omitting the row coordinates, so that column A in Fig. 7.3 is denoted by A:A.
Whole rows can also be addressed using a dual syntax. In order to add up the
contents of all the cells highlighted in that figure, we would enter the formula
“D SUM.AWA; C 2WD10; C14; F 2WI 3/”. There is no convenient way to denote a
column deprived of its first two cells, for example. Indeed, were we interested in
all the cells of column A except A1 and A2, we would have to write A3:A654536.
Besides being clumsy, this has the problem that if the maximum number of rows in
a spreadsheet were ever to be increased, this expression would not any more refer
to all cells in column A except A1 and A2. It is possible to reference a cell in a
268 7 The Traditional Interface

Fig. 7.7 Textual input of an array formula

different worksheet by prefixing the name of this sheet with “!” as a separator, as in
“D Sheet2ŠA3 C 1”. It is even possible to reference a cell in a different file.
The same input box that allows creating a new formula can be used to modify
an existing expression. Excel and other applications simplify this task somewhat
by outlining in different colors the cell referenced in a formula. Unfortunately, this
visual aid disappears as soon as one starts the editing process. Both while creating
and while modifying a formula, a user can make syntactic mistakes and enter an
ill-formed expression. In that case the application shows an error dialog with hints
at how to correct it.
Our deductive extension of the spreadsheet will interfere only minimally with the
concrete syntax of scalar formulas. The changes will indeed be limited to extending
the current notation to support partial rows and columns in a simple and general
way, and to providing a functionality to embed clauses within scalar formulas.

7.2.2 Array Formulas

Array formulas are associated not with a single cell as scalar expressions, but with
all the cells in a range. They describe the simultaneous computation of multiple
values. For instance, Fig. 7.7 extends our last example by showing an array formula
that converts a series of Celsius measurements to Fahrenheit. This formula resides in
the range B17:AX17, while its input is located in the range B16:AX16. The formula
itself, “fD B16WAX16  9=5 C 32g”, is very similar to what we used in Fig. 7.6: the
main difference is given by the fact that the range B16:AX16 takes the place of the
reference B16, which promotes the operators , =, and C to their component-wise
versions. The braces surrounding this formula are a visual reminder that this is an
array formula.
Entering an array formula into a spreadsheet is more involved than writing a
scalar expression. Indeed, simply typing “fD B16WAX16  9=5 C 32g” in the usual
text box of the formula bar will simply insert this string in the target cell, or cause a
warning if a cell range had been selected. The proper way to create this formula in
Excel (and all other commercial products we have tried) is to type “D B16WAX16 
7.2 Formulas 269

9=5C32” and then press the key combination CTRLCSHIFTCENTER. The braces
are then inserted automatically. Simply pressing ENTER, as one would do for scalar
formulas, yields a value only for the first cell in the range, here B17. We have not
been able to find a justification for this design, which appears considerably more
complicated than necessary. For lack of a better explanation, we interpret it as an
extension of the otherwise excellent user interface of the traditional spreadsheet that
has not yet reached full maturity. Indeed, this design surprisingly violates several
of the cognitive principles that we will use as guidelines in Chap. 8 for our own
extension to the spreadsheet.
In the example in Fig. 7.7, the array of values returned by the formula had
the same geometry as the range containing the formula. Whenever this is not the
case, the result is truncated, padded with default values, or marked as partially
incomplete. Besides component-wise extensions of scalar operators such as , =,
and C, commercial spreadsheets make available a few constructs that natively return
arrays. We have encountered TRANSPOSE in Chap. 4. The traditional spreadsheet
also provides support for building arrays out of scalars and smaller arrays: for
example, for inserting the constant 2  2 array with first row (1,2) and second
row (3,4), one would select a 2  2 range, enter “D f1; 2I 3; 4g”, and seal it with
CTRLCSHIFTCENTER.
Updating array formulas is even more counter-intuitive than creating them
from scratch. Indeed, simply pointing to one of its cells and typing a new
value or formula will only result in an unfriendly error message. Attempting
the CTRLCSHIFTCENTER combination will produce the same effect. The only
way to modify an array formula is to select it in its entirety, make the changes,
and seal them with CTRLCSHIFTCENTER. One can alternatively use menu
functionalities to convert it to individual scalar formulas, and then operate normally.
Again, we believe these design choices force unnecessary cognitive overhead on the
users.
The clausal definition of a predicate in the deductive spreadsheet usually spans
several cells, often an entire table. Therefore, it is conceivable that a mechanism
akin to the construction of array formulas will be selected for this purpose. We will
follow this general path in Chap. 9, but not the specific modalities outlined in this
section. Indeed, we will aspire to a simple and intuitive approach for creating and
modifying clausal formulas that naturally extends the available methodologies for
scalar expressions. We believe our approach will be naturally applicable to array
formulas, thereby removing many of the current difficulties in taking full advantage
of them.

7.2.3 Graphical Construction of Formulas

As mentioned above, typing an expression in the formula bar is not the only option
for creating and modifying a formula. All commercial spreadsheet implementations
offer a variety of alternative methods for achieving this effect, most of which
270 7 The Traditional Interface

Fig. 7.8 Building a formula using the mouse

take advantage of and contribute to the intuitive graphical user interface of these
applications. We will now examine these alternatives to writing textual formulas. We
begin with approaches that exploit the ability to point to cells and select ranges using
the mouse to simplify the construction of formulas. Then, we examine methods that
rely on dialogs to assist the user in the task of entering functions.
Excel and other products make several graphical modalities available to the user
for working with formulas. Consider for example the expense report displayed in
Fig. 7.8. In order to add up the items in cells B2 through B7, the user can of course
type the formula “D SUM.B2WB7/” in the formula input box. She can also type
just “D SUM.” and then select the range B2:B7 with the mouse: this will have the
effect of extending this formula prefix to “D SUM.B2WB7”. She will then need
only to close the parenthesis. Any reference can be included within a formula using
this method. This approach offers several benefits over textual input: first, it is often
faster and more natural; second, it visually captures the intended reference rather
than going through their mental translation via cell coordinates. This often results
in formulas with fewer mistakes.
Traditional spreadsheets also make available graphical methods for selecting the
functions that one wants to include in a formula. This does not apply however
to infix operators such as  of C, but only to functional operators. We will now
examine the most prominent options.
Because of its frequent use, SUM has been assigned to a particular button in the
default toolbar configuration: it is marked with a † and generally known as autosum
(see Fig. 7.5). Pressing the autosum button proposes a formula that adds together all
the numerical values above or to the left of the active cell. The user can then modify
this range with her mouse or keyboard.
7.2 Formulas 271

Fig. 7.9 Constructing a formula using a dialog

Once the user starts entering a formula in a cell, even just the ‘D’ sign, the
formula bar changes aspect as shown in Fig. 7.5. In particular, the leftmost textbox,
which normally displays the coordinates to the active cell, is replaced with a pull-
down menu containing the most common functions. Selecting any of these entries
inserts a template for the corresponding function and brings up a dialog with a
description of the function itself, as well as input boxes for each of its arguments.
The user can then either type the intended expression for each argument, or select it
using the mouse. It is not possible to use this same mechanism recursively to embed
other functions within an argument, although one can always type them manually.
Figure 7.9 shows an example of this capability by extending the formula in Fig. 7.8
with an additional component aimed at accounting for the per diem portion of the
expense report: it states that if the number of days of travel (held in cell A8) is
greater than 30, then the user receives double per diem, otherwise single. Here, the
constant “PerDiem” is a defined name for a cell, as discussed in Sect. 7.1.1.
The entire library of functions available in the spreadsheet can be retrieved by
pressing the insert function button, bearing the icon fx in the default toolbar, or
from the appropriate menu. This brings up a dialog which organizes the available
functions into categories for easy browsing and displays a brief summary of what
each does. Selecting a function brings up the same type of input dialog already seen
in the case of IF.
Several programming languages and environments offer visual methods for
constructing formulas, and actually programs. There, icons representing operations
can be chained by means of input/output connectors to visually build expressions.
272 7 The Traditional Interface

No commercial realization of the traditional spreadsheet offers any such tools. This
is not completely surprising since they have been received with mixed reviews over
the years [GP96].
Graphical methods for constructing and modifying formulas are a welcome
alternative to textual input, in particular when working with complex formulas or
with unfamiliar operations. Since our deductive extension attaches to a large extent
to the language of formulas and because clausal theories may appear unfamiliar at
first, it is important to aggressively extend the graphical input methods to these new
constructions. A good part of Chap. 9 will be dedicated precisely to this task, which
can be conducted in a fairly noninvasive manner.

7.2.4 Copy and Paste of Formulas

The graphical construction methods just examined greatly simplify the process of
building formulas, especially complex formulas. It is, however, frequent that the
same expression needs to be applied over and over to all the data in a list. We have
seen an example in Fig. 7.6, where an array formula was used to iterate the Celsius-
to-Fahrenheit computation to a whole range of input values. An alternative approach
to achieving the same effect is to produce one instance of the desired formula and
copy it in correspondence with all the input data. This is demonstrated on the same
problem in Fig. 7.10: cell B16 simply contains the value 5; cell C16 holds the
formula “D B16 C 1” which evaluates to 4; and cell B17 contains our original
formula, “D B16  9=5 C 32”. A copy of the contents of this cell has been pasted
into cell C17. However, a close examination reveals an altered formula: “D C16 
9=5C32”, which correctly computes the Fahrenheit equivalent of the temperature in
C16. When copying a formula to a new location, all contained references are indeed
relativized to the target cell. Figure 7.10 takes further advantage of this property:
column C has been selected and pasted over columns D through AX. Therefore,
cell I16 contains the formula “D H16 C 1”, for example, while I17 contains “D
I16  9=5 C 32”.
This clever copy-and-paste mechanism is enabled by the fact that formulas are
internally represented using a relativized variant of the R1C1 notation. The formula
“D B16  9=5 C 32” is actually stored in cell B17 as “D RŒ1C  9=5 C 32”.
The syntax RŒ1C identifies the cell that is one row above the active cell, and on
the same column, i.e., B16 in A1 notation. Because reference C16 in the formula
“D C16  9=5 C 32” bears the same relation to hosting cell C17, the R1C1 form of
this formula is again “D RŒ1C  9=5 C 32”. Indeed, every cell in the range
B17:AX17 contains this same formula, even it they all appear different in the
standard A1 notation. By the same argument, cells C16 through AX16 all contain
one identical formula, “D RC Œ1 C 1”, where the syntax RC Œ1 refers to the cell
to the immediate left and on the same row as the current cell.
The references we have encountered in these examples are relative since the cell
they identify depends on the location where the reference appears. It is occasionally
7.2 Formulas 273

Fig. 7.10 Cut and paste involving formulas

necessary to rely on absolute references, which identify the same cell independently
from where their hosting formula is located. This is achieved in the R1C1 notation
by simply using actual R1C1 addresses rather than displacements in square brackets.
For example, using absolute references in cell F17 (i.e., R17C6) the formula “D
RŒ1C  9=5 C 32” would present itself as “D R16C 6  9=5 C 32”: copying it
elsewhere would still maintain the reference to cell F16 (i.e., R16C6). References
can be partial, with for example the column expressed as a displacement and the row
in absolute terms, or vice versa. Absolute and hybrid references are also available
in the A1 notation by prefixing each coordinate that one wants to keep fixed with a
$ character. Our last example then becomes “D $F $16  9=5 C 32”.
The intuitive simplicity of the copy-and-paste mechanism over formulas has
made it the preferred method of iterating computations over a range of data. By
contrast, the alternative approach, using array formulas, requires planning and a
sequence of actions that is not as natural. It is also not as forgiving in that it is likely
that the user will do something wrong, hence prompting an annoying error dialog.
To the further detriment of the current array formula interface, these two approaches
do not work well together. Indeed, any attempt to use the copy-and-paste method on
a region intersecting an array formula will result in an error notification. This is
particularly unfortunate as it is common practice to use the copy-and-paste method
to extend a list and the data derived from it. For example, we could easily extend
the example in Fig. 7.10 past column AX by simply copying one of its columns
and pasting it at the end of this range, possibly over several columns. This would
not work in Fig. 7.7 as the size of the array formula was fixed when we pressed
CTRLCSHIFTCENTER. To achieve the same effect, we would need to delete this
formula and reenter it in the new range.
The clever copy-and-paste mechanism of formulas we just described has been
available in spreadsheet implementations since the early days of this technology,
and is clearly here to stay since it supports an increased user productivity at nearly
no cognitive cost. The extension of the user interface required by the deductive
spreadsheet will not alter this state of affairs, and indeed avail itself of this useful
option. We will tap into this mechanism to copy-and-paste clausal definitions
around a worksheet and between worksheets, honoring relative, absolute and hybrid
references as expected. We will also allow this mechanism to extend a relation
274 7 The Traditional Interface

or predicate with new records, even when calculated predicates are involved. This
will distinguish our approach from the current treatment of array formulas, which
resemble defined predicates.

7.3 Controlling Evaluation

Populating a spreadsheet with data and formulas acting on them does not always
solve the problem at hand once and for all. Indeed, there are a number of reasons
for revising a spreadsheet, which includes correcting conceptual mistakes and
maintaining the data up to date. It is often the nature of a spreadsheet to be
continuously updated, as for example with the status of one’s bank account. In this
section, we examine aspects of the more general issue of how the user can interact
with the evaluation engine to develop the solution of her problem. In particular,
we look at update propagation in Sect. 7.3.1 and at the explanation or debugging
facilities available in commercial products in Sect. 7.3.2.

7.3.1 Updates

By default, each time the content of a cell is modified, any formula depending
on it is automatically recalculated, hence accurately reflecting the new state of the
spreadsheet. We have examined the details of the underlying technology in Chap. 3.
This behavior is indeed what users expect and have come to desire. However, the
overhead due to recalculation can become noticeable, if not unbearable, for very
complex spreadsheets. For this reason, commercial applications provide a number
of alternatives. The opposite end of the spectrum is given by manual updates: there,
no recalculation is performed unless the user requires it by pressing a specific
key (in Excel, CTRLCALTCF9 for the current worksheet, SHIFTCF9 for the
current workbook, and F9 for all open workbooks). Intermediate options allow the
automatic calculation of all data except for those in a particular type of table used
for what-if analysis, the so-called “data tables” (see Sect. 7.4). Further options are
available for fine-tuning recalculation; however they are not relevant to the present
discussion.
The deductive extension of the traditional spreadsheet does not per se influence
the chosen recalculation modalities. The intricate dependencies and potentially large
defined relations available in the deductive spreadsheet however require care in
implementing efficient update algorithms and in choosing the default recalculation
policy. Details of these issues have been examined in Chap. 4 and will be further
reviewed in Chap. 9.
7.3 Controlling Evaluation 275

Fig. 7.11 Color-coded visualization of the references embedded in a formula

7.3.2 Explanation

In spite of their widespread popularity and undeniable utility, spreadsheets have

been shown to be rigged with errors of all kinds [BG87, Pan98]. One of the features
that users seem to have the greatest difficulty with are formulas. Spreadsheet
developers have attempted to address this problem from several fronts. We have
already described in Sect. 7.2.3 various graphical input methods for formulas.
Because of the immediate visual feedback they provide, these instruments have
the effect of preventing common input errors during the entry phase of a formula.
We will now describe the available tools for verifying the correctness of a formula
after it has been entered, often several weeks later or by a different user. We call
these mechanisms explanation, although Excel refers to them as “auditing”, Calc as
“detective”, and a programmer may think about them as debugging.
A first line of defense against the errors in a formula is simply inspection. This
is facilitated in commercial products by the fact that positioning the mouse on
the formula input box will highlight all the cells referenced in this formula using
different colors for easy visualization. Figure 7.11 takes this step with the formula
contained in cell B8 in the expense report example. On a color print of this book,
the range B2:B7 is outlined in blue, cell A8 is purple, and the cell E2 symbolically
named “PerDiem” is identified by a green border. These useful outlines disappear
as soon as the user starts editing the formula.
Another useful debugging tool in Excel is the audit toolbar displayed in the right
bottom corner of Fig. 7.12. Among other features, it allows tracing the locations
276 7 The Traditional Interface

Fig. 7.12 Tracing the dependents of a cell using the audit tool

that the active cells depends on. Again looking at our expense report example,
Fig. 7.12 displays arrows from all feeder cells into cell B8. It also displays indirect
dependencies, for example the fact that the value of cell A8 is calculated on the basis
of the contents of cells A2 and A7. The audit toolbar also provides methods to trace
dependents forward, and several more options. The audit arrows are uniformly blue,
while blue borders identify embedded ranges. Color-coding is not used. Indeed, it
is unfortunate that this set of tools and the inspection facility described in the last
paragraph are not better coordinated.
The intermediate values of any functional subexpressions of a formula and
their arguments can be inspected using the function insertion dialog discussed in
Sect. 7.2.3. This is visible in the central and right parts of Fig. 7.9. Using this
feature for debugging purposes requires some dexterity and it is available only for
functional subformulas. In particular, this method is not available for verifying the
value of partial arithmetic expressions such as “D B16  9=5” in our unit conversion
example.
The traditional explanation and debugging facilities just examined are extended
in the deductive spreadsheet to permit tracing the returned values of a clausal theory.
The technical underpinnings of this feature were described in Chaps. 5 and 6. This
clause-specific facilities will be integrated seamlessly into existing support. We
should remark that the traditional debugging facilities leave room for improvement,
although it is not our task to engineer better solutions.
7.5 Summary 277

7.4 Productivity Facilities

This chapter touched on the core interface functionalities of a traditional spread-

sheet. Even in that respect, we shall not claim completeness as we examined only
the aspects that are more closely related to our extension in Chap. 9. On top of this,
commercial applications offer a wealth of facilities for augmenting productivity and
improving the user’s experience. Impressive graphs and charts can be constructed
with a few mouse clicks to provide an effective representation of otherwise tabular
data. Several tools are available to support various types of what-if analysis: in
Excel, data tables allow running a formula on a number of possible input values,
scenarios name chosen groups of input/output values, goal seeking finds the input
values that enable a given desired output, hence reversing the normal data flow of
a worksheet, and pivot tables provide a convenient graphical way to summarize
information and find relationships inherent in a group of cells. Lately, a lot of
emphasis has been put on interoperability with other applications and interaction
with the Web. Finally, all traditional spreadsheet applications provide an escape to
the programming world that allows extending default functionalities with self-made
programs or purchased add-ons. For example, Microsoft supports programming
language called Visual Basic for Applications (VBA) that allows extending Excel
with new functionalities. Dedicated libraries allow interacting with some of the
graphical elements in the interface.
The deductive extension of the spreadsheet will not change this state of affairs,
and actually enrich it in the long run with additional tools aimed at visualizing
certain types of recursive data.

7.5 Summary

In this chapter, we have dissected the user interface of the traditional spreadsheet, or
at least the functionalities that are most relevant to our extension. As we were doing
so, we noted the principal points that required intervention to support the deductive
machinery developed here. The main extensions will concern:
• The definition and manipulation of tabular data.
• The syntax of a textual language for clausal theories.
• An efficient input mechanism for clausal theories, which will also be available
for array formulas.
• The development of effective graphical means for constructing clausal formulas.
• The extension of the traditional explanation mechanism to clausally derived data.
All of these issues and a few more will be addressed in Chap. 9.
Chapter 8
Cognitive Interface Design

As we saw in Chap. 6, the deductive spreadsheet allows solving a significantly

larger class of problems than its traditional counterpart. At the onset, this is
certainly beneficial to the user given that no office automation tool is readily
available for these common problems. The added expressive power will however
be wasted unless users find the deductive extension as easy to use as the traditional
functionalities of the spreadsheet. This is where the user interface comes into play
as its role is precisely to provide intuitive access to useful but possibly complex
functionalities. As we extend the user interface of the traditional spreadsheet to
make this augmented expressive power available, usability shall be a fundamental
concern. In particular, quoting [PBB03], the design process shall “maintain a sharp
focus on the people who are intended to benefit from the new [omit] features”: the
millions of current users of Microsoft Excel and other spreadsheet applications.
How to go about this? While designing good user interfaces is still more an
art than a science, cognitive psychology has produced effective methodologies
for evaluating their usability. Of particular interest to us, as we prepare to design
the user interface of the deductive spreadsheet, are recent advances such as the
cognitive dimensions of notations and the attention investment model, which provide
developers themselves with evaluation guidelines that can be used at the design stage
of a user interface. These methods have been applied with great success by Simon
Peyton-Jones, Alan Blackwell and Margaret Burnett [PBB03], who used them to
develop an orthogonal extension of the traditional spreadsheet. Once the design is
completed, more traditional and thorough approaches, centered around user testing,
can attest of the effectiveness of individual design choices. A very preliminary such
assessment will occupy Chap. 10.
In this chapter, we review the above mentioned cognitive approaches to designing
a user interface in preparation to applying them to the deductive spreadsheet in
Chap. 9. We begin in Sect. 8.1 by locating them within the larger area of cognitive
psychology that studies the usability of graphical user interfaces. Section 8.2
examines in some details the cognitive dimensions of notation approach, while
Sect. 8.3 focuses on the complementary attention investment model. Section 8.4 uses
these theories to precisely identify the target audience of the deductive spreadsheet.

I. Cervesato, The Deductive Spreadsheet, Cognitive Technologies, 279

DOI 10.1007/978-3-642-37747-1 8, © Springer-Verlag Berlin Heidelberg 2013
280 8 Cognitive Interface Design

Section 8.5 concludes with a summary of the notions introduced in this chapter and
relates them to the cognitive requirements given in Chap. 2.

8.1 Interface Usability Evaluation

Usability studies are an important aspect of any research and development effort that
targets end-users (as opposed to experts who are often more tolerant and adaptable).
Such investigations have thus become common practice since the 1990s [Nie93].
At the early stages of development, usability studies evaluate and quantify the
time savings and processing costs of a proposed functionality, and allow comparing
these parameters for various approaches to presenting these functionalities to a user.
At a later stage, they allow establishing the ease of learning, efficiency of use, ease
of remembering, number of errors, and subjective evaluations of a tool.
Cognitive psychology provides user interface designers with a number of tools
for assessing the usability of an individual feature or an entire prototype of a system.
We will now examine three general approaches, commenting on their advantages,
their drawbacks, and in particular the point in the design cycle of the interface in
which they apply. This book will make use of two of them, but future developments
will take advantage of all of them.

8.1.1 User Testing

Maybe the best known approach to assessing the usability of a user interface is
testing. A representative group of real users is monitored, often using electronic
recording software, while performing predetermined tasks on the target system.
Human-computer interaction experts compound the reactions, difficulties and over-
all performance of these users into an evaluation of individual aspects of the
interface, which is then passed to the designers to revise decisions that caused
unsatisfactory results. This process is iterated until all aspects of the design are
judged acceptable.
This approach is very valuable as it provides feedback that directly captures
the experience of actual users. It has also a number of drawbacks. It is expensive
as it involves a large number of individuals to be really effective, as well as a
controlled environment. It also takes a long time to perform, often several weeks for
an individual round, which has a negative impact on the time to market of a product.
Moreover, because the experiments occur in a controlled laboratory and the tasks
are predetermined, it is somewhat artificial as users may be more self-conscious
than in their natural environment. Finally, the mediation of the human-computer
interaction expert introduces an indirection into the feedback loop to the interface
designer. Because of these rather substantial drawbacks, it is important that user
testing be performed on a solid prototype interface that is judged unlikely to require
many changes.
8.1 Interface Usability Evaluation 281

The widespread availability of always-on broadband Internet connection enables

forms of user testing that are largely immune to these drawbacks. Indeed, a user can
consent to have the developer of an application collect usage statistics, in particular
about the time it takes for a series of related actions to be performed and about
backtracking patterns where the user undoes activities he or she has just done. This
type of user testing is transparent to a consenting user, and lacks the awkwardness
and artificiality of prescribed tasks. The one thing it cannot predict exactly is the
intent of the user, although statistical methods are now good at classifying series
of observations into recurring patterns. This approach is particularly suited to Web-
based applications along the lines of Google Spreadsheet.

8.1.2 Predictive User Models

Cognitive approaches based on predictive user models do not rely on users. Instead,
they consider core tasks of the system and compile a detailed chart of all the possible
ways to achieve each of them. If time estimates are associated with actions, they
provide a quantitative measure of how long it takes for a user to perform a given task.
They also summarize the knowledge or proficiency required of a user to perform
each individual task as the chart considers all the actions involved in achieving
it. The most widely used predictive methods are heuristic evaluation [NM94] and
cognitive walkthroughs [LRB91]—both discussed in more detail next—with GOMS
(Goals, Operators, Methods, and Selection rules) [OO90], an older, time-tested
method, worth mentioning as well.
Heuristic evaluation is a relatively simple but very effective usability method—
indeed, one of the most commonly used today [NM94]. This method relies on
expert evaluators, who examine the interface or prototype and judge its compliance
with recognized usability principles called heuristics. The main heuristics [NM94]
include visibility of system status (can the user always tell the state the system is in?
are there feedback mechanisms to assess the effect of an action?), match between
the system and the real world (does the system communicate with the user through
language, concepts and layouts that the user will relate to?), user control and
freedom (if the user makes a mistake, is there an easy way to undo it?), consistency
and standards (are similar features provided in similar ways?), help diagnose and
recover from errors (are error messages precise and constructive?), error prevention
(are there situations that are particularly error-prone?), recognition rather than
recall (are commands and information needed to perform actions visible or easily
accessible? does the user have to rely on memory to perform a sequence of tasks?),
flexibility and efficiency of use (are shortcuts available to experienced users?),
aesthetic and minimalistic design (is there irrelevant information that distracts
the user from accomplishing a task?), and help and documentation (is it easy
to find precise instructions to perform complex tasks?). Several of these features
transpire in the lightweight methods discussed in Sect. 8.1.3 and apply to design the
282 8 Cognitive Interface Design

user interface of the deductive spreadsheet in Chap. 9. The evaluators’ actions are
captured using electronic recording software similar to what used for user testing—
such tools are indeed very common in psychological research. These recordings
are then evaluated in order to infer how the actions of the evaluator are related to
the usability heuristic of interface design. The output of the heuristic evaluation
method is a list of usability issues in the interface with references to those usability
principles that were violated by the design in the opinion of the evaluator. Each
usability issue is linked to the particular heuristics. This method is also used to assess
the relative severity of the individual problems, which is useful in order to allocate
resources to fix the most serious issues and also provides an estimate of the needed
efforts. The severity of a usability issue is a combination of the frequency with which
the problem occurs, the impact of the problem if it occurs, and the persistence of
the problem (if it is a one-time problem or a repeated problem).
The cognitive walkthrough [LRB91] helps evaluate a system for ease of learning,
particularly by exploration. The evaluators in this case need to have significant
experience with the system. The cognitive walkthrough of a system or part has
two phases: a preparatory phase and an analysis phase. In the preparatory phase,
an evaluator lists a set of tasks, the action sequence for each task, the interface and
the information on how to do the walkthrough. Electronic recording software is
used to record all activities. Then, the analysis phase examines each action in the
solution path and attempts to give the reasons as to why users would choose that
action. Participants evaluate the interface and select actions they think accomplish
the task. Then, they observe the interface reactions to see if their actions have
the desired effect, and determine what action to take next. Questions such as the
following are asked during the analysis phase: “Did the user try to achieve the right
effect?”, “Did the user notice that the correct action was available?”, “Did the user
associate the correct action with the effect trying to be achieved?”, “Did the user see
that progress was being made when the correct action was performed?”. A subset
of these questions has been found to be sufficient in some cases [Spe00]. The
cognitive walkthrough method identifies the problems with the design and explains
the reasons for these problems. The walkthrough can be used to identify questions
that can be answered by specific, focused user testing, such as questions about user
terminology and task description.
Predictive approaches have proved extremely successful in organizing help
systems, training programs, and user documentation. They are also a useful
decision support for comparing alternative product choices against the needs of an
organization. They have however a few drawbacks of their own. Although much
cheaper and quicker than user testing, constructing a GOMS model of a system is
still expensive and requires the supervision of a human-computer interface expert.
Furthermore, this approach is only applicable on a finished system as measurements
depend on the perceptual characteristics of the interface, especially if timing is
considered. Finally, by concentrating on the steps to achieve a task, it tends to ignore
issues such as user skills (or lack thereof), mental workload, and errors.
8.2 Cognitive Dimensions of Notation 283

8.1.3 Lightweight Approximate Methods

This heading collects a number of recent approaches intended to provide useful

guidelines to application designers as they develop the user interface of a system.
They are lightweight in that they give tools to the designers themselves, who
may not be experts in human-interface interaction and know little about cognitive
psychology, to help them produce an adequate first cut of the user interface of
the target system. They are approximate in the sense that they do not guarantee
the effectiveness of the resulting design, which will have to be assessed at a later
stage using one of the two methodologies outlined above (or both). All in all, these
methods provide the designer with a vocabulary for concepts and objectives that
are important during the process of using an application and solving a problem
with it. By being able to articulate them as design attributes or strategic goals, he
can actively analyze them as an integral part of every decision rather than rely on
intuition or guesses.
Following the successful steps of [PBB03], we will adopt two complementary
such methods as we design the integration of the proposed deductive functionalities
into the user interface of the traditional spreadsheet. The attention investment
model [Bla02, BB02], reviewed in Sect. 8.3, defines the interaction between a
user and an application in psycho-economic terms, hence providing a high-level
framework to scrutinize design decision. The cognitive dimensions of notation
model [Gre89, GP96], examined in the next section, verbalizes a number of useful
design attributes that the designer can explicitly try to balance when deciding on
how to provide a functionality. We will use guidelines distilled from both approaches
in Chap. 9 to design the interface of the deductive spreadsheet.
Note again that lightweight methods are not meant to replace traditional
approaches such as testing and predictive modeling, which apply to a completed
design (they are summative). Rather, they complement them by helping a developer
make more informed decisions during the design phase of the interface (they are
formative), although they can also be applied to make a quick assessment of a
finished prototype. Indeed, while they will guide our design, we also perform a
preliminary set of testing experiments on users, which is reported in Chap. 10.

8.2 Cognitive Dimensions of Notation

The critical observation at the basis of the cognitive dimensions of notations,

originally proposed by Thomas Green in [Gre89], is that the designers of a
system, language or computer interface often lack names to talk about some of the
cognitive concepts they use, especially those concepts that most directly impact
the end-user. Lacking a way to talk about them, designers do not explicitly take
them into account when making decisions that will affect users, and must instead
rely on intuition, often resulting in sub-optimal choices. Providing a set of shared
284 8 Cognitive Interface Design

names, or more properly discussion tools, for concepts that are vaguely known
but unformulated has a number of advantages [Gre96a]: it promotes a higher level
of discourse, it facilitates analysis, it helps making decisions, it creates goals and
aspirations, it provides the basis of informed critique, it allows the interrelationship
of concepts to be appreciated, and it encourages the reuse of ideas in new contexts.
Furthermore, discussion tools can be used as a checklist to make sure that no
important concept lacked the proper consideration.

8.2.1 Discussion Tools for Visual Languages

The cognitive dimensions of notations framework [GP96, BG03, BBCC 01] is a

concrete, although evolving, set of discussion tools intended for the designers of user
interfaces and other user-oriented languages to make quick informed user-centered
decisions as they perform their work. In particular, they can give the designer a fairly
good idea of how users will fare. They can indeed help anyone who is not a human-
computer interaction expert evaluate the quality of a visual language as perceived
by its intended users (experts have other tools at their disposal). Users themselves
can rely on them to compare systems and effectively talk to designers.
Before we go into details about this armory of discussion tools, it is important to
fix the frame of reference and give some terminology [Gre89, Gre96b]. The particu-
lar problem that a user may want to solve or express is an information structure: both
a grocery list and next year’s budget are information structures, but so are a melody
and a computer program. The instrument used to externalize the problem outside
of the user’s head in view of solving it is called an environment: for us it will
be a spreadsheet application such as Microsoft Excel. Note that environments are
specialized to certain information structures—indeed Excel is not the right tool for
composing music or doing general programming—and that the same information
structure can be captured by different environments: many grocery lists are written
down using pen and paper, and the same used to be true for budgets up to a few years
ago. The syntax of the information structure is generally referred to as notation.
Observe that this term is not specific to an environment as the same notation may be
shared between several environments: we have seen in Chap. 7 that most spreadsheet
applications have nearly identical user interfaces, and their general organization is
not too different from an accountant’s book, which they originally tried to automate.
In the context of a spreadsheet, the notation will mostly refer to the layout of
information as a grid of cells and the syntax of formulas. Finally, the word system
refers to the combination of a notation and an environment, independently of the
information structure the user will rely on it for.
The particular discussion tools of this framework are called cognitive dimensions.
Different papers in the literature mention different dimensions. Below, we will
examine a core set of 13 of them, which are nearly undisputed and have recurrently
been used to evaluate and design systems [PBB03, GP96]. They are alphabetically
summarized in Fig. 8.2 which is partially borrowed from [PBB03]. For example,
8.2 Cognitive Dimensions of Notation 285

Fig. 8.1 Schematic effect of Dimensions

changing a system along a
dimension

Notation Environment
System

viscosity is a verbalization of how hard it is to make a small change. It can be

a property of the notation (e.g., by forcing the user to repeat the same syntactic
element over and over) but also of the environment (e.g., by not providing a global
search and replace facility). Indeed, the scope of each dimension is the system as a
whole, i.e., the match between the notation and the environment.
A dimension provides a scale on which to make a rough measurement: a system
can be more or less viscose for example. Generally speaking, there is no absolute
rule that says that one end of the scale is not preferable to the other. Viscose systems
resist change, which is often bad, but this is sometimes a useful guard against
dangerous operations. The point where a feature should fall on the scale depends
therefore on the application. Thus it is the job of the designer to decide how to
tune each dimension in consideration of the intended use of the system and its
target audience. There is a catch however: adjusting one dimension, for example
by altering the notation or changing environment, will invariably influence other
dimensions (this is the so-called law of cussedness [Gre96a]—Fig. 8.1 describes it
schematically). For example, reducing viscosity often has the effect of complicating
the system by requiring the user to learn new concepts, hence increasing abstraction,
another dimension. The designer has then to compare the drawbacks of viscosity
as it currently manifests in the system with the drawbacks of reducing it. The
interrelationship between dimensions is expressed in terms of trade-offs, which
describe how modifying one dimension may affect other dimensions. A partial chart
of trade-offs, borrowed from [Gre96a], is displayed in Fig. 8.3. Because of the law
of cussedness, a major challenge is selecting the right combination of attributes for
the target functionality and audience, i.e., striking the right balance of cognitive
dimensions for the intended application. Note again that the cognitive dimensions
of notation framework does not prescribe how to go about making a system more
usable, but only gives the designer a vocabulary to reason about alternatives, as well
as some warning about the effect of changes.

8.2.2 Principal Cognitive Dimensions

We will now review each dimension in Fig. 8.2, writing them in a slanted font for
easy identification. We will describe each of them on the basis of examples taken
286 8 Cognitive Interface Design

Dimension Intuitive Meaning

Abstraction Gradient What are the minimum and maximum levels of ab-
straction? Can parts of the problem be encapsualated?

Closeness to Mapping How far is the problem statement from the model
of its solution?

Consistency When some of the language has been learned, how

much of the rest can be inferred?

Diffuseness/Terseness How many constructs are there to learn?

Error-Proneness Does the design of the notation induce ‘careless

mistakes’?

Hard Mental Operations Does the language force convoluted constructions

for simple concepts?

Hidden Dependencies Is every dependency overtly indicated in both di-

rections? Is the indication perceptual or only symbolic?

Premature Commitment Does the user have to make decisions before she has
all the information she needs?

Progressive Evaluation Can a partially modeled problem produce results to

obtain feedback on progress toward the over all solution?

Role Expressiveness Can a reader see how each component of a model

relates to the whole?

Secondary Notation Is there a way to insert comments and reminders

outside of the executable notation?

Viscosity How much effort is required to perform a single

change?

Visibility and Juxtaposability Is every part of the model simultaneously visible?

Can any two parts be put side by side?

Fig. 8.2 Major cognitive dimensions

from the spreadsheet world, and occasionally use it to evaluate some of the specific
choices reviewed in Chap. 7. We also briefly describe how our intended extension
to the user interface of the traditional spreadsheet will affect this dimension. In
this respect, it is important to remember that we are bound by the decisions that
were made when the traditional spreadsheet was originally designed. Not only
shall we refrain from making any change that would alter the usability of existing
functionalities (our extension shall be conservative), but any addition shall strike the
same balance of dimensions as current features. In particular, using our deductive
extension shall “feel the same”, in a certain sense, as using traditional aspects
of the spreadsheet. Therefore, we will refrain from deviating from the status quo
in any major way. Paraphrasing the important consistency dimension that will be
introduced shortly, this is a form of “meta-consistency”.
8.2 Cognitive Dimensions of Notation 287

can
increase Secondary
increase Viscosity
cost
notation

Premature can
reduce
commitment
can
increase Abstraction can
increase

can Hidden
increase dependencies
complex
Visibility

Fig. 8.3 Selected trade-offs among dimensions

Abstraction Gradient

Any time there is a way to realize a sequence of functionalities with a single action,
we have an abstraction. For example, the possibility of selecting multiple worksheet
cells for manipulation is an abstraction over working with individual locations,
and similarly the ability of selecting rows abstracts scrolling a selection till the
last column. User-defined macros are an even more obvious example. The data list
and data form automatically generated for a group of columns in Excel abstract
these columns as a table. Spreadsheet formulas can be seen as a parametric form
of abstraction in that not only do they compound several operators, so that they can
be manipulated as a single object, for example via copy-and-paste, but also their
result varies with the value of the cells they reference. Spreadsheets support fairly
mild forms of abstraction, in particular when compared to programming languages.
More expressive forms of abstraction, e.g., user-defined functions, have recently
been proposed [PBB03, BADC 01].
Abstraction may appear to be a good thing, unconditionally. In fact, each time
we introduce an abstraction, we are asking the user to learn a new command if not a
new concept. Complex abstractions can however be overwhelming and therefore
detrimental to productivity. Whenever the abstraction is part of the notation,
as references in formulas, it can lead to added hidden dependencies (another
dimension, see below), but on the other hand it can reduce viscosity. Figure 8.3
describes further trade-offs involving abstraction.
The user interface of the deductive spreadsheet will for the most part leverage
existing abstractions. It will also extend the data list abstraction to unsorted
relations, and provide clausal theories as further abstractions similar in spirit to
scalar formulas.
288 8 Cognitive Interface Design

Closeness to Mapping

This dimension attempts to express how distant the notation is with respect to
the problems it is meant to solve. Considering that a spreadsheet is an electronic
rendering of an accountant’s book, this formalism has very high closeness to
mapping for most types of tabular data. Of course, one may use a spreadsheet
for solving problems that are not conveniently cast in this format, which shows
poor judgments rather than deficiencies of these applications in terms of closeness
to mapping. Because the deductive extension is inherently tabular, it will take
maximum advantage of the current state of affairs.

Consistency

A notation is consistent if related functionalities are invoked using related actions.

For example, the fact that one can click on the row header to select a row is
consistent with the fact that clicking on the column header selects a column.
A consistent notation benefits the user as she can infer the command that activates
a functionality on the basis of her knowledge of the commands that trigger related
effects, as in this example. Therefore, consistency is about how easy it is for the
user to extend her knowledge of the notation when she has learned some aspects.
The user interface of traditional spreadsheet applications has excellent consistency,
with a few exceptions, entering array formulas being an unfortunate example.
Consistency is invariably seen as a good property, unless it shall be sacrificed
to prioritize other requirements. One of the key requirements of the deductive
spreadsheet is to maintain, if not enhance, the consistency of the traditional
interface.

Diffuseness/Terseness

This dimension measures the verbosity of the notation, and therefore the space it
takes to use it to solve a problem. A succinct economical interface is terse and
has the advantage of allowing a larger part of the model to be visible at any time.
A diffuse notation takes up more space, but may be more readable. Traditional
spreadsheets tend to strike a good balance between these two extremes by providing
concise notations for formulas and graphical elements, and yet maintaining good
readability. Our deductive extension will not disrupt this order.

Error-Proneness

An error-prone notation is conducive to making careless mistakes. A number

of design decisions cause the traditional spreadsheet to be surprisingly error-
prone [Pan98]. One culprit is the syntax of references used in formulas, as it makes
8.2 Cognitive Dimensions of Notation 289

it easy to mistype cell coordinates, which sometimes results in rather subtle errors.
Using the graphical approaches to building formulas mitigates this problem, as does
using named cells. Spreadsheets are an example of visual notation in which high
error-proneness, although undesirable as an isolated cognitive dimension, has been
judged acceptable in the system as a whole. Indeed, making spreadsheets more
resistant to errors would compromise the usability of these tools.
Our deductive extension will be neutral with respect to this dimension. It will
support all the devices listed above to reduce the error-proneness of formula
construction and make them available in the logical context, but it will not force
them on the user. It is however expected that users will find building clausal theories
graphically, or on the basis of symbolic names, easier than entering cell ranges.

Hard Mental Operations

The notation contains hard mental operations if some simple concepts cannot be
expressed but by relying on complicated constructions. We have encountered one
instance in Chap. 6, where the base logical language of the deductive spreadsheet
forced expressing extensional “for all” statements by means of cascaded negations.
The issue was solved by providing an abbreviation, FORALL, as an additional
syntax element (we changed the notation). This had the effect of increasing
abstraction and diffuseness.
Other hard mental operations are primitives that invariably send the user to the
help menu. For example, the “database functions” implemented in most spreadsheet
applications fall into this category as they take unintuitive arguments such as
conditional tests (they are used to selectively aggregate records in data lists). Our
deductive extension provides a simple alternative to solving these same problems
(hence increasing abstraction ).
Hard mental operations are error-prone especially if they are used frequently.
Therefore eliminating them is desirable, as long that they are not replaced by a
more damaging combination of dimensions. This is the heuristic we have followed
in Chap. 6 when introducing FORALL, and that we will continue to follow while
designing the user interface of our language.

Hidden Dependencies

This attribute considers whether the notation or interface makes it evident when
parts depend on one another. Spreadsheets have lots of hidden dependencies . By
just looking at the cell grid, it is impossible to tell which cells contain actual values
and which are calculated using formulas. Only selecting a cell will reveal its nature
and, in the latter case, show only the addresses of the cells it directly depends on: the
cells they themselves depend on are unknown, and the cells it feeds to are not shown
either. Of course, the auditing tools we examined in Chap. 7 provide simple ways
to display all this information, but most users are not aware of their existence, they
290 8 Cognitive Interface Design

work only on the active cell, and the uniform blue color for both antecedents and
consequents makes them less useful that they could be.
Hidden dependencies are generally bad as they make the notation error-prone,
a well-documented reality of spreadsheets [Pan98]. On the other hand, a spreadsheet
with all dependencies revealed would be nearly unusable, although other languages
offer this possibility with good benefits [GP96].
Our design for the interface extension of the deductive spreadsheet will attempt
to maintain the status quo, in order not to disrupt user expectations. It will indeed
make use of the very same hidden dependencies currently present in the traditional
spreadsheet. It will also extend the auditing mechanism to work with the added
language fragment.

Premature Commitment

Premature commitment , also called enforced look-ahead, allows asking whether the
user is forced to make a decision before she has, or has thought of, all the needed
information. For the most part, spreadsheet interfaces avoid imposing an order of
action on users. For example, a formula can use references that have not been filled
with data, which will generally yield some innocuous default value. One example of
premature commitment we have encountered is the construction of array formulas.
As we saw in Chap. 7, a user is forced to decide a priori how many cells the array
shall contain, and any later change will essentially force her to undo the original
work and then redo it.
Forcing occasional spreadsheet users and novices to commit prematurely to their
choices would generally be bad as the subsequent frustration is likely to discourage
them from using these tools. On the other hand, professionals may want the system
to help them anticipate problems by requiring them to think ahead. The near absence
of premature commitment required to use a spreadsheet has certainly contributed to
the popularity of these applications by making them easy to use. On the other, hand
it may also be blamed, in part, for the high number of errors found in spreadsheets
[Pan98].
In supplying notation to use the deductive features, we shall adhere to the
general philosophy of the traditional spreadsheet which spares the user premature
commitment whenever possible (except for array formulas).

Progressive Evaluation

As the name implies, progressive evaluation asks how easy it is to execute an

incomplete model and get useful results that can help completing it. This dimension,
clearly derived from the study of programming languages, is inherently supported
by all spreadsheet applications and is indeed a fundamental aspect of their design.
Note that it can be disabled, as we saw in Chap. 7, for example when performing
updates is so slow that it would provide incorrect feedback, or act as a distraction.
Our design shall naturally extend this property to the added deductive features.
8.2 Cognitive Dimensions of Notation 291

Role Expressiveness

Role expressiveness gives a name to the concept of whether the purpose of a

component can easily be related to the whole, that is if the notation gives a cue of the
role of this component within the entire system. With their definition as flat, uniform
collections of cells, spreadsheets are hardly amenable to role expressiveness. This
notion is sometimes not even well founded as the same worksheet may contain
conceptually unrelated parts and therefore be deprived of a “whole”. In practice,
the situation is often different. Indeed, well-designed spreadsheets solve a single
problem, and the discipline of secondary notation often sheds light on how to
interpret an area of the spreadsheet and how it takes part in the overall solution
of the problem. We saw some examples in Chap. 7. Another way to relate a cell to
the whole is to make hidden dependencies explicit through auditing tools.
Role expressiveness is desirable, but as we saw, users generally find ways to
emulate it even if the primary notation does not have any structure to support it
directly. Our deductive extension of the traditional spreadsheet will neither augment
nor lower role expressiveness, but work with what is currently supported.

Secondary Notation

This dimension refers to the possibility of using the inessential aspects of the
notation to convey useful information without affecting operation. Two examples
in the spreadsheet world are cell formatting and range names. By formatting the
contents of a cell (as a currency rather than a number), we are making its intended
meaning clear. We can furthermore mark it in bold, for example to convey that
it is an important value, and use a colored background as a reminder that it is
calculated using a formula. Range names support secondary notation by allowing
the user to choose a meaningful identifier rather than anonymous A1 coordinates.
Spreadsheets provide a lot of support for secondary notation, other examples being
Post-It-looking notes that can be associated with cells, and the use of textual values
whose only purpose is to describe the meaning of adjacent cells.
The availability of secondary notation is of great convenience to users as they
tend to rely on it to convey meaning that is not part of the primary notation of cells
and formulas. It is helpful in reducing error rates, but can however promote viscosity
since it may be yet another thing to change.
The deductive spreadsheet will maintain the same secondary notations already
available in commercial products. Because predicates and relations are more
complex than individual cells, it can help the user label and manage these objects.

Viscosity

Viscosity is the amount of effort required to make a small change. Highly viscose
notations make it hard to implement small conceptual changes. One example in
292 8 Cognitive Interface Design

the traditional spreadsheet is the beloved copy-and-paste command combination.

Assume for example that we entered the formula “D A1  1” in cell A2, and then
copied it through cell A100 just to realize that what we meant was “D A1 C 1”.
Several remedies are available, but all of them imply various amounts of effort and
frustration: we can go through the same process again with the correct formula
this time, which can be tedious with more complex examples; we can manually
change every “C” into a “”, which is mind-numbing; or we can use the search
and replace feature, which is complicated by the use of relative references and can
be error-prone. Array formulas manifest another type of viscosity : not only does
changing the size of the formula require work, but it may also force changing the size
of dependent array formulas. On the other hand, fluid notations facilitate change.
Consider for example the task of highlighting a row in bold. This is simply achieved
by selecting it and pressing the appropriate button on the default toolbar.
In most circumstances, viscosity is a bad thing, although it can occasionally be
used advantageously, for example to protect users from unknowingly performing
dangerous operations. The viscosity of dialogs serves precisely this purpose for
novice users, although they soon becomes annoying once the user has passed this
stage. The viscosity of a feature can be reduced by providing commands to bypass
it. For example, the viscosity of changing the font of a row of cells one by one is
reduced by making multi-cell selection available. The trade-off is a higher form of
abstraction, as the user needs to learn new concepts to use this feature. See Fig. 8.3
for more trade-offs.
In designing our deductive extension in Chap. 9, we shall strive not to add
unnecessary viscosity to the traditional spreadsheet interface.

Visibility and Juxtaposability

The visibility dimension is an attempt to verbalize the degree to which users are
able to quickly access different parts of the data they are using the notation for.
Juxtaposability , included in the same dimension, asks whether two parts can be
placed side-by-side for comparison, or whether to use one part as a model for
another. Assuming a large enough display, spreadsheets excel in both visibility
and juxtaposability. Indeed, the scrollbars and worksheet navigation icons allow
exposing any part of a worksheet in no time. A window can be split by a simple
mouse action in order to juxtapose areas of the same worksheet, horizontally or
vertically. Putting different worksheets side by side can be done using menu options.
As an illustration of a lack of juxtaposability, comparing two formulas is not
possible unless one takes advantage of a seldom-used menu option that allows the
worksheet to display formulas, rather than their results.
Visibility and juxtaposability are generally positive properties, as they tend
to increase productivity. They are less desirable when an inexpert user could do
damage if he had easy access to all parts of the system (for example if he could
tinker with the implementation of the built-in operators) or could put some parts side
by side. For example, a right-click menu item that allowed him to decide whether
individual cells should show a formula or its result would quickly lead to confusion.
8.3 Attention Investment Model 293

Our general goal in Chap. 9 will be to maintain the level of visibility and
juxtaposability already available in the traditional spreadsheet in any worksheet
extension.

Other Dimensions

While the above dimensions are fairly standard, researchers are continuously
proposing new ones with names such as “creative ambiguity”, “specificity” and
“useful awkwardness” [Bla00]. Most of these ideas appear to be rather esoteric
in the context of a spreadsheet and will not be considered further. One proposal,
permissiveness , is quite relevant as it expresses whether the notation provides
several different ways to do the same thing. Spreadsheets are advantageously
permissive, in this sense. Indeed, the most common functionalities can be invoked
from the menu bar, from toolbars, using keyboard shortcuts, from pull-down menus
and more. This is certainly a characteristic that we will strive to maintain in our
extension.
Because the deductive spreadsheet is an extension of its traditional counterpart,
our design has another constraint, a dimension that we may call conservativity : as
we make deductive features available, every aspect of the user interface that pertains
to the traditional functionalities of a spreadsheet shall remain available. Said in
another way, we shall not take anything away from a user who does not need our
extensions. This does not mean that they should be invisible, as this would defeat
the user’s ability to stumble upon new features, hence reducing consistency. Precise
requirements for these different issues were given in Chap. 2.

8.3 Attention Investment Model

Another methodological ingredient of the spreadsheet extension of [PBB03] is

the attention investment model of abstraction use [Bla02, BB02]. Similarly to the
cognitive conditions model just examined, this approach provides the designer of an
application with a vocabulary that can help in making better decisions. Differently
from that model, it does not name attributes of the system (intended again as a
notation together with an environment), but names aspects of the cognitive processes
of the user as she works with the application, especially when confronted with
learning new abstractions. Therefore, it forces the designer to put himself in the
shoes of the users and anticipate how they will cope with aspects of the notation.
Quoting Alan Blackwell, who proposed it, this approach, which we may label as
psycho-economic, “offers a cost/benefit analysis of abstraction use that allows us to
predict the circumstances in which users will choose to engage in [them]” [Bla02].
By anticipating sources of difficulty, it also enables tool designers to provide
features that will facilitate a user’s decision to use or not to use an abstraction.
294 8 Cognitive Interface Design

8.3.1 Overview

The attention investment model analyzes users’ decisions in terms of investment of

attentional effort, in particular concentration [Bla02]. An excellent summary of this
technique is found in [PBB03], which we quote, as we could hardly provide a better
description:
[Users] have a finite supply of time to spend concentrating on their work, or attention, to
invest. They will invest their effort in activities for which their expected payoff exceeds
the cost, unless the risk is too great. The cost of the investment is the amount of attention
by the user that must be devoted to accomplishing a task. The expected payoff from that
investment will be some saving of attentional effort in the future, such as by achieving
a good abstract formulation to reduce the amount of effort required to cope with similar
problems. The perceived risk is the extent to which the user believes that the investment
will not produce the payoff, or that it will lead to even more cost that is not yet apparent.

Consider for example the decision of starting to use a spreadsheet to keep track
of one’s travel expenses as opposed to writing them down on paper. The cost is
the time spent learning to use some basic features of, say, Excel. Note that this
may be substantial for users unfamiliar with personal computers, but minimal for
users who have experience with other visual applications such as word processors.
The immediate payoff is not to have to go through the error-prone process of adding
up expenses by hand. Another more remote payoff is acquiring the skills to use
a spreadsheet for other tasks in the future. The risk is that acquainting oneself
with this new application may take too much time, possibly without really solving
the problem (recall that risk is something perceived by the user, not an objective
assessment).

8.3.2 Attention Investment Analysis of the Deductive

Spreadsheet

With these definitions in hand, let us apply the attention investment model to the
deductive spreadsheet: not to the user interface amendments that will be introduced
in Chap. 9, but to the core functionalities described in Part I of this book. Following
the steps of [PBB03], what we are trying to understand is why a user should even
bother about our deductive extension. Said in another way, is it worth making it
an integral part of the spreadsheet, a tool that has enjoyed unfaltering popularity for
over 30 years even without our extension? In order to establish this, we need to spell
out the payoffs, the costs, and the risks of embracing this extension.
Payoffs: As amply demonstrated in Chap. 6, the most important benefit of our
deductive extension is that it enlarges the class of problems that a user can
use a spreadsheet for. Considering that most home and many office computing
environments do not provide any application that could help with these new
problems, as observed in Chap. 1, the payoff is the ability to have access to
automated assistance in solving them, as opposed to handling them by hand,
8.3 Attention Investment Model 295

or giving up on them. This is identical to the expense report situation used

earlier as an illustration. Problem classes that are not solvable in the traditional
spreadsheet, but that different aspects of our deductive extension can capture, are
summarized in Fig. 6.7 in Chap. 6.
It is interesting to compare this state of affairs with the introduction of array
formulas as an abstraction over a spreadsheet with only scalar formulas. As we
saw at the beginning of Chap. 4, the traditional spreadsheet is equally expressive
with or without array formulas. The payoff is therefore much more modest,
mostly concerned with smaller file sizes and therefore faster file load and save
operations, and the possibility of typing a formula once rather than having to
copy it over a range of cells.
Costs: The cost of the deductive extension is null for a user who does not need
to avail herself of it. For those who have interest in exploring this possibility, for
example because they have a problem in the newly solvable class and want to
automate its solution, using this extension involves several forms of cost:
• First of all, there is the cost of learning to cast a problem in terms of relations,
which is substantially different from the location-based way of organizing
information of traditional spreadsheets. In truth, the magnitude of this cost
depends on the user’s experience: people who have used data lists will find this
mind-set natural, others who often typeset data in a tabular format may have
little difficulties, while users without this experience will have to acquire it.
• Second, there is the cost of working out the problem’s solution in relational
or logical terms. This cost can be substantial as it implies a mental shift of
paradigm with respect to the methods for solving a problem supported by the
traditional spreadsheet. The user must now learn to manipulate entire relations
rather than individual cells. Familiarity with cell ranges, as in the arguments of
SUM, somewhat lowers this cost. In practice, this means learning to think in
terms of logical or relational manipulations as opposed to simple arithmetic
constructions. Note that different elements of the deductive extension have
different cognitive costs, with recursion being probably the most expensive in
the learning process.
• Finally, there is the cost of learning a new syntax for the language fragment
that deals with clausal theories. This new notation is both textual if one uses
the formula entry textbox and graphical as it will be possible to construct
theories by using the mouse.
There is again some similarity with arrays and arrays formulas, with the first
form of cost corresponding to thinking in terms of ranges rather than single cells,
the second to learning to manipulate arrays, and the third to acquiring familiarity
with the textual syntax of cell ranges and the graphical concept of selection.
Risks: The principal risk of the deductive extension is that the time needed to
master the underlying mental and notational infrastructure may exceed the time
required for solving the problem by hand, or the cost of not solving it. Another
perceived risk is that the deductive extension may not be able to cope with the
problem that one is attempting to solve, and therefore the time spent learning to
use it would be wasted.
296 8 Cognitive Interface Design

It is again interesting to compare this with array formulas. The first form of risk
is quite concrete here since this feature does not extend the range or problems
solvable with just scalar formulas. Learning to use array formulas will often
exceed the time needed for a purely scalar solution. The time saving once this
feature has been mastered is not evident either.
Since we are using [PBB03] as inspiration for this methodology, it is useful to
compare these conclusions with their analysis of the extension they propose: adding
infrastructure to allow users to define their own functions. A major difference lies
in the fact that our deductive extension enlarges the class of problems solvable with
a spreadsheet while providing user-defined functions does not. Because the added
benefit of the latter is mostly convenience, users are likely to be willing to sustain a
lower cost and take smaller risks. Since we are supporting a new class of problems,
it is expected that users who need to solve any of these problems will have more
perseverance in exploring our extension and be willing to take greater risks.

8.3.3 Integration with the Cognitive Dimensions Model

The attention investment model provides designers with terms for concepts for
which they have an intuitive understanding, hence making some usability concerns
explicit for analysis purpose and during discussion with others. This is similar to
the cognitive dimensions framework. The two models are complementary: again
quoting from [PBB03],
As an intellectual tool for language designers, a strength of the attention investment model
is its generality, which allows it to provide insight not produced by the more specific CD
[cognitive dimensions] framework. On the other hand, the concrete enumeration of issues
in CDs is more prescriptive of specific problems to consider, and provides sharper questions
for language designers.

For these reasons, these two techniques work well together and we use them for the
purpose of designing the user interface extension of the deductive spreadsheet in
Chap. 9. The relationship between the two models has also been investigated from
the standpoint of cognitive psychology, where [BG99] observes that the attention
investment model can act as a rationale for the choice of dimensions of the cognitive
dimensions framework.

8.4 Target Audience

The cost/benefit analysis derived from the above application of the attention
investment model allows us to identify rather precisely the target audience of the
deductive spreadsheet. Indeed, the total cost of embracing this extension depends of
the previous investments a user has made in learning to use various features of the
traditional spreadsheet. If she already knows how to define and manipulate a data
list, then there is no cost associated with using it. The fewer spreadsheet skills a user
8.4 Target Audience 297

has, the harder it will be for her to master the deductive extension, and therefore the
greater the risk of investing time to learn about it. The minimum skills expected
to make this investment worthy are the prerequisites of the deductive spreadsheet.
Clearly, deciding what is prerequisite is a judgment call as persevering users will
be willing to make riskier investments. A prerequisite is an asset for the users who
have it, but a barrier for those who do not possess it as it is expected that they will
not be able to carry out tasks that require them [PBB03].
The prerequisites needed of a user of the traditional spreadsheet to take advantage
of our deductive extension are as follows:
1. A target user should be familiar with organizing information in a tabular form.
2. She should be able to select cell ranges, and especially entire rows or columns.
3. A target user should be comfortable with defining scalar formulas, including
those that make use of SUM-like operators that take cell ranges as arguments.
Familiarity with array formulas is not required.
Several bonus skills will further lower the learning cost: having worked with data
lists subsumes the first prerequisite and takes the target user closer to some of
the relational manipulations available in the deductive spreadsheet. Another bonus
skill is having used defined range names, as they will substantially simplify writing
clausal theories. Note that these bonus skills are not prerequisites, but offer a head
start to those who have them.
This list of prerequisites identifies an intermediate segment of the spreadsheet
user base: individuals who understand the spreadsheet paradigm fairly thoroughly
and have been using these applications frequently and for a rather long time.
Because of this exposure, they have experienced learning new features several times
and have a feel for it. They also know where to look for additional information if
they run into difficulties.
Expert users, for example those who have used Visual Basic for Applications
for implementing functionalities that Excel does not provide natively, also benefit
from the deductive extension as it allows them to solve problems that they could not
tackle in the traditional spreadsheet. Among expert users, professional developers
build customized extensions as part of their job, whether for clients or within the IT
department of their organization. We expect them to adopt the deductive extension
as yet another tool to perform their day-to-day job. For them, the cognitive cost is
relatively low because they are already familiar with sophisticated programming
and database concepts, while the payoff may be substantial as it would provide
them with native constructs that they may need to program explicitly to perform
some tasks.
The users who are left out are those who have used spreadsheets for nothing
more than managing and printing lists of information, or little more. They may be
able to write basic formulas, but they do not possess the familiarity of intermediate
users. Switching to the deductive spreadsheet is not likely to change their use of the
application, and they will probably not know what to do with the deductive features.
Quoting from [PBB03], our extension “has little to offer to these groups, because
the payoff (approximately zero) is lower than the learning cost of the prerequisites”.
298 8 Cognitive Interface Design

8.5 Summary

In this chapter, we have reviewed some recent advances in the cognitive psychology
literature, namely the cognitive dimensions of notations framework [Gre89, GP96]
and the attention investment model [Bla02], and extracted a methodology that
will prove useful in designing the first cut of a user interface for the deductive
spreadsheet that users may be comfortable with. We will indeed apply these user-
centric guidelines in Chap. 9 each time we need to decide how to provide access
to an extended functionality. Abiding by the requirements of Chap. 2, this task
will be constrained by the necessity of being conservative with respect to the
traditional interface, so that users who do not need the deductive extensions will
not be penalized, and users who want to avail themselves of them can rely on
their experience with the traditional interface to quickly learn to use them. What
these cognitive frameworks do not do is to provide any assurance that the resulting
interface will be free of defects, or even that it will be blessed by user acceptance.
These assessments must be done a posteriori using standard techniques such as user
testing, a very limited form of which will be reported in Chap. 10. Following those
two cognitive models as guiding principles will however allow us to make informed
decisions and therefore provide us with sufficient confidence in the adequacy of the
design in Chap. 9 as a first version of the user interface of the deductive spreadsheet.
While these cognitive models provide guidelines to make educated decisions,
they do not prescribe a design, and do not even lay down the possible choices.
These issues require creativity and taste, or at least awareness of what is “out
there”, both within actual applications and in the literature. Once a choice, or set
of candidate choices, has been made, these models will provide a way to make an
early evaluation. Possible designs are also constrained by the cognitive requirements
introduced in Chap. 2, which we recall at the highest level of granularity:
Cognitive conservativity: All traditional functionalities shall be accessible to
the user in the exact same way as in a traditional spreadsheet.
Consistency: The commands supporting the deductive extensions shall be con-
sistent with the existing modalities of interaction and with the user’s expectation.
Cognitive Integration: Using deductive capabilities shall be natural and intu-
itive, including when used in conjunction with traditional functionalities.
Performance: Common deductive inferences should have a performance that is
subjectively comparable to spreadsheet operations of intermediate complexity.
Discovery: The deductive extension should have a gentle learning curve so that
users can reinforce and extend their skills by simply using the tool and minimal
experimentation.
Notice that what we called “consistency” and “discovery” in Chap. 2 spell out
different aspects of the consistency dimension discussed in Sect. 8.2.2. The other
two classes of requirements derive from the fact that we are building on top of the
user interface of the traditional spreadsheet rather than starting from scratch.
Chapter 9
Toward a Deductive Interface

The present chapter completes the design of the deductive spreadsheet by proposing
an approach to making the logical functionalities engineered in Part I available for
practical use as an extension of the user interface of the traditional spreadsheet,
which we examined in Chap. 7. As we do so, our primary concern will be to
remain conservative with respect to the choices that have sedimented in the excellent
interfaces of modern spreadsheet applications, choices to which users have become
accustomed and on which sometimes dependent. Within these necessary bounds,
our major design objective is to provide the user with a simple and intuitive access to
the enhanced expressive power of the deductive infrastructure. For the most part, this
will be realized by simply extending the current interaction modalities to the new
deductive components. For example, all the graphical approaches to constructing
a scalar or array formula will be usable when building a clausal definition. We
will occasionally extend current support in order to further simplify the user’s
experience, in particular by offering a new interactive way to assemble clauses, or
to take advantage of the deductive infrastructure, for instance by providing novel
flow graph visualization mechanisms. Some decisions break away with tradition, for
example a proposal to make two distinct but equivalent input languages available
for the deductive fragment: one closely tracking the logical syntax of clauses
of Chaps. 5 and 6, and one in line with relational algebra as used for querying
databases. This choice is made on usability grounds as some users may be more
comfortable modeling a problem logically, while others may prefer the relational
view.
We will make abundant use of the cognitive techniques of Chap. 8 as we elicit
our preliminary design for the user interface of the deductive spreadsheet. Indeed,
the attention investment model will act as an overall framework which will guide
us toward providing techniques that facilitate access to the new functionalities.
Individual decision points will often be scrutinized through the web of the cognitive
dimensions in order to reach optimal compromises. Ultimately, these decisions will
have to get user approval, which will be assessed through testing and feedback.
We start in Chap. 10. Because we are designing a graphical user interface, we
will provide illustrations for most of the visual additions we propose. For the sake

I. Cervesato, The Deductive Spreadsheet, Cognitive Technologies, 299

DOI 10.1007/978-3-642-37747-1 9, © Springer-Verlag Berlin Heidelberg 2013
300 9 Toward a Deductive Interface

of continuity with Chap. 7, we architect our design as an extension of the interface

of Microsoft Excel 2003, although it can be easily ported to other products.
This chapter is organized as follows. Section 9.1 outlines the minimal alterations
to the layout of the traditional spreadsheet required to accommodate the deductive
infrastructure. We introduce the two syntactic extensions to the language of
traditional formulas available to write clausal definitions in Sect. 9.2 and discuss
graphical methods for constructing clausal theories in Sect. 9.3. We tackle the issue
of updates in the deductive spreadsheet in Sect. 9.4 and discuss new debugging
tools for the deductive additions in Sect. 9.5. Extensions to the traditional array
of productivity tools and the help system are presented in Sects. 9.6 and 9.7,
respectively. Finally, Sect. 9.8 evaluates our resulting design against the cognitive
requirements of Chap. 2.

9.1 Deductive Layout

At first glance, the user interface of the deductive spreadsheet is indistinguishable

from the screen of a typical traditional spreadsheet. Indeed, the user still sees a grid
of cells which she can format in the usual ways. They form a worksheet and several
worksheets constitute a workbook. All the traditional navigation methods, selection
techniques, and commands at large operate as usual. It is only when she starts
working with the system that the new possibilities reveal themselves as added
menu items and extensions to old functionalities. The major changes to the static
layout of the application window cater to providing simple and efficient methods for
designating areas of the worksheet as relations, and making a convenient interface
available for associating a clausal theory with a range of cells. These will be the main
topics of this section. The concrete language in which logical definitions are written,
technically another static aspect of our extension, will be examined in Sect. 9.2.
We discuss the definition of relations in Sect. 9.1.1, we describe changes to the
formula input textbox in Sect. 9.1.2, and we finally conclude with some further
considerations about the geometry of a worksheet in Sect. 9.1.3.

9.1.1 Defining Relations

As mentioned in Chap. 7, a second-class notion of relation exists in the traditional

spreadsheet. In Excel, for example, defining a data list (as a relation is called) is as
simple as writing the header of each field on a row and one or more records below
it. Then, a “data form” becomes automatically available to view, insert, modify,
delete, and search records. An example is shown in Fig. 7.4. While extremely useful
for working with tabular data in a command-oriented fashion, this approach does
not identify relations and their fields as first-class objects. In order to refer to them
in a formula, one has to rely on range coordinates or assign them an independent
9.1 Deductive Layout 301

symbolic name. Even after doing so, the traditional spreadsheet does not provide
any infrastructure to work with them relationally inside a formula. Of course, this
is precisely what the deductive spreadsheet brings to the table.
In this section, we discuss a mechanism that helps in defining a relation and
making it and its fields available for use in clausal formulas. It is not meant to replace
the traditional approaches illustrated above, but to complement them. Indeed, it
conveniently bundles together a number of enabling functionalities that are currently
scattered throughout the user interface. It should also be noted that this is not a
prerequisite for writing clausal definitions as it will be possible to rely uniquely on
cell ranges, for example (see Sect. 9.2), but it makes this task substantially easier.

Partial Unbounded Selection of Rows and Columns

A relation occupies a region of the worksheet, and therefore can be identified either
by typing range coordinates or by selecting its cells using the mouse or direction
keys. Currently, this last option works well when selecting a small rectangular area
or one or more rows or columns. The only way to highlight a region comprising
columns C and D and extending from row 25 to the very bottom of the worksheet
is to select cells C25 and D25, and tediously drag the mouse till its last row is
reached, a process that can take quite some time on older machines. We call such an
area a partial unbounded column, with a similar notion applying to rows. Because
many relations are often unbounded, a user may want to leave some space above
a columnar table, or to the left of a horizontal one, to include captions, comments
and generic calculations. In order to enable this, we propose an effective method
for quickly selecting a contiguous group of rows or columns starting at an arbitrary
point in the worksheet and extending to its last column or row, respectively. Let us
illustrate it on the above example:
1. The user first scrolls the worksheet so that cell C25 and D25 are visible.
2. She selects columns C and D from the column header.
3. With the mouse button still depressed, she moves the cursor toward cells D25
(if she initially selected column C) or C25 (otherwise). The selected region
shrinks accordingly.
4. When she has reached this cell, she releases the mouse button, hence partially
selecting columns C and D from row 25 till the bottom of the worksheet.
A partial unbounded selection spanning a number of columns so large that it
does not fit on the screen is performed in the same way, except that the initial
whole-column selection will automatically scroll the display horizontally. Note that
the target starting row of the selection can be anywhere, not necessarily near the
top: the only requirement is that it appear on the screen as the selection operation is
started.
While the above description illustrated the partial unbounded selection of
columns, the same approach applies to rows. Observe that this method is con-
servative with respect to the traditional spreadsheet as selecting a row or column
302 9 Toward a Deductive Interface

Fig. 9.1 Right-click menu access to the relation definition dialog

and moving the cursor inside the worksheet while the mouse button is depressed
simply leaves that entire row or column selected. This is cognitively consistent
with usual selection techniques. This method can participate in the selection
of heterogeneous or non-contiguous regions by keeping the CTRL key pressed
throughout the process.

The Relation Definition Dialog

Now that we have a way to select appropriate regions of the worksheet, we will
make a dialog, or wizard in Excel parlance, available to identify a group of cells as
a relation and assign symbolic names to its attributes. It can be invoked either from
the menu bar (under “Data” for example), or by selecting a multiple-cell region of
the worksheet, right-clicking on it and choosing the menu option that reads “Define
Relation . . . ”. This possibility is illustrated in Fig. 9.1 on a potential extension of
the user interface of Microsoft Excel 2003.
The “Define Relation . . . ” dialog itself is shown in Fig. 9.2, where it has been
instantiated for an area of the worksheet meant to contain the definition of the
path predicate, which we have encountered several times in Chap. 6. When initially
invoked, only the “Range” textbox contains any data. The user can modify it and fill
9.1 Deductive Layout 303

Fig. 9.2 Relation definition dialog

the remaining parts of the dialog to her taste. Note that she is not forced to undertake
any of these actions. We will now examine each input field and action button in turn.
Name: This optional field allows giving a name to the cell range on which the
relation is defined. We wrote path in it in our example. This corresponds
to selecting the range and assigning a symbolic name to it using the “Define
Name . . . ” dialog examined in Chap. 7. Additionally, it places this name on the
defined predicates list, which can conveniently be used later in order to quickly
build clauses. Predicate names are required to be unique.
Range: This field, which is automatically filled when the dialog is invoked,
describes the geometry of the selected range of cells. It will typically contain
a rectangular area, possibly spanning (partial) unbounded rows or columns.
Figure 9.1 shows columns C and D selected in their entirety, but the “Range”
in Fig. 9.2 specifies that they start at column 3 and extend to the bottom of
the spreadsheet (the notation C 3WD for partial unbounded selections will be
defined in Sect. 9.2). This incongruence is due to the fact that the two “Insert . . . ”
checkboxes are checked. Their meaning will be explained below.
Non-contiguous regions, possibly over different worksheets, can constitute a
relation, and can be entered in the “Range” field as a comma-separated list of
rectangular areas. We have seen in Chap. 7 that such regions can be assigned
names in the “Define Name . . . ” dialog. The only restriction on this possibility is
304 9 Toward a Deductive Interface

Fig. 9.3 “path” as defined by the “Define Relation: : :” dialog

that all subregions must form a rectangle if they are to be juxtaposed: otherwise,
there would be no way to ensure that each record in the relation has the same
number of elements. It is expected that a user will rarely avail herself of this
possibility.
If the relation has not been assigned a name, it is its hosting range that is entered
in the defined predicates list. The range of the relation is partitioned among its
fields in the “Fields” enclosure described below. This partition is either vertical
or horizontal depending on which of the “Fields as columns/rows” radio button
discussed next is selected.
Fields as columns/Fields as rows: Differently from commercial products, we
do not force the user to align the fields of her relations along the columns of
the worksheet. This dual radio button allows her to choose how the selected
region is to be interpreted as a relation. If “Fields as columns” is selected, the
traditional layout will be in vigor. If she clicks on “Fields as rows”, attributes
will be horizontal and records vertical. The first option could be the default.
Insert title row/column: The last word of the label is “row” if “Fields as
columns” was selected, and “column” otherwise. If this box is checked, the
relation name entered in the “Name” field will be installed across the top row
or leftmost column of the selected area once the user presses the “OK” button.
Its cells will be merged and the text will be centered and highlighted in bold, as
shown in Fig. 9.3. The user can always change this formatting later.
Checking this box has the effect of shrinking the extension of the relation that
can hold records by one row or one column. The “Range” of the relation as well
as each “Range” in the “Fields” enclosure are automatically adjusted to reflect
this effect. Unchecking it restores them to their original value.
Insert field row/column: The last word of the label is “row” if “Fields as
columns” was selected, and “column” otherwise. When checked, pressing the
“OK” button will write the “Name” values associated to the various attributes of
the relation in the “Fields” enclosure in a dedicated row or column just under or
9.1 Deductive Layout 305

right of the title if “Insert title row/column” was checked, and in the top/leftmost
row/column otherwise. Each name will be centered in its cell, as shown in
Fig. 9.3. Fields holding list of values span multiple rows or columns. In this case,
the cells where the field name is supposed to land are merged before inserting it.
Again, selecting this option will shrink the area intended for data by one record,
and this will be reflected in the various ranges of the dialog. Indeed, it is because
both “Insert title row” and “Insert field row” are checked in Fig. 9.2 that the
“Range” of path contains C 3WD rather than C WD. The same holds for the
ranges in the “Fields” enclosure.
Fields: This enclosure describes the fields of the relation being defined. Each
line corresponds to one field and contains a checkbox, a “Range” textbox and a
“Name” textbox. We will now describe these objects in turn:
Field checkbox: The purpose of this checkbox is to select fields for later
deletion using the “Delete marked fields” button. It is unchecked by default.
It is not strictly necessary as a field can always be deleted by blanking out its
“Range”. It is provided as a convenience for the user.
Field range: This entry allows specifying the extension of a field. It is
automatically filled with a single column/row when the user initially invokes
the dialog. She needs to modify the contained range if this attribute is to
correspond to a list of values, as described in Chap. 6. If she does so, overlap-
ping unnamed fields below it will be automatically removed and overlapping
named fields will be highlighted, prompting her to take action. Enlarging the
value of a field beyond the range of the relation, as specified in “Range”, will
bring up a dialog asking her if she really wants to enlarge the relation.
Field name: This entry allows the user to assign a name to a field. This is
mostly useful in conjunction with the “Insert fields row/column” option,
and when using the alternate SQL syntax for writing clausal definitions (see
Sect. 9.2). Field names need to be unique within a relation, but not necessarily
across relations.
Add fields: This button adds a new blank field line in the “Fields” enclosure.
This is useful for extending the relation with a new attribute.
Delete marked fields: This button deletes the lines in the “Fields” enclosure
whose checkbox has been checked.
OK: If all the data in the dialog are correct, pressing this button will partition the
relation as specified in the “Fields” enclosure, define its name and the name of its
attributes, and possibly insert them in the title rows or columns. The visible result
of performing this action on the contents of Fig. 9.2 is displayed in Fig. 9.3. It will
also insert this relation in the defined predicates list, either by name or by range,
for future reference. If there is any problem or potential ambiguity, it will bring
up an error dialog explaining the issues. This will flag relation names already in
use, duplicate field names, overlapping field ranges, and inconsistencies between
the range of the relation and the cumulative range of its fields. The latter can be
resolved either by editing the field ranges in the “Define Relation . . . ” dialog, or
by instructing the system to adapt the geometry of the relation to that of its fields.
306 9 Toward a Deductive Interface

Cancel: This button restores the dialog to its status when the user first invoked it.
Help: This button invokes the help facilities.
We have described the “Define Relation . . . ” dialog as a means to making the
system aware of a new relation. This is not the only use of this dialog. It can indeed
be invoked on an existing table (in which case it reads “Redefine Relation . . . ”)
to modify any aspect of this relation, as defined in the fields in Fig. 9.2. This is
particularly useful for altering the geometry of a table, and managing the addition
or deletion of attributes. Doing so brings up some non-trivial issues that will be
discussed in detail in Sect. 9.4.
It should be observed that different relations may overlap, be contained in each
other, or even be defined over the exact same worksheet cell range. In the latter case,
only symbolic names will be able to distinguish them.
As a final note, we want to stress again that one can use the deductive spreadsheet
without ever invoking the “Define Relations . . . ”. Indeed, its main function is to
manage the complexity of working with relations by providing symbolic names
for them and their attributes. However, range coordinates can be used anywhere a
symbolic predicate (or attribute) name appears. Note also that the same symbolic
association described here can be performed by repeatedly invoking the standard
“Define Name . . . ” dialog seen in Chap. 7. What the “Define Relations . . . ” dialog
provides is a convenient way to make all these definitions in one place. By inserting
relation names in the defined predicates list, it also enables quick edits of clausal
definitions, as we will see in Sect. 9.3. For these reasons, it is expected that users
will make abundant use of this feature.

9.1.2 The Formula Input Textbox

In the traditional spreadsheet, formulas tend to be rather simple. They usually con-
tain just a few operators and very rarely do they extend past the size of the formula
input textbox. When they do, the contents of this box scrolls, making the initial
portion of the formula invisible. It can be put back into focus by scrolling back to the
beginning using the cursor keys. In particular, a long formula does not wrap around
to the next line: few users know that pressing ALT-ENTER can produce a new line
on which to split their input. Using a single line makes writing and especially mod-
ifying long formulas difficult and error-prone as visibility is hindered. In practice,
this is a minor problem because, as we said, users tend to write rather short formulas.
The situation changes as soon as we start using the formula input textbox to
enter clausal subtheories. As we saw in Part I, a predicate definition can consist
of multiple clauses, and each clause can be a rather complex object as soon as its
body contains more than two or three literals or constraints. For this reason, we
believe it is necessary to facilitate the entry of multi-line contents (a set of clauses, a
traditional formula, or normal text) in the formula input textbox. In fact, we enrich
the formula bar with a syntactic editor that highlights keywords, provides visual
9.1 Deductive Layout 307

feedback, autoformats complex definitions, and monitors what the user writes in
real time, recognizing potential errors and suggesting corrections. We will comment
on these features over the next several sections.
In order to retain conservativity over the traditional spreadsheet, the formula
input textbox remains unchanged when the user starts entering text (again, not
necessarily a clause). As soon as she has filled more than a certain portion of
the available space (for example 30 characters or 50 % of the available space,
whichever is shorter), a “tip” appears instructing her to press the ALT-ENTER key
combination to produce a new line. This is illustrated on a possible extension of
the user interface of Microsoft Excel in the top part of Fig. 9.4. Of course, she can
ignore this suggestion and write text on this one line only. If she chooses to make
use of the second line, she will be able to press ALT-ENTER to create new lines
up to a predefined maximum number. Once she has reached this maximum, a scroll
bar appears allowing her to make all parts of the formula visible. She should also be
given the option to vertically enlarge the formula input box so that more is visible,
up to the height of the screen. This behavior is shown in the bottom part of Fig. 9.4.
While allowing clauses to be entered over multiple line is convenient, it leaves
the burden of managing their possible complexity on the user. For this reason, a
useful complement to this approach is to offer an autoformat option. Whenever
the formula input box consists of more than one line, a push-down button appears
allowing the user to enable or disable the autoformat facility. With this button
disabled (raised), the user would be totally responsible for any formatting of the
textbox. Whenever she pushes it down, the content of the textbox is reformatted
according to predefined heuristics. Figure 9.4 shows the autoformat button just
above the right scrollbar of the worksheet. A possible layout for an autoformatted
theory is shown in the bottom part of this figure. The dialog-assisted mechanism
outlined in Sect. 9.3 always relies on the autoformat facility. Even in this case, the
user should be able to alter the proposed formatting: the system should remember
her changes and limit autoformatting to new material she inputs. Of course, this
facility could be made available also for traditional formulas. Formula autoformat
is a property of individual cells or ranges and could be enabled or disable for each
single cell (or range). The default should be to have it enabled, but the user may
change this as a global preference.

9.1.3 Further Considerations

Each worksheet in a commercial spreadsheets consists of 256 columns and 65,536

rows. This limited number of columns is generally regarded as sufficient when
working with scalar expressions, or even simple relations (e.g., Excel’s data lists).
Applications that require more than 256 columns can always be laid out over several
worksheets within the same workbook, or even over multiple workbooks.
This attitude may have to change in the deductive spreadsheet, not because of the
core logical extension, but because we allowed lists of values as predicate arguments
308 9 Toward a Deductive Interface

Autoformat Button

Fig. 9.4 Multiline formula input textbox

in Chap. 6. It is conceivable that a user may want to write predicates containing

lists with more than 256 values, something that the current setup forbids: indeed,
although we allowed the fields of a relation not to be contiguous in Sect. 9.1.1,
possibly belonging to different worksheets, any individual fields must be fully
contained within the same worksheet. How important of an issue this is will be
established on the basis of user feedback. We suspect that a majority of users will
never break the current bounds, but a few will eventually feel the need for lists of
values larger than the current number of columns.
Currently, 1 byte is required to internally represent the standard 256 columns
of a worksheet and two for its 65,536 rows. If demand justifies this move, we
9.2 Textual Input of Clausal Theories 309

propose to increase the first figure to 2 bytes, hence supporting worksheets with
65,536 columns: in A1 coordinates, columns would be designated with letter codes
from A to BQWP. We suspect that this will satisfy the needs of the vast majority
of users without imposing undue penalty on file size and processing time. If it
becomes apparent that there is substantial demand for even larger number of
columns, both column and row coordinates may be enlarged to fit 4 bytes, or a
more economical dynamic encoding could be used.

9.2 Textual Input of Clausal Theories

Although the traditional spreadsheet provides graphical mechanisms to construct

formulas, as we have seen in Chap. 7, they are ultimately translated into the textual
syntax that appears in the formula input textbox. Therefore, any extension to the
language of formulas begins with an appropriate extension to this concrete syntax.
In this section, we will propose two alternative such external syntaxes for our
deductive additions. The first closely follows the logical presentation of Chaps. 5
and 6 by providing mechanisms to enter clauses and subtheories in the formula input
textbox. The second draws its inspiration from the roots of our deductive extension
in relational algebra, initially described in Chap. 4 and amended in Chap. 6, and
makes available an SQL-like syntax for the corresponding relational expressions.
Invisibly to the user, formulas written in either external syntax are translated into the
common internal representation on which the deductive engine operates. Similarly
to cell coordinates, the user is to be given a command for going back and forth
between the logical and the relational syntaxes. This command may be provided
as a global option or as a per-cell switch based on a global default. The graphical
construction of deductive formulas described in Sect. 9.3 will build an expression
using whatever syntax is currently selected.
Our rationale for providing two alternative syntaxes for the same extension, at
least initially, is that some users may find it easier to express their problem logically,
while others may have more of a relational mind-set. Giving a choice of concrete
languages should satisfy both camps. It certainly increases the closeness of mapping
cognitive dimension. The validity of this decision will be assessed at a later stage
by means of interviews with users and cognitive testing. It may become apparent
that this flexibility leads to confusion, in which case one of the syntaxes will have
to be eliminated. It is likely that these empirical assessments will suggest a better
syntax altogether, that is neither purely logical nor purely relational, but a mix, or
maybe even something completely different. As we say so, it should be noted that
the textual syntax of the traditional spreadsheet, as embodied in Microsoft Excel for
example, is not always very user-friendly. Although users have grown accustomed
to a suboptimal input language for formulas, we need to make our extension as
intuitive as possible so that simply writing a clause does not become an unnecessary
barrier to using the deductive functionalities. It should also be observed that the
graphical construction of formulas will alleviate some of the syntactic burden, and
310 9 Toward a Deductive Interface

that it is expected that users will avail themselves of this option even more than
in the traditional spreadsheet as clausal definitions tend to be more complex than
typical scalar formulas. We will return to this point in Sect. 9.3.
Independently of how exactly clausal theories are expressed, our design will be
conservative over the traditional spreadsheet, so that a user who does not need to
use the deductive functionalities can still write her formulas as she has always done.
In particular, existing workbooks will be accepted by the deductive spreadsheet and
behave as in the application it was originally written in.1 Moreover, the syntax for
the deductive extensions shall be consistent with the current external language of
scalar formulas so that a user does not have to invest too much cognitive effort in
learning how to use the extension.
In Sect. 9.2.1 we define the syntax for base relations and predicate names, as
they will be input by the user. Section 9.2.2 proposes a concrete syntax for clausal
definitions of predicates, while Sect. 9.2.3 presents an alternative notation stemming
from relational algebra. We describe what to do in case the user enters a formula
containing a syntactic error in Sect. 9.2.4. Finally, we outline the mechanisms that
allow logical formulas to be embedded in a scalar or array expression, and vice
versa, in Sect. 9.2.5.

9.2.1 Base Relations and Predicate Names

Both clauses and relational expressions are constructed by applying various kinds
of operators to similar atomic building blocks: predicates in the former case, base
relations in the latter. Since both notions identify the same regions of the worksheet,
we will rely on a single concrete syntax to denote them in larger expressions.
As we do so, two observations are in order: first, traditional spreadsheets do not
provide a specific syntax for denoting relational entities (for example data lists in
Excel) as they are manipulated through commands rather than operators; second,
arbitrary regions of the spreadsheet can currently be identified by means of range
such as “A1WB17” for a rectangular area, or “DWH ” for a cluster of columns, or
“A1WB17; DWH ” for non-contiguous regions. Prefixing the name of the workbook
with “Š” as a separator even allows regions to be contributed from other worksheets.
Symbolic names can be associated with each of these cases, hence enabling a more
clear and intuitive notation.
For the most part, ranges and range names are perfectly adequate for the purpose
of denoting a base relation or, equivalently, a predicate. A relation (or predicate)
is often unbounded, i.e., it extends all the way to the last row of the worksheet. If
it starts on the first row, the current notation, e.g., “DWH ”, is nicely succinct. If it

1
More precisely, a deductive extension based on Microsoft Excel 2013 will be able to load any
Excel 2013 file, for example. Spreadsheets written using earlier versions of Excel or different
applications may have to be imported, as currently done.
9.2 Textual Input of Clausal Theories 311

starts on a different row, 3 for example, one has to resort to writing “D3WH 65536”,
which is a lot less practical. Note again that this is rarely an issue in the traditional
spreadsheet as relations are not first-class objects and can only be manipulated using
commands. Similar considerations would apply to tables laid out along the rows of
the spreadsheet, a possibility that is not currently supported by any commercial
products.
Because predicates are first-class objects in the deductive spreadsheet and
because they can be unbounded either vertically or horizontally (or both), we need to
provide a succinct and natural notation to denote the bottom row and the rightmost
column. We propose to use “ ” as a marker for the last row or column in a worksheet.
Therefore, “A ” denotes the last cell on column A (currently “A65536”) while “ 3”
identifies the last cell on row 3 (currently “I V 3”). The cell at the very bottom and far
right is now “ ” rather than “I V 65536”. This device can naturally be incorporated
in the current syntax for cell ranges, so that “D3WH 65536” above is now written
“D3WH ”. Here are a few more examples of use of this notation:
• A5WC : all the cells in columns A; B; C below row 4.
• B3W 4: all the cells in rows 3 and 4 right of column B (included).
• B3W : all the cells in the worksheet right and below A2.
• H W : all the cells in the last row of the worksheet right of column G.
Entire columns are still denoted by omitting the row number, so that for example
“DWH ” abbreviates “D1WH 65536” or equivalently “D1WH ”
This notational device naturally extends to cell or range coordinates in R1C1
notation.
Besides providing a more convenient way of expressing unbounded relations,
this notation has the additional advantage that it is independent from the underlying
implementation of cell coordinates. For example, if an initial version of the
deductive spreadsheet with a maximum of 256 columns is replaced by a version that
supports 65,536 columns, a predicate denoted by the range “A3W 5” will still extend
till the last column of rows 3 through 5, while “A3WI V 5” will not take advantage of
the added space (and an automatic translation may not be what the user wants).
At this point, we can describe the grammatical rules that describe A1 cell and
range coordinates in the deductive spreadsheet:

Row Coordinates r WWD Œ1  maxRow j _

Column Coordinates c WWD ŒA  maxCol j _
Cell Coordinates C WWD cr
Range R WWD C :C j c:c j r:r j R,R

where maxRow and maxCol are the maximum number of rows and columns respec-
tively, currently 65,536 and IV. The definition is similar for R1C1 coordinates. Note
that the only addition with respect to the traditional spreadsheet is the possibility of
a row or column coordinate being “_”.
312 9 Toward a Deductive Interface

As a final note, cells and cell ranges can be assigned a symbolic name, which we
will indicate as id, exactly as in the traditional spreadsheet. This name can be entered
in the relation definition dialog introduced in Sect. 9.1.1, or by means of any of the
currently available name definition tools (in Excel, the “Define name . . . ” menu
command or by writing this name in the leftmost textbox of the formula toolbar).
Therefore, a predicate name can be entered either as a range or as a range identifier:

Predicate and Base Relation Names pn WWD R j id

9.2.2 Logical Syntax of Clauses

We will now propose a concrete syntax for our deductive extension to the traditional
spreadsheet that closely tracks the way we have been writing clauses and theories.
In order to create a clausal definition, the user will select the cell range for its
head predicate and enter the corresponding expression in the formula input textbox
exactly as for a scalar formula. The syntax has been designed so that there is no
possible clash with either scalar or array formulas, and therefore there is no need to
type a special begin-of-clause marker or to enter a particular mode.
Before describing this concrete syntax, it is worth summarizing the (logical)
abstract syntax of the formulas of the deductive spreadsheet as we have encountered
it in Chap. 6. The top five definitions in Fig. 9.5 collect the core components:
theories, clauses, heads, bodies and predicates. Note that we have included a
production for embedded implications in the body of a clause and provided an
escape for abbreviations in the same context (the non-terminal A). The last two
definitions report the examples of abbreviated forms described in Chap. 6, with the
ellipsis suggesting that this list is extensible. Within this definition, we enclose
optional syntactic elements in square brackets, so that, for example, FINDMAX
may be immediately followed by a test, written Œ æ, and the variable it binds
can itself be followed by a test, Œ æ0 , or by a filter predicate, W P 0 (or both).
The non-terminal lmod describes list modifiers, another extensible set of optional
tokens that can specialize the behavior of abbreviations that return lists, such as
some versions of the transitive closure.
With this summary in place, our task simply consists of providing an adequate
concrete notation for each of these productions. We begin by defining a syntax for
variables, continue with the core language and the abbreviations, and conclude with
a description of how to enter, update and remove a clausal definition. We also digress
on the applicability of our approach to array formulas.

Variables

Variables are an idiosyncrasy of the logical rendition of deductive spreadsheet for-

mulas, as opposed to relational expressions which do not rely on these objects. They
9.2 Textual Input of Clausal Theories 313

Clausal Theories Γ ::= · | Γ; Cl

Clauses Cl ::= H B
Heads H ::= p(X1 ; : : : ; Xn )
Bodies B ::= P | ¬P | æ1 ∼ æ2 | B1 ∧ B2
| P →B
| A
Predicates P ::= p(X1 ; : : : ; Xn )

Abbreviated Forms A ::= p+ (X1 ; X2 ) | TRANSITIVE X1 ; X2 : P

| lmod p++ (X1 ; : : : ; Xn )
| lmod TRANSLIST X1; X 2 : P
| FORALL X : P1 : P 2
| X = FINDMAX [∼ æ] X [∼ æ ] [: P ] : P | : : :
| Xs = FINDALL X [: P ] : P
| Xs = ACCUMULATE X; Xs [: P ] : P | : : :
List Modifiers lmod ::= ·
| lmod Xs =
| lmod X = COUNT [∼ æ]:
| lmod X = SUM [∼ æ] X [∼ æ ]:
| lmod : P : | : : :

Fig. 9.5 Clausal theories with abbreviated forms

appear inside clauses, either as the arguments of predicates or within constraints.

They have some rather peculiar properties, in particular the fact that their name
does not matter: a variable can be renamed at will within a clause, as long as two
different variables are not given the same name. No such restriction exists across
clauses as they are implicitly universally quantified at the head of each clause.
Because of these properties, variables are also rather unusual objects compared
to the ingredients of a traditional formula. Therefore, extreme care shall be put
in designing a concrete syntax for them in order to minimize confusion for a
novice user. Indeed, we provide various alternative and cooperating notations for
variables:
• The cognitively simplest but also most cumbersome and error-prone syntax for
variables simply relies on the range coordinates of the fields they are meant to
denote. For example in Fig. 9.4 the first occurrence of the variable shown as X
could have been entered as C WC . The second occurrence (in the predicate edge)
can either be left as C WC , implicitly indicating that the first argument of predicate
edge shall assume the same values as the first argument of predicate path, or can
be entered as the coordinates of the first field of edge, for example AWA, in which
case the constraint C WC D AWA is added to indicate that these arguments shall
coincide. The resulting representations for the first clause of the path predicate
would then be

path(C:C, D:D) IF edge(C:C, D:D).

314 9 Toward a Deductive Interface

and

path(C:C, D:D) IF edge(A:A, B:B) AND C:C=A:A AND D:D=B:B.

respectively. Ranges occurring in a constraint are interpreted as variables if they

occur in the argument of any predicate, and as cell arrays otherwise.
• If the user has associated symbolic names with the arguments of a predicate
(or the fields of a relation), she can use them instead of the corresponding
range coordinates, subject to the same conventions. Assume for example that the
arguments of predicate edge have been given names From and To while no names
have been assigned to the arguments of path. Then the following two clauses are
equivalent:

path(C:C, D:D) IF edge(From, To) AND C:C=From AND D:D=To.

and

path(From, To) IF edge(From, To).

If arguments in different predicates are given the same name and are not equated
in a clause, they can be differentiated by prefixing the name of the predicate they
belong to, separated by a period, as in path.To.
• Advanced and logically inclined users can use any identifier they fancy as a
variable as long as it does not lead to syntactic ambiguity or confusion. Because
variables are universally quantified at the head of a clause and the arguments
of predicates are necessarily variables, few identifiers need to be ruled out as
ambiguous. Confusion is more likely, and for this reason, it may be good practice
to impose restrictions banning reserved keywords and cell and range coordinates.
We did not include “identifiers in use” in this list because disallowing them
has the disadvantage that a later definition of one of these names will require
action by the user, forcing her either to choose a different name, or to replace
every previous use of that identifier with a new variable name—this is a form
of premature commitment. The exact language of allowable variables will be
tailored on the basis of user testing and reported feedback. In the rest of this book,
we will follow the convention adopted by most logic programming languages and
only use identifiers beginning with an uppercase letter (or _) as variables. Then,
our ongoing example may become:

path(X, Z) IF edge(X, Z).

Note that each of these syntaxes is interchangeable, and that they may be mixed at
leisure.
Altogether, we obtain the following grammar for variables, where productions
are listed in the same order as the informal alternatives above.
9.2 Textual Input of Clausal Theories 315

Variables X WWD R
j id
j word beginning with a capital letter

Clauses

A possible concrete syntax of clauses is simply given by choosing appropriate

keywords to denote the abstract symbols appearing in the top part of Fig. 9.5. Here
is a proposal:

Clausal Theories WWD j Cl.

Clauses ClWWD H IF B
Heads H WWD p(X , : : : ,X )
Bodies B WWD P j NOT P j æ  æ j B AND B
j B ASSUMING P
jA
j(B)
Predicates P WWD p(X , : : : ,X )

In the first line, an empty theory is the empty string (denoted by “ ”), while
the clause separator has been chosen to be a period. If this creates confusion,
an alternative keyword, such as ALSO, could be made available. The concrete
denotation of the reverse implication  separating the head of a clause from its
body is the keyword IF. In the body of a clause, the abstract symbols : (negation),
^ (conjunction) and ! (embedded implication) are mapped to the keywords NOT,
AND, and ASSUMING, the latter with its arguments reversed. The fact that NOT
and AND are reserved keyword in the traditional spreadsheet (as Boolean condition
connectives) is unimportant as their meaning can be recovered from the context (and
the Boolean equivalents always take their arguments in parentheses). Constraints
are written similarly to their abstract syntax (see Sect. 9.2.3 for a more detailed
account of attribute expressions, æ). Body items can be grouped using parentheses.
Predicates start with a predicate name and list their arguments as a comma-separated
list of variables enclosed in parentheses.
In addition to cell ranges and symbolic identifiers, we will admit array formulas
enclosed between angled brackets as per the coercion introduced in Chap. 4.

Predicate Names p WWD pn j <Ee >

We will come back to coercions in Sect. 9.2.5.

316 9 Toward a Deductive Interface

Abbreviated Forms

We follow the same principle to provide a concrete syntax for the abbreviated forms
that can appear in the body of a clause. Indeed, we simply transliterate each of the
reserved identifiers in the bottom part of Fig. 9.5 into a keyword. We add the suffix
+ to a binary predicate to denote its point-to-point transitive closure, and append ++
to represent the version of its transitive closure that collects constructed paths into a
list. The concrete syntax for abbreviated forms is as follows:

Abbreviated A WWD p+(X ,X ) j TRANSITIVE X , X . P

Forms
j lmod p++(X , : : : ,X )
j lmod TRANSLIST X , X . P
j FORALL X :P . P
j X D FINDMAX Œ æ X Œ æ Œ: P  . P j : : :
j X D FINDALL X Œ:P  . P
j X D ACCUMULATE X , X Œ: P  . P j : : :
List lmod WWD
Modifiers
j lmod X =
j lmod X = COUNT Œ æ .
j lmod X = SUM Œ æ X Œ æ .
j lmod : P . j :::

Observe that many of these abbreviations have a syntax that is more reminiscent
of constraints than predicates. Unless this leads to confusion, this similarity is
unimportant as the dedicated keywords and constructions can always disambiguate
their use.

Entry, Updates, and Removal

We will now describe the processes that allow a user to enter, modify and delete
a clausal definition in the deductive spreadsheet by means of the formula input
textbox.

Creating Clausal Definitions

The first step in creating a new clausal definition consists in selecting the range
where the output of this definition should be calculated. This can be done in several
manners:
• The easiest way to define a predicate that does not contain lists as arguments is
simply to select the target range with the mouse, type “=”, and start writing its
9.2 Textual Input of Clausal Theories 317

clauses in the formula input textbox. This method can still be used for predicates
with one or more lists as arguments, but then the variables corresponding to these
arguments will later have to be entered as ranges to prevent ambiguity.
• The preferred option is however to first identify the predicate and possibly its
fields by means of the relation definition dialog discussed in Sect. 9.1.1, select
it either from the list of defined predicates or using the mouse, enter “=”,
and then start typing a clause for it. Recall that a region of the worksheet
can be tagged as a predicate by simply selecting it, and right-clicking on the
“Define Relation . . . ” entry of the pull-down menu. The main advantage of this
approach is that it is not as error-prone as direct selection, especially because
the “Define Relation . . . ” dialog allows precisely identifying the arguments of a
predicate. It involves some premature commitment, but only minimally so since
the relation can be established just prior to typing its clausal definition. The
previous method is free from premature commitment.
Once the target range has been selected, the user can simply start entering clauses
in it. As for traditional formulas, the user types “=” to indicate that the contents of
this cell will be a formula. For aesthetic reasons, we may allow (but not require) the
user to prefix “=” to each of her clauses, not just the first one. For the time being, we
will only be concerned with purely textual input, but helpful graphical alternatives
will be discussed in Sect. 9.3. As illustrated in Sect. 9.1.2, the user can now enter a
definition over multiple lines. She moves to the next line by pressing ALT-ENTER
and can use the mouse or the direction keys to revisit other parts of the definition. As
a convenience, creating a new line after the end of a clause automatically proposes
its head predicate in preparation for extending its definition with a new clause. For
example, if the user types “=path(X, Z) IF edge(X, Z).ALT-ENTER”,
she will see

=path(X, Z) IF edge(X, Z).

=path(X, Z) IF j

where “j” indicates the cursor position. She can always delete this clause fragment
if she only wanted an empty line, or she can modify it.
The clauses entered in a cell range must define the predicate corresponding to
this cell range. Therefore, range “C WD” in Fig. 9.4 needs to contain definitions
for the predicate internally identified as “C WD” (here designated as “path”). It is
conceivable however that this entry also contains definitions for auxiliary predicates
that are only used by the target predicate of this range (“path” here). The main
advantage of this permissiveness is that a smaller region of the worksheet would be
needed to define an important predicate, hence improving visibility. The possible
disadvantages are error-proneness, confusion, and difficulties in debugging a model
since intermediate results would not be readily visible. The merits of this possibility
need to be established on the basis of user response and testing. For the moment, we
are inclined to making it available as a “use-at-your-own-risk” feature intended for
advanced users.
318 9 Toward a Deductive Interface

Once the user is finished entering a clausal definition in a cell range, she simply
presses ENTER, just as in the scalar case, to install it in these cells. Two outcomes
are possible: if this definition contains errors, a dialog will appear instructing her
on how to correct them—errors and their prevention will be examined in detail in
Sect. 9.2.4; if it is correct, its evaluated records will appear in that space, starting
from the top row and extending toward the bottom in case of a vertical table, and
stretching left to right for a horizontally specified relation. Note that the entire region
will remain blank if the clauses contained in it are unsatisfiable.

Modifying Clausal Definitions

As in the traditional spreadsheet, chances are very high that the user will need to
modify a clausal definition after she has entered it. Changes can target either the
clauses themselves, or the geometry of the target relation. We will now concentrate
on the first type of modification, delaying the examination of the second until
Sect. 9.4.
In order to modify a clausal definition, a user simply needs to select any cell in
the range that contains it and click on the formula input textbox. This has the effect
of selecting the whole target range. She can now make her changes to the clausal
definition using the same methods as when she initially created the formula. When
she is done, sealing the changes with the ENTER key will have them come into
effect for the entire target range of this definition.

Deleting Clausal Definitions

A clausal definition can be deleted in two ways. First, the user can select the target
range, either with the mouse or by searching for its name, and press the DELETE
key. Second, she can click on any cell in this range, then go to the formula input
textbox, which will highlight the whole range, and delete the definitions it contains.

About Array Formulas

Cell ranges can support both clausal theories and array formulas. These two types
of calculated expressions differ by the operators that can appear in them, to the
point that no confusion is possible as to whether a user is typing one or the other.
Notice also that the proposed modalities for entering and modifying clauses are
more similar to the current methods for working with scalar formulas than to the
arcane procedure that permits inserting and altering array formulas.
The same principles underlying the proposed methods for working with clausal
theories can be immediately adapted to array formulas. In particular, an array
formula can be sealed by just pressing ENTER rather than the complex CTRL-
SHIFT-ENTER combination, both when it is created and when it is modified.
For this to be the case, the extension of an array formula could advantageously
9.2 Textual Input of Clausal Theories 319

be automatically selected whenever the user embarks in editing any of its cells.
Furthermore, we will see in Sect. 9.4 that our proposal for clausal theories supports
simple and intuitive methods for modifying the geometry of the underlying cell
range as well as copy-and-paste operations: again, they can immediately be ported
to array formulas.
While we believe our approach can beneficially improve the usability of array
formulas, it is not our intention to modify the current behavior of the traditional
spreadsheet. In particular, we will not make it available in the deductive spreadsheet
for array formulas unless it is adopted by spreadsheet application manufacturers at
large, as it would violate our vow to conservativity.

9.2.3 Relational Syntax of Clauses

We now propose a second concrete syntax for the extended expressions inherent in
the deductive spreadsheet. This time, we will adopt relational algebra as our model,
and provide a language that closely resembles its SQL incarnation [O’N94]. Before
doing so, we recall the abstract syntax of core relational algebra in the following
table, reported with minor changes from Chap. 4:

Relational expressions eN WWD p j eN [ eN j eN  eN j eN  eN

j æ E eN j C eN
Attribute expressions æ WWD a j e j op.æ; : : : ; æ/
Selection conditions C WWD æ  æ j C ^ C j C _ C j :C
j true j false

Note that we did not include the equational support for recursion discussed
in Chap. 6, nor an escape for abbreviated forms, the equivalent of embedded
implication, or lists of values. While these extensions can easily be accommodated,
massaging the abstract syntax to take them into account would be a nearly useless
exercise as SQL does not closely follow the grammatical guidelines of relational
algebra. Instead, we will incorporate them directly in the proposed concrete syntax.
We begin by extending the denotation of relations introduced in Sect. 9.2.2 to
express attributes. We then recall the syntax of core SQL, and extend it to account
for our abbreviated forms. We conclude with a description of how to enter SQL-like
expressions in the deductive spreadsheet. This discussion will build heavily on the
conventions and techniques discussed in the previous section.

Attributes

Attributes are identifiers for the fields of a relations (sometimes the two terms are
used interchangeably). They are similar to logical variables, except that they cannot
320 9 Toward a Deductive Interface

be renamed at will: their name is fixed. For us, a field is a region of the worksheet,
and more precisely a subrange of a relations that extends all the way vertically (if the
relation is columnar) or horizontally (otherwise). Therefore, in its simplest instance,
an attribute is a cell range. If the relation has been baptized using the “Define
Relation . . . ” dialog, its fields could have been given symbolic names as well. In
this case, attributes can be identified using these names. Recall however that fields
in different relations can have the same name. Whenever two such relations are used
together and there is a risk of ambiguity, a symbolic attribute name can be prefixed
by its relation’s name, with a period in between. Altogether, an attribute, denoted by
the non-terminal a, is given by the following syntax:

Attributes a WWD R j Œid.id

Extended Relational Expressions

The SQL syntax of the core relational calculus introduced in Chap. 4 is given by the
following grammar:

Relational Expression R WWD p j R UNION R j R MINUS R

j SELECT æ; : : : ; æ FROM p; : : : ; p Œ WHERE C 
j (R)
Conditions C WWD æ  æ j C AND C j C OR C j NOT C
j æ IN R
Attribute Expressions æ WWD a j e j op(æ; : : : ; æ)

Here, p stands for a base relational expression, which shares the same syntax as
predicate names discussed in Sect. 9.2.2: it can be a cell range, an identifier, or an
array expression enclosed in angle brackets. The keywords UNION and MINUS are
concrete syntax for the union and difference operators of relational algebra. The
form “SELECT æ1 ; : : : ; æn FROM p1 ; : : : ; pm WHERE C ”, a trademark of SQL,
corresponds to the projection over attribute expressions æ1 ; : : : ; æn of the Cartesian
product of relations p1 ; : : : ; pm mediated by the selection over condition C . The
projection attributes æ1 ; : : : ; æn can be replaced with the symbol “*” if all exported
fields are to be made available without any processing. The selection condition C
and the keyword WHERE can be omitted if vacuous. Note that this expression is more
restrictive than general relational algebra as it forces projections and selections to be
taken over base predicates (p1 ; : : : ; pm above). This is remedied by allowing nested
SELECT statements in the condition C , introduced by the form æ IN R. Besides
this, selection conditions have the same form as in Chap. 4, and similarly for the
attribute expressions they depend on.
It should be noted that SQL is a lot richer than the simple fragment discussed
so far. Some additional constructs will support aspects of the abbreviated forms
9.2 Textual Input of Clausal Theories 321

examined next. Additional fragments of SQL could be incorporated into our

relational external language.

Abbreviated Forms

Differently from the logical syntax of Sect. 9.2.2, SQL natively embeds many
of the abbreviations introduced in Chap. 6. Moreover, rather than factoring them
neatly as a single grammatical extension, they are deeply interwoven in the fabric
of the core syntax of SQL. The resulting language is given by the following
grammar, where the extensions are marked by triangles on the right-hand side of
each affected line:

Relational R WWD p j R UNION R j R MINUS R

Expression
j SELECT Œg æ; : : : ; Œg æ FROM p; : : : ; p Œ WHERE C 
j (R)
j R ASSUMING p WHERE C C
j p+ ΠOVER a; a j p++ ΠOVER a; a J
Aggregators g WWD MAX j : : : j FINDALL j ACCUMULATE j : : : J
j COUNT j SUM j : : : J
Conditions C WWD æ  æ j C AND C j C OR C j NOT C
j æ IN R
j FORALL .a IN p/ p WHERE C J
Attribute æ WWD a j e j op(æ; : : : ; æ)
Expressions

First of all, notice the line “R ASSUMING p WHERE C ” which is intended

to express embedded implication. This is not part of SQL. The lines marked by
black triangles provide syntax for our abbreviations. The form p+ corresponds to
the binary transitive closure p C , with the optional “OVER a1 ; a2 ” token extending
this notation to capture the more general form TRANSITIVE X1 ; X2 : P , where the
variables X1 and X2 are rendered as the attributes a1 and a2 , respectively. Both
forms evaluate to a binary relation. Next to it, the form “p++ Œ OVER a1 ; a2 
similarly expresses p CC .X1 ; X2 / and TRANSLIST X1 ; X2 : P , returning a relation
with a single list argument. Observe that lists are not traditionally supported by
SQL, but are a new syntactic category similar to numbers and strings. As in
the logical setting, lists do not interfere with the relational functionalities. The
list modifiers of Fig. 9.5 are now made available as aggregators that optionally
prefix the attribute expressions of a projection. Abbreviations in the FINDMAX,
FINDALL, and ACCUMULATE families become another type of aggregator, with
their various optional arguments turned into a combination of FROM relations
and WHERE conditions. We have renamed FINDMAX to the standard SQL MAX
construct. Finally, the FORALL X W P: P 0 form has been turned into the special
condition FORALL .a IN p/ p 0 WHERE C where attribute a corresponds to
322 9 Toward a Deductive Interface

variable X , and the condition C captures the possible identification of variables in

p and p 0 . Naturally, this syntax is extensible as indicated by the various ellipses.
The concrete SQL-like syntax we have just presented can be considered a first
attempt in the relational direction, and therefore it may not exactly match the clausal
language discussed in Sect. 9.2.2. It can however easily be refined into a complete
alternative to the logical notation.

Entry, Updates, and Removal

Entering, modifying and removing an expression in SQL syntax is done exactly as

with the purely logical notation. The one difference regards the association between
the cell range hosting a relational formula and this very expression. In Chap. 6, we
used equations, hence expressing the relational definition of path as

path D edge [ edge:From;path:To .edge ‰edge:ToDpath:From path/

which closely corresponded to the logical syntax, where the host range corresponded
to the head of the contained clauses (except for possible auxiliary clauses).
In the SQL denotation, we follow a different approach: we omit the left-hand
side of the equation as it corresponds to the cell range hosting this expression. Note
that the recursive use to path does refer to precisely this cell range. Therefore, the
SQL rendering of this expression is as follows:

=edge UNION (SELECT edge.From, path.To FROM edge, path

WHERE edge.To = path.From)

to be located in the cell range identified as path for this expression to be correct.
Note that the omission of the equational prefix path = is similar in spirit to the
fact that scalar (and array) formulas keep the location into which they are entered
implicit. Observe also that this choice prevents using named auxiliary definitions,
although they can be anonymously entered by means of the IN form of a condition.

9.2.4 Handling and Preventing Errors

Users make errors in the traditional spreadsheet, and there is no reason to expect
that this will change in the deductive spreadsheet. In this section we will concern
ourselves with the simple syntactic errors that can be caught by a good parser. Other
types of errors are more serious and harder to debug [BG87, Pan98]. We will come
back to them in Sect. 9.5.
Commercial spreadsheets have two ways to handle parsing errors in an input
formula. Simple mistakes, for example forgetting to close a parenthesis, are
corrected automatically without bothering the user (in some cases). Situations where
9.2 Textual Input of Clausal Theories 323

Fig. 9.6 Error handling in Microsoft Excel

her intentions cannot be unambiguously reconstructed are flagged with a nearly

useless dialog box such as the one reported in Fig. 9.6 from Microsoft Excel 2003—
the error dialog of OpenOffice.org’s Calc is marginally more helpful. Both happen
after the user has sealed her erroneous formula with ENTER (or CTRL-SHIFT-
ENTER if it is an array formula).
Because clausal definitions can be more complex than typical scalar or even
array formulas, it is important to provide the user with good facilities for writing
these expressions, among which we include support for the precise detection and
early correction of errors. Syntactic errors in the deductive spreadsheet go beyond
expressions that do not obey the grammatical rules of clausal theories (or our of
SQL variant). Indeed, variables are subject to safety conditions that a novice may
find hard to grasp, and clauses should be stratifiable.
We propose to extend the traditional input modality for formulas with an
on-the-fly error tracking mechanism. As the user types her formula, potentially
erroneous subexpressions are highlighted in a different color. If she hovers the
mouse over a subexpression marked in this way, a note with details about the error
appears. An example is shown in the top part of Fig. 9.7. The second clause, as
entered at that point, contains two errors: there is an open parenthesis after “edge”
without a matching closing parenthesis, and the variable Z is unsafe as it appears in
the head of this clause but nowhere in its (incomplete) body. As the user moves her
mouse over Z, the cursor changes shape to reinforce the fact that there is an error,
and a callout appears with a precise explanation of what is wrong with Z. Note
that terms that could assist the user in understanding this error are highlighted and
clicking on them will bring up help facilities. The same mechanism could be made
available for tracking errors while typing scalar and array formulas. Highlighting
errors “as-you-type” allows for early detection, provides assistance in the process
of quickly correcting them, and augments overall productivity as less time is spent
looking for errors after the fact. It should be observed that Fig. 9.7 highlights only
some of the errors found by the parser: for example, the second clause is incomplete.
It is important to report only selected errors in order not to overwhelm the user.
If the user seals a clausal definition, or a scalar formula, while some errors
are still highlighted, easily reversible mistakes are corrected automatically while
any remaining error is summarized in an error dialog box (as currently done). The
precise location of each error is reported together with the same text as in the on-the-
fly callouts. If our user had pressed ENTER in the situation depicted by the upper
324 9 Toward a Deductive Interface

(Error tracking “as-you-type”)

(Error report dialog)

Fig. 9.7 Error handling in the deductive spreadsheet

half of Fig. 9.7, she would be greeted by the dialog box in the bottom half. Observe
that, differently from Fig. 9.6, this dialog is extremely focused.

9.2.5 Integrating Logical Formulas and Traditional

Expressions

So far, our discussion of the concrete syntax of the deductive spreadsheet has
concentrated on the notation of clausal definitions as the user would enter them in
the formula input textbox. One of the requirements of Chap. 2 was however that the
deductive functionalities be seamlessly integrated into the traditional infrastructure,
and vice versa. In Chap. 4, we abstractly achieved this effect by providing two
coercions to move between the functional world of the traditional spreadsheet
and the logical (then relational) world of our extension. An array (or even scalar)
expression Ee was promoted to relational status by enclosing it in angled brackets,
as in hEe i, and dually a relational expression eN could be interpreted as an array (or
scalar) formula by putting square brackets around it, obtaining Œe. N We also added
9.3 Graphical Construction of Clausal Theories 325

some new array operators such as SORT and various order-conscious variants of
relational algebra operations (we called them “pseudo-relational”).
We will now promote these formal devices to concrete syntax. In particular,
given the concrete denotation of a clausal theory (or relational expression), we
write [ ] for its coercion into an array (or scalar) expression. We have used
this operator several times in Chaps. 4 and 5 as it extends the expressiveness of
clausal definitions by taking advantage of the functional infrastructure. Recall that
the order of tuples resulting from evaluating [ ] can be arbitrary. If we are
interested in sorting them according to some of the fields a1 ; : : : ; ai , we would write
SORT [ ] BY a1 ; : : : ; ai , where the new operator SORT. . . BY implements the
SORT operation of Chap. 4. Note that this operator can be applied to any array, not
just those obtained by coercing a relational expression. Similar syntax is provided
to represent the pseudo-relational operators.
We have already encountered the syntax <Ee>, which allows us to use an array
(or scalar) expression Ee as a predicate (or a base relation). We do not expect
this operator to be as frequently used as the reverse coercion, in the same way
as the mechanism for constructing arrays out of scalars rarely appear in actual
spreadsheets.

9.3 Graphical Construction of Clausal Theories

We have observed in Chap. 7 that commercial spreadsheets provide a number of

graphical methods to simplify entering a scalar or array formula in a worksheet.
The same shall be true of the deductive spreadsheet for several reasons. First and
foremost, because a typical clausal definition is more complex than an average scalar
formula. Second, because graphical constructions are less error-prone than textual
input as they provide immediate visual feedback of what is being entered. Third,
because building a formula with a few mouse clicks is often faster that typing it.
And fourth, because some users are simply reluctant to use their keyboard for this
kind of activity.
In this section, we will examine several graphical methods for entering a clausal
definition in a cell range. This description will parallel the examination of graphical
approaches to building scalar formulas in the traditional spreadsheet of Chap. 7.
We will rely on the logical syntax of Sect. 9.2.2, although the SQL-like language
supports similar techniques. We begin in Sect. 9.3.1 by presenting simple ways to
use the mouse to construct individual parts of a formula. Section 9.3.2 proposes an
alternative approach that relies on a series of dialogs to achieve the same effect. In
Sect. 9.3.3, we adapt the traditional copy-and-paste of scalar formulas to replicate
clausal definitions.
326 9 Toward a Deductive Interface

9.3.1 Mouse-Assisted Clause Definition

In the traditional spreadsheet, a user can enter the name of a function and then
use the mouse to select its arguments. We saw several examples in Chap. 7. We
will now propose a series of mouse-assisted techniques that will allow constructing
simple clausal theories with no need to use the keyboard except for marking the end
of a clause. More complex clauses, especially those making use of abbreviations,
will benefit from this approach, as it will reduce the amount of typing to just their
keywords and punctuation. Section 9.3.2 will present graphical methods to handle
them as well.
Simple clauses such as the ones encountered in Chap. 5 and in the earlier sections
of Chap. 6 are essentially a sequence of literals and constraints with some variables
identified. They can be completely built with the mouse as soon as we devise
methods for selecting predicates and constraints, and for identifying variables.
We will describe such methods in the remainder of this section.

Selecting Predicates

Recall that one of the effects of using the “Define Relation . . . ” dialog on a cell
selection was to enter its range or the associated symbolic name in the defined
predicates list. Once the “=” sign has been entered in the formula input textbox,
the predicates appearing in this list are available for immediate insertion, as long as
the parser determines that we are in the process of entering a clause as opposed to a
scalar or array formula.
We will illustrate how this can be done by entering the second clause of our path
example, partially displayed in Fig. 9.7. We assume that regions AWB and C WD have
been defined as the binary relations edge and path, respectively. Other relations
may have been registered, but this will be of no concern to us. Assume also that the
first clause for path has already been entered.
With the cursor after the period terminating this clause, right-clicking and
choosing the “Insert Predicate” entry makes the defined predicates list available as a
scrollable submenu as illustrated in the top part of Fig. 9.8. All entries but “path”
are grayed out because we are expected to enter the head of a clause and the selected
region has been associated with the predicate identifier “path”. Had we wanted to
enter an auxiliary predicate, we would have to type its name and arguments. As
soon as the mouse moves over the “path” entry of the defined predicates list, the
corresponding region of the worksheet is outlined. Clicking on it installs the head

=path(X,Z)

in the formula input box. The variable names X and Z are copied from the head of
the first clause and the user can rename them later if she is so inclined. They are
highlighted in red since they are unsafe up to this point. The result can be seen in the
bottom part of Fig. 9.8, ignoring for the moment the menu. The cursor is positioned
9.3 Graphical Construction of Clausal Theories 327

Fig. 9.8 Mouse-assisted predicate selection

after the newly inserted text. Note that, because there is already a clause for path,
this text could also have been copied by simply pressing ALT-ENTER. Had we used
this method to enter the first clause of this definition, its head arguments would that
been the automatically generated variable names Var1 and Var2, which the user
could later rename.
At this point, the operation can be repeated to insert the first body literal,
which will be the recursive call to path. The bottom part of Fig. 9.8 displays the
result of choosing the “Insert Predicate” submenu. Note that all the entries in the
defined predicates list are now available. The cross to the left of each predicate
allows inserting its negation. It is grayed out for path because of the stratifiability
restrictions of Chap. 6. Clicking on the “path” entry inserts the line:

IF path(Var1,Var2)
328 9 Toward a Deductive Interface

Fig. 9.9 Mouse-assisted predicate selection (continued)

as shown in the upper part of Fig. 9.9. The variable Var1 and Var2 are automati-
cally inserted. Notice that they are different from X and Z since the system has no
way of knowing how they should be related. The possible need for user intervention
is indicated by highlighting them in blue.
The process is repeated for the last line of this definition. Notice that columns
AWB are outlined when the mouse moves over the “edge” entry of the defined
predicates list. The following line is inserted:

AND edge(Var3,Var4)

as shown in Fig. 9.9. Typing a period will signal that this clause has been completed.
9.3 Graphical Construction of Clausal Theories 329

Fig. 9.10 Mouse-assisted insertion of a constraint

Identifying Variables

The resulting clause is shown in the bottom part of Fig. 9.9. Clearly this is not
what we want since all variables are different. In order to identify two variables,
for example X and Var1, we simply drag X onto Var1. The cursor changes shape
as we do so. Both instances of X will become black as the one in the head is not
unsafe any more and the user has paid attention to the first blue variable in the first
body literal. We can similarly drag Z onto Var4, and Var2 over Var3, which will
again have the effect of making them black. The latter operation will result in both
body literals sharing the variable Var2. Had we wanted to supply another name,
for example “Y”, we would have needed to rename Var2 before dragging it onto
Var3. We can also rename both instances after the fact.
By the end of this simple process, we have produced the desired clause, by using
the mouse exclusively, except for the period that signals the end of the clause. The
process can now be repeated with a third clause for path if we had one in mind.
Observe that we could have started renaming and identifying variables as we were
inserting literals.

Inserting Constraints

We will now extend this basic process with a mouse-oriented technique borrowed
from [SBB96] for conveniently inserting constraints. We will illustrate it again on
our ongoing example, but we modify it somewhat to make it more interesting.
Assume that edge describes direct flights between airports, so that path corresponds
to an air itinerary. We enter the three-letter code of the origin and destination airport
of each direct flight in columns AWB, the result of which is partially visible in
Fig. 9.10. As we do so, a small upside-down triangle appears next to the field names
330 9 Toward a Deductive Interface

of edge whenever we are in formula input mode. Clicking on this triangle reveals a
pull-down menu with all the entries in this field, sorted alphabetically and with any
duplicate removed. This appears on the left-hand side of Fig. 9.10.
Assume that we are only interested in itineraries that do not transit through LAX.
Then, we simply need to modify the definition of path so that the intermediate
variable Y of the second clause of Fig. 9.9 is different from the text string “LAX”.
Logically this is achieved by simply adding the constraint Y ¤ LAX’.
This can be realized in several ways in the formula input textbox. Clearly, one
can always type this constraint, which is rendered as “Y<>"LAX"” in the concrete
syntax of our language. Another possibility is to drag the variable Y past the last
literal (or between any conjunct in the body), type “<>”, move the mouse over to
one of the fields of edge and select “LAX” from the pull-down menu, as shown in
Fig. 9.10. It will be sufficient to seal the clause with a period.
An alternative way to insert this constraint would begin by clicking on an
occurrence of the variable “Y” in the clause and selecting “LAX” from the pull-down
menu (or clicking on it if it appeared in the formula input textbox). This will install
the line

AND Y = "LAX"

Now, right-clicking on the default “=” reveals a submenu of comparison operators

to choose from. It is then sufficient to select “<>”.

Complex Clauses

The techniques we have just discussed do not allow us to automatically insert

complex constraints or abbreviations. The user has to type them in, or rely on
dialogs. She can however use the mouse to deliver arguments to them as just
illustrated or as in the traditional spreadsheet (see Chap. 7).

9.3.2 Dialog-Assisted Clause Definition

As we saw in Chap. 7, the traditional spreadsheet provides a way to build a formula

by choosing its constituent functions from a catalog, which the user can invoke
by pressing the fx button. Once the desired function has been selected, a dialog
appears and assists her by describing the nature of each argument as well as the
operation performed by the function itself. This is particularly useful for complex
or unfamiliar functions, and this input modality is not available for infix operators.
We provide similar facilities for the purpose of building clausal theories.
They are not limited to assembling the simple clauses we have encountered in
Sect. 9.3.1. Indeed, they also make an easy-to-use interface available for working
with abbreviated forms, which can be rather complex.
9.3 Graphical Construction of Clausal Theories 331

Entering a Clause from the Function Catalog

Consider first the problem of entering a clause from scratch. Rather than starting
to type in the formula input textbox, the user can press the fx button, which will
show the catalog displayed in the upper part of Fig. 9.11. Note that the traditional
left pane of this dialog has been extended with one new entry, “Insert a new
clause . . . ”. Clicking on it reveals the commentary at the bottom of this window,
but no function name appears on the right pane. Pressing “OK” will propose a blank
version of the dialog shown in the bottom part of this figure. If the user has already
entered a complete clause in the formula input textbox, pressing the fx button will
bring up the clause definition dialog directly since this context expects a clause and
not a traditional function. This dialog can also be triggered by entering “=” in the
formula input textbox and clicking on “Insert clause” on the simplified function
selection menu that now appears at the far left of the formula bar.

The Clause Definition Dialog

The clause definition dialog, which has been modeled after the function input dialog
of commercial spreadsheets discussed in Chap. 7, is shown in the bottom half of
Fig. 9.11. It as been partially filled as per the example in Fig. 9.10. Note that,
following the conventions of Sect. 9.3.1, variable “Z” is highlighted in red since
it is not safe, and variable “Var4” is colored in blue to suggest that the user may
want to make some changes to it.
The top text field is intended for the head of the clause and will generally have
automatically been filled, although the user can always make changes. The fields
in the “Body” enclosure are where she will specify each conjunct in the body.
Predicates and constraints are entered either using the keyboards or by means of
the graphical methods discussed in Sect. 9.3.1: in particular right-clicking brings up
the defined predicates list, constraints can be entered by clicking, and variables are
identified by dragging them over other variables. Additionally, the button labeled
fx on the right of each input field allows accessing the library of abbreviated forms,
which will be examined shortly. It can also be used to enter embedded traditional
formulas in a clause. New conjunct fields are automatically proposed once text is
entered in the last blank row, which is the way traditional multi-argument functions
such as SUM behave in dialog-assisted input mode.
Navigating among the various fields specializes the explanation at the bottom
of the windows. Clicking on any of the highlighted keywords will invoke targeted
help facilities. General help can be accessed by depressing the “Help” button, while
“Example” will display a simple example. Hovering over a portion of text marked
in red will bring up a callout describing the error, similarly to Fig. 9.7. Once the user
has finished entering a clause, pressing “OK” will validate it as if she had entered
it in the formula input textbox and pressed ENTER (see Sect. 9.2). Before doing so,
she can check the “Define another clause” checkbox to be brought back to an empty
clause definition dialog when she is done.
332 9 Toward a Deductive Interface

Fig. 9.11 Dialog-assisted clausal entry

9.3 Graphical Construction of Clausal Theories 333

Dialog-Assisted Entry of an Abbreviated Form

In the situation described by the bottom half of Fig. 9.10, assume the user wants to
specify that she only is interested in changing planes in airports such that every
outbound direct flight has a corresponding inbound direct flight from the same
city. At the logical level, this is achieved by adding the conjunct FORALL V W
edge.Y; V /: edge.V; Y /. Because of the presence of the abbreviated form FORALL,
this goes beyond what we could do so far with the mouse. It is however possible to
bring up a wizard that can help the user construct this formula step by step. In
order to access it, she simply needs to move the cursor to a blank field (as shown in
Fig. 9.10) and click on the fx button on the right-hand side of this field.
This brings up the function catalog as shown in the top part of Fig. 9.12. Notice
that it differs in several ways from what we saw in Fig. 9.10: the item “Insert new
clause . . . ” has been replaced with a proper category “Deductive”, and the right pane
lists all the available abbreviations. Notice also that all traditional functionalities are
grayed out since they cannot be entered where a body conjunct is expected. Had
we been in a scalar or array context (for example within a constraint), “Deductive”
would be disabled while all the traditional functions would be available.
Clicking on “FORALL” invokes the wizard shown at the bottom of Fig. 9.12,
which the user would fill similarly to the clause definition dialog examined earlier.
“FORALL” is particularly simple since it does not have the optional parts of other
abbreviated forms. Note also that none of the fields contains the fx button since
neither variables nor predicates can make use of any of the functionalities provided
by this dialog. Other body objects, for example embedded implication (ASSUMING
in our concrete syntax), would have this button for some of their arguments.

9.3.3 Copy and Paste

We conclude this section by revisiting the copy-and-paste facilities of the traditional

spreadsheet. For the time being, we will limit our discussion to cutting, copying and
pasting whole clausal definitions.
Cell ranges containing a set of clauses can be liberally cut, copied and pasted to
other locations, but the effect of this operation is somewhat different from what we
have seen for scalar and array formulas. Indeed, sometimes the user will want to
apply the same clauses to new data, while other times she only wants to relocate this
definition to a more convenient place. The first possibility interprets the coordinates
of all predicates mentioned in the clauses as relative, which corresponds to the
semantics of the scalar copy-and-paste. In the second case, all but the head predicate
of any contained clause is considered as if the underlying cell ranges were specified
using absolute coordinates. Other cell references are handled according to the way
they have been entered. Because the user will sometimes want to proceed one
way and other times the other way, we propose that pasting a clausal definition
be accompanied by a confirmation dialog that lets her choose how to handle the
334 9 Toward a Deductive Interface

Fig. 9.12 Entering an abbreviated form using a dialog

9.4 Updates 335

contained predicates. Once she has selected one of the modalities, she can also check
an option making this choice a default for future paste actions.
Because symbolic predicate names are meant to be unique, pasting a relation to
a new place forces us to remove the name from the copy. The names of its fields
will remain unchanged, and the relation as a whole will be entered in the defined
predicates list, but the user needs to explicitly enter a new name if she wants to refer
to it symbolically.

9.4 Updates

In this section, we examine the effect of a variety of common actions that the user
may perform to modify the layout of a relational entity. In particular, we discuss how
the system should behave when she alters the geometry of a relation in Sect. 9.4.1.
The consequences of inserting or deleting rows or columns across relations are
investigated in Sect. 9.4.2.
In the second part of this section, we look at update propagation and introduce
some new options the user can avail herself of for the purpose of controlling it. This
will be the subject of Sect. 9.4.3.

9.4.1 Altering the Geometry of a Relation

Modifying the geometry of a relation opens some challenging questions. Something

as simple as changing its length, i.e., the number of records it can hold, may have
the effect of leaving some tuples out. Inserting a field may have repercussions
on clauses that depend on it, and similarly for removing an attribute. We will
now examine these various issues in detail. In this section, we will be referring
to a relation whose fields are aligned vertically, with horizontal records. The dual
situation is handled similarly.

Modifying the Length of a Relation

We begin with an investigation of what should happen when the user alters the length
of a relation, i.e., the number of records it can contain. Before describing the effect
of such a modification, we discuss the means she has at her disposal to trigger it in
the first place.
The most general way to change the length (and indeed all geometric aspects)
of a relation is to invoke the “Define Relation . . . ” dialog examined in Sect. 9.1.1
and alter its vertical extension. As the user modifies the length portion of the
“Range” text box, the common length of all of its fields are adjusted automatically
to match the height of the relation. She could similarly modify its starting row. In
336 9 Toward a Deductive Interface

Fig. 9.13 Invoking the “Redefine Relation” dialog

order to invoke the “Define Relation . . . ” dialog for an existing relation, the user
simply needs to right-click on one of its cells and select the menu option that reads
“Redefine Relation . . . ”. If this cell belongs to more than one relation, a submenu
will appear from which she will be able to choose which of the intersecting relations
she wants to modify. This is shown in Fig. 9.13.
The height of a relation can also be modified more conveniently by using the
mouse. If the relation is not unbounded, one can select it, go to its last row and drag
the black rectangle at its bottom right corner to the desired row position. This is
practical only for small changes in height. Of course, this is particularly impractical
if one wants to make this relation unbounded. In that case, right-clicking on the
black rectangle will produce a menu with an option reading “Extend Relation To
Last Row”: selecting it will make the table unbounded. If instead the relation is
unbounded, right-clicking on any of its cells will produce the menu item “Shrink
Relation”. Selecting it will reduce the length of this relation to its last non-blank
record. If more than one unbounded relation overlaps this particular cell, the user
will have the possibility of choosing on which of them to operate by means of a
submenu. This menu item is shown in Fig. 9.13.
9.4 Updates 337

The user has now modified the height of a relation. The effect of this action
depends on whether this table was clausally defined or just identified some area of
the worksheet as a predicate, possibly used in a clause elsewhere.
In the first case, there will simply be a different amount of space to store the com-
puted records. Recall from Part I that the evaluation result of a relational expression
or clausal definition will start inserting tuples from the top of the target area down
(for a vertical table, left-to-right if it is horizontal). If this space is too small, some
records will not show (the output is truncated). If it is too large, the unused area will
be left blank. This is consistent with how traditional array formulas operate.
If the modified region did not contain a clausal definition, it will simply
encompass a different amount of data. This means that clauses that referred to this
relation as a predicate may be working with either more or fewer data tuples.

Modifying the Width of a Relation

The most challenging aspect of modifying the geometry of a relation is concerned

with making changes to its field structure. This has implications not only on the
clauses it may define, but also on any logical formula that depends on it. As for
changes to the length of a relation, we first examine how fields can be modified and
then discuss the implications of these actions.
The cleanest way to make changes to the attributes of a relation is by invoking
the “Redefine Relation . . . ” dialog. There the user can add, remove, merge, split,
and rename fields. Any inconsistencies will be flagged for immediate correction.
Alternatively, she can use traditional commands to insert or delete entire columns.
The “Paste special . . . ” of LibreOffice/OpenOffice.org’s Calc additionally allows
her to shift a column or row when pasting.
Reshaping a relation in any of these ways has an effect on its internal structure
and on how it is referenced by clauses that make use of it. The internal effect of
adding or removing a column depends on whether this happens in the midst of a
list-valued (multi-column) attribute or not:
• If a column is inserted or deleted from a multi-column field, nothing essential
changes structurally, except that this attribute has one more column, or one fewer.
Nothing changes with respect of the clauses that may reference it.
• Deleting a column corresponding to a single-valued attribute simply removes it
from the field structure of the relation. If it is cut, it may be worth remembering
its name in case it is immediately pasted within the same relation as a way to
change the ordering of attributes. In general, inserting a new column outside of
a multi-column field has the effect of creating a new unnamed single-column
attribute. The user can then later invoke the “Redefine Relation . . . ” dialog to
assign it a name or to merge it with adjacent fields.
If the relation in which a field is inserted contains a clausal definition, the
corresponding predicate will be given one new argument, i.e., a fresh variable
will appear in the appropriate position of its argument list. This new variable will
338 9 Toward a Deductive Interface

necessarily be unsafe and hence require user intervention in order to restore the
validity of this definition. A fresh variable will also appear in every clause in which
this predicate is used. This is inconsequential in the case of positive literals, but
again leads to loss of safety for negative literals. We therefore see that inserting a
new field has the potential of triggering a series of corrective actions by the user.
This leads to an increase of viscosity, although this appears to be necessary in order
to preserve the overall consistency of a deductive spreadsheet (where this word is
not used as a cognitive dimension).
Removing a field leads to similar concerns. The deletion of a head variable has
no consequence, but removing a variable in a positive body literal containing this
predicate may have the effect of making other occurrences unsafe and therefore may
require action by the user.

9.4.2 Cut, Copy and Paste Across Clausal Definitions

In the previous section, we have discussed the consequences of inserting or deleting

fields in a relation. We will now look at what happens when rows are inserted or
deleted. Insertion can happen because we paste new values over a record, or because
we insert a new row, possibly pasting some values as we are at it (this can be
done atomically in LibreOffice/OpenOffice.org’s Calc, but not in Excel). Similarly,
deletion can result from the removal of an entire row or from blanking out just a
few cell.
When either of these operations take place on an extensional relation, an
appropriate record is simply inserted, removed, or modified. The changes are
immediately propagated, but there are no other consequences on the overall relation
itself.
If this relation is clausally defined, adding or removing a record has no effect
whatsoever: the change is simply ignored. The values displayed in it remain
unchanged. We have two choices for the length of the relation: it can either stay the
same, as we have seen that the user needs to take other types of action to modify this
property, or it can be adjusted on the basis of the update as when inserting or deleting
a row in current applications. This is quite different from the current behavior of
array formulas, where any operation partially overlapping them immediately brings
up an error dialog. We believe that the model we are proposing for clausal definitions
could be ported to array formulas, although, for the sake of conservativity, it is not
our intention to implement it.

9.4.3 Controlling Evaluation

In the default configuration, any change to the data or layout of a traditional

spreadsheet is readily carried out and immediate visual feedback is provided. In
9.4 Updates 339

particular, any update to the contents of a cell, whether a value or a formula, is

instantly propagated to all the cells that depend on it, directly or indirectly. This
default recalculation policy can be modified through global preferences: advanced
users working with very large and complex spreadsheets often disable automatic
recalculation for efficiency reasons. Our deductive extension does not change this
state of affairs: instantaneous automatic recalculation remains the default, but can be
overridden by modifying global settings. However, as motivated in Chap. 4, the fact
that we are dealing with relations suggests a smarter approach, especially because
recalculating a clausal definition is more onerous in general than re-evaluating an
equal number of traditional formulas.
As observed in Chap. 4, in a traditional spreadsheet, where only scalar and array
formulas are present, modifying an individual cell often coincides with a logical
update, a change to a chunk of data that the user sees as a unit. Pasting or deleting
a multi-cell selection is the one exception to this rule. In the deductive spreadsheet,
it is natural to identify a logical update unit with a record, especially during data
entry. Indeed, launching re-evaluation after the user has entered each individual
attribute is often a waste of effort in this case. Therefore, whenever a cell which
is part of a record is updated, we delay recalculation until the rest of the record has
been updated as well, up to a maximum recalculation delay that we denoted by t .
Indeed, we proposed the following default recalculation policy for the deductive
spreadsheet:
• If the modified cell is not within the scope of a record, launch recalculation as
soon as the user has selected a different cell.
• If the modified cell is part of a record, proceed as follows when the user selects a
different cell:
– If the new cell is not part of the record, invoke recalculation;
– If it is part of the record and the user does not update it within t milliseconds,
launch recalculation;
– Otherwise, repeat the procedure for this new cell.
Note that some updates do happen one record at a time, for example when pasting a
row across a relation, or when using Excel’s data forms to insert a new tuple.
Modifying a clausal definition is clearly a logical update unit on its own and
must be propagated immediately, unless the user has selected another method in the
global preferences.
Figure 9.14 upgrades the global “Options” menu of Microsoft Excel to account
for the recalculation delay t of records. There, we have specified a default
of 750 ms as an educated guess. A more appropriate value will be determined
experimentally on the basis of user feedback.
This figure also shows an entry for the maximum number of recursive invocations
that a clausal definition containing a head constraint should be allowed to undergo.
We called this number hcmax in Chap. 6. Again, the shown default value, 500, is an
educated guess, to be confirmed or adjusted experimentally.
340 9 Toward a Deductive Interface

Fig. 9.14 Global recalculation preferences for the deductive spreadsheet

9.5 Debugging

Raymond Panko’s recent report on errors in spreadsheets [Pan98] came as an

expected surprise among researchers and practitioners alike. Indeed, it was known
that spreadsheets often contained errors, but the magnitude of this phenomenon,
clearly documented in the report, took many by surprise: the error rate is often over
70 %. Many of these errors were due to incorrect formulas, often because wrong cell
references had been entered (coding error), sometimes because the entered formula
did not solve the problem (conceptual error). Syntax errors, easily flagged by the
parser as formulas are entered, were not considered.
The traditional spreadsheet could be made more error resistant, but at the expense
of other cognitive dimensions: higher abstraction, reduced closeness to mapping,
added premature commitment to cite just a few. The net effect would be to make it
harder to use, which would have the effect of alienating a large number of users.
Instead, commercial applications provide a number of tools to facilitate finding
errors after they have been committed: clicking on the formula bar highlights
referenced cells using different colors, the audit tool traces dependencies as blue
arrows, and the function input dialog can show intermediate results of evaluation.
We have examined them in Chap. 7.
9.5 Debugging 341

Fig. 9.15 Asking for an explanation

The deductive spreadsheet must clearly extend all of these debugging modalities
to the added functionalities it embodies. In the proposed design, the one exception
to this rule concerns the clause definition dialog discussed in Sect. 9.3.2, for which
it is not clear how to provide usable feedback for intermediate values. The graphical
formula entry modalities described in Sect. 9.3 also help create formulas that are
correct on the first try, at least as far as cell references are concerned.
Clausally defined relations introduce a new debugging challenge as they do not
fall into existing patterns. As repeatedly observed in Part I, highlighting relational or
attribute-level dependencies is often of limited use because the typical questions that
a user will want to ask are “Why was this tuple returned?” and “Why was this tuple
not returned?”. What should be tracked is the process that leads to the inclusion
or exclusion of this particular tuple. We developed the technology to answer these
questions in Chaps. 5 and 6: top-down evaluation. We will not weave it into the
interface of the deductive spreadsheet.
We will first discuss positive explanation, i.e., the answer to the question “Why
was this tuple returned?”. The easiest way for the user to ask it is to select a suspect
record from a calculated clausal definition, and right-click on the “Explain . . . ” item,
as shown in Fig. 9.15 in the case of why .JFK; LAX/ is in the relation path. The
342 9 Toward a Deductive Interface

Fig. 9.16 Deductive explanation dialog

explanation facilities can also be invoked from the menu bar, even if no record
is selected (this is useful when one wants to ask negative questions or to issue a
one-time query). Additionally, it is possible to invoke the explanation dialog with
partial records (the omitted parts will be replaced by variables) and on several
records at once (all of them will be explained).
The explanation dialog is shown in Fig. 9.16. The textbox at the top displays the
tuple that has been selected as a query. The user can modify it to inquire about other
tuples. Indeed, we have replaced the constant “LAX” with the variable “Z” for the
sake of illustration. The left-hand side of the central part shows a derivation tree
for this query, displayed as a series of layered subgoals in a way reminiscent of
a directory browsing dialog. The node at the very top of this pane represents one
instance of the query. Other instances can be selected by clicking on the downward
triangle next to it: in this simple example, the other possibilities for the second
argument are “BWI”, “SFO”, “MSY”, “IAD”, “EWR”, and “VCE”.
The rest of this pane shows one derivation tree for the goal path("JFK",
"LAX"). Clicking on this node brings up one of the clause instances that make
it derivable on the right pane of the dialog. Positive literals are shown in different
colors, and the actual tuples are highlighted using the same color in the worksheet.
The user can request that they be brought into focus by double-clicking on them,
hence increasing visibility. These subgoals also appear on the left pane together with
their own subderivations, aligned and connected to path("JFK","LAX"). The
user can ask to see other clause instances that satisfy this goal by using the two small
black triangles on each side of the edge that extends out of it. In this case, there is
one additional possibility: taking a direct flight from “JFK” to “LAX”, which is not
displayed. The first subgoal, path("JFK","SFO") is also defined clausally and
clicking on it brings up the instantiated clause that derives it. The second subgoal,
edge("SFO","LAX"), is instead a fact. Clicking on it will simply show a color-
coded reference to the spreadsheet. In more complex examples, subderivations can
be rather large, and therefore we provide the ability to collapse them using the gray
9.6 Productivity Tools 343

triangles on the clause instance edges. Moreover, the user may want to compare
clause instances for several subgoals of a derivation. This can be done by dragging
the tabs on the right pane of the window.
Constraints are not directly represented in the derivation tree, but can be
inspected on the right pane. Any cell reference they may contain will appear in color
and the corresponding cell will be highlighted using the same color. Traditional
auditing mechanisms can be invoked if there is any reason to suspect an error in
this reference. Abbreviated forms are displayed on the derivation tree and clicking
on them will bring up dedicated explanation dialogs. Negative literals possibly
appearing in a clause will be discussed shortly.
Since the user can write any body formula in the “Query” text fields, she may
construct relations that she actually wants to save. Checking the box labeled “Save
query as relation” will prompt her to choose a target for the newly constructed
clause, to which she can assign a named predicate.
Answering a negative explanation request, “Why was this tuple not returned?”,
is more complicated as we need to “show” the user the non-existence of a record or
record template. She can enter such a request in the query box of the explanation
dialog, or it can emerge in the derivation of a positive literal. A non-derivable
goal is displayed in a grayed-out slanted font in the derivation pane. If it is a
factual predicate instance p.Ev/, it will not have any descendant and clicking on
it will display the note “No record .Ev/ in relation p” in the right-hand pane, with
a color-coded association to the extension of p in the spreadsheet. If instead it is
clausally defined, the body of one of its clauses will be displayed: its literals will be
instantiated to the extent to which the head is, and the ones that are derivable will
show a derivation below them. The user shall be able to instantiate the remaining
variables, which will have the effect of marking one or more of the remaining literals
as non-provable. She will then be able to repeat the process on one or more of them
until she is convinced of the non-derivability of the original goal.

9.6 Productivity Tools

As observed in Chap. 7, traditional spreadsheets come with a number of tools

aimed at visualizing raw data in an intuitive manner, hence making them more
readily usable, augmenting productivity, and improving the user’s experience (or
her customers’). Many of these tools target tabular data, allowing to summarize
them as charts or graphs, and to build complex what-if analysis scenarios with a
few mouse clicks.
The deductive spreadsheet must clearly support all of these tools. Additionally,
the emphasis on symbolic data and the presence of effective logical methods to
manipulate them open the door to new opportunities. Consider once more the path
example in its aviation instantiation of Fig. 9.10. The relation edge may conveniently
be visualized as a graph connecting two airports by means of an arrow whenever
there is a direct flight between them. If we augment edge with a new field containing
344 9 Toward a Deductive Interface

Fig. 9.17 A connection graph

the distance of each segment, as we had done in Chap. 6, we can use the distances as
labels for these arrows. Figure 9.17 shows a dialog for building such a connection
graph, and overlays it with an identical graph that only shows segments whose
distance is greater than 2,500 miles (those arrows are rendered in red in a color
version of this book). If the user wanted to see the cumulative distance of entire
itineraries, she may add a new query referring to path rather than edge.
Another potential accompanying tool for the deductive spreadsheet is a flow
graph generator. There, the input would be a recursive query returning a list, such as
what we used to describe the work-flow problem in Chap. 6, and the output would
align the elements of this list over all possible answers and possibly annotate them
with measures of time, cost, or effort.
The problem of visualizing tabular data, highlighting their dependencies, has
received a lot of attention in the literature, especially since the advent of the World-
Wide Web. Some of these visualization and query techniques, for example the
methods described in [SBB96], could conveniently be imported in the deductive
spreadsheet. A more detailed investigation is beyond the scope of this book.

9.7 Getting Help

Modern commercial spreadsheet applications come equipped with overarching help

facilities. An extensive library of help topics can be invoked from most parts of the
program with just one mouse click or a keyboard key press (Microsoft Excel used
9.8 Summary 345

to also offer the unpopular ClipIt office assistant). It allows searching help topics by
entering a few keywords, by using an alphabetic index, and by browsing categories
as they would be organized in a book. Help can be sought online through links to the
manufacturer’s website. On a smaller scale, just hovering the cursor over a button
shows a descriptive note explaining its function.
The deductive spreadsheet shall not only retain these various help modalities,
but include dedicated topics referring to the various extensions it offers over the
traditional spreadsheet. We have already seen throughout this chapter that the design
of the various dialogs and other facilities already contains buttons and links to the
help infrastructure.

9.8 Summary

In this chapter, we have presented a detailed tentative design touching the major
aspects of the user interface of the deductive spreadsheet. We have engineered it
as a conservative extension of the user interface of the traditional spreadsheet, and
indeed many of the illustrations have been grafted on top of screenshots of Microsoft
Excel 2003. The main effect of this conservative approach is that a user who is not
interested in the deductive extension may hardly notice the presence of the half
a dozen new commands sprinkled here and there, often within context-sensitive
right-click menus. A user who does want to avail herself of this far-reaching
extension can do so in a natural way, often by a simple mouse click. Clausal
definitions can be constructed using traditional methods such as by entering them in
the formula input textbox, by selecting arguments with the mouse and by invoking
dedicated dialogs. We proposed further extensions to ease the fact that clausal
definitions are often more complex than an average scalar or array formula. In order
to accommodate users with different backgrounds, we suggested providing two
input languages for logical formulas, one that transliterates the clausal definitions we
have used in Chaps. 5 and 6, and a relational one based on the SQL database query
language. We have enriched the auditing toolkit of the traditional spreadsheet with
convenient graphical ways to inquire about the derivability, or non-derivability, of a
record, and in general of an arbitrary clause body. We have also proposed extensions
to the array of productivity tools in order to tap into the new opportunities offered
by the deductive infrastructure.
Major and minor decisions concerning the design of the user interface of the
deductive spreadsheet have been guided by the desire to remain conservative with
respect to the traditional interface and by the cognitive principles discussed in
Chap. 8. In particular, we often framed decisions points in terms of the attention
investment model and dissected the merits of different options by bouncing
appropriate cognitive dimensions around. Ultimately, the wisdom of the tentative
choices we made will be decided by the end users of this technology through testing
and other feedback. A very preliminary effort in this direction is indeed the subject
of Chap. 10.
346 9 Toward a Deductive Interface

With this design for the deductive spreadsheet user interface in hand, we will
now review our achievement against the cognitive requirements stated in Chap. 2.
This design appears to meet or exceed each of these requirements:
Cognitive Conservativity All traditional functionalities are accessible to the user
in the exact same way as in a traditional spreadsheet. The deductive interface is
indeed conservative and unobtrusive to the point that the extensions are hardly
noticeable until she starts working with the deductive spreadsheet.
Consistency The commands supporting the deductive extensions are consistent
with the existing modalities of interaction. Indeed, the user can enter clausal
definitions using the keyboard or by means of intuitive mouse commands akin
to what she is used to for traditional formulas. Additionally, enhanced input
modalities have been made available.
Cognitive Integration Using the deductive interface additions is expected to be
natural and intuitive, including in conjunction with the traditional functionalities.
Deductive and traditional operations are fully integrated and accessible by means
of consistent commands.
Performance While the performance of the deductive spreadsheet, as experi-
enced by the user, cannot be assessed until a prototype has been developed,
preliminary experiments are encouraging. See Chap. 11 for details.
Discovery The deductive extension is expected to have a gentle learning curve,
so that users can reinforce and extend their skills by simply using the tool and
minimal experimentation. Commands operating on the deductive extension are
clearly marked and generally accessible by a single mouse click. Error messages
are generally more informative than in traditional spreadsheet implementations,
and help facilities are extended to cover every aspect of the deductive extension.
Clearly, the last two items in this list need to be established experimentally on the
basis of user testing and reported feedback.
Chapter 10
Preliminary Experiments

In this chapter, we describe a small-scale and very preliminary experiment on

potential users of the deductive spreadsheet and report on their feedback. We
interviewed eight current users of traditional spreadsheet applications, principally
Microsoft Excel, with diverse backgrounds. After reviewing their experience with
these tools, we explained the general idea underlying the deductive extension and
noted their reactions. We then went on to show them a variety of screenshots of the
deductive spreadsheet, adapted from Chap. 9, and asked for feedback.
Altogether, this experiment showed that our target audience, namely intermediate
and advanced users, could easily grasp the potential of the deductive spreadsheet.
Indeed, they suggested several uses for it in their current activities. They commented
favorably on the user interface as designed in Chap. 9. To our surprise, also
beginners, who were not a target category in Chap. 8, showed interest in the basic
relational functionalities, although they had difficulties with some of more advanced
concepts such as recursion. Beginners took issue with the wording of menu items
and other interface text and demanded friendlier approaches to building simple
clausal definitions. Tested users across the board were very positive about the
integration of persistent relational query facilities in a spreadsheet, a key aspect
of the deductive spreadsheet we extensively examined in Chaps. 4 and 5.
This chapter is organized as follows. In Sect. 10.1, we describe the various
aspects of the common experiment we used for this preliminary evaluation. In
Sect. 10.2, we introduce the volunteers who kindly participated in it. Section 10.3
analyzes the results. Finally, we give a summary of the gained insight in Sect. 10.4.

10.1 Outline of the Experiment

Our experiment consisted of three phases. We first asked our volunteers about their
backgrounds using spreadsheets in order to determine their skill levels. We then
illustrated the key concepts behind the deductive spreadsheet, mostly on the basis
of motivational material taken from Chap. 1 and by means of a simple example.

I. Cervesato, The Deductive Spreadsheet, Cognitive Technologies, 347

DOI 10.1007/978-3-642-37747-1 10, © Springer-Verlag Berlin Heidelberg 2013
348 10 Preliminary Experiments

We finally showed them selected screenshots of the user interface, as developed

in Chap. 9. Feedback was collected at each step and at the end of each interview,
they were asked for suggestions to improve the experiment itself. These phases are
discussed in more detail in Sects. 10.1.1 through 10.1.3. We initially considered
asking our volunteers to perform a simple exercise, but eventually desisted as it
would have been difficult to realize without a working prototype.

10.1.1 User Background

In this phase, we asked our volunteers about their experience using spreadsheets.
Its purpose was to classify them according to their skills in order to correlate the
collected results of our experiment with user segments.
The specific questions we asked include:
• For how long have you been using spreadsheets?
• How frequently do you use them now?
• Which applications do you use most?
• Do you use them for work?
• What do you use spreadsheets for?
• Which of the following activities have you performed with a spreadsheet?
1. Manage lists?
2. Perform simple calculations? (arithmetic operations, use SUM over a single
worksheet)
3. Perform complex calculations? (use complex functions, array expressions,
nested formulas, multiple worksheets)
4. Define symbolic names?
5. Work with data lists, data forms, database functions?
6. Use data tables, goal seeking, pivot tables?
7. Program new functionalities? (using Visual Basic for Applications, for
example)
• What would you like a spreadsheet to include that is not already there?
We mostly used the items under “Which of the following activities have you
performed with a spreadsheet?” to classify users in the following categories:
Beginners whose experience is mostly limited to 1 and 2.
Intermediate users who have also used 3 and maybe 4.
Advanced users who have taken advantage of 5 and 6.
Spreadsheet programmers who have done 7.
The exact form we used is shown in Fig. 10.2.
The outcome of this phase is summarized in Sect. 10.2 where we introduce our
volunteers and their backgrounds, and classify them on the basis of their skills.
10.1 Outline of the Experiment 349

10.1.2 Illustration of the Deductive Spreadsheet

In this phase, we gave our motivations for extending the traditional spreadsheet
with functionalities supporting deductive reasoning. We also outlined our approach
to incorporating deductive facilities in the spreadsheet, namely the use of clausal
statements over relational data, and illustrated it with a simple example. We also
collected feedback about our overall approach. User interface issues were not
addressed.
The purpose of this phase was to test the reaction of our volunteers to the very
idea of a deductive extension to the spreadsheet, to understand if they needed
anything like this, and to comment on our approach to realizing this objective as
described in the body of this book.
We will now go over each of the individual aspects of this phase of the protocol:
motivation, example and feedback collection. The exact phrasing of our explana-
tions and questions was tuned to the background information collected in Sect . 10.2.

Motivating the Deductive Spreadsheet

We first provided motivations for the very idea of a deductive extension to the
traditional spreadsheet. We observed that current applications provide outstanding
support for working with numerical data and making decisions based on them.
However, a large percentage of the decisions we make every day depend on informa-
tion that is not numerical, or not only numerical. Therefore, it is not always possible
or obvious how to use a spreadsheet, or any office automation tool, to help making
these decisions. We illustrated this point by presenting the following example, which
is inspired by material we have repeatedly examined in Chaps. 6 and 9:
Is there a direct flight from New Orleans to Venice? To where can I fly directly from New
Orleans? Is it possible to fly from New Orleans to Venice? What is the minimum number
of connections? What would be the itinerary? How many frequent flyer miles would I
accumulate? Would it help if I waited until the new direct flight from Atlanta to Venice
starts?

We asked each volunteer if he or she knew how to model the overall problem in the
traditional spreadsheet and how to have a spreadsheet answer these questions.

Example

We then went on describing how the deductive spreadsheet could help answer these
questions. We mentioned that the origin and destination airport of each direct flight,
together with the number of miles they are apart, could be organized in the table
shown in Fig. 10.1. We then illustrated the following solutions to each of the above
questions.
• Is there a direct flight from New Orleans to Venice? This question can be
answered by simply inspecting the relation directFlight, or by using any
350 10 Preliminary Experiments

Fig. 10.1 Specification of the direct flights between airports

standard filter function on data lists. The airport codes for New Orleans and
Venice are MSY and VCE, respectively.
• Where can I fly directly from New Orleans? We answer this question by simply
collecting all the destinations of a directFlight record with MSY as its
origin:

fromMSY(To) IF directFlight(From,To,_)
AND From="MSY".

• Is it at all possible to fly from New Orleans to Venice? We informally described

the notion of transitive closure, and used it to solve this problem by means of the
following clauses:

indirect(From,To) IF directFlight(From,To,_).

indirect(From,To) IF directFlight(From,Mid,_)
AND indirect(Mid,To).

and looking for an entry that relates New Orleans (MSY) and Venice (VCE). This
can also be abbreviated as:

indirect(From,To) IF directFlight+(From,To,_).
10.1 Outline of the Experiment 351

• What is the minimum of number of connections? We showed that the above

example can be modified as follows to take the number of hops in an itinerary
into account:

indirect(From,To,N) IF directFlight(From,To,_)
AND N = 0.

indirect(From,To,N) IF directFlight(From,Mid,_)
AND indirect(Mid,To,N’)
AND N=N’+1.

The entry with the minimum number of connections between New Orleans (MSY)
and Venice (VCE) can be found by inspecting the result of this definition and
picking the smallest value of the third argument for these airports. Since there
may be several entries for them, it can be refined by entering the following clause:

minConnections(From,To,Min)
IF Min = FINDMIN N. indirect(From,To,N).

If only the entries for MSY and VCE are of interest, it can further be specialized
as follows:

minConnections(From,To,Min)
IF Min = FINDMIN N. indirect(From,To,N)
AND From="MSY" AND To="VCE".

• What would be the itinerary? We extend the last definition for indirect by
accumulating the origin and intermediate airports into a list “Itin” as follows:

indirect(From,To,Itin,N) IF directFlight(From,To,_)
AND Itin = [From]
AND N = 0.

indirect(From,To,Itin,N) IF directFlight(From,Mid,_)
AND indirect(Mid,To,Itin’,N’)
AND Itin = Itin’ | [Mid]
AND N = N’+1.

Then, we select the result with the minimum number of connections and extend
it with the destination:

shortestItinerary(From,To,Itin)
IF _ = FINDMIN N. indirect(From,To,Itin’, N)
AND Itin = Itin’ | [To].
352 10 Preliminary Experiments

which we could further specialize to MSY and VCE as above, if desired. The
object “Itin” is a list containing all the airports visited in the shortest itinerary.
Using abbreviations, this same set of clauses could be more succinctly
entered as:

shortestItinerary(From,To,Itin)
IF N = COUNT.
Itin = directFlight++(From,To,_)
AND minConnections(From,To,N).

• How many frequent flyer miles would I accumulate? We can add up the distance
flown during a multi-segment itinerary by entering

indirect(From,To,D) IF directFlight(From,To,D).

indirect(From,To,D’’) IF directFlight(From,Mid,D)
AND indirect(Mid,To,D’)
AND D’’=D+D’.

and proceeding as above.

• Would it help if I waited until the new direct flight from Atlanta to Venice starts?
We can either insert the record (ATL,VCE,4900) in the spreadsheet, or extend
a clause with the variant ASSUMING directFlight("ATL","VCE",
4900), as in

shortestItinerary(From,To,Itin)
IF N = COUNT.
Itin = directFlight++(From,To,_)
AND minConnections(From,To,N)
ASSUMING directFlight("ATL","VCE",4900).

The form shown to the user is displayed in Fig. 10.3.

We described each solution in turn until we perceived the subject was starting
to lose interest. At that point, we moved on to presenting the user interface, as
described in Sect. 10.1.3.

Feedback

As we were illustrating each of these questions, we ascertained that our volunteers

did understand the proposed solution and asked for their feedback. We also took
note of any remark they made. For example, several interjected how this or that
functionality would help them with their work. Others objected to our choice of
wording.
10.2 Tested Users 353

10.1.3 Exposure to the User Interface

Once we had described our example and gone through its solution, and sometimes
concurrently with this, we showed the volunteers a series of 12 snapshots of
our proposed user interface adapted from Chap. 9. Nine of them are reported in
Figs. 10.4 through 10.7 at the end of this chapter; in addition, we showed them
snapshots corresponding to Figs. 9.15–9.17. We asked for feedback and collected
their impressions about what they saw.

10.2 Tested Users

In this section, we provide a brief description of the background and skill level of
the users who gracefully accepted to participate in this experiment. We also report
on what they said they would like to see as additional functionalities or features in
a spreadsheet. In order to safeguard their privacy, we will refer to them by means of
randomly assigned first names.
Ben was a postdoctoral fellow at a renowned university, where he specialized
in theoretical computer science. He had used a proprietary spreadsheet program
(not Excel) some 15 years before our conversation and hardly touched any such
applications ever since. Back then, he managed and printed lists, and performed
simple calculations. By itself, this would lead us to classify him as a beginner.
However, his deep knowledge of programming languages and logic set him in
a category apart: he had a perfect understanding of the technology and could
easily make sense of our EXTENSION, but the practical aspect of working with
a spreadsheet was missing.
He could not think of a desired extension of the traditional spreadsheet when
asked.
Catherine was an analyst in a large consulting firm. She used Excel and small
spreadsheet applications embedded in statistical programs on a daily basis for
her work. She occasionally used Excel at home for budgets, lists, etc. She is an
advanced user: she worked with large formulas and used complex statistical func-
tionalities such as regressions almost every day. She also frequently performed
what-if analyses using pivot tables.
When asked what she would like to see in a spreadsheet, she mentioned the
integration of database functionalities without the complication of working
WITH a database.
Geraldyn was a department administrator at a renowned university. She had been
using Microsoft Excel, and before that Lotus 1-2-3, on a daily basis for nearly
20 years. She used it to keep track of finances, expenditures, budgets, grades, and
occasionally to type up lists. She routinely wrote complex formulas and used data
forms and database functionalities. She had not had any use for the support for
354 10 Preliminary Experiments

what-if reasoning provided in modern applications. We will classify her as an

advanced user.
What she said she missed most in Excel was the possibility of defining her own
functions.
Ian was a consultant for the US government. He had been using Excel for the
last 10 years and works with it every day. He is an advanced user. He mostly
worked with data lists and the functions that operate on them and with statistical
functions. He had needed extensions to Excel and had others program them as
add-ons, but had never used Visual Basic for Applications himself.
When asked what he was missing in a spreadsheet, he mentioned more powerful
sorting mechanisms and the possibility of creating views akin to a database, but
without the complexity of setting up a database.
Julia was a librarian. She had been using spreadsheets for some 18 years, first
Lotus 1-2-3 and now Excel. She worked with spreadsheets every day, but her
experience was limited to managing lists and performing simple calculations.
Therefore, in spite of the many years she had been using these applications, we
classify her as a beginner. She had also been using various interfaces to query
databases for a long time, but had never set up a database herself.
When asked what she wished a spreadsheet to contain, she mentioned the
possibility of sorting and rearranging tables more easily without risking losing
data, and better facilities for captioning tables and reports. These are all instances
of database views.
Linda was an executive secretary in a major accounting and management con-
sulting firm. She had been working with Microsoft Excel on a daily basis for
nearly 15 years, mostly in the professional environment. She mostly used it to
collect and sort data, convert them to charts, and perform simple calculations.
However she had occasionally relied on complex functionalities. She had
extensive training on Excel, but never had a chance to use its what-if analysis
functionalities. She is an intermediate user.
When asked what she would like future spreadsheet applications to include, she
mentioned better help functionalities and more intuitive wizards.
Rich was a six-sigma asset manager in a Fortune 500 corporation. He had been
using Excel every day for over 15 years, mostly as an interface to database
applications and to keep track of scheduling. In spite of his extensive use, the
functionalities he relied on are rather limited, mostly sorting lists, setting up
layouts and performing simple calculations. He is somewhere between a beginner
and an intermediate user.
What he saw as missing from a spreadsheet was the ability to enter more text in
each individual cell (currently limited to 256 characters in Excel), the integration
of work-flow management tools, and having more rows (useful when transferring
data between large databases).
Sarah was an international relations specialist in a consulting company. She used
Excel rather infrequently, twice a month, for expense reports, lists, and simple
calculations. This was mostly done for work, although she used it also at home.
She had used statistical functions in the past, but not recently. She is a beginner.
10.3 Results of the Experiments 355

She could not think of new features she would like a spreadsheet to contain.
Altogether, these volunteers cover a sizable segment of the user base of the
traditional spreadsheet. The one category we were not able to cover in our informal
survey is that of spreadsheet programmers, who are comfortable with using tools
such as Visual Basic for Applications for extending the core functionalities of a
spreadsheet.

10.3 Results of the Experiments

We will now report on the sometimes surprising results of our interviews. Alto-
gether, responses suggested that all users, even beginners, long for the ability to use
spreadsheets as small homegrown databases. More precisely, they expressed interest
in full relational query capabilities within the familiar spreadsheet environment, not
as a separate application, and especially of tabular views of data where changes are
automatically propagated. Our design has catered to this need as early as Chaps. 4
and 5. Interest in and understanding of the more advanced features of our proposal,
most of which were outlined in Chap. 6, varied with user sophistication. Beginners
did not seem to care much for them. Advanced users immediately pointed to
problems to which they could be applied.
The opinions that our volunteers expressed about the user interface proposed
in Chap. 9 also tracked user sophistication. Advanced users immediately felt
comfortable with the choices we made. In contrast, beginners had issues with basic
decisions such as the wording of menu items. All classes of users complimented
us for the explanation facilities and the possibility of displaying our tables as
a connection graph. Interestingly, none of our users were moved by the clause
definition dialog, which stems from the fact that they declared never to use wizards
in general.
We will now examine the reactions gathered for each individual class of users,
starting with advanced and intermediate users, who were our target audience.

10.3.1 Advanced Users

Our representative advanced users, Catherine, Geraldyn and Ian, took great interest
in the added functionalities of the deductive spreadsheet. At several points during the
interview, they pointed to some feature or other that would help them solve current
problems and pointed to new applications of the technology. Both Catherine and
Ian declared that they would make heavy use of the deductive functionalities if they
were available in a commercial spreadsheet. They found recursion interesting, but
the high point was the possibility of writing relational expressions to use Excel as if
it were a database, with immediate visual feedback and without the heavy overhead
356 10 Preliminary Experiments

required to set up a database, even within a consumer product such as Microsoft

Access.
All three volunteers were pleased with the user interface in general, and very
impressed by the debugging facilities and the connection graph tool. Geraldyn
suggested refining the explanation window into a mechanism for interactively
refining queries, for example showing the routes from “JFK” to “LAX” and being
able to specialize the query to include intermediate stops. This is a possibility
definitely worth looking into.
These responses met our expectations with respect to both the core deductive
extension and the adaptation of the user interface. Nonetheless, it was gratifying to
get such enthusiastic feedback.

10.3.2 Intermediate Users

Our two intermediate users, Linda and Rich, were equally interested in the
extension. Linda immediately pointed to scheduling tasks such as meeting planning
and complex travel arrangements as uses she could make of the technology. Rich
saw applications in lean manufacturing processes, military logistics, and high-level
decision making in general. He said that he would probably use it if it were part of
Excel, but may not purchase it as a standalone application. Linda left us with the
same impression.
Both liked the user interface, but Rich and to a lesser extent Linda had some
problems with the choice of keywords. They suggested rewriting menus and dialogs
using less technical wording. Both appreciated the explanation and the connection
graph. Linda took great interest in the latter as a simple and intuitive scheduling
tool, and found it to provide better debugging than our explanation mechanism.
These responses are again mostly in line with our expectations. We found it
revealing that users in this class were at odds with our choice of text for menu items
and dialogs. This made us realize that users will not experiment with functionalities
if the commands to invoke them are labeled with words they do not understand.

10.3.3 Beginners

Our beginner users were Ben, Julia and Sarah. Because of his uncommon back-
ground, we will report on Ben’s comments separately.
Julia and Sarah clearly understood and manifested appreciation for the objective
of the deductive spreadsheet, but had difficulties comprehending the low-level
realization of these objectives. They had a hard time gaining an intuition about
what clauses do, their format, and the meaning of the keywords. Sarah suggested,
for example, replacing IF with BECAUSE. Both had issues with the choice of
wording used in menus and dialogs, for example “predicate”, “relation” and even
10.4 Summary 357

“clause” meant nothing to them: they referred to them as jargon and found them
not to be user-friendly. Sarah said she could not think of an immediate use for the
deductive extensions, but anticipated using some of the basic relational features if
it were made available within Excel in a user-friendly fashion. On the other hand,
Julia strongly appreciated the fact that the deductive spreadsheet offers a simple
way for non-specialists to use a spreadsheet as if it were a simple database. She
suggested a number of uses for it, both professionally and outside of work, if only
the interface to issuing queries were more user-friendly. In that respect, she was
extremely interested in the relational aspect of it, with the deductive part as a nice
curiosity.
Both expressed a rather negative view of the user interface. As we said, they
found it counterintuitive and would probably never use the added functionalities
because the wording of commands was a barrier to accessing them. On the other
hand, they both liked the possibility of building connection graphs and were curious
about the explanation facility.
Ben, the theoretical computer science expert, was extremely interested in the
technical aspects of the deductive spreadsheet, but mostly as an academic problem.
He could easily understand the technical explanation. His feedback about the
deductive spreadsheet as a user application meant to solve actual problems was
much more limited. He expressed some concern that it may involve concepts that
are too hard for beginners.
The responses gathered from this class of users came to us as a total surprise.
As they fell outside of our target audience, we expected a lack of appreciation
and even understanding for the problems we were trying to tackle and for their
solutions. Instead, these users expressed great interest in the simplest instances of
these problems, namely relational inference, and manifested almost eagerness for
simple tools to help solve them. They were very critical of the interface we provided
for this purpose, but pressed for simple, easy-to-use wizards or other mechanisms
that would allow them to build and display the results of queries on tabular data.

10.4 Summary

In this chapter, we have conducted preliminary usability experiments on a small but

revealing set of volunteers and collected feedback about both the core deductive
extension and the proposed user interface. The general reactions paralleled user
sophistication, with advanced users being very excited about both the added
functionalities and the interface, while beginners were interested in the former but
not as pleased with the latter.
Each participant, even beginners, appeared to crave a more extensive set of tools
to manipulate tabular data than what is available in the traditional spreadsheet. In
particular, they all wanted to be able to display views, in the sense of relational
databases, and have them be automatically updated whenever the base data changed.
The keen interest of beginners in this rather sophisticated task came as a surprise.
358 10 Preliminary Experiments

Our target audience, as defined in Chap. 8, comprised only advanced and inter-
mediate users. The feedback by volunteers from these classes matched expectations,
which suggests that we may have achieved the cognitive objectives stated in Chap. 2.
We write “may” because more rigorous and extensive experiments are required for
an accurate evaluation of this proposal.
Beginners were not part of our target audience, and yet they pressed for being
included as the beneficiaries of this technology. They demanded adjustments to the
user interface that would not discourage them from taking advantage of relational
queries. They also expressed interest in a set of intuitive tools that allowed them,
the unsophisticated beginners, to perform simple manipulations of tabular data
in unintimidating ways. Said differently, they wanted to be able to define useful
relational queries, without the jargon and technical mindset that our proposed
interface currently provides.
While we believe we have accomplished our goals with respect to advanced
and intermediate users, we see the keen interest of beginners for what the core
technology of the deductive spreadsheet can deliver as an unexpected niche. We
intend to improve the design of our user interface to answer their call and provide
them with intuitive tools to take advantage of basic but useful aspects of our solution.
We also intend to build on the numerous improvements suggested by the advanced
and especially intermediate users.
10.4 Summary 359

Questionnaire

• For how long have you been using spreadsheets?

• How frequently do you use them now?

• Which applications do you use most?

• Do you use them for work?

• What do you use spreadsheets for?

• Which of the following activities have you performed with a spreadsheet?

– Manage lists?

– Perform simple calculations? (arithmetic operations, use SUM over a single work-
sheet)

– Perform complex calculations? (use complex functions, array expressions, nested for-
mulas, multiple worksheets)

– Define symbolic names?

– Work with data lists, data forms, database functions?

– Use data tables, goal seeking, pivot tables?

– Program new functionalities? (using Visual Basic for Applications, for example)

• What would you like a spreadsheet to include that is not already there?

Fig. 10.2 Questionnaire

360 10 Preliminary Experiments

Example

• Is there a direct flight from New Orleans to Venice? By inspection.

• To where can I fly directly from New Orleans?

fromMSY(To) IF directFlight(From,To,_)
AND From="MSY".

• Is it possible to fly from New Orleans to Venice?

indirect(From,To) IF directFlight(From,To,_).

indirect(From,To) IF directFlight(From,Mid,_)
AND indirect(Mid,To).

Abbreviated as:

indirect(From,To) IF directFlight+(From,To,_).

• What is the minimum number of connections?

indirect(From,To,0) IF directFlight(From,To,_).

indirect(From,To,N) IF directFlight(From,Mid,_)
AND indirect(Mid,To,N’)
AND N=N’+1.

minConnections(From,To,Min) IF Min = FINDMIN N. indirect(From,To,N).

• What would be the itinerary?

shortestItinerary(From,To,Itin) IF N = COUNT.
Itin = directFlight++(From,To,_)
AND minConnections(From,To,N).

• How many frequent flyer miles would I accumulate?

indirect(From,To,D) IF directFlight(From,To,D).

indirect(From,To,D’’) IF directFlight(From,Mid,D)
AND indirect(Mid,To,D’) AND D’’=D+D’.

• Would it help if I waited until the new direct flight from Atlanta to Venice starts?

shortestItinerary(From,To,Itin) IF N = COUNT.
Itin = directFlight++(From,To,_)
AND minConnections(From,To,N)
ASSUMING directFlight("ATL","VCE",4900).

Fig. 10.3 Illustrated clausal definitions

10.4 Summary 361

Fig. 10.4 Interface screenshots used in experiments (1)

362 10 Preliminary Experiments

Fig. 10.5 Interface screenshots used in experiments (2)

10.4 Summary 363

Fig. 10.6 Interface screenshots used in experiments (3)

364 10 Preliminary Experiments

Fig. 10.7 Interface screenshots used in experiments (4)

Figures 9.11, 9.15, 9.16 and 9.17, on pages 332, 341, 342 and 344, respectively,
were also used.
Chapter 11
Future Developments

In this book, we have proposed a design for the deductive spreadsheet, an innovative
extension of traditional spreadsheet applications such as Microsoft Excel and
Google Spreadsheet with support for symbolic reasoning and deductive decision
making. The requirements for this tool have been set forth in Chap. 2. We have
designed the deductive apparatus underlying this logical extension in Part I of this
book, while the extension of the user interface and other cognitive issues have been
the subject of Part II.
With the bulk of the design, both functional and cognitive, behind us, we
dedicate this final chapter to outlining the next steps for the deductive spreadsheet.
The project is indeed continuing into an implementation and evaluation phase.
Accordingly, we discuss prototyping strategies and preliminary experiments in
Sect. 11.1, and we examine approaches for assessing the usability of our current
design in Sect. 11.2. We dedicate the rest of this chapter to open issues in the
current design and to opportunities for further extensions. Specifically, we look
at developments aimed at further extending the expressiveness of the deductive
spreadsheet in Sect. 11.3 and at its interface in Sect. 11.4. Both aspects will be
partially informed by the results of usability analysis.

11.1 Prototyping

With an advanced design in hand, an immediate task is the implementation of a

prototype of the deductive spreadsheet that faithfully realizes this design. Such
a prototype will serve two main purposes. First, it will empirically attest of the
feasibility of our design, especially in regard to achieving acceptable levels of
performance. Second, it will support dynamic usability studies in which users are
asked to perform tasks on an actual working system rather than mock-ups—this
is discussed in more detail in Sect. 11.2. We will now review the pros and cons
of a variety of prototyping strategies, and then outline the results of preliminary
experiments.

I. Cervesato, The Deductive Spreadsheet, Cognitive Technologies, 365

DOI 10.1007/978-3-642-37747-1 11, © Springer-Verlag Berlin Heidelberg 2013
366 11 Future Developments

11.1.1 Strategies

There are essentially three approaches to integrating deductive capabilities within a

traditional spreadsheet, with a few variants of each.
1. The first relies on the possibility, supported by most commercial spreadsheet
applications, to extend the range of built-in functionalities by invoking code
defined in external modules. These “add-ons” are often used to incorporate
domain-specific functions not native of the commercial distribution, for exam-
ple specialized statistical functions, interface shortcuts or new visualization
mechanisms. They are coded in the programming language supported by the
spreadsheet (e.g., Visual Basic for Applications in Excel, Java in OpenOffice and
LibreOffice, and a variant of JavaScript in Google Spreadsheet) and therefore
defining them lies well beyond the skills of typical end-users.
Implementing the functionalities of the deductive spreadsheet as one or more
add-ons to a traditional spreadsheet would have several advantages:
• All the traditional spreadsheet capabilities would be supported at any point
during development. In particular, there would be no need to provide code for
any original functionality that is not impacted by the deductive extension.
• The deductive extension would be self-contained. In particular there would be
no need to modify the original spreadsheet, a prospect that is legally difficult
for closed-source software.
Add-ons are an appealing solution for a prototype or a proof of concept where it
is not critical that every aspect of the deductive extension be smoothly integrated
in the operations of the base spreadsheet. As we will see below, a fully seamless
integration is however not achievable in current commercial products as it
requires changes to the core of the spreadsheet code, well beyond what the add-on
mechanisms provide access to. Another possible problem may be performance:
running the deductive extension as an add-on may be too slow.
2. The second approach is to implement the prototype from scratch, developing
native code for both the deductive engine and a selection of traditional spread-
sheet functionalities. This approach is potentially the most labor intensive, but
also the most flexible. The main disadvantage is that early prototypes will contain
little of the functionalities users are accustomed to.
3. The last approach to producing a deductive spreadsheet prototype is to modify the
existing code for an open-source spreadsheet such as OpenOffice’s Calc, KDE’s
KSpread, or one of the many Web-based spreadsheet plug-ins that have been
published in the last few years. Gaining a sufficient understanding of the source
code may provide an opportunity for dialog with the developers.
The main advantages of this approach are again flexibility and the availability
of code for the complete range of functionalities of the underlying spreadsheet.
The main disadvantages are the steep learning curve of understanding the source
code of a large third-party application, and the fact that extending a package of
11.1 Prototyping 367

this sort may be a moving target as patches and new versions are periodically
released.
An orthogonal dimension to the selection of an implementation strategy involves
the choice of the platform. Traditionally, spreadsheet applications have been made
available as standalone programs to be installed on one’s computer. Classical
examples are Excel, LibreOffice and Numbers. For some of them, matching the
underlying operating system is an additional choice point. Recently, the trend has
however moved to providing spreadsheets and other traditional office utilities as rich
Web applications, downloaded to the browser each time the user opens a document.
Google Spreadsheet is the dominant example.

11.1.2 Pre-prototyping

Over the last 2 years, we experimented with some of these options for implementing
a prototype of the deductive spreadsheet. We initially opted for extending an existing
commercial spreadsheet application rather than developing a prototype from scratch
to retain layout and feature familiarity during user testing. We experimented with
three major commercial spreadsheet applications: Microsoft Excel, OpenOffice
Calc and Google Spreadsheets.
In a first phase, we looked into exploiting approach (1) above, the add-on mech-
anism, to implement some of the core functionalities of our design. Specifically,
we developed a fully functional but unoptimized evaluation procedure for deductive
formulas within Microsoft Excel 2007 using Visual Basic for Applications (VBA).
It implemented the semi-naı̈ve strategy discussed in Sect. 6.2.2 on top of some
basic data structures to support fast joins. This reassured us about performance
worries: this unoptimized code could compute the join of two relations with 10,000
entries, each in under half a second on a commodity laptop.1 These results can
easily be replicated in any spreadsheet application that supports programmatic
extensions using add-ons (e.g., LibreOffice Calc and Google Spreadsheets). Further
performance gains can be achieved through standard logic programming and
database optimizations [CGT90, RSS92].
The second phase of this experiment focused on the user interface, with an
example task of embedding a syntactic editor within the formula bar. This proved
to be a lot more complex. We soon realized that the APIs provided within many
spreadsheet applications to manipulate interface elements via add-ons are too high-
level for our needs. For example, neither Microsoft nor OpenOffice nor Google
provides API functions that allow touching the text entered in the formula bar
of their respective spreadsheet applications—consequently we could not highlight

1
It featured a 2.33 GHz Intel Core Duo processor and 2 GB of RAM.
368 11 Future Developments

keywords, for example. For this reason, we began experimenting with modifying
the source code of OpenOffice Calc and Google Spreadsheets directly (this is not an
option for Excel since it is closed-source). This is approach (3) above.
Modifying the source code of OpenOffice is theoretically straightforward, once
we have gained a good understanding of what it does. This latter task is extremely
time consuming however, as Calc is a very large piece of software (its source code
has about two million lines of code) with lots of functionalities (and just a few
comments in the code)—for example, we kept finding more and more places in
the source code that manipulate the formula bar. In spite of good communications
with the developers, we eventually gave up on our attempt at extending Calc with a
syntactic editor.
Modifying the source code for Google Spreadsheets was a totally different
challenge because this is not a standalone application: it is shipped to the browser as
JavaScript code every time a user loads Google Spreadsheets. We used the Grease
Monkey extension to the Firefox browser to modify the code on the fly once it
reaches the browser but before it is rendered. Google Spreadsheets contains few
essential functionalities, which makes its code lightweight (244 Kb). However the
JavaScript code is “minimized” (compacted) to ensure fast downloads: for example,
comments and all unnecessary white spaces are removed and mnemonic identifiers
are replaced with automatically generated strings of minimal length. Needless to
say, this complicates understanding the code, even though it has a much smaller
footprint than Calc, for example. Because of this, we eventually abandoned our
attempt to integrate a syntactic editor in Google Spreadsheet as well.
Most recently, we selected a Web-based plug-in freely available on the
Internet and started modifying its source code to incorporate basic deductive
functionalities—another instance of approach (3) above. This proved straightfor-
ward as the source code is both readable (not minimized) and relatively small. This
makes it unlikely to encounter any hurdles in implementing more functionalities of
the deductive spreadsheet, whether in the inference engine or in the interface. One
drawback is however that a limited subset of the standard spreadsheet functionalities
are supported, which forces us to implement some ourselves and to do without
others. Another drawback is that the downloaded code does not incorporate many
of the optimizations found in more polished products.
Altogether, this exercise proved useful in informing a prototyping strategy for
the deductive spreadsheet. Building on a fully fledged product, which has the
advantage of already supporting all the functionalities users are accustomed to, is
an appealing but complex option: the provided add-on mechanisms do not go to
a sufficiently low level for our needs and the code is too complex when publicly
available. Instead, working with experimental open-source code bases appears to
be a promising approach for an initial prototyping effort. Developing a deductive
spreadsheet application from scratch (approach (2) above) does not seem to be an
effective strategy in the prototyping phase.
11.2 Experimental Assessment 369

11.2 Experimental Assessment

Future developments of the deductive spreadsheet need to be accompanied by exten-

sive usability studies to understand how to best present the extended functionalities
to interested users. Indeed, although the proposed design has received positive
feedback in a limited experimental validation based on screen mock-ups (see
Chap. 10), rigorous experiments aimed at systematically evaluating it on an actual
prototype will provide a broader range of information. Specific objectives include
investigating and improving the efficiency, memorability, errors, learnability and
user satisfaction of the tool. Other objectives, examined in more detail in Sects. 11.3
and 11.4, include assisting in the development of a user-oriented language that
supports an intuitive use of common data inference operations, in seamlessly
integrating the added functionalities within the widely accepted interface elements
of a spreadsheet, and in achieving a performance that, for common forms of data
inference, is subjectively comparable to spreadsheet operations of intermediate
complexity. This program will ultimately empower end-users and professionals
alike to solve problems like those listed in this book routinely and easily—all tasks
that currently require dedicated applications and costly training.
The usability of the deductive spreadsheet will be assessed by means of a
combination of some of the predictive user modeling methods, discussed in
Chap. 8, and actual user testing. For the former, we will mainly rely on heuristic
evaluation [NM94] and cognitive walkthrough [LRB91]. Each in its own way, these
two methods monitor a representative group of expert evaluators while performing
predetermined tasks on the target system. Their reactions, difficulties, and overall
performance are then compounded into an evaluation of individual aspects of the
interface, which is then passed to the designers to revise decisions that caused
unsatisfactory results—it is iterated until the tested aspects of the design are judged
acceptable. The use of different methods will broaden the range of usability issues
that are identified. We will continue using the lightweight methods that led to the
current interface design [GP96, Bla02] when introducing new elements into the
prototype, but rely on more accurate measures such as predictive user modeling and
user testing afterward. We expect experimental exposure to affect the user interface
more than the core deductive functionalities.
There are numerous aspects of the deductive spreadsheet that we are interested
in testing for usability. They can be roughly divided in two groups: the users’ ability
to understand a deductive problem and form a plan to solve it, and the efficacy
of a variety of interface mechanisms for doing so. In both cases, this analysis
will identify potential usability issues and provide design-relevant information to
mitigate them.
While it is expected that most spreadsheet users are comfortable with a tabular
organization of data (as visually promoted by the spreadsheet metaphor), it is
not clear how simple it is for them to shift from the current cell-at-a-time way
to perform calculations to an abstraction that views entire relations as first-class
objects of computation. Developing a series of scenarios can be helpful in answering
370 11 Future Developments

this question. These tests will include problems of growing complexity, where
complexity is measured in two ways: working from small to large tables (especially
tables that are larger than the screen), and using a combination of relational
or deductive mechanisms, first in isolation, and then in combination. Users will
eventually be asked to solve scenarios as complex as those exemplified in Chap. 10.
In all cases, it will be interesting to observe the cognitive pathways that lead a user
to forming the plan to solve a problem. They will also be interesting to monitor
how much resistance users put to using deductive mechanism, i.e., by trying to use
conventional operators to achieve the same result. This will allow us to gauge how
easy it is for different classes of users to grasp the added expressive power of the
deductive extension.
While the interface mechanisms for performing deductive reasoning are meant
to be unobtrusive and easy to use, it is important to assess how users are reacting
to individual design choices. One that we have mentioned earlier is the syntax for
inputting deductive formulas, which will be the subject of numerous scenarios.
Such tests will measure indicators such as the error rate, the number of incorrect
attempts, the time between attempts and the overall time to solve a problem. It will
also look for patterns in failed attempts. Other input modalities listed in Chap. 9
will be assessed as well, in particular the mouse-driven clausal input, wizards, the
use of cut-and-paste, and the drag-and-drop approach discussed in Sect. 9.3. For
some of these, we are interested not only in the difficulties that a user encounters
when relying on them, but also in whether they are used at all. A final object of
usability assessment is the explanation facility and the deductive productivity tools.
It should be observed that the gesture-based formula construction modalities and
the relational explanation facilities are new in the spreadsheet context and their
assessment is therefore of intrinsic scientific interest.
Usability inspection results are greatly influenced by the skill level and expe-
rience the participants have with respect to the traditional spreadsheet paradigm.
In a usability study for the deductive extension explored here, it is important to
include subjects with a diverse set of backgrounds to evaluate the different parts
of the design. Dividing users into the four classes listed in Chap. 10 (beginners,
intermediate users, advanced users, and programmers) seems an adequate first step.

11.3 Functional Extensions

Although this book has covered a large portion of the logical design of the
deductive spreadsheet, several technical challenges remain. These are engineering
and sometimes scientific quests that have not been explored yet or not at the scale
needed in an implementation of the deductive spreadsheet. We will examine some
of them in this section and also look at possible functional extensions of our design.
11.3 Functional Extensions 371

11.3.1 Technical Challenges

A key requirement for the deductive spreadsheet, listed in Chap. 2, is that its
performance not be overwhelmingly worse than that of a traditional spreadsheet.
This is a challenge as relational manipulations inherently operate on larger amounts
of data than typical arithmetic calculations. In Chaps. 4 and 5, we enrolled the semi-
naı̈ve strategy and a variety of graph-based data structures to achieve this effect.
While the preliminary experiments mentioned in Sect. 11.1 are encouraging, there
is room for significant performance gains by developing specialized methods. One
such method is the judicious use of indexing and other memoizing data structures.
We are unaware of attempts to specialize these techniques to a context similar to
ours. Another such method is compiling deductive formulas to gain in efficiency.
Compilation of logic programs is not new but has mostly focused on Prolog-
style top-down evaluation [Cer12], with relatively little attention dedicated to pure
bottom-up strategies [RSS92].
As mentioned in Sect. 5.6 and elsewhere, efficient techniques for propagating
positive updates (as when asserting a new base fact) have received a lot of attention
in the Datalog literature [CGT90,RSS92,GKB87]. By contrast, extending the semi-
naı̈ve evaluation strategy to support efficient propagation of negative updates (as
when the user deletes or modifies a record, or when propagating a positive update
through a negation) has only recently started to garner interest [NJLS11, LC12].
It is an open challenge to adapt these approaches to make them effective at the
scale needed for the deductive spreadsheet and to optimize them for the underlying
spreadsheet infrastructure.
We mentioned in Sect. 6.2.4 that well-behaved top-down evaluation strategies
(e.g., [War98]) can be advantageously used as an explanation mechanism for
bottom-up evaluation, which lies at the core of the deductive spreadsheet. Nobody
has done it before, however. Although few hurdles are expected, making all the
details work is likely to bring fresh scientific insight that could benefit the fields of
logic programming and databases.
Calculated values significantly extend Datalog’s expressive power, but raise the
possibility of non-termination. In Sect. 6.3, we showed an approach to support this
extension in the deductive spreadsheet through bounded recursion, a simple but
rather limiting solution. In many cases, termination can be determined through cycle
detection and other heuristics. Identifying useful problem classes where this can be
done efficiently in the spreadsheet context would shift some of the programming
burden from the user to the machine and permit flagging errors early. As mentioned
at the end of that section, term irrelevance is also a promising technology.

11.3.2 Extensions

The core logic underlying our design of the deductive spreadsheet has been
extensively studied in the past, although some of the functional extensions proposed
372 11 Future Developments

in this book are novel. As we saw, its good computational properties make it a
reasonable initial target language for the purpose of supporting symbolic reasoning
within a spreadsheet.
Assuming that our design successfully passes the empirical test in Sect. 11.2, a
number of linguistic extensions seem natural next steps. We now review some of
them:
• While termination is important for the average user, expert reasoners may want to
avail themselves of the deductive power of a fragment of first-order logic more
expressive than Datalog. Some variant of Prolog is a natural candidate for this
purpose. Because of the concrete risk of non-termination, this possible extension
would be made available on a use-at-your-own-risk manner or through add-ons.
Since bottom-up evaluation is almost invariably non-terminating on a Prolog
program, top-down search would need to be supported for formulas relying on
this larger language, although we may then want to look for a variant that is more
forgiving than the strategy commonly implemented in Prolog.
• If this extension were to successfully meet the needs of a sufficient user segment,
we may try to include an even larger fragment of first-order logic. This would
however most likely require equipping the tool with some form of theorem-
proving capabilities beyond bottom-up and top-down evaluation. A particularly
interesting instance of this goal could involve the integration of domain-specific
constraint solvers.
• Datalog and even Prolog account for only a very narrow slice of the logical
spectrum. Over the years, many logics with excellent expressiveness and com-
putational properties have been proposed. For example, linear logic can handle
resources, modal logic deals with necessity, temporal logic reasons with time,
etc. Once a solid base for the deductive spreadsheet has been established, it may
prove useful to provide support for some of these advanced languages, again
possibly in the form of add-ons.

11.4 Interface Extensions

Although the user interface of the deductive spreadsheet was designed on the
basis of well-established methodologies [GP96, Bla02], it too has several remaining
challenges, of a cognitive nature this time. Exploring ways to address them is likely
to lead to new scientific insight. We will now list some of them, concluding with a
few other suggestions for extending the interface of the deductive spreadsheet.

11.4.1 Cognitive Challenges

One open cognitive issue has to do with the syntax of deductive formulas as it
appears in the formula bar—in the examples in Chap. 9, we used a variant of
11.4 Interface Extensions 373

Datalog’s clausal syntax and a dialect of SQL. The syntax of traditional spreadsheet
formulas, although rather inflexible, has very simple ingredients: values, cells and
operations. Although it is known to be extremely error-prone [EB02, Pan98], users
seem comfortable working with it, at least for simple formulas. By contrast, the
informal experiments reported in Chap. 10 (and a few others) have hinted at the
possibility that a Datalog-style clausal syntax may belong to a different cognitive
level: understanding what clauses are and how they are constructed was not immedi-
ate for some classes of users—variables seemed particularly troublesome. The SQL
syntax appears to be equally taxing on users. Usability tests will reveal whether
this cognitive dissonance can be attributed to the novelty of the concept, or if there
is indeed something deeper. This would call for exploring alternative syntactic
means to describe logical specifications. Although, a better choice of keywords
and a more intuitive clause structure may help somewhat, we have however higher
expectations about replacing clausal forms altogether with alternative languages
of equivalent expressive power. XcelLog’s variable-free approach to specifying
deductive formulas is particularly intriguing [RRW06, RRW07]. Complementing
textual syntax with simple intuitive visual methods to building formulas (that go
beyond what traditional spreadsheets make available) is also worth exploring. Some
results from visual programming hold promises [APRS96], as do gesture-based
approaches available in products such as Microsoft Access.
Another cognitively challenging issue involves complex logical patterns. Datalog
clauses have very simple logical ingredients, most prominently conjunctions and
negation, which can however be combined in complex ways. Differently from
traditional formulas, some recurrent reasoning patterns have an inherently complex
form in Datalog: for example, an extensional universal quantification requires two
nested negations and an auxiliary clause. Users tend to find this indirection and
polarity switch hard to formulate and hard to explain [WJL72,SSFd00]. Quantitative
user testing can help identify the most common patterns that users find cognitively
taxing, suggest intuitive syntactic shortcuts, and assess their effects on users’ ability
to perform tasks.
A third open issue concerns helping users discover and explore the deductive
extensions in an unobtrusive way. Indeed, a person who wants to use the deductive
spreadsheet as a traditional spreadsheet should not be penalized, cognitively or
otherwise, by the presence of the relational infrastructure. Yet, if this user could
benefit from the deductive capabilities, she should be able to find and experiment
with them easily. Traditional spreadsheets have a mixed record on this issue: users
find some functionalities (e.g., charts) very easy to discover while others (e.g., pivot
tables) require more of an introduction. Devising the best way to help users discover
our extensions is more an art than a science, and why some functionalities are harder
to embrace than others is not well understood.
374 11 Future Developments

11.4.2 Extensions

A number of extensions to the interface design of the deductive spreadsheet in

Chap. 9 will propose themselves as the result of user testing and while addressing
the above cognitive challenges. Others are orthogonal and more far reaching. For
example, the preliminary experiments in Chap. 10 revealed an unexpected interest
coming from beginner spreadsheet users. Extending the proposed user interface with
intuitive mechanisms to allow them to define simple but useful relational views of
tabular data would be highly beneficial.
Although not exactly an interface extension, it will be interesting to grow our
suite of examples, both in scope and in diversity. It is known, for instance, that
transitive closure problems are pervasive, although they often occur in disguise—
expressing transitive closure problems drawn from a variety of fields in a deductive
spreadsheet is an instance of what we mean by growing the scope of our example
collection. More interesting is the task of finding new types of problems that could
beneficially be expressed and solved with the deductive spreadsheet. Two examples
that were recently brought to our attention are some instances of data mining and of
knowledge analysis.
Annotated Bibliography

[ABW87] Apt, K. R., Blair, H. A., & Walker, A. (1987). Towards a theory of declarative knowl-
edge. In J. Minker (Ed.), Foundations of deductive databases and logic programming [Min87]
(pp. 89–148). Los Altos: Morgan Kaufmann.

This influential paper identifies the notion of stratification as an effective approach

to embedding negation in logic programming, both computationally and cognitively.
It develops its general theory and fixpoint semantics, and specializes SLD-resolution
to it.

[And92] Andreoli, J. -M. (1992). Logic programming with focusing proofs in linear logic. Journal
of Logic and Computation, 2(3), 297–347.

This paper is the first systematic investigation of focusing, a proof technique that
underlies much of the recent research in logic programming. Focusing relies on the
observation that, when in a proof, some steps can be permuted without affecting the
ability to complete the proof. Focusing systematically delays some steps while eagerly
carrying out others, which results in a very efficient strategy.

[AOTC 03] Arni, F., Ong, K., Tsur, S., Wang, H., & Zaniolo, C. (2003). The deductive database
system LDL++. Theory and Practice of Logic Programming, 1, 61–94.

This “lessons-learned” paper describes LDL++, a deductive database system that

incorporates many of the standard features found in the commercial relational database
management systems of the time. As it does so, it pays particular attention to the
foundational aspects, in particular its model and fixpoint theory, but also to its
architecture as a system and the operational support within.

[APRS96] Agustı́, J., Puigsegur, J., Robertson, D., & Schorlemmer, M. (1996). Visual logic
programming through set inclusion and chaining. In CADE-13 workshop on visual reasoning,
New Brunswick.

This paper presents the design of a visual logic programming language that supports
visual representation and visual reasoning in a declarative way.

[ARLGC 09] Ashley-Rollman, M. P., Lee, P., Goldstein, S. C., Pillai, P., & Campbell, J. D.
(2009). A language for large ensembles of independently executing nodes. In Proceedings of
ICLP’09, Pasadena (Lecture notes in computer science, Vol. 5649, pp. 265–280). Springer.

I. Cervesato, The Deductive Spreadsheet, Cognitive Technologies, 375

DOI 10.1007/978-3-642-37747-1, © Springer-Verlag Berlin Heidelberg 2013
376 Annotated Bibliography

This paper describes Meld, a variant of Datalog designed to program modular

ensembles of millions of cyber-physical agents capable of sensing, actuation and
computation. Using Meld, these ensembles are programmed as if they were a single
entity.

[AvE82] Apt, K. R., & van Emden, M. H. (1982). Contributions to the theory of logic
programming. Journal of the ACM, 29(3), 841–862.

This paper develops the characterization of the model semantics of logic programs
consisting of Horn clauses (i.e., pure Prolog) as the least fixpoint of a continuous
operator. It also observes that some forms of finite-failure (but not all) correspond
to its greatest fixpoint.

[BABC 00] Boehm, B. W., Abts, C., Brown, A. W., Chulani, S., Horowitz, E., Madachy, R.,
Reifer, D., Clark, B. K., & Steece, B. (2000). Software cost estimation with COCOMO II.
Upper Saddle River: Prentice Hall.

This book describes the COCOMO framework for accurately estimating software cost
and schedules, and for supporting trade-off, risk, sensitivity, and business case analyses
for software decisions. It expands older versions of COCOMO to today’s software
engineering approaches, which are more evolutionary, risk-driven and collaborative,
and less focused on developing systems from the scratch.

[BADC 01] Burnett, M., Atwood, J., Djang, R. W., Reichwein, J., Gottfried, H., & Yang, S. (2001).
Forms/3: A first-order visual language to explore the boundaries of the spreadsheet paradigm.
Journal of functional programming, 11(2), 155–206.

This paper describes Forms/3, a conservative extension of the traditional spreadsheet

with procedural abstractions such as recursion, data abstractions such as graphical
values, and graphic time-based output. As it does so, it strives to maintain the usability
characteristics that have made the spreadsheet popular with end-users.

[BB02] Blackwell, A., & Burnett, M. (2002). Applying attention investment to end-user-
programming. In IEEE conference on human-centric computing languages and environments,
Arlington (pp. 28–30). IEEE Computer Society.

This paper describes uses of the attention investment model within the Forms/3
spreadsheet system [BADC 01], finding it a useful design tool for the development
of end-user programming features.

[BBCC 01] Blackwell, A. F., Britton, C., Cox, A., Green, T. R. G., Gurr, C. A., Kadoda, G. F.,
Kutar, M., Loomes, M., Nehaniv, C. L., Petre, M., Roast, C., Roes, C., Wong, A., & Young,
R. M. (2001). Cognitive dimensions of notations: Design tools for cognitive technology. In
M. Beynon, C. L. Nehaniv, & K. Dautenhahn (Eds.), Cognitive technology 2001, Coventry
(Lecture notes in artificial intelligence, Vol. 2117, pp. 325–341). Springer

This paper looks back at developments and applications of the cognitive dimensions
of notation methodology in the ten years since it was first proposed. At the time it
was written, it had become a fairly established methodology to assist the designers of
notational systems, whether graphical or not, evaluate their design with respect to the
impact they will have on users.

[BDM01] Bozzano, M., Delzanno, G., & Martelli, M. (2001). An effective bottom-up semantics
for first order linear logic programs. In Proceedings of FLOPS 2001, Tokyo, Japan (Lecture
notes in computer science, Vol. 2024, pp. 138–152). Springer
Annotated Bibliography 377

This paper explores an approach to defining an effective fixpoint semantics for a

fragment of first-order linear logic. This semantics yields a decision procedure akin
to symbolic model checking for some classes of concurrent systems.

[BG87] Brown, P., & Gould, J. (1987). Experimental study of people creating spreadsheets. ACM
Transactions on Office Information Systems, 5, 258–272.

In this early study of how people use spreadsheets, the authors monitored nine
experienced users building three predefined spreadsheets. They found that nearly half
of the spreadsheets they wrote contained errors. They also found that very little time
was dedicated to planning or to debugging.

[BG99] Blackwell, A., & Green, T. (1999). Investment of attention as an analytic approach to
cognitive dimensions. In Collected papers of the 11th annual workshop of the psychology of
programming interest group – PPIG-11, Leeds (pp. 24–35).

This paper takes a critical look at the cognitive dimensions of notations methodology.
It does not dispute its effectiveness but attempts a disciplined, scientific categorization
of the original and subsequently proposed cognitive dimensions.

[BG03] Blackwell, A., & Green, T. (2003). Notational systems – the cognitive dimensions of
notations framework. In J. M. Carroll (Ed.), HCI models, theories and frameworks: Towards
an interdisciplinary science. San Francisco: Morgan Kaufmann.

This paper advocates the cognitive dimensions of notations framework as a device

to helps designers build graphical user interfaces that with the desired cognitive
properties. This practical study of the people tho design interfaces is in contrast with
mainstream theoretical frameworks and user-centric approaches.

[Bla00] Blackwell, A. (2000). Dealing with new cognitive dimensions. In Workshop on cognitive
dimensions: Strengthening the cognitive dimensions research community.

This paper is concerned with the process of determining what new dimensions
should be included in the cognitive dimensions of notation framework. It does so
by recounting conversations with that method’s creator, Thomas Green, and explores
ideas for developing a methodology for evaluating candidate dimensions.

[Bla02] Blackwell, A. (2002). First steps in programming: A rationale for attention investment
models. In Conference on human-centric computing languages and environments, Arlington
(pp. 2–10). IEEE Computer Society.

This is the original paper on the attention investment model. It focuses on the initial
attitude of non-professional end-users when given tasks that have a programming
component (e.g., writing spreadsheet formulas or programming a VCR). In particular,
the attention investment model examines the first steps these users take and tries to
understand their actions based on task and system at hand.

[BM90] Bonner, A. J., & McCarty, L. T. (1990). Adding negation-as-failure to intuitionistic logic
programming. In Proceedings of the North American conference on logic programming –
NACLP’90, Austin (pp. 681–703). MIT.

This paper studies the model semantics of the Datalog-like language with embedded
clauses proposed in [BMV89]. One of the key devices is to port the classically defined
notion of perfect model to intuitionistic logic. Doing so retains the good properties of
the perfect model while accommodating embedded clauses.
378 Annotated Bibliography

[BMV89] Bonner, A. J., McCarty, L. T., & Vadaparty, K. (1989). Expressing database queries with
intuitionistic logic. In Proceedings of the North American conference on logic programming –
NACLP’89, Cleveland (pp. 831–850). MIT.

This paper is an early, and very thorough, proposal for extending a bottom-up Datalog-
like logic programming language with embedded clauses (embedded implications with
locally scoped universal quantifiers). While similar efforts were well-underway at
the time for top-down logic programming (see [Mil89] for example), this study is a
remarkable attempt at pushing the boundaries of deductive databases.

[Bon91] Bonner, A. J. (1991). Hypothetical reasoning in deductive databases. PhD thesis, Rutgers
University.

This dissertation explores a principled way to extend deductive databases with

hypothetical reasoning. It does so by allowing the body of a clause to mention
“embedded clauses”. This adds significantly to the expressive power of the language, a
fact that is thoroughly analyzed in the context of complexity theory. The model theory
of this extension is investigated in [BM90] while the key insight was already present
in [BMV89].

[Bon94] Bonner, A. J. (1994). Hypothetical reasoning with intuitionistic logic. In R. Demolombe

& T. Imielinski (Eds.), Non-standard queries and answers (Studies in logic and computation,
chapter 8, pp. 187–219). Oxford: Oxford University Press.

This paper develops the author’s dissertation [Bon91] by focusing on forms of

hypothetical reasoning that only add temporary facts in a deductive database and
create new constants dynamically. It explores its proof-theory, its model theory, and
the computational complexity of the extended language.

[Bri09] Bricklin, D. (2009). VisiCalc history. http://www.bricklin.com/history/intro.htm.

This website is about the ideas that led to the development of the first spreadsheet
application, VisiCalc, and about the people who were involved. It is a fascinating
collection of anecdotes, photographs, and early programs.

[CARGC 12] Cruz, F., Ashley-Rollman, M. P., Goldstein, S. C., Rocha, R., & Pfenning, F. (2012).
Bottom-up logic programming for multicores. In V. S. Costa (Ed.), Proceedings of 7th
international workshop on declarative aspects and applications of multicore programming –
DAMP’12, Philadelphia.

This paper applies ideas explored in [ARLGC 09] to the problem of executing
traditional algorithms in a multi-core setting. This is the first attempt at using a
Datalog-like language for this purpose.

[CC00] Chen, Y., & Chan, H. C. (2000). Visual checking of spreadsheets. In Conference of the
European spreadsheet risks interest group – EuSpRIG’00, London, UK (pp. 75–85).

This paper observes that spreadsheets, as built by regular users, contain both a
“surface” and a “deep” structure, and that many errors stem from the latter. It proposes
a method for visualizing the deep structure and therefore helping users debug their
spreadsheets.

[Cer12] Cervesato, I. (2012). An improved proof-theoretic compilation of logic programs. In

A. Dovier & V. Santos Costa (Eds.), 28th international conference on logic programming –
ICLP’12, Budapest (Theory and practice of logic programming). Cambridge University Press.
Annotated Bibliography 379

This paper describes a general, logic-based, methodology to compile a logic program

into an intermediate logical language that can be executed more efficiently. This is in
contrast to traditional approaches to compiling logic programs, which took a purely
operational view.

[CGT89] Ceri, S., Gottlob, G., & Tanca, L. (1989). What you always wanted to know about datalog
(and never dared to ask). IEEE Transactions on Knowledge and Data Engineering, 1(1),
146–166.

This paper contains a thorough survey of the research on Datalog as of 1989. It mostly
takes a query-oriented view, typical of deductive databases or logic programming,
in which only a small number of consequences of the basic facts are considered
interesting. Optimization techniques and extensions (e.g., negation) are discussed. The
reference section is phenomenal.

[CGT90] Ceri, S., Gottlob, G., & Tanca, L. (1990). Logic programming and databases.
Berlin/New York: Springer.

This expanded version of [CGT89] is an excellent survey with some in-depth analysis
of important concepts, in particular semantics, optimizations and the interface with
actual databases. It covers the experimental deductive database system of the time at
length.

[CH85] Chandra, A. K., & Harel, D. (1985). Horn clause queries and generalizations. Journal of
Logic Programming, 1, 1–15.

This paper provides a detailed classification of the expressive powers of various

fragments of relational algebra and (extensions of) Datalog. It is particularly concerned
with how different forms of negation impact expressiveness.

[Cha06] Chaudhuri, K. (2006). The focused inverse method for linear logic. PhD thesis, Computer
Science Department, Carnegie Mellon University.

This dissertation combines two established approaches to automate reasoning more

effectively in a refinement of traditional logic called linear logic. These are the inverse
method, due to Maslov, and Andreoli’s focusing [And92].

[Cla78] Clark, K. L. (1978). Negation as failure. In H. Gallaire & J. Minker (Eds.), Logic and data
bases (pp. 293–322). New York: Plenum.

This is the original article on negation-as-failure.

[CM87] Clocksin, W. F., & Mellish, C. S. (1987). Programming in Prolog (3rd ed.).
Berlin/New York: Springer.

This book is a classic introductory text to Prolog. Generations of students have relied
on it to learn logic programming to build applied programs.

[Cod70] Codd, E. F. (1970). A relational model of data for large shared data banks. Communica-
tion of the ACM, 13(6), 377–387.

This is the original article on the relational data model and the use of relational algebra
as a query mechanism for databases. This article offers a rare glimpse of what it was
like to use computers in the 1960s and of a vision that is still with us today (in a very
prominent way).
380 Annotated Bibliography

[Col70] Colmerauer, A. (1970). Les systèmes-Q ou un formalisme pour analyser et synthétiser

des phrases sur ordinateur. Technical report, Department of Computer Science, University of
Montreal, Montreal.

This technical report is widely considered to be the first mention on what was to
become Prolog. Its original application was the processing of natural language (French,
to be precise).

[Col98] Colomb, R. M. (1998). Deductive databases and their applications. London/Bristol:

Taylor & Francis.

This book is one of the best introductory accounts of what can be done with a deductive
database. Specifically, it explores in great depth problems that combine large sets
of data and require a truly deductive form of inference, above and beyond what the
relational data model has to offer. It targets people who will be using a deductive
database as a tool more than students interested in the underlying theory.

[CP02] Cervesato, I., & Pfenning, F. (2002). A linear logical framework. Information &
Computation, 179(1), 19–75.

This paper presents LLF, a logical framework based on the first instance of a linear
dependent type theory. Linearity enables it to represent object formalisms with state,
such as imperative programming languages. It also supports an operational semantics
in the style of top-down logic programming that allows us to execute some of these
specifications.

[CvEL88] Cheng, M. H. M., van Emden, M. H., & Lee, J. H. -M. (1988). Tables as a user interface
for logic programs. In Fifth generation computer systems, Tokyo (pp. 784–791).

This work extends [vEOT86] to tables in the sense of relational algebra. It is still
heavily based on incremental queries but now supports the display of multiple
solutions in a tabular way. The implementation is based on the recently introduced
graphical display. Several windows show different views of the system. However the
basic input mode is still essentially through the command lines, with other windows
used only for displaying results in a tabular way.

[DW88] Debray, S. K., & Warren, D. S. (1988). Automatic mode inference for logic programs.
Journal of Logic Programming, 5, 207–229.

This paper describes a method for automatically determining correct input/output

mode declarations in a Prolog program. This permits us to specialize the operational
interpretation of a program, thereby achieving greater efficiency.

[EB02] Erwig, M., & Burnett, M. (2002). Adding apples and oranges. In Practical aspects of
declarative languages, 4th international symposium – PADL’02, Portland (Lecture notes in
computer science, Vol. 2257, pp. 173–191). Springer.

This paper attempts to rein in the high error rate found in spreadsheets by relying
on “units” implicitly inferred from a spreadsheet rather than a rigid system of
types. Such units are classified and correlated into dependent units, multiplicities and
generalizations, with the aim to capture the user’s intention and fend off mistakes.

[EK76] Van Emden, M. H., & Kowalski, R. A. (1976). The semantics of predicate logic as a
programming language. Journal of the ACM, 23(4), 733–742.
Annotated Bibliography 381

This is a classic paper on the foundations of logic programming. It defines top-down

evaluation and the fixpoint semantics, and relates them to proof- and model-theory
respectively.

[Fre92] Freitag, B. (1992). Extending deductive database languages by embedded implications. In

A. Voronkov (Ed.), Proceedings of the third international conference on logic programming
and automated reasoning – LPAR’92, St. Petersburg (Lecture notes in computer science,
Vol. 642, pp. 84–95). Springer.

This paper considers an extension of a logical language for deductive databases

with embedded implication based on [Mil89]. It explores implicational stratification
of programs, a model-theoretic semantics, and a fixpoint semantics, but limits the
antecedent of embedded implications to universally closed formulas. In particular,
there is no sharing of variables with the parent clause.

[GA00] Gupta, G., & Akhter, S. (2000). Knowledgesheet: A graphical spreadsheet interface
for interactively developing a class of constraint programs. In Proceedings of the second
international workshop practical aspects of declarative languages – PADL’00, Boston
(Lecture notes in computer science, Vol. 1753, pp. 308–323). Springer.

This paper discusses the design and prototype implementation of Knowledgesheet,

an extension of the traditional spreadsheet paradigm with bidirectional constraints, so
that any number of cells linked by a system of constraints can be evaluated as soon
as a sufficient number of them has been assigned a value. This allows us to use the
spreadsheet metaphor to solve two-dimensional constraint satisfaction problems such
as scheduling and building time tables.

[Gir87] Girard, J. -Y. (1987). Linear logic. Theoretical Computer Science, 50, 1–102.

This is the original paper on linear logic. It describes in great detail its proof-theory
and several forms of semantics for it.

[GKB87] Güntzer, U., Kiessling, W., & Bayer, R. (1987). On the evaluation of recursion in
(deductive) database systems by efficient differential fixpoint iteration. In Proceedings of
the third international conference on data engineering, Los Angeles (pp. 120–129). IEEE
Computer Society.

This article contains an in-depth discussion of the semi-naı̈ve strategy.

[GM78] Gallaire, H., & Minker, J. (Eds.). (1978). Logic and data bases. New York: Plenum.

This collection contains many of the very first papers on using logic programming
in the context of data- and knowledge-bases. The seminal work on the closed world
assumption and negation-as-failure first appeared in it.

[GM02] Ganzinger, H., & McAllester, D. (2002). Logical algorithms. In Proceedings of the 18th
international conference on logic programming – ICLP’02, London (pp. 209–223). Springer.

This paper proposes a precise model of the run-time of a logic program that supports
using logic programs for studying the execution time of algorithms.

[GMS93] Gupta, A., Mumick, I. S., & Subrahmanian, V. S. (1993). Maintaining views incremen-
tally. In Proceedings of the 1993 SIGMOD international conference on management of data –
SIGMOD’93, Washington, DC (pp. 157–166). ACM.
382 Annotated Bibliography

This paper is concerned with the problem of propagating the effect of fact updates
in a Datalog program. To this effect, it associates a counter to inferred facts, thereby
counting in how many ways a fact can be derived. Relevant counters are increased or
decreased whenever an assertion or deletion changes the provability of inferred atoms.

[GP96] Green, T., & Petre, M. (1996). Usability analysis of visual programming environ-
ments: A “cognitive dimensions” framework. Journal of Visual Languages and Computing, 7,
131–174.

This paper applies the cognitive dimensions of notation framework to the evaluation
for visual programming environments, finding it particularly effective and a good
complement for other techniques.

[Gre69] Green, C. (1969). Application of theorem proving to problem solving. In D. E.

Walker & L. M. Norton (Eds.), Proceedings of the international joint conference on artificial
intelligence, Washington, DC (pp. 219–239).

This is an early proposal to use theorem proving for problem solving.

[Gre89] Green, T. (1989). Cognitive dimensions of notations. In A. Sutcliffe & L. Macaulay

(Eds.), People and computers V (pp. 443–460). Cambridge: Cambridge University Press.

This is the original paper on the cognitive dimension of notations. At the time, it
focused not so much on graphical user interfaces but on interactive devices and
programming notations.

[Gre96a] Green, T. (1996). An introduction to the cognitive dimensions framework. Available at

http://homepage.ntlworld.com/greenery/workStuff/Papers/introCogDims/.

In the talk on which this extended abstract is based, Thomas Green discusses the
reasons that led him to develop the cognitive dimensions of notation framework. This
is a good first-person motivation of the origins of the methodology.

[Gre96b] Green, T. (1996). The visual vision and human cognition. Available at http://homepage.
ntlworld.com/greenery/workStuff/VL96Talk/VLTalk.html.

In this talk, Thomas Green discusses the progress in visual programming environments
as well as some limitations. At he does this, the explains how his cognitive dimensions
of notations can benefit the design of such systems as well as some usability targets
for such designs.

[Gun04] Gunning, D. (2004). DARPA SBIR 2004.3, topic sb043-040: Deductive spreadsheets.

This call-for-proposals fueled a revival in interest in extending spreadsheets with

deductive capabilities. As a result, many researchers reconsidered the problem and
devised new and innovative solutions to it.

[GW10] Grumbach, S., & Wang, F. (2010). Netlog, a rule-based language for distributed
programming. In Proceedings of 12th international symposium on practical aspects of
declarative languages – PADL’10, Madrid (Lecture notes in computer science, Vol. 5937,
pp. 88–103). Springer.

This papers describes NetLog, an extension of Datalog with communication primitives

and aggregation for the purpose of describing communication protocols, peer-to-peer
applications, and other distributed systems. A distributed fixpoint semantics is devised,
as well as syntactic restrictions aimed at limiting the cost of network communications.
Annotated Bibliography 383

[Han94] Hanus, M. (1994). The integration of functions into logic programming: From theory to
practice. Journal of Logic Programming, 19–20, 583–628.

This paper describes Curry, a functional-logic programming language for the research,
teaching, and applications, as well as the foundations of this paradigm.

[Har93] Harland, J. (1993). Success and failure for hereditary Harrop formulae. Journal of Logic
Programming, 17(1), 1–29.

This paper explores negation-as-failure in the context of hereditary Harrop formulas,

which differ from the Horn clauses on which Prolog is founded by allowing embedded
implication and quantification in the body of clauses. The paper extends the classical
model semantics of Horn clauses to an intuitionistic one based on Kripke models.
Particularly interesting is the treatment of extensional universal quantification.

[Har94] Harland, J. (1994). Towards a sequent calculus for negation as failure. In Proceedings of
the workshop on proof-theoretical extensions of logic programming, Santa Margherita Ligure
(pp. 19–27).

This paper develops the proof-theory of negation-as-failure for hereditary Harrop

formulas, whose body formulas admit embedded clauses. To this effect, it defines a
proof theory for non-derivability in intuitionistic logic.

[Har03] Harvey, G. (2003). Excel 2003 all-in-one desk reference for Dummies. Hoboken: Wiley.

In spite of the demeaning title, this is one of the most popular reference guide with
users. It covers very well Excel from a user’s perspective and is therefore an ideal
reference for this work. We ourselves learned quite a few “Excel tricks” while reading
it. The same author has published guides in this series for Excel 2007 and 2010.

[HM94] Hodas, J. S., & Miller, D. (1994). Logic programming in a fragment of intuitionistic linear
logic. Information and Computation, 110(2), 327–365.

This paper describes the first attempt at marrying the proof-theoretic approach to
logic programming pioneered in [MNPS91] with linear logic, a then-novel logic to
reason about resources rather than truths. The result was Lolli, the first linear logic
programming language. The paper also applies it to numerous examples in areas such
as theorem proving, natural language parsing, and data base programming, and it
explores its model theory.

[How98] Howe, J. M. (1998). Proof search issues in some non-classical logics. PhD thesis,
University of St Andrews.

This dissertation expands Andreoli’s use of focusing [And92] from linear logic to
mainstream intuitionistic logic.

[HPSBC 04] Horrocks, I., Patel-Schneider, P., Boley, H., Tabet, S., Grosof, B., & Dean, M. (2004).
SWRL: A semantic web rule language combining OWL and RuleML. Available at http://
www.w3.org/Submission/SWRL/.

This paper proposes SWRL, a semantic web rule language based on a combination of
two fragments of the OWL web ontology language with a sublanguage of RuleML,
a variant of Datalog. It gives a model-theoretic semantics of SWRL ontologies and
demonstrates it on a number of examples.

[JL87] Jaffar, J., & Lassez, J. -L. (1987). Constraint logic programming. In Proceedings of the
14th symposium on principles of programming languages – POPL’87, Munich (pp. 111–119).
384 Annotated Bibliography

This is the original paper on constraint logic programming, a paradigm that seamlessly
incorporates constraints over generic domains within logic programming in the style
of Prolog. The result is a family of languages, one for each specific domain and
accompanying constraint-solving procedures, that retain the declarative nature of logic
for programming purposes, but specializes it to domains of interest in a natural way.

[JM94] Jaffar, J., & Maher, M. J. (1994). Constraint logic programming: A survey. Journal of
Logic Programming, 19/20, 503–581.

This paper surveys the foundations, applications and developments of constraint logic
programming since its inception in [JL87].

[KG07] Kassoff, M., & Genesereth, M. (2007). PrediCalc: A logical spreadsheet management
system. The Knowledge Engineering Review, 22(03), 281–295.

This paper describes PrediCalc, an extension of the traditional spreadsheet with

bidirectional constraints that blurs the distinction between input data and computed
values. Its bidirectional propagation mechanism automatically makes values available
for further computation as soon as constraints are solvable. It also embeds an approach
for handling the inconsistencies that may arise while solving constraints.

[KKR95] Kanellakis, P., Kuper, G., & Revesz, P. (1995). Constraint query languages. Journal of
Computer and System Sciences, 51(1), 26–52.

This paper proposes an approach to representing and retrieving data in a database that
stores information not as facts (or tuples), but as a conjunction of simple constraints
over a given domain. This is especially useful for applications, such as geographic
databases, where objects of interest are naturally expressed as constraints, spatial
constraints in this case.

[Kle67] Kleene, S. C. (1967). Mathematical logic. New York: Wiley.

This is a classical textbook on mathematical logic by one of the masters of the field.

[KLP00] Kuper, G., Libkin, L., & Paredaens, J. (Eds.). (2000). Constraint databases.
Berlin/New York: Springer.

This collection surveys the then-nascent field of constraint databases, with contribu-
tions by many of it pioneers. Constraint databases rely on constraints to describe some
types of data, for example temporal or spatial data, and to draw inferences from them
in an efficient way.

[Kow79] Kowalski, R. A. (1979). Logic for problem solving. Prentice Hall.

This classic book investigates the application of logic to problem solving and computer
programming using Prolog.

[Kri82] Kriwaczek, F. (1982). Some applications of Prolog to decision support systems. Master’s
thesis, Imperial College, London.

This master’s thesis is regarded as the first attempt ever to combine spreadsheets and
logic programming. See the discussion in the polished version [Kri88].

[Kri88] Kriwaczek, F. (1988). Logicalc: A Prolog spreadsheet. Machine intelligence, 11, 193–208.
Annotated Bibliography 385

This work refines [Kri82] and implements its design as a system called LogiCalc.
Its initial goal was to “reconstruct logically, in micro-Prolog, a typical spreadsheet
program”. Besides capturing most of the functionalities of early spreadsheets, it
provides powerful extensions such as relational manipulations, integrity constraints,
bidirectional variables, symbolic reasoning, and complex objects (e.g., lists, graphics
and spreadsheets themselves). It is based on the limited teletype interface that predated
the widespread availability of graphical user interfaces with windows. LogiCalc relies
a lot on user input, even for non-essential tasks.

[LC12] Lam, E. S. L., & Cervesato, I. (2012). Modeling datalog fact assertion and retraction in
linear logic. In A. King (Ed.), Proceedings of the 14th international ACM symposium on
principles and practice of declarative programming – PPDP’12, Leuven.

This paper models the propagation of the assertion of new facts or the retraction of
old facts in the Datalog system by means of inference in linear logic. To this effect,
it translates Datalog clauses into linear logic formulas that also contain specifications
about how to propagate updates.

[LCGC 06] Loo, B. T., Condie, T., Garofalakis, M., Gay, D. E., Hellerstein, J. M., Maniatis,
P., Ramakrishnan, R., Roscoe, T., & Stoica, I. (2006). Declarative networking: Language,
execution and optimization. In Proceedings of the 2006 SIGMOD international conference
on management of data – SIGMOD’06, Chicago (pp. 97–108). ACM.

This paper proposes a variant of Datalog as a language for declarative specifications

in computer networks. To this effect, it adapts the semi-naı̈ve evaluation strategy to
operate efficiently in asynchronous distributed settings and to account for bursts of
updates. It also explores optimization opportunities in network applications.

[Llo87] Lloyd, J. W. (1987). Foundations of logic programming (2nd extended ed.).

Berlin/New York: Springer.

This is the classical textbook on the theory of logic programming: it collects and
unifies results on SLD-resolution, unification, model and fixpoint semantics, negation-
as-failure, that had appeared in separate papers. It does not consider any of the recent
proof-theoretic developments, but it is a valuable reference. As such, it has been used
by generations of students and scholars of logic programming.

[LM03] Li, N., & Mitchell, J. C. (2003). Datalog with constraints: A foundation for trust
management languages. In Proceedings of the fifth international symposium on practical
aspects of declarative languages (PADL’03), New Orleans.

This paper shows how Datalog can fruitfully be used to specify access control and
trust management policies in order to decide authorization and access in a distributed
system of computers. To this effect, it extends Datalog with constraints over signed
messages. The paper explores the theoretical properties of the resulting language and
relates it to other authorization and trust management languages.

[LM09] Liang, C., & Miller, D. (2009). Focusing and polarization in linear, intuitionistic, and
classical logic. Theoretical Computer Science, 410(46), 4747–4768.

This paper synthesizes in a single presentation some of the best results that have
been proposed about focusing and polarization. Focusing guides the task of executing
a logic program or searching for a proof in logic, while polarization systematically
decorates formulas to direct and enable focusing.
386 Annotated Bibliography

[LNS82] Lassez, J. -L., Nguyen, V. L., & Sonenberg, L. (1982). Fixed point theorems and
semantics: A folk tale. Information Processing Letters, 14(3), 112–116.

This paper is an interesting account of the development of the theory of fixpoints.

[LP09] Lovas, W., & Pfenning, F. (2009). Refinement types as proof irrelevance. In P. -
L. Curien (Ed.), Proceedings of the 9th international conference on typed lambda calculi and
applications – TLCA’09, Brasilia (Lecture notes in computer science, Vol. 5608, pp. 157–
171). Springer.

This paper examines proof irrelevance, a mechanism for selectively hiding the
identities of terms in type theories, and relates it to refinement types, a powerful
approach to classify terms.

[LPPW05] López, P., Pfenning, F., Polakow, J., & Watkins, K. (2005). Monadic concurrent linear
logic programming. In A. Felty (Ed.), Proceedings of the 7th international symposium on
principles and practice of declarative programming – PPDP’05, Lisbon (pp. 35–46). ACM.

This paper introduces Lollimon, a concurrent linear logic programming language

inspired by CLF [WCPW03]. Lollimon combines Prolog-style backward-chaining
inference with several forms of forward-chaining (Datalog-style saturation for per-
sistent programs, and quiescence for terminating linear programs). Backward- and
forward-chaining are regulated by a monad.

[LRB91] Lewis, C., Rieman, J., & Bell, B. (1991). Problem-centered design for expressiveness and
facility in a graphical programming system. Human-Computer Interaction, 6(3–4), 319–355.

This paper advocates a design methodology for graphical systems that is driven by
concrete examples of tasks that users are expected to accomplish (problems) rather
than abstract cognitive targets. This expands the space of design alternatives and acts
as a benchmark to evaluate solutions.

[LS09] Liu, Y., & Stoller, S. (2009). From datalog rules to efficient programs with time and space
guarantees. ACM Transactions on Programming Languages and Systems – TOPLAS, 31(6),
1–38.

This paper describes a method for transforming a Datalog program into an efficient
specialized implementation with guaranteed worst-case time and space complexities.
The transformation exploits fixed-point computation, incremental maintenance of
invariants, and combinations of indexed and linked data structures.

[Mac94] MacGregor, R. (1994). A description classifier for the predicate calculus. In Proceedings
of the twelfth national conference on artificial intelligence – AAAI’94, Seattle (pp. 213–220).

This paper presents a general description classifier for first-order logic extended with
sets, equality and scalar inequality. A description classifier is a method to organize
concepts and relations into a taxonomy based on subsumption.

[McA02] McAllester, D. (2002). On the complexity analysis of static analyses. Journal of the
ACM, 49, 512–537.

This paper describes a method for computing run-time complexity bounds for Datalog
programs. More generally, it shows that specifying an algorithm using Datalog is
beneficial for reasoning about its complexity.
Annotated Bibliography 387

[Men97] Mendelson, E. (1997). Introduction to mathematical logic (4th ed.). Princeton: Van
Nostrand-Reinhold.

This is another good textbook on mathematical logic. It is notable for its breadth of
topics.

[Mil89] Miller, D. (1989). A logical analysis of modules in logic programming. Journal of Logic
Programming, 6(1–2), 79–108.

This paper explores the theory and application of a general language of Horn
clauses augmented with embedded implication and quantification (hereditary Harrop
formulas), which goes well beyond Datalog. It studies its proof theory, model theory
and fixpoint semantics in an intuitionistic setting. It shows how embedded clauses
provide a foundational justification for parametric modules in logic programming.

[Min87] Minker, J. (Ed.). (1987). Foundations of deductive databases and logic programming.
Los Altos: Morgan Kaufmann.

This book collects original articles by the main players of the then-nascent and very
fertile field of deductive databases. It grew out of a workshop by the same title held in
Washington, DC the year before.

[MNPS91] Miller, D., Nadathur, G., Pfenning, F., & Scedrov, A. (1991). Uniform proofs as a
foundation for logic programming. Annals of Pure and Applied Logic, 51, 125–157.

This now classic paper was the first to establish top-down logic programming on solid
proof-theoretic grounds. The methodology it embraced, based on the notion of uniform
proofs, identified fragments of logic larger than Prolog’s Horn clauses that could be
used for programming. Specifically, it shows that hereditary Harrop formulas, which
extend Horn clauses with embedded implication, fit into this pattern, and so do higher-
order variants of both languages.

[Nie93] Nielsen, J. (1993). Usability engineering. Boston: Academic.

This is a classic textbook on practical approaches to engineering usability in software.

It covers both cost-effective methods that will help developers improve their user
interfaces and software engineering principles to reduce delays due to usability issues
in a project.

[NJLS11] Nigam, V., Jia, L., Loo, B. T., & Scedrov, A. (2011). Maintaining distributed logic
programs incrementally. In Proceedings of the 13th international ACM symposium on
principles and practice of declarative programming – PPDP’11, Odense (pp. 125–136).
ACM.

This paper explores an efficient asynchronous algorithm to incrementally compute the

changes to the states of a distributed Datalog system, as proposed in some network
protocols and multi-agent systems, in response to the insertions and deletions of base
facts.

[NM94] Nielsen, J., & Mack, R. L. (Eds.). (1994). Usability inspection methods. New York: Wiley.

This collection of essays brings together the best approaches to predictive user
modeling in use at the time, with special emphasis on various types of cognitive
walkthrough. It reports on numerous case studies and gives practical guidance to
developers interested in embedding usability inspection into the software cycle.
388 Annotated Bibliography

[NM10] Nigam, V., & Miller, D. (2010). A framework for proof systems. Journal of Automated
Reasoning, 45(2), 157–188.

This paper gives a thorough characterization of polarization in the computation-as-

proof-search paradigm. In particular, it shows that the standard backward and forward
strategies for Prolog and Datalog correspond simply to two different polarizations of
atomic formulas in focused proofs.

[O’K90] O’Keefe, R. A. (1990). The craft of Prolog. Cambridge: MIT.

This classic textbook focuses on advanced programming techniques in Prolog as well

as on universal programming notions such as elegance and understanding a problem
before starting to write code. It is best read after [CM87] or [SS94].

[O’N94] O’Neil, P. (1994). Databases: Principles, programming, performance. San Francisco:

Morgan Kaufmann.

This is a basic textbook on databases that rigorously links theory to the real world of
database usage, programming and administration.

[OO90] Olson, J., & Olson, G. (1990). The growth of cognitive modeling in human-computer
interaction since GOMS. Human-Computer Interaction, 5, 221–265.

This retrospective paper explores the advances of cognitive sciences and human-
computer interaction technology since the proposals of the GOMS model. It also
spells out the numerous shortcomings of methodologies of the time, for example
their inability to account for things like user fatigue, individual differences, or mental
workload. This is a very interesting read.

[Pan98] Panko, R. R. (1998). What we know about spreadsheet errors. Journal of End User
Computing (Special issue on Scaling Up End User Development), 10(2), 15–21. Available at
http://panko.shidler.hawaii.edu/ssr/Mypapers/whatknow.htm.

This very influential article carried out a systematic study of the number and type of
errors found in spreadsheets. At the outset, the results are surprisingly high, which
should limit our confidence about decisions made on the basis of spreadsheet calcu-
lations. This research also finds that this type and number is similar to programming
in general, except that software engineering has developed methods to curb this error
rate, while spreadsheet users take a more informal approach.

[PBB03] Peyton-Jones, S., Blackwell, A., & Burnett, M. (2003). A user-centred approach to
functions in Excel. In Proceedings of the eighth ACM SIGPLAN international conference
on functional programming, Uppsala (pp. 165–176). ACM.

This paper proposes a method for extending the traditional spreadsheet with user-
defined functions without the expectation that an everyday user will acquire the
sophistication of an addon programmer. This work relies on cognitive psychology
techniques such as the cognitive dimension of notations and the attention investment
model to retain the usability that characterizes the traditional spreadsheet. This work
was extremely influential in the development of the deductive spreadsheet.

[Pfe] Pfenning, F. (Forthcoming). Computation and deduction. Cambridge University Press.

This forthcoming textbook gives a modern presentation of logic geared towards

computer scientists. It develops logic around the general notion of judgment rather
Annotated Bibliography 389

than truth. Special attention is given to the computational uses of logic, with a
rigorous development of the operational aspects of logic programming out of pure
logic foundations.

[Pfe89] Pfenning, F. (1989). Elf: A language for logic definition and verified meta-programming.
In Fourth annual symposium on logic in computer science, Pacific Grove (pp. 313–322). IEEE
Computer Society Press.

This paper describes the Elf system, which provides a type-checker, type-
reconstruction support, and top-down execution in the style of Prolog to the LF type
theory.

[Pfe92] Pfenning, F. (Ed.). (1992). Types in logic programming. Cambridge: MIT.

This volume brings together the principal approaches to weaving types in logic
programming. Essays within describe specific techniques but also emphasize the
overall benefits of types, including reduced error rates, support for modularity, and
compilation assistance.

[Pow03] Power, D. J. A brief history of spreadsheets. http://www.dssresources.com/history/

sshistory.html, 10/04/2003. DSSResources.Com.

This web site gives a fascinating description of how the modern spreadsheet came
about, how it evolved from the “books” that accountants have used for millennia and
how a Harvard Business School student, Dan Bricklin, envisioned and then realized an
electronic version of it. This is a good complement to Bricklin’s own account of the
invention of the spreadsheet [Bri09].

[Prz87] Przymusinski, T. C. (1987). On the declarative semantics of stratified deductive databases

and logic programs. In J. Minker (Ed.), Foundations of deductive databases and logic
programming [Min87] (pp. 193–216). Los Altos: Morgan Kaufmann.

This is the original paper on local stratification.

[PS99] Pfenning, F., & Schürmann, C. (1999). System description: Twelf – A meta-logical
framework for deductive systems. In Proceedings of the 16th international conference on
automated deduction – CADE-16, Trento (Lecture notes in artificial intelligence, Vol. 1632,
pp. 202–206). Springer.

This paper describes Twelf, a reimplementation of the Elf system [Pfe89] extended
with theorem-proving capabilities for specifications written in the LF type theory.

[Rei78] Reiter, R. (1978). On closed world data bases. In H. Gallaire & J. Minker (Eds.), Logic
and data bases (pp. 55–76). New York: Plenum.

This is the original paper on the closed world assumption.

[Rob65] Robinson, J. A. (1965). A machine-oriented logic based on the resolution principle.

Journal of the ACM, 12(1), 23–41.

This classic paper sets the foundations of theorem proving and introduces the resolu-
tion principle, which is one of the tenets of top-down evaluation in logic programming.

[RRW06] Ramakrishnan, C. R., Ramakrishnan, I. V., & Warren, D. S. (2006). Deductive

spreadsheets using tabled logic programming. In Proceedings of the 22nd international
conference on logic programming – ICLP’06, Seattle (pp. 391–405).
390 Annotated Bibliography

This paper describes the XcelLog deductive spreadsheet system [RRW07] with
particular emphasis on using the spreadsheet metaphor for the purpose of taming the
rule interactions. It also discusses the details of its implementation as an Excel addon
powered by the XSB tabled logic programming system [War98] as a back-end.

[RRW07] Ramakrishnan, C. R., Ramakrishnan, I. V., & Warren, D. S. (2007). XcelLog:

A deductive spreadsheet system. The Knowledge Engineering Review, 22(03), 269–279.

This paper describes the design and implementation of XcelLog, an extension of the
traditional spreadsheet with deductive capabilities. Like the proposal in this book,
XcelLog is meant to retain the usability of the spreadsheet while enabling end-users
to use it to draw logical consequences. The approach treats sets as the fundamental
data type and inference rules as specifying among sets. The spreadsheet metaphor is
used to create and view the materialized sets. The implementation relies on the XSB
system [War98].

[RSS92] Ramakrishnan, R., Srivastava, D., & Sudarshan, S. (1992). Efficient bottom-up evaluation
of logic programs. In P. Dewilde & J. Vandewalle (Eds.), The state of the art in computer
systems and software engineering. Kluwer Academic.

This paper reviews the state of the art of bottom-up evaluation and proposes a
transformation on Datalog programs to speed up that semi-naı̈ve evaluation. It also
discusses extensions to negation, set terms, constraints and quantitative reasoning.
Comparisons with numerous systems are included.

[RSSS94] Ramakrishnan, R., Srivastava, D., Sudarshan, S., & Seshadri, P. (1994). The CORAL
deductive system. Journal of Very Large Databases, 3(2), 161–210.

This paper describes CORAL, an advanced deductive database system featuring

optimized queries, modularity, and extensibility.

[RU95] Ramakrishnan, R., & Ullman, J. D. (1995). A survey of deductive database systems.
Journal of Logic Programming, 23(2), 125–149.

This brief but excellent survey of the development of deductive databases up to 1993
contains very little theory, but gives a clear intuition of what deductive databases
are good for. It describes the major projects and their contributions to the field. The
references are particularly useful.

[Saj00] Sajaniemi, J. (2000). Modeling spreadsheet audit: A rigorous approach to automatic

visualization. Journal of Visual Languages and Computing, 11, 49–82.

This paper explores approaches to curbing the error rate in spreadsheets observed
by [Pan98]. It does so through two auditing modalities for visualizing the flow of
information, both based on the underlying dependencies and on how people actually
use spreadsheets.

[Sar10] Sarnat, J. (2010). Syntactic finitism in the metatheory of programming languages. PhD
thesis, Department of Computer Science, Yale University.

This thesis examines in great detail, among other things, mode-checking and termina-
tion in the Twelf system, which embeds a higher-order logic programming language
with a top-down execution semantics.

[SB89] Spenke, M., & Beilken, C. (1989). A spreadsheet interface for logic programming. In
CHI’89: Proceedings of the SIGCHI conference on human factors in computing systems,
Austin (pp. 75–80). ACM.
Annotated Bibliography 391

This work proposes to combine logic programming and spreadsheets under the
umbrella of the PERPLEX system. It does so by making heavy use of integrity
constraints in a way that is clearly expressive, but likely to be unnatural for an
entry-level user. PERPLEX supports exploratory programming and the paradigm of
“query-by-example” to perform many programming tasks. While very powerful, it is
not clear that this blends smoothly in the spreadsheet mindset. This is the first logical
spreadsheet extension to make use of graphical input and not just output.

[SBB96] Spenke, M., Beilken, C., & Berlage, T. (1996). FOCUS: The interactive table for product
comparison and selection. In UIST ’96: Proceedings of the 9th annual ACM symposium on
user interface software and technology, Seattle (pp. 41–50). ACM.

This paper and the accompanying implementation include many very interesting ideas
for effectively displaying tabular information, specifying queries, and designing a new
relation on the basis of a few mouse clicks.

[SGG86] Smith, D. E., Genesereth, M. R., & Ginsberg, M. L. (1986). Controlling recursive
inference. Artificial Intelligence, 30(3), 343–389.

This paper studies in great detail to what depth it is sufficient to expand a derivation
tree for a top-down evaluation of a logic program. It also leverages properties such
as transitivity and subsumption, as well as characteristics of the domain to extend the
class of prunable programs.

[Sim12] Simmons, R. J. (2012). Structural focalization. Available at http://arxiv.org/pdf/1109.

6273v3.pdf.

This paper revisits the premises by which a generic logical derivation can be
transformed into a focused derivation, thereby replacing the “long and tedious proofs”,
which producing such a result has required so far, with a simple structural argument.

[Spe00] Spencer, R. (2000). The streamlined cognitive walkthrough method, working around
social constraints encountered in a software development company. In Proceedings of the
SIGCHI conference on human factors in computing systems – CHI’00, The Hague (pp. 353–
359).

This paper proposes a streamlined variant of the cognitive walkthrough approach to

usability analysis that mitigates some of the drawbacks of the original methodology
observed in software companies.

[SS94] Sterling, L., & Shapiro, E. (1994). The art of Prolog (2nd ed.). Cambridge: MIT.

This is another classic introductory textbook on programming in Prolog that has been
used by generations of students.

[SSFd00] Schroyens, W., Schaeken, W., Fias, W., & d’Ydewalle, G. (2000). Heuristic and analytic
processes in propositional reasoning with negation. Journal of Experimental Psychology:
Learning, Memory, and Cognition, 26(6), 1713–1734

This paper describes a series of experiments aimed at analyzing the cognitive impact
on analytic process of conditional reasoning with negations and double negations.

[Tar55] Tarski, A. (1955). A lattice-theoretical fixpoint theorem and its applications. Pacific
Journal of Mathematics, 5, 255–309.

This is the original article on the theory of fixpoints.

392 Annotated Bibliography

[TB08] Tallis, M., & Balzer, R. M. (2008). A functional spreadsheet framework for authoring
logic implication rules. In N. Bassiliades, G. Governatori, & A. Paschke (Eds.), Proceedings
of the seventh international symposium on rule interchange and applications – RuleML’08,
Orlando (Lecture notes in computer science, Vol. 5321, pp. 219–226). Springer.

This paper explores an extension of the traditional spreadsheet with logical reasoning
capabilities aimed at capturing the semantics of a binary relation by using spreadsheet
artifacts to specify its image. This model is internally translated into a system of
clauses. A prototype has been implemented as an Excel addon.

[TvD88] Troelstra, A. S., & van Dalen, D. (Eds.). (1988). Constructivism in mathematics: An
introduction. Amsterdam/New York: North-Holland.

This two-volume set is a now classic reference to constructive logic and its applications
in a variety of fields. It covers intuitionistic logic both at an introductory level and in
exhaustive depth.

[TWB07] Tallis, M., Waltzman, R., & Balzer, R. (2007). Adding deductive logic to a COTS
spreadsheet. The Knowledge Engineering Review, 22(03), 255–268.

This paper describes a logic-based application that extends Microsoft Excel with
mechanisms to flag a range or a cell as part of a relation. These relations are
automatically copied into a deductive logic engine implementing the OWL+SWRL
ontology language [HPSBC 04]. The extension also contains mechanisms to populate
a range with values deduced by evaluating logic rules.

[Ull88] Ullman, J. D. (1988). Principles of database and knowledge-base systems (Vol. 3).
Rockville: Computer Science Press.

This two-volume set reviews the theory and practice of databases as of the late 1980s,
with an eye towards then-emerging technologies such as object-oriented databases,
deductive databases and security issues. It also explores their application to reasoning
tasks, especially in the area of knowledge management.

[vEOT86] van Emden, M. H., Ohki, M., & Takeuchi, A. (1986). Spreadsheets with incremental
queries as a user interface for logic programming. New Generation Computing, 4(3), 287–
304.

This paper describes an implementation of the concept of incremental query, by which

a standard Prolog query can be refined interactively, and its application for solving
spreadsheet-like problems. The output takes the form of the rudimentary, teletype-
based, spreadsheets of the time; the input is however still given from the command
line. While taking limited advantage of the input facilities of a spreadsheet, this extends
their typical functionalities with forms of symbolic reasoning. Multiple solutions are
however shown one at a time. The whole system is implemented in Prolog, and
efficiently realizes top-down search with intelligent backtracking. The claims to target
relatively smart users rather than expert programmers is hardly met.

[VvBCE07] Valente, A., van Brackle, D., Chalupsky, H., & Edwards, G. (2007). Implementing
logic spreadsheets in LESS. The Knowledge Engineering Review, 22(03), 237–253.

This paper combines the traditional spreadsheet and an expressive knowledge repre-
sentation and reasoning system into LESS, a logic-based extension of the spreadsheet.
LESS supports reasoning with arbitrary first-order logic formulas, subsumption, clas-
sifications, higher-order constructs and much more. This expressiveness is brokered to
the user by offering four modalities requiring increasing levels of sophistication for
users to interact with the system.
Annotated Bibliography 393

[War98] Warren, D. S. (1998). Programming with tabling in XSB. In D. Gries & W. P. de Roever
(Eds.), Programming concepts and methods, IFIP TC2/WG2.2,2.3 international conference
on programming concepts and methods (PROCOMET’98): Vol. 125 of IFIP conference
proceedings, Shelter Island (pp. 5–6). Chapman & Hall.

XSB is Prolog with a form of memoization called tabling. Declaratively, it does not dif-
fer substantially from Prolog. Procedurally, however, it substantially improves on the
standard top-down SLDNF resolution strategy of this language by nearly eradicating
control-induced infinite loops. In particular, an XSB query always terminates against
a Datalog program. Tabling can also have a positive effect on efficiency, although
this may depend on how the user writes his programs. This makes XSB a promising
candidate as the underlying explanation engine of a deductive spreadsheet.

[WCPW03] Watkins, K., Cervesato, I., Pfenning, F., & Walker, D. (2003). A concurrent logical
framework: The propositional fragment. In Types for proofs and programs: Proceedings of the
TYPES 2003 international workshop, Torino (Lecture notes in computer science, Vol. 3085,
pp. 355–377). Springer.

This paper describes the foundations of a language where top-down and bottom-
up execution work together toward modeling concurrent and distributed problems.
This language is based on linear logic and takes a type-theoretic approach. Distinct
fragments support top-down and bottom-up execution, and they are mediated by means
of a monad.

[WJL72] Wason, P. C., & Johnson-Laird, P. N. (1972). Psychology of reasoning: Structure and
content. Cambridge: Cambridge University Press.

This classic cognitive psychology textbook is concerned with why we reason the way
we do: we are not infallible reasoners yet certain deductions come to us naturally
while others require quite a bit of effort, and often end up being erroneous. One such
difficult pattern is negation. This book is a fascinating in-depth analysis of the often
problematic relationship between man and logic.

[Zan99] Zaniolo, C. (1999). A short overview of deductive database systems. In W. Kloesgen &
J. Zytkow (Eds.), The handbook of data mining and knowledge discovery. Oxford/New York:
Oxford University Press.

In just three pages, this precious little overview describes deductive databases, recalls
their history, and outlines the main open research problems. It is followed by a much
longer set of references to the main papers that led to their development.
Symbols in Use

 Edge in graph
 Path in graph
C
 Added edge dependency

 Deleted edge dependency
Ü Functional dependency
) Constraint dependency
jˆ Intuitionistic validity
 Dependent subexpression
0 Positional dependent subexpression
v .@/ Subexpression (strict)
vr .@r / Relational subexpression
va .@a / Dependent attribute
@æ Attribute expression dependency
Env .<Env/ Environment ordering (strict)
 Comparison operator

b Comparison operation
s Component referential closure
Es Array referential closure
ΠEmpty list
j Append
[ Relational union
[EH;[ EV Array append
 Set difference
 Cartesian product
j k Spreadsheet index set
‰ Join
\ Relational intersection
;; ;I Empty relation
e@ Expression evaluation
s ˚ s Update application
? Uncalculated value

I. Cervesato, The Deductive Spreadsheet, Cognitive Technologies, 395

DOI 10.1007/978-3-642-37747-1, © Springer-Verlag Berlin Heidelberg 2013
396 Symbols in Use

?N Uncalculated relation
? Initial environment
[ Blank value
 Error value
E Error array
N Error relation
Œ1::n Index range
Œ1::j  Œ1::k Spreadsheet index set
ŒEe List of values
N
Œe Relation-to-array coercion
hEei Base relation
peqN Logical interpretation of eN
.x1 ; : : : ; xn / Extensional array or record
flgE Location array partition
jxj
E Length of vector or record xE
jGj Height of DAG G
rs  Next environment non-update

Theory, context
p Clausal theory of p
rec.p/ Mutually recursive clauses
s Clausal theory of s
&
s Clauses of stratum &
L Repartition
p Positive update to p
pCl Positive update to p by clause Cl
s Spreadsheet update
s  Next environment update
…p Factual theory of p
…s Factual theory of s
&
…s Facts of stratum &
… Definite/normal theory
… s Definite/normal theory of s
… & Subtheory of &
… si Subtheory of &Eis
…  Bounding transformation of …
˛ Substitution
ıp Negative update to p
ıpCl Negative update to p by clause Cl
 Environment
s Updated part of environment
 Index
' First-order formula
æE Projection
Symbols in Use 397

E æE Array projection
C Selection
EC Array election
& Stratum
&E Stratification
&Eis Sub-stratification of level i

E Set of scalar expressions

D Domain
HBV Herbrand base
I Index space
L Set of locations
LI Set of array references
PX Powerset of X
R Set of relational expressions
V Set of scalar values
V? Set of scalar values or ?
VI Set of array of values
VN Set of relational values
V… Admissible values of …
X Generic set
XI Set of I -arrays over X
XI Set of arrays over X
Xn Set of n-ary relations over X
XN I Set of I -relations over X
XNI ; XN Set of relations over X

Attr.æ/ Attributes of æ
Attr.C / Attributes of C
N
Attr.x/ Attributes of xN
N
Attr .x/ Selection attributes of xN
B Clause body
C Selection condition
b
C Selection function
CG… Call graph of …
CGs Call graph of s
CG… Œi  Predicates at level i
Cl Clause
DFs Dataflow graph
DGs Dependency graph
DGas Attribute-dependency graph
DGAs Two-level attribute-dep. graph
DGrs Relation-dependency graph
DGR s Two-level relation-dep. graph
E./ Generic evaluation strategy
398 Symbols in Use

EP s ./ Semi-naı̈ve evaluation step

EP s0 ./ Localized semi-naı̈ve evaluation step
ER s ./ Naı̈ve evaluation step
ER s0 ./ Localized naı̈ve evaluation step
Edges.G/ Edges of a graph
EGs ./ Evaluation graph
EGAs ./ Attribute evaluation graph
Env Set of environments
FDs .E &/ Functional dependency graph
H Clause head
I Index set, geometry
I Interpretation
IxE Geometry of array xE
IxN Geometry of relation xN
I… Least Herbrand model
I… Perfect model
I…? Partial perfect model
I… & Perfect model
I…? & Partial perfect model
InstV . / Set of ground clauses
L Spreadsheet partition function
L ˇ L Repartition application
Nodes.G/ Nodes of a graph
P Predicate, literal
Ns
RS Relational substrate of s
RSs Relational substrate graph of s

!
RSs Relational projection of RSs
fSs
R Stratified call graph
fSs&
R Call subgraph of &
SORTaE Sort function
T…; One-step logical consequence
T…?& ; & Partial one-step logical consequence
X Logical variable

a Attribute
æ Attribute expression
æO Evaluated attribute expression
c Base selection condition
cellss Updated cells
e Scalar expression
eE Array expression
eN Relational expression
gfp Greatest fixpoint
glb Greatest lower bound
hcmax Bound on bounding transformation
Symbols in Use 399

l Location
lE Array of location
lfp Least fixpoint
lub Lower upper bound
op Scalar operator
opb Scalar operation
b
op? Scalar operation over V?
opE Array-valued operator
E
opΠScalar component of op
bE
b
op Array-valued operation
opE? Array-valued operation over V?
op N Relational operator
bN
b
op Relational operation
opE? Relational operation over V?
p Predicate symbol
pji Predicate argument pair
q Predicate symbol
r Record
.r; r 0 / Record concatenation
rec.p/ Predicates mutually recursive with p
s Spreadsheet
s ˇ s Update application
t Term
t Recalculation delay
v Scalar value
Ev Array value
vN Relational value
x Generic set element
xE Generic array or record
xE ΠArray or record element
xN Generic relation
N
xŒa Extensional attribute
Index

Abbreviated form, 316 Audience, 2, 4, 296, 347, 358

Abstraction, 287 Auditing, 275
parametric, 287 Autoformat, 307
Add-on, 277, 366 Autosum, 270
Aggregator, 321
Anti-trust control problem, 240
Append, 84 Backtracking, 153
Argument Base case, 193
dependent, 61 Base relation, 310, 312
Arity, 35, 68 Beilken, Christian, 18
Array, 55, 57, 58, 311 Bill of materials problem, 16, 236
expression, 60 Blackwell, Alan, 279, 293
formula, 318 (see also Array, expression) Body, 131, 192, 243, 315
geometry, 57, 62, 335 definite, 140
location (see Array, reference) normal, 155
operation, 59 Bottom element, 40, 101, 145, 246
operator, 59 Bounding transformation, 219, 222, 251
reference, 58 Bricklin, Daniel, 255
spreadsheet, 58 Burnett, Margaret, 279
value, 58
Ascending chain, 40
Assignment. See Variable assignment Calculation. See Evaluation
Attention, 294 Call graph, 132, 141, 155, 193
Attention investment model, 279 enriched, 156, 194
Attribute, 69, 320 stratified, 137
addition, 337 strict, 156
base, 89 Cartesian product, 75
deletion, 335, 337 Cell, 262, 311. See also Location
dependent, 87 active, 262
extensional, 69 Cell coordinates
list, 227 A1, 262, 309
relation, 69 R1C1, 262, 272, 311
Attribute dependency, 102 Cell range. See Array
Attribute expression, 81, 130, 319–321 Chart, 277
evaluated, 83, 116 Circular reference, 57, 62, 91
Attribute-dependency graph, 102 Clark completion, 169
two-level, 103 Clausal theory, 133, 134, 192

I. Cervesato, The Deductive Spreadsheet, Cognitive Technologies, 401

DOI 10.1007/978-3-642-37747-1, © Springer-Verlag Berlin Heidelberg 2013
402 Index

Clause, 131, 192, 315 Data table, 277

definite, 139, 140, 193 Database, 2, 9
linear, 204 administrator, 93–95
normal, 154, 155, 193 deductive, 9, 11, 22, 212
recursive, 185 query, 10, 15, 80, 93, 97, 150, 299, 345,
safe, 140 347, 355
Clause definition relational, 91, 93, 97
dialog, 330, 331 Datalog, 8, 9, 11, 12, 22, 119, 131, 132, 139,
mouse-assisted, 326 170, 183, 212
textual, 309 Debugging, 4, 275, 340
Closed world assumption, 158, 160 Deductive database. See Database, deductive
Closeness to mapping, 288 Deductive decision-making, 3
Coercion, 72, 99, 324 Deductive engine, 11, 31
Cognitive dimension, 284 Defined predicates list, 303, 304, 306, 326, 335
Cognitive dimensions of notation, 279, 283 Definite program. See Definite spreadsheet
Cognitive walkthrough, 282 theory
Column, 261, 311 Definite spreadsheet theory, 139, 141
header, 262 bottom-up inference, 149, 150
partial unbounded, 301 fixpoint theory, 144
Command, 65, 66, 73, 76, 93, 116 model theory, 143
Complete lattice, 41, 145 non-recursive, 139
Complex objects, 249 proof theory, 148
Computation syntax, 140
deductive, 189 top-down inference, 150
functional, 12, 81, 189 Dependency
relational, 81 circular, 36
Conclusion. See Inference rule functional, 125, 193
Concrete syntax, 312 graph, 43, 102
Condition, 321, 322. See also Selection, relational, 193
condition Dependent subexpression, 61
Connection graph, 344 Derivability, 122
Conservativity, 11, 293, 299, 307, 345 Derivation, 122
cognitive, 28, 346 Destructor, 223
functional, 26, 251 Difference, 75
Consistency, 28, 288 Diffuseness, 288
Constraint, 131, 140, 329 Discovery, 29
generic, 192 Discussion tool, 284
head, 131, 140, 192, 217 Divergence, 183, 209, 217, 223
Context, 121 Domain, 122
Continuous function, 42, 98, 101, 159, 198,
246
Copy-and-paste, 272, 333, 338 Embedded implication, 183, 242, 243
Correctness, 123 bottom-up inference, 245
Cost, 101, 107, 110, 167, 201, 248 fixpoint semantics, 245
cognitive, 294 model theory, 244
Currying, 243 proof theory, 247
CWA. See Closed world assumption syntax, 243
top-down inference, 247
End-user, 2, 4, 5, 7, 9, 10, 12, 13, 49, 53, 196,
Data 280, 283, 366, 369
numerical, 1 Environment, 37, 54, 98, 163, 284
symbolic, 1 initial, 37, 98
Data form, 265 pre-evaluation, 51
Data list, 65, 265, 296 restriction, 48
Index 403

Error, 225, 322, 340 logical, 121

Error-proneness, 288 scalar, 35, 53, 266
Evaluation, 37, 95, 163, 206, 274 Formula bar, 261
array formula, 56, 99 Formula input textbox, 306
attribute expression, 100 Functional dependency, 94, 125, 133, 137–139,
base relation, 99 156, 166, 199, 215
bottom-up strategy, 8, 11 graph, 138, 156, 195
condition, 100 Functional language, 5, 64, 251
cost, 45, 95 Functional-logic programming, 12
enhanced semi-naı̈ve strategy, 206 Functional sublanguage, 189
naı̈ve strategy, 37, 38, 47, 63, 98, 101, 163,
203, 206, 246
scalar, 37, 99 Genesereth, Michael, 210
semi-naı̈ve strategy, 47, 63, 106, 166, 203, Gentzen, Gerhard, 122
206, 246, 371 Ginsberg, Matthew, 210
top-down strategy, 7 Goal seeking, 277
Evaluation graph, 45, 105 Goal selection
Existential quantification, 188 breadth-first, 152, 209
Explanation, 4, 53, 115, 177, 211, 275 depth-first, 152, 209
negative, 115, 178, 179 GOMS, 281
positive, 115, 178 Graph
Expression, 35 attribute-dependency, 102
array, 116 call (see Call graph)
relational, 71, 76, 116, 127, 319–321 dataflow, 142, 196, 230
scalar, 35, 116 dependency (see Dependency graph)
Expression update, 108 evaluation (see Evaluation graph)
Expressiveness, 26, 215, 217 functional dependency, 138, 156
Extension, 134, 168 relation-dependency, 105, 125
relational substrate (see Relational
substrate)
Fact, 3, 128, 140, 192 visualization, 277
Factual theory, 128, 192 Graphical user interface, 7, 11, 13, 18, 19, 64,
Field name, 265 253, 255, 270, 299, 367
Filter, 65, 265 Green, Thomas, 283
First-order logic, 120
classical, 121
model theory, 122 Halting problem, 218
proof theory, 122 Hard mental operation, 289
syntax, 120 Harland, James, 162
Fixpoint, 41 Head, 131, 140, 155, 192, 315
greatest, 42 Head constraint restriction, 197
least, 42, 98, 101, 145, 159, 165, 246 Help, 344
Fixpoint theory, 39 Herbrand base, 143, 145, 198, 244
Flow graph, 344 Herbrand interpretation, 143, 157, 159, 198
Form, 65 Herbrand model, 143, 158, 198
Formative methods, 283 fixpoint characterization, 144
Formatting least, 144, 145, 157, 198
cell, 263 termination, 146, 199
worksheet, 264 Herbrand model least, 199
Formula, 35, 121, 265 Herbrand universe, 143, 159
array, 56, 268 Herbrand, Jacques, 143
closed, 121 Heuristic evaluation, 281
goal, 143 Hidden dependency, 289
graphical construction, 269 Hilbert, David, 122
404 Index

Hypothetical reasoning, 183, 247 constraint, 131, 212

Hypothetical subgoal. See Embedded tabled, 210
implication Logical consequence, 122, 245
atomic, 143
Logical reasoning, 1, 66, 370
Index, 57 Löwenheim, Leopold, 143
set, 57 Lower bound, 40
space, 57 greatest, 40
Inference, 2, 26 Lower semi-lattice, 40, 101
Inference rule
conclusion, 123
premise, 123 Meeting planner, 239
Information structure, 284 Menu, 260
Insert function button, 271 Menu bar, 260
Integration, 27, 29, 72, 324 Meta-consistency, 286
Integrity constraints, 18 Model, 122, 245
Interpretation, 122, 244 perfect (see Perfect model)
Intersection, 79 Model intersection property, 143
Investment, 294 Monotonic function, 41, 101, 159, 198, 246
Irrelevant term, 230

Negation, 121, 132, 133, 138, 154, 155, 158,

Join, 78, 367 161, 195, 212, 216, 238, 242, 371,
natural, 78, 133 373
Judgment, 123 Negation-as-failure, 160
Juxtaposability, 292 Normal program. See Normal spreadsheet
theory
Normal spreadsheet theory, 154, 180
Key, 94 bottom-up inference, 161
Kleene theorem, 42, 54, 101, 145, 246 fixpoint theory, 159
Kleene, Stephen, 42 model theory, 157
Knaster-Tarski theorem, 42 non-recursive, 154
Kowalski, Robert, 144 proof theory, 160
Kripke structure, 245 syntax, 154
Kriwaczek, Frank, 18 top-down inference, 162
Notation, 284

Law of cussedness, 285

Layout, 300 One-time query, 179
List, 183, 221, 224 Operation, 35
algebra, 223 array, 59
operator, 224 scalar, 35
relation, 225 Operator, 34, 53
List modifier, 316 array, 59
Literal comparison, 77
negative, 133, 155, 192 list, 224
positive, 133, 155, 192 scalar, 34, 224
Liu, Yanhong, 202 Ordinal power, 42
Location, 34, 53 Outline, 264
Logic
classical, 242
intuitionistic, 242 Panko, Raymond, 53, 340
Logical program, 121 Partially ordered set. See Poset
Logical programming, 2, 6, 21, 131, 212 Partial order, 40
Index 405

Partition, 62, 85 absolute, 273

Payoff, 294 relative, 272
p-differential, 203 Referential closure, 35, 60, 62, 86, 90
Perfect model, 157–160, 165, 198, 199, 203 component-wise, 86
partial, 165 Relation, 2, 64, 68
Performance, 29, 367 base, 73, 76, 78, 80, 87, 95, 98, 99, 116,
Permissiveness, 293 124, 127
Peyton-Jones, Simon, 279 empty, 69
Pivot table, 277 error, 73
Polarization, 149 first-class, 66, 116, 311, 369
Poset, 40 primitive, 126, 127
Predicate, 121, 131, 140, 155, 192, 315 second-class, 66, 300
auxiliary, 317 uncalculated, 99
extensional, 134 Relational algebra, 66, 71, 74, 78, 84, 91, 99,
intensional, 134 111, 116, 212, 213, 216
level, 146 extended, 80
symbol, 121 logical interpretation, 127
Predicate name, 315 Relation-dependency graph, 105
Predictive user model, 281 abstract, 105
Pre-interpretation, 244 Relational equation, 214
Premature commitment, 290 Relational expression, 71, 76
Premise. See Inference rule extended, 82
Prerequisite, 297 Relational subexpression, 87, 102
Primitive predicate, 192 Relational substrate, 124
Productivity, 2, 10, 21, 234, 273, 277, 287, graph, 125
292, 323, 343 stratified graph, 137
Productivity tool, 64, 277, 343, 370 Repartition, 108
Programmer, 2, 5, 7, 12, 209, 250, 348, 370 Requirement
Progressive evaluation, 290 cognitive, 28
Projection, 75 functional, 25
Prolog, 6–8, 22, 131, 153, 160, 211 Resolution, 211
Proof system, 123 Restriction, 107
Pseudo-relational operator, 83, 325 Risk, 294
Role expressiveness, 291
Row, 261, 311
Qualified name, 70 header, 262
Query optimization, 97, 150, 203

Safety restriction, 140, 155, 243

Range. See Array Saturation, 207
Range name, 70, 262, 310 Scalar, 58
Recalculation. See Update Scenario, 277
Record, 64, 69 Scheduling problem, 240
Recursion, 5, 9, 64, 141, 183, 347 Secondary notation, 291
mutual, 193 Selection, 75
simple, 193 base condition, 77, 82, 116
Recursive spreadsheet theory, 192 condition, 75, 77, 82, 116, 319, 320
bottom-up inference, 207 Sequence. See List
fixpoint theory, 198 Sequent, 123
model theory, 198 Skolem, Thoralf , 143
syntax, 192 SLD-resolution, 153
top-down inference, 208 Smith, David, 210
Re-evaluation. See Update Spenke, Michael, 18
Reference Spreadsheet
406 Index

array, 62, 63 Union, 75

deductive, 2, 183, 198 Universal quantification, 188
evaluation (see Evaluation) extensional, 186, 231, 233, 289, 373
explanation (see Explanation) Update, 27, 49, 108, 168, 207, 274, 318, 322,
layout, 260 335, 338
logical, 119 application, 50
recursive, 184 atomic, 114
relational, 55, 85, 91, 116 dependency graph, 51, 109
scalar, 36, 53 negative, 172, 207, 371
traditional, 2, 6, 21, 33, 116, 255 policy, 114
update (see Update) positive, 168, 204, 207, 371
user interface, 260 Upper bound, 40
SQL, 9, 22, 92, 319, 345 least, 40
Status bar, 261 Usability, 2, 19, 21, 64, 253, 279–281, 286,
Stoller, Scott, 202 299, 319, 357, 365, 369, 370, 373
Stratifiable, 195 User, 4, 14, 25, 28, 56, 93, 210, 234, 255, 275,
Stratification, 138, 156, 195 280, 293, 296, 322, 353, 373
least, 138, 157, 195 advanced, 10, 339, 355
linear, 138, 157 intermediate, 244, 297, 356
local, 158 novice, 1, 223, 292, 313, 356
Stratum, 136, 137, 156, 195 testing, 280, 298
definite, 156
normal, 156
Strongly connected component, 195 Validity, 122, 198, 244, 245
Subarray, 58 Value, 5, 34, 53
Subexpression, 35, 60 blank, 34, 43, 50, 69, 76, 86, 227
Submenu, 260 error, 34, 59
Substitution, 121, 147, 150, 208 extended, 37, 54, 98
Summative methods, 283 relational, 69
Symbolic reasoning. See Logical reasoning uncalculated, 37, 98, 163
System, 284 van Emden, Marteen, 18, 144
Variable, 120, 312, 314, 329
assignment, 122
Tarski, Alfred, 39, 42, 122 bound, 121
Term, 120, 121 free, 121
Term language, 120 ground, 120
Termination, 6–8, 12, 27, 146, 186, 199, 217, Variable assignment, 143, 147, 245
222 Vector
Terseness, 288 column, 58
Testing, 280 concatenation, 58
Theory, 121, 315 length, 58
definite, 139–141 row, 58
factual, 140 View, 9, 94
normal, 154, 155 Viscosity, 285, 291
recursive, 192 Visibility, 292
stratifiable, 138 Visual language, 284
Toolbar, 260
Top element, 41 Warren, David S., 210
Totally ordered set, 40 What-if analysis, 3, 17, 242, 247
Total order, 40 Wizard, 260, 302
Trade-off, 285 Workbook, 262
Transitive closure, 17, 184, 186, 232, 234, 374 Workflow problem, 237
Tuple. See Record Worksheet, 261
Typing, 250 World, 244