Symbolic Parallelization of Nested Loop Programs

1st Edition Alexandru-Petru Tanase pdf download

https://textbookfull.com/product/symbolic-parallelization-of-
nested-loop-programs-1st-edition-alexandru-petru-tanase/

Download more ebook from https://textbookfull.com

We believe these products will be a great fit for you. Click
the link to download now, or visit textbookfull.com
to discover even more!

China in Symbolic Communication 1st Edition Sui Yan

https://textbookfull.com/product/china-in-symbolic-
communication-1st-edition-sui-yan/

Knowledge Management The Creative Loop 1st Edition

Jean-Louis Ermine

https://textbookfull.com/product/knowledge-management-the-
creative-loop-1st-edition-jean-louis-ermine/

Classical Statistical Mechanics with Nested Sampling

1st Edition Robert John Nicholas Baldock (Auth.)

https://textbookfull.com/product/classical-statistical-mechanics-
with-nested-sampling-1st-edition-robert-john-nicholas-baldock-
auth/

Nutrient Delivery 1st Edition Alexandru Mihai

Grumezescu

https://textbookfull.com/product/nutrient-delivery-1st-edition-
alexandru-mihai-grumezescu/
The Symbolic Politics of European Integration : Staging
Europe 1st Edition Jacob Krumrey (Auth.)

https://textbookfull.com/product/the-symbolic-politics-of-
european-integration-staging-europe-1st-edition-jacob-krumrey-
auth/

Human-in-the-Loop: Probabilistic Modeling of an

Aerospace Mission Outcome 1st Edition Ephraim Suhir

https://textbookfull.com/product/human-in-the-loop-probabilistic-
modeling-of-an-aerospace-mission-outcome-1st-edition-ephraim-
suhir/

Adventures of the symbolic post Marxism and radical

democracy Breckman

https://textbookfull.com/product/adventures-of-the-symbolic-post-
marxism-and-radical-democracy-breckman/

Historic firsts : how symbolic empowerment changes U.S.

politics 1st Edition Chisholm

https://textbookfull.com/product/historic-firsts-how-symbolic-
empowerment-changes-u-s-politics-1st-edition-chisholm/

Globalisation and Historiography of National Leaders

Symbolic Representations in School Textbooks 1st
Edition Joseph Zajda

https://textbookfull.com/product/globalisation-and-
historiography-of-national-leaders-symbolic-representations-in-
school-textbooks-1st-edition-joseph-zajda/
Alexandru-Petru Tanase · Frank Hannig
Jürgen Teich

Symbolic
Parallelization
of Nested Loop
Programs
Symbolic Parallelization of Nested Loop Programs
Alexandru-Petru Tanase • Frank Hannig
Jürgen Teich

Symbolic Parallelization
of Nested Loop Programs

123
Alexandru-Petru Tanase Frank Hannig
Friedrich-Alexander-Universität Friedrich-Alexander-Universität
Erlangen-Nürnberg (FAU) Erlangen-Nürnberg (FAU)
Erlangen, Germany Erlangen, Germany

Jürgen Teich
Friedrich-Alexander-Universität
Erlangen-Nürnberg (FAU)
Erlangen, Germany

ISBN 978-3-319-73908-3 ISBN 978-3-319-73909-0 (eBook)

https://doi.org/10.1007/978-3-319-73909-0

Library of Congress Control Number: 2018930020

© Springer International Publishing AG 2018

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology
now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this book
are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or
the editors give a warranty, express or implied, with respect to the material contained herein or for any
errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional
claims in published maps and institutional affiliations.

Printed on acid-free paper

This Springer imprint is published by the registered company Springer International Publishing AG part
of Springer Nature.
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Goals and Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Symbolic Outer and Inner Loop Parallelization . . . . . . . . . . . . . . . . . . . . . . . 4
1.3 Symbolic Multi-level Parallelization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.4 On-demand Fault-tolerant Loop Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.5 Book Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2 Fundamentals and Compiler Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.1 Invasive Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 Invasive Tightly Coupled Processor Arrays. . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2.1 Processor Array. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2.2 Array Interconnect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2.3 TCPA Peripherals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3 Compiler Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.3.1 Compilation Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.3.2 Front End . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.3.3 Loop Specification in the Polyhedron Model . . . . . . . . . . . . . . . . 22
2.3.4 PAULA Language . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.3.5 PARO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.3.6 Space-Time Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.3.7 Code Generation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.3.8 PE Code Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.3.9 Interconnect Network Configuration . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.3.10 GC and AG Configuration Stream . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3 Symbolic Parallelization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.1 Symbolic Tiling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.1.1 Decomposition of the Iteration Space . . . . . . . . . . . . . . . . . . . . . . . . 39
3.1.2 Embedding of Data Dependencies. . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.2 Symbolic Outer Loop Parallelization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.2.1 Tight Intra-Tile Schedule Vector Candidates. . . . . . . . . . . . . . . . . 48
3.2.2 Tight Inter-tile Schedule Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

v
vi Contents

3.2.3 Parametric Latency Formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

3.2.4 Runtime Schedule Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.3 Symbolic Inner Loop Parallelization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.3.1 Tight Intra-Tile Schedule Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.3.2 Tight Inter-tile Schedule Vector Candidates . . . . . . . . . . . . . . . . . 68
3.3.3 Parametric Latency Formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.3.4 Runtime Schedule Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.4 Runtime Schedule Selection on Invasive TCPAs . . . . . . . . . . . . . . . . . . . . . . 76
3.5 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.5.1 Latency. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
3.5.2 I/O and Memory Demand . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
3.5.3 Scalability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
3.6 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
3.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
4 Symbolic Multi-Level Parallelization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
4.1 Symbolic Hierarchical Tiling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
4.1.1 Decomposition of the Iteration Space . . . . . . . . . . . . . . . . . . . . . . . . 95
4.1.2 Embedding of Data Dependencies. . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
4.2 Symbolic Hierarchical Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
4.2.1 Latency-Minimal Sequential Schedule Vectors . . . . . . . . . . . . . . 101
4.2.2 Tight Parallel Schedule Vectors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
4.2.3 Parametric Latency Formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
4.2.4 Runtime Schedule Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
4.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
4.3.1 Latency. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
4.3.2 I/O and Memory Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
4.3.3 Scalability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
4.4 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
4.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
5 On-Demand Fault-Tolerant Loop Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
5.1 Fundamentals and Fault Model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
5.2 Fault-Tolerant Loop Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
5.2.1 Loop Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
5.2.2 Voting Insertion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
5.2.3 Immediate, Early, and Late Voting . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
5.3 Voting Functions Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
5.4 Adaptive Fault Tolerance Through Invasive Computing . . . . . . . . . . . . . . 142
5.4.1 Reliability Analysis for Fault-Tolerant Loop Execution. . . . . 145
5.5 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
5.5.1 Latency Overhead . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
5.5.2 Average Error Detection Latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
5.6 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
5.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
Contents vii

6 Conclusions and Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

6.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
6.2 Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
Acronyms

ABS Anti-lock Breaking System

AG Address Generator
AST Abstract Syntax Tree
CGRA Coarse-Grained Reconfigurable Array
COTS Commercial Off-The-Shelf
CPU Central Processing Unit
CUDA Compute Unified Device Architecture
DMR Dual Modular Redundancy
DPLA Dynamic Piecewise Linear Algorithm
ECC Error-correcting Code
EDC Egregious Data Corruptiony
EDL Error Detection Latency
FCR Fault Containment Region
FSM Finite State Machine
FU Functional Unit
GC Global Controller
GPU Graphics Processing Unit
HPC High-Performance Computing
iCtrl Invasion Controller
i-let Invasive-let
ILP Integer Linear Program
IM Invasion Manager
i-NoC Invasive Network-on-Chip
LPGS Locally Parallel Globally Sequential
LSGP Locally Sequential Globally Parallel
MPSoC Multi-Processor System-on-Chip
NMR N-Modular Redundancy

ix
x Acronyms

PE Processing Element
PFH Probability of Failure per Hour
PGAS Partitioned Global Address Space
PLA Piecewise Linear Algorithm
SER Soft Error Rate
SEU Single-Event Upset
SIL Safety Integrity Level
SoC System-on-Chip
SPARC Scalable Processor Architecture
TCPA Tightly Coupled Processor Array
TMR Triple Modular Redundancy
UDA Uniform Dependence Algorithm
VLIW Very Long Instruction Word
List of Symbols

D ∗ – Set of tiled dependency vectors . . . . . . . . . . . . . . . . . . . 42

E[LE,early ] – The average error detection latency for early voting . . . . . 137
E[LE,imm ] – The average error detection latency for immediate voting . . . 135
E[LE,late ] – The average error detection latency for late voting . . . . . . 139
G – The number of quantified equations . . . . . . . . . . . . . . . . . . 23
I – Original iteration vector . . . . . . . . . . . . . . . . . . . . . . . . 23
I n – Input space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
J – Intra-tile iteration vector . . . . . . . . . . . . . . . . . . . . . . . 40
K – Inter-tile iteration vector . . . . . . . . . . . . . . . . . . . . . . . 41
Kf – The tile to be executed first by a symbolic schedule vector λ . . . . . 61
Kl – The tile to be executed last by a symbolic schedule vector λ . . . . . . 61
L – Latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Lg – Global latency . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Ll – Local latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
LE,early – Error detection latency for early voting . . . . . . . . . . . . . 137
LE,imm – Error detection latency for immediate voting . . . . . . . . . . . 134
LE,late – Error detection latency for late voting . . . . . . . . . . . . . . 139
Lopt – Optimal latency . . . . . . . . . . . . . . . . . . . . . . . . . . 63
M – Maximal number of symbolic schedule candidates . . . . . . . . . . 51
Out – Output space . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
P – Tiling matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
R – Replicated iteration vector . . . . . . . . . . . . . . . . . . . . . . 127
S – Path stride matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
Φ – The allocation matrix . . . . . . . . . . . . . . . . . . . . . . . . . 33
B – Set of protected variables . . . . . . . . . . . . . . . . . . . . . . . 132
λ – Schedule vector . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
λJ – Intra-tile schedule vector . . . . . . . . . . . . . . . . . . . . . . . 47
λK – Inter-tile schedule vector . . . . . . . . . . . . . . . . . . . . . . 47
λR – Schedule vector of replicated iteration space . . . . . . . . . . . . . 128
P – Processor space . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

xi
xii List of Symbols

Pred – Replicated processor space . . . . . . . . . . . . . . . . . . . . . 128

R – Set of replicated iteration space . . . . . . . . . . . . . . . . . . . . 127
C – Set of I/O constraints . . . . . . . . . . . . . . . . . . . . . . . . . 111
F – An operation to be scheduled . . . . . . . . . . . . . . . . . . . . . 32
I – Set of original iteration space . . . . . . . . . . . . . . . . . . . . . 24
J – Set of intra-tile iteration space . . . . . . . . . . . . . . . . . . . . 40
K – Set of inter-tile iteration space . . . . . . . . . . . . . . . . . . . . 40
L – Set of feasible symbolic schedule vector candidates . . . . . . . . . . 63
M – Set of memory constraints . . . . . . . . . . . . . . . . . . . . . . 111
Vk – Voting space for a protected variable xk . . . . . . . . . . . . . . . . 131
τ – Relative start offset of an operation F . . . . . . . . . . . . . . . . . 32
d – Dependency vector . . . . . . . . . . . . . . . . . . . . . . . . . . 42
d ∗ – Tiled dependency vector . . . . . . . . . . . . . . . . . . . . . . . 43
dJ – Intra-tile dependency vector . . . . . . . . . . . . . . . . . . . . . 42
dK – Inter-tile dependency vector . . . . . . . . . . . . . . . . . . . . . 42
p – Processor index . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
rv – Replica where the voting takes place . . . . . . . . . . . . . . . . . 131
t – Start time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
wi – Execution time of a operation i . . . . . . . . . . . . . . . . . . . . 33
RS – Replication space . . . . . . . . . . . . . . . . . . . . . . . . . . 127
Chapter 1
Introduction

In 1965, Gordon Moore predicted that the number of transistors per chip would
double every two years and that chips would eventually be so small, that they could
be embedded in homes, cars, and mobile devices. As shown in Figure 1.1, the above
prophecy came true. His theory, also known as the Moore’s Law [Moo65], powered
massive revolutions in computer architecture, with the most important one being the
shift towards multi- and many-core chips.
For decades, the performance gain was determined mainly by increasing the
frequency of a single Central Processing Unit (CPU) as a result of CMOS process
advances. However, in 2005 “the free lunch was over” [Sut05]: as the size of
transistors has already become less than 90nm (hundredth of the width of one
human hair), the frequency scaling reached its limit. Additional performance gains
could not be achieved by boosting the operational frequency of a CPU since it
would result in a huge increase of power consumption [DKM+ 12] and inevitable
overheating. Therefore, systems could further scale only if the energy efficiency as
well as mapping and runtime methods would considerably improve—this reasoning
holds for both embedded and portable devices such as smartphones and tablets as
well as large scale systems used for high-performance computing.
To solve this problem, multi-core designs were proposed. This allowed to
keep Moore’s Law alive: the number of transistors still increases exponentially
through the increase of the core count (see Figure 1.1). The performance gain
is now achieved through the exploitation of different levels of parallelism—
rather than through an increased frequency—by employing a mixture between
specialized hardware accelerators as well as powerful and low power processors.
Such heterogeneous designs, also known as Multi-Processor System-on-Chips
(MPSoCs), offer high performance, and can be highly versatile in comparison to
single core processors. Additionally, MPSoCs also help to alleviate the power wall
and utilization wall [GHSV+ 11] problem, where the potentially available chip area
might not be fully utilized or at least not simultaneously because the power density
and temperature will exceed their limits.

© Springer International Publishing AG 2018 1

A.-P. Tanase et al., Symbolic Parallelization of Nested Loop Programs,
https://doi.org/10.1007/978-3-319-73909-0_1
2 1 Introduction

107 Transistors
6
(thousands)
10
Single-Thread
105
Performance
(SpecINT x 103)
104
Frequency (MHz)
103
Typical Power
102 (Watts)
Number of
101 Logical Cores

100

1970 1980 1990 2000 2010 2020

Year

Fig. 1.1 Overview of the evolution of processor architectures over the last decades. Increase in
frequency and power has saturated, but Moore’s law still holds true after 50 years: The number of
transistors still increases exponentially. Figure adapted from [Rup15]

Naturally, these benefits have encouraged the design of chips with more and more
processor cores. In the near future, it is expected to have architectures with 1,000 or
even more processor cores on a single chip [Bor07]. Examples of recent multi- and
many-core architectures include IBM’s Power7 chip [KSSF10], which has eight
processor cores, with each having 12 execution units with four-way simultaneous
multi-threading, Intel’s Many Integrated Core (MIC) architecture initiative with the
Single-Chip Cloud Computer (SCC) with 48 cores on a single chip [HDH+ 10]
or the Xeon Phi coprocessor series with more than 60 cores, Picochip’s PC-200
series [DPT03] with 200–300 cores per device, Tilera’s TILEPro 32-bit processor
family with 64 Very Long Instruction Word (VLIW) processor cores [Til13], or the
Am2045 [But07], a massively parallel processor array from Ambric that contains
336 RISC processors. This trend has become even more aggressive towards having
thousands of cores on a single chip such as in Adapteva’s Epiphany processor series,
which scales theoretically up to 4,096 cores on a single chip [Gwe11].
In the area of low-power and embedded MPSoC accelerators, Coarse-Grained
Reconfigurable Arrays (CGRAs) have been proposed, as a highly efficient solution
to speed up computations. Examples of CGRAs include architectures such as NEC’s
DRP [Mot02], PACT XPP [BEM+ 03], ADRES [BBDSG08], Kalray’s MPPA-256
manycore [dDAB+ 13], or the HyperX hx3100 [IDS12] from coherent logix that is
composed of an array of 100 DSP/GPP processor cores. One representative class of
CGRAs are the so-called Tightly Coupled Processor Arrays (TCPAs) [KHKT06b,
HLB+ 14], that are well suited for speeding up loop applications with low-power
requirements. TCPAs have the ability to avoid the global memory access bottle-
neck of many MPSoCs due to local—nearest-neighbor—communication between
1 Introduction 3

processors and the capability to exploit multiple levels of parallelism including

loop-level, instruction-level, and finally also word-level parallelism. Therefore,
heterogeneity in the form of domain-specific components such as accelerators
are the key to success for future performance improvements [SSM+ 11] and the
performance gains for the next couple of processor generations are going to be
accomplished mainly by exploiting different levels of parallelism.
Above-mentioned computing systems have become an indispensable part of
our daily life, powering a range of devices from smartphones to automotive and
avionics. They are used for a mixture of applications from different levels of
mixed-criticality with possible real-time requirements. For example, automotive ap-
plications such as Anti-lock Breaking System (ABS) (control-oriented processing),
collision and pedestrian detection (data-oriented processing), and multimedia appli-
cations fall into different levels of criticality with respect to errors in computed data.
Furthermore, nowadays, energy consumptions (especially for mobile devices) or
predictable execution (e.g., avionics industry) of programs plays an important role.
Therefore, it is of high importance to consider such non-functional requirements
while mapping applications to MPSoCs.
To efficiently exploit the computational performance of these systems while
considering non-functional execution properties, Teich proposed the idea of inva-
sive computing [Tei08]. The main novelty of invasive computing is to introduce
resource-aware programming support in the sense that an application program gets
the ability to express its desire for processor cores and dynamically spreads its
computations to such processors in a phase called invasion, and to then execute code
segments with a high degree of parallelism based on the region of claimed resources.
Afterwards, once it terminates or if the degree of parallelism should be less, it
enters a retreat phase, where occupied processors, but equally also communication
and memory resources are released again, and execution is resumed sequentially.
However, such runtime adaptivity renders compilation difficult because the ac-
tual number of executing processors becomes only known at runtime. This is a
challenging task, as a just-in-time compiler on MPSoC is prohibitive due to the
restricted memory available on many MPSoC devices. Moreover, the possibility
of dynamic runtime compilation and optimized code generation might be out of
reach for reasons of unacceptable time overheads. Therefore, novel compiler support
for adaptive parallel execution of programs on processor arrays such as TCPAs is
required.
At the same time, it is of utmost importance to take counter measures against the
increasing proneness to errors of modern MPSoCs. In safety-critical environments
such as avionics and automotive, Single-Event Upset (SEU) might change the
current program behavior either temporally or even permanently. Accordingly, novel
adaptive approaches to enable fault tolerance according to environmental conditions
and/or application requirements are also needed for parallel program execution,
where faults may otherwise propagate over multiple resources.
4 1 Introduction

1.1 Goals and Contributions

The major goals of this book are to consider and propose new static compilation
techniques in the polyhedron model [Len93, FL11] for the resource-adaptive parallel
execution of the important class of nested loop programs on processor arrays. The
goal is thereby to find optimal assignments and schedules of loop iterations at
compile time for an array of processors where the number of available cores is only
known at runtime. If this could be achieved symbolically, just-in-time compilation
as well as the need of a compiler resident on the MPSoC could be avoided. For
the first time, we formally show that it is indeed possible to jointly schedule and
assign iterations to processors for a given loop nest with uniform data dependen-
cies symbolically using: (1) symbolic outer loop parallelization [THT12, TTH13,
TTH14], (2) symbolic inner loop parallelization [TWTH14], and (3) symbolic multi-
level parallelization [TWTH15, TWTH17]. Furthermore, we present on-demand
fault-tolerant loop processing [LTHT14, TWT+ 15, LTT+ 15a, WTT+ 15, LTT+ 15b,
LWT+ 16], a new flexible compile time transformation that protects loop nests to be
mapped to a processor array—rather than single processors—for parallel execution
against soft errors. The major results of this book for each domain are summarized
in the following.

1.2 Symbolic Outer and Inner Loop Parallelization

Symbolic parallelization is needed when the size and number of available processors
for parallel loop execution is not known at compile time. Still, we are able to
proof that a schedule of loop iterations can be computed and optimized statically
for: (1) symbolic outer loop parallelization or Locally Sequential Globally Parallel
(LSGP) for scenarios with constraints on the I/O capacities and unknown number
or processors, and (2) symbolic inner loop parallelization also known as Locally
Parallel Globally Sequential (LPGS) for scenarios with constraints on the local data
memory and a yet unknown number of processors at compile time. We show that it is
possible to derive such symbolic schedules by proposing a mixed compile/runtime
approach: First, the iteration space of a given loop nest is tiled symbolically by
introducing symbolic tile sizes [TTH13, TTH14]. Here, each (virtual) processor
will be assigned, for example, to all iterations belonging to a single tile for
computation. Thus, using symbolic tile sizes, we solve the processor assignment
problem symbolically and for any number and dimension of available processor
arrays.
In the second step, we present a novel approach to statically derive symbolic
LSGP [TTH13, TTH14] and LPGS [TWTH14] schedules: At compile time, we
first determine the set of all schedule candidates, each being latency-optimal for
a different scanning order of the given loop nest. Then, we devise an exact symbolic
formula for determining the latency of the resulting symbolic schedules. At runtime,
once the size of the processor array becomes known, a simple prologue selects the
1.4 On-demand Fault-tolerant Loop Processing 5

overall latency-optimal schedule that is then dynamically activated and executed on

the processor array. Hence, our approach avoids any further runtime optimization
and expensive recompilations while achieving latency-optimality without a priori
knowledge about the size of the available processor array. Moreover, it turns out that
the number of optimal schedule candidates to select from is only a hand full in size.
Computing an optimal static schedule for each possible combination of array and
problem size is therefore not needed. These results are essential for self-organizing
computing paradigms such as invasive computing, where the number of processors
available on resources such as TCPA is not known at compile time.

1.3 Symbolic Multi-level Parallelization

LSGP and LPGS are either I/O- or memory-bounded and may exceed the ca-
pabilities of the target architecture. To solve this problem, we present a hybrid
compile/runtime technique [TWTH15, TWTH17] to symbolically parallelize loop
nests with uniform dependencies on multiple levels and unknown number of
processors. By tuning the size of the tiles on multiple levels, a tradeoff between the
necessary I/O-bandwidth and memory becomes possible, which facilitates obeying
resource constraints. At compile time, two novel transformations are proposed:
symbolic hierarchical tiling followed by symbolic multi-level scheduling. In this
context, we present: (1) a formal approach to symbolically tile loop nests on multiple
levels hierarchically. A multi-level (hierarchical) tiled loop is a loop nest where
tiling is applied multiple times to create different levels of hierarchical tiles, with
each lower-level tile nested in the one above. (2) An extension of the formal
approach described in the previous paragraph to find latency-minimal symbolic
schedules for symbolically and hierarchically tiled loop nests at compile time, where
each hierarchy level can be executed either in parallel or sequentially. The resulting
schedules are symbolic with respect to the number of tiles. Thus, the number of
processing elements to map onto does not need to be known at compile time. At
runtime, again when the number of processors becomes known, a simple prologue
selects a feasible schedule with respect to I/O and memory constraints that is
latency-optimal for the chosen tile size. In this way, our approach dynamically
chooses latency-optimal and feasible schedules while avoiding approaches like just-
in-time compilation or storage of multiple program configurations for each possible
array size.

1.4 On-demand Fault-tolerant Loop Processing

Due to shrinking feature sizes today’s MPSoCs experience a higher susceptibility

to soft errors that can be caused by phenomena like cosmic radiation [AJST82].
In this work, we thus propose to make use of the already abundant number
6 1 Introduction

of processors within a TCPA to provide fault tolerance in a flexible and on-

demand way: that is, let the application programmer choose to use the resources
either for achieving fault tolerance or higher performance. Through new loop
transformations [LTHT14, TWT+ 15, LTT+ 15a, WTT+ 15, LTT+ 15b, LWT+ 16], we
show that it becomes possible to protect loop nests—mapped for parallel execution
on processor arrays—against faults. The proposed source-to-source transformations
automatically introduce fault tolerance at the loop level by replicating the loop
computations across double (Dual Modular Redundancy (DMR)) or triple (Triple
Modular Redundancy (TMR)) number of processors.
Our proposed transformations consist of: (1) loop replication that creates the
desired level of redundancy. The compiler maps the replicated loop programs to
neighboring regions of a processor array. Note that in case no fault tolerance is
required, these processors might be used, e.g., to increase the performance of the
loop application or to execute other applications concurrently instead. (2) Voting
insertion appropriately inserts voting operations into the replicated loop program
to utilize the introduced redundancy for fault tolerance. We propose three different
placement variants for voting operations to detect and correct errors, respectively:
(a) in every loop iteration (immediate voting), (b) at the border between two
processors (early voting), and (c) at the border of the allocated processor array
region (late voting). Each of the variants exhibits a different tradeoff in terms of
latency (time to finish computation) and error detection latency (time to detect a
fault).
Our proposed fault-tolerant approach is thus able to offer diverse tradeoffs
between performance and reliability without the need to change the underlying
source code of a given loop application, and furthermore, without any hardware
changes.

1.5 Book Organization

This book is structured as follows:

Chapter 2 starts with discussions on the main principles of invasive computing
along with explanations of its underlying programming model. In Section 2.2, we
subsequently present the concept and structure of TCPAs, that form the basis for our
experiments throughout the book. Section 2.3 focuses on our compiler framework
for mapping nested loop programs onto TCPAs. We also present the fundamentals
in terms of how to specify nested loop programs and the considered class of
nested loops in the polyhedron model. In addition, we discuss important well-
known classes of loop transformations such as static tiling and static scheduling,
respectively.
In Chapter 3, we show for the first time, how to formally solve the problem of
symbolic parallelization of nested loop programs on a number of processors that
is unknown at compile time. In order to avoid any overhead of dynamic (runtime)
recompilation, we proof that it is possible to map a given loop nest with uniform
1.5 Book Organization 7

data dependencies symbolically. In Section 3.1, we first show analytically that the
iteration space of a loop program can be symbolically tiled into orthotopes (hyper
rectangles) of symbolic extensions. Subsequently, our breakthrough in symbolic
scheduling is presented for (1) I/O bounded scenarios in Section 3.2 and (2) for
memory bounded scenarios in Section 3.3. By analytical means, we show that the
resulting tiled program can also be scheduled symbolically, resulting in a set of
latency-optimal symbolic schedule candidates.
Multi-level parallelization techniques are proposed in Chapter 4 in order to
balance the I/O bandwidth with the necessary memory requirements on an unknown
number of processors. Here, for the first time, we present a formal approach to
hierarchically and symbolically tile an n-dimensional loop nest on multiple levels
in Section 4.1. This is followed by an introduction of mathematical methods to
determine a set of latency-optimal symbolically schedule vectors for hierarchically
tiled loop nests in Section 4.2.
Chapter 5 presents techniques for providing on-demand replication of parallel
loops on redundant resources available on massively parallel processor array
architectures such as TCPAs. We propose different options for duplicated (DMR)
and triplicated (TMR) loop execution in Section 5.2, and introduce two new
source-to-source transformations, namely, loop replication and voting insertion,
in Section 5.2.1 and in Section 5.2.2, respectively. Subsequently, we present in
Section 5.4 how these concepts can be used for providing adaptive fault tolerance
in the context of invasive computing.
Finally, Chapter 6 concludes the book and outlines possible future work.
Chapter 2
Fundamentals and Compiler Framework

The steady miniaturization of feature sizes enables the creation of increasingly

complex Multi-Processor System-on-Chip (MPSoC) architectures, as mentioned
in Chapter 1. Programming and managing this considerable amount of available
hardware resources poses difficult challenges, because centralized approaches are
not expected to scale [MJU+ 09]. Conventional parallelization approaches do not
match the dynamic behavior of applications and the variable status and availability
of resources in large chip multiprocessors. One approach to tackle these challenges
in a holistic way is invasive computing [Tei08, THH+ 11]. In Section 2.1, we
therefore discuss the main principles of invasive computing and give an overview for
programming invasively. Subsequently, in Section 2.2, we present the generic archi-
tecture of invasive Tightly Coupled Processor Arrays (TCPAs) [HLB+ 14, Lar16],
a class of massively parallel processor arrays. Afterwards, Section 2.3 gives an
overview of the compiler framework used for compiling invasive loop nests for
execution onto accelerators such as the invasive TCPAs. Furthermore, we introduce
the needed fundamentals of the underlying models for the specification of loop nests
in Section 2.3.3.

2.1 Invasive Computing

The continuous quest for performance faces modern computer systems with various
challenges and obstacles. Those (among others) are programmability, adaptiv-
ity, scalability, physical constraints, reliability, and fault-tolerance as mentioned
in [THH+ 11]. They affect all markets and branches ranging from High-Performance
Computing (HPC), over gaming to mobile devices and even to processors in the au-
tomotive and embedded sector. Therefore, it is of utmost importance to tackle these
issues through new programming and resource managing concepts. This motivated
the introduction and investigation of a new concept of dynamic and resource-aware

© Springer International Publishing AG 2018 9

A.-P. Tanase et al., Symbolic Parallelization of Nested Loop Programs,
https://doi.org/10.1007/978-3-319-73909-0_2
10 2 Fundamentals and Compiler Framework

programming paradigm under the notion of invasive computing [Tei08, THH+ 11].
Invasive computing proposes radical changes in (1) processor architectures, (2)
system software, and (3) programming language. In [Tei08], Teich defines invasive
computing as follows:
Definition 2.1 (Invasive Programming) “Invasive Programming denotes the ca-
pability of a program running on a parallel computer to request and temporarily
claim processor, communication and memory resources in the neighbourhood of its
actual computing environment, to then execute in parallel the given program using
these claimed resources, and to be capable to subsequently free these resources
again.”
Through invasive computing, the application programmer gains the ability to
distribute the application workload based on the availability and status of the
hardware resources. Employed appropriately, this significantly improves resource
utilization, and hence efficiency, drastically. Already demonstrated benefits include
increases in speedup (with respect to statically mapped applications) as well as
increases in resource utilization, hence computational efficiency [TWOSP12]. Here,
the invasive efficiency was analyzed carefully and traded against the overhead
caused by statically mapped applications. Invasive programs can achieve similar
speedups as programs running on statically allocated resources while reaching
significantly higher resource efficiency (almost 100 % in comparison to only
32 % of the non-invasive application). Moreover, an invasive application facilitates
the exploration of runtime tradeoffs, e.g., of quality or throughput depending
on the number of available processing resources [TLHT13]. Furthermore, the
capability to claim the exclusive access to resources (such as processors, mem-
ory, and communication) allows to isolate applications and thus makes multi-
core program execution higher predictable with respect to non-functional require-
ments such as execution time, fault tolerance, or power consumption [WBB+ 16,
LWT+ 16].
iNVASIVE computing pervasively covers all major aspects of system design,
encompassing research in languages and compilers, operating and runtime systems,
as well as in software and hardware architectures. A typical invasive application
runs through the three states invade, infect, and retreat depicted in
Figure 2.1. Programming support for these constructs has been developed in the
form of the programing language InvadeX10 [HRS+ 11], which is based on X10, a
parallel programming language proposed by IBM [CGS+ 05] using the Partitioned
Global Address Space (PGAS) model. In X10, programs are running on a fixed
number of places, where a place is a shared memory partition, such as a tile of
processors shown in Figure 2.2 on the right. The invasive language primitives
are realized through a resource-aware operating system for invasive computing,
called OctoPOS [OSK+ 11]. OctoPOS shares the same view with ROS [KRZ+ 10]
as far as application-directed resource management of many-core architectures is
concerned. Similar to ROS, it uses an event-based kernel architecture and largely
utilizes asynchronous and non-blocking system calls. However, OctoPOS differs
2.1 Invasive Computing 11

start invade infect retreat Exit

Fig. 2.1 State chart of an invasive program: After requesting a set of desired resources with
invade, invasive programs infect the claimed resources, execute on them, and finally
retreat. The application may transit back to the invade state (e.g., in case of a non-maskable
error) and adapt to changing runtime conditions (e.g., with data from monitors)

in its execution model that is designed particularly to support invasive-parallel

applications. A simple invasive program written in InvadeX10 may look like this:
1 val claim = Claim.invade(constraints);
2 claim.infect(ilet);
3 claim.retreat();

By calling the invade method, the application requests a set of desired

resources including processors, memory and network connections which are ex-
pressed as a set of constraints. These are, if available, provided in return with
a claim from the runtime system, containing a handle to the set of acquired
resources. By default, they are granted for exclusive use, but may differ from request
to request. The desired resource characteristics can be specified as a logical (“and”,
“or”) combination of constraints. During infect, the application executes on the
invaded resources. The code segments to execute on a set of invaded resources are
called Invasive-let (i-let) (from invaslet, similar to servlet in the Java community)
and are expressed as a closure in InvadeX101 .
Example 2.1 Assume that an invasive application needs to compute the product of
two matrices a ∈ ZN ×K and b ∈ ZK×M , yielding matrix c ∈ ZN ×M . In InvadeX10,
this is specified as a public static method:
1 public static def matmul(
2 a:Array[int](2), b:Array[int](2), c:Array[int](2),
3 N:int, M:int, K:int) {
4 val z = new Array[int]((0..(N-1))*(0..(M-1)));
5 for(var i:int = 0; i < N; i++) {
6 for(var j:int = 0; j < M; j++) {
7 c(i,j) = 0;
8 for(var k:int = 0; k < K; k++) {
9 z(i,j) = a(i,k) * b(k,j);
10 c(i,j) = c(i,j) + z(i,j);
11 }}}};

1 infect is implemented in terms of X10 places; an i-let is represented by an activity in X10,

which is a lightweight thread.
12 2 Fundamentals and Compiler Framework

The corresponding i-let to call the matmul method may look like this:

1 @REQUIRE(Latency(10, 15, "ms", "soft"))

2 val ilet = (id:IncarnationID) => {
3 val a = new Array[int]((0..(N-1))*(0..(K-1)));
4 val b = new Array[int]((0..(K-1))*(0..(M-1)));
5 val c = new Array[int]((0..(N-1))*(0..(M-1)));
6 matmul(a, b, c, N, M, K);
7 };

In invasive computing qualities of execution (e.g., latency, throughput) may

be expressed through a high-level interface in the form of so-called requirements
[TGR+ 16, WBB+ 16]. Requirements describe non-functional characteristics of the
application and serve as basis for an automatic constraint deduction. Through
source-to-source translations, requirements are pre-compiled and transformed into
a set of constraints that will enforce the desired non-functional characteristics
that have to hold during the i-let execution. In the above example, a performance
requirement is specified, with an upper and lower soft latency bound of 10 and 15
milliseconds, respectively. The constraints representing the concrete demands of the
application may look like:

1 var constraints:Constraint = new AND();

2 constraints.add(new PEQuantity(1, 4));
3 constraints.add(new PlaceCoherent());

In the above example, the constraints express the desire to claim anything between
one to four processors residing within a single tile of processors (equivalent to the
notion of a place in X10).
The infect method may either return after a successful i-let execution or give
a feedback to the application in the form of (exceptions or) error codes, allowing
the application to react appropriately, for example by re-execution of the i-let.
Finally, after execution, the application releases the claimed resources to the runtime
system by calling the retreat method. The application retreats, for example,
when its computations finish, when it needs different resources, or when infect
fails. In summary, invade, infect, and retreat enable a fundamental idiom
of invasive computing: the adaptive feedback loop, as shown in Figure 2.1. The
application gauges its environment (e.g., radiation, temperature, data load) to invade
and infect resources accordingly, retreats, and starts from the beginning again.
In the following, before introducing the invasive TCPA compilation flow in Sec-
tion 2.3, we will give a brief overview of invasive TCPAs, a class of massively
parallel architectures used in invasive architectures [HHB+ 12] to accelerate nested
loops.
2.2 Invasive Tightly Coupled Processor Arrays 13

2.2 Invasive Tightly Coupled Processor Arrays

The invasive computing paradigm drives state-of-the-art research not only in

programing language design and compilation (as detailed above), but also in hard-
ware. In the invasive philosophy, the different invasive hardware components form
together a tile-based MPSoC. An example of such a tiled heterogeneous MPSoC is
shown schematically in Figure 2.2 on the right-hand side. In the figure, different
compute, memory, and I/O tiles are shown that are connected by an Invasive
Network-on-Chip (i-NoC) to an Invasive Network Adapter (i-NA) [HZW+ 14]. The
compute tiles can be further subdivided into tiles with Commercial Off-The-Shelf
(COTS) RISC processors, tiles with i-Cores [GBH17], that are application-specific
instruction-set processors with a configurable instruction set, and accelerator tiles
containing an invasive TCPA.
TCPAs are the perfect candidate to accelerate computationally intensive loop
programs by jointly exploiting loop-level, instruction-level as well as word-level
parallelism while achieving a better energy efficiency compared to general purpose
embedded processors [KSHT09, HLB+ 14] and offering a fully time-predictable
execution [GTHT14, GSL+ 15]. In order to support the ideas of invasive computing,
such as resource exploration and management, hardware controllers (Invasion
Controller (iCtrl) [LNHT11]) are integrated in each Processing Element (PE) of a
TCPA to enable an invasion of PEs at a latency of two cycles only. Invasive TCPAs
are much more flexible than architectures synthesized from one algorithmic (loop

IM GC AG IM

AG I/O Buffers GC

PU PU PU PU PU

iCtrl iCtrl iCtrl iCtrl iCtrl

Configuration Manager

PU PU PU PU PU
I/O Buffers

I/O Buffers

iCtrl iCtrl iCtrl iCtrl iCtrl

PU PU PU PU PU

iCtrl iCtrl iCtrl iCtrl iCtrl

PU PU PU PU PU

iCtrl iCtrl iCtrl iCtrl iCtrl

PU PU PU PU PU

iCtrl iCtrl iCtrl iCtrl iCtrl

GC I/O Buffers AG

IM AG GC IM

AHB bus

Conf. & Com.

Proc. (LEON3) IRQ Ctrl.

Fig. 2.2 On the left, an abstract architectural view of a TCPA tile is shown. The abbreviations
stand for: Address Generator (AG), Global Controller (GC), Invasion Manager (IM), and Invasive
Network Adapter (i-NA) [HLB+ 14]. On the right, a schematic representation of a tiled heteroge-
neous MPSoC is given
14 2 Fundamentals and Compiler Framework

nest) specification [Tei93] due to their programmability. The TCPA architecture

is based on a highly customizable template, hence, offering a high degree of
flexibility, in which parameters such as number of Functional Units (FUs) per PE
have to be defined at synthesis time, whereas other parameters such as topology
of interconnect may be reconfigured at runtime. A TCPA architecture consists of
an array of programmable PEs (see Figure 2.2 on the left) each having a Very
Long Instruction Word (VLIW) structure, with local interconnections as proposed
first in [KHKT06a]. Apart from the computational core, an invasive TCPA tile also
encompasses other peripheral components such as I/O buffers as well as several
control, configuration, and communication companions.
Before describing these building blocks of a TCPA in Sections 2.2.1 to 2.2.3,
it should be mentioned that TCPAs can be integrated also into more traditional
System-on-Chip (SoC) designs, for instance, using a bus-based Central Processing
Unit (CPU) coupling architecture, shared registers, or a shared data cache.

2.2.1 Processor Array

The computational core of a TCPA is comprised of an array of tightly coupled VLIW

PEs arranged in a 1-dimensional or 2-dimensional grid. Before synthesis, the rows
and columns, defining the total number of PEs of an array can be specified.
A PE itself is a weakly-programmable [KHKT06a] (FUs have a reduced instruc-
tion set) and a highly parameterizable component with a VLIW structure (see outline
in Figure 2.3). Here, different types and numbers of FUs (e. g., adders, multipliers,
shifters, logical operations) can be instantiated as separate FUs, which can work
in parallel. For highest computational and energy efficiency, only a very limited
amount of instruction memory and a small register file is available in each PE, and
the control overhead is kept as small as possible. Furthermore, PEs have no direct
access to the main memory, but data is streamed from surrounding buffers through
the array PE by PE. The control path is kept very simple (no interrupt handling,
multi-threading, instruction caching, etc.), and only single cycle instructions and
integer arithmetic are considered. Each PE operates on two types of signals, i. e.,
data signals whose width can be defined at synthesis time and control signals which
are normally one-bit signals and used to control the program execution in a PE. The
register file transparently comprises of four different types of registers for the data
as well as the control path. The first type are general purpose registers named RDx
in case of data and RCx 2 in case of control bits, respectively. The second and third
type are input and output registers (IDx, ODx for data and ICx, OCx for control bits,
respectively), which are the only ports to communicate with neighboring processing
elements. The last type of registers are feedback shift registers (FDx or FCx) that
can be used as internal buffers for cyclic data reuse purposes (e. g., for efficient

2 For the sake of better visibility, control registers and control I/O ports are not shown in Figure 2.3.
2.2 Invasive Tightly Coupled Processor Arrays 15

General Purpose
Input Ports ID0 ID1 and
Feedback Registers

Read Ports
RD0
RD1
MUX
MUX MUX
MUX RD2
Instruction
Memory RD3
RD4
FU
FU
FD0
FD1

Write Ports
DEMUX
DEMUX FD2
f1 f0 FD3
PC regFlags FD4
Instruction
BUnit OD0 OD1
Decoder

Fig. 2.3 The internal structure of a processing element consisting of different FUs formed in a
VLIW structure, a register file having different types of registers, i. e., RD, ID, and OD (figure
adapted from [KHKT06a])

handling of loop-carried data dependencies or modulo repetitive constant tables).

The transparent usage of the different register types is illustrated by the following
3-address assembly code (instr dest, operand1, operand2) snippet,
which consists of two VLIW instructions in case of a two FU PE configuration:
1 ADD RD0, ID0, RD1 MULI OD0, ID1, #2
2 ADDI RD2, RD0, #1 MUL OD1, RD0, RD1

Noteworthy is the instantiation possibility of a multiway branch unit that is able to

evaluate multiple control bits and flags in parallel in order to keep the time overhead
for housekeeping (i. e., control flow code) minimal. Note that an n-way branch unit
leads to 2n branch targets, however, in practice, the branch unit is most often realized
as a two- or three-way, and thus affordable.

2.2.2 Array Interconnect

The PEs in a TCPA are interconnected by a circuit-switched mesh-like interconnect,

which allows data produced in one PE to be used already in the next clock cycle
by a neighboring PE. An interconnect wrapper encapsulates each PE and is used to
describe and parameterize the capabilities of switching in the network. The wrappers
are arranged in a grid fashion and may be customized at compile time to have multi-
ple input/output ports in the four directions, i. e., north, east, south, and west. Using
16 2 Fundamentals and Compiler Framework

these wrappers, different topologies between the PEs like grid and other systolic
topologies, but also (higher dimensional) topologies such as torus or 4-dimensional
hypercube can be implemented and changed dynamically [KHKT06a, KHKT06b].
To define all possible interconnect topologies, an adjacency matrix is provided for
each interconnect wrapper in the array at compile time. Each matrix defines how the
input ports of its corresponding wrapper and the output ports of the encapsulated PE
are connected to the wrapper output ports and the PE input ports, respectively.
Two different networks, one for data and one for control signals, can be defined
by their data width and number of dedicated channels in each direction. For instance,
two 16-bit channels and one 1-bit channel might be chosen as data and control
network, respectively.

2.2.3 TCPA Peripherals

TCPAs also consist of several peripheral components that together with the multi-
way branch unit enable zero-overhead loop execution. Among these, a Global
Controller (GC) generates synchronized control signals that govern the control
flow of the PEs, removing control overhead from the loops; Address Generators
(AG) produce the necessary addresses for feeding the PEs with data to/from
reconfigurable buffers, removing addressing overhead.
Global Controller: Numerous control flow decisions such as incrementing the
iteration variables, loop bound checking, and performing other static control flow
operations may cause a huge overhead compared to the actual data flow. However,
thanks to the regularity of the considered loop programs and since most of this static
information is needed in all PEs that are involved in the computation of one loop
program (in a linearly staggered fashion), the control flow can be moved out of the
PEs and compute it in one GC per loop program. The GC generates branch control
signals, which are propagated in a delayed fashion over the control network to the
PEs, where it is combined with the local control flow (program execution). This
orchestration enables the execution of nested loop programs with zero-overhead
loop, not only for innermost loops, but also for all static conditions in arbitrary
multidimensional data flow.
Address Generators and I/O Buffers: Data locality is a key factor for the acceler-
ation of loop programs on processor arrays. An Address Generator (AG) produces
the necessary addresses for feeding the PEs with data to/from reconfigurable buffers,
removing addressing overhead. As the processing elements are tightly coupled, they
do not have direct access to a global memory. Data transfers to and from the array
are performed through the border PEs, which are connected to a set of surrounding
reconfigurable buffers. These buffers can be configured, to either work as simple
FIFOs or as RAM-based addressable memory banks [HSL+ 13]. AGs generate the
correct sequence of read/write accesses according to a given loop schedule. AGs
work in parallel with the main computational units or processors to ensure efficient
storage/feed of data to/from the main memory.
2.2 Invasive Tightly Coupled Processor Arrays 17

Configuration and Communication Processor: The admission of an application on

the processor array, communication with the network via the Invasive Network
Adapter (i-NA) [HZW+ 14], and processor array reconfiguration is managed by a
companion RISC processor (Leon3) that is named Configuration and Communica-
tion Processor. That means that on the one hand, the companion handles resource
requests and on the other hand, initiates appropriate DMA transfers via the i-NA to
fill and flush the I/O buffers around the array.
Invasion Manager and Invasion Controller: Invasion Managers (IMs) handle
invasion requests to the TCPA and keep track of availability of processor regions for
placing new applications within the array. Such resource management is facilitated
in each PE by a iCtrl unit, giving it the capability to acquire, reserve, and then
release the PEs in its neighborhood. Here, each IM keeps track of the availability
of its connected iCtrl. When the Configuration and Communication Processor
receives an invasion request, it chooses the best initiation point by checking the
status of the IMs. After placing an invasion request on an IM, it forwards the
request to the invasion controller connected to it, and listens for the result of
invasion. Then, when the results are ready, the IM informs the Configuration and
Communication Processor about the readiness of the results by means of an interrupt
request.
Configuration Manager: The configuration manager consists of two parts, a mem-
ory to store the configuration streams and a configuration loader. It holds configura-
tion streams for the different TCPA components such as global controller, address
generator, and of course for the processor array itself (assembly codes to be loaded
to the PEs). Since TCPAs are coarse-grained reconfigurable architectures, the size
of their configuration streams is normally a few hundred bytes, which allows for
ultra fast context switches in the system. The configuration loader is responsible
for transferring a configuration stream to the PEs via a shared bus. It is possible
to group a set of PEs in a rectangular region to be configured simultaneously if
they receive the same configuration stream, which reduces the configuration time
significantly.
In conclusion, invasive TCPAs achieve a high energy efficiency, mainly due to a
much better resource utilization, data locality, and cheaper FUs (integer arithmetic).
Even more, TCPAs support multiple levels of parallelism and are therefore well
suited as accelerators in an MPSoC to speed up computationally expensive loop
algorithms stemming from, e. g., digital media and signal processing.
Yet, one of the most important challenges in using such programmable ac-
celerators today is the lack of mapping tools or compiler frameworks that can
increase the productivity of the programmer. Here, only a compiler can make such
architectures feasible. But, can loop nests be automatically mapped on massively
parallel architectures? In the next section, we give an affirmative answer by
introducing LoopInvader: A compiler for invasive massively parallel processor
arrays [TWS+ 16].
18 2 Fundamentals and Compiler Framework

2.3 Compiler Framework

As explained in Chapter 1, heterogeneous systems including power-efficient hard-

ware accelerators are dominating the design of nowadays and future embedded
computer architectures—as a requirement for energy-efficient system design. In this
context, in the previous sections, we discussed software and hardware solutions to
efficiently manage and dynamically exploit the availability and state of resources
in such systems, in the form of invasive computing (see Section 2.1) and invasive
TCPAs (see Section 2.2), respectively. However, for the efficient utilization of
an invasive TCPA through the concrete invasive language InvadeX10, compiler
support is paramount. Without such support, programming that leverages the
abundant parallelism in such architectures is very difficult, tedious, and error-prone.
Unfortunately, even nowadays, there is a lack of compiler frameworks for generating
efficient parallel code for massively parallel architectures. In this section, we
therefore present LoopInvader, the first compiler for mapping nested loop programs
onto invasive TCPAs. We furthermore discuss the fundamentals and background of
the underlying models for algorithm and application specification.

2.3.1 Compilation Flow

Compiler development has been for decades at the heart of computer science, and
has remained an extremely active field of research. The structure of the compiler
framework developed for the invasive programing language InvadeX10 is shown in
Figure 2.4. The front end of the compiler is based on an existing X10 compiler, but

LoopInvader
Source
Front end Code
Loop Extractor

SAC Representation
Extended
PAULA Code
X10 compiler
PARO
High-Level Transformations
AST

Space-Time Mapping
Allocation
Scheduling
LoopInvader libFirm
Resource &
Regster Binding

Machine code
Back End
Code & Interconnect gen.

GC (Configuration data)
Tightly coupled other
AG (Configuration data)
processor array
SPARC ... backend

Fig. 2.4 Compiler framework for invasive computing with the two compilation branches: the left
one, for TCPAs is shown in blue and the right one, for SPARC and i-Core is depicted in orange
2.3 Compiler Framework 19

has been extended by new transformation phases [BBMZ12, TWS+ 16] to support
TCPAs as well as Scalable Processor Architecture (SPARC) processors and i-
Cores through libFIRM [LBBG05, Lin06]. The compiler transforms X10 programs
with all modern language features such as closures, generic code, parallelization,
and synchronization through async, at, and finish. Figure 2.4 depicts the two
main compilation branches: the right one for SPARC and i-Cores, and the left one
for TCPAs, respectively.
The programs analyzed by the right branch are transformed into the libFIRM
intermediate representation to facilitate the generation of SPARC code and for sup-
porting the i-Core extensions. libFIRM is a graph-based intermediate representation
for compilers. In particular, libFIRM is based on the Static Single Assignment
(SSA) form [RWZ88], and uses bottom-up graph matching on SSA graphs for
code generation. The libFIRM representation is directly created from the X10’s
Abstract Syntax Tree (AST) through an SSA construction algorithm [BBH+ 13] that
directly constructs an intermediate representation in SSA form. Finally, libFIRM
infrastructure is then used to generate SPARC and i-Core machine code. The
detailed explanation of this compilation branch is out of the context of this work,
therefore, we refer to [BBMZ12] for more details.
This work concentrates rather on the loop compilation branch and TCPAs as
the target architecture (see Figure 2.4). Loop programs play an important role in
parallel computing in general and invasive computing in particular. For exploiting
invasive computing concepts at the level of loop programs and massively parallel
architectures such as TCPAs, the tool LoopInvader is employed.
Example 2.2 For instance, assume the following scenario where the matrix multi-
plication example introduced in Example 2.1 needs to be accelerated on a TCPA.
1 public static def matmul(
2 a:Array[int](2), b:Array[int](2), c:Array[int](2),
3 N:int, M:int, K:int) {
4 val z = new Array[int]((0..(N-1))*(0..(M-1)));
5 for(var i:int = 0; i < N; i++) {
6 for(var j:int = 0; j < M; j++) {
7 c(i,j) = 0;
8 for(var k:int = 0; k < K; k++) {
9 z(i,j) = a(i,k) * b(k,j);
10 c(i,j) = c(i,j) + z(i,j);
11 }}}};

The following i-let can be used to call the matrix multiplication:

1 val ilet = (id:TCPAId) => @TCPA(
2 /* compilation parameters */) {
3 val a = new Array[int]((0..(N-1))*(0..(K-1)));
4 val b = new Array[int]((0..(K-1))*(0..(M-1)));
5 val c = new Array[int]((0..(N-1))*(0..(M-1)));
6 matmul(a, b, c, N, M, K);
7 };
Exploring the Variety of Random
Documents with Different Content
left us. Accordingly, we kept our course until the extremity of this island, which
runs from north-west to south-east. We went to look for a port for the ships in
case it should be needed; and we found at the point of this promontory many
islets with shoals between them. Among them is a large island with a good port.
We were in want of water, and two canoes that accompanied us showed us where
to get it, with the intention of luring us there and killing us; for they came with
their weapons. They were joined by 30 other canoes, one of them carrying 30
Indian warriors. Arriving whilst we were watering, they landed, and having got
plenty of stones and arrows and spears, some went to attack the brigantine,
whilst the others went to attack those who were getting water on shore. When we
saw their determined daring, shots were fired by which some were killed and
many wounded; and so they fled, leaving behind two canoes empty, and carrying
off the rest. The large canoe was much injured, and in their precipitation they
threw themselves into the sea; but we took the canoe with four Indians, two
wounded and two unharmed. We landed them, and treating them well, gave them
their liberty and restored their canoe. And so they went away; and I kept a boy
that I took here. I found the latitude to be in 103⁄4°. On the south-south-east side
of the point, the coast trends from north-east to south-west, but from this point
we could not see the end of it. The port is 40 leagues from where we left the
ships.[261]
[261] The description of this part, its situation, and relative position to the adjoining
coasts of Malaita and St. Christoval, as stated below, all point to its identity with Marau
Sound. In the Geographical Appendix reference is made to the discrepancies in the
distances and latitudes of Gallego.

“We left this port with some difficulty as it lies among the reefs. We saw to the
south-east-by-east an island 7 leagues away;[262] but we did not go to it, as we
were going to the island of Malaita, as the Indians name it, which lies with the
island of Guadalcanal, and with the point where we had been, north-east-by-east.
We sailed to the north-east-by-east for 16 leagues, and arrived at a good harbour
which has many reefs at the entrance. There came out 25 canoes with warriors
who discharged their arrows. Some shots were fired at them, which killed some
and wounded others. This port, which is on the south-south-west coast, is in the
latitude of 101⁄4°; and the name, Escondido, was given to it, because it is almost
enclosed by reefs.[263] In this island we found apples of some size, oranges, a
metal that seemed to be a base kind of gold, and, besides, pearl-shell, with which
they inlay the club they use in battle, being the one they usually carry. These
natives, like the rest, go completely naked. In the name of His Majesty we took
possession of this island, to which we gave the name of the Isle of Ramos.” (Vide
Note VI., Geographical Appendix.)
[262] This island is evidently St. Christoval.
 [263] Future visitors to the southern portion of Malaita will doubtless be able to identify
this port with some anchorage on the west coast to the northward of the Maramasiki
Passage. In so doing they should not forget the usual error of Gallego’s latitudes (Note V.
of the Geographical Appendix).

“Leaving this port, we sailed to the south-east for four leagues, and discovered an
entrance to a harbour resembling a river dividing the lands from each other.[264]
We could not see the end of it; and on account of the strong current we were
unable to enter. We accordingly passed on another four leagues, where we found
a good port: and in it I took the latitude, and found it to be 101⁄3° south of the
equinoctial. It has an islet at the entrance which should be left close on the
starboard hand in entering the port. Two hundred Indians came out and attacked
us. To this port we gave the name of La Asuncion, because we entered it on that
day.[265] This day we sailed out and proceeded further along the coast to the
south-east. Close to the extremity of the island, we put into a small bay,[266]
where they discharged some arrows at us, and on our firing some shots they left
us. Quitting the small bay, we sailed as far as the end of the island which is in
101⁄4°.[267] It lies north-east and south-east with the isle of Jesus, which is the
first island we saw, and lies in 7°. [With the other end of Malaita, which is to the
north-east, and lies east and west with Meta in 8°, it is 85 leagues. There is
another point in 7°, with which the Isle of Jesus lies north-east-by-north 135
leagues.[268]]
[264] This is without a doubt the Maramasiki Passage which cuts through the south-
eastern portion of Malaita.
[265] Port Asuncion may, perhaps, be the large bay of Su Paina.

[266] Caleta in the Spanish. This anchorage may, perhaps, be identified with Su Oroha or
with one of the inlets or coves nearer to Cape Zélée, such as Te Oroha or Te Waina.
(“Pacific Islands:” vol. I.; “Western Groups:” p. 61, 62; “Admiralty publication,” 1885.)
[267] This latitude is not consistent with that given above for the port of Escondido,
which, according to the journal, lies more than half a degree to the north-west.
[268] I have endeavoured unsuccessfully to get at the meaning of the two sentences
enclosed in brackets.

“This island of Malaita has a length of 114 leagues. We did not go to the north
side, and for that reason we cannot say what is its breadth. The island of
Guadalcanal is very large. I do not estimate its size, because it is a great land and
half a year is needed to sail along its shores.[269] That we sailed along its length
on the north side for 130 leagues and did not reach the end, shows its great size.
Moreover, on the east[270] side of the extremity, the coast trended to the west,
where I saw a great number of fine towns.[271]
[269] “Para andallæ es menester medio anno.”

[270] This should be “west.”

 [271] See Note VIII. in Geographical Appendix for remarks on the exaggerated ideas as
to the size of this island.

“From the extremity of this island of Malaita we saw another island, which lies
east and west from this cape 8 leagues, to which we went, arriving in the night.
We anchored in front of a town on the coast, which has a small river; and whilst
we were anchoring, two canoes came off to see us, but they soon returned. At
dawn we sent the people on shore to get water: and the natives came out
peacefully with their women and their sons. They are all naked like the others.
The women carry in their hands some things like fans, which they sometimes
place before them. When the water was procured, we asked for a hog, and they
brought it; and placing it so that we should see it, they returned and carried it off.
But we did not injure them in any way; and accordingly embarked and proceeded
out to sail round the island. When the natives saw that we were going, most of
them came out in their canoes with their bows and arrows in pursuit of us. The
first man who was about to aim, we knocked over with a shot. At this, they turned
and fled; and we pursued them as far as the port, capturing some canoes that
had intended to take us. A friendly Indian, whom we carried with us, climbed a
palm tree and saw how the Indians came in regular bodies bearing their shields.
We went to arms, and sent three soldiers to see in what force the people were.
They came in their canoes in two or three divisions to attack the brigantine: and
we began to bring our musketry into action, killing two Indians and an Indian
woman. They soon retired; and our men who were on the shore having embarked
in the brigantine, we went on in pursuit of our quest. The island is named
Uraba[272] in the language of the Indians. We gave it the name of La Treguada
because they led us into a treacherous truce.[273] This island is in latitude 101⁄2°.
It is well peopled, and has plenty of provisions of their kind. Although small, it has
an area of 25 leagues. There is communication with the neighbouring islands, and
with a cape that lies to the north-west. It trends north-west and south-east until
the middle of the island, where we found these 10°, and the other . . . . . (milad)
trends north-north-west until the end of the island.
[272] The reader will have already inferred that the island of Uraba is the Ulaua of the
present chart, and will have noticed that the name of the island has remained the same
during the last three centuries. It is the Ulawa of the present natives, and the
Contrarieté of Surville.
[273] One must judge Gallego in the spirit of his times. Humane as he really was, we
cannot free him from his share in this unfortunate conflict with the natives of Ulaua: and
the name of La Treguada had been better never bestowed. The next navigator who
visited this island was Surville in 1769; who, following up his previous proceedings at
Port Praslin in Isabel, repelled its inhabitants with grape shot.

“To the south-by-west of the point of the island there are low islands, with many
shoals around them, which are three leagues distant from this island of La
Treguada, to which we went and obtained water. They are inhabited; and we gave
them the name of Las Tres Marias. They trend west-by-north and east-by-south.
[274]

[274] These three islands are without doubt identical with the three small islands which
are named the Three Sisters in the present chart. Surville, the French navigator, who
saw them in 1769, gave them the name of Les Trois Sœurs, which they still retain. At the
present day they are uninhabited, and any water that could be obtained would be of a
very doubtful quality. Fleurieu hints at the identity of Les Trois Sœurs and Las Tres
Marias.

“There is another island which lies three leagues from Las Tres Marias. It is low,
and the inhabitants are like those around. We named it the island of San Juan,
and found in it a good harbour. We took possession of it in the name of His
Majesty, as in the case of the other islands. It is 6 leagues in circuit; and is in
latitude 102⁄3°.[275]
[275] The San Juan of Gallego is evidently the island now known as Ugi. There is no
apparent reference in this journal to the small adjacent island of Biu.

“We went thence to another great island,[276] which lies north and south with it, 2
leagues away. Before we arrived, 93 canoes with warriors came out to us and
. . . . .[277] We took an Indian chief and placed him below the deck. He seized a
sword, and defending himself attempted to escape, until at last the sword was
taken from him and he was bound. We sent the people on shore, intending to
take possession; but so many natives attacked them that we were not able to do
so, and we returned to the island of San Juan. I offered to Don Fernando to take
possession of it before dawn; and it was done. In the island of San Juan, they
ransomed the Indian, and gave us for him three hogs, to which he added some
beads. As a sign of friendliness, Don Fernando Henriquez embraced him.
[276] Apparently this is the island named Santiago below. It is without doubt St.
Christoval.
[277] “y tuvimos gran guasavara.”

“On the following day, which was the 2nd of June, we arrived at dawn off the
island of Santiago.[278] More than 50 canoes came out to us; and they planned to
carry us off to their towns. It was necessary to fire some shots in order that they
should quit us; and they left us and returned. Possession was taken of this island
in the name of His Majesty; and we did no injury to the people. This island is 40
leagues in length on its north side: and it is narrow, and in part mountainous, and
is well peopled. The Indians of this island go naked and eat human flesh. Its
eastern extremity is in latitude 103⁄4°; and lies north-west and south-east with the
island of Treguada 12 leagues. The south-east extremity lies north-west and
south-east 18 leagues with the island of Malaita.
[278] The reader will now require to use some caution in following this part of the
narrative, since Gallego seems to have fallen into much confusion respecting the island
 of St. Christoval. The name of Santiago was evidently applied by him to the north side of
the island west of the prominent headlong of Cape Keibeck, which he might easily have
taken for the extremity of the island. The name of San Urban was in all probability given
to the peninsula of Cape Surville, which, as I have myself remarked while off the St.
Christoval coast, has the appearance of a detached island when first seen, in
approaching it from the northward and westward. This deceptive appearance, when
viewed from a distance, is due to the circumstance that the neck of the peninsula of
Cape Surville is raised but a few feet above the level of the sea, and is in consequence
below the horizon when this cape is first sighted. The distance of San Urban from
Guadalcanal, as given above, is inconsistent with the rest of the journal; and for 4
leagues, 40 leagues was evidently intended, the omission of the cipher being probably a
clerical error. The name of St. Christoval was subsequently given, as shown further on in
the narrative, when the Spanish ships visited the south coast of this island.

“When we were all embarked to proceed further on, a violent north-east wind
overtook us, and drove us to the extremity of Santiago, whence we saw a large
island to the south-east that trended westward. It was 18 leagues distant. It is in
latitude 101⁄2° south of the equinoctial; and is 4 leagues distant from the island of
Guadalcanal. We gave it the name of the island of San Urban.
“On account of the sickness of myself and of some of the soldiers, we did not
proceed further: and, keeping away to leeward, we arrived at the island of
Guadalcanal. We landed at a town where the Indians gave us . . . . . .[279] when
we intended to get water, and where we set free the three Indians in the canoe;
and they gave us a hog and panales. But they were in great fear of us, and
leaving us they returned to the town. Beads were given to them as a sign of
friendship. Leaving there, we continued our cruise to return to the ships, and
touched at some places where we had been before, the natives receiving us in a
friendly manner, and giving us what they had, because they were much afraid of
the muskets we carried. We sailed further on to a port, where, during our
previous stay, we had been received peacefully. We got water there; and they
gave us a hog and almost filled the brigantine with panoes, which is the food they
eat. It is a very good harbour for the ships, and lies under the shelter of an island.
There are many inhabitants.
[279] “La Guacanara.”

“We continued our return cruise, intending to explore a river where we had been
before. Sailing into the port to obtain provisions, we arrived close off a town
which the Indians abandoned when they saw us. We found there, many panoes
and ñames (yams) with which we loaded the brigantine. I tried to catch a tame
white parrot, which the Indians had together with many others of various hues.
When the Indians saw that we did no harm, they all assembled, and came and
gave us a hog to induce us to go. Presently we sailed on to another river, on the
bank of which there is a large town; we anchored in it. The Indians began to
make fires, and to cast the fire in the air;[280] it was a thing we had not seen in
any other part.
[280] “hechar por lo alto.”

“On the next day, which was the 6th of June, the Feast of the Holy Ghost, we
reached the ships, and found them all very sad. It appeared that on the Day of
the Ascension, the steward with four soldiers and five negroes were sent on shore
for water. As on previous occasions, they were sent because the cacique of that
tribe was a friend and used to come off to the ships to give us cocoa-nuts, whilst
his men used to fetch the water in the earthen jars, and because we trusted them
for the friendly manner in which they behaved in their dealings with us. This day,
however, when they were gone for the water, it seemed that the boat got aground
because they had not taken care to keep her afloat as she was being filled. At this
moment, the Indians rushed out from ambush with their weapons and were upon
them; and they did not leave a single soul alive except a negro of mine who
escaped. All the rest they hewed to pieces, cutting off their heads, and arms, and
legs, tearing out their tongues, and supping up their brains[281] with great ferocity.
The negro who escaped took to the water to swim off to an islet that was near.
However, they swam in pursuit, and with a cutlass, which he carried in his hand,
he defended himself from them in such a manner that they left him, and he
reached the islet. From there he began to make signs, and to shout out to those
in the ships, which they perceived; and as quickly as possible the General went
ashore to see what had happened. When he reached there, the ill tidings were
told. The Indians retired to the hills. In a short time, the dead Christians were
recovered; and they buried them in the place where they used to say mass, the
soldiers in one grave, and the negroes in another. Of the negroes, one belonged
to the King, two to ourselves, and one to the boatswain. It was a thing to hear
their shouting, and the noise that the Indians made with their drums. It appeared
to be a general assembling day with them, because more than 40,000 Indians[282]
had gathered together for this purpose. When our people had buried the dead,
they embarked in the ships, being in great grief with what had occurred.
[281] The New Ireland cannibals of the present day are fond of a composition of sago,
cocoa-nut, and human brains. (“The Western Pacific and New Guinea.” London 1886: p.
58: by H. H. Romilly.)
[282] This is either an exaggerated statement, or it is an error in transcribing.

“As I understand, the cause of the Indians coming to attack us was this. The
cacique came off to the ‘Capitana’ to entreat that our people would give him back
a boy belonging to his tribe, whom they had taken. He offered a hog for him; but
they would not give him up. On the following day, the cacique brought a hog off
to the ship, and said that, if they gave him the boy who was a kinsman of his, he
would give them the hog. But they would not give him up, and took the hog by
force. When the cacique saw how he had been treated, he went away and did not
return to the ships again. In a few days, the disaster happened.
“On the day after this unfortunate event, the General ordered Pedro Sarmiento to
land with as many men as he could muster to inflict punishment. He burned many
towns, and killed more than 20 Indians. Then he returned to give account of what
he had done. Each day that they landed they endeavoured to punish them the
more. On a subsequent occasion, because no more Indians were seen whom they
could punish, the General ordered Pedro Sarmiento to proceed to a point that lay
to the south-east a league and a half from the ships. For he considered that all
the Indians had been concerned in the treachery and in the death of the
Christians. Having embarked 50 soldiers in two boats, Pedro Sarmiento went
there, but he found no Indians as they had fled to the hills. After he had burned
all the buildings and habitations that he could find, he turned back on his way to
the ships. Some Indians, who came out from a point, followed him slowly; and
our people lay in ambush and killed three or four Indians, the rest escaping in
flight. They then returned to the boats, and embarking came back to the ships. An
Indian, whom we took, informed us of those who were concerned in the death of
our men. He said that the leader was a taurique, named Nobolo, who lived on the
bank of the river that lay a league to the east of the Rio Gallego; and that with
him there were many others who had collected together for that object and with
the said result.
“On Wednesday, the 9th of June, the men of the ‘Almiranta’ were engaged in
making a top-mast on the islet close to where the ships were anchored. Some
musketeers and targeteers (rodeleros), who were eight in number, were in guard
of the carpenter’s party. As it happened, the Indians were then preparing for
another attack; and more than 300 of them lay in ambush, ready for the assault.
About 10 Indians crossed over to the islet with bows and arrows concealed; and
they brought a hog, intending to beguile our men by occupying their attention in
talking, whilst the other Indian warriors should be arriving. When I saw the
Indians crossing over and this canoe heading for the islet where our people were
making the top-mast, I ordered some musketeers into the boat; and accompanied
by Pedro Sarmiento, we steered so that the islet concealed us from those in the
canoe. Approaching the islet, we passed between it and the main island and came
close up with the canoe which had only one Indian on board, the others having
thrown themselves into the sea. The canoe was captured together with the hog
which they had brought to deceive us. When we had joined the party who were
making the top-mast, we returned to the ships after having killed those who came
in the canoe. This was the most effective attack that was made, for the Indians
went away much discouraged.
“On the 12th of the same month of June, the General took the brigantine and a
boat with nearly all the people, in order to inflict further punishment at a river
which lay a league to the east of the place where the ships were anchored; and I
accompanied him. An hour before the dawn we arrived close to the river; and we
were about to conceal ourselves and fall upon the Indians, when we were seen by
their sentinels and they went to arms. I remained with four musketeers in charge
of the brigantine and the boat in the mouth of the river, so as not to allow any
canoe to escape. The General on arriving at the town, which had more than 200
houses, found it deserted. He set fire to it; and then we returned to the ships.
“The next day, which was Sunday the 13th of June, we made sail during the night
and proceeded in the ships to follow up the discoveries of the brigantine. When
we had sailed about 8 leagues to the south-east, we anchored because the wind
was contrary. The General landed here to get some provisions for the sick, of
whom there were many. In a short time he returned to the ships, when we made
sail with the land-breeze. Now died the pilot, Paladin, an experienced seaman. We
lost sight of the brigantine, as she went ahead of us: and we did not see her until
we found her anchored in a port off an islet that lay half a league to windward of
where we had anchored in the brigantine during our voyage of discovery. There
were many inhabitants here; and they came off to us as friends. On account of it
being Corpus Christi Day, we remained here all the day. Mass was said at the islet
which is close to the anchorage. We watered the ships there. The Indians gave us
of their own free will two hogs and many cocoa-nuts and ñames (yams). The
cacique of this tribe was named Meso, and the town was called Urare. This people
is at war with the people of Feday, which is the name of the place where we were
anchored. . . . . .[283]
[283] “que nos maron gente.”

“On the 18th of June, we left this port, and proceeded on our voyage, seeking the
island of Santiago or San Juan,[284] which was the island that we had discovered
and named. We beat to windward against a strong head wind in our endeavour to
arrive at the island of Santiago; but on account of this contrary wind and the
boisterous weather, we did not fetch it; and I determined to steer to the south of
the island of Santiago, the wind and the contrary currents not allowing us to find
a harbour. We coasted along an island, not seen in the brigantine[285], and we
held on our course for fourteen days, endeavouring to reach the end of the island;
but in the middle of the island, on account of the contrary wind and currents,
what we gained one day we lost the next. Accordingly I went to find a port. We
named this island San Christoval.[286] It was our Lord’s pleasure that after so
much difficulty I should find a very good port for the ships; and on the following
day I returned to the ships. We sailed to windward that night on account of the
boisterous weather, which obliged us to shorten sail and lie-to[287] for the night.
When it dawned, we found ourselves three leagues to leeward of the port, which
we tried in vain to reach; and since we kept falling to leeward, I was compelled to
take the brigantine and go in search of another anchorage, with the
understanding that when I had found one, I should signal to the ships to follow
the brigantine. The signal being made, I guided the ships to the brigantine, which
lay outside a point of reefs that formed the harbour; and so we entered it.
[284] Gallego here seems to have forgotten that he had previously applied these two
names to different islands, that of San Juan to Ugi and that of Santiago to the large
island south of it, viz., the present St. Christoval (see p. 222).
[285] This remark is inconsistent with the previous reference to their steering south of
Santiago.]
[286] I should here call attention to the circumstance that the Spaniards were navigating
the south coast of this island. Further proof of this is given in succeeding pages.
[287] “Sin velas de mar a el traves.”

“It is a good and secure anchorage; and there is a town there which has eighty
houses. The General landed with the captains and the soldiers to obtain provisions
and to take possession of the island, in the name of His Majesty, which we did
without opposition, for the Indians received us peacefully. The same evening we
landed, and went in marching order to see the town, but without doing them any
injury; and we returned to the ships with the agreement that on the following
morning we should revisit the town to get provisions, of which we were in need.
“On the morning of the 1st of July, we all landed with the determination to obtain
provisions for our present necessities; and the General entered one part of the
town with the greater number of our people, whilst Pedro Sarmiento with twelve
soldiers entered another part. When the Indians saw our determination, and that
we entered the town in two places, they began to arouse themselves and to take
up their weapons, making signs that we should embark. They held a consultation
in a small hollow, where Pedro Sarmiento and his party entered. One of the
headmen was seen to make incantations and invocations to the devil, which
caused real terror, because it seemed as though his body was possessed of a
devil. There were two other Indians, who, whilst making great contortions with
their faces and violently shaking themselves, scraped up the sand with their feet
and hands and threw it into the air. They then made towards the boats with loud
shouting and yells of rage, and tossed the water in the air. At this, our people
sounded the trumpets to assemble where the General was; for there were all the
Indians with their bows and arrows and darts and clubs, which are the weapons
with which they fight. They came very close to us, bending their bows and bidding
us to depart. It became necessary for us to fire; and accordingly some were killed
and others were wounded. Thereupon they fled and abandoned the town, in
which there was a great quantity of panaes and ñames and many cocoa-nuts and
almonds,[288] which were sufficient to load a ship. Presently we set about carrying
to the boats all that we found, and nothing more was done that day. The Indians
did not dare to return to the town again, and that night we embarked. This port is
in 11° south latitude. It is in close proximity to the island of Santiago, to the
south-east; it is narrow and mountainous, and the inhabitants are like the rest.
[289]

[288] These almonds were without doubt the almond-like kernels of the fruit of a species
of Canarium, a common article of food at the present day.
[289] This sentence refers to the island, and not to the port, judging from the context.

“After three days had passed, the General ordered that the brigantine should
proceed on a voyage of discovery; and Francisco Muñoz Rico, with ten soldiers,
and I, with thirteen seamen, embarked. We left this port on the 4th of July, and
coasted along this island of Paubro, as it is called in the language of the natives,
being that which we named San Christobal.[290] Until the middle of the island, the
coast trends north-west and south-east for 20 leagues and a point nearer east
and west; and the other half trends west-by-north and south-by-east. We entered
a harbour, which was the first we discovered in this cruise; and there we remained
for the day.
[290] This reference to the native name of Paubro is interesting, since at the present day
St. Christoval is largely known by the native name of Bauro, which is evidently the same.
This is also without a doubt the “large country named Pouro” of which the natives of
Taumaco (Duff Group) informed Quiros about forty years afterwards (vide Geographical
Appendix, Note XV.).

“On the following morning we left there, and proceeded further along the coast to
the east-by-south. We entered a small bay, enclosed by reefs, near which were
three towns. We seized two boys here. The officer in command of the soldiers
went with all our people to reconnoitre the town that was a league away; and I
remained behind in charge of the brigantine with no small risk, for there were
only three soldiers left with me to defend it. In a few hours the people returned
with two canoes that they had taken, and five sucking-pigs, and some panaes,
and plantains, with which they embarked. We then made sail to proceed further
along the coast.
“On the next day a canoe with two Indians came off to us. They were friendly,
and one of them came on board the brigantine. We sailed on in order to reach a
harbour, and proceeded further along the same coast, on which there were many
towns, and the people of them were, as we expected, very turbulent; for a canoe
preceded us, giving warning in such a manner that in all this island we were not
able to capture anything. As we approached a promontory (morro), many Indians
came out and threw stones at us with much shouting; and at the extremity of this
island we discovered two small islands. The end of this island is in 111⁄2° south of
the Equinoctial. This island is a hundred leagues in circuit and seven leagues in
width, and is well peopled.
“From the extremity, we went to one of the small islands which was the smallest
and lay to the south side.[291] On arriving there we anchored; and there came off
to us twelve Indians who came on board the brigantine and spent some time with
us. On their being asked by signs what further land there was in that part, they
said that there was none; but towards the west, where we pointed, they said that
there was much land. We saw it, and because there was no time or opportunity
we did not go to it.[292] Through the day and night we had much wind. As we
were about to disembark, the natives began to throw stones at us; and when
some shots were fired for our own defence, they fled. Accordingly, we landed and
went to the town, where we found some hogs and a quantity of almonds and
plantains. I ordered a sailor to climb a high palm to see if he could descry land to
the south, or south-east, or north-west (?)[293] but no further land appeared.
There came from that quarter a great swell which was a sign of their being no
more land there. This island, we named, Santa Catalina; in the language of the
natives it is called Aguare.[294] It is 40[295] leagues round, and it is low and level.
It has many palms and is well peopled. It has many reefs. It is in latitude 112⁄3°,
and it lies two leagues south-east from the extremity of San Christobal.
[291] This small island was subsequently named Santa Catalina; and the circumstance of
the Spaniards going to it before they visited the adjacent small island of Santa Anna, is a
proof of their having coasted along the south side of St. Christoval. Then, the description
of the trend of the coast (see page 229) applies rather to the south than to the north
coast; and this is further confirmed by the circumstance that when the Spanish ships
were soon afterwards leaving the group on their return voyage to Peru, they weathered
or doubled the two islands of Santa Anna and Santa Catalina. Again, no reference is
made to the islands visible off the north coast, which would have been certainly referred
to, even although they had previously visited them in the brigantine. I lay stress on this
point as it clears up the confusion of the different names applied to St. Christoval.
[292] There is some obscurity in this passage, and in rendering it I have been guided by
the account of Figueroa.
[293] “North-west” is an error, which the context indicates, even excluding other
circumstances; it should be “south-west.”
[294] The present native name is Orika, or Yoriki of the Admiralty chart.

[295] An evident mistake, and one inconsistent with the context. The island is scarcely
two leagues in circuit.

“On the 11th of this month, we went from this island to the other island which lies
with it north-north-west and south-south-east,[296] a short league distant from it.
[297] It is distant 3 leagues, east-by-south, from the end of San Christobal; and is
in latitude 11° 36′. We named it Santa Anna; it is called Hapa[298] in the language
of the natives. It is 7 leagues in circuit; and is a low round island with an
eminence in the centre, like a castle; it is well peopled, having abundant
provisions, with pigs and hens of Castile; and there is a very good port on the
east side.[299]
[296] This bearing is only approximate, the magnetic bearing being nearly north and
south.
[297] This distance agrees nearly with that on the chart which is about two miles.
Figueroa, in his account, gives the distance as three leagues.
[298] The village, situated on the shores of Port Mary on the west coast of the island, is
at present called Sapuna by its inhabitants. Allowing for the variation in the spelling of
native names, we can here recognise the Hapa of the Spaniards. Oo-ah or Oa, is the
name of the island.
[299] This is a good description of the appearance of this island. The port is, however, on
the west side; and the circumference of the island is not half this amount.

“On arriving there, we landed the people, and the Indians commenced to attack
us.[300] On an Indian being killed, they began to fly, and deserted the town. Our
men entered the houses in search of provisions, but they found only three hogs,
as all the rest had been placed in safety. At nightfall we embarked in the
brigantine and stood off the land; and all the night we heard no sound except the
crowing of many cocks. The next morning, which was the 13th of July, we landed
the people to obtain more provisions to carry back for the sick in the ships; and
when the Indians saw our people landing, they got into ambush. I was left with
four soldiers in charge of the brigantine. The Indians, with loud cries, began to
attack our men, discharging many darts and arrows. Their bodies were painted
with red stripes, and they had branches on their heads.[301] They wounded three
Spaniards and a negro of mine; and also the officer in command, Francisco
Muñoz, a dart piercing the shield and arm and projecting a hand’s breadth on the
other side of the shield. Rallying our men, we attacked them valiantly, killing some
Indians and wounding many others, so that they abandoned the place and fled.
We burned the town, and took water. From the higher ground near by we tried to
discover any appearance of land; but as we saw none, we embarked on our
return voyage to the ships.
[300] “A dar nos guacanara.” What “guacanara” means, I can only guess at.

[301] I cannot gather the meaning of this latter part of the sentence and have rendered
it literally. The same expression occurs in the account of Figueroa.

“Sailing all this day with a fair wind, we arrived at the island of San Christobal;
and that night we entered a port because there was a threatening appearance in
the weather. We landed in a town that was there, and the Indians fled,
discharging some arrows. A soldier was wounded in the throat, but not seriously,
and he was able to swallow some food. As we wished to leave the port with the
rising moon, we embarked; and we named the port La Palma.
“We continued our voyage back to the ships; and when we had sailed about 4
leagues from the port, a canoe came off to look at us and to learn what people
we were. As we had need of Indians for their language, we endeavoured to take
the canoe; and so we coaxed them on, and of four which came in the canoe we
took three alive, and one died whilst defending himself. In the evening, we arrived
at the Puerto de la Visitacion de Nuestra Senora, where the ships lay.[302] I found
that, on account of bad treatment, all the Indians whom we had taken in the
islands had gone.
[302] From the short description of this harbour given on page 228, it is probably not
Makira Harbour on the south coast of St. Christoval; although from the time occupied by
the brigantine in her return voyage along this south coast from Santa Anna to the ships,
it must be in its vicinity.

“I gave a report to the General of what we had seen and accomplished in the
expedition, telling him that there was no appearance of land further (in that
direction), but that all the mass of the land, which was endless, lay to the west;
and that, from this, he would perceive what ought to be done. A council of the
captains and pilots was held to determine what steps should be followed in the
prosecution of the voyage; and it was decided to refit the ships for this purpose;
this, therefore, was the result of the general consultation. The ships were
accordingly refitted;[303] but on Saturday, the 7th of August, in the same year of
1568, all mustered together and made a protestation to the General and the
captains with reference to the plan to be pursued. I told them briefly that because
the ships were getting worm-eaten and rotten, and the rigging and cordage were
not of much good, we should be determined to complete, without delay, the
object for which we had come. The General, in reply, said that it would be well
that the brigantine should go in search of more provisions, of which we were in
want; but I pointed out that this should not be done, because all the islands that
we had visited were aroused, and the provisions hidden. They asked for my
opinion as to returning to Peru, whence we had come; and I told them that we
should not sail to the south of the Equinoctial, as we should be lost, on account of
there being many people, scanty provisions, and but little water. I also said that if
we were to direct our course to positions in latitudes which we should have time
to reach, we should not have time to find land to the south-south-west and south,
which would be a work of difficulty; and that such a new navigation, with 1,700
leagues of sea to cross on our return voyage, did not seem prudent. I therefore
gave it as my opinion that we should steer north to reach the latitude of the first
land we found, because it would be necessary, in order to shape a course from
Peru, to go beyond the south tropic for thirty degrees and more; and I also said
that when they should venture to make the return voyage, they should carry an
abundance of water and provisions, because, otherwise, they would run the risk
of all perishing. And so the pilots came to my view, which satisfied the protest
that had been made; and I gave my opinion in the presence of a clerk who was
Antonio de Cieza. Concerning the idea of my asking to found a settlement in these
islands, I said that in that matter I did not know what the General intended to do,
since the instructions concerning it were in his keeping. To this opinion they all
came, and were of one mind without one that did not assent.[304]
[303] Figueroa refers to the ships being heaved down in this harbour.

[304] The impression, which this interesting passage leaves on my mind, is that the
Chief-Pilot prefers in his narrative to gloss over an incident which must have been full of
disappointment to himself. Further on in the narrative, he writes more freely on the
subject (page 237). In Note IX. of the Geographical Appendix, I have given some further
remarks on this passage.

“At midnight on the following Monday, when all were asleep, the General ordered
Gabriel Muñoz and myself to go with some soldiers and make an entrance into a
town in order to seize some Indians for interpreters (para lenguas). We went with
30 men, and took an Indian with his wife and young son; and all the rest of the
Indians fled. We then returned to the ships; and straightway we made
preparations for prosecuting our voyage.
“On the 11th of August of the same year, we left the Puerto de Nuestra Senora,
which is in 11° south of the Equinoctial, in order to follow our voyage to Peru.
Sailing to windward, at the end of 7 days after we had left the port, we weathered
the island of San Christobal with the two islands of Santa Catalina and Santa
Anna. On the Tuesday evening, having shortened sail, we had reached the islands
of Santa Catalina and Santa Anna, which lay three leagues to the north-north-
west. Looking around we did not see any more land, and here a strong south-east
wind overtook us; and we shaped our course to the north-east-by-east.”
In this manner the Spaniards left behind them the Isles of Salomon after a
sojourn of six months in these islands; and, perhaps, a few reflections on their
discoveries in this group, and on their dealings with the inhabitants, may be here
apposite. They seem to have landed on, and to have taken formal possession of,
almost every island of any size from Isabel eastward; they named all the large
islands in the group with the exception of Bougainville; and the majority of the
smaller islands also received their names. In the Geographical Appendix, I have
given a list of the islands named by the Spaniards, which do not at present bear
the names given them by their original discoverers.[305] It would be a graceful
compliment to the memory of the gallant Gallego, who was the central figure of
this expedition, if, after the lapse of more than three centuries, the Spanish names
should be associated with these islands in the Admiralty charts. The reason why
such islands as Choiseul, Contrarieté, Les Trois Sœurs, and the Ile du Golfe (Ugi),
at present bear the names given to them by the French navigators, Bougainville
and Surville, rather more than a century ago, is to be found, not in any intended
act of injustice to the Spanish discoverers, but in the circumstance that the
imperfect account of Figueroa,[306] which omits many of the discoveries made in
the brigantine, has been the only source of information available in the
construction of the Admiralty charts. Those who have written most on the history
of geographical discovery in these regions, Pingré, Dalrymple, Buache, and
Fleurieu a century ago, and Burney in the early part of the present century, had
only the account of Figueroa at their disposal.[307] The Journal of Hernan Gallego,
the existence of which was doubted, would have been invaluable to them; and
although a non-professional writer, I may be pardoned when I express my
admiration at the manner in which M. M. Buache and Fleurieu arrived at such
correct inferences, based as they were on such scanty premises. One or two
mistakes have arisen in the nomenclature of the present chart, which are due to
misconceptions in the English translations of the account given by Figueroa, to
wit, I may cite the instance of the Isle of Ramos. . . . . . . The additional names
which the Journal of Gallego enables us to identify with existing islands are, in
truth, to be found in the general description of the Salomon Islands, which
Herrera incorporated in his “Descripcion de les Indias Occidentales,” which was
published about 1601. But this description was, as just remarked, of a general
character, and beyond confirming the suspicion that there were other accounts of
Mendana’s discoveries besides the relation of Figueroa, it was but of little service
to the nautical geographer.
[305] Vide Note X.

[306] Translated in great part from the original in the works of Pingré, Dalrymple,
Fleurieu, and Burney. (Hechos de Don G. H. de Mendoza: par Dr. C. S. de Figueroa.)
[307] Pingré’s “Mémoire sur le choix et l’état des lieux où le passage de Vénus du 3 Juin,
1769;” Dalrymple’s “Historical Collection of Voyages;” Fleurieu’s “Découvertes des
François en 1768 et 1769 dans le sud-est de la Nouvelle Guinée” (also Eng. edit.);
Burney’s “Chronological History of Voyages and Discoveries,” &c.

I come now to a less pleasant task, that of reviewing the character of the
intercourse that prevailed between the Spaniards and the natives. It has been
remarked by Commander Markham in his spirited sketch of the discoveries of
Mendana, that the conduct of the Spaniards, in their intercourse with the
islanders, was not otherwise than humane;[308] but I feel assured that a different
opinion would have been expressed, if the writer had extended his inquiries
further into the narrative of Gallego. During their six months’ sojourn in this
group, the loss of the Spaniards was but trifling in comparison with the losses
they inflicted on the natives. In these numerous conflicts the natives must have
lost not less than a hundred killed, whilst the Spaniards lost ten of their number;
but a large proportion of these unfortunate islanders fell victims to the lamentable
succession of reprisals for the massacre of the watering-party at the Puerto de la
Cruz, an act of retribution which the Spaniards had entirely brought upon
themselves. In the great majority of instances the natives assumed the
aggressive, but not in all; and although the Spaniards were often justifiably
compelled to employ force in obtaining provisions, yet there was often nothing to
excuse them in seizing the canoes, in cajoling natives alongside in order to
capture them, or in carrying off with them from the group an unfortunate native
with his wife and child. The natives kept on board the ships escaped on account
of ill-treatment; and, as Gallego also writes, all the islands were aroused to such a
degree by the visit of the Spaniards, that they concealed their provisions, and the
ships began their return-voyage to Peru with scanty supplies of food and
water. . . . . . We must, however, judge of the conduct of the first discoverers of
the Solomon Islands in the spirit of the age to which they belonged. The zeal,
which led them to burn the temples dedicated to the worship of snakes and toads
in the interior of Isabel, was appropriate to the spirit of an age in which
expeditions were fitted out for the double purpose of discovering new territories
and of reclaiming the infidel. Yet, if we lay aside the religious element, I doubt
very much whether the lapse of three centuries has materially raised the standard
by which our dealings with savage races should be guided. The white man
kidnaps; the savage revenges the outrage on the next comer; the ship-of-war in
its reprisal is of necessity equally indiscriminate; and thus feuds are re-opened
with no single effort at conciliation.
[308] “The Cruise of the ‘Rosario,’” 2nd edit., 1873 (p. 8).

We left the Spanish vessels when on the eve of their departure from the Isles of
Salomon. Little could Mendana or Gallego have then believed that two centuries
would pass away before the white man should again visit the scene of their
discovery. The Chief-Pilot kept in his journal an almost daily record of the course
and usually of the distance during the first portion of this return voyage; but as he
was not so regular or so precise in noting the distance of each day’s run, the
latitudes, which he frequently records, enable me to follow this portion of the
track with some degree of confidence.[309] It was on the 18th of August that they
bore away to the north-eastward (N.E. by E.) with a strong south-east wind.
Experiencing rain-squalls and calms, they kept a little to the north of this course,
and on the 23rd they were in latitude 7° (full largos), being, as they computed, 36
leagues W. by N. from the Isle of Jesus.[310] It is apparent from the Journal that
Gallego expected to find more land in this vicinity, and that he would willingly
have gone in search of it. But the expedition had lost heart in the enterprise, and
all that they desired was to return to Peru. A look-out was kept for several days,
but not a sign of land was seen; and thereupon Gallego, stifling his own desire,
thus records his lament in his journal: “As in the case of the archipelago of the
islands, they did not allow me to explore further where I wished. And I hold for
certain that if they had allowed me to go further, I should have brought them to a
very prosperous and rich land, which will be discovered at God’s pleasure by
whomsoever He wills. We were not far from it now, and of its goodness I did not
wish to speak, because they were all disheartened and desired to return to Peru.”
[309] I have only indicated the general course in the return voyage, as a full translation
would be tedious to the reader and would occupy too much of my space.
[310] The bearing was to the southward of west, as the Isle of Jesus, according to
Gallego’s own observation, was in latitude 63⁄4°. Three days after, when they were in
latitude 51⁄2° S., Gallego gives their distance and bearing from the Isle of Jesus as 45
leagues W. by N.

Heading north-eastward with uncertain winds, they were obliged to steer S.E. by
E. for six days as the wind shifted to the north-east. Finally, they headed to the
northward again, and in the last day of August they passed the 3rd parallel of
south latitude. “Between 2° and 4° of south latitude,” as Gallego writes, “we met
abundant signs of land, such as palm-leaf matting, burnt wood, sticks, and
rosuras,[311] which the sea derived from the land. From these signs we knew that
we were near the land, although we did not discover it. We thought that it was
New Guinea,[312] because it is not in a greater latitude than 4° south of the
Equinoctial.”
[311] Not translated.

[312] Gallego here adds: “Inigo Ortez de Retes discovered it (i.e., New Guinea) and no
other: but Bernardo de la Torre did not see it: nor is there such a Cabo de Cruz (Cape of
the Cross) as he says.” I have placed this interesting reference to the discovery of New
Guinea in a foot-note, as it is suddenly interposed in the narrative. In Note XI. of the
Geographical Appendix, the reader may learn more, if so desirous.

New Guinea, however, lay some 1200 miles away; and the Spanish vessels were in
the vicinity of the Gilbert Group, which lay probably about 300 miles to the
eastward. On September 5th, with shifty and contrary winds, they crossed the
Equator at about the 168th meridian of longitude east of Greenwich. The course
pursued, in which it would appear the Chief-Pilot had not been consulted, was the
subject of a protest made to the General. Thus writes Gallego: “I said to the pilot,
Juan Henriquez, that we ought to petition the General to direct our course to one
place or another or to steer for one pole or the other, as we were expending our
provisions and water in beating to windward. Since the General followed his own
opinion and showed no desire to consult me, I made this request in the presence
of Antonio de Cieza, Clerk, all of which appears more fully in the said petition,
which is in the possession of the said Clerk.”
Steering to the north and subsequently to the N.E. by E., they reached the 4th
parallel of north latitude on September 8th. “This day,” writes the Chief-Pilot, “I
signified to the ‘Almiranta’ that they should keep a good look-out from 6° up to
11°, as we were heading for the land.” Altering their course to N.N.W., they
reached the parallel of 6° on the 14th, the needle showing no declination to the
north-east. On the 15th and 16th, they headed north-east, and on the 17th,
steering north, they found themselves in 8°. The surmise of Gallego proved
correct. In this parallel, they discovered land.
“Two hours before dawn,” as the Chief-Pilot writes, “we came upon the shoals and
islands of San Bartolomeo, which trend north-west and south-east and are 15
leagues in length. The south-east extremity is in 8°, and the north-west extremity
lies in 82⁄3°. There are two lines of reefs with apparently channels between them.
There seems to be another line about half-a-league distant. At the north-west,
there are two islets, which lie one with the other east and west one league. The
coast is steep-to; and we did not find any depth to anchor on the west side. There
were many houses and much people and villos in these islands. Between the
islands, which number more than 20, a canoe was under sail, but it made for the
shore. We launched the boat to go for water. They could only obtain a cock of
Castile, which they brought back with them. The people fled, abandoning their
houses. They came upon a chisel made from a nail, which appears to have
belonged to some ships that had been there, and some pieces of rope. They did
not find water, but the cocoa-nut palms were cut which showed how the
inhabitants got their water.[313] These Indians drink “chicha,”[314] which is made
from some fruits like pine-apples; and on this account there is an infinite number
of flies. We beat to windward for three hours trying to find depth to anchor; but
the water was a thousand fathoms (estados) deep. When the boat returned, we
continued our voyage.”
[313] This probably refers to cocoa-nut palms that had been cut for making “toddy,” a
practice to be found amongst the natives of the Line Islands at the present day.
[314] An Indian name for a drink prepared from maize.

Figueroa, in his scanty account, neither gives the name nor the latitude of this
discovery, so that previous writers, who derived their information entirely from
this source, were unable to identify these islands with those in the charts.
However, with the materials afforded to me by the journal of Gallego, I have been
able, after carefully following the track of the Spanish ships, to identify this
discovery with the Musquillo Islands in the Ralick Chain of the Marshall Group.
Having followed their course northward from the vicinity of the Gilbert Group, to
which I referred above (page 237), it was evident that they were about to pass
through the Marshall Islands, and that if they should sight land, I had only to
compare the description of Gallego with the present chart of this group, in order
to identify this discovery with one of the atolls that there exist. (Vide Note XII. of
the Geographical Appendix.)
Continuing their course to the northward, they began to get short of water, and
the people sickened and . . . . .[315] On the 22nd of September, they attained the
latitude of 111⁄2°, and running due north along the meridian, they reached the
latitude of 191⁄3° on October 2nd, when they discovered “a low islet enclosing the
sea after the manner of a fishing-net, and surrounded by reefs.” “We were hove-
to all that night,” . . . writes Gallego, . . . “believing that it was inhabited, and that
we should be able to obtain water. But there were only sea-birds living on it; and
its surface was sandy with some patches of bushes. It is probably two leagues in
circuit: and is in latitude 191⁄3° north of the Equinoctial. As it was the Day of San
Francisco, we named it the Isle of San Francisco.”
[315] “Murieron hartos.” To avoid falling into a serious mistake, I have not translated this,
more especially as Figueroa refers to no deaths on board during the voyage to Peru.

This island of San Francisco has not been identified by previous writers with any
island in the present chart, as Figueroa supplied them with the latitude alone, but
gave no reliable account from which they might be able to follow the previous
track; nor, in fact, in the times of Burney and Krusenstern, who were the last to
devote any considerable attention to the discoveries of Mendana, was this part of
the Pacific sufficiently well known to enable even a confident surmise to be made.
Commodore Wilkes, amongst others, has swept more than one phantom-island
from this region. The track of the Spanish ships northward from the Marshall
Group brought them, in fact, to a little coral-atoll, named Wake’s Island in the
present chart, and lying in 19° 10′ 54″ N. lat. This is the Isle of San Francisco,
which is but little altered in appearance in our own day.[316]
[316] Vide Note XIII. of the Geographical Appendix for further information on this
subject.

Keeping the same northerly course, they passed the limit of the tropic of Cancer
on October 7th; and in another week they had reached the latitude of 30°. They
now shaped their course north-east; and Gallego consulted the other pilots as to
the position of the land, and as to the bearing of the Cabo de Fortunas[317] (Cape
Fortune). “They told me in reply,” . . . . as the Chief Pilot informs us, . . . . “that
we were already in the vicinity of land, that this cape lay, in their opinion, 70 or 80
leagues to the north-by-west, that we were much to leeward of the land, that it
was not practicable to reach the cape with this wind as the coast trended north-
west and south-east, and that we could not live unless we fell in with the land.”
[317] This cape is evidently referred to as on the Californian coast; I cannot identify it.

Could the Spaniards have known at this time what lay before them, the bravest
heart amongst them would have quailed. Instead of being in the neighbourhood
of the Californian coast whither they were steering, they had more than 3,000
miles of ocean to traverse and two long dreary months to struggle through, before
they were fated to sight the land. They were destined to pass through storms, the
like of which Gallego had never witnessed during his 45 years’ experience of the
sea. The two ships were to be parted; and each was to pursue its solitary way in
the fear that the missing ship had foundered. Such was the lot before them with
sickness already amongst them, and with a failing store of water and provisions.
The Chief-Pilot thus continues his narrative—“On the 14th of this month
(October), I continued to steer both ships in close company to the north-east. In
the middle of the night there came a squall with a little rain. We shortened sail;
and at that time the ‘Almiranta’ was to windward; but she allowed herself to fall to
leeward for an hour, and when it dawned we could only see her from the top.
Hoping to fall in with her, we carried only the fore-sail, and made no more sail all
that day and night. We headed to the north-east until the second hour of the day;
and because we did not see her, we took in all the sails. This was the 16th day of
the month of October.
“Two hours after noon on Sunday the 17th, whilst we were yet hoping, we
shortened sail because there was much wind from the south-east. We were driven
before the gale; and as we were lying in the trough of the sea without any sails,
the wind came upon us with all its fury from the north-east, such as I never
beheld during the 45 years that I have been at sea, 30 of which I have served as
pilot. Such boisterous weather, I have never witnessed, although I have seen
storms enough. For a squall to take us when we were without sail, this was what
frightened me. A sea struck us on the port side from the water-line to the middle
hatch, which was battened down and caulked as I had ordered. We were deluged
with water. Everything went its own way; and the soldiers and sailors were
swimming about inside the ship, as they were trying to launch the boat, which
was smashed and full of cables and water. The sailors were not able of themselves
to do it; but God and His Blessed Mother willed that it should be done.[318] Then I
ordered the sailors to unfurl a little of the sail; but before two gaskets were
loosed, the fore-sail went into two thousand pieces, and only the bolt-ropes
remained. For more than half-an-hour the ship was in great peril until the main-
mast was cut away.[319] And soon I ordered them to make a sail of a frecada,[320]
and of a piece of a bonnet (boneta); with this the ship was able to answer her
helm. . . . . .[321] The weather began to clear. We were driven from our course
more than 50 leagues, because the storm overtook us in latitude 321⁄3°, and
when it began to clear we found ourselves in 30°. When this weather came upon
us we were 70 leagues south-east-by-south[322] from the Cabo de Fortunas; and
when it began to clear we were 120 leagues, rather more than less.
[318] This reference to the launching of the boat, in order, I infer, to lighten the vessel, is
ambiguously expressed. Figueroa, in his account, would appear to imply that the boat
was merely relieved from its weight of ropes and water; but further on in his account,
Gallego expressly refers to their being without a boat.
[319] Figueroa adds to this account. He says that the General gave the order to cut away
the mainmast, and that it carried away a portion of the bulwarks.
[320] Frazada in the account of Figueroa.
 [321] “Para atras hechamos el camarote de popa a la mar.”

[322] I cannot understand this bearing.

“We headed on our course with only the fore-sails, as we had no other sails, since
the sailors had lost the bonnets overboard. On the 21st of October, the wind went
round to the opposite quarter, and lasted until the 29th. Coursing north-east with
much wind and sea, we sailed close-hauled on one tack or the other, because it
was no longer possible to sail free as the sea would engulph us. The ship did not
behave well in a beam sea, for soon she shipped seas on either side, and she lost
as much way as she made. On the evening of the 29th of October, the wind went
round to the south-east, and there was a heavy sea. The wind was so strong that
we were unable to make any sails, as they were carried away. All that night we lay
in the trough of the sea with much wind and thunder and lightning, so that it
seemed like the overwhelming of the world.[323] On the following morning I
ordered them to clear away the sprit-sail and use it as a fore-sail, so that we
might steer the ship. Before we had run for a watch to the north-east, the wind
went round to the south, and with such force that it carried away the sails and we
were left without any sail. We employed las frescadas (blankets?) for sails, and
thus we went this day. Soon the wind lessened, and we hoisted the fore-sail and
coursed north-east until the next day, which was the last day of October.”
[323] Figueroa in his account states that there was always a foot and a half of water in
the hold.

The “Capitana,” to which ship the narrative for a time alone refers, was now in
29° N. lat. A very strong north-east wind, lasting until November 4th, drove them
to the south-east in latitude 26°. These north-easterly winds continued to prevail;
and being unable to sail close to the wind, the Spaniards could not keep their
latitude and were being driven from their course, to the south-east.[324] “We
were,” . . . . as Gallego writes, . . . . “much wearied and suffered from hunger and
thirst, as they did not allow us more than half a pint of stinking water and eight
ounces of biscuit, a few very black beans, and oil; besides which there was
nothing else in the ship. Many of our people were unable from weakness to eat
any more food. A soldier, who had gambled with his allowance of water and had
lost it, became desperate with thirst and cried out all the day. Being without a
boat, we could do nothing on approaching a harbour. We resolved to trust that
God would send us the means of help. He provided for us in His great mercy, and
on the day of St. Isabel (November 19th) he gave us a (fair) wind, and we sailed
in the latitude of 28° and up to 30°. This weather lasted until the 26th of
November, and we were 125 leagues further on our voyage.”
[324] Figueroa in his account tells us that they rigged a jury-mast, making use of a top-
mast for this purpose.
During the first week of December they experienced foul winds and thick weather:
but on the 9th the wind went round to the south-south-east; and they reached
the latitude of 31° on the 12th. Signs of the vicinity of land were now observed,
such as sea-birds and a goose. A sailor leapt into the sea after a floating piece of
a pine, and brought it on board, in order to bring fair weather. Rain fell, and
enough water was collected for three days. At length the land was sighted by the
watchful eye of Gallego. “It was the eve of our Lady the Virgin” . . . . . he writes
. . . . . “and whilst standing at the side of the ship, I saw the land. Some of us,
who despaired to see it, said that it could not be the land. Sailing through the
night, two hours before the dawn we found ourselves close to two islets that lay a
league from the mainland in latitude 30° north of the Equinoctial.[325]”
[325] Gallego here observes that the day before the land was sighted, the needle
remained pointing north.

At length the Spaniards had reached the coast of Old California. “The mercy of
God”—as Gallego writes—“had brought us safely through so many storms and
privations that the soldiers had despaired of seeing it. Following along the coast,
as it trended to the south-east, we entered a bay which resembles in form a pen
for shoeing cattle (corral de herrar ganado). We could not see the outside point
on account of its great distance. We found ourselves embayed; and it was
necessary to steer west to weather this point. . . . . . We were detained three
days with calms and north-west winds, as we had to beat to windward to weather
this point. We named this bay la bahia de San. tome: it is in latitude 273⁄4°. At the
point of this bay there are two large islets, named the Isles of Cacones.[326] We
doubled the point on the 23rd of December. We beached the ship for 12 days
between these islets. Having lost our boat at sea, we went ashore on a raft of
casks to get water. There we made another raft of rushes and some casks, on
which we carried on board 12 casks of water and many fish that we caught.”
[326] This large bay, which deeply indents the Californian peninsula, is named in the
present maps the bay of Sebastian Vizcaino, after the Spaniard who surveyed this coast
in 1602. Gallego’s name of San. tome, which may be a contraction for San. Bartolomeo,
has, therefore, the priority of some 30 years and more. The prominent headland, which
they had to double, is at present called Point Eugenio. The two large islets off this point
are now called Cerros and Natividad Islands.

Having obtained timber for making another boat, they continued their voyage, as
the Indians were hostile. A foul wind caused them to pass by the port of Xalosco,
and they “tacked to seaward to double the Cabo de Corrientes, which is in 21°, in
order to reach the port of Santiago, which is 50 leagues beyond Xalosco.”
On the 24th[327] of January, 1569, they entered the port of Santiago. The Chief-
Pilot tells us in his journal that he was well acquainted with this coast and with its
people: this port,[328] he says, lies six leagues from Port Natividad, and is in
latitude 191⁄4°. Before they left Santiago a joyful surprise awaited them. “On the
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

textbookfull.com