Algorithms For Big Data Lecture Notes Harvard

Cs229r Itebooks download

https://ebookbell.com/product/algorithms-for-big-data-lecture-
notes-harvard-cs229r-itebooks-23836006

Explore and download more ebooks at ebookbell.com

Here are some recommended products that we believe you will be
interested in. You can click the link to download.

Algorithms For Big Data Lecture Notes Uiuc Cs598csc Itebooks

https://ebookbell.com/product/algorithms-for-big-data-lecture-notes-
uiuc-cs598csc-itebooks-23836010

Algorithms For Big Data Dfg Priority Program 1736 Hannah Bast

https://ebookbell.com/product/algorithms-for-big-data-dfg-priority-
program-1736-hannah-bast-47546596

Algorithms For Big Data Feldman Moran

https://ebookbell.com/product/algorithms-for-big-data-feldman-
moran-34388846

Sublinear Algorithms For Big Data Applications 2015th Edition Dan Wang

https://ebookbell.com/product/sublinear-algorithms-for-big-data-
applications-2015th-edition-dan-wang-5146996
Natureinspired Algorithms For Big Data Frameworks Hema Banati

https://ebookbell.com/product/natureinspired-algorithms-for-big-data-
frameworks-hema-banati-33794902

Medical Computer Vision Algorithms For Big Data International Workshop

Mcv 2014 Held In Conjunction With Miccai 2014 Cambridge Ma Usa
September 18 2014 Revised Selected Papers 1st Edition Bjoern Menze

https://ebookbell.com/product/medical-computer-vision-algorithms-for-
big-data-international-workshop-mcv-2014-held-in-conjunction-with-
miccai-2014-cambridge-ma-usa-september-18-2014-revised-selected-
papers-1st-edition-bjoern-menze-4973824

Machine Learning Models And Algorithms For Big Data Classification

Thinking With Examples For Effective Learning 1st Edition Shan
Suthaharan Auth

https://ebookbell.com/product/machine-learning-models-and-algorithms-
for-big-data-classification-thinking-with-examples-for-effective-
learning-1st-edition-shan-suthaharan-auth-5233984

Medical Computer Vision Algorithms For Big Data International Workshop

Mcv 2015 Held In Conjunction With Miccai 2015 Munich Germany October 9
2015 Revised Selected Papers 1st Edition Bjoern Menze

https://ebookbell.com/product/medical-computer-vision-algorithms-for-
big-data-international-workshop-mcv-2015-held-in-conjunction-with-
miccai-2015-munich-germany-october-9-2015-revised-selected-papers-1st-
edition-bjoern-menze-5607052

Diskbased Algorithms For Big Data Healey Christopher Graham

https://ebookbell.com/product/diskbased-algorithms-for-big-data-
healey-christopher-graham-5699328
 CS 229r: Algorithms for Big Data Fall 2015

Lecture 1 — September 3, 2015

Prof. Jelani Nelson Scribes: Zhengyu Wang

1 Course Information

• Professor: Jelani Nelson

• TF: Jaroslaw Blasiok

2 Topic Overview

1. Sketching/Streaming

• “Sketch” C(X) with respect to some function f is a compression of data X. It allows

us computing f (X) (with approximation) given access only to C(X).
• Sometimes f has 2 arguments. For data X and Y , we want to compute f (X, Y ) given
C(X), C(Y ).
• Motivation: maybe you have some input data and I have some input data, and we want
to compute some similarity measure of these two databases across the data items. One
way is that I can just send you the database, and you can compute locally the similarity
measure, and vise versa. But image these are really big data sets, and I don’t want to
send the entire data across the wire, rather what I will do is to compute the sketch of my
data, and then send the sketch to you, which is something very small after compression.
Now the sketch C(X) is much smaller than X, and given the sketch you can compute
the function.
• Trivial example: image you have a batch of numbers, and I also have a batch of numbers.
We want to compute their sum. The sketch I can do is just locally sum all my input
data, and send you the sum.
• Streaming: we want to maintain a sketch C(X) on the fly as x is updated. In previous
example, if numbers come on the fly, I can keep a running sum, which is a streaming
algorithm. The streaming setting appears in a lot of places, for example, your router can
monitor online traffic. You can sketch the number of traffic to find the traffic pattern.

2. Dimensionality Reduction

• Input data is high-dimensional. Dimensionality reduction transforms high-dimensional

data into lower-dimensional version, such that for the computational problem you are
considering, once you solve the problem on the lower-dimensional transformed data, you
can get approximate solution on original data. Since the data is in low dimension, your
algorithm can run faster.
• Application: speed up clustering, nearest neighbor, etc.

1
3. Large-scale Machine Learning

• For example, regression problems: we collect data points {(zi , bi )|i = 1, . . . , n} such that
bi = f (zi ) + noise. We want to recover f˜ “close” to f .
• Linear regression: f (z) = hx, zi, where x is the parameter that we want to recover. If
the noise is Gaussian, the popular (and optimal to some sense) estimator we use is Least
Squares

xLS = arg min kZx − bk22 = (Z T Z)−1 Zb, (1)

where b = (b1 , . . . , bn )T and Z = (z1 , . . . , zn )T . If Z is big, matrix multiplication can

be very expensive. In this course, we will study techniques that allow us to solve least
squares much faster than just computing the closed form (Z T Z)−1 Zb.
• Other regression problems: PCA (Principal Component Analysis), matrix completion.
For example, matrix completion for Netflix problem: you are given a big product-
customer matrix of customer ratings of certain products. The matrix is very sparse
because not every user is going to rate everything. Based on limited information, you
want to guess the rest of the matrix to do product suggestions.

4. Compressed Sensing

• Motivation: compress / cheaply acquire high dimensional signal (using linear measure-
ment)
• For example, images are very high dimensional vectors. If the dimension of an image is
thousands by thousands, it means that the image has millions of pixels. If we write the
image in standard basis as pixels, it is likely that the pixels are not sparse (by sparse
we mean almost zero), because just image that if we take a photo in a dark room, most
of the pixels have some intensity. But there are some basis called wavelet basis, pictures
are usually very sparse on that basis. Once something is sparse, you can compress it.
• JPEG (image compression).
• MRI (faster acquisition of the signal means less time in machine).

5. External Memory Model

• Motivation: measure disk I/O’s instead of number of instructions (because random seeks
are very expensive).
• Model: we have infinite disk divided into blocks of size b bits, and memory of size M
divided into pages of size b bits. If the data we want to read or the location we want
to write is in the memory, we can just simply do it for free; if the location we want to
access is not in the memory, we cost 1 unit time to load the block from the disk into the
memory, and vise versa. We want to minimize the time we go to the disk.
• B trees are designed for this model.

6. Other Models (if time permitting)

• For example, map reduce.

2
3 Approximate Counting Problem

In the following, we discuss the problem appearing in the first streaming paper [1].
Problem. There are a batch of events happen. We want to count the number of events while
minimizing the space we use.
Note that we have a trivial solution - maintaining a counter - which takes log n bits where n is
the number of events. On the other hand, by Pigeonhole Principle, we cannot beat log n bits if we
want to count exactly.
For approximate counting problem, we want to output ñ such that

P(|ñ − n| > εn) < δ, (2)

where let’s say ε = 1/3 and δ = 1%.

First of all, we can say that if we want to design a deterministic algorithm for approximate counting
problem, we cannot beat against log log n bits, because similar to previous lower bound argument,
there are log n different bands (of different powers of 2), and it takes log log n bits to distinguish
them. Therefore, we maybe hope for O(log log n) bits algorithm. Actually, the following Morris
Algorithm can give us the desired bound:

1. Initialize X ← 0.
1
2. For each event, increment X with probability 2X
.

3. Output ñ = 2X − 1.

Intuitively, we have X ≈ lg n where lg x = log2 (2+x). Before giving rigorous analysis (in Section 5)
for the algorithm, we first give a probability review.

4 Probability Review

We are mainly discussing discrete random

P variables. Let random variable X takes values in S.
Expectation of X is defined to be E X = j∈S j · P(X = j).
Lemma 1 (Linearity of expectation).
E(X + Y ) = E X + E Y (3)
Lemma 2 (Markov).
EX
Xis a non-negative random variable ⇒ ∀λ > 0, P(X > λ) < (4)
λ
Lemma 3 (Chebyshev).
E(X − E X)2
∀λ > 0, P(|X − E X| > λ) < (5)
λ2

3
Proof. P(|X − E X| > λ) = P((X − E X)2 > λ2 ). It follows by Markov.

Moreover, Chebyshev can be generalized to be:

E(X − E X)p
∀p > 0, ∀λ > 0, P(|X − E X| > λ) < . (6)
λp
Lemma 4 (Chernoff). X1 , . . . , Xn are independent random variables, where Xi ∈ [0, 1]. Let X =
P
i Xi , λ > 0,
2 ·E X/3
P(|X − E X| > λ · E X) ≤ 2 · e−λ . (7)

Proof. Since it’s quite standard, and the proof detail can be found in both previous scribe1 (Lec-
ture 1 in Fall 2013) and wiki2 , we only include a proof sketch here. We can prove that both
2
P(X − E X > λ · E X) and P(X − E X < −λ · E X) are smaller than e−λ ·E X/3 , and then apply
union bound to prove the lemma.
2 2
The proof for P(X −E X < −λ·E X) < e−λ ·E X/3 is symmetric to P(X −E X > λ·E X) < e−λ ·E X/3 .
2
So we can focus on how to prove P(X − E X > λ · E X) < e−λ ·E X/3 . Since P(X − E X > λ E X) =
t(X−E t)
P(et(X−E X) > et E X ) < E eet E X for any t > 0, we can optimize t to get the desired bound.

Lemma 5 (Bernstein). X1 , . . . , Xn are independent random variables, where |Xi | ≤ K. Let X =

2 = 2
P P
X
i i and σ i E(Xi − E Xi ) . For ∀t > 0,

2 /σ 2
P(|X − E X| > t) . e−ct + e−ct/K , (8)

where . means ≤ up to a constant, and c is a constant.

Proof. First, we define p (p ≥ 1) norm for random variable Z to be kZkp = (E |Z|p )1/p . In the
proof, we will also use Jensen Inequality: f is convex ⇒ f (E Z) ≤ E f (Z).
To prove Bernstein, it’s equivalent to show (equivalence left to pset)
X X √
∀p ≥ 1, k Xi − E Xi kp . p · σ + p · K. (9)
i i

Let Yi be identically distributed as Xi , with {Xi |i = 1, . . . , n}, {Yi |i = 1, . . . , n} independent.

We have
1
http://people.seas.harvard.edu/~minilek/cs229r/fall13/lec/lec1.pdf
2
https://en.wikipedia.org/wiki/Chernoff_bound

4
 X X X X
k Xi − E Xi kp = k E( Xi − Yi )kp (10)
Y
i i i i
X
≤k (Xi − Yi )kp (Jensen Inequality) (11)
i
X
=k αi (Xi − Yi )kp (Add uniform random signs αi = ±1) (12)
i
X X
≤k αi Xi kp + k αi Yi kp (Triangle Inequality) (13)
i i
X
= 2k αi Xi kp (14)
i
r
π X
=2· · kE αi |gi |Xi kp (Let g be vector of iid Gaussians) (15)
2 g
i
X
.k αi |gi |Xi kp (Jensen Inequality) (16)
i
X
=k gi Xi kp (17)
i

2
P P
Note that i αi |gi |Xi is Gaussian with variance i Xi . The pth moment of Gaussian Z ∼ N (0, 1):

(
p
0, p is odd.
EZ = p! √ p (18)
(p/2)!2p/2
≤ p , p is even.

Therefore,

X √ X
k gi Xi kp ≤ p · k( Xi2 )1/2 kp (19)
i i
√ X 1/2
= p·k Xi2 kp/2 (20)
i
√ X
≤ p·k Xi2 k1/2
p (kZkp ≤ kZkq for p < q) (21)
i
√ X X X 1
= p[k Xi 2 − E Xi2 + E Xi2 kp2 ] (22)
i i i
√ X X X
≤ p[k E Xi2 kp1/2 + k Xi2 − E Xi2 k1/2
p ] (23)
i i i
√ √ X X
=σ p+ p·k Xi2 −E Xi2 k1/2
p (24)
i i
√ √ X
.σ p+ p·k gi Xi2 kp1/2 (Apply the same trick (10)-(17)) (25)
i

Note thatP i gi Xi2 is Gaussian with variance i Xi4 ≤ K 2 · Xi2 , and i gi Xi is Gaussian with
P P P P
variance i Xi2 ,

5
 X X
k gi Xi2 kp ≤ K · k gi Xi kp . (26)
i i

P 1/2
Let Q = k i gi Xi kp , we have

√ √ √
Q2 − Cσ p − C p KQ ≤ 0, (27)

where C is a constant.
Because it’s a quadratic form, Q is upper bounded by the larger root of

√ √ √
Q2 − Cσ p − C p KQ = 0. (28)

√ √ √ √
By calculation, Q2 ≤ C p KQ + Cσ p . p · σ + p · K.

5 Analysis

Let Xn denote X after n events in Morris Algorithm.

Claim 6.

E 2Xn = n + 1. (29)

Proof. We prove by induction on n.

1. Base case. It’s obviously true for n = 0.

2. Induction step.
∞
X
Xn+1
E2 = P(Xn = j) · E(2Xn+1 |Xn = j)
j=0
∞
X 1 1
= P(Xn = j) · (2j (1 − j
) + j · 2j+1 )
2 2
j=0
(30)
∞
X X
= P(Xn = j)2j + P(Xn = j)
j=0 j

= E 2Xn + 1
= (n + 1) + 1

6
By Chebyshev,

1 1
P(|ñ − n| > εn) < · E(ñ − n)2 = E(2X − 1 − n)2 . (31)
ε2 n 2 ε2 n 2

Furthermore, we can prove the following claim by induction.

Claim 7.
3 3
E(22Xn ) = n2 + n + 1. (32)
2 2

Therefore,

1 n2 1
P(|ñ − n| > εn) < · = 2. (33)
ε2 n 2 2 2ε

5.1 Morris+

We instantiate s independent copies of Morris and average their outputs. Then the right hand side
1 1 3 1 1
of (33) becomes 2sε 2 < 3 for s > 2ε2 = Θ( ε2 ). (or < δ for s > 2ε2 δ )

5.2 Morris++

Run t instantiations of Morris+ with failure probability 31 . So s = Θ( ε12 ). Output median estimate
from the s Morris+’s. It works for t = Θ(lg 1δ ), because if the median fails, then more than 1/2 of
Morris+ fails.
Let

(
1, if ith Morris+ fails.
Yi = (34)
0, otherwise.

By Chernoff bound,

X t X X t
P( Yi > ) ≤ P(| Yi − E Yi | > ) ≤ e−ct < δ (35)
2 6
i i i

References
[1] Robert Morris. Counting Large Numbers of Events in Small Registers. Commun. ACM, 21(10):
840-842, 1978.

7
 CS 229r: Algorithms for Big Data Fall 2015

Lecture 2 — Sept. 8, 2015

Prof. Jelani Nelson Scribe: Jeffrey Ling

1 Probability Recap
V ar[X]
Chebyshev: P (|X − EX| > λ) < λ2
P
Chernoff: For X1 , . . . , Xn independent in [0, 1], ∀0 <  < 1, and µ = E i Xi ,

2
X
P (| Xi − µ| > µ) < 2e− µ/3
i

2 Today

• Distinct elements

• Norm estimation (if there’s time)

3 Distinct elements (F0 )

Problem: Given a stream of integers i1 , . . . , im ∈ [n] where [n] := {1, 2, . . . , n}, we want to output
the number of distinct elements seen.

3.1 Straightforward algorithms

1. Keep a bit array of length n. Flip bit if a number is seen.

2. Store the whole stream. Takes m lg n bits.

We can solve with O(min(n, m lg n)) bits.

3.2 Randomized approximation

t s.t. P (|t − e
We can settle for outputting e t| > t) < δ. The original solution was by Flajolet and
Martin [2].

1
3.3 Idealized algorithm

1. Pick random function h : [n] → [0, 1] (idealized, since we can’t actually nicely store this)

2. Maintain counter X = mini∈stream h(i)

3. Output 1/X − 1

Intuition. X is a random variable that’s the minimum of t i.i.d U nif (0, 1) r.v.s.
1
Claim 1. EX = t+1 .

Proof.
Z ∞
EX = P (X > λ)dλ
0
Z ∞
= P (∀i ∈ str, h(i) > λ)dλ
0
Z ∞ t
Y
= P (h(ir ) > λ)dλ
0 r=1
Z 1
= (1 − λ)t dλ
0
1
=
t+1

2
Claim 2. EX 2 = (t+1)(t+2)

Proof.
Z 1
EX 2 = P (X 2 > λ)dλ
0
Z 1 √
= P (X > λ)dλ
0
Z 1 √ √
= (1 − λ)t dλ u=1− λ
0
Z 1
=2 ut (1 − u)du
0
2
=
(t + 1)(t + 2)

t 1
This gives V ar[X] = EX 2 − (EX)2 = (t+1)2 (t+2)
, and furthermore V ar[X] < (t+1)2
= (EX)2 .

2
4 FM+

We average together multiple estimates from the idealized algorithm FM.

1. Instantiate q = 1/2 η FMs independently

2. Let Xi come from FMi .
1 P
3. Output 1/Z − 1, where Z = q i Xi .

1 1 t 1
We have that E(Z) = t+1 , and V ar(Z) = q (t+1)2 (t+2) < q(t+1)2
.
1 
Claim 3. P (|Z − t+1 | > t+1 ) <η

Proof. Chebyshev.
1  (t + 1)2 1
P (|Z − |> )< =η
t+1 t+1 2 q(t + 1)2

Claim 4. P (|( Z1 − 1) − t| > O()t) < η

Proof. By the previous claim, with probability 1 − η we have

1
1 − 1 = (1 ± O())(t + 1) − 1 = (1 ± O())t ± O()
(1 ± ) t+1

5 FM++

We take the median of multiple estimates from FM+.

1. Instantiate s = d36 ln(2/δ)e independent copies of FM+ with η = 1/3.

t of {1/Zj − 1}sj=1 where Zj is from the jth copy of FM+.
2. Output the median b

t − t| > t) < δ

Claim 5. P (|b

Proof. Let (
1 if |(1/Zj − 1) − t| > t
Yj =
0 else

We have EYj = P (Yj = 1) < 1/3 from the choice of η. The probability we seek to bound is
equivalent to the probability that the median fails, i.e. at least half of the FM+ estimates have
Yj = 1. In other words,
Xs
Yj > s/2
j=1

3
We then get that X X
P( Yj > s/2) = P ( Yj − s/3 > s/6) (1)
Make the simplifying assumption that EYj = 1/3 (this turns out to be stronger than EYj < 1/3.
Then equation 1 becomes
X X 1 X
P( Yj − E Yj > E Yj )
2
using Chernoff,
(1 2
− 2 ) s/3
<e 3 <δ
as desired.
The final space required, ignoring h, is O( lg(1/δ)
2
) for O(lg(1/δ)) copies of FM+ that require O(1/2 )
space each.

6 k-wise independent functions

Definition 6. A family H of functions mapping [a] to [b] is k-wise independent if ∀j1 , . . . , jk ∈ [b]
and ∀ distinct i1 , . . . , ik ∈ [a],

Ph∈H (h(i1 ) = j1 ∧ . . . ∧ h(ik ) = jk ) = 1/bk

Example. The set H of all functions [a] → [b] is k-wise independent for every k. |H| = ba so h
is representable in a lg b bits.

Example. Let a = b = q for q = pr a prime power, then Hpoly , the set of degree ≤ k − 1
polynomials with coefficients in Fq , the finite field of order q. |Hpoly | = q k so h is representable in
k lg p bits.
Claim 7. Hpoly is k-wise independent.

Proof. Interpolation.

7 Non-idealized FM

t satisfies t/C ≤ e
First, we get an O(1)-approximation in O(lg n) bits, i.e. our estimate e t ≤ Ct for
some constant C.

1. Pick h from 2-wise family [n] → [n], for n a power of 2 (round up if necessary)

2. Maintain X = maxi∈str lsb(h(i)) where lsb is the least significant bit of a number

3. Output 2X

4
For fixed j, let Zj be the number of i in stream with lsb(h(i)) = j. Let Z>j be the number of i
with lsb(h(i)) > j.
Let (
1 lsb(h(i)) = j
Yi =
0 else
Yi . We can compute EZj = t/2j+1 and similarly
P
Then Zj = i∈str

1 1
EZ>j = t( + + . . .) < t/2j+1
2j+2 2j+3
and also X X X X
V ar[Zj ] = V ar[ Yi ] = E( Yi )2 − (E Yi )2 = E(Yi1 Yi2 )
i1 ,i2

Since h is from a 2-wise family, Yi are pairwise independent, so E(Yi1 Yi2 ) = E(Yi1 )E(Yi2 ). We can
then show
V ar[Zj ] < t/2j+1

Now for j ∗ = dlg t − 5e, we have

16 ≤ EZj ∗ ≤ 32
P (Zj ∗ = 0) ≤ P (|Zj ∗ − EZj ∗ | ≥ 16) < 1/5
by Chebyshev.
For j = dlg t + 5e
EZ>j ≤ 1/16
P (Z>j ≥ 1) < 1/16
by Markov.
This means with good probability the max lsb will be above j ∗ but below j, in a constant range.
This gives us a 32-approximation, i.e. constant approximation.

8 Refine to 1 + 

Trivial solution. Algorithm TS stores first C/2 distinct elements. This is correct if t ≤ C/2 .

Algorithm.

1. Instantiate TS0 , . . . , TSlg n

2. Pick g : [n] → [n] from 2-wise family

3. Feed i to TSlsb(g(i))

4. Output 2j+1 out where t/2j+1 ≈ 1/2 .

5
 distinct elements hashed by g to TSj . Then EBj = t/2j+1 = Qj . By
Let Bj be the number ofp
Chebyshev Bj = Qj ± O( Qj ) with good probability. This equals (1 ± O())Qj if Qj ≥ 1/2 .
Final space: C
2
(lg n)2 = O( 12 lg2 n) bits.
It is known that space O(1/2 + log n) is achievable [4], and furthermore this is optimal [1, 5] (also
see [3]).

References

[1] Noga Alon, Yossi Matias, Mario Szegedy The Space Complexity of Approximating the Fre-
quency Moments. J. Comput. Syst. Sci. 58(1): 137–147, 1999.

[2] Philippe Flajolet, G. Nigel Martin Probabilistic counting algorithms for data base applications.
J. Comput. Syst. Sci., 31(2):182–209, 1985.

[3] T. S. Jayram, Ravi Kumar, D. Sivakumar: The One-Way Communication Complexity of

Hamming Distance. Theory of Computing 4(1): 129–135, 2008.

[4] Daniel M. Kane, Jelani Nelson, David P. Woodruff An optimal algorithm for the distinct
elements problem. In Proceedings of the Twenty-Ninth ACM SIGMOD-SIGACT-SIGART
Symposium on Principles of Database Systems (PODS), pages 41–52, 2010.

[5] David P. Woodruff. Optimal space lower bounds for all frequency moments. In SODA, pages
167–175, 2004.

6
 CS 229r: Algorithms for Big Data Fall 2015

Lecture 03, September 11th

Prof. Jelani Nelson Scribe: Vasileios Nakos

1 Overview

In the last lecture we looked at distinct elements, k-wise independence, geometric subsampling of
streams.
In this lecture we will see lower bounds on exact and approximate deterministic algorithms on data
streams, as well as the AMS sketch and Indyk’s algorithm for the Fp moments, when 0 ≤ p ≤ 2.

2 Main Section

2.1 Space lower bounds for streaming algorithms

We begin by an impossibility result. We consider the distinct elements problems, where you want
to find the number of distinct elements in a stream, where queries and additions are permitted.
We will denote by s the space of the algorithm, n the size of the universe from which the elements
come from, and m the length of the stream. We have the following result [1].

Theorem 1. There is no deterministic exact algorithm for computing number of distinct elements
in O(minn, m) space.

Proof. We are going to make an information-theoretic argument. Using a streaming algorithm

with space s for the current problem we are going to show how to encode {0, 1}n using only s bits.
In other words, we are going to construct an injective mapping from {0, 1}n to {0, 1}s . So, this
implies that s must be at least n and we are done. We look for procedures Dec,Enc such that
∀xDec(Enc(x)) = x and Enc(x) is a function from {0, 1}n to {0, 1}s .
For the encoding procedure, given a string x, create a stream containing and append i at the end
of the stream if xi = 1. Then Enc(x) is the memory content of the algorithm on that stream.
For the decoding procedure, we are going to look at each i and append it at the end of the stream(
feed it to the streaming algorithm) and query then the number of distinct elements. If the number
of distinct elements increases this implies that xi = 0, otherwise it implies that xi = 1. So we can
recover x completely and this finishes the proof.

We move on by showing that even approximate algorithms are hopeless for this problem.

Theorem 2. Any deterministic F0 algorithm that provides 1.1 approximation requires Ω(n) space

Proof. Suppose we had a colection C satisfying:

1
 • |C| ≥ 2cn , for some constant c < 1.
n
• ∀S ∈ C, |S| = 100
n 1
• ∀S 6= T ∈ C, |S ∩ T | ≤ 2000 ≤ 20 |S|

We leave the proof of existence of such a set later and we are moving to our lower bound. We are
going to use the algorithm to encode vectors xS ∀S ∈ C, where xS is the indicator vector of set S.
The lower bound follows as before since we must have s ≥ cn The encoding procedure is going to
be the same as before.
For the decoding procedure, we are going to iterate over all sets and test for each set S if it corre-
sponds to our initial encoded set( remember we are just doing an information-theoretic argument
and we do not care about the running running- we only care that this map is an injection). For
that, we keep at each time the memory contents of M of the streaming algorithm after having
inserted our initial string. Then for each S, we initialize our algorithm with memory contents M
and then feed element i if i ∈ S. It is easy to see that if S equals the initial encoded set, the number
of distinct elements does not increase by much, whereas if it is not it almost doubles. Taking also
into account the approximation guarantee of the algorithm we see for example that if S is not our
initial set then the number of distinct elements grows by 23 ( we can tune the parametersif needed).

We now only need to prove the existence of such a family of sets C. We are going to argue via
n
probabilistic method. We partition n into 100 intervals of length 100 each in the obvious way.
To form a set S we pick one number from each interval uniformly at random. Obviously, such
n
a set has size exactly 100 . For two sets S, T chosen uniformly at random as before let Ui be the
random variable that equals 1 if they have the same number chosen from interval i. Obviously
n
1 P 100 n 1
P [Ui = 1] = 100 . So the expected size of the intersection is just E i=1 = 100 · 100 . The probability
−c0
that this interesction is bigger than five times its mean is smaller than e n for some constant c0 ,
by a standard Chernoff bound. We can then apply a union bound over all possible intersections
and get the desired result.

2.2 Linear Sketches and upper bounds

2.2.1 What is a Linear Sketch?

We introduce the turnstile model in streaming algorithms. In this model we have a vector x ∈ Rn
that starts as the all zero vector and then a sequence of updates comes. Each update is of the form
(i, ∆), where ∆ ∈ R and i ∈ {1, .., n}. This corresponds to the operation xi ← xi + ∆.
Given a function f , we want to compute or approximate f (x).For example in the problem of distinct
elements ∆ is always 1 and f (x) = |i : xi 6= 0.
The common/only technique for designing turnstile algorithms is linear sketching. The idea is
to maintain in memory y = Πx, where Π ∈ Rm×n , a matrix that is short and fat. We care that
m < n, usually much smaller. We can see that y is m-dimensional so we can store it efficiently but
what about Π? If we need to store the whole Π in memory this will not lead to a better algorithm
in terms of space. So, there are two common ways in constructing and storing Π. The one is that
Π is deterministic and so we can easily compute Πij without keeping the whole matrix in memory.

2
Other documents randomly have
different content
cut our cables, and our ships had gone ashore; but in the end he
concluded to our request, bringing the twelve hostages to ten, which
with all speed on either part were received, with a writing from the
viceroy, signed with his hand and sealed with his seal, of all the
conditions concluded, and forthwith a trumpet blown, with
commandment that none of either part should inviolate the peace
upon pain of death; and, further, it was concluded that the two
generals of the fleet should meet, and give faith each to other for
the performance of the promises, which was so done.

"Thus, at the end of three days, all was concluded, and the fleet
entered the port, saluting one another as the manner of the sea
doth require. Thus, as I said before, Thursday we entered the port,
Friday we saw the fleet, and on Monday, at night, they entered the
port; then we laboured two days, placing the English ships by
themselves and the Spanish ships by themselves, the captains of
each part, and inferior men of their parts, promising great amity of
all sides; which, even as with all fidelity was meant of our part,
though the Spanish meant nothing less of their parts, but from the
mainland had furnished themselves with a supply of men to the
number of one thousand, and meant the next Thursday, being
September 23rd, at dinner-time, to set upon us of all sides. The
same Thursday, the treason being at hand, some appearance
showed, as shifting of weapons from ship to ship, planting and
bending of ordnance from the ship to the island where our men
were, passing to and fro of companies of men more than required
for their necessary business, and many other ill likelihoods, which
caused us to have a vehement suspicion, and therewithal sent to the
viceroy to inquire what was meant by it, which sent immediately
straight commandment to unplant all things suspicious, and also
sent word that he, in the faith of a viceroy, would be our defence
from all villainies. Yet we, not being satisfied with this answer,
because we suspected a great number of men to be hid in a great
ship of nine hundred tons, which was moored next unto the Minion,
sent again unto the viceroy the master of the Jesus, which had the
Spanish tongue, and required to be satisfied if any such thing were
or not; on which the viceroy, seeing that the treason must be
discovered, forthwith stayed our master, blew the trumpet, and of all
sides set upon us. Our men which were on guard ashore, being
stricken with sudden fear, gave place, fled, and sought to recover
succour of the ships; the Spaniards, being before provided for the
purpose, landed in all places in multitudes from their ships, which
they could easily do without boats, and slew all our men ashore
without mercy, a few of them escaping aboard the Jesus. The great
ship which had, by the estimation, three hundred men placed in her
secretly, immediately fell aboard the Minion, which, by God's
appointment, in the time of the suspicion we had, which was only
one half-hour, the Minion was made ready to avoid, and so, loosing
her headfasts, and hailing away by the sternfasts, she was gotten
out; thus, with God's help, she defended the violence of the first
brunt of these three hundred men. The Minion being passed out,
they came aboard the Jesus, which also, with very much ado and
the loss of many of our men, were defended and kept out. Then
were there also two other ships that assaulted the Jesus at the same
instant, so that she had hard work getting loose; but yet, with some
time, we had cut our headfasts, and gotten out by the sternfasts.
Now, when the Jesus and the Minion were gotten two ship-lengths
from the Spanish fleet, the fight began hot on all sides, so that
within one hour the admiral of the Spaniards was supposed to be
sunk, their vice-admiral burned, and one other of their principal
ships supposed to be sunk, so that the ships were little to annoy us.

"Then is it to be understood that all the ordnance upon the

island was in the Spaniards' hands, which did us so great annoyance
that it cut all the masts and yards of the Jesus in such sort, that
there was no hope to carry her away; also it sank our small ships,
whereupon we determined to place the Jesus on that side of the
Minion, that she might abide all the battery from the land, and so be
a defence for the Minion till night, and then to take such relief of
victual and other necessaries from the Jesus as the time would
suffer us, and to leave her. As we were thus determining, and had
placed the Minion from the shot of the land, suddenly the Spaniards
had fired two great ships which were coming directly to us, and
having no means to avoid the fire, it bred among our men a
marvellous fear, so that some said, 'Let us depart with the Minion;'
others said, 'Let us see whether the wind will carry the fire from us.'
But to be short, the Minion's men, which had always their sails in
readiness, thought to make sure work, and so without either consent
of the captain or master, cut their sail, so that very hardly I was
received into the Minion.

"The most part of the men that were left alive in the Jesus made
shift and followed the Minion in a small boat, the rest, which the
little boat was not able to receive, were enforced to abide the mercy
of the Spaniards (which I doubt was very little); so with the Minion
only, and the Judith (a small barque of fifty tons) we escaped, which
barque the same night forsook us in our great misery. We were now
removed with the Minion from the Spanish ships two bow-shots, and
there rode all that night. The next morning we recovered an island a
mile from the Spaniards, where there took us a north wind, and
being left only with two anchors and two cables (for in this conflict
we lost three cables and two anchors), we thought always upon
death, which ever was present; but God preserved us to a longer
time.

"The weather waxed reasonable, and the Saturday we set sail,

and having a great number of men and little victual, our hope of life
waxed less and less. Some desired to yield to the Spaniards, some
rather desired to obtain a place where they might give themselves to
the infidels; and some had rather abide, with a little pittance, the
mercy of God at sea. So thus, with many sorrowful hearts, we
wandered in an unknown sea by the space of fourteen days, till
hunger enforced us to seek the land; for hides were thought very
good meat; rats, cats, mice, and dogs, none escaped that might be
gotten; parrots and monkeys that were had in great prize, were
thought there very profitable if they served the turn of one dinner.
Thus in the end, on October 8th, we came to the land in the bottom
of the same bay of Mexico, in twenty-three degrees and a half,
where we hoped to have found habitations of the Spaniards, relief of
victuals, and place for the repair of our ship, which was so sore
beaten with shot from our enemies, and bruised with shooting of our
own ordnance, that our weary and weak arms were scarce able to
defend and keep out the water. But all things happened to the
contrary, for we found neither people, victual, nor haven of relief,
but a place where, having fair weather, with some peril we might
land a boat. Our people, being forced with hunger, desired to be set
aland, whereunto I concluded.

"And such as were willing to land I put apart, and such as were
desirous to go homewards I put apart, so that they were
indifferently parted, a hundred of one side and a hundred of the
other side. These hundred men were set on land with all diligence,
in this little place aforesaid, which being landed, we determined
there to refresh our water, and so with our little remain of victuals to
take the sea.

"The next day, having on land with me fifty of our hundred men
that remained, for the speedier preparing of our water aboard, there
arose an extreme storm, so that in three days we could by no means
repair our ships. The ship also was in such peril that every hour we
looked for shipwreck.

"But yet God again had mercy on us, and sent fair weather. We
got aboard our water, and departed October 16th, after which day
we had fair and prosperous weather till November 16th, which day,
God be praised, we were clear from the coast of the Indians and out
of the channel and gulf of Bahama, which is between the Cape of
Florida and the Islands of Cuba. After this, growing near to the cold
country, our men, being oppressed with famine, died continually, and
they that were left grew into such weakness that we were scarcely
able to manœuvre our ship; and the wind being always ill for us to
recover England, determined to go to Galicia, in Spain, with intent
there to relieve our company and other extreme wants. And being
arrived the last day of December, in a place near unto Vigo, called
Pontevedra, our men, with excess of fresh meat, grew into miserable
diseases, and died a great part of them. This matter was borne out
as long as it might be, but in the end, although there was none of
our men suffered to go on land, yet by access of the Spaniards our
feebleness was known to them. Whereupon they ceased not to seek
by all means to betray us; but with all speed possible we departed to
Vigo, where we had some help of certain English ships, and twelve
fresh men, wherewith we repaired our wants as we might, and
departing January 20th, 1568, arrived in Mounts Bay in Cornwall the
25th of the same month, praised be God therefore."

If all the misery and troublesome affairs of this sorrowful voyage

should be perfectly and thoroughly written, there should need a
painful man with his pen, and as great time as he had that wrote the
"Lives and Deaths of the Martyrs."

Sir John Hawkins rendered great service under Lord Howard in

1588, against the Spanish Armada, acting as rear admiral on board
H.M.S. Victory, where we are told he had as large a share of the
danger and honour of the day as any man in the fleet; for which he
deservedly received the honour of knighthood, and was particularly
commended by Queen Elizabeth. In 1590 he was sent, in
conjunction with Sir Martin Frobisher—each having a squadron of
men-of-war—to infest the coast of Spain, where they met with many
adventures but not much success. Later, a proposition was made to
the queen by Sir John Hawkins and Sir Francis Drake, to fit out an
expedition for the West Indies to harry the Spaniards, a proposition
which they backed with an offer to bear the greater part of the
expense themselves. The queen favoured the design, and the two
ablest seamen of the time sailed from Plymouth on August 28th,
1595, with a squadron of twenty-seven ships and barques, and a
force of two thousand five hundred men. Divided counsels seem to
have interfered with the success of this expedition, Sir John and Sir
Francis not agreeing as to the course to be pursued. A few days
before their departure they received notice from the queen that the
Plate fleet had safely arrived in Spain, with the exception of a single
galleon, which, having lost a mast, had been obliged to return to
Porto Rico; the capture of which she recommended to them as
practical without interfering with the general design of the
expedition. Sir John was for immediately executing the queen's
commands, but Sir Francis inclined first to go to the Canaries, in
which he prevailed over his friend and colleague, but not over his
enemies. In the meantime the Spaniards had sent five stout frigates
to bring away the damaged galleon from Porto Rico, which convoy,
falling in with the Francis, the sternmost of Sir John's ships, captured
her before she could receive assistance from the admiral. This is said
to have so affected the veteran Sir John, that he died on November
21st, 1595, soon after his vessel had sighted the island of Porto
Rico.

"Sir John Hawkins," says Dr. Campbell, "was the author of more
useful inventions, and introduced into the navy better regulations
than any officer who had borne command therein before his time.
One instance of this was the institution of that noble fund the Chest
of Chatham, which was the humane and wise contrivance of this
gentleman and Sir Francis Drake, and their scheme that seamen,
safe and successful, should, by a voluntary deduction from their pay,
give relief to the wants, and reward to those who are maimed in the
service of their country, was approved by the queen, and has been
adopted by posterity."
 THE WORTHY ENTERPRISE OF
JOHN FOX,
AN ENGLISHMAN, IN DELIVERING TWO HUNDRED AND SIXTY-
SIX CHRISTIANS OUT OF THE CAPTIVITY OF THE TURKS
AT ALEXANDRIA, THE 3RD OF JANUARY, 1577. BY RICHARD
HAKLUYT.

Richard Hakluyt was born at Eyton in Herefordshire in 1553, and

was educated at Westminster School and Christ Church, Oxford,
where he graduated B.A. in 1574, and M.A. in 1577, and lectured
publicly upon geography, showing "both the old imperfectly
composed, and the new lately reformed maps, globes, spheres, and
other instruments of this art."

In 1582 Hakluyt published his "Divers Voyages touching the

Discovery of America and the Lands adjacent unto the same, made
first of all by our Englishmen, and afterwards by the Frenchmen and
Bretons; and certain Notes of Advertisements for Observations,
necessary for such as shall hereafter make the like attempt." In
1583, having taken orders, he went to Paris as chaplain to the
English ambassador, Sir Edward Stafford, returning to England for a
short time in 1584, when he laid before the queen a paper entitled
"A particular Discourse concerning Western Discoveries, written in
the year 1584 by Richard Hakluyt, of Oxford, at the request and
direction of the right worshipful Mr. Walter Raleigh, before the
coming home of his two barks."

In 1587 he translated and published in London "A Notable

History containing Four Voyages made by certain French Captains
into Florida." In 1589 he published "The Principal Navigations,
Voyages and Discoveries of the English Nation"—a work developed
into three volumes folio, published in the years 1598, 1599, and
1600 as "The Principal Navigations, Voyages, Traffics, and
Discoveries of the English Nation." Hakluyt became Archdeacon of
Westminster in 1603, and died in 1616. He was buried in
Westminster Abbey.

Four stories from Hakluyt's voyages appear in this book. "The

Troublesome Voyage of the Jesus," which is included with "The Story
of Sir John Hawkins," p. 43; "The Voyage made to Tripolis in
Barbary," p. 79; "A True Report of a Worthy Fight," p. 91; and "The
Worthy Enterprise of John Fox" which here follows.

"Among our merchants here in England, it is a common voyage

to traffic to Spain; whereunto a ship called the Three Half Moons,
manned with eight and thirty men, well fenced with munitions, the
better to encounter their enemies withal, and having wind and tide,
set from Portsmouth 1563, and bended her journey towards Seville,
a city in Spain, intending there to traffic with them. And falling near
the Straits, they perceived themselves to be beset round about with
eight galleys of the Turks, in such wise that there was no way for
them to fly or escape away, but that either they must yield or else
be sunk, which the owner perceiving, manfully encouraged his
company, exhorting them valiantly to show their manhood, showing
them that God was their God, and not their enemies', requesting
them also not to faint in seeing such a heap of their enemies ready
to devour them; putting them in mind also, that if it were God's
pleasure to give them into their enemies' hands, it was not they that
ought to show one displeasant look or countenance there against;
but to take it patiently, and not to prescribe a day and time for their
deliverance, as the citizens of Bethulia did, but to put themselves
under His mercy. And again, if it were His mind and good will to
show His mighty power by them, if their enemies were ten times so
many, they were not able to stand in their hands; putting them,
likewise, in mind of the old and ancient worthiness of their
countrymen, who in the hardest extremities have always most
prevailed, and gone away conquerors; yea, and where it hath been
almost impossible. 'Such,' quoth he, 'hath been the valiantness of
our countrymen, and such hath been the mighty power of our God.'

"With such other like encouragements, exhorting them to

behave themselves manfully, they fell all on their knees, making
their prayers briefly unto God; who, being all risen up again,
perceived their enemies, by their signs and defiances, bent to the
spoil, whose mercy was nothing else but cruelty; whereupon every
man took him to his weapon.

"Then stood up one Grove, the master, being a comely man,

with his sword and target, holding them up in defiance against his
enemies. So likewise stood up the owner, the master's mate,
boatswain, purser, and every man well appointed. Now likewise
sounded up the drums, trumpets and flutes, which would have
encouraged any man, had he never so little heart or courage in him.

"Then taketh him to his charge John Fox, the gunner, in the
disposing of his pieces, in order to the best effect, and, sending his
bullets towards the Turks, who likewise bestowed their pieces thrice
as fast towards the Christians. But shortly they drew near, so that
the bowman fell to their charge in sending forth their arrows so thick
amongst the galleys, and also in doubling their shot so sore upon
the galleys, that there were twice as many of the Turks slain as the
number of the Christians were in all. But the Turks discharged twice
as fast against the Christians, and so long, that the ship was very
sore stricken and bruised under water; which the Turks, perceiving,
made the more haste to come aboard the ship: which, ere they
could do, many a Turk bought it dearly with the loss of their lives.
Yet was all in vain, boarded they were, where they found so hot a
skirmish, that it had been better they had not meddled with the
feast; for the Englishmen showed themselves men indeed in working
manfully with their brown bills and halberds, where the owner,
master, boatswain and their company stood to it so lustily, that the
Turks were half dismayed. But chiefly the boatswain showed himself
valiant above the rest, for he fared amongst the Turks like a wood
lion; for there was none of them that either could or durst stand in
his face, till at last there came a shot from the Turks which brake his
whistle asunder, and smote him on the breast, so that he fell down,
bidding them farewell, and to be of good comfort, encouraging
them, likewise, to win praise by death, rather than to live captives in
misery and shame, which they, hearing, indeed, intended to have
done, as it appeared by their skirmish; but the press and store of the
Turks were so great, that they were not long able to endure, but
were so overpressed that they could not wield their weapons, by
reason whereof they must needs be taken, which none of them
intended to have been, but rather to have died, except only the
master's mate, who shrunk from the skirmish, like a notable coward,
esteeming neither the value of his name, nor accounting of the
present example of his fellows, nor having respect to the miseries
whereunto he should be put. But in fine, so it was, that the Turks
were victors, whereof they had no great cause to rejoice or triumph.
Then would it have grieved any hard heart to see these infidels so
violently entreating the Christians, not having any respect of their
manhood, which they had tasted of, nor yet respecting their own
state, how they might have met with such a booty as might have
given them the overthrow; but no remorse hereof, or anything else
doth bridle their fierce and tyrannous dealing, but the Christians
must needs to the galleys, to serve in new officer; and they were no
sooner in them, but their garments were pulled over their ears and
torn from their backs, and they set to the oars.

"I will make no mention of their miseries, being now under their
enemies' raging stripes. I think there is no man will judge their fare
good, or their bodies unloaden of stripes, and not pestered with too
much heat, and also with too much cold; but I will go to my
purpose, which is to show the end of those being in mere misery,
which continually do call on God with a steadfast hope that He will
deliver them, and with a sure faith that He can do it.
 "Nigh to the city of Alexandria, being a haven town, and under
the dominion of the Turks, there is a road, being made very fencible
with strong walls, whereinto the Turks do customably bring their
galleys on shore every year, in the winter season, and there do trim
them, and lay them up against the spring-time; in which road there
is a prison, wherein the captives and such prisoners as serve in the
galleys are put for all that time, until the seas be calm and passable
for the galleys; every prisoner being most grievously laden with irons
on their legs, to their great pain and sore disabling of them to any
labour; into which prison were these Christians put and fast warded
all the winter season. But ere it was long, the master and the owner,
by means of friends, were redeemed, the rest abiding still in the
misery, while that they were all, through reason of their ill-usage and
worse fare, miserably starved, saving one John Fox, who (as some
men can abide harder and more misery than other some can, so can
some likewise make more shift, and work more duties to help their
state and living, than other some can do) being somewhat skilful in
the craft of a barber, by reason thereof made great shift in helping
his fare now and then with a good meal. Insomuch, till at the last
God sent him favour in the sight of the keeper of the prison, so that
he had leave to go in and out to the road at his pleasure, paying a
certain stipend unto the keeper, and wearing a lock about his leg,
which liberty likewise five more had upon like sufferance, who, by
reason of their long imprisonment, not being feared or suspected to
start aside, or that they would work the Turks any mischief, had
liberty to go in and out at the said road, in such manner as this John
Fox did, with irons on their legs, and to return again at night.

"In the year of our Lord 1577, in the winter season, the galleys
happily coming to their accustomed harbourage, and being
discharged of all their masts, sails, and other such furnitures as unto
galleys do appertain, and all the masters and mariners of them
being then nested in their own homes, there remained in the prison
of the said road two hundred three score and eight Christian
prisoners who had been taken by the Turks' force, and were of
fifteen sundry nations. Among which there were three Englishmen,
whereof one was named John Fox, of Woodbridge, in Suffolk, the
other William Wickney, of Portsmouth, in the county of
Southampton, and the third Robert Moore, of Harwich, in the county
of Essex; which John Fox, having been thirteen or fourteen years
under their gentle entreatance, and being too weary thereof,
minding his escape, weighed with himself by what means it might be
brought to pass, and continually pondering with himself thereof, took
a good heart unto him, in the hope that God would not be always
scourging His children, and never ceasing to pray Him to further His
intended enterprise, if that it should redound to His glory.

"Not far from the road, and somewhat from thence, at one side
of the city, there was a certain victualling house, which one Peter
Vuticaro had hired, paying also a certain fee unto the keeper of the
road. This Peter Vuticaro was a Spaniard born, and a Christian, and
had been prisoner above thirty years, and never practised any
means to escape, but kept himself quiet without touch or suspect of
any conspiracy, until that now this John Fox using much thither, they
brake one to another their minds, concerning the restraint of their
liberty and imprisonment. So that this John Fox, at length opening
unto this Vuticaro the device which he would fain put in practice,
made privy one more to this their intent; which three debated of this
matter at such times as they could compass to meet together;
insomuch that, at seven weeks' end they had sufficiently concluded
how the matter should be, if it pleased God to further them thereto;
who, making five more privy to this their device, whom they thought
that they might safely trust, determined in three nights after to
accomplish their deliberate purpose. Whereupon the same John Fox
and Peter Vuticaro, and the other five appointed to meet all together
in the prison the next day, being the last day of December, where
this John Fox certified the rest of the prisoners what their intent and
device was, and how and when they minded to bring that purpose to
pass, who thereunto persuaded them without much ado to further
their device; which, the same John Fox seeing, delivered unto them
a sort of files, which he had gathered together for this purpose by
the means of Peter Vuticaro, charging them that every man should
be ready, discharged of his irons, by eight of the clock on the next
day at night.

"On the next day at night, the said John Fox, and his five other
companions, being all come to the house of Peter Vuticaro, passing
the time away in mirth for fear of suspect till the night came on, so
that it was time for them to put in practice their device, sent Peter
Vuticaro to the master of the road, in the name of one of the
masters of the city, with whom this keeper was acquainted, and at
whose request he also would come at the first; who desired him to
take the pains to meet him there, promising him that he would bring
him back again. The keeper agreed to go with him, asking the
warders not to bar the gate, saying that he would not stay long, but
would come again with all speed.

"In the mean-season, the other seven had provided them of

such weapons as they could get in that house, and John Fox took
him to an old rusty sword-blade without either hilt or pommel, which
he made to serve his turn in bending the hand end of the sword
instead of a pommel; and the other had got such spits and glaves as
they found in the house.

"The keeper being now come unto the house, and perceiving no
light nor hearing any noise, straightway suspected the matter; and
returning backward, John Fox, standing behind the corner of the
house, stepped forth unto him; who, perceiving it to be John Fox,
said, 'O Fox, what have I deserved of thee that thou shouldest seek
my death?' 'Thou, villain,' quoth Fox, 'hast been a bloodsucker of
many a Christian's blood, and now thou shalt know what thou hast
deserved at my hands,' wherewith he lift up his bright shining sword
of ten years' rust, and stroke him so main a blow, as therewithal his
head clave asunder so that he fell stark dead to the ground.
Whereupon Peter Vuticaro went in and certified the rest how the
case stood with the keeper, and they came presently forth, and
some with their spits ran him through, and the other with their
glaves hewed him in sunder, cut off his head, and mangled him so
that no man should discern what he was.

"Then marched they toward the road, whereinto they entered

softly, where were five warders, whom one of them asked, saying,
who was there? Quoth Fox and his company, 'All friends.' Which
when they were all within proved contrary; for, quoth Fox, 'My
masters, here is not to every man a man, wherefore look you, play
your parts.' Who so behaved themselves indeed, that they had
despatched these five quickly. Then John Fox, intending not to be
barren of his enterprise, and minding to work surely in that which he
went about, barred the gate surely, and planted a cannon against it.

"Then entered they into the gaoler's lodge, where they found
the keys of the fortress and prison by his bedside, and there got
they all better weapons. In this chamber was a chest wherein was a
rich treasure, and all in ducats, which this Peter Vuticaro and two
more opening, stuffed themselves so full as they could between their
shirts and their skin; which John Fox would not once touch, and
said, 'that it was his and their liberty which he fought for, to the
honour of his God, and not to make a mart of the wicked treasure of
the infidels.' Yet did these words sink nothing unto their stomachs;
they did it for a good intent. So did Saul save the fattest oxen to
offer unto the Lord, and they to serve their own turn. But neither did
Saul escape the wrath of God therefore, neither had these that thing
which they desired so, and did thirst after. Such is God's justice. He
that they put their trust in to deliver them from the tyrannous hands
of their enemies, he, I say, could supply their want of necessaries.

"Now these eight, being armed with such weapons as they

thought well of, thinking themselves sufficient champions to
encounter a stronger enemy, and coming unto the prison, Fox
opened the gates and doors thereof, and called forth all the
prisoners, whom he set, some to ramming up the gate, some to the
dressing up of a certain galley which was the best in all the road,
and was called The Captain of Alexandria, whereinto some carried
masts, sails, oars, and other such furniture as doth belong unto a
galley.

"At the prison were certain warders whom John Fox and his
company slew, in the killing of whom there were eight more of the
Turks which perceived them, and got them to the top of the prison,
unto whom John Fox and his company were fain to come by ladders,
where they found a hot skirmish, for some of them were there slain,
some wounded, and some but scarred and not hurt. As John Fox
was thrice shot through his apparel, and not hurt, Peter Vuticaro and
the other two, that had armed them with the ducats, were slain, as
not able to wield themselves, being so pestered with the weight and
uneasy carrying of the wicked and profane treasure; and also divers
Christians were as well hurt about that skirmish as Turks slain.

"Amongst the Turks was one thrust through, who (let us not say
that it was ill-fortune) fell off from the top of the prison wall, and
made such a groaning that the inhabitants thereabout (as here and
there stood a house or two) came and questioned him, so that they
understood the case, how that the prisoners were paying their
ransoms; wherewith they raised both Alexandria, which lay on the
west side of the road, and a castle which was at the city's end next
to the road, and also another fortress which lay on the north side of
the road, so that now they had no way to escape but one, which by
man's reason (the two holds lying so upon the mouth of the road)
might seem impossible to be a way for them. So was the Red Sea
impossible for the Israelites to pass through, the hills and rocks lay
so on the one side, and their enemies compassed them on the other.
So was it impossible that the walls of Jericho should fall down, being
neither undermined nor yet rammed at with engines, nor yet any
man's wisdom, policy, or help, set or put thereunto. Such
impossibilities can our God make possible. He that held the lion's
jaws from rending Daniel asunder, yea, or yet from once touching
him to his hurt, cannot He hold the roaring cannons of this hellish
force? He that kept the fire's rage in the hot burning oven from the
three children that praised His name, cannot He keep the fire's
flaming blasts from among His elect?

"Now is the road fraught with lusty soldiers, labourers, and

mariners, who are fain to stand to their tackling, in setting to every
man his hand, some to the carrying in of victuals, some munitions,
some oars, and some one thing some another, but most are keeping
their enemy from the wall of the road. But to be short, there was no
time mis-spent, no man idle, nor any man's labour ill-bestowed or in
vain. So that in short time this galley was ready trimmed up.
Whereinto every man leaped in all haste, hoisting up the sails lustily,
yielding themselves to His mercy and grace, in Whose hands is both
wind and weather.

"Now is this galley afloat, and out of the shelter of the road; now
have the two castles full power upon the galley; now is there no
remedy but to sink. How can it be avoided? The cannons let fly from
both sides, and the galley is even in the middest and between them
both. What man can devise to save it? There is no man but would
think it must needs be sunk.

"There was not one of them that feared the shot which went
thundering round about their ears, nor yet were once scarred or
touched with five and forty shot which came from the castles. Here
did God hold forth His buckler, He shieldeth now this galley, and hath
tried their faith to the uttermost. Now cometh His special help; yea,
even when man thinks them past all help, then cometh He Himself
down from Heaven with His mighty power, then is His present
remedy most ready. For they sail away, being not once touched by
the glance of a shot, and are quickly out of the Turkish cannons'
reach. Then might they see them coming down by heaps to the
water's side, in companies like unto swarms of bees, making show to
come after them with galleys, bustling themselves to dress up the
galleys, which would be a swift piece of work for them to do, for that
they had neither oars, masts, sails, nor anything else ready in any
galley. But yet they are carrying into them, some into one galley, and
some into another, so that, being such a confusion amongst them,
without any certain guide, it were a thing impossible to overtake the
Christians; beside that, there was no man that would take charge of
a galley, the weather was so rough, and there was such an
amazedness amongst them. And verily, I think their god was amazed
thereat; it could not be but that he must blush for shame, he can
speak never a word for dulness, much less can he help them in such
an extremity. Well, howsoever it is, he is very much to blame to
suffer them to receive such a gibe. But howsoever their god behaved
himself, our God showed Himself a God indeed, and that He was the
only living God; for the seas were swift under His faithful, which
made the enemies aghast to behold them; a skilfuller pilot leads
them, and their mariners bestir them lustily; but the Turks had
neither mariners, pilot, nor any skilful master, that was in readiness
at this pinch.

"When the Christians were safe out of the enemy's coast, John
Fox called to them all, telling them to be thankful unto Almighty God
for their delivery, and most humbly to fall down upon their knees,
beseeching Him to aid them to their friends' land, and not to bring
them into another danger, since He had most mightily delivered
them from so great a thraldom and bondage.

"Thus when every man had made his petition, they fell
straightway to their labour with the oars, in helping one another
when they were wearied, and with great labour striving to come to
some Christian land, as near as they could guess by the stars. But
the winds were so contrary, one while driving them this way, another
while that way, so that they were now in a new maze, thinking that
God had forsaken them and left them to a greater danger. And
forasmuch as there were no victuals now left in the galley, it might
have been a cause to them (if they had been the Israelites) to have
murmured against their God; but they knew how that their God, who
had delivered Egypt, was such a loving and merciful God, as that He
would not suffer them to be confounded in whom He had wrought
so great a wonder, but what calamity soever they sustained, they
knew it was but for their further trial, and also (in putting them in
mind of their further misery) to cause them not to triumph and glory
in themselves therefor. Having, I say, no victuals in the galley, it
might seem one misery continually to fall upon another's neck; but
to be brief the famine grew to be so great that in twenty-eight days,
wherein they were on the sea, there died eight persons, to the
astonishment of all the rest.

"So it fell out that upon the twenty-ninth day after they set from
Alexandria, they fell on the Isle of Candia, and landed at Gallipoli,
where they were made much of by the abbot and monks there, who
caused them to stay there while they were well refreshed and eased.
They kept there the sword wherewith John Fox had killed the keeper,
esteeming it as a most precious relic, and hung it up for a
monument.

"When they thought good, having leave to depart from thence,

they sailed along the coast till they arrived at Tarento, where they
sold their galley, and divided it, every man having a part thereof.
And then they came afoot to Naples, where they departed asunder,
every man taking him to his next way home. From whence John Fox
took his journey unto Rome, where he was well entertained by an
Englishman who presented his worthy deed unto the pope, who
rewarded him liberally, and gave him letters unto the King of Spain,
where he was very well entertained of him there, who for this his
most worthy enterprise gave him in fee twenty pence a day. From
whence, being desirous to come into his own country, he came
thither at such time as he conveniently could, which was in the year
of our Lord God 1579; who being come into England went unto the
court, and showed all his travel unto the council, who considering of
the state of this man, in that he had spent and lost a great part of
his youth in thraldom and bondage, extended to him their liberality
to help to maintain him now in age, to their right honour and to the
encouragement of all true-hearted Christians."
THE BATTLE OFF DOVER.
See page 25.
(View larger image)
 THE STORY OF SIR FRANCIS
DRAKE.
BY JOHN CAMPBELL.

Francis Drake is said to have been born at Crowndale, near

Tavistock, about the year 1540. Both his birth and his parentage are
involved in obscurity; but it is probable that he was born of good
family in reduced circumstances, for he was declared by the King of
Arms in 1551 to have the right "by just descent and progeniture of
birth" to bear the arms of the Drakes of Ash; while it is clear that he
began life in a humble capacity. According to Camden, he was
apprenticed at an early age to the master of a small coasting vessel,
who, dying without issue, left the barque to him. We find also that at
the age of eighteen he was purser on board a ship trading to Biscay,
and at twenty he made a voyage to Guinea. At twenty-two he had
the honour to be appointed captain of the Judith, in the harbour of
St. John de Ullua, in the Gulf of Mexico, where he behaved most
gallantly in the glorious action, fought there under his kinsman, Sir
John Hawkins, described in the story of Sir John Hawkins, and
afterwards returned with him into England with a great reputation,
but not worth a single groat.

Upon this he conceived a design of making reprisals on the King

of Spain, which, some say, was put into his head by the minister of
his ship; and, to be sure, in sea-divinity, the case was clear; the King
of Spain's subjects had undone Mr. Drake, and therefore Mr. Drake
was at liberty to take the best satisfaction he could on the subjects
of the King of Spain. This doctrine, how rudely soever preached, was
very taking in England; and therefore he no sooner published his
design than he had numbers of volunteers ready to accompany him,
though they had no such pretence even as he had to colour their
proceedings. In 1570 he made his first expedition with two ships,
the Dragon and the Swan, and the next year in the Swan alone,
wherein he returned safe, with competent advantages, if not rich;
and, having now means sufficient to perform greater matters, as
well as skill to conduct them, he laid the plan of a more important
design with respect to himself and to his enemies.

This he put in execution on May 24th, 1572, on which day he

sailed from Plymouth, himself in a ship called the Pascha, of the
burden of seventy tons, and his brother, John Drake, in the Swan, of
twenty-five tons burden, their whole strength consisting of no more
than twenty-three men and boys; and, with this inconsiderable force,
on July 22nd he attacked the town of Nombre de Dios, which he
took in a few hours by storm, notwithstanding a dangerous wound
he received early in the action; yet upon the whole he was no great
gainer, for after a very brisk action he was obliged to betake himself
to his ships with very little booty. His next attempt was to plunder
the mules laden with silver which passed from Vera Cruz to Nombre
de Dios; but in this scheme too he was disappointed. However, he
attacked the town of Vera Cruz, carried it, and got some little booty.
In returning, he met unexpectedly with a string of fifty mules laden
with plate, of which he carried off as much as he could, and buried
the rest. In these expeditions he was greatly assisted by the
Simerons, a nation of Indians who were engaged in a perpetual war
with the Spaniards. The prince, or captain of these people, whose
name was Pedro, was presented by Captain Drake with a fine
cutlass, which he at that time wore, and to which he saw the Indian
had a mind. Pedro, in return, gave him four large wedges of gold,
which Drake threw into the common stock, saying, that "he thought
it but just that such as bore the charge of so uncertain a voyage on
his credit should share the utmost advantages that voyage
produced." Then embarking his men with all the wealth he had
obtained, which was very considerable, he bore away for England,
and was so fortunate as to sail in twenty-three days from Cape
Florida to the isles of Scilly, and thence without any accident to
Plymouth, where he arrived August 9th, 1573.

His success in this expedition, joined to his honourable behaviour

towards his owners, gained him a high reputation, and the use he
made of his riches still a greater; for, fitting out three stout frigates
at his own expense, he sailed with them to Ireland, where, under
Walter, Earl of Essex (the father of the unfortunate earl who was
beheaded), he served as a volunteer, and did many glorious actions.
After the death of his noble patron he returned to England, where
Sir Christopher Hatton, who was then vice-chamberlain to Queen
Elizabeth, and a great favourite, took him under his protection,
introduced him to Her Majesty, and procured him her countenance.
By this means he acquired facilities for undertaking that glorious
expedition which will render his name immortal. His first proposal
was to voyage into the South Seas through the Straits of Magellan,
an enterprise which hitherto no Englishman had ever attempted.
This project was well received at court, and in a short time Captain
Drake saw himself at the height of his wishes; for in his former
voyage, having had a distant prospect of the South Seas from the
top of a tree which he ascended for the purpose, he framed an
ardent prayer to God that he might sail an English ship in them,
which he found now an opportunity of attempting; the queen's
permission furnishing him with the means, and his own fame quickly
drawing to him a force sufficient.

The squadron with which he sailed on this extraordinary

undertaking consisted of the following ships: the Pelican,
commanded by himself, of the burden of one hundred tons; the
Elizabeth, vice-admiral, eighty tons, under Captain John Winter; the
Marygold, a barque of thirty tons, commanded by Captain John
Thomas; the Swan, a fly-boat of fifty tons, under Captain John
Chester; and the Christopher, a pinnace of fifteen tons, under
Captain Thomas Moon. In this fleet were embarked no more than
one hundred and sixty-four able men, and all the necessary
provisions for so long and dangerous a voyage; the intent of which,
however, was not openly declared. Thus equipped, on November
15th, 1577, about three in the afternoon, he sailed from Plymouth;
but a heavy storm taking him as soon as he was out of port, forced
him, in a very bad condition, into Falmouth, to refit; which, being
expeditiously performed, he again put to sea on the 13th of
December following. On the 25th of the same month he fell in with
the coast of Barbary; and on the 29th with Cape Verd; the 13th of
March he passed the equinoctial; the 5th of April he made the coast
of Brazil in 30° N. Lat. and entered the river De la Plata, where he
lost the company of two of his ships; but meeting them again, and
having taken out of them all the provisions they had on board, he
turned them adrift.

On August 20th, with his squadron reduced to three ships, he

entered the Straits of Magellan; on September 25th he passed them;
having then only his own ship, which, in the South Seas, he re-
named the Golden Hind. It may not be amiss to take notice here of a
fact very little known, as appearing in no relation of this famous
voyage. Sir Francis Drake himself reported to Sir Richard, son to Sir
John Hawkins, that meeting with a violent tempest, in which his ship
could bear no sail, he found, when the storm sank, he was driven
through or round the Straits into the latitude of fifty degrees. Here,
lying close under an island, he went on shore, and, leaning his body
over a promontory as far as he could safely, told his people, when he
came on board, he had been farther south than any man living. This
we find confirmed by one of our old chronicle writers, who farther
informs us that he bestowed on this island the name of Elizabetha,
in honour of his royal mistress. On November 25th he came to
Machao, in the latitude of thirty degrees, where he had appointed a
rendezvous in case his ships separated; but the Marygold had gone
down with all hands, and Captain Winter, having repassed the
Straits, had returned to England. Thence he continued his voyage
along the coasts of Chili and Peru, taking all opportunities of seizing
Spanish ships, or of landing and attacking them on shore, till his
crew were sated with plunder. While off the island of Mocha Drake
landed with some of his men to seek water; but the inhabitants,
mistaking them for Spaniards, attacked them, killed two of their
number and wounded several others, including Drake himself, who
was shot in the face with an arrow. As the surgeon of the Golden
Hind was dead, Drake had to be his own doctor as well as surgeon
to his crew. Realising that the attack had been made in mistake, and
not wishing to risk more casualties, Drake did not attempt to punish
the natives, but put to sea and made his way to Valparaiso, where
he made free with the stores and valuables he found, and then
proceeded further in search of his missing vessels, and finding
others which added to his booty; from one of which he took a
number of charts of seas then utterly unknown to the English
mariners. While pursuing this course he gained intelligence of a rich
ship laden with gold and silver for Panama, which he fell in with off
Cape Francisco on March 1st, 1579, and captured. The booty in this
case amounted to twenty-six tons of silver, eighty pounds of gold,
thirteen chests of money and a quantity of jewels and precious
stones; valued in all at nearly £200,000. Coasting North America to
the height of forty-eight degrees, he endeavoured to find a passage
back into our seas on that side, but being disappointed of what he
sought, he landed, and called the country New Albion, taking
possession of it in the name, and for the use of Queen Elizabeth;
and, having trimmed his ship, set sail thence, on September 29th,
1579, for the Moluccas; choosing this passage round, rather than
returning by the Straits of Magellan, owing to the danger of being
attacked at a great disadvantage by the Spaniards, and the lateness
of the season, whence dangerous storms and hurricanes were to be
apprehended.

On November 4th he sighted the Moluccas, and on December

10th made Celebes, where his ship unfortunately ran on a rock on
the 9th of January; whence, beyond all expectation, and in a manner
miraculously, they got off, and continued their course. On March
16th he arrived at Java, where he determined on returning directly
home. On March 25th, 1580, he put this design in execution, and on
June 15th doubled the Cape of Good Hope, having then on board his
Welcome to our website – the perfect destination for book lovers and
knowledge seekers. We believe that every book holds a new world,
offering opportunities for learning, discovery, and personal growth.
That’s why we are dedicated to bringing you a diverse collection of
books, ranging from classic literature and specialized publications to
self-development guides and children's books.

More than just a book-buying platform, we strive to be a bridge

connecting you with timeless cultural and intellectual values. With an
elegant, user-friendly interface and a smart search system, you can
quickly find the books that best suit your interests. Additionally,
our special promotions and home delivery services help you save time
and fully enjoy the joy of reading.

Join us on a journey of knowledge exploration, passion nurturing, and

personal growth every day!

ebookbell.com