Visit https://ebookultra.

com to download the full version and

explore more ebooks or textbooks

An Introduction to Search Engines and Web

Navigation 2nd Edition Mark Levene

_____ Click the link below to download _____

https://ebookultra.com/download/an-introduction-to-search-
engines-and-web-navigation-2nd-edition-mark-levene/

Explore and download more ebooks or textbooks at ebookultra.com

Here are some recommended products that we believe you will be
interested in. You can click the link to download.

WordPress for Web Developers An Introduction for Web

Professionals 2nd Edition Stephanie Leary

https://ebookultra.com/download/wordpress-for-web-developers-an-
introduction-for-web-professionals-2nd-edition-stephanie-leary/

Professional Microsoft Search FAST Search SharePoint

Search and Search Server Wrox Programmer to Programmer 1st
Edition Mark Bennett
https://ebookultra.com/download/professional-microsoft-search-fast-
search-sharepoint-search-and-search-server-wrox-programmer-to-
programmer-1st-edition-mark-bennett/

An Introduction to Psychological Science Canadian Edition

Mark Krause

https://ebookultra.com/download/an-introduction-to-psychological-
science-canadian-edition-mark-krause/

An Introduction to Native North America 5th Edition Mark

Q. Sutton

https://ebookultra.com/download/an-introduction-to-native-north-
america-5th-edition-mark-q-sutton/
An Introduction to Islam 2nd Edition David Waines

https://ebookultra.com/download/an-introduction-to-islam-2nd-edition-
david-waines/

Pathways in Theodicy an Introduction to the Problem of

Evil Mark S.M. Scott

https://ebookultra.com/download/pathways-in-theodicy-an-introduction-
to-the-problem-of-evil-mark-s-m-scott/

An Introduction to Thermodynamics and Statistical

Mechanics 2nd Edition Keith Stowe

https://ebookultra.com/download/an-introduction-to-thermodynamics-and-
statistical-mechanics-2nd-edition-keith-stowe/

An Introduction to Science and Technology Studies 2nd

Edition Sergio Sismondo

https://ebookultra.com/download/an-introduction-to-science-and-
technology-studies-2nd-edition-sergio-sismondo/

An Introduction to Health Psychology 2nd Edition Val

Morrison

https://ebookultra.com/download/an-introduction-to-health-
psychology-2nd-edition-val-morrison/
An Introduction to Search Engines and Web Navigation
2nd Edition Mark Levene Digital Instant Download
Author(s): Mark Levene
ISBN(s): 9780470526842, 047052684X
Edition: 2
File Details: PDF, 5.80 MB
Year: 2010
Language: english
 AN INTRODUCTION TO
SEARCH ENGINES AND
WEB NAVIGATION

MARK LEVENE
Department of Computer Science and Information Systems
Birkbeck University of London, UK

A JOHN WILEY & SONS, INC., PUBLICATION

AN INTRODUCTION TO
SEARCH ENGINES AND
WEB NAVIGATION
 AN INTRODUCTION TO
SEARCH ENGINES AND
WEB NAVIGATION

MARK LEVENE
Department of Computer Science and Information Systems
Birkbeck University of London, UK

A JOHN WILEY & SONS, INC., PUBLICATION

Copyright  2010 by John Wiley & Sons, Inc. All rights reserved

Published by John Wiley & Sons, Inc., Hoboken, New Jersey

Published simultaneously in Canada

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any
form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise,
except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without
either the prior written permission of the Publisher, or authorization through payment of the
appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers,
MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at www.copyright.com. Requests
to the Publisher for permission should be addressed to the Permissions Department, John Wiley &
Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at
http://www.wiley.com/go/permission.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best
efforts in preparing this book, they make no representations or warranties with respect to the
accuracy or completeness of the contents of this book and specifically disclaim any implied
warranties of merchantability or fitness for a particular purpose. No warranty may be created or
extended by sales representatives or written sales materials. The advice and strategies contained
herein may not be suitable for your situation. You should consult with a professional where
appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other
commercial damages, including but not limited to special, incidental, consequential, or other
damages.

For general information on our other products and services or for technical support, please contact
our Customer Care Department within the United States at (800) 762-2974, outside the United
States at (317) 572-3993 or fax (317) 572-4002.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print
may not be available in electronic formats. For more information about Wiley products, visit our
web site at www.wiley.com.

Library of Congress Cataloging-in-Publication Data:

Levene, M. (Mark), 1957-

An introduction to search engines and web navigation / Mark Levene.
p. cm.
ISBN 978-0-470-52684-2 (pbk.)
1. Internet searching. 2. Web search engines. I. Title.
ZA4230.L48 2010
025.0425– dc22
2010008435

Printed in Singapore

10 9 8 7 6 5 4 3 2 1
To my wife Sara and three children
Tamara, Joseph and Oren
 CONTENTS

PREFACE xiv

LIST OF FIGURES xvii

CHAPTER 1 INTRODUCTION 1

1.1 Brief Summary of Chapters 2

1.2 Brief History of Hypertext and the Web 3
1.3 Brief History of Search Engines 6

CHAPTER 2 THE WEB AND THE PROBLEM OF SEARCH 9

2.1 Some Statistics 10

2.1.1 Web Size Statistics 10
2.1.2 Web Usage Statistics 15
2.2 Tabular Data Versus Web Data 18
2.3 Structure of the Web 20
2.3.1 Bow-Tie Structure of the Web 21
2.3.2 Small-World Structure of the Web 23
2.4 Information Seeking on the Web 24
2.4.1 Direct Navigation 24
2.4.2 Navigation within a Directory 25
2.4.3 Navigation using a Search Engine 26
2.4.4 Problems with Web Information Seeking 27
2.5 Informational, Navigational, and Transactional Queries 28
2.6 Comparing Web Search to Traditional Information Retrieval 29
2.6.1 Recall and Precision 30
2.7 Local Site Search Versus Global Web Search 32
2.8 Difference Between Search and Navigation 34

CHAPTER 3 THE PROBLEM OF WEB NAVIGATION 38

3.1 Getting Lost in Hyperspace and the Navigation Problem 39

3.2 How Can the Machine Assist in User Search and Navigation 42
3.2.1 The Potential Use of Machine Learning Algorithms 42
3.2.2 The Naive Bayes Classifier for Categorizing Web Pages 43
3.3 Trails Should be First Class Objects 46
3.4 Enter Markov Chains and Two Interpretations of Its Probabilities 49
3.4.1 Markov Chains and the Markov Property 49
3.4.2 Markov Chains and the Probabilities of Following Links 50
3.4.3 Markov Chains and the Relevance of Links 52

vii
viii CONTENTS

3.5 Conflict Between Web Site Owner and Visitor 54

3.6 Conflict Between Semantics of Web Site and the Business Model 57

CHAPTER 4 SEARCHING THE WEB 60

4.1 Mechanics of a Typical Search 61

4.2 Search Engines as Information Gatekeepers of the Web 64
4.3 Search Engine Wars, is the Dust Settling? 68
4.3.1 Competitor Number One: Google 69
4.3.2 Competitor Number Two: Yahoo 70
4.3.3 Competitor Number Three: Bing 70
4.3.4 Other Competitors 72
4.4 Statistics from Studies of Search Engine Query Logs 73
4.4.1 Search Engine Query Logs 73
4.4.2 Search Engine Query Syntax 75
4.4.3 The Most Popular Search Keywords 77
4.5 Architecture of a Search Engine 78
4.5.1 The Search Index 79
4.5.2 The Query Engine 80
4.5.3 The Search Interface 81
4.6 Crawling the Web 81
4.6.1 Crawling Algorithms 82
4.6.2 Refreshing Web Pages 84
4.6.3 The Robots Exclusion Protocol 84
4.6.4 Spider Traps 85
4.7 What Does it Take to Deliver a Global Search Service? 85

CHAPTER 5 HOW DOES A SEARCH ENGINE WORK 91

5.1 Content Relevance 94

5.1.1 Processing Web Pages 94
5.1.2 Interpreting the Query 96
5.1.3 Term Frequency 96
5.1.4 Inverse Document Frequency 99
5.1.5 Computing Keyword TF–IDF Values 100
5.1.6 Caching Queries 102
5.1.7 Phrase Matching 102
5.1.8 Synonyms 102
5.1.9 Link Text 103
5.1.10 URL Analysis 104
5.1.11 Date Last Updated 104
5.1.12 HTML Structure Weighting 104
5.1.13 Spell Checking 105
5.1.14 Non-English Queries 106
5.1.15 Home Page Detection 107
5.1.16 Related Searches and Query Suggestions 107
5.2 Link-Based Metrics 108
5.2.1 Referential and Informational Links 109
5.2.2 Combining Link Analysis with Content Relevance 110
5.2.3 Are Links the Currency of the Web? 110
 CONTENTS ix

5.2.4 PageRank Explained 112

5.2.5 Online Computation of PageRank 116
5.2.6 Monte Carlo Methods in PageRank Computation 116
5.2.7 Hyperlink-Induced Topic Search 117
5.2.8 Stochastic Approach for Link-Structure Analysis 120
5.2.9 Counting Incoming Links 122
5.2.10 The Bias of PageRank against New Pages 123
5.2.11 PageRank within a Community 123
5.2.12 Influence of Weblogs on PageRank 124
5.2.13 Link Spam 125
5.2.14 Citation Analysis 127
5.2.15 The Wide Ranging Interest in PageRank 129
5.3 Popularity-Based Metrics 130
5.3.1 Direct Hit’s Popularity Metric 130
5.3.2 Document Space Modification 132
5.3.3 Using Query Log Data to Improve Search 132
5.3.4 Learning to Rank 133
5.3.5 BrowseRank 134
5.4 Evaluating Search Engines 136
5.4.1 Search Engine Awards 136
5.4.2 Evaluation Metrics 136
5.4.3 Performance Measures 138
5.4.4 Eye Tracking Studies 139
5.4.5 Test Collections 141
5.4.6 Inferring Ranking Algorithms 142

CHAPTER 6 DIFFERENT TYPES OF SEARCH ENGINES 148

6.1 Directories and Categorization of Web Content 150

6.2 Search Engine Advertising 152
6.2.1 Paid Inclusion 152
6.2.2 Banner Ads 153
6.2.3 Sponsored Search and Paid Placement 153
6.2.4 Behavioral Targeting 157
6.2.5 User Behavior 158
6.2.6 The Trade-Off between Bias and Demand 160
6.2.7 Sponsored Search Auctions 161
6.2.8 Pay per Action 165
6.2.9 Click Fraud and Other Forms of Advertising Fraud 166
6.3 Metasearch 168
6.3.1 Fusion Algorithms 169
6.3.2 Operational Metasearch Engines 170
6.3.3 Clustering Search Results 173
6.3.4 Classifying Search Results 175
6.4 Personalization 178
6.4.1 Personalization versus Customization 180
6.4.2 Personalized Results Tool 180
6.4.3 Privacy and Scalability 182
6.4.4 Relevance Feedback 182
6.4.5 Personalized PageRank 184
x CONTENTS

6.4.6 Outride’s Personalized Search 186

6.5 Question Answering (Q&A) on the Web 187
6.5.1 Natural Language Annotations 188
6.5.2 Factual Queries 190
6.5.3 Open Domain Question Answering 191
6.5.4 Semantic Headers 193
6.6 Image Search 194
6.6.1 Text-Based Image Search 195
6.6.2 Content-Based Image Search 196
6.6.3 VisualRank 198
6.6.4 CAPTCHA and reCAPTCHA 200
6.6.5 Image Search for Finding Location-Based Information 200
6.7 Special Purpose Search Engines 201

CHAPTER 7 NAVIGATING THE WEB 209

7.1 Frustration in Web Browsing and Navigation 211

7.1.1 HTML and Web Site Design 211
7.1.2 Hyperlinks and Surfing 211
7.1.3 Web Site Design and Usability 212
7.2 Navigation Tools 213
7.2.1 The Basic Browser Tools 213
7.2.2 The Back and Forward Buttons 214
7.2.3 Search Engine Toolbars 215
7.2.4 The Bookmarks Tool 216
7.2.5 The History List 219
7.2.6 Identifying Web Pages 219
7.2.7 Breadcrumb Navigation 221
7.2.8 Quicklinks 222
7.2.9 Hypertext Orientation Tools 223
7.2.10 Hypercard Programming Environment 224
7.3 Navigational Metrics 225
7.3.1 The Potential Gain 226
7.3.2 Structural Analysis of a Web Site 228
7.3.3 Measuring the Usability of Web Sites 229
7.4 Web Data Mining 230
7.4.1 Three Perspectives on Data Mining 230
7.4.2 Measuring the Success of a Web Site 231
7.4.3 Web Analytics 233
7.4.4 E-Metrics 233
7.4.5 Web Analytics Tools 234
7.4.6 Weblog File Analyzers 235
7.4.7 Identifying the Surfer 236
7.4.8 Sessionizing 237
7.4.9 Supplementary Analyses 237
7.4.10 Markov Chain Model of Web Site Navigation 238
7.4.11 Applications of Web Usage Mining 242
7.4.12 Information Extraction 244
7.5 The Best Trail Algorithm 245
7.5.1 Effective View Navigation 245
 CONTENTS xi

7.5.2 Web Usage Mining for Personalization 246

7.5.3 Developing a Trail Engine 246
7.6 Visualization that Aids Navigation 252
7.6.1 How to Visualize Navigation Patterns 252
7.6.2 Overview Diagrams and Web Site Maps 253
7.6.3 Fisheye Views 255
7.6.4 Visualizing Trails within a Web Site 257
7.6.5 Visual Search Engines 258
7.6.6 Social Data Analysis 259
7.6.7 Mapping Cyberspace 262
7.7 Navigation in Virtual and Physical Spaces 262
7.7.1 Real-World Web Usage Mining 262
7.7.2 The Museum Experience Recorder 264
7.7.3 Navigating in the Real World 265

CHAPTER 8 THE MOBILE WEB 272

8.1 The Paradigm of Mobile Computing 273

8.1.1 Wireless Markup Language 274
8.1.2 The i-mode Service 275
8.2 Mobile Web Services 277
8.2.1 M-commerce 277
8.2.2 Delivery of Personalized News 278
8.2.3 Delivery of Learning Resources 281
8.3 Mobile Device Interfaces 282
8.3.1 Mobile Web Browsers 282
8.3.2 Information Seeking on Mobile Devices 284
8.3.3 Text Entry on Mobile Devices 284
8.3.4 Voice Recognition for Mobile Devices 286
8.3.5 Presenting Information on a Mobile Device 287
8.4 The Navigation Problem in Mobile Portals 291
8.4.1 Click-Distance 291
8.4.2 Adaptive Mobile Portals 292
8.4.3 Adaptive Web Navigation 294
8.5 Mobile Search 295
8.5.1 Mobile Search Interfaces 296
8.5.2 Search Engine Support for Mobile Devices 298
8.5.3 Focused Mobile Search 299
8.5.4 Laid Back Mobile Search 300
8.5.5 Mobile Query Log Analysis 301
8.5.6 Personalization of Mobile Search 302
8.5.7 Location-Aware Mobile Search 303

CHAPTER 9 SOCIAL NETWORKS 309

9.1 What is a Social Network? 311

9.1.1 Milgram’s Small-World Experiment 312
9.1.2 Collaboration Graphs 313
9.1.3 Instant Messaging Social Network 314
xii CONTENTS

9.1.4 The Social Web 314

9.1.5 Social Network Start-Ups 316
9.2 Social Network Analysis 320
9.2.1 Social Network Terminology 320
9.2.2 The Strength of Weak Ties 322
9.2.3 Centrality 322
9.2.4 Web Communities 324
9.2.5 Pajek: Large Network Analysis Software 326
9.3 Peer-to-Peer Networks 326
9.3.1 Centralized P2P Networks 327
9.3.2 Decentralized P2P Networks 328
9.3.3 Hybrid P2P Networks 330
9.3.4 Distributed Hash Tables 331
9.3.5 BitTorrent File Distribution 331
9.3.6 JXTA P2P Search 332
9.3.7 Incentives in P2P Systems 332
9.4 Collaborative Filtering 333
9.4.1 Amazon.com 333
9.4.2 Collaborative Filtering Explained 334
9.4.3 User-Based Collaborative Filtering 335
9.4.4 Item-Based Collaborative Filtering 337
9.4.5 Model-Based Collaborative Filtering 338
9.4.6 Content-Based Recommendation Systems 339
9.4.7 Evaluation of Collaborative Filtering Systems 340
9.4.8 Scalability of Collaborative Filtering Systems 341
9.4.9 A Case Study of Amazon.co.uk 341
9.4.10 The Netflix Prize 342
9.4.11 Some Other Collaborative Filtering Systems 346
9.5 Weblogs (Blogs) 347
9.5.1 Blogrolling 348
9.5.2 Blogspace 348
9.5.3 Blogs for Testing Machine Learning Algorithms 349
9.5.4 Spreading Ideas via Blogs 349
9.5.5 The Real-Time Web and Microblogging 350
9.6 Power-Law Distributions in the Web 352
9.6.1 Detecting Power-Law Distributions 353
9.6.2 Power-Law Distributions in the Internet 355
9.6.3 A Law of Surfing and a Law of Participation 355
9.6.4 The Evolution of the Web via Preferential Attachment 357
9.6.5 The Evolution of the Web as a Multiplicative Process 359
9.6.6 The Evolution of the Web via HOT 360
9.6.7 Small-World Networks 361
9.6.8 The Robustness and Vulnerability of a Scale-Free Network 366
9.7 Searching in Social Networks 369
9.7.1 Social Navigation 369
9.7.2 Social Search Engines 370
9.7.3 Navigation Within Social Networks 373
9.7.4 Navigation Within Small-World Networks 375
9.7.5 Testing Navigation Strategies in Social Networks 379
9.8 Social Tagging and Bookmarking 379
 CONTENTS xiii

9.8.1 Flickr—Sharing Your Photos 380

9.8.2 YouTube—Broadcast Yourself 380
9.8.3 Delicious for Social Bookmarking 382
9.8.4 Communities Within Content Sharing Sites 382
9.8.5 Sharing Scholarly References 383
9.8.6 Folksonomy 383
9.8.7 Tag Clouds 384
9.8.8 Tag Search and Browsing 385
9.8.9 The Efficiency of Tagging 388
9.8.10 Clustering and Classifying Tags 389
9.9 Opinion Mining 390
9.9.1 Feature-Based Opinion Mining 391
9.9.2 Sentiment Classification 392
9.9.3 Comparative Sentence and Relation Extraction 393
9.10 Web 2.0 and Collective Intelligence 393
9.10.1 Ajax 394
9.10.2 Syndication 395
9.10.3 Open APIs, Mashups, and Widgets 396
9.10.4 Software as a Service 398
9.10.5 Collective Intelligence 399
9.10.6 Algorithms for Collective Intelligence 401
9.10.7 Wikipedia—The World’s Largest Encyclopedia 402
9.10.8 eBay—The World’s Largest Online Trading Community 407

CHAPTER 10 THE FUTURE OF WEB SEARCH AND NAVIGATION 419

BIBLIOGRAPHY 424

INDEX 463
 PREFACE

MOTIVATION
Searching and navigating the web have become part of our daily online lives.
Web browsers and the standard navigation tools embedded in them provide a
showcase of successful software technology with a global user-base, that has
changed the way in which we search for and interact with information. Search
engine technology has become ubiquitous, providing a standard interface to the
endless amount of information that the web contains. Since the inception of the
web, search engines have delivered a continuous stream of innovations, satisfy-
ing their users with increasingly accurate results through the implementation of
advanced retrieval algorithms and scalable distributed architectures. Search and
navigation technologies are central to the smooth operation of the web and it is
hard to imagine finding information without them. Understanding the computa-
tional basis of these technologies and the models underlying them is of paramount
importance both for IT students and practitioners.
There are several technical books on web search and navigation but the
ones I have seen are either very academic in nature, that is, targeted at the post-
graduate student or advanced researcher, and therefore have a limited audience,
or they concentrate on the user interface and web site usability issues, ignoring
the technicalities of what is happening behind the scenes. These books do not
explain at an introductory level how the underlying computational tools work.
This book answers the need for an introductory, yet technical, text on the topic.
My research into web search and navigation technologies started during
the beginning of the 1990s just before the internet boom, when, together with
my colleagues, we began looking at hypertext as a model for unstructured (or
semistructured) data connected via a network of links, much in the same way web
pages are connected. Of particular interest to us was the infamous “navigation
problem” when we lose our way navigating (or what has become known as
“surfing”) through the myriad of information pages in the network. Tackling this
problem has provided continued impetus for my research.
In a wider context, the activity of information seeking, that is, the process
we go through when searching and locating information in order to augment our
state of knowledge, has been of major concern to all involved in the development
of technologies that facilitate web interaction.
I have been using browser navigation tools and search engines since their
early days, and have been fascinated by the flow of new ideas and the improve-
ments that each new tool has delivered. One of my aims in this text is to demystify
the technology underlying the tools that we use in our day-to-day interaction with

xiv
 PREFACE xv

the web, and another is to inform readers about upcoming technologies, some of
which are still in the research and development stage.
I hope that this book will instill in you some of my enthusiasm for the pos-
sibilities that these technologies have and are creating to extend our capabilities
of finding and sharing information.

AUDIENCE AND PREREQUISITES

The book is intended as an undergraduate introductory text on search and navi-
gation technologies, but could also be used to teach an option on the subject. It
is also intended as a reference book for IT professionals wishing to know how
these technologies work and to learn about the bigger picture in this area.
The course has no formal prerequisites, all that is required is for the learner
to be a user of the web and to be curious to know how these technologies work.
All the concepts that are introduced are explained in words, and simple examples
from my own experience are given to illustrate various points. Occasionally,
to add clarity to an important concept, a formula is given and explained. Each
chapter starts with a list of learning objectives and ends with a brief bullet-pointed
summary. There are several exercises at the end of each chapter. Some of these
aim to get the student to explore further issues, possibly with a reference which
can be followed up, some get the student to discuss an aspect of the technology,
and others are mini-projects (which may involve programming) to add to the
student’s understanding through a hands-on approach. The book ends with a
set of notes containing web addresses to items mentioned in the book, and an
extensive bibliography of the articles and books cited in the book.
Readers should be encouraged to follow the links in the text and to discover
new and related links that will help them understand how search and navigation
tools work, and to widen their knowledge with related information.

TIMELINESS
I believe that due to the importance of the topic it is about time that such a book
should appear. Search and navigation technologies are moving at a very fast pace
due to the continued growth of the web and its user base, and improvements in
computer networking and hardware. There is also strong competition between
different service providers to lock-in users to their products. This is good news
for web users, but as a result some of the numerics in the text may be out of
date. I have qualified the statistics I have given with dates and links, which can
be found in the notes, so the reader can follow these to get an up-to-date picture
and follow the trends. I do not expect the core technologies I have covered to
radically change in the near future and I would go so far as to claim that in
essence they are fundamental to the web’s working, but innovation and new
ideas will continue to flourish and mold the web’s landscape.
xvi PREFACE

If you find any errors or omissions please let me know so that I can list
them on the book’s web site. I will also be grateful to receive any constructive
comments and suggestions, which can be used to improve the text.

ACKNOWLEDGMENTS
First I would like to thank my wife and family who have been extremely support-
ive throughout this project, encouraging me to put in the extra hours needed to
complete such a task. I would also like to thank my colleagues at the Department
of Computer Science and Information Systems at Birkbeck, who have read and
commented on parts of the book. Special thanks to my editors at Wiley, Lucy
Hitz and George Telecki, who have patiently guided me through the publica-
tion process. Finally, I would like to thank the reviewers for their constructive
comments.
The people who have built the innovative technologies that drive today’s
web are the real heroes of the revolution that the World Wide Web has brought
upon us. Without them, this book could not have been written. Not only in terms
of the content of the book, but also in terms of the tools I have been using daily
to augment my knowledge on how search and navigation technologies work in
practice.

Mark Levene
London, June 2010
 LIST OF FIGURES

1.1 Bush’s memex 3

1.2 Nelson’s Xanadu 5
1.3 Engelbart’s NLS 5
1.4 Mosaic browser initially released in 1993 6

2.1 A database table versus an example web site 19

2.2 Partial map of the Internet (2005) 21
2.3 A view of web pages related to www.dcs.bbk.ac.uk 22
2.4 Bow-tie shape of the Web 23
2.5 Information seeking 26
2.6 Recall versus precision 31

3.1 The navigation problem 40

3.2 Being lost in hyperspace 41
3.3 Example web site 47
3.4 Four trails within the web site 47
3.5 Search engine results for the query “mark research” submitted to Google 48
3.6 Relevant trail for the query “mark research” 49
3.7 Markov chain for example web site 51
3.8 Two trails in the Markov chain 51
3.9 Probabilities of the four trails 52
3.10 Scoring web pages 53
3.11 Constructing a Markov chain from scores 54

4.1 Query “computer chess” submitted to Google 62

4.2 Results for “computer chess” from Google 62
4.3 Relevant category from the directory for “computer chess” from Google 64
4.4 Results for phrase “computer chess” from Google 65
4.5 Simplified search engine architecture 78
4.6 Basic crawler algorithm 82

5.1 Results from “∼ food ∼ facts” Google 103

5.2 Web pages both A and B have a link to C 109
5.3 Example web site to illustrate PageRank 113
5.4 Example of a rank sink 113
5.5 Page A is an authority 118
5.6 Page A is a hub 119
5.7 The base set for the HITS technique 120
5.8 Web pages related to Yahoo 129
5.9 Heatmap of Google’s golden triangle 140

xvii
xviii LIST OF FIGURES

6.1 Query “chess” submitted to Overture 155

6.2 Query “netbook” submitted to Google 155
6.3 Query “netbook” submitted to Yahoo 156
6.4 Query “netbook” submitted to Bing 156
6.5 Query “chess” submitted to Clusty 173
6.6 Query “chess” submitted to CatS 176
6.7 The right-hand window, generated by PResTo! when the query “salsa” is
typed into Google 181
6.8 Question “who is the prime minister of the uk?” submitted to Ask Jeeves 190
6.9 Query “who is the prime minister of the uk?” submitted to Wolfram
Alpha 191
6.10 Similarity graph generated from the top 1000 search results of
“Mona-Lisa” 199

7.1 Navigation tools built-in to the browser 214

7.2 GroupLab Internet Explorer History System 220
7.3 Google sitelinks for query “Birkbeck” 222
7.4 Nielsen’s hypertext implemented in Apple’s Hypercard 225
7.5 Example web site 226
7.6 Pie chart showing the page request to my site 236
7.7 Pie chart showing the keywords that led to referrals to my site 236
7.8 Markov chain example 239
7.9 Frequent trails 240
7.10 Markov chain constructed from search engine scores 247
7.11 A navigation tree 249
7.12 Trail search for the query “knowledge technologies” 250
7.13 Nav-search for the query “knowledge technologies” 250
7.14 Visual search for the query “knowledge technologies” 251
7.15 The WebBrain user interface 254
7.16 Open Directory home page 254
7.17 Hierarchical site map 255
7.18 A web site for an online journal application 256
7.19 Query-specific site map 256
7.20 RSVP browser on a small screen 258
7.21 VISVIP visualization of user trails laid over the web site 259
7.22 Sea anemones in an aquarium 260
7.23 Anemone web site usage visualization 260
7.24 Carrot’s topic map for the query “beatles” 261
7.25 Kartoo’s topic map for the query “beatles” 261
7.26 The trail of a visitor to a museum 263
7.27 Navigation engine architecture 266

8.1 The Daily Learner adaptive news agent 279

8.2 A map in the Knowledge Sea system 282
8.3 Standard mobile phone 12-key keypad 285
8.4 A web page thumbnail overview (a) and a detailed view of a selected
segment (b) as presented by SmartView 288
8.5 Accordion page summary on Power Browser 289
8.6 Example summary generated by BCL Technologies (www.bcltechnologies.
com) summarizer 290
Other documents randomly have
different content
 CHAPTER VIII
ANXIETIES AND HARDSHIPS DURING OUR STAY IN
THE RUFIJI COUNTRY

O
UR march from Kibata was on the first day carried out according
to plan. On the following day I rode ahead with a few
companions, in the expectation that the troops who had
several native guides with them would not fail to find the way.
In the Kissi mountains we came upon large numbers of natives who,
however, were very timid and often deserted their flourishing rice
plantations on our approach. Later in the day I regretted that I did
not appropriate some of this abundant produce for our own use.
During the midday heat we rested at Pori. Some of my companions
who knew the country called my attention to the acid Mbinji fruit,
which we found very refreshing. Unfortunately we did not know at
that time that the stone of this fruit, when roasted, makes an
excellent dish, tasting like our hazel-nut. The heat was overpowering,
but as we were in the neighbourhood of the enemy patrols we had to
keep a sharp look-out. The springs and water-courses were now dried
up; after a long search we at last found a small pool of dirty water,
which, however, we were told was not injurious to health. Towards
evening we reached the great deserted settlement. Here we were
fortunate enough to find a negro in the employ of the German
Government, who informed us that we were at Ungwara, our
destination for that day. After we had walked through the place, the
man showed us a pool near which we pitched our camp. My old black
cook, the bearded Baba, well known to many East Africans, had very
nearly kept up with our horses, and, following our trail, soon arrived.
He had soon prepared his uzeli (boiled rice), and was sitting
contentedly by the fire. We watched him enviously, for we had
nothing, and were waiting for our baggage and the troops. But no
one came and we lay down, hungry, to sleep. The friend in need,
however, was approaching in the shape of a splendid sable-antelope,
which in the brilliant moonlight was coming down to drink. Almost
simultaneously the rifles of two of my companions, van Booyen and
Nieuwenhuizen, experienced Boer hunters, who had become
Germans, rang out. We sprang from our blankets as though we had
received an electric shock, and within a short time the first pieces of
delicate flesh were roasting on the spit.
On the following day we reached Lake Utungi, where Captain
Feilke was awaiting us, and we refreshed ourselves with bread, coffee
and sausage made from antelope flesh. There was still no trace of the
troops. They had lost us in Pori, and almost all completely lost their
bearings. One detachment did not get into touch with us until several
days later, when they came upon our telephone line in the
neighbourhood of Utete. In view of the difficulty of communication, it
had hitherto been impossible to get an accurate estimate of the state
of our supplies. I had expected to find well-filled depots at
Mpanganya on Lake Utungi and in the neighbourhood of Madaba.
This was why I had pressed on out of the fertile country north of the
lower Rufiji through Mpanganya to the line-of-communication area.
The question of supplies had developed quite differently from my
expectations.
In the line-of-communication area, in addition to the large
numbers of bearers necessary for the transport of war material to the
south, a numerous personnel was maintained, who were employed on
road-making, building grass huts and for other purposes. Even in the
small depots there were always a number of men who, whenever
possible, did nothing but fetch supplies, which they ate themselves.
Often the supplies were even fetched by others who, in their turn,
had to be fed. In many places it was almost the case that a load of
supplies collected and forwarded by the fighting troops in the north
finally landed in a small depot, and were devoured by these people
who had nothing else to do. In view of the difficulties of transport,
and the great distances, even the energy and thoroughness of
Captain Stemmermann, who had taken charge of the depots, did not
succeed in detecting and putting a stop to these abuses. Further,
there were too many people in Africa whose propensity for diverting
valuable energy to non-essentials to the prejudice of the really
important things that it would take a very strong broom to sweep
them away. The general result of all these obstacles was that
thousands and thousands of useless mouths were devouring the
supplies which had been collected with great effort in the region
occupied by the fighting forces. The depot did nothing for the
supplies, but, on the other hand, lived on them, and the most serious
point was that the moment was at hand when the areas from which
the supplies were drawn would have to be evacuated by the fighting
forces. It was a difficult situation. It was necessary to lose no time in
putting under cultivation the territory we were then occupying—that
is to say, the country round Madaba and Livale, and in the southern
parts of the Protectorate, which were likely to be the scene of the
subsequent operations. But months must elapse before any results
could be obtained from these measures. During these months we
should have to remain on the Rufigi and live there. Here, it is true,
some hundreds of acres of maize were standing, but even these
would require months to ripen. Until this time came the force could
not move south; it would have to remain in the unproductive area
which it was then occupying.
The accomplishment of this task was difficult. The order had at
once to be given for the removal of every man who was not
absolutely indispensable for carrying on the war during the next few
months. This meant that thousands of bearers and workmen in the
line-of-communication area were sent home. The most serious
drawback to this step had to be reckoned with; we were sending over
to the enemy thousands of men from whom he was bound to gain
detailed information as to our strength, the condition of our supplies
and our internal organization. Nor was it enough to cut down the
personnel of our lines of communication. The non-combatant
personnel of the companies was also reduced. Among other things, it
was laid down that henceforward no European should have more than
five native attendants. That sounds a generous allowance to
European ears, but under African conditions native attendance is
really indispensable to the European. He requires at least one man or
boy to cook for him and attend to his personal needs, and, in
addition, it must be remembered that all baggage, kit, rations,
blankets and tent-material, has to be carried whenever he moves.
When one considers that in peace-time a travelling official on a long
safari (journey) took with him from eleven to thirteen bearers, in
addition to two or three personal servants, it will be understood how
drastic this new order was and what a storm of indignation it aroused.
Fortunately I was in a position, when appealed to on grounds of
health and decency, to point to the fact that I myself had for months
managed with three, or at a pinch two, loads—that is, four negroes—
and had kept in good health. I am still particularly grateful to those
regimental officers who, as on so many other occasions, saw the
necessity of this vexatious regulation and set the example. They
upheld the tradition of our officer-corps by not claiming any special
comforts for themselves, and were the first to submit to the
unavoidable discomfort. I believe that among all the soldiers and non-
combatants up to the highest civilian official, there is not one who still
condemns this order, at first so strongly opposed.
But reduction of the number of food-consumers alone was not
sufficient to solve the problem of existence; the supplies would not go
round. It was already obvious that the supplies from the area of the
fighting force, which were, of course, being worked at high pressure,
would not suffice to feed us until the new harvest at the end of
March. After close and mature consideration, we found it impossible
to avoid the necessity of cutting down the rations, a measure which
went very much against the grain, as even the native, if he is to be
relied on, must be well fed. This gave rise to a fresh and much
stronger outburst of indignation. From all sides came telegrams and
messages to say that it was impossible to get the calories of
nourishment necessary for a fighting man from the daily cereal ration,
fixed at six hundred grammes of meal. But the hard fact had to be
faced that only a certain quantity was available, and we must make
the best of it. The reduction of the cereal ration could not be avoided.
For the rest, each man and each company would have to try to make
good the shortage by hunting, which in this region, where game
abounded, could be managed with the exercise of a little agility. But
logic is apt to go to pieces with many people when it comes to a
question of daily food, and many did not scruple to lay the whole
blame for the at times barely sufficient ration at the door of the
wicked commander-in-chief, and even to do all they could to have the
daily ration increased to its former amount. This I had to bear calmly,
and I made my own observations as to who were the men to make
the best of an unavoidable necessity and who were not.
In carrying out these drastic measures new difficulties were
encountered. A crowd of Askari women had followed the force, and
had attached themselves to various camps on the Rufiji, where they
were very comfortable. I was most anxious to send them south,
where the question of supplies was less difficult. The necessary
transport was arranged for, and the women were given rations for the
march. After one short day’s march, however, the women simply lay
down, and declared that they could go no further. Their rations, which
were intended to last a considerable time, were all eaten by the third
day, and they were crying out for more. Some even went so far as to
attack and beat the European who was in charge of the transport.
Even under a dark skin the gentler sex did not always scruple to make
full use of their prerogatives, which are usually justified.
Finally we got over this difficulty, and a tolerable solution was
found to the ration problem. The Askari, to whom the position was
explained, saw the difficulty and were very reasonable. Skilled
hunters were sent to the different hunting-grounds, and the empty
stomachs from time to time more generously filled. I remember that
with us on Lake Utungi our two hundred blacks in one day completely
devoured a big buffalo and an elephant. It was often found possible
to give a piece of antelope to the passing caravans of bearers.
In the course of February the stores in our supply dumps, of
which I took stock every day, ran out. I began to fear that for reasons
of supply we would not be able to wait for the ripening of the corn on
the Rufiji. In that case, not only would the harvest be lost, but the
crops growing further south could not be used to the best advantage.
There we should have to use the grain that was actually ripe and pass
on, leaving the unripe portion standing. A lucky chance came to my
aid in this dilemma. I went one day from Lake Utungi to Mpanganya
to see Captain Tafel, who was handling the tactical and commissariat
problems there with admirable efficiency. I spent the night in his
camp, and he set before me an excellent dish of young maize
prepared like asparagus. This led us to speak of the maize fields of
Mpanganya and the neighbourhood. These were full of women and
other natives who had swarmed over them like a flock of birds, and
were living on the young, unripe corn. This was as bad economy as
well could be, but it gave me the idea that in case of need the maize
crops could be largely used before they were ripe. This need very
soon occurred, and an experiment with the ears which had ripened
most showed that these could be artificially dried and a very good
meal made from them. After this, the ripest ears were gathered daily,
and as the whole crop ripened the food situation improved from day
to day. By 1st March it was found possible to increase the ration to
seven hundred grammes, or nearly the normal allowance.
The increasing severity of the whole campaign called for a more
intensive and energetic exploitation of our food resources; the slow,
deliberate supply methods of the civil authorities, which had sufficed
for the first phase of the campaign, were no longer adequate. Twice,
at Kissaki and on the Rufiji I had been put in a most difficult position
with regard to supplies, which had almost made it impossible to carry
on the operations. A more efficient supply service which would know
the military needs, look ahead and work more quickly and
energetically was a vital necessity for the further carrying on of the
campaign. Fortunately I was able to convince the Governor on this
point, and, as a result, a new supply detachment was raised from the
force, and sent ahead to Massassi, via Livale. They established
several subsidiary detachments, which were attached to the
administrative stations in the Lindi area, and in this way worked side
by side with the civil authorities in organizing, and, later, in carrying
out, the cultivation and storing of food. In this way the desired
impregnation of the supplies and transport service with the necessary
military spirit was completely attained.
At this time there was no appreciable shortage of kit, and there
was also an adequate supply of arms and ammunition.
With a view to the envelopment of the enemy at Mkalinzo, where
he was reported to be in strong force, Captain Otto had marched his
detachment south from Kungulio. North of Mawa he covered the
fertile area of Madaba, and the line of transport and telephone
communication running from Lake Utungi, via Mawa to Madaba. On
24th January, 1917, Captain Otto was attacked north of Mawa by
several battalions of the Nigerian brigade. The enemy was beaten off
with heavy losses and pursued several miles through the bush to an
entrenched position, where he took refuge. The troops under Captain
Schulz, who had been left behind after our departure from Kibeta,
were gradually withdrawn to Ungwara. They had been reinforced and
relieved from time to time after the fighting in the region of the
Kibeta-Utete-Kissi mountains. Strong enemy forces—identified as an
infantry brigade—had followed them. In spite of his numerical
superiority, the single engagements were very costly, and for the
most part unfavourable for the enemy. Captains von Lieberman,
Goering and Koehl, and numerous patrol leaders on many occasions
completely routed more than twice their number of Indian or negro
troops, and captured rifles, machine guns and ammunition. The long
war had produced a large number of capable leaders, and their
example, as in the case of Lieutenant Kroeger, who was afterwards
killed, roused unbounded enterprise and daring. Over and over again,
and without stopping to ask the strength of the opposing force, this
officer, followed by a handful of men with fixed bayonets and cheering
loudly, had charged the enemy in the thick of the bush. He had even
trained the Askari. Several of these distinguished themselves as patrol
leaders, and when later the brave Effendi, of the 4th Field Company,
with his patrol, routed an entire Indian company, we owed the
success to this training at Ungwara.
Our line of communication to the south, passing through Madeba
and Livale, was in danger from a strong enemy force west of Kibata,
and it was necessary that we should afford it adequate protection.
This meant a gradual move south of our forces from the Rufiji,
especially as our supplies on this part of the river were coming to an
end and the rainy season was at hand.
It was particularly important that we should not evacuate this part
of the Rufiji until the rains had set in. This would mean a considerable
gain in time for us, as, during the actual rainy season and
immediately after, the operations would, of necessity, come to a
standstill, and the corn, particularly the mtema (millet), would have
time to ripen.
When the migrations of the ants warned us that the rains were at
hand, orders were given, as a precautionary measure, that the
women, children and non-combatants should as far as possible be
transferred to the north bank of the Rufiji, and thence transported to
Dar-es-Salaam. This step, which the approaching rains and the state
of the supplies made necessary, aroused much discontent, which I
was obliged to treat with the same indifference as the previous
outbursts of indignation. I am, however, still of the opinion that the
timely removal of these people was much better for them than
spending part of the rainy season on the drenched ground or in
flooded dwellings with insufficient food.
The rains, which set in at the end of March, were particularly
heavy in 1917. The site of our camp, which was slightly elevated,
became an island, from which access to the outer world was only
possible by boat through the Rufiji wood. A number of people were
drowned in the wood during the rains; others had to take refuge for
days in the trees. The water rose so high that in Mpanganya it
reached the high-lying dwellings of the Europeans, and invaded the
hospitals and disturbed every kind of filth. It was impossible for
women and children, sick and wounded to remain, and after the
withdrawal of the troops they had to give themselves up to the
English, who took pity on their need, and provided them with food
and transport.
The majority of the troops marched south out of the flooded
districts on the Rufiji and Lake Utungi in good time, after using up the
available crops almost to the last grain. The evacuation was carried
out gradually and in echelon; the greater part of the troops were
assembled in Mpotora, which was occupied by Captain Rothe, in a
fortified camp, with his two companies which had defeated the
Portuguese at Nevala. Only a few small detachments were left on the
Rufiji, and these were gradually reduced to the strength of patrols.
Four days’ march east of Madaba the detachments of Koehl and
Goering had the opportunity of some successful skirmishes against
enemy detachments on the western edge of the Matumbi Mountains.
Gradually, however, all these detachments were brought to Mpotora,
and only Captain Otto remained in the higher regions of Madaba.
 CHAPTER IX
THE END OF THE FRONTIER DEFENCE IN THE
SUBSIDIARY THEATRES

I
N August, 1916, Major Kraut had gradually retired from Kilossa on
Mahenge, leaving only Schoenfeld’s division at Kidodi, on the
Ruaha. Captain Braunschweig’s force was embodied in Major
Kraut’s command. Of these Captain Falkenstein, with the 5th
Field Company, had retired, at the end of May, 1916, from Ipyana,
and Captain Aumann, with his company, from the Mbozi region in the
direction of Lupembe and Maubire. During the retirement there was
continual skirmishing. Our weak divisions had to make a stand
against the pursuing enemy, at least a brigade strong.
At the end of June, 1916, Captain Braunschweig, who was then
at Dodoma, was sent through Iringa, and his force was strengthened
to five companies by the addition of the Kondoa troops and others
brought from Dar-es-Salaam, including the two companies from
Langenburg. One hundred of the crew of the Königsberg from Dar-
es-Salaam and a field howitzer were added to this force. At Malangali
he accepted battle with the enemy, and apparently inflicted heavy
losses. Then, however, he evacuated the position, and abandoned
the howitzer, which was difficult to move, first making it useless. The
difficulties of Braunschweig’s position were increased by the action of
an important Wahehe chief in his rear, who rebelled and went over
to the enemy with all his people and cattle. Captain Braunschweig
then retired on Mahenge, fighting a succession of minor rearguard
skirmishes, and put himself under the orders of Major Kraut.

Fig. xv. March of Major-General Wahle in the West

After numerous minor engagements Major Kraut’s retiring

divisions established themselves on the line of the Ruhudje and
Ruaha rivers. In the fertile region round Mahenge the supplies were
excellent, even after the evacuation of a large part of the rice-field
west of the Ruhudje. On this river the enemy had established a
strongly fortified camp at Mkapira. With our insufficient resources it
was impossible to take this position by force, but there was a chance
that by cutting the enemy’s line of communication with Lupembe, we
might force him to evacuate the camp owing to shortage of food.
Major Kraut crossed the river with five companies and a light field
gun, and occupied a position in a semi-circle of hills in the enemy’s
rear and right across his line of communication. In the enemy’s front
weak forces covered the river bank on the Mahenge side.
Unfortunately the fortified positions of our companies were so
extended that, owing to the difficulties of the country, there was no
guarantee that support could be brought up in time. On the——,
before daybreak, the 10th Company on the right wing was surprised
by a heavy enemy attack. The enemy also cleverly took the
company’s position in the rear and, after inflicting heavy losses, put
the machine guns out of action. On the left wing Lieutenant von
Schroetter’s company was also attacked from all sides, and had to
cut its way out with the bayonet, losing the light field gun and a
machine gun. In view of the heavy casualties of the enemy, Major
Kraut would have been able to remain on the west bank of the
Ruhudje, in spite of this partial disaster, but fighting could be heard
from the direction of Lupembe, where the 25th Field Company was
covering his rear. Major Kraut thought, wrongly, that there, too, a
sharp attack had been made, and, therefore, retired again to the
east bank of the Ruhudje. To his astonishment the enemy’s strong
entrenchments at Mkapira were found to be evacuated a few days
later, the enemy having withdrawn in the night. Closer inspection
showed that he had suffered heavy losses in the recent fighting.
This, however, was not enough to explain his withdrawal; this riddle
was not solved until later, on the appearance of General Wahle, with
whom no communication had been established.
In expectation of the opening of the big operations of 1916 the
reinforcements that had been provisionally sent to Victoria Nyanza,
Ruanda, the Russissi and the Tanganyika area were brought back
and embodied into our main forces along the North railway. A single
command for these minor theatres of war was required, and with
this object a “western command” was established under Major-
General Wahle, who for the most part co-ordinated and directed
these operations from Tabora. In April and May, 1916, when the
British main forces in the Kilima Njaro area had completed their
march, and, after the rainy season, were beginning a fresh advance
to the south, English and Belgians from Muansa, Lake Kiwu, the
Russissi and Bismarckburg began to advance concentrically on
Tabora through these minor theatres of war. Our weak divisions
retired on this place.
Major von Langenn retired at once from Tschangugu to Issawi,
followed by Captain Wintgens from Kissenji. Heavy casualties were
inflicted on the pursuing Belgian brigades in successful rearguard
actions. The German detachment later continued its retirement on
Mariahilf. The danger to our district from the strong Belgian forces
on our heels had been correctly estimated by Captain Gudovius.
When in June, 1916, strong English forces advanced across the
Kagera, he retired south from Bukoba with his division. Owing to the
difficulties of communication and getting information, a part of his
force unfortunately ran upon strong Belgian forces in Ussuwi district.
Captain Gudovius himself was wounded in the abdomen and fell into
the enemy’s hands. The engagement went badly for us and cost us
heavy losses. Individual bodies of the detachment, however,
managed to fight their way through to Muansa and Uschirombo.
In the middle of July, 1916, the English succeeded in effecting a
surprise landing with about a brigade in the neighbourhood of
Muansa. There, too, there was some skirmishing, favourable to us,
and there the Commanding Officer, Captain von Chappuis, retired in
the direction of Tabora. The troops from Muansa and those under
Major von Langenn and Captain Wintgens established a new front,
approximately on the line Schinjanga-St. Michael, and repulsed
several Belgian attacks. Captain Zimmer had sunk the steamer
Goetzen at Kigoma and blown up the Wami. He then retired slowly
along the railway to Tabora. Captain Hering von Usumbura followed
suit. The fact that the operations were nearing Tabora gave General
Wahle the opportunity to bring up quickly part of the troops from the
north of Tabora, to make a dash west by rail and retire again at
once. In this raid the 8th Field Company completely routed a Belgian
battalion west of Tabora, and Wintgens’ detachment brought off a
successful surprise attack west and north of Tabora. These minor
victories were often considerable, and on several different days of
skirmishing the enemy losses amounted to hundreds; several light
howitzers were also captured in these raids.
On 2nd June, 1916, the 29th Field Company was surrounded in
its fortified position in the Namema mountains. In fighting his way
through, the brave company commander, Lieutenant Franken, was
severely wounded and taken prisoner. Lieutenant Hasslacher retired
step by step on Tabora. In an affair of patrols south of this place he
met with a hero’s death.
In this way the troops of the western command were actually
assembled at Tabora, and the moment had come for a systematic
retirement to the south-east. These last operations and the capture
of Tabora were not known at Headquarters until long afterwards.
There was no means of communication with the western command.
Major-General Wahle was aware that this retirement of our main
forces was of first importance for the Mahenge area. Accordingly he
gave orders for the march. At first the railway could be used for
supplies and transport. The eastern column, under Major von
Langenn, marched on Iringa, the centre column, under Captain
Wintgens, on Madibira, and the western column, under Lieutenant
Huebener, on Ilembule. Major Wahle accompanied the centre
column. In this way they came upon the line of communication
between Neu-Langenburg and Iringa, and the enemy’s dumps along
this line. Huebener’s detachment lost touch, and surrendered, being
enveloped by a superior force of the enemy at Ilembule. Langenn’s
detachment was most unfortunately surprised by a burst of fire while
crossing a ford near Iringa and lost heavily. The subsequent attack
on Iringa was also costly and without success.
Wintgens’ detachment surprised enemy dumps and columns near
Madibira, and also captured a gun and some wireless apparatus. In
spite of several days of stubborn fighting, they were unable to take
Lupembe and the surrounding farms. The influence of Wahle’s
advance immediately made itself felt in the Mahenge district. The
apparently strong enemy troops, who from their fortified positions at
Mkapira had carried out the successful raid against Major Kraut, now
felt themselves seriously threatened in their rear. They evacuated
their strong positions and retired on Lupembe. General Wahle took
over the command of all the forces at Mahenge.
At the end of 1916 the troops of General Wahle’s western
command were grouped round Mahenge. From here he directed the
operations extending approximately to the line Ssongea—Lupembe—
Iringa—Kidodi.
It has been said that all touch with General Wahle had been lost
since July, 1916, until in October, 1916, his patrols joined up with
those of Major Kraut south of Iringa.
It was not, therefore, until after the fighting at Mkapira that
Major Kraut, and through him Headquarters, learned of General
Wahle’s advance; the development of the situation made a very
different impression on the enemy. He must have regarded the
advance of General Wahle’s columns against the English line of
communication from Iringa to Langenburg, and Major Kraut’s
accidentally simultaneous threat to Mkapira, as a widely-planned
joint operation, which was seriously endangering his troops at
Mkapira, even after Major Kraut had withdrawn to the east bank of
the Ruhudje. He avoided this danger by a hasty retirement from
Mkapira in a westerly direction.
General Wahle’s columns at once concentrated in the Lupembe-
Mkapira area. No news was received of Huebener’s western column.
Its capitulation was not known until much later.
Welcome though this reinforcement of the forces in the west was,
there were difficulties of supplies, and it became necessary to put
under cultivation a considerable area, stretching almost to Ssongea.
Major Grawert’s detachment advanced to Likuju on the Ssongea—
Liwale road, that of Major Kraut to the Mpepo region and Captain
Wintgens’ surrounded an enemy detachment in a fortified camp at
Kitenda. The enemy quickly marched to the relief of this force, but
the relieving troops were driven off with heavy losses. At the same
time the position of Grawert’s detachment took a very unfavourable
turn. The enemy had succeeded in driving off this force’s live-stock.
As other supplies in the district were scanty, Major von Grawert,
exaggerating the difficulties of supply, thought his position hopeless
and surrendered in January, 1917. A transportable 8.8 air naval gun
which had been brought to Lihuju with great difficulty fell into the
enemy’s hands, as well as a number of good machine guns. In
reality the position of Grawert’s force does not appear to have been
so desperate as he supposed; at any rate, a strong patrol under
Sergeant-Major Winzer, who refused to surrender, made its way
south without being molested by the enemy, and, a few days later,
found abundant supplies at small cost in the districts west of
Tunduru. The conduct of this patrol gave further proof that there is
almost always a way out, even of an apparently hopeless position, if
the leader makes up his mind to face the risks.
Meanwhile General Wahle’s supply difficulties were increasing.
Whether they could have been modified by ruthlessly reducing the
number of non-combatants, as had been done on the Rufiji, or
whether the material welfare of the western command could have
been substantially improved by greater care in procuring and
rationing the available resources, could not be decided from my
position on Lake Utungi. The temporary telegraph to Mahenge was
very inefficient and often interrupted, and it took several days to get
a despatch through from General Wahle in Mahenge to the troops.
This made it difficult for me to get a view of the situation from the
incomplete information at hand. Suffice it to say that the difficulties
of supply in Mahenge were regarded as so acute that it was not
considered possible to keep such strong forces concentrated there,
and part of them would have to be withdrawn.
Kraut’s and Wintgens’ forces were marched west to Gumbiro,
whence they were to press on across the Ssongea-Wiedhafen road.
It was thought that they would find adequate supplies in the
mountains south of Ssongea. The report of this move reached me
too late for me to interfere. From Gumbiro Captain Wintgens turned
north and, near Lake Rukwa, successfully engaged an enemy column
which had been following him; on nearing Tabora he got typhus and
was taken prisoner. Captain Naumann led the force on until finally he
surrendered to the pursuing enemy column near Kilima-Njaro
towards the end of 1917. It is to be regretted that this operation,
carried out with so much initiative and determination, became
separated so far from the main theatre of war as to be of little use.
Major Kraut had separated from Captain Wintgens in Gumbiro,
and, carrying out General Wahle’s orders, had marched south. There
was no difficulty about crossing the line of communication Ssongea-
Wiedhafen, but as the enemy had strongly entrenched and secured
his supply dumps, no booty was captured. In the open little was to
be found in March, 1917, the poorest season of the year, a few
months before the new harvest. After some rearguard actions
against English troops a success was scored in a surprise attack on
the small Portuguese camp at Mitomoul, on the Rovuma. Major Kraut
then followed the river downstream to Tunduru and himself came to
Headquarters to report. Two of his companies remained at Tunduru
to guard the fertile district. The other three marched further east and
were temporarily taken over by Captain Loof at Lindi.
 CHAPTER X
LINDI AND KILWA

T
HE operations of the last few months had narrowed the area
from which supplies for the troops could be obtained. The
productive areas of Lupembe, Iringa, Kissaki and the lower
Rufiji had been lost, and the newly-occupied districts included
wide stretches of barren land. The productivity of the more fertile
areas was for the most part unknown; for instance, it was not known
until during the subsequent operations what yield could be expected
south-west of Kilwa and south-west of Livale, for example. At that
time I only had a general idea that the eastern part of the Lindi area
was very fertile and known as the granary of the colony. But this
fertile region, owing to its nearness to the coast, was in a very
precarious position, and it was already necessary to consider what
should be done if it were lost.
Our eyes naturally turned to the Portuguese territory across the
Rovuma, but we had even less information about this than about
parts of the German colony. Fortunately, however, a number of
Portuguese chiefs had immigrated into German territory out of
hatred for their oppressors, and, apart from this, we Germans
enjoyed a very good reputation among the intelligent natives of
Portuguese East Africa, many of whom worked on German
plantations. Thus we were able to get at least an approximate
picture of the district east of Lake Nyassa, and to take it as probable
that south of the steppe-like zone of the Rovuma, in the
neighbourhood of Mwembe, several days’ march, broad and thinly
populated, lay a fertile region. An expeditionary force of a few
hundred rifles under Major von Stuemer, crossed the Rovuma south
of Tundura, and quickly took possession of Mwembe from where our
patrols explored the banks of Lake Nyassa as far as the
neighbourhood of Fort Jackson, and east half-way to Port Amelia.

Fig. xvi. March of Main Force during operations on

interior lines west of Lindi, June to November, 1917. Fig.
xvii. Battle of Mahiwa.
 In view of the difficulty of communication—messengers from the
telegraph station at Livale took about three days to get to Tunduru
and five from there to Mwembe—it was difficult to get a clear idea of
the situation at Mwembe. We had no definite news until Lieutenant
Brucher personally reported at Headquarters in January, 1917. The
European potatoes he brought with him gave us good hopes that
supplies could be expected there. He reported that the country was
fertile, as was also the region round Tunduru, where the war had so
far hardly been felt. There were still large numbers of eggs and fowls
in the richly cultivated district. When Brucher slept on the ground in
Tunduru, this was regarded as a piece of bravado by the inhabitants,
so little did they know about war. In view of the difficulties of
transport and the constant movement of the troops, it became
increasingly necessary to make the force less and less dependent on
their inadequate line of communication. With this object the forces of
Captains Goering and von Lieberman were also moved to the region
south of Kilwa, where, according to the stories of some Europeans in
the Kiturika mountains, there was plenty of food. In order to relieve
the transport of supplies from the rear the troops were marched off
to Kilwa without waiting for further investigation, and it was
fortunate that the reports as to the fertility of this district were
realized. In order to take the enemy, who had already moved some
small forces half-way to Livale, as far as possible from south of his
point of debarkation, and at the same time to secure the fertile
districts south of Kilwa to Mbemkuru, Goering’s and von Lieberman’s
divisions made a détour from Mpotora southwards and pressed
forward, Goering’s force following the coast straight to Kilwa, and
von Lieberman keeping further west and making for the Kilwa-Livale
road. A weaker force followed this road to Kilwa and served as a
reserve for the patrols, which several times surprised the enemy in
his camps and threw him back. Our patrols were soon swarming in
the neighbourhood of Kilwa. Several enemy dumps were surprised
and part of the garrisons killed. On one of these occasions brave
Sergeant-Major Struwe, who was afterwards killed, skilfully forced
his way, with a large part of the 3rd Field Company, inside a dump,
and, taking cover behind the sacks of flour, inflicted heavy losses on
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebookultra.com