Experience Seamless Full Ebook Downloads for Every Genre at ebookmeta.

com

Using Python for Introductory Econometrics 1st

Edition Florian Heiss Daniel Brunner

https://ebookmeta.com/product/using-python-for-introductory-
econometrics-1st-edition-florian-heiss-daniel-brunner/

OR CLICK BUTTON

DOWNLOAD NOW

Explore and download more ebook at https://ebookmeta.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Health Econometrics Using Stata 1st Edition Partha Deb

https://ebookmeta.com/product/health-econometrics-using-stata-1st-
edition-partha-deb/

ebookmeta.com

Environmental Econometrics Using Stata 1st Edition

Christopher F. Baum

https://ebookmeta.com/product/environmental-econometrics-using-
stata-1st-edition-christopher-f-baum/

ebookmeta.com

Introduction to Python for Econometrics, Statistics and

Data Analysis. 5th Edition Kevin Sheppard.

https://ebookmeta.com/product/introduction-to-python-for-econometrics-
statistics-and-data-analysis-5th-edition-kevin-sheppard/

ebookmeta.com

Rick Steves England 10th Edition Rick Steves

https://ebookmeta.com/product/rick-steves-england-10th-edition-rick-
steves/

ebookmeta.com
Step by Step Hair Transplantation Expand the Dermatologist
s Knowledge on Concepts and Techniques of Fue Hair
Transplantation 1st Edition Pradeep Sethi Editor Arika
Bansai Editor Abhinav Kumar Editor Sarita Sanke Editor
https://ebookmeta.com/product/step-by-step-hair-transplantation-
expand-the-dermatologist-s-knowledge-on-concepts-and-techniques-of-
fue-hair-transplantation-1st-edition-pradeep-sethi-editor-arika-
bansai-editor-abhinav-kumar-editor/
ebookmeta.com

Manias, Panics, and Crashes: A History of Financial

Crisis, 8th 8th Edition Robert Z. Aliber

https://ebookmeta.com/product/manias-panics-and-crashes-a-history-of-
financial-crisis-8th-8th-edition-robert-z-aliber/

ebookmeta.com

Learning Disabilities Sourcebook (Health Reference), 6th

Edition Angela L. Williams

https://ebookmeta.com/product/learning-disabilities-sourcebook-health-
reference-6th-edition-angela-l-williams/

ebookmeta.com

Big Data Analytics and Knowledge Discovery: 23rd

International Conference, DaWaK 2021, Virtual Event,
September 27–30, 2021, Proceedings 1st Edition Golfarelli
https://ebookmeta.com/product/big-data-analytics-and-knowledge-
discovery-23rd-international-conference-dawak-2021-virtual-event-
september-27-30-2021-proceedings-1st-edition-golfarelli/
ebookmeta.com

The Story of Black Military Officers 1861 1948 1st Edition

Krewasky A. Salter

https://ebookmeta.com/product/the-story-of-black-military-
officers-1861-1948-1st-edition-krewasky-a-salter/

ebookmeta.com
South Asian Women and International Relations 1st Edition
Abhiruchi Ojha

https://ebookmeta.com/product/south-asian-women-and-international-
relations-1st-edition-abhiruchi-ojha/

ebookmeta.com
Using Python for Introductory Econometrics
1st edition

Florian Heiss
Daniel Brunner
Using Python for Introductory Econometrics
© Florian Heiss, Daniel Brunner 2020. All rights reserved.

Companion website: http://www.UPfIE.net

Address:
Universitätsstraße 1, Geb. 24.31.01.24
40225 Düsseldorf, Germany
 1.6.4. Random Draws from Prob-
ability Distributions . . . . 50
1.7. Confidence Intervals and Statisti-
cal Inference . . . . . . . . . . . . . 52
Contents 1.7.1. Confidence Intervals . . . .
1.7.2. t Tests . . . . . . . . . . . .
52
55
1.7.3. p Values . . . . . . . . . . . 57
1.8. Advanced Python . . . . . . . . . . 60
Preface . . . . . . . . . . . . . . . . . . . 1
1.8.1. Conditional Execution . . . 60
1. Introduction 3 1.8.2. Loops . . . . . . . . . . . . . 60
1.1. Getting Started . . . . . . . . . . . 3 1.8.3. Functions . . . . . . . . . . 61
1.1.1. Software . . . . . . . . . . . 3 1.8.4. Object Orientation . . . . . 62
1.8.5. Outlook . . . . . . . . . . . 66
1.1.2. Python Scripts . . . . . . . . 4
1.9. Monte Carlo Simulation . . . . . . 66
1.1.3. Modules . . . . . . . . . . . 8
1.9.1. Finite Sample Properties of
1.1.4. File Names and the Work-
Estimators . . . . . . . . . . 66
ing Directory . . . . . . . . 9
1.9.2. Asymptotic Properties of
1.1.5. Errors and Warnings . . . . 9
Estimators . . . . . . . . . . 69
1.1.6. Other Resources . . . . . . 10
1.9.3. Simulation of Confidence
1.2. Objects in Python . . . . . . . . . . 11
Intervals and t Tests . . . . 70
1.2.1. Variables . . . . . . . . . . . 11
1.2.2. Objects in Python . . . . . . 11
1.2.3. Objects in numpy . . . . . . 15
I. Regression Analysis with
1.2.4. Objects in pandas . . . . . 19
1.3. External Data . . . . . . . . . . . . 23 Cross-Sectional Data 75
1.3.1. Data Sets in the Examples . 23
2. The Simple Regression Model 77
1.3.2. Import and Export of Data
2.1. Simple OLS Regression . . . . . . . 77
Files . . . . . . . . . . . . . 24
2.2. Coefficients, Fitted Values, and
1.3.3. Data from other Sources . . 26 Residuals . . . . . . . . . . . . . . . 82
1.4. Base Graphics with matplotlib . 27 2.3. Goodness of Fit . . . . . . . . . . . 85
1.4.1. Basic Graphs . . . . . . . . 27 2.4. Nonlinearities . . . . . . . . . . . . 88
1.4.2. Customizing Graphs with 2.5. Regression through the Origin and
Options . . . . . . . . . . . 29 Regression on a Constant . . . . . 89
1.4.3. Overlaying Several Plots . . 30 2.6. Expected Values, Variances, and
1.4.4. Exporting to a File . . . . . 31 Standard Errors . . . . . . . . . . . 91
1.5. Descriptive Statistics . . . . . . . . 33 2.7. Monte Carlo Simulations . . . . . . 94
1.5.1. Discrete Distributions: Fre- 2.7.1. One Sample . . . . . . . . . 94
quencies and Contingency 2.7.2. Many Samples . . . . . . . 96
Tables . . . . . . . . . . . . 33 2.7.3. Violation of SLR.4 . . . . . 98
1.5.2. Continuous Distributions: 2.7.4. Violation of SLR.5 . . . . . 99
Histogram and Density . . 38
1.5.3. Empirical Cumulative Dis- 3. Multiple Regression Analysis: Estima-
tribution Function (ECDF) . 40 tion 101
1.5.4. Fundamental Statistics . . . 42 3.1. Multiple Regression in Practice . . 101
1.6. Probability Distributions . . . . . . 44 3.2. OLS in Matrix Form . . . . . . . . 107
1.6.1. Discrete Distributions . . . 44 3.3. Ceteris Paribus Interpretation and
1.6.2. Continuous Distributions . 47 Omitted Variable Bias . . . . . . . 110
1.6.3. Cumulative Distribution 3.4. Standard Errors, Multicollinearity,
Function (CDF) . . . . . . . 47 and VIF . . . . . . . . . . . . . . . . 112
4. Multiple Regression Analysis: Inference115 8.2. Heteroscedasticity Tests . . . . . . 168
4.1. The t Test . . . . . . . . . . . . . . . 115 8.3. Weighted Least Squares . . . . . . 171
4.1.1. General Setup . . . . . . . . 115
4.1.2. Standard Case . . . . . . . . 116 9. More on Specification and Data Issues 177
4.1.3. Other Hypotheses . . . . . 118 9.1. Functional Form Misspecification . 177
4.2. Confidence Intervals . . . . . . . . 121 9.2. Measurement Error . . . . . . . . . 180
4.3. Linear Restrictions: F-Tests . . . . 123 9.3. Missing Data and Nonrandom
Samples . . . . . . . . . . . . . . . 184
5. Multiple Regression Analysis: OLS 9.4. Outlying Observations . . . . . . . 188
Asymptotics 127 9.5. Least Absolute Deviations (LAD)
5.1. Simulation Exercises . . . . . . . . 127 Estimation . . . . . . . . . . . . . . 190
5.1.1. Normally Distributed Error
Terms . . . . . . . . . . . . . 127
5.1.2. Non-Normal Error Terms . 128 II. Regression Analysis with Time
5.1.3. (Not) Conditioning on the
Series Data 191
Regressors . . . . . . . . . . 132
5.2. LM Test . . . . . . . . . . . . . . . . 135
10. Basic Regression Analysis with Time
Series Data 193
6. Multiple Regression Analysis: Further
Issues 137 10.1. Static Time Series Models . . . . . 193
6.1. Model Formulae . . . . . . . . . . . 137 10.2. Time Series Data Types in Python . 194
6.1.1. Data Scaling: Arithmetic 10.2.1. Equispaced Time Series in
Operations Within a Formula 137 Python . . . . . . . . . . . . 194
6.1.2. Standardization: Beta Coef- 10.2.2. Irregular Time Series in
ficients . . . . . . . . . . . . 138 Python . . . . . . . . . . . . 197
6.1.3. Logarithms . . . . . . . . . 140 10.3. Other Time Series Models . . . . . 199
6.1.4. Quadratics and Polynomials 140 10.3.1. Finite Distributed Lag
6.1.5. Hypothesis Testing . . . . . 142 Models . . . . . . . . . . . . 199
6.1.6. Interaction Terms . . . . . . 143 10.3.2. Trends . . . . . . . . . . . . 201
6.2. Prediction . . . . . . . . . . . . . . 144 10.3.3. Seasonality . . . . . . . . . 202
6.2.1. Confidence and Prediction
Intervals for Predictions . . 11. Further Issues in Using OLS with Time
144
6.2.2. Effect Plots for Nonlinear Series Data 205
Specifications . . . . . . . . 147 11.1. Asymptotics with Time Series . . . 205
11.2. The Nature of Highly Persistent
7. Multiple Regression Analysis with Time Series . . . . . . . . . . . . . . 210
Qualitative Regressors 151 11.3. Differences of Highly Persistent
7.1. Linear Regression with Dummy Time Series . . . . . . . . . . . . . . 213
Variables as Regressors . . . . . . . 151 11.4. Regression with First Differences . 213
7.2. Boolean Variables . . . . . . . . . . 154
7.3. Categorical Variables . . . . . . . . 155 12. Serial Correlation and Heteroscedas-
7.3.1. ANOVA Tables . . . . . . . 157 ticity in Time Series Regressions 217
7.4. Breaking a Numeric Variable Into 12.1. Testing for Serial Correlation of the
Categories . . . . . . . . . . . . . . 159 Error Term . . . . . . . . . . . . . . 217
7.5. Interactions and Differences in Re- 12.2. FGLS Estimation . . . . . . . . . . 222
gression Functions Across Groups 161 12.3. Serial Correlation-Robust Infer-
ence with OLS . . . . . . . . . . . . 223
8. Heteroscedasticity 165 12.4. Autoregressive Conditional Het-
8.1. Heteroscedasticity-Robust Inference 165 eroscedasticity . . . . . . . . . . . . 224
III. Advanced Topics 227 18. Advanced Time Series Topics 289
18.1. Infinite Distributed Lag Models . . 289
13. Pooling Cross-Sections Across Time: 18.2. Testing for Unit Roots . . . . . . . 291
Simple Panel Data Methods 229 18.3. Spurious Regression . . . . . . . . 292
13.1. Pooled Cross-Sections . . . . . . . 229 18.4. Cointegration and Error Correc-
13.2. Difference-in-Differences . . . . . . 230 tion Models . . . . . . . . . . . . . 295
13.3. Organizing Panel Data . . . . . . . 233 18.5. Forecasting . . . . . . . . . . . . . . 295
13.4. First Differenced Estimator . . . . 234
19. Carrying Out an Empirical Project 299
14. Advanced Panel Data Methods 239 19.1. Working with Python Scripts . . . 299
14.1. Fixed Effects Estimation . . . . . . 239 19.2. Logging Output in Text Files . . . 301
14.2. Random Effects Models . . . . . . 240 19.3. Formatted Documents with
14.3. Dummy Variable Regression and Jupyter Notebook . . . . . . . . . . 302
Correlated Random Effects . . . . 244 19.3.1. Getting Started . . . . . . . 302
14.4. Robust (Clustered) Standard Errors 247 19.3.2. Cells . . . . . . . . . . . . . 302
19.3.3. Markdown Basics . . . . . . 303
15. Instrumental Variables Estimation
and Two Stage Least Squares 249
15.1. Instrumental Variables in Simple IV. Appendices 309
Regression Models . . . . . . . . . 249
15.2. More Exogenous Regressors . . . . Python Scripts
251 311
15.3. Two Stage Least Squares . . . . . . 1.
254 Scripts Used in Chapter 01 . . . . 311
15.4. Testing for Exogeneity of the Re- 2. Scripts Used in Chapter 02 . . . . 334
gressors . . . . . . . . . . . . . . . .256 3. Scripts Used in Chapter 03 . . . . 343
15.5. Testing Overidentifying Restrictions 257 4. Scripts Used in Chapter 04 . . . . 347
15.6. Instrumental Variables with Panel 5. Scripts Used in Chapter 05 . . . . 349
Data . . . . . . . . . . . . . . . . . .259 6. Scripts Used in Chapter 06 . . . . 352
7. Scripts Used in Chapter 07 . . . . 357
16. Simultaneous Equations Models 261 8. Scripts Used in Chapter 08 . . . . 361
16.1. Setup and Notation . . . . . . . . . 261 9. Scripts Used in Chapter 09 . . . . 365
16.2. Estimation by 2SLS . . . . . . . . . 262 10. Scripts Used in Chapter 10 . . . . 371
16.3. Outlook: Estimation by 3SLS . . . 263 11. Scripts Used in Chapter 11 . . . . 374
12. Scripts Used in Chapter 12 . . . . 378
17. Limited Dependent Variable Models 13. Scripts Used in Chapter 13 . . . . 383
and Sample Selection Corrections 265 14. Scripts Used in Chapter 14 . . . . 386
17.1. Binary Responses . . . . . . . . . . 265 15. Scripts Used in Chapter 15 . . . . 390
17.1.1. Linear Probability Models . 265 16. Scripts Used in Chapter 16 . . . . 395
17.1.2. Logit and Probit Models: 17. Scripts Used in Chapter 17 . . . . 396
Estimation . . . . . . . . . . 267 18. Scripts Used in Chapter 18 . . . . 405
17.1.3. Inference . . . . . . . . . . . 270 19. Scripts Used in Chapter 19 . . . . 409
17.1.4. Predictions . . . . . . . . . . 271
17.1.5. Partial Effects . . . . . . . . 273 Bibliography 411
17.2. Count Data: The Poisson Regres-
List of Wooldridge (2019) Examples 413
sion Model . . . . . . . . . . . . . . 276
17.3. Corner Solution Responses: The Index 415
Tobit Model . . . . . . . . . . . . . 279
17.4. Censored and Truncated Regres-
sion Models . . . . . . . . . . . . . 281
17.5. Sample Selection Corrections . . . 286
List of Tables
1.1. Logical Operators . . . . . . . . . . 12
1.2. Python Built-in Data Types . . . . . 15
1.3. Important numpy Functions and
Methods . . . . . . . . . . . . . . . 18
1.4. Important pandas Methods . . . . 21
1.5. numpy Functions for Descriptive
Statistics . . . . . . . . . . . . . . . 41
1.6. scipy Functions for Statistical
Distributions . . . . . . . . . . . . . 44

4.1. One- and Two-tailed t Tests for

H0 : β j = a j . . . . . . . . . . . . . . 118
 5.1. Density of β̂ 1 with Different Sam-
ple Sizes: Normal Error Terms . . 129
5.2. Density Functions of the Simu-
lated Error Terms . . . . . . . . . . 130
List of Figures 5.3. Density of β̂ 1 with Different Sam-
ple Sizes: Non-Normal Error Terms 131
5.4. Density of β̂ 1 with Different Sam-
ple Sizes: Varying Regressors . . . 134
1.1. Python in the Command Line . . . 3
1.2. Spyder User Interface . . . . . . . . 5 6.1. Nonlinear Effects in Example 6.2 . 149
1.3. Executing a Script with I . . . . . 6
1.4. Executing a Script Line by Line . . 7 9.1. Outliers: Distribution of Studen-
1.5. Examples of Text Data Files . . . . 24 tized Residuals . . . . . . . . . . . 189
1.6. Examples of Point and Line Plots
using plot(x, y) . . . . . . . . . 28 10.1. Time Series Plot: Imports of Bar-
1.7. Examples of Function Plots using ium Chloride from China . . . . . 196
plot . . . . . . . . . . . . . . . . . 29 10.2. Time Series Plot: Stock Prices of
1.8. Overlayed Plots . . . . . . . . . . . 31 Ford Motor Company . . . . . . . 198
1.9. Examples of Exported Plots . . . . 32
1.10. Pie and Bar Plots . . . . . . . . . . 37 11.1. Time Series Plot: Daily Stock Re-
1.11. Histograms . . . . . . . . . . . . . . 39 turns 2008–2016, Apple Inc. . . . . 209
1.12. Kernel Density Plots . . . . . . . . 40 11.2. Simulations of a Random Walk
1.13. Empirical CDF . . . . . . . . . . . . 41 Process . . . . . . . . . . . . . . . . 211
11.3. Simulations of a Random Walk
1.14. Box Plots . . . . . . . . . . . . . . . 43
Process with Drift . . . . . . . . . . 212
1.15. Plots of the PMF and PDF . . . . . 46
11.4. Simulations of a Random Walk
1.16. Plots of the CDF of Discrete and
Process with Drift: First Differences 214
Continuous RV . . . . . . . . . . . 49
1.17. Simulated and Theoretical Density 17.1. Predictions from Binary Response
of Ȳ . . . . . . . . . . . . . . . . . . 69 Models (Simulated Data) . . . . . . 273
1.18. Density of Ȳ with Different Sample 17.2. Partial Effects for Binary Response
Sizes . . . . . . . . . . . . . . . . . 70 Models (Simulated Data) . . . . . . 274
1.19. Density of the χ2 Distribution with 17.3. Conditional Means for the Tobit
1 d.f. . . . . . . . . . . . . . . . . . 71 Model . . . . . . . . . . . . . . . . . 279
1.20. Density of Ȳ with Different Sample 17.4. Truncated Regression: Simulated
Sizes: χ2 Distribution . . . . . . . . 71 Example . . . . . . . . . . . . . . . 285
1.21. Simulation Results: First 100 Con-
fidence Intervals . . . . . . . . . . . 74 18.1. Spurious Regression: Simulated
Data from Script 18.3 . . . . . . . . 293
2.1. OLS Regression Line for Example 18.2. Out-of-sample Forecasts for Un-
2-3 . . . . . . . . . . . . . . . . . . . 80 employment . . . . . . . . . . . . . 298
2.2. OLS Regression Line for Example
2-5 . . . . . . . . . . . . . . . . . . . 82 19.1. Creating a Jupyter Notebook . . . 302
2.3. Regression through the Origin and 19.2. An Empty Jupyter Notebook . . . 303
on a Constant . . . . . . . . . . . . 91 19.3. Cells in Jupyter Notebook . . . . . 304
2.4. Simulated Sample and OLS Re- 19.4. Example of an Exported Jupyter
gression Line . . . . . . . . . . . . . 96 Notebook . . . . . . . . . . . . . . . 306
2.5. Population and Simulated OLS Re- 19.5. Example of an Exported Jupyter
gression Lines . . . . . . . . . . . . 99 Notebook (cont’ed) . . . . . . . . . 307
Preface
An essential part of learning econometrics is the application of the methods to real-world problems
and data. The practical implementation and application of econometric methods and tools helps
tremendously with understanding the concepts. But learning how to use a software package also
has great benefits in and of itself. Nowadays, a vast majority of our students will have to deal with
some sort of data analysis in their careers. So a solid understanding of some serious data analysis
software is an invaluable asset for any student of economics, business administration, and related
fields.
But what software package is the right one for learning econometrics? That’s a tough question.
Possibly the most important aspect is that it is widely used both in and outside of academia. A
large and active user community helps the software to remain up to date and increases the chances
that somebody else has already solved the problem at hand. And fluency in a software package is
especially valuable on the job market as well as on the job if it is popular. Another aspect for the
software choice is that it is easily (and ideally freely) available to all students.
Python is an ideal candidate for starting to learn econometrics and data analysis. It has a huge
user base, especially in the fields of data science, machine learning, and artificial intelligence, where
it arguably is the most popular software overall. These are very exciting areas and there is a lot
of cutting edge research in the integration of their tools into the econometrics toolbox. So why not
kill two birds with one stone and master a powerful and important software package while learning
econometrics at the same time? Because Python must be hard to learn and to apply to econometrics?
It is not at all, as this book shows.
And Python is completely free and available for all relevant operating systems. When using it
in econometrics courses, students can easily download a copy to their own computers and use it
at home (or their favorite cafés) to replicate examples and work on take-home assignments. This
hands-on experience is essential for the understanding of the econometric models and methods. It
also prepares students to conduct their own empirical analyses for their theses, research projects,
and professional work.
A problem we encountered when teaching introductory econometrics classes is that the textbooks
that also introduce Python do not discuss econometrics. Conversely, our favorite introductory econo-
metrics textbooks do not cover Python. Although it is possible to combine a good econometrics text-
book with an unrelated introduction to Python, this creates substantial hurdles because the topics
and order of presentation are different, and the terminology and notation are inconsistent.
This book does not attempt to provide a self-contained discussion of econometric models and
methods. Instead, it builds on the excellent and popular textbook “Introductory Econometrics” by
Wooldridge (2019). It is compatible in terms of topics, organization, terminology, and notation, and
is designed for a seamless transition from theory to practice.
The first chapter provides a gentle introduction to Python, covers some of the topics of basic
statistics and probability presented in the appendix of Wooldridge (2019), and introduces Monte
Carlo simulation as an additional tool. The other chapters have the same names and cover the same
material as the respective chapters in Wooldridge (2019). Assuming the reader has worked through
the material discussed there, this book explains and demonstrates how to implement everything
in Python and replicates many textbook examples. We also open some black boxes of the built-in
functions for estimation and inference by directly applying the formulas known from the textbook
2 Preface

to reproduce the results. Some supplementary analyses provide additional intuition and insights.
We want to thank Lars Grönberg providing us with many suggestions and valuable feedback about
the contents of this book.
The book is designed mainly for students of introductory econometrics who ideally use
Wooldridge (2019) as their main textbook. It can also be useful for readers who are familiar with
econometrics and possibly other software packages. For them, it offers an introduction to Python
and can be used to look up the implementation of standard econometric methods. Because we are
explicitly building on Wooldridge (2019), it is useful to have a copy at hand while working through
this book.
Note that there is a sister book Using R for Introductory Econometrics, just published as a second
edition, see http://www.URfIE.net. We based this book on the R version, using the same struc-
ture, the same examples, and even much of the same text where it makes sense. This decision was
not only made for laziness. It also helps readers to easily switch back and forth between the books.
And if somebody worked through the R book, she can easily look up the pythonian way to achieve
exactly the same results and vice versa, making it especially easy to learn both languages. Which one
should you start with (given your professor hasn’t made the decision for you)? Both share many of
the advantages like having a huge and active user community, being widely used inside and outside
of academia and being freely available. R is traditionally used in statistics, while Python is domi-
nant in machine learning and artificial intelligence. These origins are still somewhat reflected in the
availability of specialized extension packages. But most of all data analysis and econometrics tasks
can be equally well performed in both packages. At the end, it’s most important point is to get used
to the workflow of some dedicated data analysis software package instead of not using any software
or a spreadsheet program for data analysis.
All computer code used in this book can be downloaded to make it easier to replicate the results
and tinker with the specifications. The companion website also provides the full text of this book
for online viewing and additional material. It is located at:

http://www.UPfIE.net
1. Introduction
Learning to use Python is straightforward but not trivial. This chapter prepares us for implementing
the actual econometric analyses discussed in the following chapters. First, we introduce the basics
of the software system Python in Section 1.1. In order to build a solid foundation we can later rely
on, Chapters 1.2 through 1.4 cover the most important concepts and approaches used in Python like
working with objects, dealing with data, and generating graphs. Sections 1.5 through 1.7 quickly
go over the most fundamental concepts in statistics and probability and show how they can be
implemented in Python. More advanced Python topics like conditional execution, loops, functions
and object orientation are presented in Section 1.8. They are not really necessary for most of the
material in this book. An exception is Monte Carlo simulation which is introduced in Section 1.9.

1.1. Getting Started

Before we can get going, we have to find and download the relevant software, figure out how the
examples presented in this book can be easily replicated and tinkered with, and understand the most
basic aspects of Python. That is what this section is all about.

1.1.1. Software
Python is a free and open source software. Its homepage is https://www.python.org/. There, a
wealth of information is available as well as the software itself. We recommend installing the Python
distribution Anaconda (also open source), which includes Python plus many tools needed for data
analysis. For more information and installation files, see https://www.anaconda.com.
Distributions are available for Windows, Mac, and Linux systems and come in two versions. The
examples in this book are based on the installation of the latest version, Python 3. It is not backwards
compatible to Python 2.

Figure 1.1. Python in the Command Line

After downloading and installing, Python can be accessed by the command line interface. In
Windows, run the program “Anaconda Prompt”. In Linux or macOS you can simply open a terminal
4 1. Introduction

window. You start Python by typing python and pressing the return key ( ←- ). This will look similar
to the screenshot in Figure 1.1. It provides some basic information on Python and the installed
version. Right to the “>>>” sign is the prompt where the user can type commands for Python to
evaluate.
We can type whatever we want here. After pressing ←- , the line is terminated, Python tries to
make sense out of what is written and gives an appropriate answer. In the example shown in Figure
1.1, this was done four times. The texts we typed are shown next to the “>>>” sign, Python answers
under the respective line.
Our first attempt did not work out well: We have got an error message. Unfortunately, Python does
not comprehend the language of Shakespeare. We will have to adjust and learn to speak Python’s
less poetic language. The second command shows one way to do this. Here, we provide the input
to the command print in the correct syntax, so Python understands that we entered text and knows
what to do with it: print it out on the console. Next, we gave Python simple computational tasks and
got the result under the respective command. The syntax should be easy to understand – apparently,
Python can do simple addition and deals with the parentheses in the expected way. The meaning of
the last command is less obvious,
√ because it uses the pythonian way of calculating an exponential
term: 16**0.5 = 160.5 = 16 = 4.
Python is used by typing commands such as these. Not only Apple users may be less than im-
pressed by the design of the user interface and the way the software is used. There are various
approaches to make it more user friendly by providing a different user interface added on top of
plain Python. Notable examples include IDLE, PyCharm, Visual Studio and Spyder. The latter was
already set up during the installation of Anaconda and we use it for all what follows. The easiest
way to start Spyder is by selecting it in the Anaconda Navigator that was also set up during the
installation of Anaconda.
A screenshot of the user interface on a Mac computer is shown in Figure 1.2 (on other systems it
will look very similar). There are several sub-windows. The one on the bottom right named “IPython
console” looks very similar and behaves exactly the same as the command line. The usefulness of
the other windows will become clear soon.
Here are a few quick tricks for working in the console of Spyder:
• When starting to type a command, press the tabulator key − →−−
→− to see a list of suggested
commands. Typing pr, for example, followed by − → −
−
−→ gives a list of all Python commands
starting with pr, like the print command.
• Use help(command) to print the help page for the provided command.
• With the ↑ and ↓ arrow keys, we can scroll through the previously entered commands to
repeat or correct them.

1.1.2. Python Scripts

As already seen, we will have to get used to interacting with our software using written commands.
While this may seem odd to readers who do not have any experience with similar software at this
point, it is actually very common for econometrics software and there are good reasons for this. An
important advantage is that we can easily collect all commands we need for a project in a text file
called Python script.
A Python script contains all commands including those for reading the raw data, data manip-
ulation, estimation, post-estimation analyses, and the creation of graphs and tables. In a complex
project, these tasks can be divided into separate Python scripts. The point is that the script(s) together
with the raw data generate the output used in the term paper, thesis, or research paper. We can then
ask Python to evaluate all or some of the commands listed in the Python script at once.
1.1. Getting Started 5

Figure 1.2. Spyder User Interface

This is important since a key feature of the scientific method is reproducibility. Our thesis adviser
as well as the referee in an academic peer review process or another researcher who wishes to build
on our analyses must be able to fully understand where the results come from. This is easy if we can
simply present our Python script which has all the answers.
Working with Python scripts is not only best practice from a scientific perspective, but also very
convenient once we get used to it. In a nontrivial data analysis project, it is very hard to remember
all the steps involved. If we manipulate the data for example by directly changing the numbers in a
spreadsheet, we will never be able to keep track of everything we did. Each time we make a mistake
(which is impossible to avoid), we can simply correct the command and let Python start from scratch
by a simple mouse click if we are using scripts. And if there is a change in the raw data set, we can
simply rerun everything and get the updated tables and figures instantly.
Using Python scripts is straightforward: We just write our commands into a text file and save it
with a “.py” extension. When using a user interface like Spyder, working with scripts is especially
convenient since it is equipped with a specialized editor for script files. To use the editor for working
on a new Python script, use the menu File→New file....
The window in the left part of Figure 1.2 is the script editor. We can type arbitrary text, begin
a new line with the return key, and navigate using the mouse or the ↑ ↓ ← → arrow keys.
Our goal is not to type arbitrary text but sensible Python commands. In the editor, we can also use
tricks like code completion that work in the Console window as described above. A new command
is generally started in a new line, but also a semicolon “;” can be used if we want to cram more than
one command into one line – which is often not a good idea in terms of readability.
An extremely useful tool to make Python scripts more readable are comments. These are lines
beginning with a “#”. These lines are not evaluated by Python but can (and should) be used to
structure the script and explain the steps. Python scripts can be saved and opened using the File
menu.
Figures 1.3 and 1.4 show a screenshot of Spyder with a Python script saved as “First-Python-
Script.py”. It consists of six lines in total including three comments. We can send lines of code to
Random documents with unrelated
content Scribd suggests to you:
the application of capital to agriculture will have increased the
available food. The result will be the same tolerable degree of
comfort as at the beginning of the cycle, and the same relapse from it
as at the second stage. He conceives the two stages to follow each
other as naturally as sunshine rain and rain sunshine. The existence
of such a cycle may remain concealed from the ordinary historian, if
he looks merely to the money wages of the labourer, for it frequently
happens that the labourer gets the same sums of money for his wages
during a long series of years when the real value of the sums has not
remained the same,—the price of bread in what we have called the
second stage of the cycle being much dearer than it was in the first,
and than it will be in the third.[153] Though Malthus expressly
qualifies his statements by showing that civilization tends to
counteract these fluctuations, it certainly seemed to be his belief in
1803 that on the whole the working classes of Europe, and especially
of England, were powerless to escape from them. How far this view is
justified will be seen presently.
 CHAPTER IV.
THE SAVAGE, BARBARIAN, AND ORIENTAL.
Simile supplanted by Fact—Savage Life—Population dependent not
on possible but on actual Food—Indirect Action of Positive
Checks—Hunger not a Principle of Progress—Otaheite a Crux for
Common Sense—Cycle in the Movement of Population—Pitcairn
Island—Barbarian and Oriental—Nomad Shepherds—Abram and
Lot—Cimbri and Teutones—Gibbon versus Montesquieu—“At
bay on the limits of the Universe”—Misgovernment an indirect
Check on Population—Ancient Europe less populous than
Modern—Civilization the gradual Victory of the third Check.

The main position of the essay was so incontrovertible, that when

the critics despaired to convict Malthus of a paradox, they charged
him with a truism. To the friendly Hallam[154] the mathematical basis
of the argument appeared as certain as the multiplication table, and
the unfriendly Hazlitt “did not see what there was to discover after
reading the tables of Noah’s descendants, and knowing that the
world is round.”[155] If the essayist had done nothing more than put
half-truths together into a whole, he would have “entrenched himself
in an impregnable fortress” and given his work a great “air of
mastery.”[156] But he would have convinced the understanding
without convincing the imagination. Adam Smith himself would have
done no more than half his work, if he had been content to prove the
reasonableness of free trade without showing, in detail, the effect of
it and its opposites. Even the most competent reader has seldom all
the relevant facts marshalled in his memory, ready to command; and
he will always be thankful for illustrations. The Essay on Population
in its second form certainly excelled all economical works, save one,
in its pertinent examples from life and history.
Imagination in the narrower sense of the word is to be out of court.
Malthus, like Adam Smith, not only leaves little to his reader’s fancy,
but makes little use of his own. His own had misled his readers in the
first essay, though it had certainly given that little book much of its
piquancy; and he resolves for the sake of truth to chain it up, as
Coleridge chained up his understanding. The self-denying ordinance
is only too fully executed. The style of his essay is truly described by
himself[157] as having gradually “lost all pretensions to merit.” Edition
follows edition, each with its footnotes, supplements,
rearrangements, and corrections, till the reader feels that this writer
“would be clearer if he were not so clear.”
But the title-page supplies a guiding thread. From the second
edition onwards to the last, “Past” and “Present” appear in large
letters, “Future” in small. The entire work may therefore be divided
according to the three tenses, with the emphasis on the two former.
The first book is devoted to the past, the second to the present, and
the third and fourth to the future.
The First deals with the less civilized parts of the world as it now
is, and the uncivilized past times; the Second with the different states
of modern Europe; the Third criticizes popular schemes of future
improvement; while the Fourth gives the author’s own views of the
possible progress of humanity.
After explaining his principles, Malthus takes a survey of human
progress, if not from brute to savage, at least from savage to citizen.
He shows us how the rude and simple positive checks become
complicated with the preventive; and he leads us up from barbarism
to civilization till we find ourselves in a society where the citizens
think less of check than of chief end, and less of self-sacrifice than of
self-devotion, to some cause or person, and even the inferior
members act, at worst, from mixed motives, containing good as well
as evil. These are the two extreme ends of his line. It would be
useless to deny that he lingers longest over the less pleasing, and
gives Godwin some excuse for questioning his logical right to believe
in the more pleasing at all.[158] At the same time it would have been
(even logically) impossible for him to have attacked Godwin for
taking abstract views of human nature, and then to have persisted in
an abstraction of his own, after all his own European travel and
historical studies. His fault had lain in defective premises, not in
false reasoning; and he remedies the fault.
Let us take his account in his own order. Beginning with present
savagery, which with some qualifications is a picture of our own past,
he sifts out the descriptions of Cook, Vancouver, and other travellers,
to see what checks to population operate in different grades of savage
humanity. At the very bottom of the scale comes Tierra del Fuego, by
general consent the abode of pure misery, and therefore naturally the
home of a sparse population. Next come the natives of the Andaman
Islands and of Van Diemen’s Land. “Their whole time is spent in
search of food,” which consists of the raw products of the soil and
sea; the whole time of every individual is devoted to this one labour,
and there is neither room nor inducement for any other industries.
Vice is hardly needed; misery in the shape of perpetual scarcity and
famine keeps down the people to the food. Third in the scale of
human beings are the New Hollanders, the original inhabitants of
North-West Australia, among whom can be traced not only the check
of misery, but the check of vice. The women are so cruelly treated at
all times, and the children have so harsh an upbringing, that there is
no difficulty in understanding how population does not even reach
the full limit of the scanty food. War and pestilence make the
assurance doubly sure. As savages are entirely innocent of sanitary
science, the dirt of their persons and their houses deprives them of
“the advantage which usually attends a thinly-peopled country,”
comparative exemption from pestilence.[159] Even the North
American Indians, who are one step higher than the New Hollanders,
come under the same condemnation for overcrowding, and for much
else besides. The account which Malthus gives of them may be
compared with that of De Tocqueville half a century later. Romance
has clung to them only because they were the nearest and best
known savages of their kind, and their necessary labours were in
Europe rich men’s pleasures. But hunting and river-fishing cannot
yield much food unless pursued over a wide area. A hunter is so far
like the beast of prey which he pursues, that he must go long
distances for his food, and must either fly from or overcome every
rival. The North American Indian must therefore either go West after
his old food, or else he must stay where he is, to beat off the
Europeans, or to adopt their food and their habits. “The Indians have
only two ways of saving themselves, war and civilization. They must
either destroy the Europeans or become their equals.”[160] As the
civilization of a nation of hunters is almost impossible, their
extinction seems inevitable. The question remains, How is this
population cut down to the level of its food?
In Malthus’ answer to the question occur three remarks of great
general importance. First, what limits the numbers of a people is not
the possible but the actual food.[161] Second, want destroys a
population less often directly by starvation than indirectly through
the medium of manners and customs.[162] Third, the mere pressure of
impending starvation does not lead to progress.[163]
Malthus is never tired of insisting on the first of these remarks;
and a proper understanding of it is essential to a fair judgment on his
doctrine. He never says that it is the tendency of a population to
increase up to the limits of the greatest possible amount of food that
can be produced in a given country. The valley of the Mississippi
when highly cultivated may possibly support a hundred millions; but
the question is not what it would do when highly cultivated, but what
it can do when cultivated as it now is and as men now are. “In a
general view of the American continent as described by historians,
the population seems to have been spread over the surface very
nearly in proportion to the quantity of food which the inhabitants of
the different parts in the actual state of their industry and
improvements could as a matter of fact obtain; and that with few
exceptions it pressed hard against this limit, rather than fell short of
it, appears from the frequent recurrence of distress for want of food
in all parts of America.”[164] What is said here of the Indians a
hundred years ago applies to the Colonists now. “The actual state of
industry” is of course a much more improved one; but the population
the land will bear is still in proportion to it, and the amount could
not have been increased till the actual state of the industry had first
been bettered. One cause of the decay of the numbers of the Indians
was that their method of industry, so far from becoming better,
became worse by their contact with Europeans, and therefore the
limit of population was actually contracted instead of being
extended.[165] This explains how it is that their diminishing numbers
do not bring them greater comfort. Whether the numbers in any
given case are too great or too small depends always on the quantity
of the food that is divided among them; and, where the food
decreases faster than the population, a population that has become
smaller numerically becomes actually larger in proportion to the
food. The statement that England or any other country could bear
millions more than it does now is a mere reference to unexplored
possibilities, landing us in the infinite. It may be answered in the
same way as the Eleatic puzzles about motion; land infinitely
improvable does not mean land infinitely improved, as matter
infinitely divisible does not mean matter infinitely divided. The
position of Malthus is therefore as follows: given a people’s skill, and
given its standard of living at any time, its numbers are always
tending to be the utmost that can be furnished by that skill with a
living up to that standard,—that is to say, with what, according to
that standard, are the necessaries of human life. Either a diminution
of that skill or an increase in that standard would cause over-
population. The question is always a relative one.
The human as distinguished from the animal character of the
problem appears not only in that relativity (which affects mainly the
preventive checks), but in the indirect way in which the positive
checks, if we may say so, prefer to act. It is as if they were always
desirous of resolving themselves as far as might be into preventive.
The ultimate check, Malthus says, is starvation; but, he adds, it is
seldom the immediate one. The higher up we go in the scale, the
more it is hidden away out of sight. Starvation is interpreted, by all
grades of society above the lowest, to mean the loss of what they
conceive to be the necessaries not of a bare living but of endurable
life; and even the lowest, instead of apprehending some pain,
apprehend some bringer of it. They do not allow famine to kill them;
they create manners and customs that do the work for it, keeping the
famine itself afar off. “Both theory and experience uniformly instruct
us that a less abundant supply of food operates with a gradually
increasing pressure for a very long time before its progress is
stopped. It is difficult indeed to conceive a more tremendous shock
to society than the event of its coming at once to the limits of the
means of subsistence, with all the habits of abundance and early
marriages, which accompany a largely increasing population. But,
happily for mankind, this never is nor ever can be the case. The event
is provided for by the concurrent interests and feelings of individuals
long before it arrives; and the gradual diminution of the real wages of
the labouring classes of society slowly and almost insensibly
generates the habits necessary for an order of things in which the
funds for the maintenance of labour are stationary.... The causes [of
the retardation of population] will be generally felt and [will]
generate a change of habits long before the period arrives.”[166] “An
insufficient supply of food to any people does not show itself merely
in the shape of famine, but in other more permanent forms of
distress, and in generating certain customs which operate sometimes
with greater force in the prevention of a rising population” than in
the destruction of the risen.[167] Robertson the historian truly says,
that whether civilization has improved the lot of men may be
doubtful, but it has certainly improved the condition of women.
Among the Indians and almost all savages, “servitude is a name too
mild to describe their wretched state.” The hard life of the men kills
their instinctive fondness for the women; the latter are therefore less
likely to become mothers, while, if they do, their own hardships and
heavy tasks are a great hindrance to nursing. It is not surprising that
the surviving children are of good physique; none but the
exceptionally strong could weather the cruel discipline of childhood.
[168]
In South America the difficulty of upbringing actually led to an
enforced monogamy, as well as to late marriages and their not
unfrequent accompaniment, irregularities before marriage. Such
customs diminish numbers. But even the adult savages do not find
life easy. They are not the men to think of providing for a rainy day;
in the short moments of plenty they do not think of the long days of
want. Intemperate living as well as the rigour and the accidents of a
hunting life cut off numbers in their prime. They are subject to
diseases and invent no remedies. Their acquiescence in dirt leads to
pestilences, but they invent no sanitary reforms; and their thinly-
peopled country loses its natural exemption from epidemics. Their
wars are internecine, for they are largely prompted by sheer self-
preservation, and the thought that if the one combatant lives the
other cannot. Cannibalism itself was at first due to extreme want,
though what occasional hunger had begun, hate perpetuated in a
custom. This and the low cunning and mean strategy, due to a
resolve to survive at all costs, are the prime inventions of the struggle
for existence on these low levels.
Such are the causes by which the numbers of the North American
Indians are kept down to a very low figure; but, low as it is, the figure
is high enough for the food. Apart from a difference in the standard
of living, the proportion of population to food is similar over the
inhabited world; and in the same neighbourhood or among cognate
races it will be almost identical. A diminution in one Indian tribe, not
being voluntary, will not be the cause of plenty to the survivors; it
has been the effect of want, and it will simply weaken the collective
force of the tribe in the struggle against others.[169]
 The supremacy of want as the ultimate check on population is
illustrated by the instant expansion of population which is produced
in these grades of humanity by an accession of plenty. When a tribe
falls upon fertile land, its numbers swell, and its collective might,
depending on numbers, becomes greater. The increase of food,
however, seems in this case to lead to nothing else than increase of
numbers. There is a melancholy equality of suffering between tribe
and tribe, as well as between members of the same tribe. There is no
distinction of rank, but only of sex and bodily strength, as regards
endurance of hardships.
It is in this connection that Malthus throws some light on the
question how progress could ever take place at all. His answer is not
unlike Adam Smith’s remark about the connection of high wages
with good work. He says, that beyond a certain limit, hard fare and
great want depress men below the very capacity of improvement;
comfort must reach a certain height before the desires of civilized life
can come into being at all. If the American tribes, he says, have
remained hunters, it is not simply because they have not increased in
numbers sufficiently to render the pastoral or agricultural state
necessary to them. Reasons which Malthus does not pretend to
particularize,[170] and which he allows to be unconnected with mere
increase or decrease of numbers, have prevented these tribes from
ever trying to raise cattle or grow corn at all. “If hunger alone could
have prompted the savage tribes of America to such a change in their
habits, I do not conceive that there would have been a single nation
of hunters and fishers remaining; but it is evident that some
fortunate train of circumstances, in addition to this stimulus, is
necessary for this purpose; and it is undoubtedly probable that these
arts of obtaining food will be first invented and improved in those
spots which are best suited to them, and where the natural fertility of
the situation, by allowing a greater number of people to subsist
together, would give the fairest chance to the inventive powers of the
human mind.” “A certain degree of [political] security is perhaps still
more necessary than richness of soil to encourage the change from
the pastoral to the agricultural state.”[171] These passages are
remarkable because they seem to contradict the general tenor of the
author’s writings. We were told with great emphasis in the first
edition of the essay that difficulties generate talents,[172] and even the
second and later are full of approving commentaries on the proverb,
“Necessity is the mother of invention.”[173] The contradiction is soon
solved. Malthus has no faith in the civilizing power of competition
when it means a struggle among starved men for bare life, but much
faith in it when it means the struggle for greater comfort among
those who already have the animal necessaries.[174] The significance
of his admissions will be noticed later. Meanwhile it must be
observed that the passage just quoted is not perfectly precise. The
larger the society, the greater might be the division of labour and
consequent stimulus to invention; but a tribe might be large and yet
have little in it of a society, and still less of a division of labour.
Without such favouring circumstances as Malthus mentions the
progress cannot take place; but even with them it need not; they are
therefore not the real motive power.
The account of the state of population among the South Sea
Islanders,[175] which comes next in order to the chapter on the
American Indians, is an illustration of these remarks. These savages
live in a fertile country and yet they make no progress. As this is not
the only point illustrated, it is worth while to look at the chapter in
detail.
Malthus begins by observing that population must not be thought
more subject to checks on an island than on a continent. The Abbé
Raynal, in his book on the Indies, had tried to explain a number of
modern customs that retarded population[176] by referring them to an
insular origin. He thought that they were caused at first by the over-
population of Britain and other islands, and were imported
therefrom into the continents, to the perplexity of later ages. But as a
matter of fact population on the mainlands is subject to the same
laws as on the islands, though the limits are not so obvious to
common observation, and the case is not put so neatly in a nutshell.
A nation on the continent may be as completely surrounded by its
enemies or its rivals, savage or civilized, as any islanders by the sea;
and emigration may be as difficult in the one case as in the other.
Both continent and island are peopled up to their actual produce.
“There is probably no island yet known, the produce of which could
not be further increased. This is all that can be said of the whole
earth. Both are peopled up to their actual produce. And the whole
earth is in this respect like an island.”[177] The earth is indeed more
isolated than any island of the sea, for no emigration from it is
possible. The question, therefore, to be asked about the whole earth
as about any part of it, is, “By what means the inhabitants are
reduced to such numbers as it can support?”
This was the question which forced itself on Captain Cook when he
visited the islands of the Pacific and Indian Oceans. Some of his
experiences there, especially in New Zealand, show that the native
population was kept down in nearly the same way as the American.
Their chief peculiarity is the extreme violence of their local feuds.
The people of every village he visited petitioned him to destroy the
people of the next, and “if I had listened to them I should have
extirpated the whole race.” A sense of human kinship is impossible at
so low a level of being; and the internecine wars of the New
Zealanders were the chief check to their numbers, which, from the
distressing effects of occasional scarcities, would seem always at the
best to have been close to the limits of the food.
The first impression of common sense is that distress is natural
where food is scanty, and unnatural where it is plentiful. But “if we
turn our eyes from the thinly-scattered inhabitants of New Zealand
to the crowded shores of Otaheite and the Society Islands” we find no
such phenomenon. “All apprehension of death seems at first sight to
be banished from a country that is described to be fruitful as the
garden of the Hesperides.” But reflection tells us that happiness and
plenty are the most powerful causes of increase. We might, therefore,
expect a large population in Otaheite; at its first start it might double
itself not in twenty-five but in fifteen years. Captain Cook estimated
it (on his second voyage in 1773) as 204,000. How could a country
about one hundred and twenty miles in circuit support an increase
that doubled these numbers in twenty-five years? Emigration is
impossible, for the other islands are in the same situation. Further
cultivation is inadequate, for scientific invention is quite wanting.
The answer is that the increase does not take place, and yet there is
no miracle. Licentiousness among the higher classes, and infanticide
amongst all classes, are freely practised. The free permission of
infanticide no doubt, as Hume remarks,[178] tends as a rule rather to
increase than to diminish population, for “by removing the terrors of
too numerous a family it would engage many people in marriage,”
and such is the force of natural affection, that comparatively few
parents would carry out their first intentions. But in Otaheite in its
old state custom had made infanticide easy, and it was a real check.
War against other islands was a third check, frequently destroying
the food as well as the people, thus striking down two generations at
once. All these checks notwithstanding, the population was up to the
level of the food, and there was as much scarcity and keen distress as
on any barren island.
Such at least was the state of things discovered by Captain Cook in
his three voyages (the last in 1778) and Captain Vancouver (in 1791).
On the other hand, the author of A Missionary Voyage to the South
Pacific Ocean in 1796–8 (London, 1799) found a people very scanty
as compared with the food. The accuracy of both accounts is borne
out by the description of the habits of the people at these two
periods. Captain Cook says they were careful to save up every scrap
of food, and yet suffered often from famine. The missionaries
observe the frequency of famine in the Friendly Islands and the
Marquesas, but say of the Otaheitans that they are extremely
wasteful, and yet never seem to be in want. Even in the intervals
between one of Cook’s voyages and another the state of the island
had altered. Malthus sees here an illustration of two facts. The one is
that, apart from changes in the standard of living, population
fluctuates between great excess and great defect, great numbers with
great mortality, and great comfort with rapid multiplication of
numbers. The other, which explains the first, is that any cause
affecting population, either towards increase or towards decrease,
continues to act for some time after the disappearance of the
circumstances that first occasioned it. For example, over-
populousness would lead to wars,[179] and the enmities of these wars
would long survive their first occasion. Again, over-populousness
would lead to greater infanticide and vice, which would become
habitual. New circumstances would, no doubt, after a time bring new
habits, and, to use the author’s words, would “restore the population,
which could not long be kept below its natural level without the most
extreme violence. How far European contact may operate in Otaheite
with this extreme violence and prevent it from recovering its former
population is a point which experience only can determine. But,
should this be the case, I have no doubt that on tracing the causes of
it, we should find them to be aggravated vice and misery.”[180] As a
matter of fact either European contact has caused a diminution, or
exacter inquiry has made a lower estimate of the population of all
Polynesia. The people of the whole Society Islands is reckoned at
between 15,000 and 18,000,[181] which is a long way from Cook’s
estimate of 204,000 for Otaheite alone. We can hardly believe,
however, that the vice and misery of Otaheite are more than ten
times as great as they were in 1773; and perhaps we may suppose
Malthus to mean that, if the European influences were of the same
character at the end as they were at the beginning, and were as
pernicious to the Polynesian as to the Red Indian, the language of
pessimism would be justified. The passage at least shows how unfair
it is to suppose Malthus to desire at all costs a small population; he is
careful to say that, while vice in Otaheite by reducing the numbers
caused a transient plenty among the survivors, still “a cause which
may prevent any particular evil may be beyond comparison worse
than the evil itself.”[182] Life itself may be bought too dear.
No good is done, however, by denying that excessive numbers are
an evil, or by optimistic assertion that if men are only good they will
be happy. There is at least one Polynesian island whose past history
gives a picturesque proof of the contrary. Pitcairn, “the lonely isle of
the Mutineers,” was a moral contrast to Otaheite. The inhabitants
owed nothing good to their parents, who were the mutineers of
H.M.S. ‘Bounty,’ and the women of Otaheite that came with them in
1790, when they first took refuge in Pitcairn Island. They owed all to
the religious teaching of John Adams, which made them so good,
that there were few like them on the earth.[183] But in latitudes just
touching the tropics, with a single square mile of poor soil,
surrounded by wide ocean, they had no outlet for trade and modern
arts. Like the inhabitants of Godwin’s Utopia,[184] they soon peopled
the little country to the full extent of the food that could be got by the
old methods, and, unlike the Utopians, they had not skill to invent
new. If they had not drawn the line for themselves, misery would
have done it for them. Their little colony at its first founding
consisted of fifteen men and twelve women. Fourteen men and many
women died off in the course of the ten years which passed before
the time of moral regeneration. But they left many children; and,
when the patriarch John Adams was visited by a passing ship in
1814, he was surrounded by a happy circle of devout families.
Rapidly outgrowing the resources of the place, these simple folk
removed in 1831 to Tahiti, eighty-seven strong. Some remained
there; others had no pleasure in their new abode, and came back to
suffer affliction with the people of God, believing with Malthus that
“a cause which may prevent any particular evil may be beyond all
comparison worse than the evil itself.” The evil was real, however,
and, in default of celibacy or new ways of bread winning, their only
cure seemed emigration. So in 1855, Tahiti seeming ineligible, they
journeyed further west to Norfolk Island. Though there are more
than four hundred and forty to the square mile in England and
Wales, two hundred people of this primitive sort had been certainly
too many for the single square mile of Pitcairn Island; and they did
not leave a moment too soon. Home sickness brought back two
entire families (of seventeen persons) in 1859. One or two stray
travellers joined them five years afterwards; but, with allowance for
these, we find that the increase of population on Pitcairn Island
reaches the highest estimate of Malthus. When the English Admiral
D’Horsey visited the place in 1878, the quarter of a hundred had
grown in nineteen years, at the moderate cost of twelve deaths, to a
population of ninety[185] persons. The primeval virtues will avail little
without the modern arts.
Returning to Malthus, we find him following an order of his own,
in rough conformity with the orthodox progress from deer to sheep,
and from sheep to corn. He takes us from Polynesian savages to the
nomad pastoral nations of ancient Europe.[186] The vast migrations
and their momentous historical effects he ascribes to the “constant
tendency in the human race to increase” beyond its food, and thinks
that when history has been rewritten it will contain more of this.[187]
“The misfortune of history is, that while the particular motives of a
few princes and leaders are sometimes detailed with accuracy, the
general causes which crowd their standards with willing followers
are totally overlooked.”[188] At first sight the phenomenon of civilized
agricultural nations unable to repel the invasion of shepherds seems
incredible; a country in pasture cannot possibly support so many
inhabitants as a country in tillage. A shepherd, it is true, is nearer to
the skilled labourer than a hunter; he does not simply take what
nature gives him, where nature puts it; he keeps the desired objects
of consumption under his own control, and his life is stronger
because more social. Early African colonization, as Adam Smith
pointed out, was less successful than early American, because the
natives, being shepherds and even farmers rather than fishermen,
were stronger in their resources and more united than the American
aborigines, so that the European intruders were not able to displace
them.[189] We should have expected the Scythian, Cimbrian, and
Gothic invaders of ancient times to have had a similar rebuff. “But
what renders nations of shepherds so formidable is the power which
they possess of moving altogether, and the necessity they frequently
feel of exerting this power in search of fresh pasture for their
herds.”[190] They have always in their breeding stock a reserve of food
for an emergency. The mere consciousness that their mode of life
does not bind them to one place gives them less anxiety about
providing for a family. Therefore, when they exhaust one region and
begin to feel the pinch of want, they make an armed emigration on
the scale of whole tribes at once, for the occupation of more fruitful
regions, and, as a rule, the conquest of them by force. The law of
their life is a series of periodical “struggles for existence”[191] between
one nation and another, in which the fittest survive at the cost of a
prodigious waste of human life.
The milder initial stage of this process is illustrated by the
separation of Abram and Lot in the book of Genesis.[192] Abram “was
very rich in cattle.” “Lot also had flocks and herds and tents. And the
land was not able to bear them that they might dwell together, for
their substance was great, so that they could not dwell together. And
there was a strife between the herdmen of Abram’s cattle and the
herdmen of Lot’s cattle.” They agreed, therefore, to separate, Lot
choosing the fertile valley of the Jordan, Abram going to the left into
the land of Canaan. Migrations of the same sort, more or less
peaceable, are described by modern writers as extending the Russian
people from time to time farther and farther to the south and east.
[193]
In the instances best known to history the migrations were far
from peaceable, and the puzzle has been to account for their
recurrence. The slaughter of the German barbarians by Marius,
Cæsar, Drusus, Tiberius, Germanicus did not prevent the
reappearance of similar hordes of invaders a generation later.
Claudius destroyed a quarter of a million of Goths; Aurelian and
Probus had the same work to do again. Under Diocletian the
barbarians, finding the conquest of Rome too much for them,
slaughtered one another in frontier wars. No losses seemed to
exhaust the permanent possibilities of population in those quarters.
At last in the fourth century “clouds of barbarians seemed to collect
from all parts of the northern hemisphere. Gathering fresh darkness
and terror as they rolled on, the congregated bodies at length
obscured the sun of Italy and sunk the Western world in night.”[194]
Why were the resources of the North so inexhaustible? Simply
because the power of increase is inexhaustible. The North was not, it
is true, more densely peopled then than now. “The climate of ancient
Germany has been mollified and the soil fertilized by the labour of
ten centuries from the time of Charlemagne. The same extent of
ground which at present maintains in ease and plenty a million of
husbandmen and artificers, was unable to supply a hundred
thousand lazy warriors with the simple necessaries of life. The
Germans abandoned their immense forests to the exercise of
hunting, employed in pasturage the most considerable part of their
lands, bestowed on the small remainder a rude and careless
cultivation, and then accused the scantiness and sterility of a country
that refused to maintain the multitude of its inhabitants. When the
return of famine severely admonished them of the importance of the
arts, the national distress was sometimes alleviated by the
emigration of a third, perhaps, or a fourth part of their youth.”[195] In
short, the countries were more than fully peopled up to their actual
produce; and, though by agriculture the actual produce would have
been made greater, yet agriculture was not extended. The passion of
the Germans for wine did not lead them to plant vineyards by the
Rhine and Danube, but to rob the vintage of Italy. “Pigrum quin
immo et iners videtur sudore acquirere quod possis sanguine
parare.”[196] Malthus supposes that even the Mark system of land-
holding, with its absence of cities and its periodical redistributions of
land, may have sprung from a political motive, the fear of
accustoming the people to a settled agricultural life, and the desire to
make emigration less irksome to them.[197] So long as there were
weaker peoples to be plundered, the northern nations might freely
double their numbers every twenty-five years, or oftener, and
descend again on Italy and the South. Only when the whole was
occupied by their own people who were not likely to be less stout for
defence than for conquest, were the hordes forced back. Not perhaps
till gunpowder was invented was Europe finally safe against them.
Long after their last inland invasions, the Norsemen found their way
by sea to the shores of England and France.
 Gibbon’s account of the matter is, according to Malthus,
substantially true. The only flaw is that he thinks it necessary, in
denying the greater populousness of North Europe in ancient times,
to deny the possibility of a rapid increase of population.[198] The
German people were on the whole virtuous and healthy in their
manners of living, and, the checks to increase being mainly the
positive ones of war and famine, the increase itself was prodigious.
But Gibbon is greatly in advance of Montesquieu, who believes with
Sir William Temple, Mariana, and Machiavelli, that the northern
countries were, as a matter of fact, more densely peopled then than
they are now, and that, further, when the Romans repelled them, a
huge multitude was driven far north and remained there biding its
time. The same (says Montesquieu) happened under Charlemagne,
and would happen again if a modern prince were to make the same
ravages in Europe; “the nations repulsed to the north, backed against
the limits of the universe, would there make a firm stand, till the
moment when they would inundate and conquer Europe a third
time.”[199] We are to suppose these immense multitudes living “at the
limits of the universe” on ice and air for some hundreds of years. If
this is to answer to the question-begging question, why the North is
less fully peopled than it once was, it involves a miracle. But nothing
more supernatural than ordinary laws is really needed to explain the
movements of pastoral nations a thousand years ago. They are the
same that govern the Tartars and Bedouins now.[200]
In the modern nomads,[201] it is true, the comparative simplicity of
the circumstances and the comparative thoroughness of our
knowledge about them, enable us to see plainly that the local
distribution of the people is in strict accordance with the local
distribution of the food, in other words, with “the quantity of food
the people can obtain in the actual state of their industry and habits.”
We should see the same thing of the rest of the world’s inhabitants, if
the complicated commerce of civilized nations did not make it less
gross and palpable. The power of the earth to support life may be
compared with the power of a horse to carry burdens. He is strong in
proportion to the strength of his weakest part, as a chain to the
strength of its weakest link; and the earth’s powers of nourishment
are great in proportion to their greatness in the worst seasons of the
year.[202] Again, owing to imperfect facilities for distribution, one part
of a society may suffer want when another is in plenty.[203] Among
the Tartars and the Arabs this is plainly seen; and it is clear, too, how
the waste of life from war not only acts as a direct check on
population, but checks it indirectly by repressing productive
industry. Its fruits would have no chance of preservation. “Even the
construction of a well requires some funds or labour in advance, and
war may destroy in one day the work of many months and the
resources of a whole year.”[204] When once warlike habits have
become fixed, the two evils, war and scarcity, reproduce and
perpetuate one another. The encouragements held out to large
families by the Mohammedan religion have a like effect. “The
promise of Paradise to every man who had ten children would but
little increase their numbers, though it might greatly increase their
misery.”[205] It could only increase their numbers if it increased their
food, and it could not increase their food without changing their
warlike habits into habits of industry. Failing this, it simply creates a
constant uneasiness (through want and poverty) that multiplies
occasions of war. Fortunately for himself, the Arab often proportions
his religious obedience to the extent of his resources,[206] and in hard
times, “when there is a pig at hand and no Koran,” he thinks best to
eat what God has given him.
Nothing but increase of food will permanently increase population,
and where there is food the increase will reach up to it. In those parts
of Africa that have furnished the Western slave supplies, there has
been no discernible gap from the “hundred years’ exportation of
negroes which has blackened half America.”[207] Even in Egypt, where
there is a striking contrast between natural fertility and human
lethargy, the cause is not any deficiency in the principle of increase.
It is that property is insecure, the government being despotic and its
exactions indefinite. It is not the want of population that has checked
industry, but the want of industry that has checked population; and
it is bad government that has occasioned the want of industry.
“Ignorance and despotism seem to have no tendency to destroy the
passion which prompts to increase, but they effectually destroy the
checks to it from reason and forethought.... Industry cannot exist
without foresight and security; the indolence of the savage is well
known, and the poor Egyptian or Abyssinian farmer without capital,
who rents land which is let out yearly to the highest bidder, and who
is constantly subject to the demands of his tyrannical masters, to the
casual plunder of an enemy, and not unfrequently to the violation of
his miserable contract, can have no heart to be industrious, and, if he
had, could not exercise that industry with success. Even poverty
itself, which appears to be the great spur to industry, when it has
once passed certain limits almost ceases to operate. The indigence
which is hopeless destroys all vigorous exertion, and confines the
efforts to what is sufficient for bare existence.[208] It is the hope of
bettering our condition, and the fear of want rather than want itself,
that is the best stimulus to industry; and its most constant and best
directed efforts will almost invariably be found among a class of
people above the class of the wretchedly poor.”[209] This passage
repeats an idea expressed in every book of the essay.[210] Government
can retard the increase of population both directly and indirectly, but
can only advance it indirectly, namely, by encouraging industry,
more especially agriculture. For example, industrious agriculture has
made China capable of bearing a great population, though other
causes of a more equivocal character have made it exceed its great
capacities, and its excessive numbers are cut down by famine and
child murder.[211] The Roman emperors found it impossible by
legislation to promote the increase of the old Roman stock, because
they found it impossible to restore the old Roman habits of industry,
though believers in the superior populousness of ancient nations
used to mistake their intentions for accomplished facts.
In the eighteenth-century dispute about the populousness of
ancient nations (one particular skirmish in the general battle of the
books) we have seen that Malthus declares for the moderns. He gives
his opinion in detached passages; but, putting together the different
parts wherever we can find them, we discover his proof to depend on
two principles, which are corollaries of the primary doctrine of the
essay. The first is, that without the extension of agriculture or the
better distribution of its fruits there can be no increase of population;
[212]
the second is, that whatever is unfavourable to industry is to that
extent unfavourable to population.[213]
Now in the early days of Greece and Rome[214] the population
ought on these principles to have been a large one, for not only was
agriculture actively prosecuted, but property and wealth were more
equally divided among the people than in later times. On the other
hand, the numbers were always up to the level of the resources; and
the smallness of the political divisions made law-givers like Solon,
and theorists like Plato and Aristotle, conscious of the risk of over-
population and full of plans to provide against it. It is one of
Aristotle’s criticisms of Plato’s Republic that Plato has not
sufficiently met this difficulty, or realized that a community of goods
or an equal distribution of property is impossible without a
limitation of families. If every one may have as many children as he
pleases, the result will soon be poverty and sedition. Of the
preventive checks actually recommended by the highest wisdom of
the Greek world, the mildest is late marriages; the rest include
exposure and abortion. Colonization was rather adopted in practice
than recommended in theory. Frequent wars and occasional plagues
were the chief positive checks.
In Rome even more evidently than in Greece[215] the causes that
produced inequality of property led also to thinness of population. In
our own days the absorption of small proprietors by large would have
this effect in a less degree, because the large would need to employ
the labour of the small. In Rome the labour was done by slaves; and
the wonder was not that the number of free citizens should decrease,
but that any should exist at all, except the proprietors.[216]
Yet the legislation of Augustus in favour of marriage, and the
universal lamentation of the later Roman writers over the extinction
of the old Roman stock, are no more than a presumption that the
population was decreasing, not a proof of its actual smallness, while
the prevalence of war and infanticide, so often used to prove the
same point, tend really to do the opposite. They are for the time
positive encouragements to marriage, for people will not hesitate to
bring children into the world, if they are either free to kill off the
superfluous or certain to find sad vacancies ready for them.[217] In the
former case, as we noticed, parental feeling will often interfere with
the infanticide, and save rather too many than too few.[218] Wars, on
the other hand, may injure the quality of the population by removing
the most stalwart and even the most intelligent men; but there is as
much food as before, there is more room, and there are therefore
more marriages, till all the gaps are filled, even to overflowing.[219]
Livy need not have wondered that in the Volscian wars the more
were killed the more seemed to come on. The like is true of plague
and famine; epidemics, like the small-pox, have never permanently
lessened the population, though they have increased the mortality of
the infected countries.[220] To take only one instance (from
Süssmilch)—a third of the people in Prussia and Lithuania were
destroyed by the plague in 1710, and in 1711 the number of marriages
was very nearly double the average.[221] Emigration in like manner
may drain off the best blood of a nation, but cannot reduce its
numbers for any length of time, unless the nation is learning a new
standard of comfort. Greece and Rome were not less populous
because they were great colonizers.[222] The known existence of a
number of very active checks to population, instead of proving that
the population was absolutely small, might more naturally, other
things being equal, prove it to have been absolutely large. It might be
argued that, if the population had not been great, fewer and less
potent checks would have done the work.[223]
But other things were not equal. We know that the gratuitous
distribution of foreign corn had ruined Roman husbandry.[224] We
know that even the labour of the slaves who had supplanted the free
labourers of Italy had not been sufficiently (or sufficiently well)
directed to agriculture. Moreover, the increase by marriage in the
number of slaves did not even balance the decrease in the number of
the free men; else why should the Romans need to import fresh
cargoes of slaves every year from all parts of the world?[225]
In short, the Roman habits had become “unfavourable to industry,
and therefore to population.” The very necessity for such a law as the
Papia Poppæa would indicate a moral depravity inconsistent with
habits of industry. This strong argument had escaped even Hume,
who thought that the people would increase very fast under the
Peace of Trajan and the Antonines, forgetting that the people could
not unlearn their habits in so short a time; unlearning is harder than
learning, especially for a whole people; and, “if wars do not
depopulate much when industry is in vigour, peace does not increase
population much when industry is languishing.”[226] Contrariwise, it
might be argued that the prevention of child-murder in India will not
cause over-population, when it is part of a general policy
accustoming the people to European habits.
Allow, then, that general viciousness is inconsistent with general
industry, and it follows that those ancient nations in which the first
prevailed were less populous than the modern. This seems to be the
argument of Malthus brought to a focus. From the absence of