0% found this document useful (0 votes)
44 views

The Elements of Statistical Data Mining, Inferece and Prediction

This document provides a review of several books related to statistics and risk measurement. It discusses how statistics has expanded from traditional hypothesis testing to also include areas like Bayesian inference, data mining, and risk analysis. The review examines key ideas and applications discussed in the books.

Uploaded by

cmgarciasilva
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
44 views

The Elements of Statistical Data Mining, Inferece and Prediction

This document provides a review of several books related to statistics and risk measurement. It discusses how statistics has expanded from traditional hypothesis testing to also include areas like Bayesian inference, data mining, and risk analysis. The review examines key ideas and applications discussed in the books.

Uploaded by

cmgarciasilva
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

level.

This has the advantage that an in- Like the Incredible Hulk, statistics
structor can choose articles depending
on the level of the students.
The Elements of Statistical has burst out of its constricting gar-
ments in several directions. In the
Another example is the choice of pa- Learning: Data Mining, foundational direction, Bayesians, es-
pers concerned with reaction-diffusion pecially those of an objectivist stamp
equations and pattern formation in Inference and Prediction like E. T. Jaynes, have reconnected sta-
Chapter 18. We all know that the Tur- by Trevor Hastie, Robert Tibshirani, tistics with inference under uncertainty,
ing mechanism creates wonderful pat- and Jerome Friedman or rational degree of belief on non-con-
terns, but it is still unclear if this mech- clusive evidence. In the direction of en-
anism is responsible for animal skin NEW YORK, SPRINGER-VERLAG,2001. 533 PP. $82.95 gagement with the large and messy
HARDBACK ISBN 0 387 95284-5
patterns, for example. Taubes's selec- data sets thrown up by the computer
tion of papers shows the controversy REVIEWED BY JAMES FRANKLIN revolution, the disciplines of data min-
quite nicely. An initial publication on ing and risk measurement, represented
fish-pattern is opposed by a second ar- by the books of Hastie et al. and Mar-
ticle, which then is commented on by A standard view of probability and rison, have developed data analysis
the authors of the first article. This statistics centres on distributions and tools well outside the traditional
leaves the true impression that this dis- and hypothesis testing. To solve a real boundaries.
cussion is still open. problem, say in the spread of disease, The essence of Jaynes's position is
Other topics of the text: ODEs, one chooses a "model," a distribution that (some) probability is logic, a rela-
phase-plane analysis, linearization, or process that is believed from tradi- tion of partial implication between ev-
vector-matrix notation, advection, dif- tion or intuition to be appropriate to idence and conclusion. According to
fusion, separation of variables, reaction- the class of problems in question. One this point of view, statistical inference
diffusion equations, pattern formation, uses data to estimate the parameters of is in the same line of business as "proof
traveling waves, periodic solutions, the model, and then delivers the re- beyond reasonable doubt" in law and
fast and slow dynamics, and chaos. sulting exactly specified model to the the evaluation of scientific hypotheses
Although the text is not suitable for customer for use in prediction and in the light of experimental evidence.
a course in mathematics, the enormous classification. As a gateway to these Just as "all ravens are black and this is
number of well-chosen references mysteries, the combinatorics of dice a raven" makes it logically certain that
makes it a useful addition for the shelf and coins are recommended; the ener- this is black, so "99% of ravens are
of a generally interested researcher. As getic youth who invest heavily in the black and this is a raven" makes it log-
Taubes says in his preface, his "goal is calculation of relative frequencies will ically highly probable that this is black
to introduce to future experimental bi- be inclined to protect their investment (in the absence of further relevant ev-
ologists some potentially useful tools through faith in the frequentist philos- idence). That is why the results of drug
and modes of thought." ophy that probabilities are all really rel- trials give rational confidence in the ef-
ative frequencies. Those with a taste fects of drugs. Galileo and Kepler used
Department of Mathematical and Statistical for foundational questions are referred the language of objective probability
Sciences to measure theory, an excursion from about the way evidence supported
University of Alberta which few return. their theories, and in the last hundred
Edmonton, Alberta T6G 2G1 That picture, standardised by Fisher years a number of books have filled out
Canada and Neyman in the 1930s, has proved the theory of logical probability--
e-mail: [email protected] in many ways remarkably serviceable. Keynes's Treatise on Probability (the
It is especially reasonable where it is great work of his early years, before he
known that the data are generated by went on to easier pickings in econom-

Probability Theory: The a physical process that conforms to


the model. It is not so useful where the
ics), D. C. Williams's The Ground of In-
duction, George P61ya's Mathematics
Logic of Science data is a large and little-understood
mess, as is typical in, for example, in-
and Plausible Reasoning, and now
E. T. Jaynes's posthumous master-
by E. T. Jaynes
surance data being investigated for piece, Probability Theory: The Logic
CAMBRIDGE, CAMBRIDGEUNIVERSITYPRESS, 2003. fraud. Nor is it suitable where one has of Science.
727 PP., $65.00, HARDBACK ISBN 0-521-59271 2 several speculations about possible Jaynes's school are called "objective
models and wishes to compare them, Bayesians" or "maxent Bayesians,"

The Fundamentals of Risk or where the data is sparse and there


is a need to argue about prior knowl-
to distinguish them not only from
frequentists but from "subjective
Measurement edge. It is also weak philosophically,
in failing to explain why information
Bayesians," who think that any degrees
of belief are allowable, provided they
by Chris Marrison
on relative frequencies should be rele- are consistent (that is, obey the axioms
BOSTON, McGRAW HILL, 2002. 415 PP. $44.95 vant to belief revision and decision- of probability such as that the proba-
HARDBACK iSBN 0 07 138627 0 making. bility of a proposition and its negation

9 2005 Springer Science+BusinessMedia, Inc,,Volume27, Number2, 2005 83


must sum to one). The objectivists em- the Bank of England in the U.K.). Mar- play a role similar to Basel II in bank-
phasise, on the contrary, that one's de- rison's book is an excellently written ing, in enforcing higher standards of
gree of belief ought to conform to the introduction to the standard ideas in mathematical competence. It will be
degree to which one's evidence does the field. It avoids the unnecessary el- necessary to price options reasonably
logically support the conclusion, in ements in usual statistics courses and in the interests of their truthful display
asking why evidential support should goes immediately to the most applica- on balance sheets, for example. The
satisfy the axioms of probability the- ble concepts. These include the "value- book for accountants corresponding to
ory, objectivists have been much im- at-risk" formalism, which measures Marrison's appears not yet to be writ-
pressed by the proof of R. T. Cox the loss such that worse losses occur ten, so there may be a gap in the mar-
(American Journal of Physics, 1946) exactly 1% (say) of the time, and the ket for an ambitious textbook writer
that any assignment of numbers to the concepts needed for precision in han- who would like to become very rich
relation of support between proposi- dling rare losses, such as heavy-tailed very quickly.
tions which satisfies very minimal and distributions and correlations be- If there is a dispute in statistics as
natural logical requirements must obey tween the losses of different financial heated as that between frequentists
the standard axioms of conditional instruments. It is significant, for ex- and Bayesians, it is that between tra-
probability. They have been corre- ample, that foreign exchange rate ditional statisticians and data miners.
spondingly unimpressed by supposed changes resemble a random walk, but Data mining, with its roots in the neural
paradoxes of logical probability that are heavy-tailed, are heteroskedastic networks and decision trees developed
purport to demonstrate that one can- by computer scientists in the 1980s, is
not consistently assign initial probabil- a collection of methods aiming to un-
ities. In some of his most entertaining Mathematicians, derstand and make money from the
pages, Jaynes exposes these "para- massive data sets being collected by
doxes" as exercises in pretending not pure and applied, supermarket scanners, weather buoys,
to know what everyone really does intelligence satellites, and so on.
know. His reliance on symmetry prin- think there is "Drink from the firehose of data," says
ciples to assign initial probabilities the science journalist M. Mitchell Wal-
shows its worth, however, well beyond something weirdly drop. It is not easy--and especially not
such philosophical polemics. In in- with the model-based methods devel-
verse problems like image reconstruc- different about oped by twentieth-century statisticians
tion, where the data grossly under- for small and expensive data sets. With
determines the answer, it is essential statistics. a large data set, there is a need for very
to assign initial probabilities as non- flexible forms to model the possibly
dogmatically as possible, in order to (variable in "volatility," that is, stan- complicated structure of the data, but
give maximum room for the data to dard deviation), and have some ten- also for appropriate methods of
speak and point towards the truth. dency to revert to the mean. It is per- smoothing so that one does not "over-
Jaynes's maximum entropy formalism haps surprising to learn that credit fit," that is, learn the idiosyncracies of
allows that to be done. ratings are intended to mean absolute the particular data set in a way that will
In the business world, there is the probabilities--a AAA rating means not generalise to other sets of the same
same need as in science to learn from one chance in 10,000 of failure within kind. Are specialists in data mining (or
data and make true predictions. But a year; naturally it is hard to ground so "analytics" as they now often prefer)
among other forces driving the expan- small a probability in data, so one pre- pioneers of new and exciting statistical
sion of commercial statistics are the sumes that credit rating agencies will methodologies, or dangerous cowboys
new compliance regimes in banking need to use priors and qualitative evi- lacking elementary knowledge of sta-
and accounting. Following a number dence (a euphemism for market ru- tistical models? Those who enjoy vig-
of corporate scandals and unexpected mours?) in the style of Jaynes. Marri- orous intellectual debate will want
collapses, the world governing bodies son's insights into how bank risk to read data miner Leo Breiman's pug-
in banking and accounting have de- teams really work is enlivened by oc- nacious "Statistical modelling: the
cided on standards that include, casionai dry humour: in pointing out two cultures," Statistical Science 16
among other things, risk measure- that profits from risky trades need to (2001), 199-219, with a marvellously
ment. The Basel II standard in banking be discounted, he adds, "Convincing supercilious reply on behalf of the tra-
says, in effect, that banks may use any traders that their bonuses should be ditionalists by Sir David Cox. As an in-
sophisticated statistical methodology reduced according to Allocated Capi- troduction to the field for practitioners
to measure their overall risk position tal • HT is left as an exercise for the in the business world, Michael Berry's
(in order to determine the necessary reader." Mastering Data Mining (New York,
reserves), provided they disclose their In accounting, the forthcoming IFRS Wiley, 2000) is often recommended,
methods to their national banking reg- (International Financial Reporting but for mathematicians interested in
ulator (the Federal Reserve in the U.S., Standards) compliance standard will understanding the field, Hastie et al.'s

84 THE MATHEMATICALINTELUGENCER
Elements of Statistical Learning is the heading "ethno-sciences"? But what
ideal introduction. Assuming basic sta-
tistical concepts and an ability to read
Mathematics Across Cultures then are the criteria that include sci-
ence in Mesopotamia or ancient Egypt
formulas, it runs through the methods Helaine Selin and Ubiratan in ethno-sciences, but exclude Greek
of supervised learning (that is, gener- D'Ambrosio, editors science, although all the authors in
alisation from data) that have come Greco-Hellenistic Antiquity regard sci-
KLUWER ACADEMIC PUBLISHERS,2000, 479 PAGES
from many sources: neural networks, HARDBOUND, ISBN 0-7923-6481-3, ~ 195.50
ence in ancient Egypt as the origin of
kernel smoothing, smoothed splines, PAPERBACK, ISBN 1-4020-0260-2, ~63.00 Greek science? Why should Arab math-
nearest-neighbour techniques, logistic ematics, the heir to Greek mathemat-
REVIEWED BY HELENE BELLOSTA
regression and newer techniques like ics, whose contribution is essential to
bagging and boosting. The unified understand the constitution of classical
treatment and illustration with well- his book is meant as a supplement mathematics in 17th-century Europe, be
chosen (and well-graphed) real data- T to the Encyclopaedia of the His- included in ethno-mathematics?
sets makes for efficient understanding tory of Science, Technology and Med- This book seems to make a rather
of the whole field. It is possible to icine in Non-Western Cultures (Kluwer strange division. On one side we have
appreciate how different methods are Academic Publishers, 1997) and is ethno-sciences, bringing together sci-
really attempting the same task--for aimed at a more scholarly audience; ences as different as science in ancient
example, that classification trees de- the aim is to explore the same topics China and science in present-day Abo-
veloped by computer scientists to suit in greater depth. riginal societies, these being viewed as
their discrete mindset are really per- The book is divided into two parts: sciences of unusual societies, the pe-
forming non-linear regression. But the the authors of the six essays in the first culiarities of which, together with their
differences between methods are well section try to define the field of ethno- incommunicabihty, some papers dili-
laid out too: the table on p. 313 com- mathematics and to make a general gently stress; and on the other, by de-
pares the methods with respect to such study of the connection between math- fault, Greek science, European science
crucial qualities as scalability to large ematics and culture as well as the vari- from the Renaissance to nowadays as
data sets, robustness to outliers, han- ability of the concept of rationality, well as science in the USA, would be
dling of missing values, and inter- while the second part is devoted to the left as non-ethnic sciences (white sci-
pretability. The less-tamed territory of description of fifteen individual cul- ence versus colored science?). If we
unsupervised learning, such as cluster tures and their mathematics in various continue to follow the unspoken logic
analysis, is also well covered. One "non Euro-Anmrican" areas: the Middle of this division, these sciences should
topic of current interest missing is the East, America (native cultures), the Pa- then show the opposite qualities and be
attempt to infer causes from data, but, cific and Australia, Africa, and the Far a contrario universal. We should not
as is clear from Richard Neapolitan's East. be surprised then to find here and there
Learning Bayesian Networks (Har- If the intention behind the b o o k - - t o in some papers hasty judgments and
low, Prentice Hall, 2004), that theory is rehabilitate the so-called non-Western worn-out commonplaces on these "dif-
still in a primitive state. Spatial statis- cultures and to denounce the damag- ferent" civilizations, which could be de-
tics and text mining are not covered ei- ing effects of cultural imperialism and fined as the eurocentrism the editors
ther; they too await readable textbooks eurocentrism, the consequence of intended to stigmatize: "The transfor-
of their own. which is a certain contemptuous dis- mation of the word science as a dis-
Mathematicians, pure and applied, regard for these cultures--is highly tinct rationality valued above magic is
think there is something weirdly dif- laudable, this enterprise is not entirely uniquely European" (H. Selin, p. vi) or
ferent about statistics. They are right. free from danger. The main difficulty is "the development of this concept of ra-
It is not part of combinatorics or mea- defining and naming the field of study: tionality (i.e., European's 17th century)
sure theory but an alien science with how should we divide sciences into was not universal. For example, it was
its own modes of thinking. Inference Western and non-Western, or Euro- not paralleled in Islamic society where
is essential to it, so it is, as Jaynes pean and non-European? The criterion men were denied rational agency; they
says, more a form of (non-deductive) is not geographical but cultural (H. were held to lack the capacity to
logic. And, unlike mathematics, it Selin, p. v), for the studies in this book change nature or to understand it.
does have a nice line in colourful deal with mathematics in the Far and Knowledge was instead to be derived
polemic. Middle East, as well as mathematics in from traditional authority" (D. Turn-
Aboriginal, Amerindian, or African so- bull, Rationality and the disunity of
cieties. Should we, as some authors do, the sciences, p. 47). One of the authors
School of Mathematics speak of "non-modern" or "traditional" (R. Eglash, Anthropological perspec-
University of New South Wales sciences, even though this mixes up tives on ethnomathematics) is clearly
Sydney 2052 different eras, from the 3rd millennium conscious of the difficulty of defining
Australia BC to today? Should we then group what ethno-mathematics or non-West-
e-mail: [email protected] these sciences together under the ern mathematics actually are, and also

9 2005 Springer Science+BusinessMedia, Inc. Volume27, Number 2, 2005 8 5

You might also like