The Diagnostic Process: Graphic Approach to Probability and Inference in Clinical Medicine

Copyright © 2013 by RUDOLF ZALTER, M.D.

Library of Congress Control Number: 2013910345

ISBN: Hardcover  978-1-4836-5031-9

           Softcover   978-1-4836-5030-2

           Ebook        978-1-4836-5032-6

All rights reserved. No part of this book may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without permission in writing from the copyright owner.

Rev. date: 08/16/2013

Xlibris LLC

1-888-795-4274

www.Xlibris.com

127427

This book addresses the decision making process under uncertainty. The process commonly encountered in all fields of human endeavor is called the diagnostic process in this monograph.

The thrust of this book is to help the struggling student, of all ages, in all fields, to cross the threshold from rote to comprehension, thus bridging an intuitive gap left in many a reader’s mind regarding the significance and clinical implication of the accompanying probability data.

The text is, in essence, a verbal and graphic portrait of the basic ideas and symbolic structure of probability and statistical inference with particular stress on the Bayesian version. It aims to expound in words, simile, and diagrams the inherent connections obtained between a given event and its sample space or between a given random sample and a hypothesized population. In this sense, no formula is left naked to be absorbed on its face value without the support of a graphic cover. The final result is a firm grasp of the simple concepts that make the infrastructure (not the superstructure) of the subject.

Nonetheless, this is not another book on statistics. It certainly is not a textbook geared for the classroom, and it contains no problem to solve other than those structured and graphed examples needed to clarify and illustrate the thrust of the point under consideration. The book deals exclusively with the two topics that I tend to believe are the core thesis of statistics, namely, probability and its counterpoint, inference, supported by the necessary exposition of sets. Thus, the book does not include the mandatory and important chapters on analysis of variance, regression, and correlation.

Contents

Introduction

Part I

Sets, Counting, and Probability

Chapter 1: Sets

1.1 Sets in Perspective

1.2 Definition

1.3 Sets and Subsets

1.4 The Universal Set and Its Subsets

1.5 Rules Governing Set Operations

1.6 Basic Set Operations

1.7 Combined Set Operations

1.8 Extended Venn Diagrams

1.9 Blood Phenotypes

Chapter 2A: Counting Techniques and Enumeration Methods

2A.1 The Multiplication Principle

2A.2 Permutation

2A.2.1 Counting Ordered Arrangements with

No Repetition or Replacement

2A.2.2 Counting Ordered Arrangements with

Repetition or Replacement

2A.2.3 Counting Ordered Arrangements with

Fixed Repetition or Replacement

2A.3 Combination

2A.3.1 Counting Unordered Combinations with

No Repetition or Replacement

2A.3.2 Counting Subsets with Repetition or Replacement

2A.4 Practical Application

2A.5 Questionnaire and Review

Chapter 2B: Partition and Occupancy

2B.1 The Distribution of Balls into Cells

2B.2 Ordered Partition

2B.3 Allocation in Practice

2B.4 Partition and Occupancy

Chapter 3: Switching Circuits and Gates:

The Alternative to Sets

3.1 Sets and Switching Circuits Operations

3.2 Logic and Set Operations

3.3 Probability and Set Operations

Chapter 4: Probability

4.1 An Overview

4.2 A Simple Experiment

4.3 The Sample Space

4.4 The Event: Simple and Compound

4.5 Probability and Randomness

4.6 Classical Probability: Probability of Simple Events

4.7 Probability of Compound Events

4.8 Probability of Not Equally Likely Outcome:

Relative Frequency Model

4.9 Probability Mode: Simple versus Conditional

4.10 Probability Interaction:

The Addition Rule of Probability

4.11 The Multiplication Rule of Probability

4.12 Contingency Tables, Venn Diagrams,

and Tree Diagrams

4.13. The Odds Alternative to Probability

4.14 The Random Variable

Chapter 5: Conditional Probability

5.1 Conditional Probability

5.2 Graphic Overview

5.3 Illustrative Examples

5.4 Basic Rules of Probability Theory

5.4.1 Summation Rule

5.4.2 Multiplication Rule

5.5 Probability in Words, Symbols, and Venn Diagrams

5.6 Uniform Distribution: The Classical Model

5.6.1 Permutation Approach without Replacement

5.6.2 Permutation Approach with Replacement

5.6.3 Combination Approach

5.6.4 Conditional Probability Approach

5.6.5 Complement Approach

5.6.6 Double Combination Approach

5.6.7 Generalized Double Combination Approach

5.6.8 Triple Combination Approach: Red, White,

and Yellow Balls

5.7 The Sample Space Out of the Urn

5.8 Mad Cow Disease

5.9 The Probability of Being on Time

5.10 The Sample Space Whose Elements are not Equally Likely

5.11 The Double Urn Problem in Jest and Reality

5.12 Lyme Disease: Proved and Unproved

5.13 Prostatic Cancer: Confirmed and Excluded

5.14 Probability in the World of Chance

5.14.1 The New York Lotto

5.14.2 The Poker Game

5.15 The Game of Craps

5.16 The New York Times: Opinionator

5.17 Example

Chapter 6: Dependence versus Independence

6.1 Mutually Exclusive Case

6.2 Mutually Nonexclusive Case

Chapter 7: Clinical versus Diagnostic Conditional Probability

7.1 Diagnostic Marker Cooperative Study for the

Diagnosis of Myocardial Infarction

7.2 Timed Conditional Probability: Risk Assessment

7.3 Prevalence, Incidence, and Risk

Part II

Bayes’ Theorem

Chapter 8: Bayes’ Theorem:

Descriptive and Graphic Approach

8.1 Numerical and Graphic Equivalents of the

Diagnostic Procedure

8.2 Evaluation of the Operating Characteristics of

Diagnostic Test

8.3 Posttest Diagnostic Procedure

Chapter 9: Bayes’ Theorem: Formal Approach

9.1 Bayes’ Theorem: Odds and Likelihood Alternative

Chapter 10: Prevalence, Sensitivity, Specificity,

and Predictive Value

10.1 Graphic Overview

10.2 Sensitivity

10.3 Levels of Sensitivity: Diagnostic Implication

10.4 Indication for High-Sensitivity Tests

10.5 The Complement of Sensitivity

10.6 Specificity

10.7 Levels of Specificity: Diagnostic Implication

10.8 Indication for High-Specificity Tests

10.9 The Complement of Specificity

10.10 Sensitivity versus Specificity

Chapter 11: The Discrete Random Variable Distribution:

An Overview

Chapter 12: The Odds Paradigm as the Popular Mode

Part III

Clinical Application

Chapter 13: HIV Infection

13.1 HIV Screen in Blood Banks:

Excerpts from a Debate in the Literature

Chapter 14: Tumor Markers

Chapter 15: Myocardial Infarction in the

Emergency Department

Chapter 16: Stress Test

16.1 Coronary Artery Disease

16.2 The Ischemic Response: Physiologic Background

16.3 The Diagnostic Impact of the Predictive Value:

Exercise Stress Test in Perspective

Chapter 17: Syphilis and Lyme Disease: Diagnostic Strategy

17.1 Targeting a Test Order

17.2 Testing for Syphilis and Lyme disease

Part IV

Statistical Inference

Chapter 18A: The Discrete Random Variable

Probability Distribution

18A.1 The Conversion of Raw Data into Empirical

Frequency Distribution

18A.2 The Transformation of Empirical Frequency Distribution

18A.3 The Concept of Function as an Operation

18A.4 The Random Variable

18A.5 The Probability Mass Function: The Probability Distribution Function for Discrete Random Variable

Chapter 18B: The Binomial Probability Distribution

18B.1 Bernoulli Random Variable

18B.2 The Binomial Random Variable

18B.3 Binomial Probability Distribution: p = q

18B.4 Graphic Display of the Discrete Binomial

Probability: p = q

18B.5 The Expected Value

18B.6 Binomial Probability Distribution: p ≠ q

18B.7 Probability Distribution: b(x;n,p)

18B.8 Finite versus Continuous Probability Distribution

18B.9 The Binomial Probability Distribution

18B.10 Clinical Application of the Binomial Distribution

18B.11 Malignant Viral Infection

18B.12 New Antibiotic

18B.13 Heart Transplant Program

18B.14 Public Health Problem

18B.15 Critical Care Technique

18B.16 Critical Care Procedures Proficiency

18B.17 Monitoring System Failure

18B.18 Frequency of Procedures

18B.19 Question of Life Expectancy

18B.20 The Birthday Problem

18B.21 Terminology in Probability

18B.22 Probability of Error

Chapter 18C: The Continuous Probability Distribution

18C.1 Probability Density Function (PDF)

18C.2 The Normal Distribution as an Approximation of the Binomial Distribution

Chapter 19A: The Normal Probability Distribution

19A.1 The Normal Distribution

19A.2 The Normal Probability Density Function:

The Normal Curve

19A.3 The Standard Normal Distribution Curve

Chapter 19B: Z Tables

Chapter 20: Statistical Inference

20.1 Probability Revisited

20.2 The Interrelation Between Random Sample,

Sampling Distribution, and Population

20.3 Random Sample: The First Entity

20.4 Population: The Second Entity

20.5 Sampling Distribution: The Third Entity

20.6 The Population and Its Sampling Distribution:

The First Connection

20.7 Relation of the Sample Mean to the Sampling

Distribution: The Second Connection

(Known Population Parameters)

20.8 Relation of the Sample Mean to the Sampling

Distribution: The Second Connection

(Unknown Population Parameters)

20.9 Relation of the Sample Mean to the Population Mean:

The Third Connection (Known Population Parameters)

20.10 Relation of the Sample Mean to the Population Mean:

The Third Connection (Unknown Population Parameters)

20.11 Statistical Estimation

Chapter 21A: Statistical Inference: Hypothesis Testing

21A.1 The Sample-Population Relation

21A.2 Formal Approach to Hypothesis Testing

21A.3 The p Value and the Observed Significance Level

21A.4 Definitions of α

21A.5 The Interrelationship of p and α

21A.6 Difference between p and α

21A.7 Reporting p and/or α

Chapter 21B: Hypothesis Testing in Action

21B.1 Population versus Sampling Distribution

21B.2 Population Mean versus Sample Mean

21B.3 Sample Mean vs. Sampling Distribution of the Mean

21B.4 Statement and Formulation of Two Opposing Hypotheses

21B.5 Placement and Siting of the Sample Mean

21B.6 Prior Specification and Setting of the Significance Level

21B.7 Weighing the Observed versus the

Assigned Significance Lever

21B.8 Statistical Decision and Conclusion

Chapter 21C: Controlling Errors in Hypothesis Testing

21C.1 Two-Tailed Test

21C.2 Type I Error: α Error

(Rejecting a True Null Hypothesis)

21C.3 Type II Error: β Error

(Failing to Reject a False Null Hypothesis)

21C.4 Control of the Type I Error, α

21C.5 Control of Type II error, β

21C.6 One-Tailed Test

21C.7 Operating-Characteristic Curve versus Power Curve

21C.8 Factors Affecting Type II Error

21C.9 Simultaneous Control of Type I and Type II Errors

(α and ß)

Chapter 22: Bayes’ Theorem from a Statistical Perspective

22.1 Inductive versus Deductive Reasoning

22.2 Probability versus Statistics:

Deductive Inference versus Inductive Inference

22.3 The Scientific Method: Applied Inductive Reasoning; Applied Hypothesis Testing

22.4 The Reference Interval: Arbiter of the Test Result

22.5 Statistical versus Bayesian Inference

The Graphic Interpretative Approach of Symbolic Idiom

The Ideal Mode of Comprehending the Subject

Introduction

The diagnostic enterprise is a multifaceted systematic approach to the central issue of clinical medicine. This monograph is an attempt to introduce the interested reader to the use of probabilistic logic as a practical tool in clinical medicine. To this end, I tend to believe that the use and manipulation of mathematical symbols as a pure logical instrument, although scientifically irreplaceable, seem to detract from its wider application as a practical tool in everyday communication.

This is particularly true in dealing with uncertainty, where an intuitive appreciation of probability, perhaps more than the capacity to manipulate its symbolic language, is essential. In this case, more so than in other branches of mathematics, the exclusive use of symbolic notation devoid of its graphic equivalent is a restrictive impediment that tends to curtail the imaginative thrust in visualizing the interrelations the problem presents. The same reservation could be raised at the higher levels of mathematical endeavor, although here again the advent of computer-generated graphic analogs to the complex mathematical functions makes it possible to configure at least in the three-dimensional space the structural anatomy of the function.

Nonetheless, this is not another textbook on statistics. It is, in essence, a verbal and graphic portrait of the basic ideas of statistical inference and Bayesian decision analysis so that once a firm grasp of the simple concepts that make the infrastructure of this subject, indeed of all subjects including this one, has been assimilated, the subject matter, no matter how esoteric, is at once rendered comprehensible, logically obvious, and certainly less formidable.

Bayesian decision theory in its modern context is being increasingly applied in diverse fields ranging from business management to medical diagnosis, where the decision-making process under uncertainty is commonly encountered.

In the medical field, the focal point of the diagnostic process is concerned with the resolution of uncertainty through the ability to zero in on the tentative diagnosis and either confirm or rule it out. It is only with this purpose in mind that a given test is ordered and its results evaluated. With the tentative diagnosis in the background, a given test result confronts the physician with the problem of assessing the test result’s credibility in contrast to the prior perception in the given case. At its basic level, the credibility factor may be reduced to one of two possible alternatives, either in support or in denial of what had been formulated up to this moment as a tentative diagnosis or working hypothesis to use the mathematical jargon, which is considered as the best fit to account for the clinical data thus far available.

Yet despite the seemingly complex rules and quaint mathematical script, the core idea underlying the act of decision making is a common tool of everyday experience, which can be summarized as follows.

Given a test result, specifically ordered in connection with a tentative diagnosis previously promulgated on the available evidence, the formal approach, impractical as it may appear, is to determine the predictive accuracy of a positive test result in support of the disease in question or equally the predictive accuracy of the negative result in excluding the disease in question.

Alternatively, the question may be phrased in terms of the predictive error incurred, given a positive result in the absence of disease or a negative result albeit the presence of disease. To answer the two basic accuracy parameters, one needs to appreciate that predictive value or accuracy is a complex function of three independent variables, namely, that of sensitivity, specificity, and prevalence.

What Bayes’ theorem offers is a quantitative appraisal of the credibility factor attached to the given test result. The quantification of a yes or no decision is the extra dimension required to fine-tune the diagnostic process, which is, in essence, an iterative and sequential pattern of inductive reasoning leading to a final conclusive diagnosis. This approach to diagnosis may not be practical at all levels of the diagnostic process. Other modalities of diagnosis, including the intuitive pattern recognition, are powerful tools in the hands of experienced clinicians. It is however in the area of decision making, which requires the integration of diverse and specific information, that a quantitative probabilistic approach to decision assumes its unique function. Under certain conditions, decision making is not an optional exercise in judgment strategy that can be dispensed with; it is in effect the only mode of rational decision applicable under the complex circumstances of therapeutic or diagnostic decisions.

This definitive assertion must be partially retracted or modified in field. The numerical manipulation of the data, other than being not always practical in the clinical setting, is of marginal accuracy in view of the fact that prevalence data in particular for the relevant patient subset is most often lacking and at best subjective.

In the modern medical literature, Bayes’ theorem in its esoteric probabilistic form and in its simplified variant of sensitivity, specificity, and predictive value has now been in vogue for several decades. It has by all accounts penetrated the medical literature such that all new therapeutic modalities and diagnostic procedures are routinely bolstered by the basic quota of relevant probabilistic data. Yet despite the ubiquitous routine derivation of the necessary fundamental characteristics of a given procedure or test, an intuitive gap is left in many a reader’s mind regarding the significance and clinical implication of the accompanying probabilistic data.

Given the previously mentioned considerations, the road to a graphic interpretation of probability theory including its Bayes’ formulation must perforce detour through the domain of set theory as the graphic adjunct necessary to the understanding of probabilistic logic of inductive and deductive inference.

With this in mind, a great deal of effort has been made at the graphic level to unravel among other ideas the delicate concepts of dependence and independence in the context of conditional probability.

The monograph is subdivided into four parts:

Part I is an attempt to translate the key concepts of probability into a structured descriptive language, supported and amplified by its graphic counterpart. This approach presupposes a working knowledge of set theory of which the first chapter is a brief introduction supplemented by the necessary counting and enumeration methods.

Part II is formal approach to Bayes’ theorem. It is developed utilizing probability rules and set notations. The clinical correlates of sensitivity, specificity, and predictive value are specified both verbally and graphically. The odds and likelihood alternative to probability is considered in a separate chapter.

A special three-part chapter is interposed before the clinical section as an overview of probability and its corollary of frequency distribution supported by a variety of practical examples that may be familiar to the general reader in order to demonstrate the common bond of probabilistic logic, both medical and otherwise.

Part III presents the practical application of the theorem brought forth in a series of detailed relevant clinical situation in which the decision-making process is formulated to reflect the conditional probabilities extant under a given situation.

An optional side trip to symbolic logic as applied to electronic switching circuits is suggested as an additional perspective or a complement to set theory. To this end, a special chapter (chapter 3) has been added to supplement set theory. This may sound as a superfluous assignment for a simple nonmathematical introduction to probability because the diversity of the three subjects may appear on casual inspection to be lacking a common thread with questionable justification for an artificial although optional side trip. Nevertheless, it may be demonstrated that despite the divergence in the subject matter, the theoretical foundation of all three is analogous. It is this analogy in the axiomatic underpinning and the derived operative laws that unifies and simplifies the understanding and practical application of all three subjects. The common unifying language shared by the three related subjects is Boolean algebra using specialized fonts and notation to accommodate the specific subject matter. The diversity of subject as well as the divergent historical development of the individual subject resulted in the application of different sets of notation (one might call it special script or font) to cover the algebraic operations of sets, symbolic logic, and switching circuits. Indeed, it could be argued that all three subjects are special, and a unique application of Boolean algebra if the historical development of the individual subjects is not taken into consideration.

Part IV presents the classical statistical inference both as an estimation or a hypothesis testing approached via the sampling distribution. Here again, the graphic approach to the concept is paramount.

Given the limitations imposed by this approach, the focus on the graphic elements of the theory may prove rewarding if the core idea of probability theory can be transmitted with a minimum of mathematical luggage. In this somewhat unorthodox attempt to present a complex subject, mathematical rigor is perforce a casualty, and the mathematically oriented reader may object to the unwarranted intrusion into a sanctioned territory through simplistic expositions of well-defined algebraic operations and rules. The only consolation that one can offer is that all modalities of human thought are ultimately defined and shared by one unique reasoning process.

Personal Note

This manuscript was relegated to the dormant shelves until now.

A second look prompted by a friendly reviewer initiated this delayed attempt at self-publishing. This monograph is neither a statistical treatise nor a clinical dissertation on selective medical topics. It is essentially an attempt to demystify the logical symbolic steps involved in the diagnostic process and to place them in their correct category as an exercise in Bayesian decision framework with particular emphasis on their graphic analog as the ideal medium of bridging the comprehension gap. Excerpts from a New York Times article on a relevant pulse-quickening topic are reprinted at the end of the introduction, illustrating the extent of the logical gap in the process.

In the present medical setting, the diagnostic process is sometimes intuitive, frequently based on pattern identification by the astute clinician and very often unwittingly invoking a probability paradigm, albeit without the solid backing of the conditional order involved.

Please note that tests, procedures, and clinical data used in this book may not be the current standard in medical practice. This should in no way deflect one’s attention from the leitmotif of this treatise, namely, the diagnostic process and its graphic match. This of course is independent of new or old medical tests or procedures because it is not the relevance of outmoded clinical data and/or procedures that is at issue but the context in which the data were used in order to graphically illustrate the probabilistic and or inferential model and thus bring to light what may not be apparent to the general science reader, given the abstract contents of solid mathematical reasoning. Nevertheless, a strong caveat about the mathematical rigor needs to be considered, given the graphic and verbal slant of this presentation.

Finally, I am fully aware the no manuscript can escape the editorial scissors and remain intact given that errors, both inadvertent and heuristic, are unavoidable at first try.

Historical Footnote

The principles of the theory of probability are credited to a series of correspondence between Blaise Pascal (1623-1662) and Pierre Fermat (1602-1665) trying to resolve a question regarding the actual odds encountered in a gambling controversy.

Thomas Bayes’ (1702-1761) classic paper, An Essay toward Solving a Problem in the Doctrine of Chance, which is the main topic of this monograph, was published posthumously in 1763. The work antedates Boole’s algebraic formulations. It occupies a central position because it is the first attempt at a precise quantitative definition of inductive inference.

Symbolic logic, however, was a fully developed field of mathematics formulated by Gottfried Wilhelm Von Leibnitz (1646-1716) as a system of mathematical logic.

George Boole’s (1815-1864) monumental publication, An Investigation of the Laws of Thought on Which Are Founded Mathematical Theories of Logic and Probabilities, laid the foundation for the unique algebra common to both fields.

Set theory in its present form owes its formal inception to George Cantor (1845-1918). The Venn diagrams extensively used in this monograph owes its current perfected version to John Venn (1834-1923).

Symbolic logic transformation from a language technique to an indispensable tool of modern digital electronic may be dated as late as 1938 when Claude Shannon (1916-2001) published his seminal paper A Symbolic Analysis of Relay and Switching Circuits, which was to become the basis for the logical design of digital computers and switching systems.

The diversity of subject ranging from symbolic logic to switching circuits as well as the divergent historical development of the individual subject may have been a factor in the application of different sets of notation (one might call it special Script or Font) to cover the algebraic operations of the individual subject.

The New York Times: Opinionator

April 25, 2010

Chances Are

by Steven Strogatz

Excerpts from the Article

Perhaps the most pulse-quickening topic of all is conditional probability. In one study, Gerd Gigerenzer (the author of Calculated Risk) asked doctors in Germany and the United States to estimate the probability that women with a positive mammogram actually has breast cancer given the following statistics:

The probability that one of these women has breast cancer is 0.8 percent. If a woman has breast cancer, the probability is 90 percent that she will have a positive mammogram.

If a woman does not have breast cancer, the probability is 7 percent that she will still have a positive mammogram.

Imagine a woman who has a positive mammogram. What is the probability that she actually has breast cancer?

When Gigerenzer asked twenty-four German doctors the question, their estimate whipsawed from 1 percent to 90 percent. Imagine how upsetting it would be as a patient to hear such divergent opinions.

Of one hundred American doctors, ninety-five estimated the woman’s probability of having breast cancer to be somewhere around 75 percent.

For a detailed answer, please refer to the end of chapter 5 on conditional probability.

Part I

Sets, Counting, and Probability

Chapter 1

Sets

1.1 Sets in Perspective

In the context of this monograph, the concept of sets and its corollary of counting the elements in a given set need to be fully explored as the necessary background to the all-important subject of probability.

1.2 Definition

In general, a set defines a collection or aggregate of distinct elements (objects and points), real or conceptual. The elements of a set run the gamut from the finite number of the letters of the alphabet to the infinite number of integers.

We call the objects that make up an arbitrary set the members of the set of the elements of the set.

Sets are represented by one of two methods:

1. Roster method. In this method, the elements that belong to the set are individually listed and enclosed in braces. The order of listing being immaterial. Thus, each of the symbols {a, b, c}, {a, c, b}, and {b, a, c} stands for the same set, namely, the set consisting of the elements a, b, and c. In general, a set is known once its members are known.

N = {1, 2, 3, 4, 5, 6}, a set of six elements comprising the integers 1 through 6.

S = {a, b, c, d}; S is the set of elements a, b, c, and d.

2. Descriptive method. In this method, the elements are not listed individually. Instead, the properties common to all elements are listed as necessary qualification for each and every element of the set in question.

N = {n | 0 < n <7}, the set of all numbers, such that n is between 0 and 7.

D = {d | male patient members of a pulmonary clinic, with resting PO2 < 60 mm Hg}, the set of all male patients, d, such that d satisfies the previously noted restrictions.

Lowercase letters are usually used for elements and capital letters for sets.

1.3 Sets and Subsets

If A and B are sets, and if every member of A is also a member of B, then set A is said to be a proper subset of the set B (i.e., A is included in, or is contained in, B). Thus, A is a proper subset of B if and only if every element of A is an element of B and there exists at least one member of B that is not a member of A.

For example, the set {1, 2} is a subset of the set {1, 2, 3}, but {4} is not a subset of {1, 2, 3}.

The improper subset of set B is the set itself. Thus, the improper set of {1, 2, 3} is identically the set itself.

1.4 The Universal Set and Its Subsets

The universal set U is defined as the set containing all the elements, objects, subjects, or members under consideration. All other sets under current consideration are thus subsets of the universal set.

The null set or empty set, { }, denoted by Ø (to be distinguished from the Greek letter phi φ), is the set whose members are listed inside the braces { }. In other words, it has no members.

Thus, it is the flip side or reverse of any given full set or subset. Because the null set is an empty set, it is easy to visualize Ø as a legitimate subset of every set.

The empty set (Ø) is unique in set theory in that it is the set that has no elements and is therefore a subset of every set. The empty set must not be confused with zero number of elements despite the apparent similarity in context. Zero by itself as 0 is a decimal digit, not a set, but {0} is the set containing one element, namely, 0. On the other hand, Ø is the empty set { } = Ø, that is, the set with no elements.

Note that a set consisting of a single object is not equivalent to the object itself. In other words, {1} is not the same as 1, in the sense that an object within a box is not the same as the naked object itself.

The relationship between the universal set and its possible subsets is best illustrated by enumerating the subsets of a universal set consisting of three elements only; U = {a, b, c} (Table 1.1).

Table 1.1. Subsets of a three-element universal set

Note that besides the six subsets composed of one or two elements, two additional subsets need to be included, the null subset and the set itself, U in this case, for a total of eight possible subsets. This relationship can be generalized to apply to any set containing n elements where the number of all possible subsets is equal to 2n subsets of U.¹ In the case just cited, the number 2³ = 8 subsets, as noted earlier. Note also that the order in which the elements of a set are listed does not affect the basic property of the set because the elements only and not their order characterize the set. Similarly, each subset is unique as to its constituent elements irrespective of their order within the subset.

1.4.1 Combining and Comparing Sets

Two sets are defined to be equal or identical if they contain exactly the same number of identical elements irrespective of order, so that if A = {1, 2, 3} and B = {2, 3, 1}, then A = B. That is, a set A is equal to a set B if and only if every member of A is a member of B, and every member of B is a member of A.

In general, two sets are unequal provided at least one of them has a member that is not a member of the other.

The union of any two sets A and B, written A ∪ B, is the set of all elements that are members of A, B, or both.

Thus, if A = {1, 2, 3} and B = {4, 5}, then A ∪ B = {1, 2, 3, 4, 5}.

The intersection of any two sets A and B, written A ∩ B, is the set of all elements common to A and B, that is, elements shared by both A and B.

Thus, if A = {1, 2, 3, 4} and B = {4, 5}, then A ∩ B = {4}.

The relationship between a set and its complement may be graphically represented and verbally described as follows:

Set A and its complement 39723.png are subsets of the universal set U (Figure 1.1). Note that set A and its complement, 39729.png , subdivides the universal set into mutually exclusive, exhaustive, and disjoint sets—mutually exclusive because no element in A is shared by its complement 39733.png , exhaustive because the universal set is fully occupied by set A and its complement 39740.png , and disjoint because set A and its complement 39744.png do not intersect and thus have no elements in common.

Figure 1.1. Venn diagram of the exhaustive and exclusive partition of the U set by set A and its complement

1.5 Rules Governing Set Operations

1. The union of any set and its complement equals the universal set, and the intersection of any set and its complement equals the null set (Figure 1.7a, panels 2 and 4).

2. The union of any set with the null set is equal to itself, and the intersection of any set with the null set equals the null set (Figure 1.7a, panels 1 and 3).

3. The union of any set with the universal set equals the universal set, and the intersection of any set with the universal set equals itself (Figure 1.7a, panels 1 and 3).

4. The null set and the universal set are complements of each other; all that is not in the universal set is the empty set, and conversely, all that is not in the empty set is in the universal set.

The relationship between the two sets and their complements may be graphically represented and verbally described as follows:

1. Sets A and B are disjoint, A ∩ B = Ø, and mutually exclusive, that is, they have no elements in common; they exclude each other, and no two events occur simultaneously among them (Figure 1.2). The universal set is now partitioned into three mutually exclusive and exhaustive sets in which set A could be construed as the intersection of A with the complement of B, and set B could be construed as the intersection of B with the complement of A and the balance of the universal set as the intersection of the complements of A and B. In formal notation,

Figure 1.2. Venn diagram of the partition of the U set into three subsets

2. Sets A and B are disjoint, mutually exclusive, and exhaustive, that is, they have no elements in common, and their union comprises the universal set (Figure 1.3). That is, it is impossible that none of them occurred as a result of the experiment.

Figure 1.3. Venn diagram of the exhaustive and exclusive partition of the U set by sets A and B

3. Sets A and B are not disjoint, that is, they share a common element(s) at the intersection (Figure 1.4a). Note that A and B and the intersection divide the universal set into four mutually exclusive and exhaustive disjoint subsets, where the intersection area constitute the fourth subset (in contrast to the three subsets of Figure 1.2). The four subsets may thus be categorized as follows (Figures 1.5 and 1.7b):

Figure 1.4a. Venn diagram of the partition of the U set into four subsets through the intersection of two sets

Figure 1.5. Venn diagram of the disjointed subsets of the U set defined by the intersection of the sets and/or their complements

4. Set A is a subset of B, which is a subset of U, which is another way of saying that anything in A is also in B and anything in B is also in U (Figure 1.6). In this case, three disjoint, mutually exhaustive but not mutually exclusive subsets can be recognized.

Note that the intersection of A with the complement of B yields the empty set.

Figure 1.6. Venn diagram of the partition of the U set into three mutually exhaustive, but not mutually exclusive, subsets

1.6 Basic Set Operations

The Venn diagram constitutes the basic graphic element necessary for the intuitive grasp of set interrelations (Figure 1.7b). Traditionally, a rectangle represents the universal set, denoted by U, whereas enclosed circles represent subsets of the universal set. In addition, in order to make full use of the potential of the graphic approach, each given set and its complement will be identified throughout by a distinguishable fill-in pattern, thus allowing instant appraisal of the resultant pattern derived from the interaction of two or more sets.

Four basic operations that may be performed on sets are recognized.

1. Cartesian product. The Cartesian product set of two sets A and B, denoted by A × B, is defined as the set consisting of all possible ordered pairs formed when the first element is taken from the A set and the second element is taken from the B set (Table 1.3).

Thus, if A = {a, b} and B = {x, y, z}, then A × B = {(a, x), (a, y), (a, z), (b, x), (b, y), (b, z)}.

Note that B × A = {(x, a), (x, b), (y, a), (y, b), (z, a), (z, b)}.

Table 1.3. Cartesian product of sets A and B

The elements (ordered pairs) of B × A are not the same as those of A × B because the orders are reversed. In other words, (a, x) and (x, a) are distinct elements of their respective sets. Note that the ordered pairs making the elements of the product set A × B are enclosed in parentheses (not braces) to denote that the order of the individual enclosed elements is important in the differentiation between the resulting ordered pairs.

An ordered pair of elements is not a set because in a set, the order of the elements is completely immaterial: {a, b} = {b, a}. However, in an ordered pair, the additional property of order is essential. The ordered pair (a, b) is to be distinguished from the ordered pair (b, a) if a ≠ b.

For the sets B = {a, b} and C = {c, d, e}, all possible ordered pairs formed by choosing an element from the set B for the first element and an element from the set C for the second element: (a, c); (a, d); (a, e); (b, c); (b, d); (b, e). This set of ordered pairs is denoted by B × C.

The definition of a Cartesian product set does not specify that the sets used in forming the product set need to be the same or different.

For example, if A = {1, 2, 3} and B = {a, b}, then

(a) A × B = {(1,a), (1,b), (2,a), (2,b), (3,a), (3,b)}.

(b) B × A = {(a,1), (a,2), (a,3), (b,1), (b,2), (b,3)}.

(c) A × A = {(1,1), (1,2), (1,3), (2,1), (2,2), (2,3), (3,1), (3,2), (3,3)}.

(d) B × B = {(a,a), (a,b), (b,a), (b,b)}.

The ordered pair idea may be generalized to include any ordered arrangements, which may be formed from n objects (ordered n-tuples) constituting the elements of a set. A set may therefore be composed of ordered pairs, ordered triplets, ordered 4-tuples, . . . , ordered n-tuples, but not duplicate elements.

2. Union. The union of two sets, A + B, denoted by A ∪ B, is the set of elements contained within A or B or both, such that an element that belongs to A or B belongs to their union (panel 8).

3. Intersection. The intersection of sets A • B, denoted by A ∩ B, is the set of the elements that belong to both A and B. That is the set of elements shared or common to both A and B.

Note that the shaded area of intersection in panel 5 is actually crosshatched through the intersection the diagonal patterns of set A and set B so as to reflect the shared or common elements of both sets (Figure 1.7b, panel 5).

4. Complement. The complement of a set is the set of elements outside the set but within the universal set U, usually denoted by 39749.png or A′ (Figure 1.7a, panels 2 and 4).

As a concrete example, consider the experiment of rolling a die. The single digit numbers generated are elements of the universal set S = {1, 2, 3, 4, 5, 6}. Applying the rule defined earlier, the number of subsets that could be derived from the set of six numbers (2⁶) equals 64 different subsets ranging in size from no elements of the empty subset to one element per subset up to the total of six elements of the universal set. Within this constraint, we may have 6 one-element subsets, 15 two-element subsets, and so on.

Consider now the subset of even numbers and its complement of odd numbers so that

The union of the even subset with the subset made up two elements larger the four is graphically and symbolically denoted as follows:

Figure 1.4b

The union of the two subsets is the set that consists of all the elements of Seven, S>4, or both.

Note that the word or, when used in the context of sets interrelationship, carries the exclusive meaning of the equivalent word of union, which is normally symbolized by the set notation ∪ or its algebraic counterpart, the plus sign (+).

The intersection of the two subsets, on the other hand, is denoted as follows and graphically as previously mentioned:

The intersection of the two subsets is the set consisting of all the elements common to Seven and S>4. Note that the word and, when used in the context of sets interrelationship, carries the exclusive meaning of the equivalent word of intersection, which is normally symbolized by the set notation ∩ or its algebraic counterpart, the dot sign (•).

Comparing Figure 1.4a with Figure 1.4b, it can be readily appreciated that the patterns associated with the subsets of Figure 1.4a correspond to the elements associated with each subset.

1.7 Combined Set Operations

1.7.1 Intersection of a Set with the Complement of Another Set

In Figure 1.7b, panel 6, set A is intersecting the complement of set B. The resulting shaded area in this case is crosshatched through the intersection of the diagonal pattern, \, of set A with the vertical pattern, ||, of the complement of set B.

In Figure 1.7b, panel 7, set B is now intersecting the complement of set A. The resulting shaded area reflects the intersection of the diagonal pattern of set B, //, with the horizontal pattern, =, of the complement of set A.

1.7.2 Union of a Set with the Complement of another Set

In Figure 1.7b, panel 10, the vertical pattern, ||, of