Introduction to Parallel Processing Algorithms and Architectures 1st Edition by Behrooz Parhami ISBN 9780306469640 0306469642 pdf download
Introduction to Parallel Processing Algorithms and Architectures 1st Edition by Behrooz Parhami ISBN 9780306469640 0306469642 pdf download
https://ebookball.com/product/introduction-to-parallel-
processing-algorithms-and-architectures-1st-edition-by-behrooz-
parhami-isbn-9780306469640-0306469642-19872/
https://ebookball.com/product/introduction-to-game-development-using-
processing-1st-edition-by-james-parker-1942270658-9781942270652-25160/
Adapting Parallel Algorithms to the W Stream Model with
Applications to Graph Problems 1st Edition by Camil
Demetrescu, Bruno Escoffier, Gabriel Moruz, Andrea
Ribichini ISBN 9783540744566
https://ebookball.com/product/adapting-parallel-algorithms-to-the-w-
stream-model-with-applications-to-graph-problems-1st-edition-by-camil-
demetrescu-bruno-escoffier-gabriel-moruz-andrea-ribichini-
isbn-9783540744566-12246/
https://ebookball.com/product/algorithms-for-image-processing-and-
computer-vision-2nd-edition-by-parker-0470643854-978-0470643853-17240/
FUNDAMENTALS OF X PROGRAMMING
Graphical User Interfaces and Beyond
Theo Pavlidis
INTRODUCTION TO PARALLEL PROCESSING
Algorithms and Architectures
Behrooz Parhami
Introduction to
Parallel Processing
Algorithms and Architectures
Behrooz Parhami
University of California at Santa Barbara
Santa Barbara, California
No part of this eBook may be reproduced or transmitted in any form or by any means, electronic,
mechanical, recording, or otherwise, without written consent from the Publisher
The field of digital computer architecture has grown explosively in the past two decades.
Through a steady stream of experimental research, tool-building efforts, and theoretical
studies, the design of an instruction-set architecture, once considered an art, has been
transformed into one of the most quantitative branches of computer technology. At the same
time, better understanding of various forms of concurrency, from standard pipelining to
massive parallelism, and invention of architectural structures to support a reasonably efficient
and user-friendly programming model for such systems, has allowed hardware performance
to continue its exponential growth. This trend is expected to continue in the near future.
This explosive growth, linked with the expectation that performance will continue its
exponential rise with each new generation of hardware and that (in stark contrast to software)
computer hardware will function correctly as soon as it comes off the assembly line, has its
down side. It has led to unprecedented hardware complexity and almost intolerable devel-
opment costs. The challenge facing current and future computer designers is to institute
simplicity where we now have complexity; to use fundamental theories being developed in
this area to gain performance and ease-of-use benefits from simpler circuits; to understand
the interplay between technological capabilities and limitations, on the one hand, and design
decisions based on user and application requirements on the other.
In computer designers’ quest for user-friendliness, compactness, simplicity, high per-
formance, low cost, and low power, parallel processing plays a key role. High-performance
uniprocessors are becoming increasingly complex, expensive, and power-hungry. A basic
trade-off thus exists between the use of one or a small number of such complex processors,
at one extreme, and a moderate to very large number of simpler processors, at the other.
When combined with a high-bandwidth, but logically simple, interprocessor communication
facility, the latter approach leads to significant simplification of the design process. However,
two major roadblocks have thus far prevented the widespread adoption of such moderately
to massively parallel architectures: the interprocessor communication bottleneck and the
difficulty, and thus high cost, of algorithm/software development.
vii
viii INTRODUCTION TO PARALLEL PROCESSING
The above context is changing because of several factors. First, at very high clock rates,
the link between the processor and memory becomes very critical. CPUs can no longer be
designed and verified in isolation. Rather, an integrated processor/memory design optimiza-
tion is required, which makes the development even more complex and costly. VLSI
technology now allows us to put more transistors on a chip than required by even the most
advanced superscalar processor. The bulk of these transistors are now being used to provide
additional on-chip memory. However, they can just as easily be used to build multiple
processors on a single chip. Emergence of multiple-processor microchips, along with
currently available methods for glueless combination of several chips into a larger system
and maturing standards for parallel machine models, holds the promise for making parallel
processing more practical.
This is the reason parallel processing occupies such a prominent place in computer
architecture education and research. New parallel architectures appear with amazing regu-
larity in technical publications, while older architectures are studied and analyzed in novel
and insightful ways. The wealth of published theoretical and practical results on parallel
architectures and algorithms is truly awe-inspiring. The emergence of standard programming
and communication models has removed some of the concerns with compatibility and
software design issues in parallel processing, thus resulting in new designs and products with
mass-market appeal. Given the computation-intensive nature of many application areas (such
as encryption, physical modeling, and multimedia), parallel processing will continue to
thrive for years to come.
Perhaps, as parallel processing matures further, it will start to become invisible. Packing
many processors in a computer might constitute as much a part of a future computer
architect’s toolbox as pipelining, cache memories, and multiple instruction issue do today.
In this scenario, even though the multiplicity of processors will not affect the end user or
even the professional programmer (other than of course boosting the system performance),
the number might be mentioned in sales literature to lure customers in the same way that
clock frequency and cache size are now used. The challenge will then shift from making
parallel processing work to incorporating a larger number of processors, more economically
and in a truly seamless fashion.
The field of parallel processing has matured to the point that scores of texts and reference
books have been published. Some of these books that cover parallel processing in general
(as opposed to some special aspects of the field or advanced/unconventional parallel systems)
are listed at the end of this preface. Each of these books has its unique strengths and has
contributed to the formation and fruition of the field. The current text, Introduction to Parallel
Processing: Algorithms and Architectures, is an outgrowth of lecture notes that the author
has developed and refined over many years, beginning in the mid-1980s. Here are the most
important features of this text in comparison to the listed books:
the notation and terminology from the reference source. Such an approach has the
advantage of making the transition between reading the text and the original
reference source easier, but it is utterly confusing to the majority of the students
who rely on the text and do not consult the original references except, perhaps, to
write a research paper.
SUMMARY OF TOPICS
The six parts of this book, each composed of four chapters, have been written with the
following goals:
Part I sets the stage, gives a taste of what is to come, and provides the needed
perspective, taxonomy, and analysis tools for the rest of the book.
Part II delimits the models of parallel processing from above (the abstract PRAM
model) and from below (the concrete circuit model), preparing the reader for everything
else that falls in the middle.
Part III presents the scalable, and conceptually simple, mesh model of parallel process-
ing, which has become quite important in recent years, and also covers some of its
derivatives.
Part IV covers low-diameter parallel architectures and their algorithms, including the
hypercube, hypercube derivatives, and a host of other interesting interconnection
topologies.
Part V includes broad (architecture-independent) topics that are relevant to a wide range
of systems and form the stepping stones to effective and reliable parallel processing.
Part VI deals with implementation aspects and properties of various classes of parallel
processors, presenting many case studies and projecting a view of the past and future
of the field.
For classroom use, the topics in each chapter of this text can be covered in a lecture
spanning 1–2 hours. In my own teaching, I have used the chapters primarily for 1-1/2-hour
lectures, twice a week, in a 10-week quarter, omitting or combining some chapters to fit the
material into 18–20 lectures. But the modular structure of the text lends itself to other lecture
formats, self-study, or review of the field by practitioners. In the latter two cases, the readers
can view each chapter as a study unit (for 1 week, say) rather than as a lecture. Ideally, all
topics in each chapter should be covered before moving to the next chapter. However, if fewer
lecture hours are available, then some of the subsections located at the end of chapters can
be omitted or introduced only in terms of motivations and key results.
Problems of varying complexities, from straightforward numerical examples or exercises
to more demanding studies or miniprojects, have been supplied for each chapter. These problems
form an integral part of the book and have not been added as afterthoughts to make the book
more attractive for use as a text. A total of 358 problems are included (13–16 per chapter).
Assuming that two lectures are given per week, either weekly or biweekly homework can
be assigned, with each assignment having the specific coverage of the respective half-part
PREFACE xi
(two chapters) or full part (four chapters) as its “title.” In this format, the half-parts, shown
above, provide a focus for the weekly lecture and/or homework schedule.
An instructor’s manual, with problem solutions and enlarged versions of the diagrams
and tables, suitable for reproduction as transparencies, is planned. The author’s detailed
syllabus for the course ECE 254B at UCSB is available at http://www.ece.ucsb.edu/courses/
syllabi/ece254b.html.
References to important or state-of-the-art research contributions and designs are
provided at the end of each chapter. These references provide good starting points for doing
in-depth studies or for preparing term papers/projects.
xii INTRODUCTION TO PARALLEL PROCESSING
New ideas in the field of parallel processing appear in papers presented at several annual
conferences, known as FMPC, ICPP, IPPS, SPAA, SPDP (now merged with IPPS), and in
archival journals such as IEEE Transactions on Computers [TCom], IEEE Transactions on
Parallel and Distributed Systems [TPDS], Journal of Parallel and Distributed Computing
[JPDC], Parallel Computing [ParC], and Parallel Processing Letters [PPL]. Tutorial and
survey papers of wide scope appear in IEEE Concurrency [Conc] and, occasionally, in IEEE
Computer [Comp]. The articles in IEEE Computer provide excellent starting points for
research projects and term papers.
ACKNOWLEDGMENTS
GENERAL REFERENCES
[Akl89] Akl, S. G., The Design and Analysis of Parallel Algorithms, Prentice–Hall, 1989.
[Akl97] Akl, S. G., Parallel Computation: Models and Methods, Prentice–Hall, 1997.
[Alma94] Almasi, G. S., and A. Gottlieb, Highly Parallel Computing, Benjamin/Cummings, 2nd ed., 1994.
[Bert89] Bertsekas, D. P., and J. N. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods,
Prentice–Hall, 1989.
[Code93] Codenotti, B., and M. Leoncini, Introduction to Parallel Processing, Addison–Wesley, 1993.
[Comp] IEEE Computer, journal published by IEEE Computer Society: has occasional special issues on
parallel/distributed processing (February 1982, June 1985, August 1986, June 1987, March 1988,
August 1991, February 1992, November 1994, November 1995, December 1996).
[Conc] IEEE Concurrency, formerly IEEE Parallel and Distributed Technology, magazine published by
IEEE Computer Society.
[Cric88] Crichlow, J. M., Introduction to Distributed and Parallel Computing, Prentice–Hall, 1988.
[DeCe89] DeCegama, A. L., Parallel Processing Architectures and VLSI Hardware, Prentice–Hall, 1989.
[Desr87] Desrochers, G. R., Principles of Parallel and Multiprocessing, McGraw-Hill, 1987.
[Duat97] Duato, J., S. Yalamanchili, and L. Ni, Interconnection Networks: An Engineering Approach, IEEE
Computer Society Press, 1997.
[Flyn95] Flynn, M. J., Computer Architecture: Pipelined and Parallel Processor Design, Jones and Bartlett,
1995.
[FMPC] Proc. Symp. Frontiers of Massively Parallel Computation, sponsored by IEEE Computer Society and
NASA. Held every 1 1/2–2 years since 1986. The 6th FMPC was held in Annapolis, MD, October
27–31, 1996, and the 7th is planned for February 20–25, 1999.
[Foun94] Fountain, T. J., Parallel Computing: Principles and Practice, Cambridge University Press, 1994.
[Hock81] Hockney, R. W., and C. R. Jesshope, Parallel Computers, Adam Hilger, 1981.
[Hord90] Hord, R. M., Parallel Supercomputing in SIMD Architectures, CRC Press, 1990.
[Hord93] Hord, R. M., Parallel Supercomputing in MIMD Architectures, CRC Press, 1993.
[Hwan84] Hwang, K., and F. A. Briggs, Computer Architecture and Parallel Processing, McGraw-Hill, 1984.
[Hwan93] Hwang, K., Advanced Computer Architecture: Parallelism, Scalability, Programmability, McGraw-
Hill, 1993.
PREFACE xiii
[Hwan98] Hwang, K., and Z. Xu, Scalable Parallel Computing: Technology, Architecture, Programming,
McGraw-Hill, 1998.
[ICPP] Proc. Int. Conference Parallel Processing, sponsored by The Ohio State University (and in recent
years, also by the International Association for Computers and Communications). Held annually since
1972.
[IPPS] Proc. Int. Parallel Processing Symp., sponsored by IEEE Computer Society. Held annually since
1987. The 11th IPPS was held in Geneva, Switzerland, April 1–5, 1997. Beginning with the 1998
symposium in Orlando, FL, March 30–April 3, IPPS was merged with SPDP. **
[JaJa92] JaJa, J., An Introduction to Parallel Algorithms, Addison-Wesley, 1992.
[JPDC] Journal of Parallel and Distributed Computing, Published by Academic Press.
[Kris89] Krishnamurthy, E. V., Parallel Processing: Principles and Practice, Addison–Wesley, 1989.
[Kuma94] Kumar, V., A. Grama, A. Gupta, and G. Karypis, Introduction to Parallel Computing: Design and
Analysis of Algorithms, Benjamin/Cummings, 1994.
[Laks90] Lakshmivarahan, S., and S. K. Dhall, Analysis and Design of Parallel Algorithms: Arithmetic and
Matrix Problems, McGraw-Hill, 1990.
[Leig92] Leighton, F. T., Introduction to Parallel Algorithms and Architectures: Arrays, Trees, Hypercubes,
Morgan Kaufmann, 1992.
[Lerm94] Lerman, G., and L. Rudolph, Parallel Evolution of Parallel Processors, Plenum, 1994.
[Lipo87] Lipovski, G. J., and M. Malek, Parallel Computing: Theory and Comparisons, Wiley, 1987.
[Mold93] Moldovan, D. I., Parallel Processing: From Applications to Systems, Morgan Kaufmann, 1993.
[ParC] Parallel Computing, journal published by North-Holland.
[PPL] Parallel Processing Letters, journal published by World Scientific.
[Quin87] Quinn, M. J., Designing Efficient Algorithms for Parallel Computers, McGraw-Hill, 1987.
[Quin94] Quinn, M. J., Parallel Computing: Theory and Practice, McGraw-Hill, 1994.
[Reif93] Reif, J. H. (ed.), Synthesis of Parallel Algorithms, Morgan Kaufmann, 1993.
[Sanz89] Sanz, J. L. C. (ed.), Opportunities and Constraints of Parallel Computing (IBM/NSF Workshop, San
Jose, CA, December 1988), Springer-Verlag, 1989.
[Shar87] Sharp, J. A., An Introduction to Distributed and Parallel Processing, Blackwell Scientific Publica-
tions, 1987.
[Sieg85] Siegel, H. J., Interconnection Networks for Large-Scale Parallel Processing, Lexington Books, 1985.
[SPAA] Proc. Symp. Parallel Algorithms and Architectures, sponsored by the Association for Computing
Machinery (ACM). Held annually since 1989. The 10th SPAA was held in Puerto Vallarta, Mexico,
June 28–July 2, 1998.
[SPDP] Proc. Int. Symp. Parallel and Distributed Systems, sponsored by IEEE Computer Society. Held
annually since 1989, except for 1997. The 8th SPDP was held in New Orleans, LA, October 23–26,
1996. Beginning with the 1998 symposium in Orlando, FL, March 30–April 3, SPDP was merged
with IPPS.
[Ston93] Stone, H. S., High-Performance Computer Architecture, Addison–Wesley, 1993.
[TCom] IEEE Trans. Computers, journal published by IEEE Computer Society; has occasional special issues
on parallel and distributed processing (April 1987, December 1988, August 1989, December 1991,
April 1997, April 1998).
[TPDS] IEEE Trans. Parallel and Distributed Systems, journal published by IEEE Computer Society.
[Varm94] Varma, A., and C. S. Raghavendra, Interconnection Networks for Multiprocessors and Multicomput-
ers: Theory and Practice, IEEE Computer Society Press, 1994.
[Zoma96] Zomaya, A. Y. (ed.), Parallel and Distributed Computing Handbook, McGraw-Hill, 1996.
*The 27th ICPP was held in Minneapolis, MN, August 10–15, 1998, and the 28th is scheduled for September
21–24, 1999, in Aizu, Japan.
**The next joint IPPS/SPDP is sceduled for April 12–16, 1999, in San Juan, Puerto Rico.
This page intentionally left blank.
Random documents with unrelated
content Scribd suggests to you:
☐ Ring the alarm and try to put out the fire.
☐ Run in the other direction.
☐ Watch it burn and calculate your insurance.
5. What should one do if he accidently steps on someone else’s
toes?
☐ Call for help.
☐ Run for the doctor.
☐ Ask the person’s pardon.
☐ Take his own part.
6. Why do the leaves fall off the trees in the autumn?
☐ The frost has killed them.
☐ To protect the flowers from freezing.
☐ To enrich the ground.
☐ So that one can see farther.
7. Why do people wear heavier clothing in January than in June?
☐ To protect them from the colder weather.
☐ Because it looks better with furs.
☐ Everybody else does it, especially in January.
☐ It makes a good impression on other people.
8. Where might one expect to find the largest number of expert
swimmers?
☐ At the circus.
☐ At the beach of a summer resort.
☐ At a Sunday School picnic.
☐ At a moving picture show.
9. What should a person do when he is late getting started to work
in the morning?
☐ Wait until the next day.
☐ Think up some excuse to make.
☐ Try to make time by hurrying.
☐ Blame it on the street cars.
10. Why do school houses usually have flag-poles?
☐ For the boys to exercise on.
☐ To show where to have a flag drill.
☐ To display the flag and inspire patriotism.
☐ To decorate the school yard.
11. Why does water freeze in winter?
☐ It is warm in summer and we need ice.
☐ So the children can skate.
☐ Water always becomes solid at low temperatures.
☐ So it can be put in refrigerators.
12. What is the best way to stop up a hole by which mice enter the
kitchen?
☐ Stuff it with paper.
☐ Place a pile of rags in front of it.
☐ Put a corn cob in it.
☐ Nail a piece of tin over it.
13. Why is milk a good thing to feed young children?
☐ It comes from cows that eat fresh grass.
☐ It is an easily digested and wholesome food.
☐ It is so pure and white to look at.
☐ It can be bought in pint or quart bottles.
14. What kind of light is best for a reading table?
☐ A tallow candle.
☐ A mercury vapour lamp.
☐ A coal oil lantern.
☐ An incandescent electric bulb.
15. What is the purpose of advertising food products?
☐ To make people hungry.
☐ To decorate street cars and magazines.
☐ To make people think about eating.
☐ To create a demand for special brands.
16. Why is harness put on horses?
☐ So that their strength may be utilized.
☐ So that their beauty will be recognized.
☐ To match the colour of the carriage.
☐ To keep them warm.
17. Why do people put food in refrigerators?
☐ To get it out of the way.
☐ The low temperature keeps it fresh.
☐ To help ice men make a living.
☐ Because the law requires it.
18. Why are fire escapes more frequently put on tall buildings than
on one-story buildings?
☐ A tall building is more likely to burn down.
☐ They would spoil the looks of a low building.
☐ Low buildings have fire extinguishers.
☐ One could jump from one-story buildings.
19. What should one do with a baby when it cries?
☐ Discover and remove the cause of its crying.
☐ Spank it and put it to bed.
☐ Get it a drink of water and rock its cradle.
☐ Give it a bottle of milk or sing to it.
20. Why is country air considered more healthful than city air?
☐ It has fewer impurities in it.
☐ Trees and grass grow in the country.
☐ More people die in the city.
☐ The wind is stronger in the country.
21. Why do railroads use electric engines in some cities?
☐ They run faster than steam engines.
☐ They look better than steam engines.
☐ To avoid making the city smoky.
☐ In order to make less noise.
22. What is the main purpose of lightning rods?
☐ To decorate the roof of the house.
☐ To make the lightning strike somewhere else.
☐ To show which way the wind blows.
☐ To remove the electricity from the air.
23. What is the safest altitude and speed for flying in an airplane?
☐ Low and slowly.
☐ Low and rapidly.
☐ High and rapidly.
☐ High and slowly.
24. What is the chief purpose of newspaper headlines?
☐ To make the paper attractive.
☐ To show what actually happened.
☐ To help one decide where to read.
☐ To guide public opinion wisely.
Directions for Scoring the Test.
The score in this test is the number of questions correctly
answered. A stencil is furnished with each package of tests, which
makes it possible to check up at a very rapid rate the accuracy of the
votes cast, without ever reading a single word of the answers. The
stencil is merely to be adjusted to the page, according to directions
given on its face, and where the check mark made by the candidate
corresponds to the printed mark on the stencil the question has been
correctly answered, while if there is not agreement between stencil
and candidate’s check no credit is to be allowed on the question. Any
fairly careful clerical worker can learn in two minutes to score such a
test with a stencil as rapidly and accurately as a thoroughly trained
psychologist could do it.
The final score should be entered in the lower right-hand corner of
the front or title page, where it will be easily associated with the
name and other information about the candidate.
Scores from 0 to 3 indicate Inferior Ability
Scores from 4 to 8 indicate Low Average Ability
Scores from 9 to 14 indicate Average Ability
Scores from 15 to 19 indicate High Average Ability
Scores from 20 to 24 indicate Superior Ability
Mentimeter No. 30
LOGICAL CONCLUSIONS TEST
“The only one of the four endings which can be true, if the first
part of the sentence is true, is the third, ‘violets do not cost as much
as roses,’ so the square in front of this third conclusion should be
check-marked.
“You will have five minutes in which to read and mark the twelve
sentences. Think carefully and get all of your marks correctly placed.
Ready! Go!”
At the end of five minutes exactly, call “Stop! Time up! Give me
your papers.” All papers should be collected at once.
MARK (✓) THE SQUARE IN FRONT OF THE TRUEST
ENDING TO EACH SENTENCE
The score in this test is the number of sentences for which the
correct conclusion is checked. The stencil furnished with the test
booklets makes this process of counting the number correct so
simple that a child can do it almost at a glance. The degree of
intellectual capacity is indicated roughly by the score as follows:
Attention should be invited here again to the fact that this is a new
test and that its reliability and implications will need to be carefully
tested and measured before one can be sure what it measures or how
accurate it is.
CHAPTER XI
TRADE TESTS OR TESTS OF SKILL
Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.
ebookball.com