[FREE PDF sample] Extending and Embedding Perl Tim Jenness ebooks
[FREE PDF sample] Extending and Embedding Perl Tim Jenness ebooks
com
https://ebookname.com/product/extending-and-embedding-perl-
tim-jenness/
OR CLICK BUTTON
DOWNLOAD EBOOK
https://ebookname.com/product/web-graphics-perl-tk-best-of-the-perl-
journal-1st-edition-jon-orwant-ph-d/
ebookname.com
https://ebookname.com/product/learning-perl-5th-edition-randal-
schwartz/
ebookname.com
https://ebookname.com/product/embedded-android-porting-extending-and-
customizing-1st-edition-karim-yaghmour/
ebookname.com
https://ebookname.com/product/cultural-tourism-and-identity-
rethinking-indigeneity-1st-edition-keyan-g-tomaselli/
ebookname.com
Last House on the Hill BACH Area Reports from Çatalhöyük
Turkey Cotsen Monumenta Archaeologica BACH Area Reports
from Catalhoyuk Turkey 27 Mirjana Stevanovic (Editor)
https://ebookname.com/product/last-house-on-the-hill-bach-area-
reports-from-catalhoyuk-turkey-cotsen-monumenta-archaeologica-bach-
area-reports-from-catalhoyuk-turkey-27-mirjana-stevanovic-editor/
ebookname.com
https://ebookname.com/product/macrosociology-four-modern-
theorists-1st-edition-frank-w-elwell/
ebookname.com
https://ebookname.com/product/soft-tissue-composition-mechanisms-of-
injury-and-repair-1st-edition-antonio-j-chavez-ruiz/
ebookname.com
https://ebookname.com/product/roosevelts-1st-edition-robert-grayson/
ebookname.com
https://ebookname.com/product/tristan-and-isolde-a-casebook-1st-
edition-joan-tasker-grimbert-editor/
ebookname.com
Whose Story Translating the Verbal and the Visual in
Literature for Young Readers 1st Edition Riitta Oittinen
https://ebookname.com/product/whose-story-translating-the-verbal-and-
the-visual-in-literature-for-young-readers-1st-edition-riitta-
oittinen/
ebookname.com
Extending and Embedding Perl
Extending and
Embedding Perl
TIM JENNESS
SIMON COZENS
MANNING
Greenwich
(74° w. long.)
For online information and ordering of this and other Manning books,
go to www.manning.com. The publisher offers discounts on this book
when ordered in quantity. For more information, please contact:
Special Sales Department
Manning Publications Co.
209 Bruce Park Avenue Fax: (203) 661-9018
Greenwich, CT 06830 email: [email protected]
Many of the designations used by manufacturers and sellers to distinguish their products are
claimed as trademarks. Where those designations appear in the book, and Manning
Publications was aware of a trademark claim, the designations have been printed in initial
caps or all caps.
Recognizing the importance of preserving what has been written, it is Manning’s policy to have the
books we publish printed on acid-free paper, and we exert our best efforts to that end.
ISBN 1930110820
Printed in the United States of America
1 2 3 4 5 6 7 8 9 10 – VHG – 06 05 04 03 02
To the Perl open source community
contents
preface xiii
acknowledgments xv
about this book xvi
author online xix
about the cover illustration xx
vii
2.2 Interfacing to another language: C from XS 30
The Perl module 30 ✦ The XS file 31 ✦ Example: “Hello, world” 32
Return values 36 ✦ Arguments and return values 37
2.3 XS and C: taking things further 38
Modifying input variables 38 ✦ Output arguments 39
Compiler constants 40
2.4 What about Makefile.PL? 44
It really is a Perl program 47
2.5 Interface design: part 1 47
Status and multiple return arguments 48 ✦ Don’t supply what is
already known 48 ✦ Don’t export everything 49
Use namespaces 49 ✦ Use double precision 49
2.6 Further reading 50
2.7 Summary 50
3 Advanced C 51
3.1 Arrays 51
3.2 Pointers 53
Pointers and arrays 55 ✦ Pointers to functions 57
3.3 Strings 58
Arrays of strings 59
3.4 Structures 60
3.5 File I/O 62
3.6 Memory management 63
Allocating memory at runtime 64 ✦ Altering the size of memory 65
Manipulating memory 65 ✦ Memory manipulation and Perl 67
3.7 C Traps for the Perl programmer 68
3.8 Further reading 69
3.9 Summary 69
viii CONTENTS
4.4 Array variables 85
4.5 Hashes 87
4.6 Globs 91
4.7 Namespaces and stashes 94
4.8 Lexical “my” variables 95
4.9 Code blocks 96
Important CV flags 97
4.10 Further reading 99
4.11 Summary 99
CONTENTS ix
6.7 Callbacks 202
Immediate callbacks 203 ✦ Deferred callbacks 206 ✦ Multiple callbacks 207
6.8 Other languages 209
Linking Perl to C++ 209 ✦ Linking Perl to Fortran 216
Linking Perl to Java 223
6.9 Interface design: part 2 223
6.10 Older Perls 224
6.11 What’s really going on? 225
What does xsubpp generate? 226
6.12 Further reading 230
6.13 Summary 230
7 Alternatives to XS 231
7.1 The h2xs program 232
7.2 SWIG 233
Data structures 236
7.3 The Inline module 238
What is going on? 239 ✦ Additional Inline examples 240
Inline and CPAN 245 ✦ Inline module summary 246
7.4 The PDL::PP module 247
The .pd file 248 ✦ The Makefile.PL file 249 ✦ Pure PDL 251
7.5 Earlier alternatives 251
7.6 Further reading 252
7.7 Summary 253
x CONTENTS
9 Embedding case study 268
9.1 Goals 268
9.2 Preparing the ground 269
9.3 Configuration options 270
9.4 Testing options 273
Binary options 273 ✦ Quad-state options 274 ✦ String options 275
9.5 Summary 276
CONTENTS xi
11.4 Perl 6: the future of Perl 321
A history 321 ✦ Design and implementation 322
What happens next 323 ✦ The future for Perl 5 323
11.5 Further reading 323
11.6 Summary 323
xii CONTENTS
preface
Perl is a wonderful language. We, along with at least a million other programmers, love it dearly.
It’s great for all kinds of applications: text processing, network programming, system administra-
tion, and much more. But there are times when we need to go beyond the core of the language
and do something not provided by Perl.
Sometimes we do this without noticing it: several modules that ship with Perl call out to C
routines to get their work done, as do some of the most commonly used CPAN modules. Other
times, we do it deliberately: the various modules for building graphical applications in Perl almost
all, directly or indirectly, use external C libraries. Either way, writing extensions to Perl has his-
torically been a bit of a black art. We don’t believe this situation is fair, so we’ve written this book
to attempt to demystify the process of relating Perl and C.
That’s not to say we’re fed up with writing modules in Perl. Both of us write many of our mod-
ules in Perl, and although sometimes it might be easier to interface to C, we’ve decided to stick
with Perl. In fact, writing a module in Perl has a number of advantages over using other languages:
• It is far easier to write portable cross-platform code in Perl than in C. One of the successes
of Perl has been its support of varied operating systems. It is unlikely that a module written
in C could be as portable as a Perl version without much more effort on the part of the pro-
grammer, precisely because Perl strives to hide the complexities of differences between
underlying operating systems from the programmer.
• Some problems do not need the speed gain that comes from using a compiled language or
the added complexity of interfacing to another language. For many programmers who are
proficient in Perl (and/or Java), writing a Perl module is much more efficient (in terms of
programming effort) than writing the equivalent in C.
People program in Perl for a reason, and this fact should not be forgotten when it comes to
deciding whether to use Perl for an extension module. These issues were addressed in the devel-
opment of the standard File::Temp module (part of Perl 5.6.1). This module provides a stan-
dard interface for creating temporary files from Perl. The original intention was that this module
would be written in C, but it quickly became apparent that a Perl implementation would be eas-
ier because of portability problems (it was to be a standard module, so it would have to work on
all platforms supported by Perl); in addition, speed would not be an issue unless thousands of
temporary files were required.
xiii
Having addressed why not to use a language other than Perl, we must now present two impor-
tant reasons why another language is sometimes required:
• Speed—In some cases, Perl is simply too slow for a particular program. In this case, the
choice is either to change the algorithm or to use a faster programming language. The Perl
Data Language was created specifically to address the case of processing N-dimensional
arrays, but there are times when another language is required. Similarly, Perl is definitely
too slow for some tasks (for instance, the graphics manipulation functions provided by
Imager and GD).
• Functionality—Many useful libraries have been written in other languages (especially C,
and for numerical applications, Fortran). If new functionality is required that is present in
an external library, then it is usually far better to provide a Perl interface to the library than
to recode the library in Perl. For instance, the XML::Parser module provides a reason-
ably direct mapping onto the underlying functions of the expat library. Functionality is
particularly an issue for things that simply can’t be written in Perl, such as interfaces to cer-
tain system libraries (for instance, the Macintosh Carbon library) or to particular pieces of
hardware.
As well as extending Perl by writing modules in C, sometimes it’s advantageous to go the other
way around: to add the flexibility of a Perl interpreter to an existing C program. Like extending,
this process has a fearsome reputation, and so we provide two chapters on the topic. Chapter 8
introduces embedding Perl, and chapter 9 includes a full working example that explains Perl
embedding.
We also realize that people want to know what’s really going on under the hood, so we con-
clude our study of the interaction between C and Perl by examining the C sources of the perl
interpreter itself, together with details of how to get involved in becoming a developer maintain-
ing perl. Finally, we look ahead to Perl 6 and the Parrot project.
xiv PREFACE
acknowledgments
The authors would like to thank the staff at Manning for their hard work, encouragement, and,
where necessary, coercion. Susan Capparelle first approached us and got us to commit to the book;
Marjan Bace oversaw the project. Ted Kennedy set up reviews and helped get some of the best
people around to give us feedback on the book. Mary Piergies headed up production; Syd Brown
and Dottie Marsico were the poor sods who had to deal with our awkward choice of markup and
file formats. Tiffany Taylor was our eagle-eyed copy editor and Liz Welch our proofreader.
Our reviewers were wonderfully pedantic, catching every mistake from the unintentional slip
to the outright lie. Martien Verbruggen headed up this exercise, and many thanks are also due
to Alasdair Allan, Chuck Bailey, Clinton Pierce, Abhijit Menon-Sen, John Tobey, John Linder-
man, Norman Gray, Bennett Todd, Dan Sugalski, David Zempel, Doug MacEachern, Graham
Barr, James Freeman, Spider Boardman, and Alex Gough.
The perl5-porters have been wonderfully helpful, answering obscure questions about the hid-
den corners of Perl. Particular thanks to Jarkko Hietaniemi, Gurusamy Sarathy, Nick Ing-Sim-
mons, Hugo van der Sanden, Abhijit Menon-Sen, and Arthur Bergman. We hope that our
explorations have been as beneficial to you as they have been intriguing to us.
TIM JENNESS I would also like to thank Frossie for leading me onto the path to becoming a
Perl hacker and for forcing me to write this book. Thanks to my brother, Matt, because he
wanted to be thanked, and to the rabbits, Buns and Neo, for not eating all the draft versions of
this book. Thanks to the Tuesday night movie gang for putting up with my terrible selections—
the book can no longer be used as an excuse! Finally, thanks again to Frossie for expanding my
horizons beyond measure and for being the center of my universe.
SIMON COZENS I would also like to thank the denizens of #perl and That Other IRC Chan-
nel, without whom this book would have been finished much earlier but would have been con-
siderably more boring. Thanks to Ray, pod, and Aaron at Oxford University Computing
Services, who have to put up with me on a daily basis and even seem to try not to mind; and to
Oxford Go Club for providing welcome diversion and keeping me more or less sane. And thanks
to Eiko, who has possibly been told “nearly finished now!” more times than the entirety of the
Manning staff.
xv
about this book
Guide to this book
This book is roughly divided into four sections. The first section covers the groundwork of deal-
ing with C and the basics of XS, the extension language used to communicate between Perl and C:
• Chapter 1 provides an introduction to the C language from a Perl perspective. We explain
the similarities and differences between the languages, and show how to write, compile,
and run programs written in C.
• Chapter 2 presents the basics of the extension language, XS. By the end of this chapter, you
will be able to create Perl modules that call simple C functions.
• Chapter 3 discusses more advanced features of the C language, such as strings, arrays, struc-
tures, and memory handling.
The next section explains XS in more detail:
• Chapter 4 describes how Perl values are stored internally: that is, how scalars, arrays, and
hashes work. This concept is fundamental to doing any advanced work with XS, because
you will be manipulating these types of values.
• Chapter 5 introduces the Perl API—the range of functions you have at your disposal as an
XS programmer for manipulating values and interacting with the perl interpreter. As well
as being a reference guide to the API, it contains many real-world examples of how the API
functions are used in XS situations, including modules such as Tk and DBI, the
mod_perl extension to the Apache web server, and the perl interpreter itself. In other
words, it’s intended to be a hands-on tutorial to using the API functions. (Note that appen-
dix C provides an index of all Perl API references in the book.)
• Chapter 6 describes many more advanced uses of XS, such as how to create Perl arrays,
hashes, and objects based on C structures; how to interface with Fortran; how to deal with
files; how to call back from C to Perl; and much more. We believe this chapter represents,
in a distilled format, the highest levels of XS magic, much of which has never been clearly
written down before.
• Chapter 7 describes alternatives to XS that also help with C-Perl interaction: SWIG,
Inline, and PDL::PP, among others.
xvi
The third section deals with embedding Perl in other projects:
• Chapter 8 describes embedding Perl in generic terms, including why and when you should
consider embedding Perl in a C program, the fundamentals required to embed Perl, and
how to call back from embedded Perl into C.
• Chapter 9 turns the description from chapter 8 into a working example, to help you under-
stand the thought processes and programming involved in embedding Perl in a real-world
application.
The fourth section deals with the internals of the perl interpreter:
• Chapter 10 provides an introduction to perl internals, including the path a Perl program
takes from input to compilation to execution.
• Chapter 11 is a grab-bag of useful information on developing perl: how to debug the
interpreter, how to contribute code back to the maintainers, and how the Perl development
process works. We close by looking into the future at the Perl 6 development effort.
The final section consists of three appendices:
• Appendix A describes all of Perl’s typemap entries used in the book.
• Appendix B lists further reading material.
• Appendix C provides an index of all Perl API references in the book.
Intended audience
We’ve worked hard to make this book the definitive tutorial and reference to all topics involved
in the interaction of Perl and C. This means we’ve had to make some broad assumptions. Natu-
rally, we assume you, the reader, are a competent Perl programmer.
We don’t assume proficiency in C. Although we include an introduction to C at the beginning
of this book, and it should be possible to gain a lot of benefit from this book without any prior
exposure to C, this isn’t intended to be a substitute for a good C tutorial—the idea is to whet your
appetite regarding what can be done while extending Perl with C, and give you what you need
to know to understand the majority of the examples provided. The book is also intended to inspire
those who know the ground rules of C programming but find it hard to translate that knowledge
into practical programs.
If you’re an experienced C programmer, you’ll naturally gain the most from this book, because
you are likely to have practical ideas about how to apply the information in it. To avoid attacks
of boredom, however, we recommend that the experienced C programmer skip chapters 1 and 3.
Source code downloads
The source code for all examples presented in this book is freely available from the publisher’s
web site, http://www.manning.com/jenness. Should errors be discovered after publication, all
code updates will be made availabe via the Web.
The % indicates the shell or command prompt (on some systems this may be written as >), the
emboldened text indicates what you would type, and the rest of the text is the output you
would see.
Code annotations accompany many segments of code. Certain annotations are marked with
chronologically ordered bullets such as B . These annotations have further explanations that fol-
low the code.
xix
about the cover illustration
The figure on the cover of Extending and Embedding Perl is a “Gauro o Larsi,” a man from one of
the many tribes that inhabited the mountainous and ethnically diverse region of the Indus River
between Kashmir and Kabul. This illustration is taken from a Spanish compendium of regional
dress customs first published in Madrid in 1799. The book’s title page informs us:
Coleccion general de los Trages que usan actualmente todas las Nacionas del Mundo desubierto, dibu-
jados y grabados con la mayor exactitud por R.M.V.A.R. Obra muy util y en special para los que
tienen la del viajero universal
which we loosely translate as:
General Collection of Costumes currently used in the Nations of the Known World, designed and
printed with great exactitude by R.M.V.A.R. This work is very useful especially for those who hold
themselves to be universal travelers.
Although nothing is known of the designers, engravers, and artists who colored this illustration
by hand, the “exactitude” of their execution is evident in this drawing. The Gauro o Larsi is just
one of a colorful variety of figures in this collection which reminds us vividly of how distant and
isolated from each other the world’s towns and regions were just 200 years ago. Dress codes have
changed since then and the diversity by region, so rich at the time, has faded away. It is now often
hard to tell the inhabitant of one continent from another. Perhaps we have traded a cultural and
visual diversity for a more varied personal life—certainly a more varied and interesting world of
technology.
At a time when it can be hard to tell one computer book from another, Manning celebrates
the inventiveness and initiative of the computer business with book covers based on the rich diver-
sity of regional life of two centuries ago—brought back to life by the pictures from this collection.
xx
C H A P T E R 1
When using C and Perl together, the first thing you need to realize is that they are
very different languages, requiring different styles and different thought patterns. Perl
spoils programmers by doing much of the hard work for you; if you’ve never pro-
grammed in C, the language can feel very barren and empty. C is close to the
machine—Perl is close to the user.
That said, Perl’s syntax borrows heavily from C’s, and so most of the elements of
a C program should be familiar to a competent Perl programmer with a little thought
and a little preparation.
1
The classic book on Perl, on the other hand, is Programming Perl, by Larry Wall, Tom
Christiansen, and Randal Schwartz. It begins with a similar program:
print "Howdy, world!\n";
Notice that Perl is much more compact: there’s no waffle, no housekeeping. We want
to print something, and we tell Perl to print it. C, on the other hand, requires more
support from the programmer.
Let’s first look at how to compile and run the C program, before we discuss how
it’s constructed.
If we have a collection of C files that make up one program, we can list them all:
% gcc -Wall -o bigproject one.c two.c three.c four.c
1
It has this name for historical reasons; a.out was the name of a particular type of executable file
format.
2 Of course, if you are not using GCC, the particular option will be different. Ironically, -Wall does
not turn on all the warnings in GCC; it only turns on those the GCC programmers feel are important.
See the GCC documentation for the full list of warnings. If you want to make sure your program will
be ANSI compliant and you’re not accidentally slipping in some GCC extensions, consider using the
-ansi -pedantic option.
There are more complicated ways to build large programs, using static and dynamic
libraries, but we won’t go into those in our examples.
3
It’s almost like use strict is always on—you can’t use any function without telling C where it’s
coming from (or at least you shouldn’t, because if you fail to declare the prototype, the compiler will
make some assumptions that are unlikely to be correct).
HEADER FILES 3
1.4 THE MAIN FUNCTION
The Perl version of the “Hello world” program is much smaller because everything in
C must be inside a function; if you have Perl code outside any subroutine, it will all
be executed in order. C needs all this code in one place: the main function. When
your program begins, C arranges for this function to be called.
NOTE What you’d call a subroutine in Perl is called a function in C.
This function must have a prototype, and here it is again:4
int main(int argc, char* argv[]);
Perl’s prototypes tell you only what type of data is coming into the function: sub
main($@) tells you that the subroutine main takes a scalar and an array. The proto-
types can also coerce values into a given type; for instance, sub take_ref(\%) will
make the first element into a reference to a hash.
In C, you’re not only told what’s coming into the function, you’re also told what
variables it should be stored in, and also what type of data the function should return.
C’s prototypes do not coerce values to a type—instead, if you pass the wrong type, the
C compiler will give an error.
In this case, we’re returning an integer, and we’re being given an integer called
argc and something called argv. We’ll look at what char * means later, but you
might be able to guess that argv is similar to Perl’s @ARGV and [] might denote an
array—argv is an array of the command-line parameters. argc is the number of ele-
ments in the array—that is, the number of command-line parameters passed to the
program (the argument count). One difference between Perl and C is that in C, argv
contains the name of the command in element 0, whereas in Perl, @ARGV does not
contain the command name (that can be retrieved using $0).
NOTE main is a special C function in the sense that it can have multiple proto-
types and you can decide which one to use. If you are not interested in
using command-line arguments, you can use a much simpler prototype
that has no arguments:
int main(void);
Some compilers also support a third argument to main:
int main(int argc, char *argv[], char **envp);
Here, envp provides access to the process environment. It is not part of
POSIX or the ANSI C89 standard, so we won’t mention it again.
4
ISO standard C main is defined as being int main(int argc, char** argv), but many pro-
grammers use the (roughly) equivalent char* argv[]. The difference is horribly subtle. We use
char* argv[] here because it’s easier to understand.
TIP This example includes the use of a C comment. Comments are made up of
matched pairs of /* and */.5 Note that comments cannot be nested. This
is legal:
/*
Comment out some code:
printf("This won't print\n");
5
Many modern compilers (and the C99 standard) also implement C++-style // comments. These com-
ments act just like Perl # comments.
Declaration of Sentiments.
“The history of mankind is a history of repeated injuries and
usurpations on the part of man toward woman, having in direct
object the establishment of an absolute tyranny over her. To prove
this, let facts be submitted to a candid world.
“He has never permitted her to exercise her inalienable right
to the elective franchise.
“He has compelled her to submit to laws in the formation of
which she had no voice.
“He has withheld from her rights which are given to the most
ignorant and degraded men—both natives and foreigners.
“Having deprived her of this first right of a citizen, the elective
franchise, thereby leaving her without representation in the halls of
legislation, he has oppressed her on all sides.
“He has made her, if married, in the eye of the law, civilly
dead.
“He has taken from her all right in property, even to the wages
she earns.
“He has so framed the laws of divorce as to what shall be the
proper causes, and, in case of separation, to whom the guardianship
of the children shall be given, as to be wholly regardless of the
happiness of women—the law in all cases going upon a false
supposition of the supremacy of man, and giving all power into his
hands.
“After depriving her of all rights as a married woman, if single
and the owner of property, he has taxed her to support a government
which recognizes her only when her property can be made profitable
to it.
“He has monopolized nearly all the profitable employments,
and from those she is permitted to follow she receives but a scanty
remuneration. He closes against her all the avenues of wealth and
distinction which he considers most honorable to himself. As a
teacher of theology, medicine, or law, she is not known.
“He allows her in church, as well as state, but a subordinate
position, claiming Apostolic authority for her exclusion from the
ministry and, with some exceptions, from any public participation in
the affairs of the church.
“He has created a false public sentiment by giving to the world
a different code of morals for men and women, by which moral
delinquencies which exclude women from society are not only
tolerated but deemed of little account in man.
“He has usurped the prerogative of Jehovah himself, claiming
it as his right to assign for her a sphere of action, when that belongs
to her conscience and God.
“He has endeavored, in every way that he could, to destroy her
confidence in her own powers, to lessen her self-respect, and to make
her willing to lead a dependent and abject life.
“Now, in view of this disfranchisement of one-half the people
of this country, their social and religious degradation; in view of the
unjust laws mentioned, and because women do feel themselves
aggrieved, oppressed and fraudulently deprived of their most sacred
rights, we insist that they have immediate admission to all the rights
and privileges which belong to them as citizens of the United States.”
SUSAN B. ANTHONY.