Practical pairing of generative programming with functional programming.

The new wave of
JavaScript techniques
Practical pairing of generative programming
with functional programming
Eugene Lazutkin, ClubAjax, 6/4/2013
Wednesday, June 5, 13

Generative programming

What is GP?
Generative programming (GP):
Creates source code automatically to improve
programmer productivity.
Can be considered as a form of code reuse.
Sometimes called “automatic programming”.

Examples of GP
Macro processors:
C preprocessor.
C++ templates and inline functions.
IDE “wizards” in Eclipse, MS Visual Studio.
Meta-languages and their compilers.
Domain-speciﬁc languages (DSL).

Why do we need GP?
Higher-level programming.
Meta-programming.
Modeling.
DSL.

Why do we need GP?
Performance improvements.
Program transformations.
Data transformations.
Code optimization and generation.
Applies to both compilers and interpreters.

Why do I need GP?
Common narrative goes like that:
“Compilers and interpreters are now very
sophisticated – no need to optimize my code.”
“Modern CPUs with their fancy pipelining,
parallel execution, and branch prediction
eliminate code inefﬁciencies.”

Don’t write inefﬁcient code
Always estimate the algorithmic complexity of
your code (Big O notation).
Remember simple things:
Linear code usually works faster than code
with branches or loops.
Calling a function is not free. If its body is
small, it may make sense to inline it.

Don’t write inefﬁcient code
Remember simple things:
Simplify!
Simple shorter code is usually faster.
Eliminate unnecessary variables.
Proﬁle!!!
BTW, this was an advice by Jared Jurkiewicz.

Performance summary
Do not expect that an inefficient code will be
magically efficient. Always profile!
In a battle of a code clarity with a code
efficiency support the clarity, if possible.
Avoid premature optimization by taking complex
steps, yet do not write obviously slow code.

JavaScript and GP
People coming from static languages frequently
overlook or discount GP facilities in JS.
JS has four ways to generate code dynamically:
eval(), new Function(), setTimeout(),
setInterval().
Browser allows to inline code.

JavaScript and GP
Amazing code-generation facilities are a big
differentiator for Javascript.
Yet they are frequently shunned for numerous
reasons, including security, which is not applied
in all scenarios.

JavaScript and GP
Recently GP crept in JavaScript libraries.
Examples:
Many templating languages are actually
compiled in JavaScript or can be compiled:
Mustache, Handlebars, jQuery templates...

JavaScript and GP
Examples:
All JavaScript-based templates:
EJS, JST (e.g., Underscore, Lo-Dash)...
All transpilers:
CoffeeScript, GWT, Emscripten...
Numerous code-level tools (miniﬁers...).

JavaScript and GP
Examples:
Some libraries generate code for similar
functions from the same template in order to
reduce their downloadable size, and optimize
performance.
Lo-Dash...

Functional programming

What is FP?
Functional programming (FP):
Treats computations as the evaluation of
mathematical functions.
Avoids state, mutations, side-effects.
Emphasizes “scenario” over “a list of actors”.
Concerns more with “how” rather than
“who” and “what”.

JavaScript and FP
“JavaScript is Lisp in C’s clothing.” – said
Douglas Crockford back in 2001.
Functions are ﬁrst-class objects.
Lambdas are supported (functions can be
inlined, and passed around).
Closures are supported (functions can access
their environment).

Practical applications of FP
Abstracting control ﬂow details with array extras:
var total = data
.ﬁlter(function(value){ return value % 2; })
.map(function(value){ return value * value; })
.reduce(function(acc, value){
return acc + value;
}, 0);

Practical applications of FP
We can even use “building blocks” now:
var total = data
.ﬁlter(isOdd)
.map(square)
.reduce(sum, 0);

Why I like this code
It is easy to understand.
It is easy to decompose or refactor.
There is no trivial technical noise like names of
index and value variables, and exit conditions.

Building blocks
In general we are assumed to create our
programs from right-sized building blocks.
Bigger blocks are built from smaller blocks with a
mortar/language.
It applies to any programming paradigms.

...and reality
How come in real projects we see methods and
functions in 100s and even 1000s lines of code?
Sometimes there are “good” reasons for that.
It can take more time to ﬁgure out our
building blocks than to code everything from
scratch.

...and more reality
Sometimes there are “good” reasons for that.
Calling myriad of small functions can be
detrimental to performance.
Is it really the case? It can be veriﬁed with a
proﬁler, which takes time too.
It is possible to create a different system of
“building blocks”.

Case for pipes
Pipes are individual components of a pipeline.
Each pipe has an input and an output
(sometimes more than one of those).
Pipeline combines individual pipes by
connecting their inputs and outputs producing a
program.
A pipeline can be used as a pipe.

Case for pipes
Unix shells are a classic examples of pipes.
Closely related concept: dataﬂow.
Complex applications frequently can be
expressed in terms of event ﬂow.
Hot topics in JS: event streams, data streams.

Pipes and chaining
While chaining is a frequent implementation
technique for pipes, it doesn’t mean that all
chaining equates to pipes.
Pipes encapsulate streams of data and
orchestrate processing of individual bits.
Frequently chaining operates in terms of
transferred objects (streams), not data bits.

Chaining
Examples in JS of chaining without piping:
jQuery
Underscore
JavaScript array extras
All examples above create a container object
before passing it down.

Theory and practice

Can we do better?
Let’s use our simple example as a start point and
optimize it by inlining functions:
var total = data
return acc + value;
}, 0);

Can we do better?
var a1 = [];
for(var i = 0; i < data.length; ++i){
if(data[i] % 2) a1.push(data[i]);
}
var a2 = new Array(a1.length);
for(var i = 0; i < a1.length; ++i){
a2[i] = a1[i] * a1[i];
}
var acc = 0;
for(var i = 0; i < a2.length; ++i){
acc += a2[i];
}
var total = acc;

Can we do better?
Let’s merge loops and eliminate intermediate
arrays:
var acc = 0;
for(var i = 0; i < data.length; ++i){
var value = data[i];
if(value % 2) acc += value * value;
}
var total = acc;

Performance results
0 750 1500 2250 3000
ms
Manual (38ms) Extra (2885ms)

We did better!
And the results are in:
It is faster.
We loop over values manually.
We cannot easily move this snippet around
because our technical variable names may
clash.
Ultimately it was a trade-off.

Can we have our cake and eat it too?
Yes!

Can we have our cake and eat it too?
We need a way to describe a result.
Not how exactly get it, but its logical inference.
Then we can apply GP to generate an optimal
code.

FP and pipes
FP gives us some answers:
Iterations can be described by higher-order
operations:
map, ﬁlter, fold/reduce, zipping, unzipping,
partitioning.
Secondary: forEach, every, some, indexOf.

Heya Pipe

Heya: ctr and pipe
Heya is a library that provides a modern
foundation for building web applications (both
server and client).
Heya Pipe is a functional toolkit to build efﬁcient
data and event processing pipes using GP.
Heya Ctr provides a constructor to build
generated components.

The Heya way
Let’s use our simple example and optimize it with
Heya Pipe:
var total = data
return acc + value;
}, 0);

The Heya way
Here it is:
var f = array(new Pipe()
.ﬁlter(“value % 2”)
.map(“value *= value;”)
.fold(“accumulator += value;”, 0)
).compile();
var total = f(data);

Performance results
0 750 1500 2250 3000
ms
Manual (38ms) Pipe (53ms) Extra (2885ms)

What do we see?
We are as expressive as array extras.
We are as performant as manual code.
The performance gap can be reduced.
In any case pipes are >40x faster than array
extras.

Heya Pipe conventions
Heya Pipe is modeled after array extras.
It can accept the same parameters.
If a string is speciﬁed instead of a function it is
assumed to be a code snippet.
Code snippets work by conventions.

Heya Pipe conventions
Snippets can have several lines of code.
“Boolean” snippets assume that the last line
returns a boolean value.
Snippet’s input is a “value” variable. A snippet
can modify it to return a different value.
Snippets for fold-able operations can access and
update an “accumulator” variable.

Debugging
People make mistakes ⇒ code should be
debugged.
How to debug a generated code?
What is Heya Pipe story here?

Debugging
The screenshot shows node.js debugging with
node-inspector.
The generated code appears in the ﬁle panel.
The code is fully readable.

Heya Pipe
Provides a rich foundation for pipes:
forEach, transform, map, ﬁlter, indexOf, every,
some, fold, scan, skip, take, skipWhile,
takeWhile, voidResult.
Can work with potentially inﬁnite streams.

Heya Pipe
Can iterate over numerous containers:
Array (sparse and dense, including in reverse),
array slices (including in reverse), Object (keys,
values, own and inherited), iterators,
enumerators, generators, and perform unfold.
Includes a compiler, and an interpreter.
Can use pipes as sub-pipes.

Heya Ctr
Implements GP helpers.
Includes a simple formatting-aware
templating.
Allows managing closures.
Serves as a foundation for Heya Pipes.

Closures with Heya Ctr
Allows to inject variables in a closure, which will
be directly accessible from snippets by name.
Does not use “with” statement because it is
incompatible with strict mode.
Fully strict mode compatible.
Allows to access and modify closure-bound
variables.

!at’s all
Folks!

Practical pairing of generative programming with functional programming.

Recommended

More Related Content

What's hot (20)

Viewers also liked (16)

Similar to Practical pairing of generative programming with functional programming. (20)

More from Eugene Lazutkin (16)

Recently uploaded (20)

Practical pairing of generative programming with functional programming.