The Functional Programming Toolkit (NDC Oslo 2019)

The
Functional Programming
Toolkit
(NDC Oslo 2019)
@ScottWlaschin
fsharpforfunandprofit.com

Why do functional programmers
use so many strange words?

Functional programming is scary

Object oriented programming is scary

These aren't just academic concepts.
They are actually useful tools!

Functional programmers have a
standard set of tools

The Functional Toolkit
• Composition
• Combination/Aggregation
• Iteration
• Working with effects
– Mixing effects and non-effects
– Chaining effects in series
– Working with effects in parallel
– Pulling effects out of a list

The Functional Toolkit
• Composition: compose
• Iteration: fold
• Combination/Aggregation: combine & reduce
– Mixing effects and non-effects: map & return
– Chaining effects in series: bind/flatMap
– Working with effects in parallel: apply, lift, zip
– Pulling effects out of a list: sequence, traverse

FunctionalToolkit (FP jargon version)
• Combination/Aggregation: Monoid
– Mixing effects and non-effects: Functor
– Chaining effects in series: Monad
– Working with effects in parallel: Applicative

FunctionalToolkit (FP jargon version)
• Combination/Aggregation: Monoid
– Mixing effects and non-effects: Functor
– Chaining effects in series: Monad
– Working with effects in parallel: Applicative
This talk

This talk
A whirlwind tour of many sights
Don't worry if you don't understand everything

What I'll talk about
• The core principles of FP
• Function transformers
• Some tools in the functional toolkit
– map
– bind
– lift
• An example of using all the tools together

Core principles of
statically-typed FP
Part I

Core principles of (statically-typed) FP
Function
Functions are things
Composition everywhere

Core FP principle:
Functions are things
Function

The Tunnel of
Transformation
Function
apple -> banana
A function is a thing which
transforms inputs to outputs

A function is a standalone thing,
not attached to a class
It can be used for inputs and outputs
of other functions

input
A function can be an output
A function can be an output

output
A function can be an input
A function can be an input

input output
A function can be a parameter
A function can be a parameter
You can build very complex systems
from this simple foundation!
Most of the tools in the functional toolkit are "function
transformers" that convert functions to functions

Core FP principle:
Composition everywhere

Lego Philosophy
1. All pieces are designed to be connected
2. Connect two pieces together and get
another "piece" that can still be connected
3. The pieces are reusable in many contexts

All pieces are designed to be connected

Connect two pieces together and
get another "piece" that can still be connected

The pieces are reusable in different contexts

The Lego philosophy let's you make big things!

The Functional Programming Toolkit (NDC Oslo 2019)

Can we apply these ideas to
programming?

Functional Programming Philosophy
• Design functions that do one thing well
– Functions can be reused in different contexts
• Design functions to work together
– Expect the output of every function to become
the input to another, as yet unknown,function
• Use types to ensure that inputs match outputs

Function 1
apple -> banana
Function 2
banana -> cherry

>>
Function 1
apple -> banana
Function 2
banana -> cherry

New Function
apple -> cherry
Can't tell it was built from
smaller functions!
Where did the banana go?

Building big things from functions
It's compositions all the way up

Low-level operation
ToUpper
stringstring

Low-level operation
Service
AddressValidator
A “Service” is just like a microservice
but without the "micro" in front
Validation
Result
Address
Low-level operation Low-level operation

Service
Use-case
UpdateProfileData
ChangeProfile
Result
ChangeProfile
Request
Service Service

Use-case
Web application
Http
Response
Http
Request
Use-case Use-case

Http
Response
Http
Request
A web application built from
functions only (no classes!)

Http
Response
Http
Request
I have a whole talk on "The Power of Composition" at
fsharpforfunandprofit.com/composition
A web application built from
functions only (no classes!)

let add1 x = x + 1
let double x = x + x
Introducing F#

let add1 x = x + 1
let double x = x + x
let square = fun x -> x * x
Introducing F#

add1 5 // = 6
double (add1 5) // = 12
square (double (add1 5)) // = 144
How to compose functions together

5 |> add1 // = 6
5 |> add1 |> double // = 12
5 |> add1 |> double |> square // = 144
Piping in F#

Composition works well when
the types match up



Composition doesn't work well
when the types don't match up

Fix: Insert a converter function
into the pipeline
Converter
function

5 |> add1 |> strLen
// ^ error: expecting an int
5 |> add1 |> intToStr |> strLen
// ^ fixed!
F# example


What if the types match up but
they're wrapped in something?
Option< >


What if the types match up but
they're wrapped in something?
List< >

FunctionTransformers
Part II
What do railways
have to do with
programming?

Receive request
Validate request
Lowercase the email
Update user record in DB
Return result to user
type Request = {
name: string;
email: string }
"As a user I want to update my name and email address"

string UpdateCustomer()
{
var request = receiveRequest();
validateRequest(request);
lowercaseEmail(request);
db.updateDbFromRequest(request);
return "OK";
}

string UpdateCustomerWithErrorHandling()
{
var isValidated = validateRequest(request);
if (!isValidated) {
return "Request is not valid"
}
db.updateDbFromRequest(request);
smtpServer.sendEmail(request.Email)
return "OK";
}

{
if (!isValidated) {
}
var result = db.updateDbFromRequest(request);
if (!result) {
return "Customer record not found"
}
return "OK";
}

{
if (!isValidated) {
}
try {
var result = db.updateDbFromRequest(request);
if (!result) {
return "Customer record not found"
}
} catch {
return "DB error: Customer record not updated"
}
return "OK";
}

Use a Result type
for error handling

Request SuccessValidate
Failure
type Result =
| Ok of SuccessValue
| Error of ErrorValue
A choice type, aka sum type

Request SuccessValidate
Failure
let validateInput input =
if input.name = "" then
Error "Name must not be blank"
else if input.email = "" then
Error "Email must not be blank"
else
Ok input // happy path

Functional flow without error handling
let updateCustomer =
receiveRequest()
|> validateRequest
|> lowercaseEmail
|> updateDbFromRequest
|> returnMessage
Before
One track

let updateCustomerWithErrorHandling =
receiveRequest()
|> validateRequest
|> lowercaseEmail
|> updateDbFromRequest
|> returnMessage
Functional flow with error handling
After
Two track

How to implement
Railway Oriented Programming

Validate UpdateDbon success
bypass

How to compose
these functions?

Step 1 Step 2 Step 3
>> >>
Composing one-track functions is fine...

>> >>
... and composing two-track functions is fine...

 
... but composing switches is not allowed!

How to combine the
mismatched functions?

“Bind” is the answer!
Bind all the things!

Two-track input Two-track output
One-track input Two-track output



Two-track input
Slot for switch function
Two-track output

let bind nextFunction twoTrackInput =
match twoTrackInput with
| Ok success -> nextFunction success
| Error err -> Error err

let bind nextFunction twoTrackInput =
| Ok success -> nextFunction success
| Error err -> Error err
©

Composing switches - review

Validating input
type Input = {
Name : string
Email : string
}

nameLessThan50
let nameNotBlank input =
if input.Name = "" then
Error "Name must not be blank"
else Ok input
Let nameLessThan50 input =
if input.Name.Length > 50 then
Error "Name must not be longer than 50 chars"
else Ok input
Let emailNotBlank input =
if input.Email = "" then
Error "Email must not be blank"
else Ok input
nameNotBlank
emailNotBlank

nameNotBlank (composed with)
nameLessThan50 (composed with)
emailNotBlank
nameNotBlank nameLessThan50 emailNotBlank

nameLessThan50 emailNotBlanknameNotBlank
nameNotBlank
bind nameLessThan50
bind emailNotBlank

request
|> nameNotBlank
|> Result.bind nameLessThan50
|> Result.bind emailNotBlank
name50 emailNotBlanknameNotBlank

validateInput
let validateInput input =
input
|> nameNotBlank
|> Result.bind nameLessThan50
|> Result.bind emailNotBlank

FunctionB
Result<'TEntity,string>
FunctionA

Correct shape AND types match
Correct shape, but types don't match

How do single track functions
fit this model?

// trim spaces and lowercase
let lowercaseEmail input =
{input with
email = input.email.Trim().ToLower() }
lowercaseEmail

Won't compose!
UpdateDb EtcValidate
LowercaseEmail

Two-track input
Slot for one-track function
Two-track output

lowercaseEmaillowercaseEmail

let map singleTrackFunction twoTrackInput =
| Ok s -> Ok (singleTrackFunction s)
| Error e -> Error e

Converting one-track functions
UpdateDb EtcValidate
LowercaseEmail
Will compose

map
Converting everything to two-track

bind
map
Converting everything to two-track
These are "function
transformers"!

Validate LowercaseEmail DbUpdate
Using function transformers so that *can*
compose different shaped functions
map bind

Understanding "effects"
Part III

What is an effect?
• A collection type
List<_>
• A type enhanced with extra data
Option<_>, Result<_>
• A type that interacts with the outside world
Async<_>, Task<_>, Random<_>
• A type that carries state
State<_>, Parser<_>

What is an effect?
• A collection type
List<_>
• A type enhanced with extra data
Option<_>, Result<_>
• A type that interacts with the outside world
Async<_>, Task<_>, Random<_>
• A type that carries state
State<_>, Parser<_>
We'll focus on
three for this talk

"Normal" world vs.
"Effects" world

"Normal" world
int
string
bool
int -> string
int -> bool

"Option" world
Option<int>
Option<string>
Option<bool>
Option<int> -> Option<string>
Option<int> -> Option<bool>

"List" world
List<int>
List<string>
List<bool>
List<int> -> List<string>
List<int> -> List<bool>

"Async" world
Async<int>
Async<string>
Async<bool>
Async<int> -> Async<string>
Async<int> -> Async<bool>

Generic "Effects" world
E<int>
E<string>
E<bool>
E<int> -> E<string>
E<int> -> E<bool>

Different names, same concept
• "Effect" world
• "Enhanced" world
• "Elevated" world
– Because we use the word "lift" a lot!

How to work with effects?
Challenge:

Example scenario
• Download a URL into a JSON object
• Decode the JSON into a Customer DTO
• Convert the DTO into a valid Customer
• Store the Customer in a database

NormalWorld
Result World
AsyncWorld
Url
AsyncResult<Json,Err>
Download the json file

World of normal values
Result World
R<CustomerDto,Err>
Json
Decode the JSON into a DTO

Result World
R<name> R<email>
CustomerDto
Validate the fields of the customer

validName validEmail
validCustomer
Construct a customer from the fields

NormalWorld
Result World
AsyncWorld
Customer
AsyncResult<unit,Err>
Store the customer in the DB

How do we compose these
functions together?
None the worlds match up ...
... but we can use the functional toolkit!

int string bool
World of options
Option<int> Option<string> Option<bool>

World of options
int string bool


World of options

int string bool

add1 1 // 2
add1 (Some 1) // error

let add1ToOption opt =
if opt.IsSome then
let newVal = add1 opt.Value
Some newVal
else
None 

World of options
add1 

World of options
add1

Moving functions between
worlds with "map"
Tool #1

let add1ToOption opt =
if opt.IsSome then
Some (add1 opt.Value)
else
None
Let's take this code and turn it
into a generic, reusable tool

let optionMap f opt =
if opt.IsSome then
Some (f opt.Value)
else
None

let optionMap f =
fun opt ->
if opt.IsSome then
Some (f opt.Value)
else
None

let optionMap f
fun opt ->
if opt.IsSome then
Some (f opt.Value)
else
None

World of options
Option<T> -> -> Option
optionMap
T -> -> U

let add1 x = ...
(optionMap add1) (Some 1)


Example:
Working with List world

let add1ToEach aList =
let newList = new List()
for item in aList do
let newItem = add1 item
newList.Add(newItem)
// return
newList


World of lists
add1 

let listMap f aList =
let newItem = f item
// return
newList
Let's make a generic, reusable
tool again

let listMap f aList =
// return
newList

let listMap f =
fun aList ->
// return
newList

World of lists
listMap
List<T> -> -> List
T -> -> U

let add1 x = ...
(listMap add1) [1;2;3]
Q: Why is this any better than writing
your own loops every time?
A: Because it's a pattern you
can learn to recognize.

World of async
async<T> -> -> async
asyncMap
T -> -> U
We do the same for other worlds too

Guideline:
Most wrapped generic types
have a “map”. Use it!

Guideline:
If you create your own generic type,
create a “map” for it.

FP terminology
A functor is
i. An effect type
– e.g. Option<>, List<>, Async<>
ii. Plus a "map" function that "lifts" a function to
the effects world
– a.k.a. select, lift
iii. And it must have a sensible implementation
– the Functor laws

Moving values between worlds
with "return"
Tool #2

World of options
Option<int>
Option.return
int

let x = 42
let intOption = Some x

World of lists
List<int>
List.return
int

Chaining world-crossing
functions with "bind"
Tool #3

What's a
world-crossing function?

let range max = [1..max]
// int -> List<int>

ListWorld
Normal world
A world crossing function
List<int>
int
range

let getCustomer id =
if customerFound then
Some customerData
else
None
// CustomerId -> Option<CustomerData>

OptionWorld
Normal world
A world crossing function
Option<CustomerData>
CustomerId
getCustomer

Problem:
How do you chain
world-crossing functions?

let optionExample input =
let x = doSomething input
if x.IsSome then
let y = doSomethingElse (x.Value)
if y.IsSome then
let z = doAThirdThing (y.Value)
if z.IsSome then
let result = z.Value
Some result
else
None
else
None
else
None

let taskExample input =
let taskX = startTask input
taskX.WhenFinished (fun x ->
let taskY = startAnotherTask x
taskY.WhenFinished (fun y ->
let taskZ = startThirdTask y
taskZ.WhenFinished (fun z ->
z // final result
)
)
)

if x.IsSome then
if y.IsSome then
if z.IsSome then
// do something with z.Value
// in this block
else
None
else
None
else
None
Let's fix this!
There is a pattern we can exploit...

if x.IsSome then
if y.IsSome then
if z.IsSome then
// do something with z.Value
// in this block
else
None
else
None
else
None

if x.IsSome then
if y.IsSome then
// do something with y.Value
// in this block
else
None
else
None

if x.IsSome then
// do something with x.Value
// in this block
else
None
Can you see the pattern?

if opt.IsSome then
//do something with opt.Value
else
None

let ifSomeDo f opt =
if opt.IsSome then
f opt.Value
else
None

let example input =
doSomething input
|> ifSomeDo doSomethingElse
|> ifSomeDo doAThirdThing
|> ifSomeDo ...
let ifSomeDo f opt =
if opt.IsSome then
f opt.Value
else
None

Some
None
Input ->
Let's revisit the railway analogy

“Bind” is the answer (again!)

Option input Option output
One-track input Option output



Option input
Slot for switch function
Option output

let bind nextFunction optionInput =
match optionInput with
| Some s -> nextFunction s
| None -> None
Option input Option output

Pattern:
Use "bind" to chain options

if x.IsSome then
if y.IsSome then
if z.IsSome then
let result = z.Value
Some result
else
None
else
None
else
None
Before

let optionBind f opt =
match opt with
| Some v -> f v
| None -> None
After

Pattern:
Use "bind" to chain tasks
a.k.a "promise" "future"

let taskX = startTask input
taskX.WhenFinished (fun x ->
let taskY = startAnotherTask x
taskY.WhenFinished (fun y ->
let taskZ = startThirdTask y
taskZ.WhenFinished (fun z ->
z // final result
)
)
)
Before

let taskBind f task =
task.WhenFinished (fun taskResult ->
f taskResult)
startTask input
|> taskBind startAnotherTask
|> taskBind startThirdTask
|> taskBind ...
After

Why is bind so important?
It makes world-crossing functions
composable

EffectsWorld
Normal World
a
E
Before bind:
A diagonal function
(world crossing)
bind

bind
EffectsWorld
Normal World
a
E
EffectsWorld
Normal World
E<a> E
After bind:
A horizontal function
(all in E-world)

bind
EffectsWorld
Normal World
a
E
EffectsWorld
Normal World
E<a> E

NormalWorld
Effects World
"Diagonal" functions can't be composed

NormalWorld
Effects World
Bind

NormalWorld
Effects World
"Horizontal" functions can be composed

FP terminology
A monad is
i. An effect type
ii. Plus a return function
– a.k.a. pure unit
iii. Plus a bind function that converts a "diagonal"
(world-crossing) function into a "horizontal" (E-
world-only) function
– a.k.a. >>= flatMap SelectMany
iv. And bind/return must have sensible implementations
– the Monad laws

TLDR: If you want to chain effects-
generating functions in series,
use a Monad

Combining effects in parallel
with applicatives
Tool #4

How to combine effects?
Option<T> Option+
Option<T,U>

Some 42 Some "hello"+
Some (42,"hello")
Combining options
This is what you expect!

Some 42 None+
None
Combining options

How to combine Lists?
List<T> List+
List<T,U>

[1,2,3] ["a","b","c"]+
[ (1,"a"), (1,"b"), (1,"c")
(2,"a"), (2,"b"), (2,"c")
(3,"a"), (3,"b"), (3,"c") ]
Combining lists (cross product)

[1,2,3] ["a","b","c"]+
[ (1,"a")
(2,"b")
(3,"c") ]
Combining lists (zip)

The general term for this is
"applicative functor"
Option, List, Async are all applicatives

FP terminology
A applicative (functor) is
i. An effect type
ii. Plus a return function
– a.k.a. pure unit
iii. Plus a function that combines two effects into one
– a.k.a. <*> apply pair liftA2
iv. And apply/return must have sensible implementations
– the Applicative Functor laws
So why is this useful?

Problem:
How to validate multiple fields
in parallel?

type Customer = {
Name : String50
Email : EmailAddress
Birthdate : Date
}
validateName validateEmail validateBirthdate
So we create some validation functions:
Each field must be validated

validateName validateEmail validateBirthdate
Problem: Validation done in series.
So only one error at a time is returned

type CustomerDto = {
name : string
email : string
birthdate : string
} validateName
validateEmail
validateBirthdate
Combine
output
Now we do get all
errors at once!
... But how to
combine them?

Result World
R<name> R<email> R<bdate>

validName validEmail validDate
validCustomer
We know how to combine the normal values
(use a constructor)

Result World
R<Customer>
The output is also in
Result world
Use the magic
of Applicatives!

Introducing "liftA2", "liftA3", etc

ResultWorld
NormalWorld
Option<T>, Option -> -> Option<V>
liftA2
T,U -> -> V
"liftA2" is just like map but works
on functions with two parameters

ResultWorld
NormalWorld
Opt<T>,Opt,Opt<V> -> -> Option<W>
liftA3
T,U,V -> -> W
"liftA3" is just like map but works
on functions with three parameters

let dtoToCustomer (dto:CustomerDto) =
// get the validated values
let nameOrError = validateName dto.name
let emailOrError = validateEmail dto.email
let birthdateOrError =
validateBirthdate dto.birthdate
// call the constructor
(liftA3 makeCustomer)
nameOrError
emailOrError
birthdateOrError
// final output is Result<Customer,ErrMsg list>
Here's where the
magic happens!
What the code looks like

The FunctionalToolbox
• "map"
– Lifts functions into an effects world
• "return"
– Lifts values into an effects world
• "bind"
– Converts "diagonal" functions into "horizontal" ones so
they can be composed.
• "apply"
– Combines two effects in parallel
– "liftA2", "liftA3" for example

Using all the tools together
Part IV

Revisiting the example scenario
• Download a URL into a JSON object
• Decode the JSON into a Customer DTO
• Convert the DTO into a valid Customer
• Store the Customer in a database

NormalWorld
Result World
AsyncWorld
Download the json file
Url
AsyncResult<Json>

Result World
R<CustomerDto>
Json
Decode the json

Result World
CustomerDto
Validate fields

Construct the customer
validName validEmail validDate
validCustomer

NormalWorld
Result World
AsyncWorld
Customer
AsyncResult<unit>

We now have the tools to compose
these functions together!

ResultWorld
CustomerDto
Validate fields AND create a customer
Use Result type for validation
R<Customer>
Use "lift3"

Result World
CustomerDto
R<Customer>
Validate fields AND create a customer
We now have a world crossing function from
the DTO to the Customer

Result World
CustomerDto
R<Customer>
Parse json AND create a customer
R<CustomerDto>
Json
Use "bind" to turn the diagonal
functions into horizontal ones
Bind Bind

Result World
R<Customer>
R<CustomerDto>R<Json>
Bind Bind

Result World
R<Customer>R<Json>
Then compose them into one function

let jsonToCustomer jsonOrError =
jsonOrError
|> Result.bind jsonToCustomerDto
|> Result.bind dtoToCustomer
It takes much more time to explain
than to write it!

NormalWorld
Result World
AsyncWorld
R<Customer>R<Json>
Then lift it up to Async
world using map
Map

NormalWorld
Result World
AsyncWorld
AsyncResult<Customer>AsyncResult<Json>

NormalWorld
AsyncWorld
Customer
Use "bind" to turn the
diagonal function horizontal
AsyncResult<unit>
Bind

NormalWorld
Result World
AsyncWorld
AsyncResult<Customer> AsyncResult<unit>
Bind

NormalWorld
AsyncWorld
All steps are now composable
Url
AsyncResult<Json>
AsyncResult<Customer>AsyncResult<Json>
AsyncResult<Customer> AsyncResult<unit>
Convert JSON to customer
Store customer in DB

NormalWorld
AsyncWorld
All steps are now composable
Url
AsyncResult<Json> AsyncResult<Customer> AsyncResult<unit>

NormalWorld
AsyncWorld
All steps are now composable into one single function
Url
AsyncResult<unit>

let jsonToCustomer jsonOrError =
jsonOrError
|> Result.bind jsonToCustomerDto
|> Result.bind dtoToCustomer
let downloadAndStoreCustomer url =
url
|> downloadFile
|> Async.map jsonToCustomer
|> AsyncResult.bind storeCustomerInDb
The patterns might be unfamiliar but once you get
use to them, you can compose code quickly.
Again, it takes much more time to explain than to write it!

Language support for monads
• F# has computation expressions
• Haskell has "do" notation
• Scala has "for" comprehensions
• C# has "SelectMany" 
a.k.a. using "bind" everywhere gets ugly

In conclusion…
• FP jargon is not that scary
– Can you see why monads are useful?
• The FP toolkit is very generic
– FP's use these core functions constantly!
• You can now recognize "map", "lift" and "bind"
– Don’t expect to understand them all straight away.

"The Functional ProgrammingToolkit"
– Slides and video will be posted at
• fsharpforfunandprofit.com/fptoolkit
Related talks
– "Functional Design Patterns"
• fsharpforfunandprofit.com/fppatterns
– "The Power of Composition"
• fsharpforfunandprofit.com/composition
– "Domain Modeling Made Functional"
• fsharpforfunandprofit.com/ddd
Thanks!
Twitter:@ScottWlaschin

The Functional Programming Toolkit (NDC Oslo 2019)

More Related Content

What's hot (20)

Similar to The Functional Programming Toolkit (NDC Oslo 2019) (20)

More from Scott Wlaschin (11)

Recently uploaded (20)

The Functional Programming Toolkit (NDC Oslo 2019)