Towards Reliable Lookups - Scala By The Bay

TOWARDS
RELIABLE LOOKUPS
Scala by the Bay 2015, August 14th
Patrick Prémont - @ppremont
Functional Programming Architect

MAP LOOKUPS
trait Map[K, V]{ 
def get(key: K): Option[V] 
}

DEFENSIVE
PROGRAMMING
.getOrElse( 
// Not sure how this 
// can ever happen, 
// but just in case… 
“Unknown” 
)()

INVALID REFERENCES
Incorrect judgements yield waste or bugs!
Impossible: Ignore/terminate
Possible:
Handle appropriately or 
revise program
?

INVALID REFERENCES
WITH TOTAL-MAP
Impossible: Types prove it!
?

RELATIONAL DATABASES
Foreign Key
quantity partId
10 102
100 102
25 101
id name
101 Monitor
102 Keyboard
Shipment Part
Brittle insertion Brittle removal

@ppremont
CURRENT PRACTICE
case class Data( 
parts: Map[Int, String], 
shipments: List[(Int, Int)])

@ppremont
DISTINCT IDTYPES
case class PartId(id: Int) 
 
case class Data( 
parts: Map[PartId, String], 
shipments: List[(Int, PartId)])

TOTAL MAPS
trait Map[K, V]{ 
def get(key: K): Option[V] 
} 
 
trait Total[V]{ 
type Id 
def apply(id: Id): V 
}

PATH-DEPENDENT TYPE
trait Data { 
val parts: Total[String] 
val shipments: List[(Int, parts.Id)] 
}

SAFER UPDATES TOO
trait Total[V]{ 
type Id 
def update(id: Id, f: V=>V) 
}

IDENTIFIER TYPES
• As we insert, need types with more values
• As we remove, need types with fewer values

INSERTION
def insert(value: V): { 
val total: 
Total[V] { 
type Id >: Total.this.Id 
} 
val newId: total.Id 
}

REMOVAL
def remove(removedKey: Id): { 
val total: 
Total[V] { 
type Id <: Total.this.Id 
} 
 
def filter(k: Total.this.Id):  
Option[total.Id] 
}

COVARIANT
ID TYPE PARAMETERS
case class Shipment[+P]( 
quantity: Int, 
part: P)
val parts: Total[String]
val shipments: 
List[Shipment[parts.Id]]

MULTIPLE CONSTRAINTS
case class Country(name: String) 
case class Address[+CountryId](country: CountryId) 
case class Customer[+AddressId](address: AddressId) 
 
trait Data { 
val countries : Total[Country] 
val addresses : Total[Address[countries.Id]] 
val customers : List[Customer[addresses.Id]] 
}

CYCLICAL CONSTRAINTS
case class User[+U]( 
name: String,  
follows: List[U]) 
 
trait Data { 
val users : Total[User[users.Id]] 
}

BREAKING THE CYCLE
case class User[+U]( 
name: String,  
follows: List[U]) 
 
trait Data { 
val userSet : Total[Unit] 
val users : userSet.Map[User[userSet.Id]] 
}

COMPLEMENT TYPES
trait Total { 
type Id 
type Comp 
def allocate: Comp 
def insertAt(id: Comp) : Total { 
type Id >: Total.this.Id 
} 
}

ID TYPE FAMILY
Nothing {}
Option[Nothing] {None}
Option[
Option[Nothing]]
{None, 
Some(None)}
Option[
Option[
Option[Nothing]]]
{None, 
Some(None), 
Some(Some(None))}

SELECTIVE SUBSETS
Nothing {}
Either[Unit, 
Nothing]
{Left(())}
Either[Unit, 
Either[Unit,
Nothing]]
{Left(()), 
Right(Left(()))}
Either[Nothing, 
Either[Unit,
Nothing]]
{Right(Left(()))}

BINARY TREE
Nothing {}
Id2[Unit, 
Nothing, 
Nothing]
{Element}
Id2[Unit, 
Id2[Unit,Nothing,Nothing],  
Nothing]
{Element, 
Left(Element)}
Id2[Unit, 
Id2[Unit,Nothing,Nothing]]
{Element, 
Left(Element), 
Right(Element)}

CONSTANT SPACE
Nothing {}
Id2[Unit, 
Nothing, 
Nothing]
{Id2(0)}
Id2[Unit, 
Nothing]
{Id2(0), 
Id2(1)}
Id2[Unit, 
Id2[Unit,Nothing,Nothing]]
{Id2(0), 
Id2(1), 
Id2(2)}

PERFORMANCE
• Total maps are binary trees: O(log n) operations
• No rebalancing: identifiers designate fixed paths
to a node
• n is the highest index in use

PERFORMANCE
DEGRADATION
Total.empty 
.insertAll(0 to size)
(1 to size) 
.map(i => (i, i))(breakOut) 
: Map[Int, Int]
us per inserted element
0
0.5
1
1.5
100 000 1 000 000
Total Map

BENEFITS
• Safe lookups, updates, inserts and removals
• Descriptive identifier types (parts.Id)

LIMITATIONS
• Traversal on removal
• Narrowing identifier types generates garbage
• Advanced, path-dependent types
• Type inference

ORTHOGONAL
PERSISTENCE
• Run a non-persistent program in a persistent
manner.
• Referential integrity must be built into the
program

TRANSACTIONS
type Txn = Data => (Data, A)
def run(txn: Txn) : IO[A] = IO{ 
val in = db.read[Data]  
val (out, a) = txn(in) 
db.write(out) 
a 
}

TOTAL MAP LIBRARY
github.com/boldradius/total-map

TOWARDS
RELIABLE LOOKUPS
Scala Symposium 2015 Paper: 
Referential Integrity with Scala Types
github.com/boldradius/total-map
Patrick Prémont - @ppremont

Towards Reliable Lookups - Scala By The Bay

More Related Content

What's hot (20)

Viewers also liked (8)

Similar to Towards Reliable Lookups - Scala By The Bay (20)

More from BoldRadius Solutions (14)

Recently uploaded (20)

Towards Reliable Lookups - Scala By The Bay