Scala Days Highlights | BoldRadius

Who?
● BoldRadius Solutions
○ boldradius.com
○ Typesafe Partner
○ Scala, Akka and Play specialists
○ Ottawa, Saskatoon, San Francisco, Boston, Chicago, Montreal, New
York, Toronto
● Michael Nash, VP Capabilities
● Adam Murray, VP Business

What?
What was ScalaDays all about?
● Held in San Francisco March 16th thru 18th
○ About 800 attendees
○ Three categories (intermediate, beginner,
advanced),
○ Four tracks, 55 presentations, three keynotes
● Followed by Two Intensive Training days
○ All the Typesafe courses offered

Keynotes
● Martin Odersky, Chief Architect & Co-Founder at Typesafe
○ Scala - where it came from, where it’s going
● Danese Cooper, Distinguished Member of Technical Staff -
Open Source at Paypal
○ Open Languages
● Dianne March, Director of Engineering Tools at Netflix
○ Technical Leadership from wherever you are

Highlights
Too many great sessions to summarize them all, we have
to extract a few recurring themes…
○ Distributed Application Design and Development
○ Big Data/Fast Data
○ Types and Safe Scala
○ Performance and Scalability

Distributed Applications
● Life Beyond the Illusion of Present
● Reactive Reference Architectures
● Akka in Production: Why and How
● Easy Scalability with Akka
● Scalable task distribution with Scala, Akka
and Mesos
● A Guided Tour of a Distributed Application

Performance and Scalability
● Scala Collections and Performance
● Shattering Hadoop’s Large-Scala Sort Record with
Spark and Scala
● Type-safe off-heap memory for Scala
● Akka in Production: Why and How
● Easy Scalability with Akka
● The JVM Backend and Optimizer in Scala 2.12

Big Data, Fast Data
● Shattering Hadoop’s Large Scala Sort Record with
Spark and Scala
● Scala - The Real Spark of Data Science
● Apache Spark: A Large Community Project in Scala
● S3 at Scala: Async Scala Client with Play Iteratees and
Composable Operations
● The Unreasonable Effectiveness of Scala for Big Data
● Scalable task distribution with scala, Akka and Mesos

Types and Safer Scala
● Keynote: Scala - where it came from, where it’s going
● Towards a Safer Scala
● Leveraging Scala Macros for Better Validation
● Type-level programming in Scala 101
● Improving Correctness with Types
● Happy Paths: Functional Constructs in the Wild
● The Scalactic Way
● Delimited dependency-typed monadic checked
exceptions

The Rest...
Many excellent talks were outside these categories
● Why Scala.js
● Reactive Slick for Database Programming
● Exercise in machine learning
● Functional Natural Language Processing
● If I Only Had a Brain...in Scala
● Akka HTTP: the Reactive Web Toolkit
● many many others

Highlights of Specific Sessions
A quick taster of what was in some of the more
popular sessions...

Scala - Where it came from, where it’s going

Where it’s from, where it’s going
“Scala is a gateway drug to Haskell” (in actual fact it’s going well beyond Haskell.)
Slides: http://www.slideshare.net/Odersky/scala-days-san-francisco-45917092
Came from a practical combination of OOP and functional programming
- Funny story about hipster syntax (..) instead of [..], <..> instead of [..], ??? instead of <..>
- Trend in Type Systems
- Scala JS is no longer experimental
- TASTY: new scala-specific platform
- Introduction to DOT
- Type Parameters
- Better treatment of effects with implicit Functions instead of Monads

TASTY
Scala faces challenges:
● binary compatibility
● having to pick a platform: JDK( 7,8,9,10, ?) or Javascript.
Proposing a scala-specific platform called TASTY, (serialized typed abstract syntax tree),
as an intermediate representation before bytecode - carries metadata of types, can be compiled with
different versions of JDK, and to Javascript.

Tasty will enable:
● instrumentation
● optimization
● code analysis
● refactoring
● publish once run anywhere
● automated remapping to solve binary compatibility issues.

Explorations:
Implicits that compose

Explorations:
Hope to find something cooler than Monads to handle effects.
● Monads don’t commute
● Require Monad transformers for composition
● Monad transformers make Martin’s head explode

Toward a Safer Scala
Leif Wickland
http://tinyurl.com/sd15lint
● Scalac enables some error-prone code.
○ head of empty List?
● Using Static Analysis to detect errors early
● IDE based solutions
○ Inconsistencies
○ If not in release build process, doesn’t exist
● Web-based analysis
○ outside of compile loop
○ relatively immature analysis

● Scala Style
● Wart Remover
● Linter
● Watch
○ Abide
○ Scapegoat
○ Supersafe

Life Beyond the Illusion of Present
Jonas Bonér:
The idea of the present is an illusion. Everything we see, hear and feel is just
an echo from the past. But this illusion has influenced us and the way we view
the world in so many ways.
There is no present, all we have is facts derived from the merging of multiple
pasts. The truth is closer to Einstein’s physics where everything is relative to
one’s perspective. As developers we need to wake up and break free from the
perceived reality of living in a single globally consistent present.

The advent of multicore and cloud computing architectures meant that most
applications today are distributed systems—multiple cores separated by the
memory bus or multiple nodes separated by the network—which puts a harsh
end to this illusion.
The only way to design truly scalable and performant systems that can
construct a sufficiently consistent view of history—and thereby our local
“present”—is by treating time as a first class construct in our programming
model and to model the present as facts derived from the merging of multiple
concurrent pasts.

How do we deal with failure and communication unreliability in real life?
Confirmation and repetition
We can’t force the world into a globally consistent present (CRUD).
Mentioned 2 paradigms/theories:
● CALM (consistency as logical monotonicity)
● CRDT (Commutative Replicated Data Type)
CRDT (Commutative Replicated Data Type) eventually consistent data types
● minimize contention / coordination in a distributed system.
● set, maps, graphs: rich data types.
● monotonic merge function: all state change is monotonically increasing, no way
back.

Improving Correctness with Types
http://workday.github.io

● Using Wrapper Types (aka Tiny Types) instead of primitives
● Never use null, or throw Exceptions
● use === org.scalactic.TypeCheckedTripleEquals
○ requires the types of the two values compared to be in a subtype/supertype
● Never Use Non-Empty Lists when a list must be populated (org.scalactic.Every)
● Use Type Tags (ala Shapeless, Scalaz)
● Use Path Dependent Types
● Other reading
○ Self recursive types
○ Phantom Types
○ Shapeless
○ Scalactic
Types: Defensive Programming. Fail Fast. Design By Contract.

Function Passing Style
Heather Miller:
● A new programming model called function passing designed to overcome many of
imperative / weakly-typed issues found in traditional “big data” processing
systems.
● Provides a more principled substrate on which to build data-centric distributed
systems.
● Pass safe, well-typed serializable functions to immutable distributed data
● Based on her work on Pickling and Spores
● Uses Spores (Serializable Functions) for a distributed model.
● Kind of an Inverse of the Actor Model
● Stateless. data is stationary, functions are passed around.
● Uses Data Silos accessed through a Silo Ref.

The Unreasonable Effectiveness of Scala for Big Data
Dean Wampler
● How Hadoop Works - Map Reduce
○ Problems
■ Hard to implement algorithms
■ The Hadoop API is horrible
● Scalding
○ An improved Hadoop API in Scala
○ Problems
■ Still uses a batch mode
● Spark
○ An elegant, functional API
○ Still in Batch Mode, but with mini batches which
approach real time.

Akka HTTP: The Reactive Web Toolkit
Roland Kuhn
● Replaces Spray
● Uses Akka Streams
○ Sources emit values to the stream
○ Sinks receive values, act on them
○ Sources can compose using Zip and Graph shapes
● “The pinball interpreter”
○ produce data
○ move downstream through transformations
○ get to the effect
○ go up and ask for more data
○ Filters interrupt the flow before getting to the effect, make the pinball go back
upstream.
● A live coded demonstration of using Streams and Http
● Expected timeline for Streams - 4 weeks
● Expected timeline for Http - 8 weeks

The Scalactic Way
Bill Venners
http://www.slideshare.net/bvenners/the-scalactic-way
ScalaTest: quality through tests
Scalactic: quality through types
SuperSafe: quality through static analysis

Scala Days Highlights | BoldRadius

Reactive Slick for Database Programming
Stefan Zeiger
http://slick.typesafe.com/talks/scaladays2015sf/Reactive_Slick_for_Database_Programming.pdf
Slick 3.0
● JDBC is inherently blocking (and blocking ties up threads)
● Traditional Model
○ Fully synchronous
○ One thread per web request
○ Contention for Connections (getConnection blocks)
○ Database back-pressure creates more blocked threads
○ Doesn’t scale

New Slick Architecture makes use of a new DataType to provide
Asynchronous Database I/O
● based on State, IO and Free Monads.
● Returns a Future[R]
● Creates a separate ExecutionContext, avoiding blocking of current thread
● Works with akka-streams to create back pressure, so DB only gives as much data as client can
process.
● For performance purposes it pre-fetches some data to keep client busy while it waits for the next
portion from DB.
sealed trait DBIOAction[+R, +S <: NoStream, -E <: Effect]{
def map[R2](f: R => R2)(implicit executor: ExecutionContext): DBIOAction[R2, NoStream, E]
def flatMap[R2, S2 <: NoStream, E2 <: Effect](f: R => DBIOAction[R2, S2, E2])(implicit
executor: ExecutionContext): DBIOAction[R2, S2, E with E2]
...
}

Easy Scalability with Akka
Michael Nash
● Reviewed Akka, CQRS, ES
● Introduced Distributed DDD
● Identical clustered system with DDDD and Without
● Gatling performance tests on both

ConductR
● Application manager that empowers ops to
deploy distributed systems
● Uses Akka, Play, Aka Streams, Akka
Cluster, FSM, Akka Data Replication
● How can we run cluster based apps ensuring
the seed nodes are started first?
○ State replicated using Data
Replication
● How can we consolidate logging?
○ Using Akka Streams
● How can we avoid batching?
○ Use Event Driven Architecture
● How can monitor/test multiple nodes?
○ Use the visualizer built into conductor
● How can we share state among the nodes?
○ Use Akka Data Replication

Scala.js
Building a data-driven map of the world in 10
minutes….

Whats wrong with Webapps?
● Everything String/Map[String, String]
● Compiler cannot help you!
○ document.getElementByld("my-id")
● Javascript!
○ ["10", "10", "10", "10"].map(parseInt) [10, NaN, 2, 3] // WTF
ScalaJS
● lets you write webapp in one language
● Swap String-typing for Strong-typing
● Type Everything!!
○ val res: Future[Seq[String]] = Ajax[Api].list(inputBox.value).call()
● IS NOT JAVASCRIPT
○ scala.js> List("10", "10", "10", "10").map(parseInt) List(10, 10, 10, 10)
// Yay!

The BoldRadius Crew Packing up...

ScalaDays Amsterdam!
● June 8th to 10th
● Followed by Training

Q & A
michael.nash@boldradius.com
adam.murray@boldradius.com

Scala Days Highlights | BoldRadius

Recommended

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Scala Days Highlights | BoldRadius (20)

More from BoldRadius Solutions (9)

Scala Days Highlights | BoldRadius