Introducing Scala to your Ruby/Java Shop : My experiences at IGN

Introducing Scala to your
Java/Ruby Shop
My experiences at IGN

Manish Pandit
Silicon Valley Code Camp ‘12
Oct 7th, 2012

About me

Manish Pandit
Director of Engineering, IGN
@lobster1234

linkedin.com/in/mpandit

About IGN
We are a leading online media and services
company
obsessed with gaming and entertainment.

56MM Monthly Unique Views
737MM Monthly Page Views
20MM Monthly Video Views

My Objective
Is:
To share my story of introducing Scala to IGN, and
(hopefully) show you the path to enlightenment.

Isn't:
To incite a functional vs. imperative, or Scala vs. ____
debate.

Best tool for the job
• Front End : JS, CSS3, HTML5, backbone,
Coffeescript, jQuery
• Middle Tier : ZF2 (PHP), Rails
• APIs : Scala, some Java
• Persistence : MongoDB, MySQL, Redis
• Search : ElasticSearch
• Caching : Memcached, Varnish

APIs at IGN : Numbers
• ~5 Billion requests a month
• Average Response time of under
20ms on cache misses
• 17 APIs between Social, OAuth,
and Content
• Hardest hit APIs doing about
25K RPM at peak

The Kitchen Sink
• CMS – JSPs talking to Oracle DB and some Java
services
• V1 API : Java Services using Oracle DB with
Memcached
• V2 API : Rails based API layer using MongoDB and a
Java/Memcached front cache
• Lots of duct tape like messaging and cron jobs for
simple tasks
• Too diverse of a stack in the API infrastructure

The API Evolution : V3 [2012]
• A fresh look at the APIs and IGN’s direction
• First attempt to
– Integrate all content types
– Open up the APIs to external consumers
– Evolve the CMS, traditionally a monolithic system
– ElasticSearch
– Varnish Response Caching
• A learning opportunity for traditional Java
stack engineers

Roadblocks
• Resistance to change
– Culture
– Willingness and commitment
• Too many choices – making the right pick
– Involve every one
– Do not decide by committee, decide by learning
– Time box and measure everything
– Be accountable, as you’re looked upon as the
expert

The hunt for…
• A runtime as fast, if not faster than Java
• Concise, yet expressive
• Less boilerplate
• Smooth learning curve and ramp up
• Re-use ecosystem of libraries
• Growing adoption and developer velocity
• Built in support for concurrency
• Fast and fun development
• Next level of programming – disrupt!

Why Scala : Strategic Reasons
• Evolve the talent brand along side evolving
the platform
• Establish an API platform by replacing
“rewrite” with learning instead of porting
• Start thinking functional
• Align with the leaders in the APIs
• Get involved with a technology as it is evolving
(vs. established). Helps with influence.

Why Scala : Tactical Reasons
• Performance tied to cores than processor
speed, i.e. concurrency
• Ideal for API development – (relatively) simple
to handle concurrency and immutability
• JVM based = Performance
• Re-use Java toolkits and libraries
• Concise, yet expressive code
• Actor model makes the difficult parts easy

Why Scala : Tactical Reasons
• Richer Collections API
• Traits enabling Mix-Ins and behavior re-use
• Statically Typed w/Type inference
• Functional, but not alien (it can be!)
• REPL

Yes, Virginia, Scala is hard : DPP
So, how can you figure out if Scala will be "easy" or "hard" for your
organization:
•Your company has speakers at JavaOne, OSCON, Strangle Loop, QCon: Scala
will be easy
•Lunch-time discussions involve the criteria for moving from a developer to a
senior developer: Scala will be hard
•Your developers can write code in NotePad if they have to: Easy
•Your developers stare blankly or say 3 "Hail Marys" when they hear the
name "Zed Shaw": Scala == Hard
•Developers all follow Dean Wampler on Twitter: Scala Easy
•Your developers come in at 9:15 and leave before 6 and don't check work
email at night: Hard

http://blog.goodstuff.im/yes-virginia-scala-is-hard

Introducing Scala to your Ruby/Java Shop : My experiences at IGN

Picking up Scala : Week 1
• Scala for Java Developers fast-tracked the
learning
• Syntax Familiarity
• Java without semicolons?
def total(nums: List[Int]) :Int = {
var sum:Int=0
for(num<-nums) {
sum+=num
}
sum
}

Picking up Scala : Week 4
• Reduce the lines of code, embrace
immutability, use the functional paradigms
built into the language
• Unmystrify the implicits magic
nums.foldLeft(0)((n,c) => n+c)

Picking up Scala : Today
• Focus on immutability
• Think functional – with the knowledge gained
from using the functional aspects of the
language, build control abstractions
• Explore and implement ScalaZ
• Keep learning, and applying
• Move all the Actor processing to Akka

Less Boilerplate
public class HelloWorld{
public static void main(String... args){
System.out.println("Hello, World!");
}
}

object HelloWorld extends App{
println("Hello, World!")
}

scala> println("Hello, World!")

Expressive, yet concise
val x = if(n%2==0) "Even" else "Odd"

val x = for(i <- 1 to 10) yield i*2

val x = for(i <- 1 to 10; if(i%2==0)) yield i*2

val y = 20 match{
case p if(p%2==0) => "Even"
case _ => "Odd"
}

val largest = List(1,5,2,6).foldLeft(0)((a,b)=> if(a>b)
a else b)

Less pain points
No more dreaded null or throwing unwanted exceptions :
Use Option[T]

def thisMayReturnNull[T](x:T):Option[T]={
if(true) Some(x) else None
}

No more creating classes only to return more than 1 value :
Use Tuples

def doublePair(x:Int)= (x,x*2)

Try-catch mess
try{
//Construct a URL which can throw a MalformedURLException
//do something with a URL which can throw IOException
//do something with string encoding which can throw
// UnsupportedEncodingException
} catch{
case mal:MalformedURLException => //something
case ioe: IOException => //something
case e: Exception => //something
}

Type Inference
scala> val x = "Sunday"
x: java.lang.String = Sunday

scala> def x(n:Int) = n*3
x: (n: Int)Int

scala> 1 to 5
res2: scala.collection.immutable.Range.Inclusive =
Range(1, 2, 3, 4, 5)

scala> def x = if(true) Left("Yes!") else Right(new
Exception("Didn’t work out!"))
x: Product with Serializable with
Either[java.lang.String,java.lang.Exception]

Dynamic Mix-ins
class Home(bedrooms:Int=3,bathrooms:Double=2.5)

trait Garage{
def park(n:Int) = println("Parking " + n + "cars")
}

val someHome = new Home

val someHomeWithGarage = new Home with Garage

someHomeWithGarage.park(2)

Concurrency
• Actors make concurrency so much easier
– Messaging vs. Blocking
– Lightweight, 300-500 bytes per instance
– Isolated

Scala API Components
• Scalatra
• Lift-MongoRecord for MongoDB
• Casbah for MongoDB
• PlayFramework 2
• Actors for tasks like syndication
• sbt and Maven for Builds
• MongoDB for persistence

Other API Components
• Varnish Cache
• Elasticsearch
• Yammer Metrics (ping, healthcheck)
• Swagger (self-documenting RESTful APIs)
• 3Scale for Partner APIs
• IntelliJ IDEA and Eclipse with ScalaIDE
• PlayFramework 2.0 (New!)
• ScalaTest for Testing

Challenges with Scala *

• Steepness of the Learning curve depends
entirely on your org culture
• Scala is what you make of it
• http://scalaz.github.com/scalaz/scalaz-2.9.1-
6.0.4/doc.sxr/scalaz/BKTree.scala.html

• Middle Ground between Simplicity and,
well..elegance

* None of these challenges outweigh the benefits. They’re just something to
be aware of.

Challenges with Scala
• Slow compilation based on the source
• No (binary) compatibility of dependencies for
the major releases (2.8, 2.9, 2.10(?))
• Tooling not as rich as Java
– Changing (rapidly) with Typesafe investing in
ScalaIDE
– ScalaIDE comes with a Scala Worksheet, which is
like REPL but richer

Lessons Learned : Culture
• Culture of Meetups, Hack
Nights and Code Jams
• Being able to connect with
the contributors and
language creators
• Willingness to invest in
learning, evolving vs.
maintaining

Lessons Learned : Team
• Small team (~5 engineers)
• Java background helped a lot
• 10% time set aside for skill building
• Learn and share
• It is a commitment
• Communication!

Lessons Learned : Material
• Books that have exercises, so you
can quantify your learning.
• Online tutorials
• StackOverflow
• Coursera : A course on Functional
Programming by Martin Odersky

Resources
• Books
– Programming in Scala by Odersky
– Scala for the Impatient by Horstmann
• Interwebz
– http://debasishg.blogspot.com
– http://www.reddit.com/r/scala
– http://stackoverflow.com/tags/scala/info
– http://twitter.github.com/scala_school
– https://www.coursera.org/course/progfun

scala> case class Me(name:String="Manish
Pandit",twitter:String="@lobster1234")
defined class Me

scala> val me = new Me
me: Me = Me(Manish Pandit,@lobster1234)

scala> println(“Questions!”)
Questions!

Introducing Scala to your Ruby/Java Shop : My experiences at IGN

Recommended

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Introducing Scala to your Ruby/Java Shop : My experiences at IGN (20)

More from Manish Pandit (20)

Recently uploaded (20)

Introducing Scala to your Ruby/Java Shop : My experiences at IGN

Editor's Notes