Scala

• Scala is an acronym for “Scalable Language”. It is a general-

purpose programming language designed for the
programmers who want to write programs in a concise,
elegant, and type-safe way. Scala enables programmers to
be more productive. Scala is developed as an object-
oriented and functional programming language.
• If you write a code in Scala, you will see that the style is
similar to a scripting language. Even though Scala is a new
language, it has gained enough users and has a wide
community support. It is one of the most user-friendly
languages.
• The design of Scala started in 2001 in the
programming methods laboratory at EPFL
(École Polytechnique Fédérale de Lausanne).
Scala made its first public appearance in
January 2004 on the JVM platform and a few
months later in June 2004, it was released on
the .(dot)NET platform. The .(dot)NET support
of Scala was officially dropped in 2012.
Scala is pure Object-Oriented programming language

• Scala is an object-oriented programming language.

Everything in Scala is an object and any operations
you perform is a method call. Scala, allow you to add
new operations to existing classes with the help of
implicit classes.
• One of the advantages of Scala is that it makes it
very easy to interact with Java code. You can also
write a Java code inside Scala class. The Scala
supports advanced component architectures
through classes and traits.
 Scala is a functional language

• Scala is a programming language that has implemented

major functional programming concepts. In Functional
programming, every computation is treated as a
mathematical function which avoids states and
mutable data. The functional programming exhibits
following characteristics:
• Power and flexibility
• Simplicity
• Suitable for parallel processing
• Scala is not a pure functional language
 Scala is a compiler based language (and not
interpreted)

• Scala is a compiler based language which

makes Scala execution very fast if you
compare it with Python (which is an
interpreted language). The compiler in Scala
works in similar fashion as Java compiler. It
gets the source code and generates Java byte-
code that can be executed independently on
any standard JVM (Java Virtual Machine).
• There are more important points about Scala
which I have not covered. Some of them are:
• Scala has thread based executors
• Scala can execute Java code
• You can do concurrent and Synchronized
processing in Scala
• Scala is JVM based languages
 Companies using Scala

• LinkedIn
• Twitter
• Foursquare
• Netflix
• Tumblr
• The Guardian
• Precog
• Sony
• AirBnB
• Klout
• Apple
 Scala Basics Terms

• Object: An entity that has state and behavior is known as an object.

For example: table, person, car etc.
• Class: A class can be defined as a blueprint or a template for
creating different objects which defines its properties and behavior.
• Method: It is a behavior of a class. A class can contain one or more
than one method. For example: deposit can be considered a
method of bank class.
• Closure: Closure is any function that closes over the environment in
which it’s defined. A closure returns value depends on the value of
one or more variables which is declared outside this closure.
• Traits: Traits are used to define object types by specifying the
signature of the supported methods. It is like interface in java.
 Things to note about Scala

• It is case sensitive
• If you are writing a program in Scala, you should
save this program using “.scala”
• Scala execution starts from main() methods
• Any identifier name cannot begin with numbers.
For example, variable name “123salary” is invalid.
• You can not use Scala reserved keywords for
variable declarations or constant or any identifiers.
•  
 Variable declaration in Scala

• In Scala, you can declare a variable using ‘var’

or ‘val’ keyword. The decision is based on
whether it is a constant or a variable. If you
use ‘var’ keyword, you define a variable as
mutable variable. On the other hand, if
you use ‘val’, you define it as immutable. Let’s
first declare a variable using “var” and then
using “val”.
• var Var1 : String = "Ankit“
• In the above Scala statement, you declare a
mutable variable called “Var1” which takes a
string value. You can also write the above
statement without specifying the type of
variable. Scala will automatically identify it.
For example:
• var Var1 = "Gupta"
• Declare using val
• val Var2 : String = "Ankit"In the above Scala
statement, we have declared an immutable
variable “Var2” which takes a string “Ankit
 basic concepts about Apache Spark

• Lazy operation: Operations which do not execute until

we require results.
• Spark Context:  holds a connection with Spark cluster
manager.
• Driver and Worker: A driver is in charge of the process
of running the main() function of an application and
creating the SparkContext.
• In-memory computation: Keeping the data in RAM
instead of Hard Disk for fast processing.
•  
 Spark has three data representations viz RDD, Dataframe,
Dataset. To use Apache Spark functionality, we must use one of
them for data manipulation
• RDD: RDD (Resilient Distributed Database) is a collection of elements, that can be divided
across multiple nodes in a cluster for parallel processing. It is also fault tolerant collection of
elements, which means it can automatically recover from failures. RDD is immutable, we can
create RDD once but can’t change it.
• Dataset: It is also a distributed collection of data. A Dataset can be constructed from JVM
objects and then manipulated using functional transformations (map, flatMap, filter, etc.). As
I have already discussed in my previous articles, dataset API is only available in Scala and Java.
It is not available in Python and R.
• DataFrame: In Spark, a DataFrame is a distributed collection of data organized into named
columns. It is conceptually equivalent to a table in a relational database or a data frame. It is
mostly used for structured data processing. In Scala, a DataFrame is represented by a Dataset
of Rows. A DataFrame can be constructed by wide range of arrays for example, existing RDDs,
Hive tables, database tables.
• Transformation: Transformation refers to the operation applied on a RDD to create new RDD.
• Action: Actions refer to an operation which also apply on RDD that perform computation and
send the result back to driver.
• Broadcast: We can use the Broadcast variable to save the copy of data across all node.
• Accumulator: In Accumulator, variables are used for aggregating the information.