Scala for Machine Learning

Editor's Notes

• #2: Context of the presentation: The transition from Java and Python to Scala is not that easy: It goes beyond selecting Scala for its obvious benefits.- support functional concepts- leverage open source libraries and framework if needed- fast, distributed enough to handle large data setsScala was the most logical choice. Scientific programming may very well involved different roles in a project: Mathematicians for formulas Data scientists for data processing and modeling Software engineering for implementation Dev. Ops and performance engineers for deployment in productionIn order to ease the pain, we tend to to learn/adopt Scala incrementally within a development team.. The problem is that you end up with an inconsistent code base with different levels of quality and the team developed a somewhat negative attitude toward the language.The solution is to select a list of problems or roadblocks (in our case Machine learning) and compare the solution in Scala with Java, Python ... (You sell the outcome not the process).PresentationA set of diverse Scala features or constructs the entire team agreed that Scala is a far better solution than Python or Java.
• #3: There are really 3 dimensions to building machine learning apps. Abstraction is critical to encode or translate mathematical concepts Ability to create workflow is needed for chaining data transformation and reduction Scalable execution comes into play when deploying application in production How can the main players (Mathematician, data scientific, developers or dev. Ops.) leverage Scala in their daily tasks?
• #4: Being an object oriented and functional language, Scala has a lot of features and powerful constructs to choose from…. Here is a list of some of the features of Scala that are particularly valuable for writing machine learning algorithms or building applications that use these algorithms.
• #5: Let’s start with ability in Scala to represent or implement abstractions such as Mathematical concepts, equations Programming concepts already defined in the Scala standard library.
• #6: Geometric entities on a differential (Riemann) manifold are defined as tensors. Tensor can be Covariant Contra variant A bilinear form such as Tensor product Inner product n-differential forms …
• #7: A contravariant vector or vector field is associated with a covariant functor. A covariant vector (linear map V -> Field) is associated with a contravariant functor A Tensor product is a bilinear form associated with a BiFunctor (or Hom functor) An inner product is associated with a ProFunctor ……
• #8: Mathematician and data scientists are faced with this problem.
• #9: Covariant functors are used to define the morphisms or transformations on vectors fields (contravariant) with type VField[U] = List[U] where U is a field ( u(x, y,…)). If U and V are groups of fields f(xi) then F[U => V] := F[V] => F[V] Functors as defined in Scala are actually covariant functor as defined in algebraic topology.
• #10: Contravariant functors are used to define the morphisms or transformations on covariant vectors (or linear maps) with type CoField[U,T] = U => T where U is a (Contravariant). A U is an type argument in a higher Kind CoFunctor, this functor has to be contravariant: F[U => V] := F[V] => F[V] Functors as defined in Scala are actually covariant functor as defined in algebraic topology. Note: The inversion of the arrow for the morphism f in the map method is equivalent to raising/lowering the index in the tensor (Einstein)
• #11: Morphisms applied to each type of tensor which require a specific functors (ProductFunctor, Bifunctor, HOM functor….). As example the definition of the functors for morphisms on the inner product, ProdFunctor CoField dot VField => Set and the bi-functor BiFunctor. Paul Phillips has a great open source library of various type of functors, applicatives and monads.
• #12: Machine learning is used to group or classify observations into clusters, classes, patterns…. Two observations are compared using a cosine (or inner product). The inner product works well for linear models (tangent plane on the manifold). For non-linear model the inner product (known as the tensor or Riemann metric) has to be projected to the manifold. The projection of the inner product (linear plane) to the manifold is mathematically resolved using the Laplace-Beltrami operator and the heat equation. The solution is a exponential map defined by a kernel function. I assume you are familiar with Monads are used for collection, error handling in Scala standard library. There is no need to know kernel to understand the benefits of using Monads in this particular case. A kernel function is used in a lot of supervised learning such as support vector machine and neural networks as well as dimension reduction, kernel principal components analysis as an example. The challenge is to create and reuse kernel functions during multiple training runs in an attempt to extract the fittest model from the data set. By 1st class programming element, I mean Kernel functions have to be defined as a class Kernel functions can be categorized by sharing basic group functions (or monadic operations)
• #13: The purpose here is to generate and experiment with any kind of explicit kernels by defining and composing two g and h functions A function h operates on each feature or component of the vector A function g is the transformation of the dot product of the two vectors. The dot product is computed by applying the function to all the elements and compute the sum. The “dot” product K is computed by traversing the two observations (vector of features), computing the sum and finally applying the g transform. The variable type is the type of the function g (F1 = Double => Double)
• #14: Once the functor is defined, the monad is created by adding the flatMap method. The monad, KFMonad which take a kernel function as argument is defined as an implicit class so kernel functions “inherits” the monadic methods. The map and flatMap transformation applies to the g function or transformation on the inner product. The flatMap method is implemented by creating a new Kernel and applying the transformation to only one of the component of the Kernel function: function h (in red). This “partial” monadic operation is good enough for building Kernel functions on the fly.
• #15: Kernel functions that project the inner product to the manifold for non-linear models belong to the family of exponential functions Polynomial functions and Radius basis functions are two of the most commonly used kernel functions. Note: The source code is shown here to illustrate the fact that the implementation in any other language would be a lot more messy and won’t be able to fit in any of those slides.
• #16: Finally we can chain flatMap to map to compose two kernel function and compute the dot/inner product of the resulting kernel function. Note that composed kernel function used the h function of the last invocation in the for instruction. The method does not expose the functor or KF classes that wraps the components of the kernel function.
• #17: Implicit kernels that are defined iteratively or as a solution of equation or ODE such as Laplace-Beltrami operator on Manifold require a different class and monadic implementation The two filter functions in Scala standard library are filter and filterWith. For-comprehension is a “sugar” syntax that wraps a stack of flatMap and map invocation. It can be used to create any complex container or programming entity including dynamic workflows. You will be introduced to a different approach to create workflows by weaving transformation/reduction on datasets.
• #18: We looked at the abstraction of mathematical concepts. What about abstraction of programming concepts? Let’s consider the problem of handling a computation error and recovery the workflow to its previous known stable state (transactional computation). There are two outcomes If the current step in the computation succeeds, the next computation stage is started If the step fails, the workflow rolls back to the previous known stable state. Can we leverage existing Scala constructs without the need to introduce a new pattern? Standard library classes are either final or implemented a sealed trait or abstract class. Creating a helper or utility class (reuse by composition) adds a new indirection and clutter the design.
• #19: Here is the dilemma: On one hand, the concept of transactional computation (all or nothing) is better expressed with the Either class On the other hand, exceptions are implemented through the Try class (as in C++ and Java) We need to combine these two concepts (Try and Either) by extending “transparently” the Try class with Either. This is accomplished by wrapping Try with an implicit class containing toEither (Try -> Either) method.
• #20: Let’s apply the implicit class Try2Either to a simple data transformation: normalization. If the denominator “sum” were too small, downstream computation would incur significant rounding errors. In this case, the normalization aborts and a recovery mechanism (instance of class Recover) is invoke. If the normalization proceeds without rounding errors, the next computation step (creating a log of the vector x) is triggered.
• #21: As I mentioned previously, such a pattern can be used as a micro-router. I used the Try.toEither to roll back to the previous known stable state of the computation which is stored in file. Note: Delimited continuation seems would be an interesting alternative to investigate for transactional computation.
• #22: The second requirement for a programming language (to be a good fit for implementing machine learning) is the ability to create dynamic and configurable data flows/workflows as a sequence of transformation/reduction on data.
• #23: The first thing to come to mind in creating complex system from existing objects (or classes) is a factory design pattern. Design patterns have been introduced by the “Gang of four” in the eponymous “Design Patterns: Elements of Reusable Object-Oriented software” some 20 years ago… The list of factory design patterns includes Builder, Prototype, Factory method, Composite, Bridge and obviously Singleton. Those patterns are not very convenient for weaving data transformation (these transformation being defined as class or interface). This is where dependency injection popularized by the Spring framework comes into play. Beyond composition and inheritance, Scala enables us to implement and chain data transformations/reductions by stacking the traits that declare these transformations or reductions.
• #24: We briefly mentioned that the for comprehension can be used to chain/stack data transformation. Dependency injection provides a very flexible approach to create workflows dynamically, sometimes referred as the Cake pattern. Note: As far the 3rd point, deployment of tasks, it usually involves a actor-based (non blocking) distributed architecture such as Akka and Spark. We will mention it briefly later in this presentation is introducing mailbox back-pressure mechanism.
• #25: The implementation in Scala matches perfectly the universal mathematical formalism Here is another example Declaration variable 𝑥∈ℝ, 𝑦∈ℝ Declaration of model f(x,y)=𝑥+𝑦 Instantiation of variable 𝑥=5, 𝑦=7; 𝑓 5,7 =12
• #26: Let’s look at the Training module/task as an example. The task of training a model is executed by a Supervisor instance that can be either a support vector machine or a multi-layer perceptron, in this simplistic case. Each of these two “supervisors” can have several implementations (single host, distributed through a low-latency network,…) Once defined, the modules are to be weaved/chained by making sure that output of a module/tasks matches the input of the subsequent task. Notes: The training module can be broken down further into generative and discriminative models. Real-world applications are significantly more complex and would include REST service, DAO to access relational database, caches…. The terms “module”, “tasks” or “computational tasks” are used interchangeably in this section.
• #27: Let’s consider the Preprocessing module (or task) implemented as trait Preprocessing of a data set is performed by a processor of type Preprocessor that is defined at run time. Therefore the preprocessor has to be declared as an abstract value. The three preprocessors defined in the preprocessing modules are Kalman filter, Moving Average (MovAv) and Discrete Fourier filter (DFTF). Those 2 inner classes act as adapter or stub to the actual implementation of those algorithm.
• #28: Here is an implementation of the Kalman and Discrete Fourier transform band-pass filter. The2 inner classes, Kalman and DFTF act as adapter or stub to the actual implementation of those algorithm. It allows the implementation may consist of multiple version. For instance filtering.Kalman is a trait with several implementation of the algorithm (single host, distributed, using Spark…) Such design allows to Select the type of preprocessing algorithm within the Preprocessing module or namespace Select the implementation of this particular type of algorithm/preprocessor in the filtering package
• #29: Clustering workflow is created by weaving (or stacking) a preprocessor and a reducer. Modeling workflow is generated by stacking a preprocessor, a supervisor and a validator.
• #30: From the data management perspective, Clustering implements two consecutive data transformations: preprocessing and dimension reduction.
• #31: Modeling workflow is created by chaining an implementation of the filter, training and validator, all selected at run-time. Modeling is therefore implemented as a stack of 3 traits, each representing a transformation or reduction on data sets.
• #32: Computational tasks related to machine learning can be complex and lengthy. The process should be able to select the appropriate date flow (or sequence of data transformation or reduction) at run time, according to the state of the computation. In this simple case, a clustering task is triggered if anomalyDetection is needed, training a model is launched otherwise. These conditional path execution are important for complex analysis or lengthy computation that require unattended execution (i.e. overnight or over the week-end). Note: The overriding of the abstract value for the Modeling workflow are omitted here for the sake of clarity Summary: This factory pattern operates on 3 level of componentization: Dynamic selection of 1- Workflow or sequence of tasks according to the objective of the computation (i.e. Clustering => Preprocessing) 2- Task processing algorithm according to the data (i.e. Preprocessing => Kalman filter) 3- Implementation of task processing according to the environment (i.e. Kalman filter => Implementation on Apache Spark)
• #33: At last, Scala has to be performant and scalable to execute some lengthy computations.
• #34: Let’s start with the ubiquitous tail recursion. Although tail recursion is widely known and used, I have to include in this presentation because it is essential to the implementation of dynamic programming techniques in Scala. Quite a few machine learning algorithm does not rely on a close, explicit, global minimization of the loss function (or maximization of the log likelihood). Dynamic programming techniques are therefore required for such classifier as Q-learning, hidden Markov model or back propagation neural network. For better or worse, recursion has the reputation of being an elegant but inefficient option for implement dynamic programming algorithm. Scala provides us with a viable alternative with tail recursion.
• #35: The Viterbi algorithm is an excellent candidate for tail recursion because the number recursions (equal to the length of the sequence of states) is finite and reasonable and the number of parameters to be passed is small.
• #36: Here is the skeleton of the implementation of the Viterbi algorithm that traverses a sequence of observations. This implementation is representative of implementation of dynamic programming algorithms in Scala: First invocation (on observation of index 0) initializes the context of the tail recursion (hidden states in our case) Subsequent invocations update the states parameters and recurse to the next observation (index t+1) Finally the last invocation on the last observation (index obs.size -1) updates the last state’ parameter, performs some cleaning activities if needed and exits. As a side note, using a closure for which the recursive call update a global values or class attribute will degrade the performance of the recursion.
• #37: Simple performance test using Fibonacci sequence shows a 3 fold improvement. This “back of the envelop” performance test consists of Viterbi to compare the tail recursion and the recursion without tail elimination using 5 samples with increasing number of observations. Although it is not a rigorous and scientific finding, the tail recursion becomes very effective for test with more than 60 observations It would be interesting to compare the performance of those two recursions with the performance of an iterative function.
• #38: Real-time streaming is becoming popular (i.e. Apache Spark streaming library, Akka reactive streams….). Short of using one of these frameworks, you can create a simple streaming mechanism for large data sets that require. Streams vs. Iterator: Iterator does not allow to dynamically select the chunk of memory or preserve it if necessary for future computation. It is not uncommon to have to train a model as labeled data becomes available (online training). In this case, the size of the data set can be very large and unknown. The processing of the data would result in high memory consumption and heavy load on the Garbage collector. Finally, the traversing the entire data sets (~ allocating large memory chunk) may not even needed as some computation may abort if some condition is met. Scala’s streams can help!
• #39: Most of machine learning algorithms consists of minimizing a loss function or maximizing a likelihood. For each iteration (or recursion) the cumulative loss is computed and passed to the optimizer (i.e. Gradient descent or variant) to update the model parameters. In order to minimize the memory footprint, two actions have to take place Allocate slice of the data set (memory chunk) from the heap using take method. Release the memory chunk back to the Garbage collection through a drop method. Each slice of n observations is allocated, processed by the loss function then released back to the Garbage collector.
• #40: The Loss class has a single method, exec, that traverses the stream. Once again, the loss is computed using a tail recursion. An observation is defined as y = f(x) where x the feature set containing for instance, age, ethnicity of patient, body temperature and y is the label value such as the diagnosed disease.. The tail recursion allocates the next slice of STEP observations through the take method, computes the lost, nextLoss, then drop the slice. The reference is recursively redefined as the reference to the remaining stream. The problem is that the garbage collector cannot reclaim the memory because the first reference to the stream is created outside the recursion. The solution is to declare the reference to the stream as weak so it chunks of memory associated to the slices/batches of observations already processed can be reclaimed.
• #41: The reference of the stream is created as a Weak java reference to an instance created by the stream constructor, Stream.iterate. In this case, the weak reference has been used to show Java concepts are still relevant.
• #42: Let’s compare the memory consumption of three strategies to compute the loss function on a very large dataset. A list A stream with standard reference A stream with weak reference In the first scenario, and as expected, the memory for the entire data set is allocated before processing. The memory requirement for the stream with a strong reference increases each time a new slice is instantiated, because the memory block is held by the reference to the original stream. Only the stream with a weak reference guarantees that only the memory for a slice of STEP observations is needed through the entire execution.
• #43: Let’s compare the memory consumption of three strategies to compute the loss function on a very large dataset. A list A stream with standard reference A stream with weak reference In the first scenario, and as expected, the memory for the entire data set is allocated before processing. The memory requirement for the stream with a strong reference increases each time a new slice is instantiated, because the memory block is held by the reference to the original stream. Only the stream with a weak reference guarantees that only the memory for a slice of STEP observations is needed through the entire execution.
• #44: We mentioned earlier that a learning platform requires implementing, wiring and deploying tasks. Akka or framework derived from Akka, are commonly used to deploy workflow for large datasets because of the immutability, non-blocking and supervision characteristic for actors. Scala/Akka actors are resilient because that are defined with hierarchical context for which an actor because a supervisor to other actors. In this slide, a router is a supervising actor to the workers and, depending on the selected strategy, is responsible of restarting a worker in case of failure. But, what about the case for which the load (number of messages in the mail box) for each actor increases indefinitely?
• #45: The objective is to avoid bottleneck in the computation data flow which would result in overflowing actors’ mail box/local buffers. A strategy to control the flow (or back-pressure flow) is needed to regulate the data flow across all modules. This example use a back-pressure handling mechanism that consists of monitoring bounded mail boxes. This is a simplistic approach to flow control described for the sake of illustrating the concept. As we will see later, there is a far more effective mechanism to deal with back-pressure.
• #46: In this example, an actor, Controller loads chunk of data, partitions and distributed across multiple Worker actors, along with a data transformation. Upon receiving the message ‘Compute’ the workers process data given a transformation function. The workers returns the processed data through a Completed message. The purpose of the watch dog actor, ‘Watcher’ is to monitor the utilization of mailbox and report it to the Controller This is a simple feedback control: 1- Watcher monitors the utilization of the mailbox (average length) 2- The controller adjust the size of each batch in the load message handler (throttling) 3- The workers process the next batch
• #47: Let’s start with the Worker actor. The load on a worker depends on three variables 1- The amount of data to process 2- The complexity of the data transformation 3- The underlying system (cores, memory..) The controller provides the slice of data to be processed by the workers msg.xt as well as the data transformation msg.fct.
• #48: Let’s look at our watch dog actor, Watcher: It computes the load as the average mailbox utilization and send it back to the controller through a Status message.
• #49: As its name implies, the controller configure and manage the dynamic workflow and control the back pressure from the worker actors. As far as the configuration is concerned, the Controller generates a list of workers, the bounded mailboxes, msgQueues, for the workers and ultimately the watcher actor. The worker and watcher are created within the Controller context using the Akka actorOf constructor.
• #50: As far as the management of data flow and feedback control loop, the Controller loads partition and distribute batch of data points to the worker actors (message: Load) processes the results of the computation in workers (message: Completed) - Throttle up or down the flow upon receiving the status on utilization of mail boxes from the watcher (message: Status)
• #51: The composition of the messages processed by the controller are self-explanatory. It Adjusts the size of the next batch if required (throttle method) Extracts the next batch of data from the input stream Partition and distribute the batch across the worker actors Send the partition along with the data transformation to the workers.
• #52: The implementation of the throttle method is rather simple. It takes the load computed by the watcher actor and the current batch size (number of data points to be processed) as input. It update the batch size using a simple ratio relative to the watermark. For instance if load is below the watermark, the batch size is increased.
• #53: The bottom graph describes the throttle action and the complexity of data transformation. The complexity of the data transformation and has an impact on the load on workers. It varies from 0 (simple map operation) to 2 (complex data processing involving recursion or iterations). The throttle intensity ranges between -6 (rapid decrease of size of batches) and +6 (rapid increase of size of batches of data) The top graph displays the actual utilization of the mail boxes with capacity of 512 messages as regulated by the feedback control loop (executed by the controller).
• #54: The deployment of a reactive data flow in production would require significant improvement on our Naïve model. The feedback control loop could be smoothed with a moving average technique or Kalman filter to avoid erratic behavior We would need to provide a larger range of options for control actions beside adjusting the size of data batches: increase of number of workers, mail box capacity, caching strategy, .. A fine grained set of actions reduces also the risk of instable systems. The watch dog should be able to handle dead letters in case of failure (mailbox overflowing) Reactive streams control the flow back pressure at the TCP connection level. It is far more accurate and responsive that mailbox utilization.
• #55: The implementation of machine learning algorithm would benefit from few other interesting features of the Scala language, I have not used yet. DSL are used to introduce domain specific semantic in the library. It is also intensively used in reactive programming. As stated earlier, Akka reactive streams are an efficient approach to managing data flows. It would be particularly valuable to combine the flexibility of dependency injection with the efficiency of Akka streams and reactors.
• #56: I hope you will find some of these references, useful