From Experimentation to Production - Scala & Python APIs for DL4J

Nov 7, 20172 likes236 views

This document discusses bridging the gap between experimentation and production for deep learning. It summarizes Skymind's tools for experimentation using Python like ND4S and Jumpy, and production using Scala like ScalNet. It outlines the different requirements for experimentation vs production, and how Skymind's tools can help meet both through high performance tensor libraries, deep learning frameworks, and model import capabilities.

From Experimentation to Production - Scala & Python APIs for DL4J

AGENDA
● About Skymind
● Experimentation vs production
● ND4S: Tensors for Scala
● Jumpy: Numpy for the JVM
● Model import for DL4J
● ScalNet: Scala API for DL4J

Founded 2014
Clients 14 Enterprises
3,500 GH Forks, 7,200 Stars
300,000+ DL4J downloads/mo.
Team ~35; mostly engineers; 7 PhDs
COMPANY OVERVIEW

Deeplearning4j
Build, train, and deploy neural
networks on JVM
DataVec
Data ingestion, normalization,
and vectorization
RL4J
Reinforcement learning for the
JVM
SKYMIND TOOLS
ND4J
High performance linear algebra CPU
and GPU libraries
Arbiter
Hyperparameter search for optimizing
neural networks

Deeplearning4j
Build, train, and deploy neural
networks on JVM
Model import
Import of models from Keras
(Tensorflow, Theano, CNTK, Caffe)
ScalNet
Scala API for Deeplearning4J
SCALA & PYTHON APIS
ND4J
High performance linear algebra CPU
and GPU libraries
Jumpy
Python interface for ND4J: a numpy
wrapper for the JVM
ND4S
Scala wrapper for ND4J

EXPERIMENTATION
REQUIREMENTS
● Open source (preferably python)
● Insightful evaluation & debugging
● Fast experimentation cycle
● Reproducibility
● Collaboration

PRODUCTION
Proper ETL
pipes for prod
Real-time data
preparation
Model serving
Low latency
predictions
Model history
Model &
prediction
history
Parallel trainingStreaming data
Distributed
storage
Model updates
Various data
sources

PRODUCTION
REQUIREMENTS
● Open-source (Linux, Hadoop)
● Integrates With Existing Tech (JVM)
● Cross-Team Solution (DevOps, DS)
● Scalable, Containerized, Fast
● High availability
● General-Purpose, Customizable Framework

BRIDGING THE GAP
● Technical:
○ Programming languages
○ Eco systems / tooling
● Cultural:
○ Data science vs. engineering
● Cognitive:
○ Specialisation vs. silos

POSSIBLE SOLUTIONS
● Provide tools many groups can pick up
easily
○ Tensor libraries (ND4S, Jumpy)
○ Deep learning libraries (ScalNet)
● Empowering data scientists by bringing
their models to production
○ Model import

ND4J
● Performant and versatile Java tensor library
● Hardware acceleration (CPU & GPU)
● Roughly twice as fast as numpy
● Samediff: auto-differentiation module
● ND4S: Scala-wrapper, close to numpy
● Jumpy: ND4J for python, interoperates with
numpy

DEEPLEARNING4J
● General purpose DL library
● Parallel training and inference
● Powered by ND4J
● Keras model import
● ScalNet Scala wrapper, Keras lookalike

MODEL IMPORT
Use case: product recommendations
● Users buy and rate products
● Want to recommend products
● Input features are one-hot
encoded users and products
● labels are normalized ratings
● learn features
Users
Products
Users
Products

MODEL IMPORT: KERAS
0 … 0 1 0 … 0
Users
0 … 0 1 0 … 0
Products
Embeddings
Merge
Dense
Dense
Dense
Prediction

MODEL IMPORT
Store Keras model:
Load model and predict:

SCALNET
● Intuitive high-level API (Keras)
● Models compile into regular DL4J models
● Scala as “python for production”

OUTLOOK
● Use autodiff for jumpy to create a DL4J
Keras backend
● Feature parity with Keras (for model import)
● Make ScalNet first class member

Max Pumperla
Kevin Ferguson
Deep Learning &
The Game of Go
BOOKS

TAKE AWAYS
● Take production seriously and find ways to
work together
● Consider our wrappers to bridge the gap
● You can have both

max@skymind.io
twitter: maxpumperla
github: maxpumperla

This document discusses distributed deep learning with Keras and TensorFlow on Apache Spark. It begins with introductions and background on the speaker. It then provides overviews of deep learning, popular frameworks like TensorFlow and Keras, and challenges with distributed deep learning. The core topics cover what distributed deep learning on Spark is, why it is difficult, and how to implement it in Python using frameworks like Keras, TensorFlow and DeepLearning4J that integrate with Spark. It discusses techniques like data and model parallelism and provides examples of importing models between frameworks and retraining imported models on Spark. Finally, it discusses memory utilization and tools for visualization.

Distributed Deep Learning with Keras and TensorFlow on Apache SparkGuglielmo Iozzia

This document provides an overview of distributed deep learning with Keras/TensorFlow on Apache Spark. It discusses TensorFlow and Keras as deep learning frameworks and Apache Spark as a unified analytics engine. It then introduces DeepLearning4J (DL4J) as an open source distributed deep learning library for the JVM that is integrated with Hadoop and Spark. The document explains how to perform tasks like importing models, testing models, serializing models, and distributed training in DL4J on Spark. It also discusses challenges of training neural networks in Spark and shows screenshots of code examples using DL4J on Spark.

Why scala for data scienceGuglielmo Iozzia

This document discusses why Scala is a good programming language for data science. It begins by providing background on Scala as a functional programming language that runs on the Java Virtual Machine. The main reasons given for using Scala in data science are its robustness for large datasets, integration with common big data tools that run on JVM, and available libraries like Spark MLlib, DeepLearning4J, and ND4J. Code examples are provided showing how to perform tasks with these libraries in Scala. The document also discusses how Scala, Python, and Keras can be used together via TensorFlow for prototyping models in Python and deploying them in Scala applications using DeepLearning4J.

Building machine learning service in your business — Eric Chen (Uber) @PAPIs ...PAPIs.io

AKKA and Scala @ InneractiveGal Aviv

Akka & Scala were chosen as the concurrency framework and programming language respectively for an advertising company to optimize machine utilization and handle high throughput and low latency requirements. Akka provides an actor model for building concurrent and distributed applications, while Scala is a functional programming language that integrates well with Akka and Java. The architecture was refactored over time to use Akka actors and Spray REST services built on top of Akka for asynchronous request handling at scale.

Javantura v4 - Getting started with Apache Spark - Dinko SrkočHUJAK - Hrvatska udruga Java korisnika / Croatian Java User Association

This document provides an introduction to Apache Spark and Scala. It discusses that Apache Spark is a general purpose cluster computing system that is faster than Hadoop MapReduce, runs locally and in the cloud. It has high-level APIs for Scala, Python, Java and R. The document outlines Spark's core components including SQL, MLlib, GraphX and streaming. It describes Spark's main data collections of RDDs for unstructured data and DataFrames/Datasets for structured data. Finally, it provides an overview of demonstrations that will be covered including the Spark shell, notebook, streaming and deploying a mini project to Google Cloud Dataproc.

Accelerated Training of Transformer ModelsDatabricks

Language models help in automating a wide range of natural language processing (NLP) tasks such as speech recognition, machine translation, text summarization and more. Transformer architecture was introduced a few years back and it has significantly changed the NLP landscape since then. Transformer based models are getting bigger and better to improve the state of the art on language understanding and generation tasks.

Tweet4Beer (atualizada): Torneira de Chopp Controlada por Java, JavaFX, IoT ...Marco Antonio Maciel

TensorFlowOnSpark Enhanced: Scala, Pipelines, and Beyond with Lee Yang and An...Databricks

TensorFlowOnSpark (TFoS) was open sourced in Q1 2017, and it has gained strong adoption within the Spark community for running TensorFlow training and inferencing jobs on Spark clusters. At Spark Summit 2017, we explained how TFoS enables Python applications to conduct distributed TensorFlow training and inference efficiently by leveraging key built-in capabilities of PySpark and TensorFlow. In this talk, we cover the major enhancements of TFoS in recent months. We will introduce a new Scala API for users who want to integrate previously trained models into an existing Scala/Spark workflow. We will describe a new Python API for Spark ML pipelines to train all types of TensorFlow models, and conduct inference/featurization without any custom code. Additionally, we will cover the support for TensorFlow Keras API, and TensorFlow Datasets.

Why & Where Knoldus Uses Rust?Knoldus Inc.

Memory bugs are difficult to catch since it’s not an easy job writing tests to detect them. If these bugs are not found out during the beta phase, they can remain in your code for years and pop up at any time causing havoc. When it comes to Systems programming language, Rust is ahead of C++ as it was created with safer memory management in mind. This helps developers to build software which is less prone to bugs & security breaches. Also, Rust helps build really fast software applications. When it comes to building embedded electronics, IoT applications, operating systems, device drivers, or game engines, Rust is your go-to language. It's because of its advantages and use cases that Knoldus also chooses the Rust programming language. In our next webinar, our experts will walk you through the advantages of using the Rust programming language. Also, there are a number of projects that we have built on Rust in the areas of Command Line Interface, Networking, and Embedded programming that we will cover during the webinar.

Apache Spark MLlib 2.0 Preview: Data Science and ProductionDatabricks

This talk highlights major improvements in Machine Learning (ML) targeted for Apache Spark 2.0. The MLlib 2.0 release focuses on ease of use for data science—both for casual and power users. We will discuss 3 key improvements: persisting models for production, customizing Pipelines, and improvements to models and APIs critical to data science. (1) MLlib simplifies moving ML models to production by adding full support for model and Pipeline persistence. Individual models—and entire Pipelines including feature transformations—can be built on one Spark deployment, saved, and loaded onto other Spark deployments for production and serving. (2) Users will find it much easier to implement custom feature transformers and models. Abstractions automatically handle input schema validation, as well as persistence for saving and loading models. (3) For statisticians and data scientists, MLlib has doubled down on Generalized Linear Models (GLMs), which are key algorithms for many use cases. MLlib now supports more GLM families and link functions, handles corner cases more gracefully, and provides more model statistics. Also, expanded language APIs allow data scientists using Python and R to call many more algorithms. Finally, we will demonstrate these improvements live and show how they facilitate getting started with ML on Spark, customizing implementations, and moving to production.

Intuitive & Scalable Hyperparameter Tuning with Apache Spark + FugueDatabricks

This presentation introduces Tune and Fugue, frameworks for intuitive and scalable hyperparameter optimization (HPO). Tune supports both non-iterative and iterative HPO problems. For non-iterative problems, Tune supports grid search, random search, and Bayesian optimization. For iterative problems, Tune generalizes algorithms like Hyperband and Asynchronous Successive Halving. Tune allows tuning models both locally and in a distributed manner without code changes. The presentation demonstrates Tune's capabilities through examples tuning Scikit-Learn and Keras models. The goal of Tune and Fugue is to make HPO development easy, testable, and scalable.

Micro-Benchmarking Considered HarmfulThomas Wuerthinger

Measuring the time spent on small individual fractions of program code is a common technique for analysing performance behavior and detecting performance bottlenecks. The benefits of the approach include a detailed individual attribution of performance and understandable feedback loops when experimenting with different code versions. There are however severe pitfalls when following this approach that can lead to vastly misleading results. Modern dynamic compilers use complex optimisation techniques that take a large part of the program into account. There can be therefore unexpected side-effects when combining different code snippets or even when running a presumably unrelated part of the code. This talk will present performance paradoxes with examples from the domain of dynamic compilation of Java programs. Furthermore, it will discuss an alternative approach to modelling code performance characteristics that takes the challenges of complex optimising compilers into account.

Running R on AWS Lambda by Ana-Maria NiculescuParis Women in Machine Learning and Data Science

This document discusses running R code on Amazon Lambda. It presents a solution using custom R runtimes provided by Bakdata to deploy R packages and functions to Lambda. Shell scripts automate the deployment process, creating the necessary AWS infrastructure including VPC, S3, API Gateway and Lambda function. A simple example R function is shown running on Lambda. While Lambda has limitations like memory and timeout restrictions, it is suitable for running modularized R code on an as-needed basis without managing servers.

MLlib sparkmeetup_8_6_13_final_reducedChao Chen

The document describes the MLbase stack, which aims to make machine learning development and use easier and more scalable. It discusses three levels of the stack: MLlib for low-level ML functionality, MLI which provides higher-level abstractions and feature extraction tools, and ML Optimizer which automates model selection. Examples are given showing how text classification can be done more easily using MLI compared to directly with MLlib. The goal is to release initial functionality in the summer and expand it over the following winter.

Scalablity and benchmark in mysql performanceAmrendra Kumar

The document discusses MySQL performance, scalability, and benchmarks. It emphasizes the importance of monitoring MySQL performance and understanding workload limits. It also discusses how to test the scalability of a single MySQL instance by increasing hardware resources and measuring throughput. Finally, it provides examples of common benchmarking tools used to test MySQL performance, such as Sysbench, DBT2, and iiBench, and benchmarks showing over 1.8 million queries per second for point selects on MySQL 5.7.

Rapids erase the waiting hassle by Andrada OlteanuParis Women in Machine Learning and Data Science

RAPIDS is a suite of open-source software libraries that allows users to execute end-to-end data science and analytics pipelines on GPUs. It includes libraries like cudf, cupy, and cuml that are analogous to popular Python data science libraries like pandas, numpy, and scikit-learn, but are optimized to run calculations on GPUs. This allows for faster performance for large-scale tasks like big data, machine learning, and image processing compared to CPUs.

Backup and Recovery in MySQL Clusterpriyanka_sangam

OpenCV for Embedded: Lessons LearnedYury Gorbachev

This document discusses lessons learned from using OpenCV for embedded vision applications. It notes that while OpenCV works well out of the box for desktop applications, embedded platforms present additional challenges like different processors, interfaces, and unpredictable performance. It recommends prototyping on desktop for faster development, then optimizing algorithms and porting to embedded hardware. Specific optimizations discussed include using ARMv8 processors, vendor-optimized OpenCV packages, and custom NEON-accelerated functions. An example product from Itseez that runs computer vision algorithms in real-time on ARM using these techniques is also presented.

IBM Runtimes Performance Observations with Apache SparkAdamRobertsIBM

In this talk presented at the Spark London meetup on the 23rd of November 2016 I have detailed our findings in IBM's Runtime Technologies department around Apache Spark. I share best practices we observed by profiling Spark on a variety of workloads I have covered and help Spark users to profile their own applications. I've also touched on how anybody can develop using fast networking capabilities (RDMA) and can achieve substantial performance speedups using GPUs.

Neo4j tms_mdev_

High Performance Continuous Delivery - Versioning and Release Management AlignedPerforce

EclairJS = Node.Js + Apache SparkJen Aman

EclairJS allows developers to use JavaScript and Node.js to interact with Apache Spark for large-scale data processing and analytics. It provides a Spark API for Node.js so that compute-intensive workloads can be handed off to Spark running in the backend. EclairJS also enables the use of JavaScript with Jupyter notebooks, so data engineers and web developers can experiment with Spark from within the browser using familiar JavaScript syntax.

ROCm and Distributed Deep Learning on Spark and TensorFlowDatabricks

ROCm, the Radeon Open Ecosystem, is an open-source software foundation for GPU computing on Linux. ROCm supports TensorFlow and PyTorch using MIOpen, a library of highly optimized GPU routines for deep learning. In this talk, we describe how Apache Spark is a key enabling platform for distributed deep learning on ROCm, as it enables different deep learning frameworks to be embedded in Spark workflows in a secure end-to-end machine learning pipeline. We will analyse the different frameworks for integrating Spark with Tensorflow on ROCm, from Horovod to HopsML to Databrick's Project Hydrogen. We will also examine the surprising places where bottlenecks can surface when training models (everything from object stores to the Data Scientists themselves), and we will investigate ways to get around these bottlenecks. The talk will include a live demonstration of training and inference for a Tensorflow application embedded in a Spark pipeline written in a Jupyter notebook on Hopsworks with ROCm.

アドテク×Scala @DynalystSangwon Han

This document summarizes a presentation about building and monitoring applications with Scala, Spray, and Akka. It discusses using Spray's FutureDirectives to handle asynchronous requests non-blockingly. It recommends separating applications into layers with different dispatchers and implementing services abstractly using type classes and implicits to allow for different execution contexts in testing. It also covers using Akka's master-worker pattern and monitoring tools like Typesafe Console and Kamon.

RESTful Machine Learning with Flask and TensorFlow Serving - Carlo MazzaferroPyData

Big data analysing genomics and the bdg projectsree navya

This document discusses a project analyzing genomics using big data techniques. It introduces Spark, a framework for large-scale data analysis, and ADAM, a Spark-based framework for genomic data. Key points covered include an overview of Spark and Hadoop, data types and formats in genomics, and using ADAM and Spark to perform genomic analysis and queries on large datasets like the 1000 Genomes Project.

Advanced MLflow: Multi-Step Workflows, Hyperparameter Tuning and Integrating ...Databricks

Because MLflow is an API-first platform, there are many patterns for using it in complex workflows and integrating it with existing tools. In this talk, we’ll demo a few best practices for using MLflow in a more complex workflow. These include: * Run multi-step workflows on MLflow, such as data preparation steps followed by training, and organizing your projects so you can automatically reuse past work. * Tune Hyperparameter on MLflow with open source hyperparameter tuning packages. * Save a model in MLflow (eg, from a new machine learning library) and deploying it to the existing deployment tools.

Advanced deeplearning4j featuresAdam Gibson

Guglielmo iozzia - Google I/O extended dublin 2018Guglielmo Iozzia

This document discusses deep learning and options for implementing it on the Java Virtual Machine (JVM). It introduces DeepLearning4J (DL4J) as an open source deep learning library for the JVM. It also discusses TensorFlow and Keras, noting that while TensorFlow is mostly Python-based, Keras models can be imported into DL4J. An example is provided of using DataVec to transform data and DL4J to build and train a convolutional neural network model on Spark. The document encourages exploring existing neural network models online and moving deep learning implementations into production.

More Related Content

What's hot (20)

TensorFlowOnSpark Enhanced: Scala, Pipelines, and Beyond with Lee Yang and An...Databricks

Why & Where Knoldus Uses Rust?Knoldus Inc.

Apache Spark MLlib 2.0 Preview: Data Science and ProductionDatabricks

Intuitive & Scalable Hyperparameter Tuning with Apache Spark + FugueDatabricks

Micro-Benchmarking Considered HarmfulThomas Wuerthinger

Running R on AWS Lambda by Ana-Maria NiculescuParis Women in Machine Learning and Data Science

MLlib sparkmeetup_8_6_13_final_reducedChao Chen

Scalablity and benchmark in mysql performanceAmrendra Kumar

Rapids erase the waiting hassle by Andrada OlteanuParis Women in Machine Learning and Data Science

Backup and Recovery in MySQL Clusterpriyanka_sangam

OpenCV for Embedded: Lessons LearnedYury Gorbachev

IBM Runtimes Performance Observations with Apache SparkAdamRobertsIBM

Neo4j tms_mdev_

High Performance Continuous Delivery - Versioning and Release Management AlignedPerforce

EclairJS = Node.Js + Apache SparkJen Aman

ROCm and Distributed Deep Learning on Spark and TensorFlowDatabricks

アドテク×Scala @DynalystSangwon Han

RESTful Machine Learning with Flask and TensorFlow Serving - Carlo MazzaferroPyData

Big data analysing genomics and the bdg projectsree navya

Advanced MLflow: Multi-Step Workflows, Hyperparameter Tuning and Integrating ...Databricks

TensorFlowOnSpark Enhanced: Scala, Pipelines, and Beyond with Lee Yang and An...Databricks

Why & Where Knoldus Uses Rust?Knoldus Inc.

Apache Spark MLlib 2.0 Preview: Data Science and ProductionDatabricks

Intuitive & Scalable Hyperparameter Tuning with Apache Spark + FugueDatabricks

Micro-Benchmarking Considered HarmfulThomas Wuerthinger

Running R on AWS Lambda by Ana-Maria NiculescuParis Women in Machine Learning and Data Science

MLlib sparkmeetup_8_6_13_final_reducedChao Chen

Scalablity and benchmark in mysql performanceAmrendra Kumar

Rapids erase the waiting hassle by Andrada OlteanuParis Women in Machine Learning and Data Science

Backup and Recovery in MySQL Clusterpriyanka_sangam

OpenCV for Embedded: Lessons LearnedYury Gorbachev

IBM Runtimes Performance Observations with Apache SparkAdamRobertsIBM

Neo4j tms_mdev_

High Performance Continuous Delivery - Versioning and Release Management AlignedPerforce

EclairJS = Node.Js + Apache SparkJen Aman

ROCm and Distributed Deep Learning on Spark and TensorFlowDatabricks

アドテク×Scala @DynalystSangwon Han

RESTful Machine Learning with Flask and TensorFlow Serving - Carlo MazzaferroPyData

Big data analysing genomics and the bdg projectsree navya

Advanced MLflow: Multi-Step Workflows, Hyperparameter Tuning and Integrating ...Databricks

Similar to From Experimentation to Production - Scala & Python APIs for DL4J (20)

Advanced deeplearning4j featuresAdam Gibson

Guglielmo iozzia - Google I/O extended dublin 2018Guglielmo Iozzia

Running deep neural nets in your Java application with Deeplearning4jAlexander Fedintsev

Bringing Deep Learning into production Paolo Platter

- The document discusses deep learning frameworks and how to choose one for a given environment. It summarizes the strengths, weaknesses, opportunities and threats of popular frameworks like TensorFlow, Theano, Torch, Caffe, DeepLearning4J and H2O. - It recommends H2O as a good choice for enterprise environments due to its ease of use, scalability on big data, integration with Spark, Java/Scala support and commercial support. DeepLearning4J is also recommended for more advanced deep neural networks and multi-dimensional arrays. - The document proposes using Spark as a middleware to leverage multiple frameworks and avoid vendor lock-in, and describes Agile Lab's recommended stack for enterprises which combines H

Building Deep Learning Workflows with DL4JJosh Patterson

In this session we will take a look at a practical review of what is deep learning and introduce DL4J. We’ll look at how it supports deep learning in the enterprise on the JVM. We’ll discuss the architecture of DL4J’s scale-out parallelization on Hadoop and Spark in support of modern machine learning workflows. We’ll conclude with a workflow example from the command line interface that shows the vectorization pipeline in Canova producing vectors for DL4J’s command line interface to build deep learning models easily.

Build, Scale, and Deploy Deep Learning Pipelines with Ease Using Apache SparkDatabricks

Deep Learning has shown a tremendous success, yet it often requires a lot of effort to leverage its power. Existing Deep Learning frameworks require writing a lot of code to work with a model, let alone in a distributed manner. In this talk, we’ll survey the state of Deep Learning at scale, and where we introduce the Deep Learning Pipelines, a new open-source package for Apache Spark. This package simplifies Deep Learning in three major ways: • It has a simple API that integrates well with enterprise Machine Learning pipelines. • It automatically scales out common Deep Learning patterns, thanks to Spark. • It enables exposing Deep Learning models through the familiar Spark APIs, such as MLlib and Spark SQL. In this talk, we will look at a complex problem of image classification, using Deep Learning and Spark. Using Deep Learning Pipelines, we will show: • how to build deep learning models in a few lines of code; • how to scale common tasks like transfer learning and prediction; and • how to publish models in Spark SQL.

Suneel Marthi - Deep Learning with Apache Flink and DL4JFlink Forward

http://flink-forward.org/kb_sessions/deep-learning-with-apache-flink-and-dl4j/ Deep Learning has become very popular over the last few years in areas such as Image Recognition, Fraud Detection, Machine Translation etc. Deep Learning has proved to be very useful in handling unstructured data and extracting value from them. A big challenge with having to build deep learning models was the high cost of training them. With the recent advent of distributed frameworks like Apache Flink, Apache Spark etc.. it’s faster to train Deep Learning models in parallel on modern platform architecture. In this talk, we’ll be showing how to use Apache Flink Streaming with the open source Deep Learning framework, DeepLearning4j to perform large scale deep learning model training. We will show a demo of a Recurrent Neural Net that is trained for language modeling and have it generate text.

Intro to DeepLearning4J on ApacheSpark SDS DL Workshop 16Romeo Kienzler

AI and Spark - IBM Community AI DayNick Pentreath

Apache Spark’s machine learning library provides a simple, elegant, yet powerful framework for creating scalable machine learning pipelines. It provides out of the box components for feature extraction and transformation, as well as various machine learning algorithms. However, in recent years specialized systems (such as TensorFlow, Caffe, PyTorch and Apache MXNet) have been dominant in the domain of AI and deep learning, as they allow greater performance and flexibility for training complex models. While there are a few deep learning frameworks that are Spark specific, in most cases these frameworks are separate from Spark and the ease of integration and feature set exposed varies considerably. This session will explore the role of Spark within the AI landscape, the current state of deep learning on top of Spark and the most recent developments in the Spark project to better integrate Spark with the deep learning ecosystem.

Training Week: Build APIs with Neo4j GraphQL LibraryNeo4j

Which Is Deeper - Comparison Of Deep Learning Frameworks On SparkSpark Summit

This document compares several deep learning frameworks that run on Apache Spark, including SparkNet, Deeplearning4J, CaffeOnSpark, and Tensorflow on Spark. It outlines the theoretical principles behind data parallelism for distributed stochastic gradient descent. It then evaluates and benchmarks each framework based on criteria like ease of use, functionality, performance, and community support. SparkNet, CaffeOnSpark, and Tensorflow on Spark are shown to have stronger communities and support from organizations. The document concludes that while these frameworks currently lack model parallelism and could experience network congestion, integrating GPUs and improving scalability are areas for future work.

Transitioning from Java to Scala for Spark - March 13, 2019Gravy Analytics

Gravy Analytics ingests ~17 billion records daily of data and improve and refine that data into many data products at various levels of aggregation. To meet the challenges of our product requirements and scale we constantly evaluate new technologies. Spark has become central to our ability to process ever increasing amounts of data through our data factory. In late 2017 and throughout 2018, we have improved our ability to work with Spark by migrating all Spark jobs to Scala. In this discussion, we’ll cover areas which were more difficult from a Spark perspective to develop in Java than Scala as well as some of the challenges we met along the way.

Machine Learning with ScalaSusan Eraly

This document discusses machine learning concepts like neural networks, deep learning, and recurrent neural networks. It introduces Deeplearning4j, an open-source deep learning library for Java and Scala, and its commercial support arm Skymind. Specific machine learning applications mentioned include optical character recognition, Netflix recommendations, topic modeling, sentiment analysis, machine translation, and question answering. Parallelization techniques for model training like stochastic gradient descent are also covered.

Smart Data Conference: DL4J and DataVecJosh Patterson

This document discusses using DL4J and DataVec to build production-ready deep learning workflows for time series and text data. It provides an example of modeling sensor data with recurrent neural networks (RNNs) and character-level text generation with LSTMs. Key points include: - DL4J is a deep learning framework for Java that runs on Spark and supports CPU/GPU. DataVec is a tool for data preprocessing. - The document demonstrates loading and transforming sensor time series data with DataVec and training an RNN on the data with DL4J. - It also shows vectorizing character-level text data from beer reviews with DataVec and using an LSTM in DL4J to generate new

BigDL webinar - Deep Learning Library for SparkDESMOND YUEN

Deep learning with DL4J - Hadoop Summit 2015Josh Patterson

This document discusses deep learning and DL4J. It begins with an overview of deep learning, describing it as automated feature engineering through chained techniques like restricted Boltzmann machines. It then introduces DL4J, describing it as an enterprise-grade Java implementation of deep learning that supports parallelization on Hadoop, Spark, and GPUs. The rest of the document discusses building deep learning workflows with DL4J and related tools like Canova and Arbiter, providing an example of vectorizing and modeling iris data from a CSV file on the command line.

Enterprise Deep Learning with DL4JJosh Patterson

Josh Patterson presented on deep learning and DL4J. He began with an overview of deep learning, explaining it as automated feature engineering where machines learn representations of the world. He then discussed DL4J, describing it as the "Hadoop of deep learning" - an open source deep learning library with Java, Scala, and Python APIs that supports parallelization on Hadoop, Spark, and GPUs. He demonstrated building deep learning workflows with DL4J and Canova, using the Iris dataset as an example to show how data can be vectorized with Canova and then a model trained on it using DL4J from the command line. He concluded by describing Skymind as a distribution of DL4J with enterprise

Deep Learning with DL4J on Apache Spark: Yeah it's Cool, but are You Doing it...DataWorks Summit

DeepLearning4J (DL4J) is a powerful Open Source distributed framework that brings Deep Learning to the JVM (it can serve as a DIY tool for Java, Scala, Clojure and Kotlin programmers). It can be used on distributed GPUs and CPUs. It is integrated with Hadoop and Apache Spark. ND4J is a Open Source, distributed and GPU-enabled library that brings the intuitive scientific computing tools of the Python community to the JVM. Training neural network models using DL4J, ND4J and Spark is a powerful combination, but the overall cluster configuration can present some unespected issues that can compromise performances and nullify the benefits of well written code and good model design. In this talk I will walk through some of those problems and will present some best practices to prevent them. The presented use cases will refer to DL4J and ND4J on different Spark deployment modes (standalone, YARN, Kubernetes). The reference programming language for any code example would be Scala, but no preliminary Scala knowledge is mandatory in order to better understanding the presented topics.

Deep learning and Apache SparkQuantUniversity

Interest is growing in the Apache Spark community in using Deep Learning techniques and in the Deep Learning community in scaling algorithms with Apache Spark. A few of them to note include: · Databrick’s efforts in scaling Deep learning with Spark · Intel announcing the BigDL: A Deep learning library for Spark · Yahoo’s recent efforts to opensource TensorFlowOnSpark In this lecture we will discuss the key use cases and developments that have emerged in the last year in using Deep Learning techniques with Spark.

Josh Patterson, Advisor, Skymind – Deep learning for Industry at MLconf ATL 2016MLconf

This document discusses using DL4J and DataVec to build deep learning workflows for modeling time series sensor data with recurrent neural networks. It provides an example of loading and transforming time series data from sensors using DataVec, configuring an RNN using DL4J to classify the trends in the sensor data, and training the network both locally and distributed on Spark. The document promotes DL4J and DataVec as tools that can help enterprises overcome challenges to operationalizing deep learning and producing machine learning models at scale.

Advanced deeplearning4j featuresAdam Gibson

Guglielmo iozzia - Google I/O extended dublin 2018Guglielmo Iozzia

Running deep neural nets in your Java application with Deeplearning4jAlexander Fedintsev

Bringing Deep Learning into production Paolo Platter

Building Deep Learning Workflows with DL4JJosh Patterson

Build, Scale, and Deploy Deep Learning Pipelines with Ease Using Apache SparkDatabricks

Suneel Marthi - Deep Learning with Apache Flink and DL4JFlink Forward

Intro to DeepLearning4J on ApacheSpark SDS DL Workshop 16Romeo Kienzler

AI and Spark - IBM Community AI DayNick Pentreath

Training Week: Build APIs with Neo4j GraphQL LibraryNeo4j

Which Is Deeper - Comparison Of Deep Learning Frameworks On SparkSpark Summit

Transitioning from Java to Scala for Spark - March 13, 2019Gravy Analytics

Machine Learning with ScalaSusan Eraly

Smart Data Conference: DL4J and DataVecJosh Patterson

BigDL webinar - Deep Learning Library for SparkDESMOND YUEN

Deep learning with DL4J - Hadoop Summit 2015Josh Patterson

Enterprise Deep Learning with DL4JJosh Patterson

Deep Learning with DL4J on Apache Spark: Yeah it's Cool, but are You Doing it...DataWorks Summit

Deep learning and Apache SparkQuantUniversity

Josh Patterson, Advisor, Skymind – Deep learning for Industry at MLconf ATL 2016MLconf

More from Max Pumperla (7)

Data science in practiceMax Pumperla

Snakes on a plane - Ship your Python on enterprise machinesMax Pumperla

Data scientists want Python for experimentation, engineers want production-gradesystems. This can create friction between departments and often leads to suboptimal solutions. In this talk we show how to access Deeplearning4J (DL4J) directly from Python, and discuss how to import some of your favorite frameworks into DL4J. This approach narrows the gap between science and engineering and brings Deep Learning models to production more easily. We close by giving a demo of real-time object detection with YOLO, using Skymind's intelligence layer (SKIL).

Bridging the gap in enterprise AIMax Pumperla

Hitchhiker's guide to the kerasverseMax Pumperla

“Other hitchhikers had seen to modify their towels in exotic ways, weaving all kinds of esoteric tools and utilities and even computer equipment into the fabric.” - Douglas Adams Keras has established itself as one of the most popular and accessible Deep Learning frameworks in the world. It is widely used among practitioners and has a buzzing community. On top of that, over the last years many powerful third party extensions have been built. In this talk we'll showcase some of these extensions, with a focus on tools that ease and enhance your data science workflow.

Deep Recommender systems - Shibsted, OsloMax Pumperla

The document discusses recommender systems and how deep learning techniques can be applied. It provides an overview of traditional recommender approaches like content-based filtering, collaborative filtering, and hybrid systems. It then outlines how deep learning methods such as neural networks, item embeddings, deep matrix factorization, and session-based models can be used to power recommender systems. The document emphasizes starting simple, experimenting with deep learning, knowing the optimization goal, having strong evaluation practices, and bringing in experts to help build and scale systems.

GTC Europe 2017 - How to predict ICU mortality with digital health dataMax Pumperla

In this presentation, we will discuss modeling electronic health record (EHR) data with deep learning and Deeplearning4j (DL4J). We draw inspiration from recent research showing that carefully designed neural network architectures can learn effectively from the complex, messy data collected in EHRs. Specifically, we describe how to train a long short-term memory recurrent neural network (LSTM RNN) to predict in-hospital mortality among patients hospitalized in the intensive care unit (ICU). Of particular note, our results show that even for a dataset of moderate size, the LSTM is competitive with alternative approaches, including decision trees and multilayer perceptrons, using hand-engineering features. We will also show how to parallelize model training on a Spark cluster. Finally, we will highlight potential extensions of this work and other use cases for EHR data and deep learning. All code and data are publicly available, so that attendees may reproduce our work.

EclipseCon 2017 - Introduction to Machine Learning with Eclipse Deeplearning4jMax Pumperla

In this talk we give an overview of the Eclipse Deeplearning4J ecosystem. Skymind builds open source machine learning tools for the JVM and with a focus on deep learning. We give a quick introduction into the basics of deep learning and discuss a practical use case for illustration. Among other libraries, we introduce ND4J for powerful tensor operations and DL4J for building and deploying deep neural networks.

Data science in practiceMax Pumperla

Snakes on a plane - Ship your Python on enterprise machinesMax Pumperla

Bridging the gap in enterprise AIMax Pumperla

Hitchhiker's guide to the kerasverseMax Pumperla

Deep Recommender systems - Shibsted, OsloMax Pumperla

GTC Europe 2017 - How to predict ICU mortality with digital health dataMax Pumperla

EclipseCon 2017 - Introduction to Machine Learning with Eclipse Deeplearning4jMax Pumperla

Recently uploaded (20)

Manifest Pre-Seed Update | A Humanoid OEM Deeptech In Francechb3

Build 3D Animated Safety Induction - Tech EHSTECH EHS Solution

Train Smarter, Not Harder – Let 3D Animation Lead the Way! Discover how 3D animation makes inductions more engaging, effective, and cost-efficient. Check out the slides to see how you can transform your safety training process! Slide 1: Why 3D animation changes the game Slide 2: Site-specific induction isn’t optional—it’s essential Slide 3: Visitors are most at risk. Keep them safe Slide 4: Videos beat text—especially when safety is on the line Slide 5: TechEHS makes safety engaging and consistent Slide 6: Better retention, lower costs, safer sites Slide 7: Ready to elevate your induction process? Can an animated video make a difference to your site's safety? Let's talk.

UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager APIUiPathCommunity

Join this UiPath Community Berlin meetup to explore the Orchestrator API, Swagger interface, and the Test Manager API. Learn how to leverage these tools to streamline automation, enhance testing, and integrate more efficiently with UiPath. Perfect for developers, testers, and automation enthusiasts! 📕 Agenda Welcome & Introductions Orchestrator API Overview Exploring the Swagger Interface Test Manager API Highlights Streamlining Automation & Testing with APIs (Demo) Q&A and Open Discussion Perfect for developers, testers, and automation enthusiasts! 👉 Join our UiPath Community Berlin chapter: https://community.uipath.com/berlin/ This session streamed live on April 29, 2025, 18:00 CET. Check out all our upcoming UiPath Community sessions at https://community.uipath.com/events/.

AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...Alan Dix

Talk at the final event of Data Fusion Dynamics: A Collaborative UK-Saudi Initiative in Cybersecurity and Artificial Intelligence funded by the British Council UK-Saudi Challenge Fund 2024, Cardiff Metropolitan University, 29th April 2025 https://alandix.com/academic/talks/CMet2025-AI-Changes-Everything/ Is AI just another technology, or does it fundamentally change the way we live and think? Every technology has a direct impact with micro-ethical consequences, some good, some bad. However more profound are the ways in which some technologies reshape the very fabric of society with macro-ethical impacts. The invention of the stirrup revolutionised mounted combat, but as a side effect gave rise to the feudal system, which still shapes politics today. The internal combustion engine offers personal freedom and creates pollution, but has also transformed the nature of urban planning and international trade. When we look at AI the micro-ethical issues, such as bias, are most obvious, but the macro-ethical challenges may be greater. At a micro-ethical level AI has the potential to deepen social, ethnic and gender bias, issues I have warned about since the early 1990s! It is also being used increasingly on the battlefield. However, it also offers amazing opportunities in health and educations, as the recent Nobel prizes for the developers of AlphaFold illustrate. More radically, the need to encode ethics acts as a mirror to surface essential ethical problems and conflicts. At the macro-ethical level, by the early 2000s digital technology had already begun to undermine sovereignty (e.g. gambling), market economics (through network effects and emergent monopolies), and the very meaning of money. Modern AI is the child of big data, big computation and ultimately big business, intensifying the inherent tendency of digital technology to concentrate power. AI is already unravelling the fundamentals of the social, political and economic world around us, but this is a world that needs radical reimagining to overcome the global environmental and human challenges that confront us. Our challenge is whether to let the threads fall as they may, or to use them to weave a better future.

tecnologias de las primeras civilizaciones.pdffjgm517

TrsLabs - Fintech Product & Business ConsultingTrs Labs

Hybrid Growth Mandate Model with TrsLabs Strategic Investments, Inorganic Growth, Business Model Pivoting are critical activities that business don't do/change everyday. In cases like this, it may benefit your business to choose a temporary external consultant. An unbiased plan driven by clearcut deliverables, market dynamics and without the influence of your internal office equations empower business leaders to make right choices. Getting things done within a budget within a timeframe is key to Growing Business - No matter whether you are a start-up or a big company Talk to us & Unlock the competitive advantage

Mastering Advance Window Functions in SQL.pdfSpiral Mantra

Build Your Own Copilot & Agents For DevsBrian McKeiver

Generative Artificial Intelligence (GenAI) in BusinessDr. Tathagat Varma

Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...Aqusag Technologies

How analogue intelligence complements AIPaul Rowe

Artificial Intelligence is providing benefits in many areas of work within the heritage sector, from image analysis, to ideas generation, and new research tools. However, it is more critical than ever for people, with analogue intelligence, to ensure the integrity and ethical use of AI. Including real people can improve the use of AI by identifying potential biases, cross-checking results, refining workflows, and providing contextual relevance to AI-driven results. News about the impact of AI often paints a rosy picture. In practice, there are many potential pitfalls. This presentation discusses these issues and looks at the role of analogue intelligence and analogue interfaces in providing the best results to our audiences. How do we deal with factually incorrect results? How do we get content generated that better reflects the diversity of our communities? What roles are there for physical, in-person experiences in the digital world?

Big Data Analytics Quick Research Guide by Arthur MorganArthur Morgan

This is a Quick Research Guide (QRG). QRGs include the following: - A brief, high-level overview of the QRG topic. - A milestone timeline for the QRG topic. - Links to various free online resource materials to provide a deeper dive into the QRG topic. - Conclusion and a recommendation for at least two books available in the SJPL system on the QRG topic. QRGs planned for the series: - Artificial Intelligence QRG - Quantum Computing QRG - Big Data Analytics QRG - Spacecraft Guidance, Navigation & Control QRG (coming 2026) - UK Home Computing & The Birth of ARM QRG (coming 2027) Any questions or comments? - Please contact Arthur Morgan at [email protected]. 100% human made.

Greenhouse_Monitoring_Presentation.pptx.hpbmnnxrvb

Cybersecurity Identity and Access Solutions using Azure ADVICTOR MAESTRE RAMIREZ

TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...TrustArc

Most consumers believe they’re making informed decisions about their personal data—adjusting privacy settings, blocking trackers, and opting out where they can. However, our new research reveals that while awareness is high, taking meaningful action is still lacking. On the corporate side, many organizations report strong policies for managing third-party data and consumer consent yet fall short when it comes to consistency, accountability and transparency. This session will explore the research findings from TrustArc’s Privacy Pulse Survey, examining consumer attitudes toward personal data collection and practical suggestions for corporate practices around purchasing third-party data. Attendees will learn: - Consumer awareness around data brokers and what consumers are doing to limit data collection - How businesses assess third-party vendors and their consent management operations - Where business preparedness needs improvement - What these trends mean for the future of privacy governance and public trust This discussion is essential for privacy, risk, and compliance professionals who want to ground their strategies in current data and prepare for what’s next in the privacy landscape.

Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...BookNet Canada

Book industry standards are evolving rapidly. In the first part of this session, we’ll share an overview of key developments from 2024 and the early months of 2025. Then, BookNet’s resident standards expert, Tom Richardson, and CEO, Lauren Stewart, have a forward-looking conversation about what’s next. Link to recording, presentation slides, and accompanying resource: https://bnctechforum.ca/sessions/standardsgoals-for-2025-standards-certification-roundup/ Presented by BookNet Canada on May 6, 2025 with support from the Department of Canadian Heritage.

Are Cloud PBX Providers in India Reliable for Small Businesses (1).pdfTelecoms Supermarket

Electronic_Mail_Attacks-1-35.pdf by xploitniftliyevhuseyn

Procurement Insights Cost To Value Guide.pptxJon Hansen

Top 10 IT Help Desk Outsourcing ServicesInfrassist Technologies Pvt. Ltd.

Manifest Pre-Seed Update | A Humanoid OEM Deeptech In Francechb3

Build 3D Animated Safety Induction - Tech EHSTECH EHS Solution

UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager APIUiPathCommunity

AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...Alan Dix

tecnologias de las primeras civilizaciones.pdffjgm517

TrsLabs - Fintech Product & Business ConsultingTrs Labs

Mastering Advance Window Functions in SQL.pdfSpiral Mantra

Build Your Own Copilot & Agents For DevsBrian McKeiver

Generative Artificial Intelligence (GenAI) in BusinessDr. Tathagat Varma

Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...Aqusag Technologies

How analogue intelligence complements AIPaul Rowe

Big Data Analytics Quick Research Guide by Arthur MorganArthur Morgan

Greenhouse_Monitoring_Presentation.pptx.hpbmnnxrvb

Cybersecurity Identity and Access Solutions using Azure ADVICTOR MAESTRE RAMIREZ

TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...TrustArc

Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...BookNet Canada

Are Cloud PBX Providers in India Reliable for Small Businesses (1).pdfTelecoms Supermarket

Electronic_Mail_Attacks-1-35.pdf by xploitniftliyevhuseyn

Procurement Insights Cost To Value Guide.pptxJon Hansen

Top 10 IT Help Desk Outsourcing ServicesInfrassist Technologies Pvt. Ltd.

From Experimentation to Production - Scala & Python APIs for DL4J

2. FROM EXPERIMENTATION TO PRODUCTION Scala & Python APIs for DL4J

3. AGENDA ● About Skymind ● Experimentation vs production ● ND4S: Tensors for Scala ● Jumpy: Numpy for the JVM ● Model import for DL4J ● ScalNet: Scala API for DL4J

4. Founded 2014 Clients 14 Enterprises 3,500 GH Forks, 7,200 Stars 300,000+ DL4J downloads/mo. Team ~35; mostly engineers; 7 PhDs COMPANY OVERVIEW

5. Deeplearning4j Build, train, and deploy neural networks on JVM DataVec Data ingestion, normalization, and vectorization RL4J Reinforcement learning for the JVM SKYMIND TOOLS ND4J High performance linear algebra CPU and GPU libraries Arbiter Hyperparameter search for optimizing neural networks

6. OUR DEEP LEARNING STACK

7. Deeplearning4j Build, train, and deploy neural networks on JVM Model import Import of models from Keras (Tensorflow, Theano, CNTK, Caffe) ScalNet Scala API for Deeplearning4J SCALA & PYTHON APIS ND4J High performance linear algebra CPU and GPU libraries Jumpy Python interface for ND4J: a numpy wrapper for the JVM ND4S Scala wrapper for ND4J

8. EXPERIMENTATION VS PRODUCTION

9. EXPERIMENTATION

10. EXPERIMENTATION REQUIREMENTS ● Open source (preferably python) ● Insightful evaluation & debugging ● Fast experimentation cycle ● Reproducibility ● Collaboration

11. PRODUCTION Proper ETL pipes for prod Real-time data preparation Model serving Low latency predictions Model history Model & prediction history Parallel trainingStreaming data Distributed storage Model updates Various data sources

12. PRODUCTION

13. PRODUCTION REQUIREMENTS ● Open-source (Linux, Hadoop) ● Integrates With Existing Tech (JVM) ● Cross-Team Solution (DevOps, DS) ● Scalable, Containerized, Fast ● High availability ● General-Purpose, Customizable Framework

14. BRIDGING THE GAP ● Technical: ○ Programming languages ○ Eco systems / tooling ● Cultural: ○ Data science vs. engineering ● Cognitive: ○ Specialisation vs. silos

15. POSSIBLE SOLUTIONS ● Provide tools many groups can pick up easily ○ Tensor libraries (ND4S, Jumpy) ○ Deep learning libraries (ScalNet) ● Empowering data scientists by bringing their models to production ○ Model import

16. ND4J ● Performant and versatile Java tensor library ● Hardware acceleration (CPU & GPU) ● Roughly twice as fast as numpy ● Samediff: auto-differentiation module ● ND4S: Scala-wrapper, close to numpy ● Jumpy: ND4J for python, interoperates with numpy

17. ND4S VS NUMPY

18. JUMPY & NUMPY

19. DEEPLEARNING4J ● General purpose DL library ● Parallel training and inference ● Powered by ND4J ● Keras model import ● ScalNet Scala wrapper, Keras lookalike

20. MODEL IMPORT

21. MODEL IMPORT Use case: product recommendations ● Users buy and rate products ● Want to recommend products ● Input features are one-hot encoded users and products ● labels are normalized ratings ● learn features Users Products Users Products

22. MODEL IMPORT: KERAS 0 … 0 1 0 … 0 Users 0 … 0 1 0 … 0 Products Embeddings Merge Dense Dense Dense Prediction

23. MODEL IMPORT Store Keras model: Load model and predict:

24. SCALNET ● Intuitive high-level API (Keras) ● Models compile into regular DL4J models ● Scala as “python for production”

25. SCALNET VS KERAS

26. OUTLOOK ● Use autodiff for jumpy to create a DL4J Keras backend ● Feature parity with Keras (for model import) ● Make ScalNet first class member

27. Max Pumperla Kevin Ferguson Deep Learning & The Game of Go BOOKS

28. TAKE AWAYS ● Take production seriously and find ways to work together ● Consider our wrappers to bridge the gap ● You can have both

29. [email protected] twitter: maxpumperla github: maxpumperla

30. MAJOR DEEP LEARNING LIBRARIES COMPARED

From Experimentation to Production - Scala & Python APIs for DL4J

Recommended

More Related Content

What's hot (20)

Similar to From Experimentation to Production - Scala & Python APIs for DL4J (20)

More from Max Pumperla (7)

Recently uploaded (20)

From Experimentation to Production - Scala & Python APIs for DL4J