H2O World - Intro to R, Python, and Flow - Amy Wang
6 likes2,378 views
The document provides an introduction to loading data into H2O from R and Python, building logistic regression and deep learning models on an airline departure delays dataset, and reviewing the model outputs, with hands-on examples of commands to run these analyses in R, Python, and the H2O Flow web interface. It also advertises additional sessions that will provide more in-depth learning about generalized linear models and deep learning techniques in H2O.
1 of 20
Downloaded 129 times
![Import Airlines Data into H2O
importFiles [ "https://s3.amazonaws.com/h2o-airlines-unpacked/allyears2k.csv" ]
setupParse paths: [ "https://s3.amazonaws.com/h2o-airlines-unpacked/
allyears2k.csv" ]
…
airlines.hex <- h2o.importFile(path = normalizePath(“../data/allyears2k.csv”),
destionation_frame = “allyearks2k.hex”)
** Note: Make sure that the working directory is set to where your
intro-to-r.md.R file is located otherwise the path to the data will be wrong.
airlines.hex = h2o.import_file(path = os.path.realpath(“../data/allyears2k.csv”),
destionation_frame = “allyearks2k.hex”)
airlines.hex.describe()
Flow Users
R Users
Python Users](https://image.slidesharecdn.com/introtorpythonflow-151111005224-lva1-app6891/85/H2O-World-Intro-to-R-Python-and-Flow-Amy-Wang-12-320.jpg)
![Build a Logistic Regression Model
y <- "IsDepDelayed"
x <- c("Dest", "Origin", "DayofMonth", "Year", "UniqueCarrier", "DayOfWeek", "Month", "Distance")
glm_model <- h2o.glm(x = x, y = y, training_frame = airlines.hex, model_id = "glm_model_from_R",
solver = "IRLSM", standardize = T, link = "logit",
family = "binomial", alpha = 0.5, lambda = 1e-05)
myY = “IsDepDelayed”
myX = ["Dest", "Origin", "DayofMonth", "Year", "UniqueCarrier", "DayOfWeek", "Month", “Distance"]
glm_model = H2OGeneralizedLinearEstimator(family = "binomial",standardize = True, solver = “IRLSM", link =
"logit", alpha = 0.5, model_id = "glm_model_from_python" )
glm_model.train( x = myX, y = myY, training_frame = airlines_hex)
Flow Users
R Users
Python Users](https://image.slidesharecdn.com/introtorpythonflow-151111005224-lva1-app6891/85/H2O-World-Intro-to-R-Python-and-Flow-Amy-Wang-14-320.jpg)
![Build a Deep Learning Model
dl_model <- h2o.deeplearning(x = x, y = y, training_frame = airlines.hex,
distribution = “bernoulli”, model_id = "deeplearning_model_from_R", epochs = 100,
hidden = c(200,200), target_ratio_comm_to_comp = 0.02, seed =
6765686131094811000, variable_importances = T)
deeplearning_model = H2ODeepLearningEstimator( distribution = "bernoulli",
model_id = “deeplearning_model_from_python”, epochs = 100, hidden = [200,200], seed =
6765686131094811000, variable_importances = True)
deeplearning_model.train(x = myX, y = myY, training_frame = airlines_hex)
Flow Users
R Users
Python Users](https://image.slidesharecdn.com/introtorpythonflow-151111005224-lva1-app6891/85/H2O-World-Intro-to-R-Python-and-Flow-Amy-Wang-17-320.jpg)
Ad
Recommended
H2O World - PySparkling Water - Nidhi Mehta
H2O World - PySparkling Water - Nidhi MehtaSri Ambati PySparkling Water allows users to run H2O algorithms on Spark clusters from Python. It initializes an H2O context using Py4J, connecting a Python client to an H2O cluster on Spark executors. The document demonstrates building an arrest prediction model on a Chicago crime dataset combined with census and weather data using PySparkling. It provides instructions on prerequisites like installing Spark, PySparkling, H2O and the H2O Python client, and how to start a PySparkling cluster from a Python notebook or shell to run the demo.
H2O World - Munging, modeling, and pipelines using Python - Hank Roark
H2O World - Munging, modeling, and pipelines using Python - Hank RoarkSri Ambati The document discusses the integration of Python and H2O for data munging, modeling, and pipeline creation, particularly in the context of prognostics and health management for turbofan jet engines. It highlights feature engineering, data enrichment, and model evaluation processes, emphasizing the importance of community feedback and Pythonic interfaces for rapid learning and production scalability. Additionally, it provides resources and tools for leveraging existing knowledge within the PyData ecosystem.
2014 09 30_sparkling_water_hands_on
2014 09 30_sparkling_water_hands_onSri Ambati The document discusses Sparkling Water, a tool that integrates H2O with Apache Spark for efficient data analysis and machine learning. It provides an overview of the installation process, leveraging Spark APIs, and running deep learning models on distributed data. The document also highlights configurations for Spark, usage examples, and resources for further learning.
PythonBrasil[8] - CPython for dummies
PythonBrasil[8] - CPython for dummiesTatiana Al-Chueyr This document provides an overview of how to contribute to the cPython source code. It discusses running benchmarks to understand performance differences between loops inside and outside functions. It encourages contributing to improve coding skills and help the open source community. The steps outlined are to clone the cPython source code repository, resolve any dependencies during building, review open issues on bugs.python.org, and work on resolving issues - starting with easier ones. Tips are provided such as commenting when taking ownership of an issue, reproducing bugs before working on them, writing tests for code changes, and updating documentation.
A Beginner's Guide to Building Data Pipelines with Luigi
A Beginner's Guide to Building Data Pipelines with LuigiGrowth Intelligence This document serves as a guide for building data pipelines, particularly focusing on enhancing outbound sales and marketing efforts for UK limited companies through data handling. It discusses the use of a command line interface and introduces Luigi, an open-source tool for managing batch processing jobs, task dependencies, and incorporating custom logging. Additionally, it covers various tasks for counting companies and handling data persistence while emphasizing tasks and dependencies within the processing framework.
Spark Summit EU talk by Nimbus Goehausen
Spark Summit EU talk by Nimbus GoehausenSpark Summit This document discusses automatic checkpointing in Spark. It describes the current Spark development cycle and issues with it. It then presents an approach called Distributed Collections (DC) that allows defining data pipelines in a lazy manner and generating logical signatures to enable automatic checkpointing without rerunning previous stages. This is achieved by introducing the concepts of Deferred Results (DR) and following dependencies recursively to hash functions and classes. DC builds upon but does not modify Spark, aiming to improve the developer workflow while avoiding complexity. Sample code is provided and the approach is available as an open source project.
Presto overview
Presto overviewShixiong Zhu Presto is a distributed SQL query engine that allows users to run SQL queries against various data sources. It consists of three main components - a coordinator, workers, and clients. The coordinator manages query execution by generating execution plans, coordinating workers, and returning final results to the client. Workers contain execution engines that process individual tasks and fragments of a query plan. The system uses a dynamic query scheduler to distribute tasks across workers based on data and node locality.
Sparkling Water Meetup
Sparkling Water MeetupSri Ambati The document outlines the capabilities and features of Sparkling Water, an integration of H2O with Apache Spark, aimed at enhancing machine learning workflows through efficient data processing and user-friendly APIs. It includes practical examples for building models using flight and weather data, instructions for setup, and references for further exploration of Sparkling Water and H2O functionalities. Additionally, it encourages community involvement and provides links for resources and tutorials.
Chapel-on-X: Exploring Tasking Runtimes for PGAS Languages
Chapel-on-X: Exploring Tasking Runtimes for PGAS LanguagesAkihiro Hayashi The document discusses the exploration of dynamic tasking runtimes for PGAS languages, particularly focusing on Chapel's tasking constructs and the potential integration of alternative runtimes like Open Community Runtime (OCR) and Habanero-C/C++. It evaluates various tasking and synchronization functions provided in Chapel, comparing their performance with existing runtimes through application tests. The findings indicate that while HCLib generally outperforms other runtimes, there are scenarios where differences in performance become evident, influencing the choice of runtime based on specific application needs.
Luigi presentation NYC Data Science
Luigi presentation NYC Data ScienceErik Bernhardsson Luigi is a workflow management system that allows users to build complex data pipelines. It provides tools to define dependencies between tasks, run workflows on Hadoop, and visualize data flows. The speaker describes how they developed Luigi at Spotify to manage thousands of Hadoop jobs run daily for music recommendations and other applications. Key features of Luigi include defining Python tasks, easy command line execution, automatic dependency resolution, and failure recovery through atomic file operations. The speaker demonstrates how Luigi can run multi-step workflows on the command line, including a music recommendation example involving feature extraction, model training, and evaluation.
Debugging PySpark: Spark Summit East talk by Holden Karau
Debugging PySpark: Spark Summit East talk by Holden KarauSpark Summit The document discusses debugging PySpark applications, focusing on understanding PySpark architecture, logging options, and interpreting error messages, particularly JVM stack traces. It emphasizes the challenges and nuances of Python and Spark integration, alongside practical advice for improving debugging processes. Additionally, it touches on performance pitfalls and mentions relevant resources, including co-authored books and tools for testing and logging in PySpark.
Flink 0.10 @ Bay Area Meetup (October 2015)
Flink 0.10 @ Bay Area Meetup (October 2015)Stephan Ewen Flink 0.10 focuses on operational readiness with improvements to high availability, monitoring, and integration with other systems. It provides first-class support for event time processing and refines the DataStream API to be both easy to use and powerful for stream processing tasks.
Maximilian Michels – Google Cloud Dataflow on Top of Apache Flink
Maximilian Michels – Google Cloud Dataflow on Top of Apache FlinkFlink Forward This document discusses Google Cloud Dataflow and how it can be executed using Apache Flink. It provides an overview of Dataflow and its API, which is similar to batch and streaming concepts in Flink. It then describes how a Dataflow program is translated to an Abstract Syntax Tree (AST) and how the AST is converted to a Flink execution graph by implementing translators for specific Dataflow transforms like ParDo and Combine. Finally, it mentions the FlinkPipelineRunner that is available on GitHub to execute Dataflow pipelines using Flink.
SparkR - Play Spark Using R (20160909 HadoopCon)
SparkR - Play Spark Using R (20160909 HadoopCon)wqchen The document provides a comprehensive overview of 'sparkr', an R interface for Apache Spark, highlighting its origins, features, and usage. It covers key concepts such as data frames, RDDs, machine learning libraries, and practical applications for data analysis and visualization using R. Additionally, it discusses the future development plans for sparkr and includes various references and resources for further learning.
Introduction to Apache Beam & No Shard Left Behind: APIs for Massive Parallel...
Introduction to Apache Beam & No Shard Left Behind: APIs for Massive Parallel...Dan Halperin Apache Beam is a unified programming model for efficient and portable data processing pipelines. It provides abstractions like PCollections, sources/readers, ParDo, GroupByKey, side inputs, and windowing that hide complexity and allow runners to optimize efficiency. Beam supports both batch and streaming workloads on different distributed processing backends. It gives runners control over bundle size, splitting, and triggering to make tradeoffs between latency, throughput, and efficiency based on workload and cluster resources. This allows the same pipeline to be executed efficiently in different contexts without changes to the user code.
Big Data Science with H2O in R
Big Data Science with H2O in RAnqi Fu The document provides an overview of H2O, an open source math engine for big data and machine learning, emphasizing its capability to analyze large datasets using R programming. It includes installation instructions, examples of basic GLM (Generalized Linear Models) implementations with the prostate cancer dataset, and guidelines for remote connection to H2O. The document also outlines future goals for integration with R and additional algorithm support.
Interactive Session on Sparkling Water
Interactive Session on Sparkling WaterSri Ambati Sparkling Water provides a transparent integration of H2O algorithms and data structures into the Spark ecosystem. It allows users to use H2O machine learning algorithms on data stored in Spark and HDFS. The presentation demonstrates loading weather and flight data using Spark and H2O APIs, building regression models to predict flight delays, and accessing prediction results from R for residual analysis. Sparkling Water applications can be developed and run as standalone jobs by creating a SparkContext and H2OContext and submitting to a Spark cluster.
Monitoring Spark Applications
Monitoring Spark ApplicationsTzach Zohar Monitoring Spark Applications
Tzach Zohar @ Kenshoo, March/2016
The document discusses monitoring Spark applications. It covers using the Spark UI to monitor jobs, stages and tasks; using the Spark REST API to programmatically access monitoring data; configuring Spark metric sinks like Graphite to export internal Spark metrics; and creating applicative metrics to monitor your own application metrics. The key points are monitoring is important for failures, performance, correctness and understanding data; Spark provides built-in tools but applicative metrics are also useful; and Graphite is well-suited to analyze metrics trends over time.
Yahoo! Mail antispam - Bay area Hadoop user group
Yahoo! Mail antispam - Bay area Hadoop user groupHadoop User Group This document discusses shape-shifting spam and how Hadoop can help fight against it. It describes the challenges of spam filtering at scale given spammers' ability to rapidly change their techniques. It outlines the evolution of anti-spam techniques from manual and rule-based approaches to machine learning models. It then introduces Hadoop and how its distributed processing capabilities allow for more expressive models that can learn from large datasets and quickly update to match spammers' changes. Examples are given of how Hadoop enables standard reporting, ad hoc queries, campaign discovery, and other applications to provide insights into spammer behavior.
Getting to Know Airflow
Getting to Know AirflowRosanne Hoyem The document provides an overview of Apache Airflow, highlighting its origin at Airbnb in 2014 and its evolution into a widely used tool for data engineering and workflow automation. It explains the concept of Directed Acyclic Graphs (DAGs) in managing task dependencies and outlines the functionalities and limitations of Airflow. Additionally, it briefly mentions competitors and the challenges regarding documentation in the rapidly evolving project.
R and C++
R and C++Romain Francois - Rcpp is a package that facilitates interoperability between R and C++ by providing data structures and functions that make it easy to write C++ code that integrates with R. It has been released 54 times since 2008 with over 170 CRAN packages depending on it.
- Rcpp allows users to source C++ code from R using sourceCpp() and export C++ functions to R using attributes like // [[Rcpp::export]]. This improves performance over pure R code by leveraging fast C++ implementations.
- dplyr is a popular R package for data manipulation that achieves great performance through its use of Rcpp. Functions like arrange(), filter(), and summarise() are much faster when
Data correlation using PySpark and HDFS
Data correlation using PySpark and HDFSJohn Conley This document discusses using PySpark and HDFS to correlate different types of data at scale. It describes some challenges with correlating out-of-order and high-volume data. It then summarizes three approaches tried: using Redis, RDD joins, and writing bindings to HDFS. The current recommended approach reads relevant binding buckets from HDFS to correlate records in small windows, supporting different temporal models. Configuration and custom logic can be plugged in at various points in the correlation process. While scalable, further improvements in latency and throughput are still needed.
Extending Flux to Support Other Databases and Data Stores | Adam Anthony | In...
Extending Flux to Support Other Databases and Data Stores | Adam Anthony | In...InfluxData The document outlines the process of extending Flux by creating custom data sources, particularly using SQL for integration. It details the structure of Flux data models, stream processing, and the implementation of source decoders to fetch and decode data into Flux tables. Additionally, it highlights the framework's flexibility to connect to various data sources and potentially write data to them.
Everyday I'm Shuffling - Tips for Writing Better Spark Programs, Strata San J...
Everyday I'm Shuffling - Tips for Writing Better Spark Programs, Strata San J...Databricks The document provides insights on optimizing Apache Spark jobs by understanding key functional elements such as shuffling, appropriate use of operations like reduceByKey versus groupByKey, and how to handle joins effectively. It emphasizes the importance of understanding Spark's internal workings to write efficient, well-tested, and reusable jobs while highlighting common pitfalls and best practices. Additionally, it discusses strategies for reusing code between batch and streaming processes, testing scenarios in Spark, and diagnosing performance issues.
Scalding - the not-so-basics @ ScalaDays 2014
Scalding - the not-so-basics @ ScalaDays 2014Konrad Malawski This document discusses various big data technologies and how they relate to each other. It explains that Summingbird is built on top of Scalding and Storm, which are built on top of Cascading, which is built on top of Hadoop. It also discusses how Spark relates and compares to these other technologies.
Unit testing of spark applications
Unit testing of spark applicationsKnoldus Inc. The document provides an overview of unit testing for Spark applications, detailing what Spark is and the importance of unit testing in software development. It discusses different methods for unit testing with Spark, comparing a more manual method with one using the 'spark-testing-base' library that streamlines the process. Key benefits of unit testing, such as early bug detection and easing integration, are also highlighted.
PyConUK 2018 - Journey from HTTP to gRPC
PyConUK 2018 - Journey from HTTP to gRPCTatiana Al-Chueyr The presentation details the evolution of a recommender system at BBC from an HTTP-based monolithic architecture to a more efficient microservices architecture using gRPC and protocol buffers. It outlines the challenges faced with CPU and I/O-bound tasks in earlier versions, the benefits of introducing gunicorn and asynchronous workers to improve performance, and a transition to gRPC for better scalability and type safety. The speaker emphasizes the importance of adapting technologies to enhance system capabilities while maintaining performance.
Introduction to Apache Airflow - Data Day Seattle 2016
Introduction to Apache Airflow - Data Day Seattle 2016Sid Anand Apache Airflow is a platform for authoring, scheduling, and monitoring workflows or directed acyclic graphs (DAGs) of tasks. It includes a DAG scheduler, web UI, and CLI. Airflow allows users to author DAGs in Python without needing to bundle many XML files. The UI provides tree and Gantt chart views to monitor DAG runs over time. Airflow was accepted into the Apache Incubator in 2016 and has over 300 users from 40+ companies. Agari uses Airflow to orchestrate message scoring pipelines across AWS services like S3, Spark, SQS, and databases to enforce SLAs on correctness and timeliness. Areas for further improvement include security, APIs, execution scaling, and on
H2O World - Intro to Data Science with Erin Ledell
H2O World - Intro to Data Science with Erin LedellSri Ambati This document provides an introduction to data science. It defines data science as using data to solve problems through the scientific method. The roles of data scientists, data analysts, and data engineers on a data science team are discussed. Popular tools for data science include Python, R, and APIs that connect data processing engines. Machine learning algorithms are used to perform tasks like classification, regression, and clustering by learning from data rather than being explicitly programmed. Deep learning and ensemble methods are also introduced. Resources for learning more about data science and machine learning are provided.
Introduction to R for Data Science :: Session 3
Introduction to R for Data Science :: Session 3Goran S. Milovanovic This document provides an introduction to lists and functions in R. It discusses how lists can contain elements of different types and classes, and can be recursively nested. It also covers subsetting lists, adding and removing list elements, and converting lists to vectors. The document then introduces functions in R, including defining simple functions, taking multiple arguments, and using curly braces to denote blocks of code. It demonstrates how everything in R is a function, including operators. Finally, it shows how to use lapply() and apply() to iterate functions over list elements or matrix rows and columns.
More Related Content
What's hot (20)
Chapel-on-X: Exploring Tasking Runtimes for PGAS Languages
Chapel-on-X: Exploring Tasking Runtimes for PGAS LanguagesAkihiro Hayashi The document discusses the exploration of dynamic tasking runtimes for PGAS languages, particularly focusing on Chapel's tasking constructs and the potential integration of alternative runtimes like Open Community Runtime (OCR) and Habanero-C/C++. It evaluates various tasking and synchronization functions provided in Chapel, comparing their performance with existing runtimes through application tests. The findings indicate that while HCLib generally outperforms other runtimes, there are scenarios where differences in performance become evident, influencing the choice of runtime based on specific application needs.
Luigi presentation NYC Data Science
Luigi presentation NYC Data ScienceErik Bernhardsson Luigi is a workflow management system that allows users to build complex data pipelines. It provides tools to define dependencies between tasks, run workflows on Hadoop, and visualize data flows. The speaker describes how they developed Luigi at Spotify to manage thousands of Hadoop jobs run daily for music recommendations and other applications. Key features of Luigi include defining Python tasks, easy command line execution, automatic dependency resolution, and failure recovery through atomic file operations. The speaker demonstrates how Luigi can run multi-step workflows on the command line, including a music recommendation example involving feature extraction, model training, and evaluation.
Debugging PySpark: Spark Summit East talk by Holden Karau
Debugging PySpark: Spark Summit East talk by Holden KarauSpark Summit The document discusses debugging PySpark applications, focusing on understanding PySpark architecture, logging options, and interpreting error messages, particularly JVM stack traces. It emphasizes the challenges and nuances of Python and Spark integration, alongside practical advice for improving debugging processes. Additionally, it touches on performance pitfalls and mentions relevant resources, including co-authored books and tools for testing and logging in PySpark.
Flink 0.10 @ Bay Area Meetup (October 2015)
Flink 0.10 @ Bay Area Meetup (October 2015)Stephan Ewen Flink 0.10 focuses on operational readiness with improvements to high availability, monitoring, and integration with other systems. It provides first-class support for event time processing and refines the DataStream API to be both easy to use and powerful for stream processing tasks.
Maximilian Michels – Google Cloud Dataflow on Top of Apache Flink
Maximilian Michels – Google Cloud Dataflow on Top of Apache FlinkFlink Forward This document discusses Google Cloud Dataflow and how it can be executed using Apache Flink. It provides an overview of Dataflow and its API, which is similar to batch and streaming concepts in Flink. It then describes how a Dataflow program is translated to an Abstract Syntax Tree (AST) and how the AST is converted to a Flink execution graph by implementing translators for specific Dataflow transforms like ParDo and Combine. Finally, it mentions the FlinkPipelineRunner that is available on GitHub to execute Dataflow pipelines using Flink.
SparkR - Play Spark Using R (20160909 HadoopCon)
SparkR - Play Spark Using R (20160909 HadoopCon)wqchen The document provides a comprehensive overview of 'sparkr', an R interface for Apache Spark, highlighting its origins, features, and usage. It covers key concepts such as data frames, RDDs, machine learning libraries, and practical applications for data analysis and visualization using R. Additionally, it discusses the future development plans for sparkr and includes various references and resources for further learning.
Introduction to Apache Beam & No Shard Left Behind: APIs for Massive Parallel...
Introduction to Apache Beam & No Shard Left Behind: APIs for Massive Parallel...Dan Halperin Apache Beam is a unified programming model for efficient and portable data processing pipelines. It provides abstractions like PCollections, sources/readers, ParDo, GroupByKey, side inputs, and windowing that hide complexity and allow runners to optimize efficiency. Beam supports both batch and streaming workloads on different distributed processing backends. It gives runners control over bundle size, splitting, and triggering to make tradeoffs between latency, throughput, and efficiency based on workload and cluster resources. This allows the same pipeline to be executed efficiently in different contexts without changes to the user code.
Big Data Science with H2O in R
Big Data Science with H2O in RAnqi Fu The document provides an overview of H2O, an open source math engine for big data and machine learning, emphasizing its capability to analyze large datasets using R programming. It includes installation instructions, examples of basic GLM (Generalized Linear Models) implementations with the prostate cancer dataset, and guidelines for remote connection to H2O. The document also outlines future goals for integration with R and additional algorithm support.
Interactive Session on Sparkling Water
Interactive Session on Sparkling WaterSri Ambati Sparkling Water provides a transparent integration of H2O algorithms and data structures into the Spark ecosystem. It allows users to use H2O machine learning algorithms on data stored in Spark and HDFS. The presentation demonstrates loading weather and flight data using Spark and H2O APIs, building regression models to predict flight delays, and accessing prediction results from R for residual analysis. Sparkling Water applications can be developed and run as standalone jobs by creating a SparkContext and H2OContext and submitting to a Spark cluster.
Monitoring Spark Applications
Monitoring Spark ApplicationsTzach Zohar Monitoring Spark Applications
Tzach Zohar @ Kenshoo, March/2016
The document discusses monitoring Spark applications. It covers using the Spark UI to monitor jobs, stages and tasks; using the Spark REST API to programmatically access monitoring data; configuring Spark metric sinks like Graphite to export internal Spark metrics; and creating applicative metrics to monitor your own application metrics. The key points are monitoring is important for failures, performance, correctness and understanding data; Spark provides built-in tools but applicative metrics are also useful; and Graphite is well-suited to analyze metrics trends over time.
Yahoo! Mail antispam - Bay area Hadoop user group
Yahoo! Mail antispam - Bay area Hadoop user groupHadoop User Group This document discusses shape-shifting spam and how Hadoop can help fight against it. It describes the challenges of spam filtering at scale given spammers' ability to rapidly change their techniques. It outlines the evolution of anti-spam techniques from manual and rule-based approaches to machine learning models. It then introduces Hadoop and how its distributed processing capabilities allow for more expressive models that can learn from large datasets and quickly update to match spammers' changes. Examples are given of how Hadoop enables standard reporting, ad hoc queries, campaign discovery, and other applications to provide insights into spammer behavior.
Getting to Know Airflow
Getting to Know AirflowRosanne Hoyem The document provides an overview of Apache Airflow, highlighting its origin at Airbnb in 2014 and its evolution into a widely used tool for data engineering and workflow automation. It explains the concept of Directed Acyclic Graphs (DAGs) in managing task dependencies and outlines the functionalities and limitations of Airflow. Additionally, it briefly mentions competitors and the challenges regarding documentation in the rapidly evolving project.
R and C++
R and C++Romain Francois - Rcpp is a package that facilitates interoperability between R and C++ by providing data structures and functions that make it easy to write C++ code that integrates with R. It has been released 54 times since 2008 with over 170 CRAN packages depending on it.
- Rcpp allows users to source C++ code from R using sourceCpp() and export C++ functions to R using attributes like // [[Rcpp::export]]. This improves performance over pure R code by leveraging fast C++ implementations.
- dplyr is a popular R package for data manipulation that achieves great performance through its use of Rcpp. Functions like arrange(), filter(), and summarise() are much faster when
Data correlation using PySpark and HDFS
Data correlation using PySpark and HDFSJohn Conley This document discusses using PySpark and HDFS to correlate different types of data at scale. It describes some challenges with correlating out-of-order and high-volume data. It then summarizes three approaches tried: using Redis, RDD joins, and writing bindings to HDFS. The current recommended approach reads relevant binding buckets from HDFS to correlate records in small windows, supporting different temporal models. Configuration and custom logic can be plugged in at various points in the correlation process. While scalable, further improvements in latency and throughput are still needed.
Extending Flux to Support Other Databases and Data Stores | Adam Anthony | In...
Extending Flux to Support Other Databases and Data Stores | Adam Anthony | In...InfluxData The document outlines the process of extending Flux by creating custom data sources, particularly using SQL for integration. It details the structure of Flux data models, stream processing, and the implementation of source decoders to fetch and decode data into Flux tables. Additionally, it highlights the framework's flexibility to connect to various data sources and potentially write data to them.
Everyday I'm Shuffling - Tips for Writing Better Spark Programs, Strata San J...
Everyday I'm Shuffling - Tips for Writing Better Spark Programs, Strata San J...Databricks The document provides insights on optimizing Apache Spark jobs by understanding key functional elements such as shuffling, appropriate use of operations like reduceByKey versus groupByKey, and how to handle joins effectively. It emphasizes the importance of understanding Spark's internal workings to write efficient, well-tested, and reusable jobs while highlighting common pitfalls and best practices. Additionally, it discusses strategies for reusing code between batch and streaming processes, testing scenarios in Spark, and diagnosing performance issues.
Scalding - the not-so-basics @ ScalaDays 2014
Scalding - the not-so-basics @ ScalaDays 2014Konrad Malawski This document discusses various big data technologies and how they relate to each other. It explains that Summingbird is built on top of Scalding and Storm, which are built on top of Cascading, which is built on top of Hadoop. It also discusses how Spark relates and compares to these other technologies.
Unit testing of spark applications
Unit testing of spark applicationsKnoldus Inc. The document provides an overview of unit testing for Spark applications, detailing what Spark is and the importance of unit testing in software development. It discusses different methods for unit testing with Spark, comparing a more manual method with one using the 'spark-testing-base' library that streamlines the process. Key benefits of unit testing, such as early bug detection and easing integration, are also highlighted.
PyConUK 2018 - Journey from HTTP to gRPC
PyConUK 2018 - Journey from HTTP to gRPCTatiana Al-Chueyr The presentation details the evolution of a recommender system at BBC from an HTTP-based monolithic architecture to a more efficient microservices architecture using gRPC and protocol buffers. It outlines the challenges faced with CPU and I/O-bound tasks in earlier versions, the benefits of introducing gunicorn and asynchronous workers to improve performance, and a transition to gRPC for better scalability and type safety. The speaker emphasizes the importance of adapting technologies to enhance system capabilities while maintaining performance.
Introduction to Apache Airflow - Data Day Seattle 2016
Introduction to Apache Airflow - Data Day Seattle 2016Sid Anand Apache Airflow is a platform for authoring, scheduling, and monitoring workflows or directed acyclic graphs (DAGs) of tasks. It includes a DAG scheduler, web UI, and CLI. Airflow allows users to author DAGs in Python without needing to bundle many XML files. The UI provides tree and Gantt chart views to monitor DAG runs over time. Airflow was accepted into the Apache Incubator in 2016 and has over 300 users from 40+ companies. Agari uses Airflow to orchestrate message scoring pipelines across AWS services like S3, Spark, SQS, and databases to enforce SLAs on correctness and timeliness. Areas for further improvement include security, APIs, execution scaling, and on
Viewers also liked (20)
H2O World - Intro to Data Science with Erin Ledell
H2O World - Intro to Data Science with Erin LedellSri Ambati This document provides an introduction to data science. It defines data science as using data to solve problems through the scientific method. The roles of data scientists, data analysts, and data engineers on a data science team are discussed. Popular tools for data science include Python, R, and APIs that connect data processing engines. Machine learning algorithms are used to perform tasks like classification, regression, and clustering by learning from data rather than being explicitly programmed. Deep learning and ensemble methods are also introduced. Resources for learning more about data science and machine learning are provided.
Introduction to R for Data Science :: Session 3
Introduction to R for Data Science :: Session 3Goran S. Milovanovic This document provides an introduction to lists and functions in R. It discusses how lists can contain elements of different types and classes, and can be recursively nested. It also covers subsetting lists, adding and removing list elements, and converting lists to vectors. The document then introduces functions in R, including defining simple functions, taking multiple arguments, and using curly braces to denote blocks of code. It demonstrates how everything in R is a function, including operators. Finally, it shows how to use lapply() and apply() to iterate functions over list elements or matrix rows and columns.
Introduction to R for Data Science :: Session 2
Introduction to R for Data Science :: Session 2Goran S. Milovanovic This document provides an introduction to vectors, matrices, and data frames in R. It discusses that in R, everything starts as a vector, including the columns of a data frame. It demonstrates how to create, subset, order, and concatenate vectors. It also shows how to create matrices and discusses their similarities to vectors. Finally, it covers key aspects of working with data frames such as subsetting, joining, and using names(), rownames(), and colnames().
R language tutorial
R language tutorialDavid Chiu R is a programming language for statistical analysis and graphics. It is an open-source language developed by statisticians to allow for easy statistical analysis and visualization of data. The document provides an overview of R, discussing its origins, functionality, uses in data science, and popular packages and IDEs used with R. Examples are given of basic R syntax for vectors, matrices, data frames, plotting, and applying functions to data.
H2O World - Ensembles with Erin LeDell
H2O World - Ensembles with Erin LeDellSri Ambati The document details ensemble learning in statistics and machine learning, highlighting various methods like bagging, boosting, and stacking, which combine multiple algorithms to improve predictive performance. It describes the super learner algorithm, which optimally integrates base learners through cross-validation and a metalearner, and is applicable for tasks in regression and classification. Additionally, it outlines the H2O ensemble R package and emphasizes its effectiveness in machine learning competitions, like those on Kaggle.
H2O Overview with Amy Wang at useR! Aalborg
H2O Overview with Amy Wang at useR! AalborgSri Ambati H2O.ai is a machine learning company founded in 2012 with 35 employees based in Mountain View, CA. It was started by Stanford engineers and is an open source leader in machine and deep learning. H2O's software provides interfaces for R, Python, Spark and Hadoop and expands predictive analytics capabilities to large datasets across many industries. The executive team is led by CEO Sri Satish Ambati and CTO Cliff Click, and the scientific advisory council includes experts from Stanford like Trevor Hastie and Stephen Boyd.
H2O World - GLM - Tomas Nykodym
H2O World - GLM - Tomas NykodymSri Ambati Generalized linear models (GLMs) are a well-known statistical method for fitting linear models to different types of prediction problems. GLMs allow modeling of non-normal distributions and non-identity link functions. Regularization techniques like lasso, ridge, and elastic net are commonly used with GLMs to prevent overfitting. H2O's GLM implementation handles large datasets with many predictors efficiently using distributed and parallel processing. It supports automatic handling of categorical variables, multiple solvers, and regularization including elastic net. GLMs scaled to "big data" problems can be fit in seconds on H2O using all CPU cores.
R Intro Workshop
R Intro Workshop Saad Chahine This document provides an introduction to using R for statistical analysis. It outlines how to get started with R, perform basic calculations and logic, work with variables, vectors, matrices and factors. It also demonstrates how to import and export data, call data within R, and describes some useful R packages. Examples are provided for each concept to help users get hands-on experience with the R programming language.
Distributed R: The Next Generation Platform for Predictive Analytics
Distributed R: The Next Generation Platform for Predictive AnalyticsJorge Martinez de Salinas The document discusses 'Distributed R', a next-generation platform for predictive analytics that leverages horizontal scaling and enhances the performance of R through a multi-node architecture. It highlights features such as parallel processing, support for large datasets, and built-in distributed algorithms for various analytics tasks. Additionally, the platform aims to facilitate collaboration and open-source engagement within the data science community.
Deciphering voice of customer through speech analytics
Deciphering voice of customer through speech analyticsR Systems International This document discusses speech analytics and its use for analyzing customer call center conversations. It begins by explaining the challenges of analyzing speech data and how speech recognition systems work to transform speech into structured data. It then discusses common use cases for speech analytics in call centers, such as sentiment analysis and agent performance monitoring. Next, it provides an overview of major vendors in the speech analytics market. It proposes a two-phase architecture for speech analytics involving speech recognition and predictive analytics. Finally, it presents a case study using speech analytics to predict customer loyalty scores for a health insurance provider.
Optimizing Facebook Campaigns with R
Optimizing Facebook Campaigns with RDomino Data Lab The document is a detailed presentation by Gergely Daroczi on managing and optimizing Facebook ad campaigns using R. It covers key aspects such as understanding when to advertise, utilizing Facebook's marketing API, creating custom and lookalike audiences, and setting up campaigns and adsets. The content includes practical examples and coding snippets for implementing these strategies in R.
R lecture oga
R lecture ogaOsamu Ogasawara R is a statistical programming language used for data analysis and visualization. The document discusses R's data types and structures, functions for probability distributions like the normal distribution, and plotting functions. It also covers creating vectors, lists, and data frames in R. The final section briefly mentions using R for RNA-seq analysis in bioinformatics.
The Next List: R&D Breakthroughs that are Changing the World
The Next List: R&D Breakthroughs that are Changing the WorldGE We believe in a future made better by brilliant minds and brilliant machines. The Next List is our blueprint for progress therein.
Implementing a highly scalable stock prediction system with R, Geode, SpringX...
Implementing a highly scalable stock prediction system with R, Geode, SpringX...William Markito Oliveira The document provides an overview of implementing a highly scalable stock prediction system using R, Apache Geode, and Spring XD, highlighting data ingestion, machine learning model training, and real-time evaluation of stock data. Key concepts include the integration of live and historical data to predict stock price movements and the use of Apache Geode for performance and fault tolerance. Additionally, it discusses practical applications in industries such as transportation, specifically referencing Indian and Chinese railway systems.
Cloud Conf 2015 - Develop and Deploy IOT Applications
Cloud Conf 2015 - Develop and Deploy IOT ApplicationsCorley S.r.l. The document discusses the development and deployment of an IoT application for monitoring environmental data like temperature and humidity. It emphasizes the importance of using a REST server and a graphical integrated development environment (Flowcode) to simplify programming for microcontrollers. Additionally, it provides insights into utilizing components for data collection and presents useful resources for further information.
IMCSummit 2015 - Day 2 Developer Track - Implementing a Highly Scalable In-Me...
IMCSummit 2015 - Day 2 Developer Track - Implementing a Highly Scalable In-Me...In-Memory Computing Summit The document discusses a stock prediction system utilizing open-source software, focusing on machine learning techniques like neural networks and real-time data processing. It highlights challenges in traditional models and emphasizes the need for real-time, stream-based analytics. The architecture proposed includes the integration of various data sources with a framework for scalable and efficient processing using tools like Spring XD and GemFire.
Implementing a highly scalable stock prediction system with R, Geode, SpringX...
Implementing a highly scalable stock prediction system with R, Geode, SpringX...William Markito Oliveira
IMCSummit 2015 - Day 2 Developer Track - Implementing a Highly Scalable In-Me...
IMCSummit 2015 - Day 2 Developer Track - Implementing a Highly Scalable In-Me...In-Memory Computing Summit
Ad
Similar to H2O World - Intro to R, Python, and Flow - Amy Wang (20)
Living With Legacy Code
Living With Legacy CodeRowan Merewood The document discusses the challenges of dealing with legacy code, defined as code that lacks tests and is often not understood by current developers. It provides a strategy for managing legacy systems, including documentation review, engaging with original developers, and gradually refactoring code while testing changes. Key recommendations include using version control, setting up continuous integration, and applying dependency injection for better testability.
Container (Docker) Orchestration Tools
Container (Docker) Orchestration ToolsDhilipsiva DS This document discusses various Docker orchestration tools including Kubernetes, Marathon, Rancher, Helios, Ansible Container, Docker Swarm, and others. It provides brief descriptions and usage examples for several of the tools. Kubernetes and Marathon are covered in slightly more detail, with notes on prerequisites and basic usage commands provided. The document aims to introduce developers to common options for orchestrating Docker containers at scale.
Pemrograman Python untuk Pemula
Pemrograman Python untuk PemulaOon Arfiandwi This document provides an introduction to programming with Python for beginners. It covers basic Python concepts like variables, data types, operators, conditional statements, functions, loops, strings and lists. It also demonstrates how to build simple web applications using Google App Engine and Python, including templating with Jinja2, storing data in the Datastore and handling web forms. The goal is to teach the fundamentals of Python programming and get started with cloud development on Google Cloud Platform.
What's New In Laravel 5
What's New In Laravel 5Darren Craig This document provides an overview and introduction to Laravel 5, a PHP web application framework. It discusses key Laravel 5 concepts and features such as Eloquent ORM, routing, middleware, contracts, form requests, the IoC container, file drivers, scheduling commands, and the command bus pattern. The document is intended to explain Laravel 5 concepts through code examples and brief explanations.
Using and scaling Rack and Rack-based middleware
Using and scaling Rack and Rack-based middlewareAlona Mekhovova Rack provides a standard interface between web servers and web applications. It allows a web application to return a status, headers, and a body in response to an HTTP request. Middleware can be plugged into a Rack application to modify requests and responses. Popular Rack middleware includes Rack::Cache, Rack::Middleware, and Warden for authentication. In Rails, middleware is configured through an initializer and plugged into the middleware stack to run before or after other middleware.
Python Code Camp for Professionals 3/4
Python Code Camp for Professionals 3/4DEVCON The document provides a comprehensive guide on building a Python web application using MongoDB, covering installation, basic operations, and API development with the Bottle framework. It includes examples of CRUD operations, creating dynamic HTML pages, and returning various data types like JSON and HTML. Additionally, it discusses advanced topics such as templating with Tenjin and handling user input through forms.
Swift Cloud Workshop - Swift Microservices
Swift Cloud Workshop - Swift MicroservicesChris Bailey The document discusses deploying Swift microservices with Docker and Kubernetes. It covers building a Swift microservice into a Docker container, deploying the container with Kubernetes, and integrating resiliency features like health checks, circuit breakers, and monitoring. Key points include using Dockerfiles to package the application, Helm charts to deploy to Kubernetes, liveness probes to check service health, and circuit breakers to handle failures of downstream services.
Best practices tekx
Best practices tekxLorna Mitchell The document discusses several software development best practices including source control, coding standards, design patterns, documentation, testing, and deployment. It provides examples and explanations of common design patterns like factory, registry, adapter, decorator, and observer. Resources for further information on topics like Subversion, Git, coding standards, and design patterns are also referenced.
服务框架: Thrift & PasteScript
服务框架: Thrift & PasteScriptQiangning Hong Thrift and PasteScript are frameworks for building distributed applications and services. Thrift allows defining data types and interfaces using a simple definition language that can generate code in multiple languages. It uses a compact binary protocol for efficient RPC-style communication between clients and servers. PasteScript builds on WSGI and provides tools like paster for deploying and managing Python web applications, along with reloading and logging capabilities. It integrates with Thrift via server runners and application factories.
Introduction to Data Science with H2O- Mountain View
Introduction to Data Science with H2O- Mountain ViewSri Ambati This document provides an overview of H2O.ai, an open source in-memory machine learning platform. It discusses that H2O.ai was founded in 2011 and is venture-backed, with a team of 37 people working on distributed systems for machine learning. It also summarizes that H2O provides easy to use APIs for Java, R, Python and other languages, and allows for scalable machine learning on large datasets using distributed algorithms to make full use of data without downsampling. Finally, it highlights how H2O works with other technologies like Spark, Hadoop, and HDFS to enable reading of large datasets for machine learning.
Introduction to data science with H2O-Chicago
Introduction to data science with H2O-ChicagoSri Ambati This document provides an overview of H2O.ai, an open source in-memory machine learning platform. It describes H2O.ai's product as an in-memory prediction engine, its team of 37 distributed systems engineers doing machine learning, and its headquarters in Mountain View, CA. It also provides details on how to use H2O with R and Python for scalable machine learning on large datasets across distributed systems.
CodeIgniter PHP MVC Framework
CodeIgniter PHP MVC FrameworkBo-Yi Wu The document discusses CodeIgniter, an open source PHP MVC framework, and provides information about CodeIgniter features such as controllers, models, views, helpers, libraries, and working with databases using CodeIgniter's active record functions. It also covers topics like installing CodeIgniter, creating controllers and models, and loading views, helpers, and libraries.
OSSBarCamp Talk on Dexy
OSSBarCamp Talk on Dexyananelson The document discusses the Dexy documentation tool. It provides 10 ways that Dexy can help with projects, including promoting projects, helping users, improving code quality, getting involved in open source, contributing confidently, expressing yourself, writing productively, keeping documentation current, learning, and earning. It also discusses Dexy techniques like templating, sectioning, JSON, writing filters, using subprocesses, configuring Dexy, running Dexy, and the output. It emphasizes writing the right kind of documentation for the purpose.
Python and Oracle : allies for best of data management
Python and Oracle : allies for best of data managementLaurent Leturgez The document provides an overview of using Python in conjunction with Oracle databases for data management and manipulation, highlighting features of Python and how to interact with Oracle through installation and coding examples. It discusses libraries like NumPy and Pandas for data analysis and manipulation, as well as methods for querying and calling PL/SQL procedures. Additionally, the document touches upon deploying resources in Oracle Cloud Infrastructure and integrates practical coding demonstrations.
Hack Like It's 2013 (The Workshop)
Hack Like It's 2013 (The Workshop)Itzik Kotler This document introduces Pythonect, a dataflow programming language based on Python, highlighting its installation, syntax, and applications in developing domain-specific languages (DSLs) and security tools. Additionally, it presents Hackersh, a shell designed for security tasks integration with Pythonect, featuring built-in security components and multi-threading capabilities. The content emphasizes ease of use, flexibility, and the potential for creating efficient workflows within the realms of programming and cybersecurity.
Django
DjangoHarmeet Lamba Django is a Python-based web framework that follows the MTV (Model-Template-View) design pattern. It allows developers to create dynamic websites and web applications by handling common tasks like database access, session management, and rendering templates. The document outlines how to create a Django project and app, configure settings like the database, and link views to URLs to handle requests and return responses.
Python Functions Tutorial | Working With Functions In Python | Python Trainin...
Python Functions Tutorial | Working With Functions In Python | Python Trainin...Edureka! The document is about Python certification training, focusing on functions, their importance, and different types. It explains the concept of functions, including built-in and user-defined functions, provides examples of common built-in functions, and discusses the usage of lambda functions. Additionally, it emphasizes the benefits of using functions to reduce code repetition and outlines how functions can be structured and invoked in Python.
Burn down the silos! Helping dev and ops gel on high availability websites
Burn down the silos! Helping dev and ops gel on high availability websitesLindsay Holmwood The document discusses a DevOps case study focused on applying technologies for a fundraising website, emphasizing the need for consistency, repeatability, and visibility in deployment and configuration management. It highlights tools like Puppet and Capistrano for automating deployments and managing complex application environments while detailing the challenges and solutions around configuration drift and metrics collection. The case study illustrates the importance of collaboration between development and operations teams and incorporates real-world examples of deployment strategies and monitoring practices.
Tips
Tipsmclee The document provides tips for optimizing PHP code, including using string functions instead of regular expressions where possible, passing references to reduce memory usage, using persistent database connections, and checking mysql_unbuffered_query() for faster queries. It also discusses HTTP requests and responses, cookie expiry, references in PHP, returning references from functions, and the debug_backtrace() function. The document concludes with tips for improving security such as checking for uninitialized variables, validating user input, and restricting access to included files.
Python from zero to hero (Twitter Explorer)
Python from zero to hero (Twitter Explorer)Yuriy Senko This document outlines steps to build a Twitter explorer application using Python and Flask. It begins with setting up the virtual environment and cloning the GitHub repository. It then walks through steps to add basic functionality like configuration, templates, a database with SQLAlchemy ORM, user authentication with Flask plugins, and finally integrating the Twitter API. Each step includes changes to files, dependencies in requirements.txt, and commands to test and view progress. The goal is to create a full-stack web application to explore tweets from the Twitter API.
Ad
More from Sri Ambati (20)
H2O.ai Agents : From Theory to Practice - Support Presentation
H2O.ai Agents : From Theory to Practice - Support PresentationSri Ambati This is the support slide deck for H2O Agents AI: From Theory to Practice course.
These slides cover AI agent architecture, h2oGPTe capabilities, industry applications across finance, healthcare, telecom, and energy sectors, plus implementation best practices.
They're designed as a helpful reference while following the video course or for quick review of key concepts in agentic AI.
To access the full course and more AI learning resources, visit https://h2o.ai/university/
H2O Generative AI Starter Track - Support Presentation Slides.pdf
H2O Generative AI Starter Track - Support Presentation Slides.pdfSri Ambati H2O Generative AI Starter Track introduces you to practical applications of Generative AI using Enterprise h2oGPTe—a secure, flexible, and enterprise-ready platform designed for real-world AI adoption.
Explore core AI concepts, prompt engineering, Retrieval-Augmented Generation (RAG), and enterprise integration through a structured, hands-on approach.
Use the slides above to follow along and deepen your understanding.
Learn more at:
https://h2o.ai/university/
Learn more at :
https://h2o.ai/university/
H2O Gen AI Ecosystem Overview - Level 1 - Slide Deck
H2O Gen AI Ecosystem Overview - Level 1 - Slide DeckSri Ambati The document provides a comprehensive introduction to large language models (LLMs), detailing their definitions, construction steps, and the significance of data cleaning in their development. It outlines the lifecycle of LLMs, benefits, challenges, and the features of the LLM Datastudio, which aids in data preparation and fine-tuning. Additionally, the text emphasizes the advantages of open-source LLMs over proprietary models, highlighting issues of cost, customization, and data privacy.
An In-depth Exploration of Enterprise h2oGPTe Slide Deck
An In-depth Exploration of Enterprise h2oGPTe Slide DeckSri Ambati The document outlines an end-to-end pipeline for utilizing H2O.ai’s h2ogpt, focusing on AI-powered search and question-answering functionalities. It describes how to implement solutions using the h2ogpte UI and Python API, and emphasizes integration with enterprise document AI technologies. The document also highlights the ease of use in developing apps and deploying them within the H2O platform.
Intro to Enterprise h2oGPTe Presentation Slides
Intro to Enterprise h2oGPTe Presentation SlidesSri Ambati The document introduces Andreea Turcu, head of global training at H2O.ai, and outlines key topics related to generative AI, including its benefits, challenges, and the H2O generative AI ecosystem. It details the H2O Gen AI framework which empowers organizations to create custom large language models while ensuring data integrity, and highlights the capabilities of enterprise H2O GPT for document analysis and question-answering. The presentation concludes with a summary of features and a Q&A session.
Enterprise h2o GPTe Learning Path Slide Deck
Enterprise h2o GPTe Learning Path Slide DeckSri Ambati The document provides an overview of H2O.ai's generative AI ecosystem, including its functionalities, benefits, and challenges of generative AI technologies. It details the H2O GPT-E platform, which offers AI-powered document question-answering capabilities and supports integration with various data extraction technologies. The presentation also discusses the open-source nature of H2O's solutions and wraps up with an exploration of its applications for enterprise use.
H2O Wave Course Starter - Presentation Slides
H2O Wave Course Starter - Presentation SlidesSri Ambati The document outlines an H2O Wave starter course aimed at teaching users to build AI applications using the H2O Wave framework, which is an open-source low-code tool for Python and R. It covers the setup of the development environment, writing and deploying wave apps, as well as practical applications like chatbots. The course encourages exploration of the GenAI app store to further enhance learning and contribution.
Large Language Models (LLMs) - Level 3 Slides
Large Language Models (LLMs) - Level 3 SlidesSri Ambati The document introduces h2oGPT, an open-source large language model (LLM) designed for enhancing productivity in various business applications, including content automation and data analysis. It discusses the limitations of existing proprietary LLMs, such as concerns over data privacy and customization constraints, while emphasizing the benefits of open-source models like cost-effectiveness and user control. The document also outlines the evaluation metrics for LLM performance and highlights the importance of thorough assessment frameworks.
Data Science and Machine Learning Platforms (2024) Slides
Data Science and Machine Learning Platforms (2024) SlidesSri Ambati The document outlines H2O.ai's various data science and machine learning capabilities, including tools for data visualization, model creation, and deployment. It details features such as Driverless AI for automating model building, Label Genie for data labeling, and capabilities for managing large language models. Additionally, it discusses the application of generative AI and training methodologies within H2O.ai's ecosystem.
Data Prep for H2O Driverless AI - Slides
Data Prep for H2O Driverless AI - SlidesSri Ambati The document discusses principles for data preparation in machine learning, emphasizing the importance of data quality and structured data formats. It outlines the differences between supervised and unsupervised learning, and highlights practices for handling different types of data issues, including missing or inaccurate data. Additionally, it covers best practices for time series data preparation and considerations for model training and scoring in the context of H2O.ai's platform.
H2O Cloud AI Developer Services - Slides (2024)
H2O Cloud AI Developer Services - Slides (2024)Sri Ambati The document outlines various AI and machine learning services provided by H2O.ai, including tools for data preparation, feature engineering, and model building across different data types. It emphasizes the company's offerings in no-code and low-code solutions, as well as AI operations (MLOps) for model management and deployment. Additionally, it highlights advanced features such as automated machine learning, computer vision, and natural language processing capabilities aimed at democratizing AI for users.
LLM Learning Path Level 2 - Presentation Slides
LLM Learning Path Level 2 - Presentation SlidesSri Ambati The document outlines a comprehensive learning path for fine-tuning large language models (LLMs) utilizing H2O.ai's Data Studio, emphasizing the importance of clean, relevant data and various workflows. Key features include preparing datasets for question-answering, text summarization, and supporting different NLP tasks while addressing challenges such as bias, data quality, and selecting appropriate model backbones. The document also covers technical aspects like synthetic datasets, quantization, and LORA for optimizing model performance and deployment.
LLM Learning Path Level 1 - Presentation Slides
LLM Learning Path Level 1 - Presentation SlidesSri Ambati The document outlines the learning path for fine-tuning large language models (LLMs) at H2O.ai, covering foundational concepts, architecture, and practical applications. It emphasizes the importance of pre-training, fine-tuning, and transfer learning in adapting LLMs for specific tasks. Additionally, key characteristics and challenges of language models, as well as their uses in various applications like chatbots and content generation, are discussed.
Hydrogen Torch - Starter Course - Presentation Slides
Hydrogen Torch - Starter Course - Presentation SlidesSri Ambati H2O Hydrogen Torch is a platform designed to enhance deep learning model training with features such as hyperparameter tuning, user-friendly interface, and flexible deployment options. It supports various applications, including image, text, and audio analysis, and outlines a structured workflow for dataset handling, model training, and evaluation. The document also details assignments aimed at improving model performance through practical experiments with hyperparameter tuning and model optimization.
Presentation Resources - H2O Gen AI Ecosystem Overview - Level 2
Presentation Resources - H2O Gen AI Ecosystem Overview - Level 2Sri Ambati The document outlines a comprehensive framework for developing and refining generative AI applications and large language models (LLMs), detailing components such as data preparation, fine-tuning, retrieval augmented generation (RAG), and prompt engineering. It emphasizes the importance of foundational models and their applications, as well as the evaluation methods for assessing LLM performance across various tasks. Additionally, it introduces practical labs dedicated to implementing and enhancing generative AI systems.
H2O Driverless AI Starter Course - Slides and Assignments
H2O Driverless AI Starter Course - Slides and AssignmentsSri Ambati The document outlines the H2O Driverless AI starter course, detailing its content and assignments designed to teach users about navigating the Driverless AI interface, data preparation, visualization, and experiment creation. It includes a structured outline of course topics and specific assignments aimed at hands-on learning, such as modifying datasets and creating visualizations. The course emphasizes practical application through interactive assignments to solidify understanding and application of Driverless AI features.
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...Sri Ambati The document discusses the democratization of Artificial General Intelligence (AGI) through open-source community efforts, emphasizing its significance like that of music and mathematics. H2O.ai is highlighted for its role in various sectors, using data science and machine learning to drive innovation and improve public services, while also showcasing the achievements of its expert community. The overarching theme is the need for responsible AI practices and the integration of AI applications across multiple industries to foster collaboration and drive societal benefits.
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DaySri Ambati This document provides an overview of H2O.ai, an AI company that offers products and services to democratize AI. It mentions that H2O products are backed by 10% of the world's top data scientists from Kaggle and that H2O has customers in 7 of the top 10 banks, 4 of the top 10 insurance companies, and top manufacturing companies. It also provides details on H2O's founders, funding, customers, products, and vision to make AI accessible to more organizations.
Generative AI Masterclass - Model Risk Management.pptx
Generative AI Masterclass - Model Risk Management.pptxSri Ambati Here are some key points about benchmarking and evaluating generative AI models like large language models:
- Foundation models require large, diverse datasets to be trained on in order to learn broad language skills and knowledge. Fine-tuning can then improve performance on specific tasks.
- Popular benchmarks evaluate models on tasks involving things like commonsense reasoning, mathematics, science questions, generating truthful vs false responses, and more. This helps identify model capabilities and limitations.
- Custom benchmarks can also be designed using tools like Eval Studio to systematically test models on specific applications or scenarios. Both automated and human evaluations are important.
- Leaderboards like HELM aggregate benchmark results to compare how different models perform across a wide range of tests and metrics.
AI and the Future of Software Development: A Sneak Peek
AI and the Future of Software Development: A Sneak Peek Sri Ambati Pritika Mehta, Co-Founder, Butternut.ai
H2O Open Source GenAI World SF 2023
Recently uploaded (20)
Zoneranker’s Digital marketing solutions
Zoneranker’s Digital marketing solutionsreenashriee Zoneranker offers expert digital marketing services tailored for businesses in Theni. From SEO and PPC to social media and content marketing, we help you grow online. Partner with us to boost visibility, leads, and sales.
ERP Systems in the UAE: Driving Business Transformation with Smart Solutions
ERP Systems in the UAE: Driving Business Transformation with Smart Solutionsdheeodoo ERP solutions are driving innovative change for businesses across the UAE through improved efficiency, reduced operational costs, and provide real-time visibility of operational data. By offering native Arabic language support, VAT compliance, and some level of cloud readiness, ERP solutions also supports and are consistent with the initiative for Smart Dubai 2030. Odoo is an example of a flexible & scalable solution. If you're looking for expert support with implementation contact Banibro. www.banibro.com
MOVIE RECOMMENDATION SYSTEM, UDUMULA GOPI REDDY, Y24MC13085.pptx
MOVIE RECOMMENDATION SYSTEM, UDUMULA GOPI REDDY, Y24MC13085.pptxMaharshi Mallela Movie recommendation system is a software application or algorithm designed to suggest movies to users based on their preferences, viewing history, or other relevant factors. The primary goal of such a system is to enhance user experience by providing personalized and relevant movie suggestions.
Zoho Creator Solution for EI by Elsner Technologies.docx
Zoho Creator Solution for EI by Elsner Technologies.docxElsner Technologies Pvt. Ltd. Working with Elsner’s Zoho Creator experts, Educational Initiatives implemented a custom asset management system that: Our Zoho Creator solutions have made real changes across the company, and each team has felt the difference.
CodeCleaner: Mitigating Data Contamination for LLM Benchmarking
CodeCleaner: Mitigating Data Contamination for LLM Benchmarkingarabelatso The pdf version of "CodeCleaner: Mitigating Data Contamination for LLM Benchmarking" presented in Internetware 2025.
From Data Preparation to Inference: How Alluxio Speeds Up AI
From Data Preparation to Inference: How Alluxio Speeds Up AIAlluxio, Inc. Alluxio Webinar
June 17, 2025
For more Alluxio Events: https://www.alluxio.io/events/
Speaker:
Jingwen Ouyang (Sr. Product Manager @ Alluxio)
In this talk, Jingwen Ouyang, Senior Product Manager at Alluxio, will share how Alluxio make it easy to share and manage data from any storage to any compute engine in any environment with high performance and low cost for your model training, model inference, and model distribution workload.
arctitecture application system design os dsa
arctitecture application system design os dsaza241967 wow amazing see and enjoy for this the best pdf slideshow you could see random shit is what i do best
IObit Driver Booster Pro 12 Crack Latest Version Download
IObit Driver Booster Pro 12 Crack Latest Version Downloadpcprocore 👉𝗡𝗼𝘁𝗲:𝗖𝗼𝗽𝘆 𝗹𝗶𝗻𝗸 & 𝗽𝗮𝘀𝘁𝗲 𝗶𝗻𝘁𝗼 𝗚𝗼𝗼𝗴𝗹𝗲 𝗻𝗲𝘄 𝘁𝗮𝗯🔴▶ https://pcprocore.com/ ◀✅
IObit Driver Booster Pro is the solution. It automatically downloads and updates drivers with just one click, avoiding hardware failures, system instability, and security vulnerabilities.
Decipher SEO Solutions for your startup needs.
Decipher SEO Solutions for your startup needs.mathai2 A solution deck that gives you an idea of how you can use Decipher SEO to target keywords, build authority and generate high ranking content.
With features like images to product you can create a E-commerce pipeline that is optimized to help your store rank.
With integrations with shopify, woocommerce and wordpress theres a seamless way get your content to your website or storefront.
View more at decipherseo.com
Download Adobe Illustrator Crack free for Windows 2025?
Download Adobe Illustrator Crack free for Windows 2025?grete1122g COPY & PASTE LINK 👉👉👉 https://click4pc.com/after-verification-click-go-to-download-page/
Adobe Illustrator CC Crack will make your illustration level upgrade by moving toward real modification purpose using adobe pro tools there ..
Heat Treatment Process Automation in India
Heat Treatment Process Automation in IndiaReckers Mechatronics Automate your heat treatment processes for superior precision, consistency, and cost savings. Explore solutions for furnaces, quench systems. Heat treatment is a critical manufacturing process that alters the microstructure and properties of materials, typically metals, to achieve desired characteristics such as hardness, strength, ductility, and wear resistance.
Modern Platform Engineering with Choreo - The AI-Native Internal Developer Pl...
Modern Platform Engineering with Choreo - The AI-Native Internal Developer Pl...WSO2 Building and operating internal platforms has become increasingly complex — sprawling toolchains, rising costs, and fragmented developer experiences are slowing teams down. It’s time for a new approach.
This slide deck explores modern platform engineering with Choreo, WSO2’s AI-native Internal Developer Platform as a Service. Discover how you can streamline platform operations, empower your developers, and drive faster innovation — all while reducing complexity and cost.
Learn more: https://wso2.com/choreo/platform-engineering/
Best Practice for LLM Serving in the Cloud
Best Practice for LLM Serving in the CloudAlluxio, Inc. Alluxio Webinar
June 17, 2025
For more Alluxio Events: https://www.alluxio.io/events/
Speaker:
Nilesh Agarwal (Co-founder & CTO @ Inferless)
Nilesh Agarwal, co-founder & CTO at Inferless, shares insights on accelerating LLM inference in the cloud using Alluxio, tackling key bottlenecks like slow model weight loading from S3 and lengthy container startup time. Inferless uses Alluxio as a three-tier cache system that dramatically cuts model load time by 10x.
OpenChain Webinar - AboutCode - Practical Compliance in One Stack – Licensing...
OpenChain Webinar - AboutCode - Practical Compliance in One Stack – Licensing...Shane Coughlan OpenChain Webinar - AboutCode - Practical Compliance in One Stack – Licensing, Vulnerabilities, and More
Enable Your Cloud Journey With Microsoft Trusted Partner | IFI Tech
Enable Your Cloud Journey With Microsoft Trusted Partner | IFI TechIFI Techsolutions Start your cloud journey with IFI Tech—Microsoft’s trusted partner delivering secure, scalable Azure solutions tailored for your business.
On-Device AI: Is It Time to Go All-In, or Do We Still Need the Cloud?
On-Device AI: Is It Time to Go All-In, or Do We Still Need the Cloud?Hassan Abid As mobile hardware becomes more powerful, the promise of running advanced AI directly on-device is closer than ever. With Google’s latest on-device model Gemini Nano, accessible through the new ML Kit GenAI APIs and AI Edge SDK, alongside the open-source Gemma-3n models, developers can now integrate lightweight, multimodal intelligence that works even without an internet connection. But does this mean we no longer need cloud-based AI? This session explores the practical trade-offs between on-device and cloud AI for mobile apps.
Automated Migration of ESRI Geodatabases Using XML Control Files and FME
Automated Migration of ESRI Geodatabases Using XML Control Files and FMESafe Software Efficient data migration is a critical challenge in geospatial data management, especially when working with complex data structures. This presentation explores an automated approach to migrating ESRI Geodatabases using FME and XML-based control files. A key advantage of this method is its adaptability: changes to the data model are seamlessly incorporated into the migration process without requiring modifications to the underlying FME workflow. By separating data model definitions from migration logic, this approach ensures flexibility, reduces maintenance effort, and enhances scalability.
Azure AI Foundry: The AI app and agent factory
Azure AI Foundry: The AI app and agent factoryMaxim Salnikov Discover how Azure AI Foundry is transforming the way organizations innovate and operate. This session explores the pivotal role of AI in reshaping business processes, enhancing employee experiences, and redefining customer engagement. Attendees will gain insights into the strategic value of AI investments, including the measurable returns and the growing momentum behind generative AI experimentation.
The talk introduces the emerging paradigm of agentic AI - intelligent agents that are revolutionizing how we build, manage, and interact with software. Through real-world examples, the session demonstrates how Azure AI Foundry empowers teams to design, customize, and scale AI applications and agents, unlocking new levels of efficiency and creativity across industries.
Application Modernization with Choreo - The AI-Native Internal Developer Plat...
Application Modernization with Choreo - The AI-Native Internal Developer Plat...WSO2 In this slide deck, we explore the challenges and best practices of application modernization. We also deep dive how an internal developer platform as a service like Choreo can fast track your modernization journey with AI capabilities and end-to-end workflow automation.
H2O World - Intro to R, Python, and Flow - Amy Wang
- 1. Introduction to R, Python, and Flow Amy Wang [email protected]
- 2. Getting Started with H2O • Learn how R, Flow, and Python sends commands to compute in H2O • FAQ on writing R, Flow, and Python expressions • Hands on introduction into data science • Understanding model outputs • Note the limitations of the basic workflow to improve upon later Objective
- 3. I have H2O Installed I have Python installed I have R installed I have the H2O World data sets Pick up stickers or get install help at the information booth
- 4. Reading Data into H2O with R R User h2o_df = h2o.importFile(“../data/allyears2k.csv”) STEP 1
- 5. Reading Data into H2O with R STEP 2 H2O H2O H2O H2O ClusterInitiate distributed ingest 2.3 HTTP REST API request to H2O carries the path argument 2.2 allyears2k.csvRequest data 2.4 R h2o.importFile() 2.1 R function call
- 6. Reading Data into H2O with R Data Provided 3.1 allyears2k.csv R Cluster IP Cluster Port Pointer to Data 3.4 h2o_df object created in R h2o_df H2O H2O H2O H2O Frame 3.2 Distributed H2O Frame in DKV H2O Cluster Return pointer to data in REST API JSON Response 3.3 STEP 3
- 7. R Script Starting H2O GLM HTTP REST/JSON .h2o.startModelJob() POST /3/ModelBuilders/glm h2o.glm() R script Standard R process TCP/IP HTTP REST/JSON /3/ModelBuilders/glm endpoint Job GLM algorithm GLM tasks Fork/Join framework K/V store framework H2O process Network layer REST layer H2O - algos H2O - core User process H2O process Legend
- 8. R Parity
- 9. Hands on Introduction to Running H2O • Import & parse a small 44000 row airlines dataset • Run a Logistic Regression • Build a Deep Learning model • Review the Model Outputs
- 10. Starting up H2O and Preloaded Workbook From terminal, change the directory to where h2o.jar file is sitting and run: > java -jar h2o.jar Then access the Flow UI at: https://localhost:54321 Open the intro-to-r.md.R file and run from R (Native R or Rstudio): library(h2o) h2o.init(nthreads = -1) Open either intro-to-python.ipynb or intro-to-python.py with python: import h2o import … h2o.init() Flow Users R Users Python Users
- 11. Load up Preinstalled Flow Pack For Flow Users
- 12. Import Airlines Data into H2O importFiles [ "https://s3.amazonaws.com/h2o-airlines-unpacked/allyears2k.csv" ] setupParse paths: [ "https://s3.amazonaws.com/h2o-airlines-unpacked/ allyears2k.csv" ] … airlines.hex <- h2o.importFile(path = normalizePath(“../data/allyears2k.csv”), destionation_frame = “allyearks2k.hex”) ** Note: Make sure that the working directory is set to where your intro-to-r.md.R file is located otherwise the path to the data will be wrong. airlines.hex = h2o.import_file(path = os.path.realpath(“../data/allyears2k.csv”), destionation_frame = “allyearks2k.hex”) airlines.hex.describe() Flow Users R Users Python Users
- 13. The Airlines Data • Goal: To predict departure delays using historical airlines data. • Enumerator columns: Dest, Origin, and Unique Carrier have a cardinality of 134, 132, and 10 respectively • Numeric Columns: DayOfMonth, Year, DayOfWeek, Month, and Distance • Binary Response Column: IsDepDelayed
- 14. Build a Logistic Regression Model y <- "IsDepDelayed" x <- c("Dest", "Origin", "DayofMonth", "Year", "UniqueCarrier", "DayOfWeek", "Month", "Distance") glm_model <- h2o.glm(x = x, y = y, training_frame = airlines.hex, model_id = "glm_model_from_R", solver = "IRLSM", standardize = T, link = "logit", family = "binomial", alpha = 0.5, lambda = 1e-05) myY = “IsDepDelayed” myX = ["Dest", "Origin", "DayofMonth", "Year", "UniqueCarrier", "DayOfWeek", "Month", “Distance"] glm_model = H2OGeneralizedLinearEstimator(family = "binomial",standardize = True, solver = “IRLSM", link = "logit", alpha = 0.5, model_id = "glm_model_from_python" ) glm_model.train( x = myX, y = myY, training_frame = airlines_hex) Flow Users R Users Python Users
- 15. GLM Model Output
- 16. To Learn More About GLM • Ramanujan Stage @ 10:45 AM • Erdos Stage @ 2:15 PM
- 17. Build a Deep Learning Model dl_model <- h2o.deeplearning(x = x, y = y, training_frame = airlines.hex, distribution = “bernoulli”, model_id = "deeplearning_model_from_R", epochs = 100, hidden = c(200,200), target_ratio_comm_to_comp = 0.02, seed = 6765686131094811000, variable_importances = T) deeplearning_model = H2ODeepLearningEstimator( distribution = "bernoulli", model_id = “deeplearning_model_from_python”, epochs = 100, hidden = [200,200], seed = 6765686131094811000, variable_importances = True) deeplearning_model.train(x = myX, y = myY, training_frame = airlines_hex) Flow Users R Users Python Users
- 18. Deep Learning Model Output
- 19. To Learn More About Deep Learning • Ramanujan Stage @ 5:45 PM • Erdos Stage @ 1:30 PM
- 20. Overview • Write R and Python expression to clean and munge the data in a parallelized and distributed fashion in H2O. • Automate model builds by writing R and Python code and because all frames and models are generate in H2O, it is also accessible from the Flow UI. • From the Web UI, the user can readily access the POJO which is the H2O independent Java representation of the models.