Big Data Day LA 2015 - NoSQL: Doing it wrong before getting it right by Lawrence Trinh of Fandango
1 like596 views
The team at Fandango heartily embraced NoSQL, using Couchbase to power a key media publishing system. The initial implementation was fraught with integration issues and high latency, and required a major effort to successfully refactor. My talk will outline the key organizational and architectural decisions that created deep systemic problems, and the steps taken to re-architect the system to achieve a high level of performance at scale.
1 of 8
Ad
Recommended
Big Data Day LA 2015 - Introducing N1QL: SQL for Documents by Jeff Morris of ...
Big Data Day LA 2015 - Introducing N1QL: SQL for Documents by Jeff Morris of ...Data Con LA NoSQL has exploded on the developer scene promising alternatives to RDBMS that make rapidly developing, Internet scale applications easier than ever. However, as a trade off to the ease of development and scale, some of the familiarity with other well-known query interfaces such as SQL, has been lost. Until now that is...N1QL (pronounced ‘N1QL’) is a SQL like query language for querying JSON, which brings the familiarity of RDBMS back to the NoSQL world. In this session you will learn about the syntax and basics of this new language as well as Integration with the Couchbase SDKs.
The Future of Hadoop by Arun Murthy, PMC Apache Hadoop & Cofounder Hortonworks
The Future of Hadoop by Arun Murthy, PMC Apache Hadoop & Cofounder HortonworksData Con LA Arun Murthy will be discussing the future of Hadoop and the next steps in what the big data world would start to look like in the future. With the advent of tools like Spark and Flink and containerization of apps using Docker, there is a lot of momentum currently in this space. Arun will share his thoughts and ideas on what the future holds for us.
Bio:-
Arun C. Murthy
Arun is a Apache Hadoop PMC member and has been a full time contributor to the project since the inception in 2006. He is also the lead of the MapReduce project and has focused on building NextGen MapReduce (YARN). Prior to co-founding Hortonworks, Arun was responsible for all MapReduce code and configuration deployed across the 42,000+ servers at Yahoo!. In essence, he was responsible for running Apache Hadoop’s MapReduce as a service for Yahoo!. Also, he jointly holds the current world sorting record using Apache Hadoop. Follow Arun on Twitter: @acmurthy.
A Thorough Comparison of Delta Lake, Iceberg and Hudi
A Thorough Comparison of Delta Lake, Iceberg and HudiDatabricks Recently, a set of modern table formats such as Delta Lake, Hudi, Iceberg spring out. Along with Hive Metastore these table formats are trying to solve problems that stand in traditional data lake for a long time with their declared features like ACID, schema evolution, upsert, time travel, incremental consumption etc.
Presto @ Uber Hadoop summit2017
Presto @ Uber Hadoop summit2017Zhenxiao Luo This document discusses Presto, an interactive SQL query engine for big data. It describes how Presto is optimized to quickly query data stored in Parquet format at Uber. Key optimizations for Parquet include nested column pruning, columnar reads, predicate pushdown, dictionary pushdown, and lazy reads. Benchmark results show these optimizations improve Presto query performance. The document also provides an overview of Uber's analytics infrastructure, applications of Presto, and ongoing work to further optimize Presto and Hadoop.
Hadoop and HBase @eBay
Hadoop and HBase @eBayDataWorks Summit eBay has one of the largest Hadoop clusters in the industry with many petabytes of data. This talk will give an overview of how Hadoop and HBase have been used within eBay, the lessons we have learned from supporting large-scale production clusters, as well as how we plan to use and improve Hadoop and HBase moving forward. Specific use cases, production issues and platform improvement work will be discussed.
Database Camp 2016 @ United Nations, NYC - Michael Glukhovsky, Co-Founder, Re...
Database Camp 2016 @ United Nations, NYC - Michael Glukhovsky, Co-Founder, Re...✔ Eric David Benari, PMP Advancing Real-Time Responses in Web Applications
Michael Glukhovsky, Co-Founder, RethinkDB
Video of this session at the Database Camp conference at the UN is on http://www.Database.Camp
Membase Meetup 2010
Membase Meetup 2010Membase This document provides an overview and technical discussion of Membase. It begins with introducing Membase and how it allows both applications and databases to scale horizontally. The rest of the document discusses Membase architecture, deployment options, use cases, and a demo. It also briefly explores developing with Membase and the future direction of NodeCode, which will allow extending Membase through custom modules.
Apache Hive 2.0: SQL, Speed, Scale
Apache Hive 2.0: SQL, Speed, ScaleDataWorks Summit/Hadoop Summit This document discusses the new features of Apache Hive 2.0, including:
1) The addition of procedural SQL capabilities through HPLSQL to add features like cursors and loops.
2) Performance improvements for interactive queries through LLAP which uses in-memory caching and persistent daemons.
3) Using HBase as the metastore to speed up query planning by reducing metadata access times.
4) Enhancements to Hive on Spark such as dynamic partition pruning and vectorized joins.
5) Improvements to the cost-based optimizer including better statistics collection.
Hadoop Infrastructure @Uber Past, Present and Future
Hadoop Infrastructure @Uber Past, Present and FutureDataWorks Summit Uber’s mission is to provide transportation as reliable as running water and for fulfilling that mission data plays a critical role. In Uber, Hadoop plays a critical role in Data Infrastructure. We want to talk about the journey of Hadoop @Uber and our future plans in terms of scaling for billions of trips. We will talk about most unique use case Uber have and how Hadoop and eco system which we built, helped us in this journey. We want to talk about how we scaled from 10 -> 2000 and In future to scale up to 10’s X1000 of Nodes. We will talk about our mistakes, learning and wins and how we process billions of events per day. We will talk about the unique challenges and real world use-cases and how we will co-locate the Uber’s service architecture with batch (e.g data pipelines, machine learning and analytical workloads). Uber have done lot of improvements to current Hadoop eco system and uniquely solved some of the problems in a way which is never been solved in the past. This presentation will help audience to use this as an example and even encourage them to enhance the eco system. This will help to increase the community of these project and overall help the whole big data space. Audience is anybody who is working on Big Data and want to understand how to scale Hadoop and eco system for 10s of thousands of node. This talk will help them understand the Hadoop ecosystem and how to efficiently use that. It will also introduce them to some of the awesome technologies which Uber team is building in big data space.
Hadoop World 2011: Data Ingestion, Egression, and Preparation for Hadoop - Sa...
Hadoop World 2011: Data Ingestion, Egression, and Preparation for Hadoop - Sa...Cloudera, Inc. One of the first challenges Hadoop developers face is accessing all the data they need and getting it into Hadoop for analysis. Informatica PowerExchange accesses a variety of data types and structures at different latencies (e.g. batch, real-time, or near real-time) and ingests data directly into Hadoop. The next step is to parse the data in preparation for analysis in Hadoop. Informatica provides a visual IDE to deploy pre-built parsers or design specific parsers for complex data formats and deploy them on Hadoop. Once the analysis is complete, Informatica PowerExhange delivers the resulting output to other information management systems such as a data warehouse. Learn in this session from Informatica and one of their customers, how to get all the data you need into Hadoop, parse a variety of data formats and structures, and egress the resultant output to other systems.
Data Engineer's Lunch #55: Get Started in Data Engineering
Data Engineer's Lunch #55: Get Started in Data EngineeringAnant Corporation In Data Engineer's Lunch #55, CEO of Anant, Rahul Singh, will cover 10 resources every data engineer needs to get started or master their game.
Accompanying Blog: Coming Soon!
Accompanying YouTube: Coming Soon!
Sign Up For Our Newsletter: http://eepurl.com/grdMkn
Join Data Engineer’s Lunch Weekly at 12 PM EST Every Monday:
https://www.meetup.com/Data-Wranglers-DC/events/
Cassandra.Link:
https://cassandra.link/
Follow Us and Reach Us At:
Anant:
https://www.anant.us/
Awesome Cassandra:
https://github.com/Anant/awesome-cassandra
Email:
[email protected]
LinkedIn:
https://www.linkedin.com/company/anant/
Twitter:
https://twitter.com/anantcorp
Eventbrite:
https://www.eventbrite.com/o/anant-1072927283
Facebook:
https://www.facebook.com/AnantCorp/
Join The Anant Team:
https://www.careers.anant.us
Big Data Day LA 2016/ Use Case Driven track - Hydrator: Open Source, Code-Fre...
Big Data Day LA 2016/ Use Case Driven track - Hydrator: Open Source, Code-Fre...Data Con LA This talk will present how to build data pipelines with no code using the open-source, Apache 2.0, Cask Hydrator. The talk will continue with a live demonstration of creating data pipelines for two use cases.
03-NOV-1510-Ognjen-Antonic-Telemach-stream-1
03-NOV-1510-Ognjen-Antonic-Telemach-stream-1Ognjen Antonic This document discusses how Telemach Slovenia leveraged open source big data technologies like Elasticsearch, Logstash, and Kibana (ELK stack) to build analytics dashboards for fraud detection and network monitoring. It summarizes their initial success building a roaming fraud dashboard in 8 days using these technologies. This proof of concept led them to expand usage to additional fraud and network performance dashboards. The ELK stack provided scalable and cost-effective log analytics capabilities compared to commercial options like Splunk. This enabled both IT and business users to gain new visual insights into network operations and issues.
Embeddable data transformation for real time streams
Embeddable data transformation for real time streamsJoey Echeverria This document summarizes Joey Echeverria's presentation on embeddable data transformation for real-time streams. Some key points include:
- Stream processing requires the ability to perform common data transformations like filtering, extracting, projecting, and aggregating on streaming data.
- Tools like Apache Storm, Spark, and Flink can be used to build stream processing topologies and jobs, but also have limitations for embedding transformations.
- Rocana Transform provides a library and DSL for defining reusable data transformation configurations that can be run within different stream processing systems or in batch jobs.
- The library supports common transformations as well as custom actions defined through Java. Configurations can extract metrics, parse logs, and perform
Scaling Deep Learning on Hadoop at LinkedIn
Scaling Deep Learning on Hadoop at LinkedInDataWorks Summit Deep learning has become widespread as frameworks such as TensorFlow and PyTorch have made it easy to onboard machine learning applications. However, while it is easy to start developing with these frameworks on your local developer machine, scaling up a model to run on a cluster and train on huge datasets is still challenging. Code and dependencies have to be copied to every machine and defining the cluster configurations is tedious and error-prone. In addition, troubleshooting errors and aggregating logs is difficult. Ad-hoc solutions also lack resource guarantees, isolation from other jobs, and fault tolerance.
To solve these problems and make scaling deep learning easy, we have made several enhancements to Hadoop and built an open-source deep learning platform called TonY. In this talk, Anthony and Keqiu will discuss new Hadoop features useful for deep learning, such as GPU resource support, and deep dive into TonY, which lets you run deep learning programs natively on Hadoop. We will discuss TonY's architecture and how it allows users to manage their deep learning jobs, acting as a portal from which to launch notebooks, monitor jobs, and visualize training results.
Big Telco - Yousun Jeong
Big Telco - Yousun JeongSpark Summit This document provides an overview of SK Telecom's use of big data analytics and Spark. Some key points:
- SKT collects around 250 TB of data per day which is stored and analyzed using a Hadoop cluster of over 1400 nodes.
- Spark is used for both batch and real-time processing due to its performance benefits over other frameworks. Two main use cases are described: real-time network analytics and a network enterprise data warehouse (DW) built on Spark SQL.
- The network DW consolidates data from over 130 legacy databases to enable thorough analysis of the entire network. Spark SQL, dynamic resource allocation in YARN, and integration with BI tools help meet requirements for timely processing and quick
What database
What databaseRegunath B Slides from my talk at the Hasgeek FifthElephant conference 2017 - a practical guide to database selection
What's new in SQL on Hadoop and Beyond
What's new in SQL on Hadoop and BeyondDataWorks Summit/Hadoop Summit Presto is an open source distributed SQL query engine that allows interactive analysis of data across multiple data stores. At Facebook, Presto is used for ad-hoc queries of their Hadoop data warehouse, which processes trillions of rows and scans petabytes of data daily. Presto's low latency also makes it suitable for powering analytics in user-facing products. New features of Presto include improved SQL support, performance optimizations, and connectors to additional data sources like Redis and MongoDB.
Presto Strata London 2019: Cost-Based Optimizer for interactive SQL on anything
Presto Strata London 2019: Cost-Based Optimizer for interactive SQL on anythingPiotr Findeisen The Presto Cost-Based Optimizer for interactive SQL on anything
Jeremy Engle's slides from Redshift / Big Data meetup on July 13, 2017
Jeremy Engle's slides from Redshift / Big Data meetup on July 13, 2017AWS Chicago "Strategies for supporting near real time analytics, OLAP, and interactive data exploration" - Dr. Jeremy Engle, Engineering Manager Data Team at Jellyvision
Real time fraud detection at 1+M scale on hadoop stack
Real time fraud detection at 1+M scale on hadoop stackDataWorks Summit/Hadoop Summit Rocketfuel processes over 120 billion ad auctions per day and needs to detect fraud in real time to prevent losses. They developed Helios, which ingests event data from Kafka and HDFS into Storm in real time, joins the streams in HBase, then runs MapReduce jobs hourly to populate an OLAP cube for analyzing feature vectors and detecting fraud patterns. This architecture on Hadoop allows them to easily scale real-time processing and experiment with different configurations to quickly react to fraud.
Built-In Security for the Cloud
Built-In Security for the CloudDataWorks Summit Today enterprises desire to move more and more of their data lakes to the cloud to help them execute faster, increase productivity, drive innovation while leveraging the scale and flexibility of the cloud. However, such gains come with risks and challenges in the areas of data security, privacy, and governance. In this talk we cover how enterprises can overcome governance and security obstacles to leverage these new advances that the cloud can provide to ease the management of their data lakes in the cloud. We will also show how the enterprise can have consistent governance and security controls in the cloud for their ephemeral analytic workloads in a multi-cluster cloud environment without sacrificing any of the data security and privacy/compliance needs that their business context demands. Additionally, we will outline some use cases and patterns as well as best practices to rationally manage such a multi-cluster data lake infrastructure in the cloud.
Speaker:
Jeff Sposetti, Product Management, Hortonworks
Reshape Data Lake (as of 2020.07)
Reshape Data Lake (as of 2020.07)Eric Sun SF Big Analytics 2020-07-28
Anecdotal history of Data Lake and various popular implementation framework. Why certain tradeoff was made to solve the problems, such as cloud storage, incremental processing, streaming and batch unification, mutable table, ...
Qubole - Big data in cloud
Qubole - Big data in cloudDmitry Tolpeko Big Data Challenges and How to Overcome Them with Qubole - a Self-Service Platform for Big Data Analytics built on Amazon Web Services, Microsoft and Google Clouds. Storing, accessing, and analyzing large amounts of data from diverse sources and making it easily accessible to deliver actionable insights for users can be challenging for data driven organizations. The solution for customers is to optimize scaling and create a unified interface to simplify analysis. Qubole helps customers simplify their big data analytics with speed and scalability, while providing data analysts and scientists self-service access in Cloud. The platform is fully elastic and automatically scales or contracts clusters based on workload. We will try to overview main features, advantages and drawback of this platform.
Owning Your Own (Data) Lake House
Owning Your Own (Data) Lake HouseData Con LA Data Con LA 2020
Description
In this session, I introduce the Amazon Redshift lake house architecture which enables you to query data across your data warehouse, data lake, and operational databases to gain faster and deeper insights. With a lake house architecture, you can store data in open file formats in your Amazon S3 data lake.
Speaker
Antje Barth, Amazon Web Services, Sr. Developer Advocate, AI and Machine Learning
HBaseCon 2013: Real-Time Model Scoring in Recommender Systems 
HBaseCon 2013: Real-Time Model Scoring in Recommender Systems Cloudera, Inc. This document describes a real-time product recommendation system called KijiShopping that uses content-based modeling with TF-IDF. It explains how KijiShopping collects user and product data, computes TF-IDF to find useful features, associates words with products using batch MapReduce jobs, determines a user's preferred words, and generates recommendations by combining user ratings and models using producers that access models via key-value stores. The goal is to provide real-time recommendations by leveraging the Apache Kiji framework for real-time analytics.
Drilling into Data with Apache Drill
Drilling into Data with Apache DrillDataWorks Summit Apache Drill is an open source SQL query engine for analyzing data in non-relational data stores like HDFS, HBase, MongoDB, and others. It allows users to query data across these systems using SQL without requiring schemas or transformation of the data. Drill uses a JSON data model and columnar storage to provide fast performance on large datasets. It is optimized to work across distributed systems and enables analysis of complex nested and hierarchical data.
A Non-Standard use Case of Hadoop: High Scale Image Processing and Analytics
A Non-Standard use Case of Hadoop: High Scale Image Processing and AnalyticsDataWorks Summit 1. The Hadoop Image Processing (HIP) pipeline acquires vehicle images, identifies updates, generates URLs, crops and resizes images, copies them to asset servers, and removes duplicates.
2. It uses HBase for image storage and archiving, MapReduce for image processing, Kafka for publishing to asset servers, OpenCV for image processing, and Avro for data serialization.
3. Performance testing showed HIP scales linearly and is at least 10x faster than the previous system, and using cascading downloads provided a 20% performance gain.
La big datacamp2014_vikram_dixit
La big datacamp2014_vikram_dixitData Con LA Big Data Camp LA 2014, Hive 0.13: An upgrade in performance, scaling, security and multi-tenancy by Vikram Dixit of Hortonworks
Big Data Day LA 2015 - HBase at Factual: Real time and Batch Uses by Molly O'...
Big Data Day LA 2015 - HBase at Factual: Real time and Batch Uses by Molly O'...Data Con LA This talk will explore how factual uses some of the advanced features in version .98 and beyond to build a robust and flexible data store.
Ad
More Related Content
What's hot (20)
Hadoop Infrastructure @Uber Past, Present and Future
Hadoop Infrastructure @Uber Past, Present and FutureDataWorks Summit Uber’s mission is to provide transportation as reliable as running water and for fulfilling that mission data plays a critical role. In Uber, Hadoop plays a critical role in Data Infrastructure. We want to talk about the journey of Hadoop @Uber and our future plans in terms of scaling for billions of trips. We will talk about most unique use case Uber have and how Hadoop and eco system which we built, helped us in this journey. We want to talk about how we scaled from 10 -> 2000 and In future to scale up to 10’s X1000 of Nodes. We will talk about our mistakes, learning and wins and how we process billions of events per day. We will talk about the unique challenges and real world use-cases and how we will co-locate the Uber’s service architecture with batch (e.g data pipelines, machine learning and analytical workloads). Uber have done lot of improvements to current Hadoop eco system and uniquely solved some of the problems in a way which is never been solved in the past. This presentation will help audience to use this as an example and even encourage them to enhance the eco system. This will help to increase the community of these project and overall help the whole big data space. Audience is anybody who is working on Big Data and want to understand how to scale Hadoop and eco system for 10s of thousands of node. This talk will help them understand the Hadoop ecosystem and how to efficiently use that. It will also introduce them to some of the awesome technologies which Uber team is building in big data space.
Hadoop World 2011: Data Ingestion, Egression, and Preparation for Hadoop - Sa...
Hadoop World 2011: Data Ingestion, Egression, and Preparation for Hadoop - Sa...Cloudera, Inc. One of the first challenges Hadoop developers face is accessing all the data they need and getting it into Hadoop for analysis. Informatica PowerExchange accesses a variety of data types and structures at different latencies (e.g. batch, real-time, or near real-time) and ingests data directly into Hadoop. The next step is to parse the data in preparation for analysis in Hadoop. Informatica provides a visual IDE to deploy pre-built parsers or design specific parsers for complex data formats and deploy them on Hadoop. Once the analysis is complete, Informatica PowerExhange delivers the resulting output to other information management systems such as a data warehouse. Learn in this session from Informatica and one of their customers, how to get all the data you need into Hadoop, parse a variety of data formats and structures, and egress the resultant output to other systems.
Data Engineer's Lunch #55: Get Started in Data Engineering
Data Engineer's Lunch #55: Get Started in Data EngineeringAnant Corporation In Data Engineer's Lunch #55, CEO of Anant, Rahul Singh, will cover 10 resources every data engineer needs to get started or master their game.
Accompanying Blog: Coming Soon!
Accompanying YouTube: Coming Soon!
Sign Up For Our Newsletter: http://eepurl.com/grdMkn
Join Data Engineer’s Lunch Weekly at 12 PM EST Every Monday:
https://www.meetup.com/Data-Wranglers-DC/events/
Cassandra.Link:
https://cassandra.link/
Follow Us and Reach Us At:
Anant:
https://www.anant.us/
Awesome Cassandra:
https://github.com/Anant/awesome-cassandra
Email:
[email protected]
LinkedIn:
https://www.linkedin.com/company/anant/
Twitter:
https://twitter.com/anantcorp
Eventbrite:
https://www.eventbrite.com/o/anant-1072927283
Facebook:
https://www.facebook.com/AnantCorp/
Join The Anant Team:
https://www.careers.anant.us
Big Data Day LA 2016/ Use Case Driven track - Hydrator: Open Source, Code-Fre...
Big Data Day LA 2016/ Use Case Driven track - Hydrator: Open Source, Code-Fre...Data Con LA This talk will present how to build data pipelines with no code using the open-source, Apache 2.0, Cask Hydrator. The talk will continue with a live demonstration of creating data pipelines for two use cases.
03-NOV-1510-Ognjen-Antonic-Telemach-stream-1
03-NOV-1510-Ognjen-Antonic-Telemach-stream-1Ognjen Antonic This document discusses how Telemach Slovenia leveraged open source big data technologies like Elasticsearch, Logstash, and Kibana (ELK stack) to build analytics dashboards for fraud detection and network monitoring. It summarizes their initial success building a roaming fraud dashboard in 8 days using these technologies. This proof of concept led them to expand usage to additional fraud and network performance dashboards. The ELK stack provided scalable and cost-effective log analytics capabilities compared to commercial options like Splunk. This enabled both IT and business users to gain new visual insights into network operations and issues.
Embeddable data transformation for real time streams
Embeddable data transformation for real time streamsJoey Echeverria This document summarizes Joey Echeverria's presentation on embeddable data transformation for real-time streams. Some key points include:
- Stream processing requires the ability to perform common data transformations like filtering, extracting, projecting, and aggregating on streaming data.
- Tools like Apache Storm, Spark, and Flink can be used to build stream processing topologies and jobs, but also have limitations for embedding transformations.
- Rocana Transform provides a library and DSL for defining reusable data transformation configurations that can be run within different stream processing systems or in batch jobs.
- The library supports common transformations as well as custom actions defined through Java. Configurations can extract metrics, parse logs, and perform
Scaling Deep Learning on Hadoop at LinkedIn
Scaling Deep Learning on Hadoop at LinkedInDataWorks Summit Deep learning has become widespread as frameworks such as TensorFlow and PyTorch have made it easy to onboard machine learning applications. However, while it is easy to start developing with these frameworks on your local developer machine, scaling up a model to run on a cluster and train on huge datasets is still challenging. Code and dependencies have to be copied to every machine and defining the cluster configurations is tedious and error-prone. In addition, troubleshooting errors and aggregating logs is difficult. Ad-hoc solutions also lack resource guarantees, isolation from other jobs, and fault tolerance.
To solve these problems and make scaling deep learning easy, we have made several enhancements to Hadoop and built an open-source deep learning platform called TonY. In this talk, Anthony and Keqiu will discuss new Hadoop features useful for deep learning, such as GPU resource support, and deep dive into TonY, which lets you run deep learning programs natively on Hadoop. We will discuss TonY's architecture and how it allows users to manage their deep learning jobs, acting as a portal from which to launch notebooks, monitor jobs, and visualize training results.
Big Telco - Yousun Jeong
Big Telco - Yousun JeongSpark Summit This document provides an overview of SK Telecom's use of big data analytics and Spark. Some key points:
- SKT collects around 250 TB of data per day which is stored and analyzed using a Hadoop cluster of over 1400 nodes.
- Spark is used for both batch and real-time processing due to its performance benefits over other frameworks. Two main use cases are described: real-time network analytics and a network enterprise data warehouse (DW) built on Spark SQL.
- The network DW consolidates data from over 130 legacy databases to enable thorough analysis of the entire network. Spark SQL, dynamic resource allocation in YARN, and integration with BI tools help meet requirements for timely processing and quick
What database
What databaseRegunath B Slides from my talk at the Hasgeek FifthElephant conference 2017 - a practical guide to database selection
What's new in SQL on Hadoop and Beyond
What's new in SQL on Hadoop and BeyondDataWorks Summit/Hadoop Summit Presto is an open source distributed SQL query engine that allows interactive analysis of data across multiple data stores. At Facebook, Presto is used for ad-hoc queries of their Hadoop data warehouse, which processes trillions of rows and scans petabytes of data daily. Presto's low latency also makes it suitable for powering analytics in user-facing products. New features of Presto include improved SQL support, performance optimizations, and connectors to additional data sources like Redis and MongoDB.
Presto Strata London 2019: Cost-Based Optimizer for interactive SQL on anything
Presto Strata London 2019: Cost-Based Optimizer for interactive SQL on anythingPiotr Findeisen The Presto Cost-Based Optimizer for interactive SQL on anything
Jeremy Engle's slides from Redshift / Big Data meetup on July 13, 2017
Jeremy Engle's slides from Redshift / Big Data meetup on July 13, 2017AWS Chicago "Strategies for supporting near real time analytics, OLAP, and interactive data exploration" - Dr. Jeremy Engle, Engineering Manager Data Team at Jellyvision
Real time fraud detection at 1+M scale on hadoop stack
Real time fraud detection at 1+M scale on hadoop stackDataWorks Summit/Hadoop Summit Rocketfuel processes over 120 billion ad auctions per day and needs to detect fraud in real time to prevent losses. They developed Helios, which ingests event data from Kafka and HDFS into Storm in real time, joins the streams in HBase, then runs MapReduce jobs hourly to populate an OLAP cube for analyzing feature vectors and detecting fraud patterns. This architecture on Hadoop allows them to easily scale real-time processing and experiment with different configurations to quickly react to fraud.
Built-In Security for the Cloud
Built-In Security for the CloudDataWorks Summit Today enterprises desire to move more and more of their data lakes to the cloud to help them execute faster, increase productivity, drive innovation while leveraging the scale and flexibility of the cloud. However, such gains come with risks and challenges in the areas of data security, privacy, and governance. In this talk we cover how enterprises can overcome governance and security obstacles to leverage these new advances that the cloud can provide to ease the management of their data lakes in the cloud. We will also show how the enterprise can have consistent governance and security controls in the cloud for their ephemeral analytic workloads in a multi-cluster cloud environment without sacrificing any of the data security and privacy/compliance needs that their business context demands. Additionally, we will outline some use cases and patterns as well as best practices to rationally manage such a multi-cluster data lake infrastructure in the cloud.
Speaker:
Jeff Sposetti, Product Management, Hortonworks
Reshape Data Lake (as of 2020.07)
Reshape Data Lake (as of 2020.07)Eric Sun SF Big Analytics 2020-07-28
Anecdotal history of Data Lake and various popular implementation framework. Why certain tradeoff was made to solve the problems, such as cloud storage, incremental processing, streaming and batch unification, mutable table, ...
Qubole - Big data in cloud
Qubole - Big data in cloudDmitry Tolpeko Big Data Challenges and How to Overcome Them with Qubole - a Self-Service Platform for Big Data Analytics built on Amazon Web Services, Microsoft and Google Clouds. Storing, accessing, and analyzing large amounts of data from diverse sources and making it easily accessible to deliver actionable insights for users can be challenging for data driven organizations. The solution for customers is to optimize scaling and create a unified interface to simplify analysis. Qubole helps customers simplify their big data analytics with speed and scalability, while providing data analysts and scientists self-service access in Cloud. The platform is fully elastic and automatically scales or contracts clusters based on workload. We will try to overview main features, advantages and drawback of this platform.
Owning Your Own (Data) Lake House
Owning Your Own (Data) Lake HouseData Con LA Data Con LA 2020
Description
In this session, I introduce the Amazon Redshift lake house architecture which enables you to query data across your data warehouse, data lake, and operational databases to gain faster and deeper insights. With a lake house architecture, you can store data in open file formats in your Amazon S3 data lake.
Speaker
Antje Barth, Amazon Web Services, Sr. Developer Advocate, AI and Machine Learning
HBaseCon 2013: Real-Time Model Scoring in Recommender Systems
HBaseCon 2013: Real-Time Model Scoring in Recommender Systems Cloudera, Inc. This document describes a real-time product recommendation system called KijiShopping that uses content-based modeling with TF-IDF. It explains how KijiShopping collects user and product data, computes TF-IDF to find useful features, associates words with products using batch MapReduce jobs, determines a user's preferred words, and generates recommendations by combining user ratings and models using producers that access models via key-value stores. The goal is to provide real-time recommendations by leveraging the Apache Kiji framework for real-time analytics.
Drilling into Data with Apache Drill
Drilling into Data with Apache DrillDataWorks Summit Apache Drill is an open source SQL query engine for analyzing data in non-relational data stores like HDFS, HBase, MongoDB, and others. It allows users to query data across these systems using SQL without requiring schemas or transformation of the data. Drill uses a JSON data model and columnar storage to provide fast performance on large datasets. It is optimized to work across distributed systems and enables analysis of complex nested and hierarchical data.
A Non-Standard use Case of Hadoop: High Scale Image Processing and Analytics
A Non-Standard use Case of Hadoop: High Scale Image Processing and AnalyticsDataWorks Summit 1. The Hadoop Image Processing (HIP) pipeline acquires vehicle images, identifies updates, generates URLs, crops and resizes images, copies them to asset servers, and removes duplicates.
2. It uses HBase for image storage and archiving, MapReduce for image processing, Kafka for publishing to asset servers, OpenCV for image processing, and Avro for data serialization.
3. Performance testing showed HIP scales linearly and is at least 10x faster than the previous system, and using cascading downloads provided a 20% performance gain.
Viewers also liked (20)
La big datacamp2014_vikram_dixit
La big datacamp2014_vikram_dixitData Con LA Big Data Camp LA 2014, Hive 0.13: An upgrade in performance, scaling, security and multi-tenancy by Vikram Dixit of Hortonworks
Big Data Day LA 2015 - HBase at Factual: Real time and Batch Uses by Molly O'...
Big Data Day LA 2015 - HBase at Factual: Real time and Batch Uses by Molly O'...Data Con LA This talk will explore how factual uses some of the advanced features in version .98 and beyond to build a robust and flexible data store.
Aziksa hadoop for buisness users2 santosh jha
Aziksa hadoop for buisness users2 santosh jhaData Con LA This document discusses big data, including its drivers, characteristics, use cases across different industries, and lessons learned. It provides examples of companies like Etsy, Macy's, Canadian Pacific, and Salesforce that are using big data to gain insights, increase revenues, reduce costs and improve customer experiences. Big data is being used across industries like financial services, healthcare, manufacturing, and media/entertainment for applications such as customer profiling, fraud detection, operations optimization, and dynamic pricing. While big data projects show strong financial benefits, the document cautions that not all projects are well-structured and Hadoop alone is not sufficient to meet all business analysis needs.
Kiji cassandra la june 2014 - v02 clint-kelly
Kiji cassandra la june 2014 - v02 clint-kellyData Con LA Big Data Camp LA 2014, Don't re-invent the Big-Data Wheel, Building real-time, Big Data applications on Cassandra with the open-source Kiji project by Clint Kelly of Wibidata
Yarn cloudera-kathleenting061414 kate-ting
Yarn cloudera-kathleenting061414 kate-tingData Con LA This document summarizes Kathleen Ting's presentation on migrating to MapReduce v2 (MRv2) on YARN. The presentation covered the motivation for moving to MRv2 and YARN, including higher cluster utilization and lower costs. It then discussed common misconfiguration issues seen in support tickets, such as memory, thread pool size, and federation misconfigurations. Specific examples were provided for resolving task memory errors, JobTracker memory errors, and fetch failures in both MRv1 and MRv2. Recommendations were given for optimizing YARN memory usage and CPU isolation in containers.
2014 bigdatacamp asya_kamsky
2014 bigdatacamp asya_kamskyData Con LA The document discusses various options for processing and aggregating data in MongoDB, including the Aggregation Framework, MapReduce, and connecting MongoDB to external systems like Hadoop. The Aggregation Framework is described as a flexible way to query and transform data in MongoDB using a JSON-like syntax and pipeline stages. MapReduce is presented as more versatile but also more complex to implement. Connecting to external systems like Hadoop allows processing large amounts of data across clusters.
Summit v4 dave wolcott
Summit v4 dave wolcottData Con LA The document summarizes the Hadoop stack and its components for storing and analyzing big data. It includes file storage with HDFS, data processing with MapReduce, data access tools like Hive and Pig, and security/monitoring with Kerberos and Nagios. HDFS uses metadata to track file locations across data nodes in a fault-tolerant manner similar to a file system.
20140614 introduction to spark-ben white
20140614 introduction to spark-ben whiteData Con LA This document provides an introduction to Apache Spark. It begins by explaining how Spark improves upon MapReduce by leveraging distributed memory for better performance and supporting iterative algorithms. Spark is described as a general purpose computational framework that retains the advantages of MapReduce like scalability and fault tolerance, while offering more functionality through directed acyclic graphs and libraries for machine learning. The document then discusses getting started with Spark and its execution modes like standalone, YARN client, and YARN cluster. Finally, it introduces Spark concepts like Resilient Distributed Datasets (RDDs), which are collections of objects partitioned across a cluster.
140614 bigdatacamp-la-keynote-jon hsieh
140614 bigdatacamp-la-keynote-jon hsiehData Con LA The document discusses the evolution of big data stacks from their origins inspired by Google's systems through imitation via Hadoop-based stacks to ongoing innovation. It traces the development of major components like MapReduce, HDFS, HBase and their adoption beyond Google. It also outlines the timeline of open source projects and companies in this space from 2003 to the present.
Big datacamp june14_alex_liu
Big datacamp june14_alex_liuData Con LA This document discusses how big data analytics can provide insights from large amounts of structured and unstructured data. It provides examples of how big data has helped organizations reduce customer churn, improve customer acquisition, speed up loan approvals, and detect fraud. The document also outlines IBM's big data platform and analytics process for extracting value from large, diverse data sources.
Big Data Day LA 2015 - Solr Search with Spark for Big Data Analytics in Actio...
Big Data Day LA 2015 - Solr Search with Spark for Big Data Analytics in Actio...Data Con LA Apache Solr makes it so easy to interactively visualize and explore your data. Create a dashboard, add some facets, select some values, cross it with the time… and just look at the results.
Apache Spark is the growing framework for performing streaming computations, which makes it ideal for real time indexing.
Solr also comes with new Analytics Facets which are a major weapon added to the arsenal of the data explorer. They bring another dimension: calculations. We can now do the equivalent of SQL, just in a much simpler and faster way. These calculations can operate over buckets of data. For example, it is now possible to see the sum of Web traffic by country over the time, the median price of some categories of products, which ads are bringing more money by location...
This talks puts in practice some of the leading features of Solr Search. It presents the main types of facets/stats and which advanced properties and usage make them shine. A demo in parallel with the open source Search App in Hue will demonstrate how these facets can power interactive widgets or your own analytic queries. The data will be indexed in real time from a live stream with Spark.
Ag big datacampla-06-14-2014-ajay_gopal
Ag big datacampla-06-14-2014-ajay_gopalData Con LA This document provides an overview of CARD.COM, a company that offers prepaid debit cards customized with different designs. They collect data from card transactions, member interactions on their site/app, and marketing platforms to test different designs and better understand customer behavior. Their goal is to use data science to personalize the financial experience for members and potentially offer services like credit scores for the unbanked. They are hiring various technical roles and use open source tools like R, Python, and PHP to build out their analytics platforms and infrastructure.
Hadoop and NoSQL joining forces by Dale Kim of MapR
Hadoop and NoSQL joining forces by Dale Kim of MapRData Con LA More and more organizations are turning to Hadoop and NoSQL to manage big data. In fact, many IT professionals consider each of those terms to be synonymous with big data. At the same time, these two technologies are seen as different beasts that handle different challenges. That means they are often deployed in a rather disjointed way, even when intended to solve the same overarching business problem. The emerging trend of “in-Hadoop databases” promises to narrow the deployment gap between them and enable new enterprise applications. In this talk, Dale will describe that integrated architecture and how customers have deployed it to benefit both the technical and the business teams.
Big Data Day LA 2015 - Lessons Learned from Designing Data Ingest Systems by ...
Big Data Day LA 2015 - Lessons Learned from Designing Data Ingest Systems by ...Data Con LA 1. The document discusses lessons learned from designing data ingest systems. Key lessons include structuring endpoints wisely, accepting at least once semantics, knowing that change data capture is difficult, understanding service level agreements, considering record format and schema, and tracking record lineage.
2. The document also provides examples of real-world data ingest scenarios and different implementation strategies with analyses of their tradeoffs. It concludes with recommendations to track errors and keep transformations minimal.
Hadoop Innovation Summit 2014
Hadoop Innovation Summit 2014Data Con LA Slides from my presentation at Hadoop Innovation Summit 2014
The Future of Hadoop: Choosing the right options
Big Data Day LA 2015 - Deep Learning Human Vocalized Animal Sounds by Sabri S...
Big Data Day LA 2015 - Deep Learning Human Vocalized Animal Sounds by Sabri S...Data Con LA Investigated a couple audio based, deep learning strategies for identifying human vocalized car sounds. In one case Mel Frequency Cepstral Coefficients, MFCCs, were used as inputs into a supervised, logistic regression neural network. In a separate case, Short Term Fourier Transforms ,STFT, were used to generate PCA whitened spectograms, which were used as inputs into a supervised, convolutional neural network. The MFCC method trained quickly on a relative small dataset of 4 sounds. The STFT method resulted in a much larger input matrix, resulting in much longer times for converging onto a solution
Big Data Day LA 2016/ Data Science Track - Decision Making and Lambda Archite...
Big Data Day LA 2016/ Data Science Track - Decision Making and Lambda Archite...Data Con LA This document discusses decision making systems and the lambda architecture. It introduces decision making algorithms like multi-armed bandits that balance exploration vs exploitation. Contextual multi-armed bandits are discussed as well. The lambda architecture is then described as having serving, speed, and batch layers to enable low latency queries, real-time updates, and batch model training. The software stack of Kafka, Spark/Spark Streaming, HBase and MLLib is presented as enabling scalable stream processing and machine learning.
Big Data Day LA 2016/ Hadoop/ Spark/ Kafka track - Introduction to Kafka - Je...
Big Data Day LA 2016/ Hadoop/ Spark/ Kafka track - Introduction to Kafka - Je...Data Con LA Kafka is a distributed publish-subscribe system that uses a commit log to track changes. It was originally created at LinkedIn and open sourced in 2011. Kafka decouples systems and is commonly used in enterprise data flows. The document then demonstrates how Kafka works using Legos and discusses key Kafka concepts like topics, partitioning, and the commit log. It also provides examples of how to create Kafka producers and consumers using the Java API.
Big Data Day LA 2016/ Big Data Track - Twitter Heron @ Scale - Karthik Ramasa...
Big Data Day LA 2016/ Big Data Track - Twitter Heron @ Scale - Karthik Ramasa...Data Con LA Twitter generates billions and billions of events per day. Analyzing these events in real time presents a massive challenge. Twitter designed and deployed a new streaming system called Heron. Heron has been in production nearly 2 years and is widely used by several teams for diverse use cases. This talk looks at Twitter's operating experiences and challenges of running Heron at scale and the approaches taken to solve those challenges.
Big Data Day LA 2016/ Hadoop/ Spark/ Kafka track - Alluxio (formerly Tachyon)...
Big Data Day LA 2016/ Hadoop/ Spark/ Kafka track - Alluxio (formerly Tachyon)...Data Con LA Alluxio, formerly Tachyon, is a memory speed virtual distributed storage system. The Alluxio open source community is one of the fastest growing open source communities in big data history with more than 300 developers from over 100 organizations around the world. In the past year, the Alluxio project experienced a tremendous improvement in performance and scalability and was extended with key new features including tiered storage, transparent naming, and unified namespace. Alluxio now supports a wide range of under storage systems, including Amazon S3, Google Cloud Storage, Gluster, Ceph, HDFS, NFS, and OpenStack Swift. This year, our goal is to make Alluxio accessible to an even wider set of users, through our focus on security, new language bindings, and further increased stability.
Ad
Similar to Big Data Day LA 2015 - NoSQL: Doing it wrong before getting it right by Lawrence Trinh of Fandango (20)
A Scalable Data Transformation Framework using Hadoop Ecosystem
A Scalable Data Transformation Framework using Hadoop EcosystemDataWorks Summit This document summarizes a presentation about Penton's use of Hadoop and related technologies to improve their data processing capabilities. It describes Penton's data challenges with siloed and slow ETL processes. A proof of concept used HBase for ingesting and manipulating CSV files, Drools for data scrubbing rules, and MapReduce jobs for exports. This new architecture improved performance for mapping and uploads. Lessons included HBase tuning and challenges translating SQL queries to scans. The new system provided better scalability and relieved load from their RDBMS.
David Max SATURN 2018 - Migrating from Oracle to Espresso
David Max SATURN 2018 - Migrating from Oracle to EspressoDavid Max This talk is about our experience at LinkedIn migrating our content ingestion system from using Oracle to using our internal database system Espresso. I explain some of the reasons for doing the migration as well as how we met the challenges of swapping database technologies with no down time and in a way that was transparent to our clients.
This talk was delivered at the SATURN 2018 conference in Plano, TX on May 9, 2018.
A Scalable Data Transformation Framework using the Hadoop Ecosystem
A Scalable Data Transformation Framework using the Hadoop EcosystemSerendio Inc. Hadoop-HBase framework to ingest and process TBs of data. A case study in migrating a legacy Oracle implementation to Hadoop.
Apache Hadoop Hive
Apache Hadoop HiveSome corner at the Laboratory The initiation of The Hadoop Apache Hive began in 2007 by Facebook due to its data growth.
This ETL system began to fail over few years as more people joined Facebook.
In August 2008, Facebook decided to move to scalable a more scalable open-source Hadoop environment; Hive
Facebook, Netflix and Amazons support the Apache Hive SQL now known as the HiveQL
Couchbase Chennai meetup #3 What's new in Couchbase Server & Couchbase Mobile
Couchbase Chennai meetup #3 What's new in Couchbase Server & Couchbase MobileKarthik Babu Sekar This session provided an overview of Couchbase Solutions and whats latest and greatest in the new release and product announcement after Couchbase Connect 16 in the official third meetup of Couchbase Chennai
Operations for databases – the agile/devops journey
Operations for databases – the agile/devops journeyEduardo Piairo The presentation discusses the journey from a traditional development process to adopting Agile and DevOps practices for database operations. It covers moving from siloed teams to Scrum and then combining Scrum and Kanban. Key aspects included establishing source control for database scripts, implementing continuous integration and delivery pipelines for databases, and defining a "contract" to facilitate collaboration and communication around database changes. Automating processes, tracking changes, and integrating databases into feature development helped increase delivery speed, reliability, and number of supported customers.
SPTechCon Austin - The Slippery Slope of SharePoint Migrations
SPTechCon Austin - The Slippery Slope of SharePoint MigrationsJill Hannemann This document summarizes a workshop on SharePoint migrations presented by Jill Hannemann and Adam Levithan. It discusses common reasons why SharePoint migrations fail, such as failing to fully scope the effort, budget appropriately, or gain buy-in from stakeholders. The workshop covers how to define the scope of a migration, plan and budget appropriately, and gain buy-in. It also discusses content cleanup and taking advantage of the migration effort to reorganize content.
The Path Through SharePoint Migrations
The Path Through SharePoint MigrationsBrian Caauwe Whether you are planning on upgrading to SharePoint 2016 or moving to the cloud, transitioning from one version to another has many moving parts that tend to bring out the worst in your SharePoint environment. This session will give you information on:
• How to prepare yourself BEFORE you spin up your new environment or buy that Office 365 licensing
• Different migration patterns and why you may want to use these different methods
• How to prepare yourself for life after migration, which won't include updating your resume.
NoSql Brownbag
NoSql BrownbagSandeep Kumar This document provides an introduction to NoSQL databases, including the motivation behind them, where they fit, types of NoSQL databases like key-value, document, columnar, and graph databases, and an example using MongoDB. NoSQL databases are a new way of thinking about data that is non-relational, schema-less, and can be distributed and fault tolerant. They are motivated by the need to scale out applications and handle big data with flexible and modern data models.
Intro to Azure Data Factory v1
Intro to Azure Data Factory v1Eric Bragas An introduction to the concepts and requirements to get started developing data pipelines in Azure Data Factory version 1.
scopp_chris_resume
scopp_chris_resumeChris Scopp The document provides a summary of an IT professional with over 18 years of experience in database administration, development, and web administration using technologies such as Oracle, MySQL, PHP, Laravel, and content management systems. They have experience developing applications, databases, and websites for a variety of clients as a consultant.
scopp_chris_resume
scopp_chris_resumeChris Scopp The document provides a summary of an IT professional with over 18 years of experience in database administration, development, and web administration using technologies such as Oracle, MySQL, PHP, Laravel, and content management systems. They have experience developing applications, databases, and websites for a variety of clients as a consultant.
Using Amazon RDS to power enterprise applications (Peoplesoft) 
Using Amazon RDS to power enterprise applications (Peoplesoft) Tom Laszewski AWS Oracle RDS can be used to run Oracle Peoplesoft. This presentation is done by AWS, DLZP, and Select Family.
Case Study: Implementing Hadoop and Elastic Map Reduce on Scale-out Object S...
Case Study: Implementing Hadoop and Elastic Map Reduce on Scale-out Object S...Cloudian This document discusses implementing Hadoop and Elastic MapReduce on Cloudian's scale-out object storage platform. It describes Cloudian's hybrid cloud storage capabilities and how their approach reduces costs and provides faster analytics by analyzing log and event data directly on their storage platform without needing to transform the data for HDFS. Key benefits highlighted include no redundant storage, scaling analytics with storage capacity by adding nodes, and taking advantage of multi-core CPUs for MapReduce tasks.
Plan, prepare & overall process of upgrade and migrate to SharePoint 2013
Plan, prepare & overall process of upgrade and migrate to SharePoint 2013Kashish Sukhija This Presentation cover step-by-step instructions on how upgrade to SharePoint 2013. Learn how to plan & prepare for Upgrade & Migration. What’s new in SharePoint 2013 for upgrade and what is the upgrade process and best practices. How to clean up an environment before an upgrade, Strategy on Claims Migration and how to prepare for known Issues, Customization Upgrade tips and tricks.
The Future of Data Engineering - 2019 InfoQ QConSF
The Future of Data Engineering - 2019 InfoQ QConSFChris Riccomini The document summarizes the six stages of data pipeline maturity: from no pipeline (Stage 0) to fully decentralized pipelines (Stage 5). It uses the evolution of data infrastructure at WePay as an example, starting with a monolithic MySQL database (Stage 0) and progressing to a real-time integration of data across services using Kafka and automated operations/data management (Stages 3-4). The talk argues that as pipelines become fully automated, data teams should expose self-service tools to decentralize control and management of data to individual engineering teams through micro data warehouses.
Cloud-native Data
Cloud-native Datacornelia davis <November 2017 Updated from earlier presentations on Cloud-native Data>
Cloud-native applications form the foundation for modern, cloud-scale digital solutions, and the patterns and practices for cloud-native at the app tier are becoming widely understood – statelessness, service discovery, circuit breakers and more. But little has changed in the data tier. Our modern apps are often connected to monolithic shared databases that have monolithic practices wrapped around them. As a result, the autonomy promised by moving to a microservices application architecture is compromised.
What we need are patterns and practices for cloud-native data. The anti-patterns of shared databases and simple proxy-style web services to front them give way to approaches that include use of caches (Netflix calls caching their hidden microservice), database per service and polyglot persistence, modern versions of ETL and data integration and more. In this session, aimed at the application developer/architect, Cornelia will look at those patterns and see how they serve the needs of the cloud-native application.
Cloud-Native-Data with Cornelia Davis
Cloud-Native-Data with Cornelia DavisVMware Tanzu This document discusses cloud-native data and patterns for managing data in microservices architectures. It describes using data services and APIs to interface with existing data sources. Patterns like caching data at the edge with various caching strategies are discussed. The document also covers using multiple small databases with each microservice rather than a shared database. Event sourcing and CQRS patterns are presented as ways to integrate data across services. Finally, the impact on roles like database administrators is considered in cloud-native data environments.
Tah 03302015 withendclient
Tah 03302015 withendclientTerry Hendrickson Terry Hendrickson has over 15 years of experience as a software/database developer with expertise in SQL Server, Oracle, Visual Basic, VB.NET, T-SQL, PL/SQL, ETL, Crystal Reports, and other technologies. He has worked on projects involving database development, reporting, ETL, and application development for companies across various industries. Hendrickson holds an Associate's Degree in Computer Information Systems and Business Management.
Introduction to Big Data Technologies: Hadoop/EMR/Map Reduce & Redshift
Introduction to Big Data Technologies: Hadoop/EMR/Map Reduce & RedshiftDataKitchen Boston Data Mining Meetup introduction slides from Big Data Infrastructure workshop - A hands-on introduction
Ad
More from Data Con LA (20)
Data Con LA 2022 Keynotes
Data Con LA 2022 KeynotesData Con LA 1. LAUSD has been developing its enterprise data and reporting capabilities since 2000, with various systems and dashboards launched over the years to provide different types of data and reporting, including student outcomes and achievement reports, individual student records, and teacher/staff data.
2. Current tools include MyData (with over 20 million student records), GetData (with instructional and business data), Whole Child (with academic and wellness data), OpenData, and Executive Dashboards.
3. Upcoming improvements include dashboards for social-emotional learning, physical education, and tools to support the Intensive Diagnostic Education Centers and Black Student Achievement Plan initiatives.
Data Con LA 2022 Keynotes
Data Con LA 2022 KeynotesData Con LA The document discusses the County of Los Angeles' efforts to better coordinate services across various departments by creating an enterprise data platform. It notes that the county serves over 750,000 patients annually through its health systems and oversees many other services related to homelessness, justice, child welfare, and public health. The proposed data platform would create a unified client identifier and data store to integrate client records across departments in order to generate insights, measure outcomes, and improve coordination of services.
Data Con LA 2022 Keynote
Data Con LA 2022 KeynoteData Con LA Fastly is an edge cloud platform provider that aims to upgrade the internet experience by making applications and digital experiences fast, engaging, and secure. It has a global network of 100+ points of presence across 30+ countries serving over 1 trillion daily requests. The presentation discusses how internet requests are handled traditionally versus more modern approaches using an edge cloud platform like Fastly. It emphasizes that the edge must be programmable, deliver general purpose compute anywhere, and provide high reliability, security, and data privacy by default.
Data Con LA 2022 - Startup Showcase
Data Con LA 2022 - Startup ShowcaseData Con LA The document summarizes how Aware Health can save self-insured employers millions of dollars by reducing unnecessary surgeries, imaging, and lost work time for musculoskeletal conditions. It notes that 95% of common spine, wrist, and other surgeries are no more effective than non-surgical treatments. Aware Health uses diagnosis without imaging to prevent chronic pain and has shown real-world savings of $9.78 to $78.66 per member per month for employers, a 96% net promoter score, and over $2 million in annual savings for one enterprise customer.
Data Con LA 2022 Keynote
Data Con LA 2022 KeynoteData Con LA - Project Lightspeed is the next generation of Apache Spark Structured Streaming that aims to provide faster and simpler stream processing with predictable low latency.
- It targets reducing tail latency by up to 2x through faster bookkeeping and offset management. It also enhances functionality with advanced capabilities like new operators and easy to use APIs.
- Project Lightspeed also aims to simplify deployment, operations, monitoring and troubleshooting of streaming applications. It seeks to improve ecosystem support for connectors, authentication and authorization.
- Some specific improvements include faster micro-batch processing, enhancing Python as a first class citizen, and making debugging of streaming jobs easier through visualizations.
Data Con LA 2022 - Using Google trends data to build product recommendations
Data Con LA 2022 - Using Google trends data to build product recommendationsData Con LA Mike Limcaco, Analytics Specialist / Customer Engineer at Google
Measure trends in a particular topic or search term across Google Search across the US down to the city-level. Integrate these data signals into analytic pipelines to drive product, retail, media (video, audio, digital content) recommendations tailored to your audience segment. We'll discuss how Google unique datasets can be used with Google Cloud smart analytic services to process, enrich and surface the most relevant product or content that matches the ever-changing interests of your local customer segment.
Data Con LA 2022 - AI Ethics
Data Con LA 2022 - AI EthicsData Con LA Melinda Thielbar, Data Science Practice Lead and Director of Data Science at Fidelity Investments
From corporations to governments to private individuals, most of the AI community has recognized the growing need to incorporate ethics into the development and maintenance of AI models. Much of the current discussion, though, is meant for leaders and managers. This talk is directed to data scientists, data engineers, ML Ops specialists, and anyone else who is responsible for the hands-on, day-to-day of work building, productionalizing, and maintaining AI models. We'll give a short overview of the business case for why technical AI expertise is critical to developing an AI Ethics strategy. Then we'll discuss the technical problems that cause AI models to behave unethically, how to detect problems at all phases of model development, and the tools and techniques that are available to support technical teams in Ethical AI development.
Data Con LA 2022 - Improving disaster response with machine learning
Data Con LA 2022 - Improving disaster response with machine learningData Con LA Antje Barth, Principal Developer Advocate, AI/ML at AWS & Chris Fregly, Principal Engineer, AI & ML at AWS
The frequency and severity of natural disasters are increasing. In response, governments, businesses, nonprofits, and international organizations are placing more emphasis on disaster preparedness and response. Many organizations are accelerating their efforts to make their data publicly available for others to use. Repositories such as the Registry of Open Data on AWS and Humanitarian Data Exchange contain troves of data available for use by developers, data scientists, and machine learning practitioners. In this session, see how a community of developers came together though the AWS Disaster Response hackathon to build models to support natural disaster preparedness and response.
Data Con LA 2022 - What's new with MongoDB 6.0 and Atlas
Data Con LA 2022 - What's new with MongoDB 6.0 and AtlasData Con LA Sig Narvaez, Executive Solution Architect at MongoDB
MongoDB is now a Developer Data Platform. Come learn what�s new in the 6.0 release and Atlas following all the recent announcements made at MongoDB World 2022. Topics will include
- Atlas Search which combines 3 systems into one (database, search engine, and sync mechanisms) letting you focus on your product's differentiation.
- Atlas Data Federation to seamlessly query, transform, and aggregate data from one or more MongoDB Atlas databases, Atlas Data Lake and AWS S3 buckets
- Queryable Encryption lets you run expressive queries on fully randomized encrypted data to meet the most stringent security requirements
- Relational Migrator which analyzes your existing relational schemas and helps you design a new MongoDB schema.
- And more!
Data Con LA 2022 - Real world consumer segmentation
Data Con LA 2022 - Real world consumer segmentationData Con LA Jaysen Gillespie, Head of Analytics and Data Science at RTB House
1. Shopkick has over 30M downloads, but the userbase is very heterogeneous. Anecdotal evidence indicated a wide variety of users for whom the app holds long-term appeal.
2. Marketing and other teams challenged Analytics to get beyond basic summary statistics and develop a holistic segmentation of the userbase.
3. Shopkick's data science team used SQL and python to gather data, clean data, and then perform a data-driven segmentation using a k-means algorithm.
4. Interpreting the results is more work -- and more fun -- than running the algo itself. We'll discuss how we transform from ""segment 1"", ""segment 2"", etc. to something that non-analytics users (Marketing, Operations, etc.) could actually benefit from.
5. So what? How did team across Shopkick change their approach given what Analytics had discovered.
Data Con LA 2022 - Modernizing Analytics & AI for today's needs: Intuit Turbo...
Data Con LA 2022 - Modernizing Analytics & AI for today's needs: Intuit Turbo...Data Con LA Ravi Pillala, Chief Data Architect & Distinguished Engineer at Intuit
TurboTax is one of the well known consumer software brand which at its peak serves 385K+ concurrent users. In this session, We start with looking at how user behavioral data & tax domain events are captured in real time using the event bus and analyzed to drive real time personalization with various TurboTax data pipelines. We will also look at solutions performing analytics which make use of these events, with the help of Kafka, Apache Flink, Apache Beam, Spark, Amazon S3, Amazon EMR, Redshift, Athena and Amazon lambda functions. Finally, we look at how SageMaker is used to create the TurboTax model to predict if a customer is at risk or needs help.
Data Con LA 2022 - Moving Data at Scale to AWS
Data Con LA 2022 - Moving Data at Scale to AWSData Con LA George Mansoor, Chief Information Systems Officer at California State University
Overview of the CSU Data Architecture on moving on-prem ERP data to the AWS Cloud at scale using Delphix for Data Replication/Virtualization and AWS Data Migration Service (DMS) for data extracts
Data Con LA 2022 - Collaborative Data Exploration using Conversational AI
Data Con LA 2022 - Collaborative Data Exploration using Conversational AIData Con LA Anand Ranganathan, Chief AI Officer at Unscrambl
Conversational AI is getting more and more widely used for customer support and employee support use-cases. In this session, I'm going to talk about how it can be extended for data analysis and data science use-cases ... i.e., how users can interact with a bot to ask analytical questions on data in relational databases.
This allows users to explore complex datasets using a combination of text and voice questions, in natural language, and then get back results in a combination of natural language and visualizations. Furthermore, it allows collaborative exploration of data by a group of users in a channel in platforms like Microsoft Teams, Slack or Google Chat.
For example, a group of users in a channel can ask questions to a bot in plain English like ""How many cases of Covid were there in the last 2 months by state and gender"" or ""Why did the number of deaths from Covid increase in May 2022"", and jointly look at the results that come back. This facilitates data awareness, data-driven collaboration and joint decision making among teams in enterprises and outside.
In this talk, I'll describe how we can bring together various features including natural-language understanding, NL-to-SQL translation, dialog management, data story-telling, semantic modeling of data and augmented analytics to facilitate collaborate exploration of data using conversational AI.
Data Con LA 2022 - Why Database Modernization Makes Your Data Decisions More ...
Data Con LA 2022 - Why Database Modernization Makes Your Data Decisions More ...Data Con LA Anil Inamdar, VP & Head of Data Solutions at Instaclustr
The most modernized enterprises utilize polyglot architecture, applying the best-suited database technologies to each of their organization's particular use cases. To successfully implement such an architecture, though, you need a thorough knowledge of the expansive NoSQL data technologies now available.
Attendees of this Data Con LA presentation will come away with:
-- A solid understanding of the decision-making process that should go into vetting NoSQL technologies and how to plan out their data modernization initiatives and migrations.
-- They will learn the types of functionality that best match the strengths of NoSQL key-value stores, graph databases, columnar databases, document-type databases, time-series databases, and more.
-- Attendees will also understand how to navigate database technology licensing concerns, and to recognize the types of vendors they'll encounter across the NoSQL ecosystem. This includes sniffing out open-core vendors that may advertise as “open source,"" but are driven by a business model that hinges on achieving proprietary lock-in.
-- Attendees will also learn to determine if vendors offer open-code solutions that apply restrictive licensing, or if they support true open source technologies like Hadoop, Cassandra, Kafka, OpenSearch, Redis, Spark, and many more that offer total portability and true freedom of use.
Data Con LA 2022 - Intro to Data Science
Data Con LA 2022 - Intro to Data ScienceData Con LA Zia Khan, Computer Systems Analyst and Data Scientist at LearningFuze
Data Science tutorial is designed for people who are new to Data Science. This is a beginner level session so no prior coding or technical knowledge is required. Just bring your laptop with WiFi capability. The session starts with a review of what is data science, the amount of data we generate and how companies are using that data to get insight. We will pick a business use case, define the data science process, followed by hands-on lab using python and Jupyter notebook. During the hands-on portion we will work with pandas, numpy, matplotlib and sklearn modules and use a machine learning algorithm to approach the business use case.
Data Con LA 2022 - How are NFTs and DeFi Changing Entertainment
Data Con LA 2022 - How are NFTs and DeFi Changing EntertainmentData Con LA Mariana Danilovic, Managing Director at Infiom, LLC
We will address:
(1) Community creation and engagement using tokens and NFTs
(2) Organization of DAO structures and ways to incentivize Web3 communities
(3) DeFi business models applied to Web3 ventures
(4) Why Metaverse matters for new entertainment and community engagement models.
Data Con LA 2022 - Why Data Quality vigilance requires an End-to-End, Automat...
Data Con LA 2022 - Why Data Quality vigilance requires an End-to-End, Automat...Data Con LA Curtis ODell, Global Director Data Integrity at Tricentis
Join me to learn about a new end-to-end data testing approach designed for modern data pipelines that fills dangerous gaps left by traditional data management tools—one designed to handle structured and unstructured data from any source. You'll hear how you can use unique automation technology to reach up to 90 percent test coverage rates and deliver trustworthy analytical and operational data at scale. Several real world use cases from major banks/finance, insurance, health analytics, and Snowflake examples will be presented.
Key Learning Objective
1. Data journeys are complex and you have to ensure integrity of the data end to end across this journey from source to end reporting for compliance
2. Data Management tools do not test data, they profile and monitor at best, and leave serious gaps in your data testing coverage
3. Automation with integration to DevOps and DataOps' CI/CD processes are key to solving this.
4. How this approach has impact in your vertical
Data Con LA 2022-Perfect Viral Ad prediction of Superbowl 2022 using Tease, T...
Data Con LA 2022-Perfect Viral Ad prediction of Superbowl 2022 using Tease, T...Data Con LA 1. The document discusses methods for predicting and engineering viral Super Bowl ads, including a panel-based analysis of video content characteristics and a deep learning model measuring social media effects.
2. It provides examples of ads from Super Bowl 2022 that scored well using these methods, such as BMW and Budweiser ads, and compares predicted viral rankings to actual results.
3. The document also demonstrates how to systematically test, tweak, and target an ad campaign like Bajaj Pulsar's to increase virality through modifications to title, thumbnail, tags and content based on audience feedback.
Data Con LA 2022- Embedding medical journeys with machine learning to improve...
Data Con LA 2022- Embedding medical journeys with machine learning to improve...Data Con LA Jai Bansal, Senior Manager, Data Science at Aetna
This talk describes an internal data product called Member Embeddings that facilitates modeling of member medical journeys with machine learning.
Medical claims are the key data source we use to understand health journeys at Aetna. Claims are the data artifacts that result from our members' interactions with the healthcare system. Claims contain data like the amount the provider billed, the place of service, and provider specialty. The primary medical information in a claim is represented in codes that indicate the diagnoses, procedures, or drugs for which a member was billed. These codes give us a semi-structured view into the medical reason for each claim and so contain rich information about members' health journeys. However, since the codes themselves are categorical and high-dimensional (10K cardinality), it's challenging to extract insight or predictive power directly from the raw codes on a claim.
To transform claim codes into a more useful format for machine learning, we turned to the concept of embeddings. Word embeddings are widely used in natural language processing to provide numeric vector representations of individual words.
We use a similar approach with our claims data. We treat each claim code as a word or token and use embedding algorithms to learn lower-dimensional vector representations that preserve the original high-dimensional semantic meaning.
This process converts the categorical features into dense numeric representations. In our case, we use sequences of anonymized member claim diagnosis, procedure, and drug codes as training data. We tested a variety of algorithms to learn embeddings for each type of claim code.
We found that the trained embeddings showed relationships between codes that were reasonable from the point of view of subject matter experts. In addition, using the embeddings to predict future healthcare-related events outperformed other basic features, making this tool an easy way to improve predictive model performance and save data scientist time.
Data Con LA 2022 - Data Streaming with Kafka
Data Con LA 2022 - Data Streaming with KafkaData Con LA Jie Chen, Manager Advisory, KPMG
Data is the new oil. However, many organizations have fragmented data in siloed line of businesses. In this topic, we will focus on identifying the legacy patterns and their limitations and introducing the new patterns packed by Kafka's core design ideas. The goal is to tirelessly pursue better solutions for organizations to overcome the bottleneck in data pipelines and modernize the digital assets for ready to scale their businesses. In summary, we will walk through three uses cases, recommend Dos and Donts, Take aways for Data Engineers, Data Scientist, Data architect in developing forefront data oriented skills.
Recently uploaded (20)
Quantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur MorganArthur Morgan This is a Quick Research Guide (QRG).
QRGs include the following:
- A brief, high-level overview of the QRG topic.
- A milestone timeline for the QRG topic.
- Links to various free online resource materials to provide a deeper dive into the QRG topic.
- Conclusion and a recommendation for at least two books available in the SJPL system on the QRG topic.
QRGs planned for the series:
- Artificial Intelligence QRG
- Quantum Computing QRG
- Big Data Analytics QRG
- Spacecraft Guidance, Navigation & Control QRG (coming 2026)
- UK Home Computing & The Birth of ARM QRG (coming 2027)
Any questions or comments?
- Please contact Arthur Morgan at [email protected].
100% human made.
Special Meetup Edition - TDX Bengaluru Meetup #52.pptx
Special Meetup Edition - TDX Bengaluru Meetup #52.pptxshyamraj55 We’re bringing the TDX energy to our community with 2 power-packed sessions:
🛠️ Workshop: MuleSoft for Agentforce
Explore the new version of our hands-on workshop featuring the latest Topic Center and API Catalog updates.
📄 Talk: Power Up Document Processing
Dive into smart automation with MuleSoft IDP, NLP, and Einstein AI for intelligent document workflows.
UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager API
UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager APIUiPathCommunity Join this UiPath Community Berlin meetup to explore the Orchestrator API, Swagger interface, and the Test Manager API. Learn how to leverage these tools to streamline automation, enhance testing, and integrate more efficiently with UiPath. Perfect for developers, testers, and automation enthusiasts!
📕 Agenda
Welcome & Introductions
Orchestrator API Overview
Exploring the Swagger Interface
Test Manager API Highlights
Streamlining Automation & Testing with APIs (Demo)
Q&A and Open Discussion
Perfect for developers, testers, and automation enthusiasts!
👉 Join our UiPath Community Berlin chapter: https://community.uipath.com/berlin/
This session streamed live on April 29, 2025, 18:00 CET.
Check out all our upcoming UiPath Community sessions at https://community.uipath.com/events/.
Big Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur MorganArthur Morgan This is a Quick Research Guide (QRG).
QRGs include the following:
- A brief, high-level overview of the QRG topic.
- A milestone timeline for the QRG topic.
- Links to various free online resource materials to provide a deeper dive into the QRG topic.
- Conclusion and a recommendation for at least two books available in the SJPL system on the QRG topic.
QRGs planned for the series:
- Artificial Intelligence QRG
- Quantum Computing QRG
- Big Data Analytics QRG
- Spacecraft Guidance, Navigation & Control QRG (coming 2026)
- UK Home Computing & The Birth of ARM QRG (coming 2027)
Any questions or comments?
- Please contact Arthur Morgan at [email protected].
100% human made.
Technology Trends in 2025: AI and Big Data Analytics
Technology Trends in 2025: AI and Big Data AnalyticsInData Labs At InData Labs, we have been keeping an ear to the ground, looking out for AI-enabled digital transformation trends coming our way in 2025. Our report will provide a look into the technology landscape of the future, including:
-Artificial Intelligence Market Overview
-Strategies for AI Adoption in 2025
-Anticipated drivers of AI adoption and transformative technologies
-Benefits of AI and Big data for your business
-Tips on how to prepare your business for innovation
-AI and data privacy: Strategies for securing data privacy in AI models, etc.
Download your free copy nowand implement the key findings to improve your business.
AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...
AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...SOFTTECHHUB I started my online journey with several hosting services before stumbling upon Ai EngineHost. At first, the idea of paying one fee and getting lifetime access seemed too good to pass up. The platform is built on reliable US-based servers, ensuring your projects run at high speeds and remain safe. Let me take you step by step through its benefits and features as I explain why this hosting solution is a perfect fit for digital entrepreneurs.
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...Aqusag Technologies In late April 2025, a significant portion of Europe, particularly Spain, Portugal, and parts of southern France, experienced widespread, rolling power outages that continue to affect millions of residents, businesses, and infrastructure systems.
TrsLabs - Fintech Product & Business Consulting
TrsLabs - Fintech Product & Business ConsultingTrs Labs Hybrid Growth Mandate Model with TrsLabs
Strategic Investments, Inorganic Growth, Business Model Pivoting are critical activities that business don't do/change everyday. In cases like this, it may benefit your business to choose a temporary external consultant.
An unbiased plan driven by clearcut deliverables, market dynamics and without the influence of your internal office equations empower business leaders to make right choices.
Getting things done within a budget within a timeframe is key to Growing Business - No matter whether you are a start-up or a big company
Talk to us & Unlock the competitive advantage
Complete Guide to Advanced Logistics Management Software in Riyadh.pdf
Complete Guide to Advanced Logistics Management Software in Riyadh.pdfSoftware Company Explore the benefits and features of advanced logistics management software for businesses in Riyadh. This guide delves into the latest technologies, from real-time tracking and route optimization to warehouse management and inventory control, helping businesses streamline their logistics operations and reduce costs. Learn how implementing the right software solution can enhance efficiency, improve customer satisfaction, and provide a competitive edge in the growing logistics sector of Riyadh.
Procurement Insights Cost To Value Guide.pptx
Procurement Insights Cost To Value Guide.pptxJon Hansen Procurement Insights integrated Historic Procurement Industry Archives, serves as a powerful complement — not a competitor — to other procurement industry firms. It fills critical gaps in depth, agility, and contextual insight that most traditional analyst and association models overlook.
Learn more about this value- driven proprietary service offering here.
Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...
Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...BookNet Canada Book industry standards are evolving rapidly. In the first part of this session, we’ll share an overview of key developments from 2024 and the early months of 2025. Then, BookNet’s resident standards expert, Tom Richardson, and CEO, Lauren Stewart, have a forward-looking conversation about what’s next.
Link to recording, presentation slides, and accompanying resource: https://bnctechforum.ca/sessions/standardsgoals-for-2025-standards-certification-roundup/
Presented by BookNet Canada on May 6, 2025 with support from the Department of Canadian Heritage.
Semantic Cultivators : The Critical Future Role to Enable AI
Semantic Cultivators : The Critical Future Role to Enable AIartmondano By 2026, AI agents will consume 10x more enterprise data than humans, but with none of the contextual understanding that prevents catastrophic misinterpretations.
AI and Data Privacy in 2025: Global Trends
AI and Data Privacy in 2025: Global TrendsInData Labs In this infographic, we explore how businesses can implement effective governance frameworks to address AI data privacy. Understanding it is crucial for developing effective strategies that ensure compliance, safeguard customer trust, and leverage AI responsibly. Equip yourself with insights that can drive informed decision-making and position your organization for success in the future of data privacy.
This infographic contains:
-AI and data privacy: Key findings
-Statistics on AI data privacy in the today’s world
-Tips on how to overcome data privacy challenges
-Benefits of AI data security investments.
Keep up-to-date on how AI is reshaping privacy standards and what this entails for both individuals and organizations.
What is Model Context Protocol(MCP) - The new technology for communication bw...
What is Model Context Protocol(MCP) - The new technology for communication bw...Vishnu Singh Chundawat The MCP (Model Context Protocol) is a framework designed to manage context and interaction within complex systems. This SlideShare presentation will provide a detailed overview of the MCP Model, its applications, and how it plays a crucial role in improving communication and decision-making in distributed systems. We will explore the key concepts behind the protocol, including the importance of context, data management, and how this model enhances system adaptability and responsiveness. Ideal for software developers, system architects, and IT professionals, this presentation will offer valuable insights into how the MCP Model can streamline workflows, improve efficiency, and create more intuitive systems for a wide range of use cases.
Cyber Awareness overview for 2025 month of security
Cyber Awareness overview for 2025 month of securityriccardosl1 Cyber awareness training educates employees on risk associated with internet and malicious emails
Role of Data Annotation Services in AI-Powered Manufacturing
Role of Data Annotation Services in AI-Powered ManufacturingAndrew Leo From predictive maintenance to robotic automation, AI is driving the future of manufacturing. But without high-quality annotated data, even the smartest models fall short.
Discover how data annotation services are powering accuracy, safety, and efficiency in AI-driven manufacturing systems.
Precision in data labeling = Precision on the production floor.
Mobile App Development Company in Saudi Arabia
Mobile App Development Company in Saudi ArabiaSteve Jonas EmizenTech is a globally recognized software development company, proudly serving businesses since 2013. With over 11+ years of industry experience and a team of 200+ skilled professionals, we have successfully delivered 1200+ projects across various sectors. As a leading Mobile App Development Company In Saudi Arabia we offer end-to-end solutions for iOS, Android, and cross-platform applications. Our apps are known for their user-friendly interfaces, scalability, high performance, and strong security features. We tailor each mobile application to meet the unique needs of different industries, ensuring a seamless user experience. EmizenTech is committed to turning your vision into a powerful digital product that drives growth, innovation, and long-term success in the competitive mobile landscape of Saudi Arabia.
#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025
#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025BookNet Canada Book industry standards are evolving rapidly. In the first part of this session, we’ll share an overview of key developments from 2024 and the early months of 2025. Then, BookNet’s resident standards expert, Tom Richardson, and CEO, Lauren Stewart, have a forward-looking conversation about what’s next.
Link to recording, transcript, and accompanying resource: https://bnctechforum.ca/sessions/standardsgoals-for-2025-standards-certification-roundup/
Presented by BookNet Canada on May 6, 2025 with support from the Department of Canadian Heritage.
Heap, Types of Heap, Insertion and Deletion
Heap, Types of Heap, Insertion and DeletionJaydeep Kale This pdf will explain what is heap, its type, insertion and deletion in heap and Heap sort
What is Model Context Protocol(MCP) - The new technology for communication bw...
What is Model Context Protocol(MCP) - The new technology for communication bw...Vishnu Singh Chundawat
Big Data Day LA 2015 - NoSQL: Doing it wrong before getting it right by Lawrence Trinh of Fandango
- 1. NOSQL: DOING IT WRONG BEFORE GETTING IT RIGHT 6-27-2015/Lawrence Trinh
- 2. Content
- 3. Goals • Accommodate new products – Content – Marketing – Schema changes • Start using Node.js for services and web tier • Move content out to AWS – Improve abilities to scale up/down
- 4. Background • Web App – Host website (www.fandango.com) • Backoffice tools – Map data – Modify existing content • SQL Server – Store data
- 5. Enter Couchbase • CDC – Monitor SQL Server changes • Export – Grabs changes and pushes to • Import – Contains rules to load data • Custom query language – Sit in front of couchbase to pull/join data
- 6. Challenges • Rules – Large number /data type – Import grows quickly • Breaking points – Lots of places things can go wrong • Second source of truth – New tools write directly to Couchbase
- 7. Enter Microservices • Data store – Simplified flow – Centralized • Flexibility – (Re)building should be less risky • Service ownership – Easily distributable
- 8. Q&A