The Evolving Landscape of Data Engineering
0 likes545 views
Data Engineering is a relatively new, but fast evolving discipline that spans multiple environments and technologies, from traditional data centers to hyper-scale cloud providers, a discipline that combines closed-source, homegrown and open source software to create scalable data pipelines and power incredible new product features. In this presentation, we will go over the last 5-10 years of technology trends and advancements and bring all of that together in a story about modern day Data Engineering and the magic behind it.
1 of 9
Download to read offline
Ad
Recommended
Big Data on EC2: Mashing Technology in the Cloud
Big Data on EC2: Mashing Technology in the CloudGeorge Ang This document discusses how a startup serving widgets on popular online publications scaled their infrastructure using Amazon Web Services to handle spikes in traffic from over 1 billion users sharing over 10 billion URLs. They used a hub-and-spoke architecture with components like Cascading, Amazon Elastic MapReduce, and AsterData to analyze user sharing patterns in a cost-effective and horizontally scalable way.
Metrics for Web Applications - Netcamp 2012
Metrics for Web Applications - Netcamp 2012Andrei Savu A short presentation designed to be a short overview on relevant business metrics for web applications (SaaS).
Edge processing and High Level Visualization versus Cloud computing and Kibana
Edge processing and High Level Visualization versus Cloud computing and KibanaStefano Milani Check out our Hackster post: https://www.hackster.io/159824/edgent-and-high-level-visualization-vs-cloud-and-kibana-c9407b
Netflix Big Data Paris 2017
Netflix Big Data Paris 2017Jason Flittner How "Stranger Things" can happen with Visual Analytics at Netflix. This is the presentation given by Jason Flittner at Big Data Paris 2017.
Using BigQuery as a main Big Data solution
Using BigQuery as a main Big Data solutionNikolay Novozhilov Nikolay Novozhilov of Wego.com introduces BigQuery, Google's service for interactive analysis of massive datasets, as Wego's main Big Data solution. Wego was founded in 2005 in Singapore as Asia Pacific and the Middle East's leading flight/hotel metasearch engine. BigQuery allows querying of billions of rows in seconds, uses a SQL-style query syntax, and pays only for resources used. Novozhilov discusses replacing MySQL with BigQuery and addresses concerns about data security, pricing, and open-source alternatives.
Big data groningen
Big data groningenWillem Hendriks This document discusses Apache Spark, a framework for large-scale data processing. It encourages attendees to learn Spark through free online resources and IBM's partnership with universities. It also presents a toy example analyzing Dutch police and sales data to see if item advertisements predict crime in neighborhoods. The document promotes IBM's goal of educating 1 million data scientists on Spark and announces an upcoming Spark hackathon in the Netherlands.
Realtime stream analytics with momentum
Realtime stream analytics with momentumShamshad Ansari Momentum provides easy to use platform for processing large volume of data streams in realtime. This is an ideal solution for IoT and click stream analytics
Spark Summit West 2017: Real-Time Image Recognition with MemSQL and Spark
Spark Summit West 2017: Real-Time Image Recognition with MemSQL and SparkSingleStore Spark Summit West 2017: Real-Time Image Recognition with MemSQL and Spark
Nikita Shamgunov, CTO, MemSQL
Big Data and ML on Google Cloud
Big Data and ML on Google CloudWlodek Bielski This document discusses Google Cloud Platform tools for big data and machine learning. It provides an overview of BigQuery for cloud data warehousing and big data analytics. It also covers managed Hadoop and data processing tools like Cloud Composer, Dataproc, and Dataflow. Finally, it discusses Google's machine learning tools ranging from high-level APIs to Cloud ML Engine for building custom models and AutoML for automating the machine learning process.
Hw09 Counting And Clustering And Other Data Tricks
Hw09 Counting And Clustering And Other Data TricksCloudera, Inc. This document summarizes the early use of Hadoop at The New York Times to generate PDFs of archived newspaper articles and analyze web traffic data. It describes how over 100 Amazon EC2 instances were used with Hadoop to pre-generate over 11 million PDFs from 4.3TB of source data in under 24 hours for a total cost of $240. The document then discusses how the Times began using Hadoop to perform web analytics, counting page views and unique users, and merging this data with demographic and article metadata to better understand user traffic and behavior.
Hadoop World - Oct 2009
Hadoop World - Oct 2009Derek Gottfrid Review of the different things that nytimes.com has been up to w/ Hadoop from the simple to the less simple.
Big data groningen
Big data groningenWillem Hendriks This document discusses Apache Spark, a framework for large-scale data processing. It summarizes that Spark is fast, has a nice library, and may be easy to use quickly. It provides an example of combining police reports and advertisements to see if thieves prefer certain neighborhoods or items. The document encourages attending an upcoming IBM Spark hackathon and learning Spark through free online resources and IBM's goal of educating one million data scientists on Spark.
Momentum v2.0
Momentum v2.0Shamshad Ansari Accure Analytics is a big data analytics company that has developed Momentum, a big data analytics platform for machine learning, natural language processing, and IoT. Momentum allows enterprises to perform advanced analytics on large volumes of data faster and at a lower cost compared to traditional data management systems. It provides benefits such as superfast data processing, built-in analytics functions, ability to join disparate data sources, and easy custom analytics in Java or R. Momentum has been implemented across industries such as healthcare, banking, retail, and more to help customers derive meaningful insights from their complex data.
Real-Time Supply Chain Analytics with Machine Learning, Kafka, and Spark
Real-Time Supply Chain Analytics with Machine Learning, Kafka, and SparkSingleStore This document discusses real-time supply chain analytics using machine learning, Kafka, and Spark. It outlines four key requirements for real-time supply chain databases: supporting massive data ingestion, serving as a system of record while providing real-time analytics, integrating with familiar ecosystems, and allowing for online scaling. The document then introduces MemSQL as a database platform that can meet these requirements using an in-memory approach. It provides an example called MemEx that combines MemSQL, Kafka, and Spark with machine learning for global supply chain management and real-time predictive analytics.
Atmosphere Conference 2015: Oktawave Horizon Project: the future of real-time...
Atmosphere Conference 2015: Oktawave Horizon Project: the future of real-time...PROIDEA Speaker: Maciej Kuzniar
Language: English
The presentation highlights the foundation for a new project of Polish cloud: Oktawave Horizon. This project aims to provide access to efficient, scalable and cost-optimized set of services which form a platform allowing real-time stream data processing.
Visit our website: http://atmosphere-conference.com/
End To End Business Intelligence On Google Cloud
End To End Business Intelligence On Google CloudTu Pham This document summarizes Tu Pham's presentation on building an end-to-end business intelligence system on Google Cloud. It describes collecting raw user data from partners using Compute Engine, processing the data into Apache Parquet files, storing the files in Cloud Storage, and analyzing the data using tools like DataPrep, BigQuery, and Grafana. The system aggregates data to calculate metrics like unique users per topic and average user engagement. Tu Pham emphasizes principles like keeping things simple, separating realtime and batch workflows, and optimizing costs.
Google cloud
Google cloudvipin ojha Google Cloud Platform provides cloud computing services such as compute power, database storage, applications, and other IT services via remote servers accessed over the internet. It launched services starting in 2008 and has grown to include compute engine, storage, networking, databases, machine learning, and more. Pricing is based on usage with free tiers and discounts for heavy usage to reduce costs compared to maintaining own hardware.
re:Invent re:Peat
re:Invent re:PeatSteve Houël The document discusses various updates and new services announced at AWS re:Invent 2018 across analytics, databases, storage, AI/ML, security, monitoring, hybrid cloud, serverless computing, and managed services. Key releases included Amazon Timestream, Amazon Quantum Ledger Database, Amazon Comprehend for Healthcare, Amazon Forecast, Amazon Textract, Amazon Personalize, Transit Gateway, AWS Outposts, DynamoDB Transactions, Lambda programming languages and code sharing, AWS DeepRacer, and Amazon Managed Streaming for Kafka.
PowerStream: Propelling Energy Innovation with Predictive Analytics 
PowerStream: Propelling Energy Innovation with Predictive Analytics SingleStore This document discusses a presentation about MemSQL PowerStream, a product for predicting the global health of wind turbines. The presentation covers renewable energy news stories, introduces PowerStream, demonstrates high-speed data ingestion and predictive analytics using MemSQL and Spark, and shows how SQL queries can be pushed down to MemSQL for faster processing. It concludes with a question and answer section.
Autonomous analytics on streaming data
Autonomous analytics on streaming dataClaudiu Barbura This document describes an autonomous analytics platform that allows users to analyze streaming data. The platform uses a unified big data technology stack including Spark, Cassandra, Hadoop, Kafka and Elasticsearch. It has a cloud-agnostic architecture and supports multiple machine learning frameworks. The platform includes a Domain Specific Language (DSL) that allows power users to create full data pipelines and analytics workflows with a few lines of code. It also includes a DSL Workbench for interactively building, editing and publishing analytical pipelines. Additionally, the document introduces "Auto Curious", which harnesses user interactions to autonomously discover insights and compose DSL commands through a question graph interface.
R, Spark, Tensorflow, H20.ai Applied to Streaming Analytics
R, Spark, Tensorflow, H20.ai Applied to Streaming AnalyticsKai Wähner Slides from my talk at Codemotion Rome in March 2017. Development of analytic machine learning / deep learning models with R, Apache Spark ML, Tensorflow, H2O.ai, RapidMinder, KNIME and TIBCO Spotfire. Deployment to real time event processing / stream processing / streaming analytics engines like Apache Spark Streaming, Apache Flink, Kafka Streams, TIBCO StreamBase.
Satwik resume
Satwik resumeSatwik Mishra Satwik Mishra has pursued a Master's in Computer Science from Rochester Institute of Technology with a CGPA of 3.89/4 and has a Bachelor's in Information Technology from Manipal Institute of Technology with a CGPA of 8.29/10. He has work experience as a Software Engineer Intern at Trimble Inc. and as a Data Science Intern at Apollo Munich Health Insurance. His technical skills include programming languages like Python, Java, C++, JavaScript, and frameworks like ROS, TensorFlow, PyTorch and databases like MySQL, PostgreSQL and MongoDB. Some of his projects include handwritten math symbol classification, automated canonical cover computation, synchronized producer-consumer across servers and lung cancer detection using deep learning.
Cloud computer
Cloud computerRajesh Kumar Cloud computing is defined as computing resources and application services delivered over networks from massive collections of servers and storage devices. It is a model that enables on-demand access to a shared pool of configurable computing resources like networks, servers, storage and applications. Cloud computing provides five essential characteristics: virtualization, multi-tenancy, real-time provisioning, pay-per-use, and delivery of computing as a service. It discusses cloud computing infrastructure, developer tools, and the impact on the consulting industry. It also provides a demo of the Apache Hadoop MapReduce framework.
Microservices Live
Microservices LiveData Driven Innovation Glynn Bird - Building the "microservices way" involves breaking monolithic IT systems into small, decoupled services that each to one job well. This talk builds a practical microservices architecture during the talk using small Node.js apps that perform storage, analytics and visualisation tasks. Learn how to orchestrate your own microservice architecture using simple, easily-tested building blocks.
Cloud Capacity Planning Tooling - South Bay SRE Meetup Aug-09-2016
Cloud Capacity Planning Tooling - South Bay SRE Meetup Aug-09-2016Coburn Watson The document summarizes Netflix's cloud capacity planning operations. It notes that Netflix tracks over 2,400 applications running on 80 instance types deployed across 30 zones and 60 accounts in AWS. To provide transparency and actionable insights at scale, Netflix built interactive dashboards and focuses on deviations and trends. Its tool called Picsou comprehensively monitors Netflix's current cloud footprint, forecasts future capacity needs, and optimizes on-demand and reserved capacity for cost and reliability. Going forward, Picsou will provide capacity alerts, forecasts, and recommendations to optimize reserved capacity usage.
Automate scalable micro services with Kubernetes on Google Cloud by Billme , ...
Automate scalable micro services with Kubernetes on Google Cloud by Billme , ...TeK Charnsilp Chinprasert The document discusses automating scalable microservices with Kubernetes on Google Cloud. It introduces Kubernetes and how it can be used to run jobs on Google Cloud with microservices split into pools for the API and jobs. This allows scaling the jobs independently while ensuring high availability of the API through Kubernetes features like node selection and node pools.
Near-real time reporting in Azure | 2017 Cloudbrew Radu Vunvulea
Near-real time reporting in Azure | 2017 Cloudbrew Radu VunvuleaRadu Vunvulea One of the most common requirement on a projects nowadays is real time monitoring and reporting. Easy to say, expensive to implement and complex to maintain. In this session we'll take a look on the Azure Services that enable us to fulfil this requirements with minimal effort and with maximum benefits. We have on our radar services like Azure Time Series Insights, Analysis Services and PowerBI. After this session you will know what are the services can be used for near-real time and what does near-real time means in the real world.
PhD Projects in Green Cloud Computing Research Guidance
PhD Projects in Green Cloud Computing Research GuidancePhD Services Pertinent Concepts in Green Cloud Computing
Typical Topics in Green Cloud Computing
Foremost Subjects in Green Cloud Computing
The Evolving Landscape of Data Engineering
The Evolving Landscape of Data EngineeringAndrei Savu The document discusses the evolving landscape of data engineering. It provides context on the past, present, and future of data engineering. Specifically, it notes that in the past, data engineering was driven by open source communities and the early histories of AWS and Google Cloud. It describes common present-day patterns like serverless architectures and data locality. Finally, it outlines a future wish list, including data catalogs, monitoring systems, and more intelligent data infrastructure. The document concludes by offering recommendations on where to start with technologies, Google Cloud courses, and developing domain knowledge.
云计算及其应用
云计算及其应用lantianlcdx The document discusses cloud computing technology and applications. It provides an introduction to cloud computing concepts, distributed systems, MapReduce, and technologies like Google File System, BigTable and AppEngine. It then outlines the syllabus for a cloud computing course, including topics on virtualization, data centers, and guest lectures. Project presentations will account for 60% of the grading.
Ad
More Related Content
What's hot (20)
Big Data and ML on Google Cloud
Big Data and ML on Google CloudWlodek Bielski This document discusses Google Cloud Platform tools for big data and machine learning. It provides an overview of BigQuery for cloud data warehousing and big data analytics. It also covers managed Hadoop and data processing tools like Cloud Composer, Dataproc, and Dataflow. Finally, it discusses Google's machine learning tools ranging from high-level APIs to Cloud ML Engine for building custom models and AutoML for automating the machine learning process.
Hw09 Counting And Clustering And Other Data Tricks
Hw09 Counting And Clustering And Other Data TricksCloudera, Inc. This document summarizes the early use of Hadoop at The New York Times to generate PDFs of archived newspaper articles and analyze web traffic data. It describes how over 100 Amazon EC2 instances were used with Hadoop to pre-generate over 11 million PDFs from 4.3TB of source data in under 24 hours for a total cost of $240. The document then discusses how the Times began using Hadoop to perform web analytics, counting page views and unique users, and merging this data with demographic and article metadata to better understand user traffic and behavior.
Hadoop World - Oct 2009
Hadoop World - Oct 2009Derek Gottfrid Review of the different things that nytimes.com has been up to w/ Hadoop from the simple to the less simple.
Big data groningen
Big data groningenWillem Hendriks This document discusses Apache Spark, a framework for large-scale data processing. It summarizes that Spark is fast, has a nice library, and may be easy to use quickly. It provides an example of combining police reports and advertisements to see if thieves prefer certain neighborhoods or items. The document encourages attending an upcoming IBM Spark hackathon and learning Spark through free online resources and IBM's goal of educating one million data scientists on Spark.
Momentum v2.0
Momentum v2.0Shamshad Ansari Accure Analytics is a big data analytics company that has developed Momentum, a big data analytics platform for machine learning, natural language processing, and IoT. Momentum allows enterprises to perform advanced analytics on large volumes of data faster and at a lower cost compared to traditional data management systems. It provides benefits such as superfast data processing, built-in analytics functions, ability to join disparate data sources, and easy custom analytics in Java or R. Momentum has been implemented across industries such as healthcare, banking, retail, and more to help customers derive meaningful insights from their complex data.
Real-Time Supply Chain Analytics with Machine Learning, Kafka, and Spark
Real-Time Supply Chain Analytics with Machine Learning, Kafka, and SparkSingleStore This document discusses real-time supply chain analytics using machine learning, Kafka, and Spark. It outlines four key requirements for real-time supply chain databases: supporting massive data ingestion, serving as a system of record while providing real-time analytics, integrating with familiar ecosystems, and allowing for online scaling. The document then introduces MemSQL as a database platform that can meet these requirements using an in-memory approach. It provides an example called MemEx that combines MemSQL, Kafka, and Spark with machine learning for global supply chain management and real-time predictive analytics.
Atmosphere Conference 2015: Oktawave Horizon Project: the future of real-time...
Atmosphere Conference 2015: Oktawave Horizon Project: the future of real-time...PROIDEA Speaker: Maciej Kuzniar
Language: English
The presentation highlights the foundation for a new project of Polish cloud: Oktawave Horizon. This project aims to provide access to efficient, scalable and cost-optimized set of services which form a platform allowing real-time stream data processing.
Visit our website: http://atmosphere-conference.com/
End To End Business Intelligence On Google Cloud
End To End Business Intelligence On Google CloudTu Pham This document summarizes Tu Pham's presentation on building an end-to-end business intelligence system on Google Cloud. It describes collecting raw user data from partners using Compute Engine, processing the data into Apache Parquet files, storing the files in Cloud Storage, and analyzing the data using tools like DataPrep, BigQuery, and Grafana. The system aggregates data to calculate metrics like unique users per topic and average user engagement. Tu Pham emphasizes principles like keeping things simple, separating realtime and batch workflows, and optimizing costs.
Google cloud
Google cloudvipin ojha Google Cloud Platform provides cloud computing services such as compute power, database storage, applications, and other IT services via remote servers accessed over the internet. It launched services starting in 2008 and has grown to include compute engine, storage, networking, databases, machine learning, and more. Pricing is based on usage with free tiers and discounts for heavy usage to reduce costs compared to maintaining own hardware.
re:Invent re:Peat
re:Invent re:PeatSteve Houël The document discusses various updates and new services announced at AWS re:Invent 2018 across analytics, databases, storage, AI/ML, security, monitoring, hybrid cloud, serverless computing, and managed services. Key releases included Amazon Timestream, Amazon Quantum Ledger Database, Amazon Comprehend for Healthcare, Amazon Forecast, Amazon Textract, Amazon Personalize, Transit Gateway, AWS Outposts, DynamoDB Transactions, Lambda programming languages and code sharing, AWS DeepRacer, and Amazon Managed Streaming for Kafka.
PowerStream: Propelling Energy Innovation with Predictive Analytics
PowerStream: Propelling Energy Innovation with Predictive Analytics SingleStore This document discusses a presentation about MemSQL PowerStream, a product for predicting the global health of wind turbines. The presentation covers renewable energy news stories, introduces PowerStream, demonstrates high-speed data ingestion and predictive analytics using MemSQL and Spark, and shows how SQL queries can be pushed down to MemSQL for faster processing. It concludes with a question and answer section.
Autonomous analytics on streaming data
Autonomous analytics on streaming dataClaudiu Barbura This document describes an autonomous analytics platform that allows users to analyze streaming data. The platform uses a unified big data technology stack including Spark, Cassandra, Hadoop, Kafka and Elasticsearch. It has a cloud-agnostic architecture and supports multiple machine learning frameworks. The platform includes a Domain Specific Language (DSL) that allows power users to create full data pipelines and analytics workflows with a few lines of code. It also includes a DSL Workbench for interactively building, editing and publishing analytical pipelines. Additionally, the document introduces "Auto Curious", which harnesses user interactions to autonomously discover insights and compose DSL commands through a question graph interface.
R, Spark, Tensorflow, H20.ai Applied to Streaming Analytics
R, Spark, Tensorflow, H20.ai Applied to Streaming AnalyticsKai Wähner Slides from my talk at Codemotion Rome in March 2017. Development of analytic machine learning / deep learning models with R, Apache Spark ML, Tensorflow, H2O.ai, RapidMinder, KNIME and TIBCO Spotfire. Deployment to real time event processing / stream processing / streaming analytics engines like Apache Spark Streaming, Apache Flink, Kafka Streams, TIBCO StreamBase.
Satwik resume
Satwik resumeSatwik Mishra Satwik Mishra has pursued a Master's in Computer Science from Rochester Institute of Technology with a CGPA of 3.89/4 and has a Bachelor's in Information Technology from Manipal Institute of Technology with a CGPA of 8.29/10. He has work experience as a Software Engineer Intern at Trimble Inc. and as a Data Science Intern at Apollo Munich Health Insurance. His technical skills include programming languages like Python, Java, C++, JavaScript, and frameworks like ROS, TensorFlow, PyTorch and databases like MySQL, PostgreSQL and MongoDB. Some of his projects include handwritten math symbol classification, automated canonical cover computation, synchronized producer-consumer across servers and lung cancer detection using deep learning.
Cloud computer
Cloud computerRajesh Kumar Cloud computing is defined as computing resources and application services delivered over networks from massive collections of servers and storage devices. It is a model that enables on-demand access to a shared pool of configurable computing resources like networks, servers, storage and applications. Cloud computing provides five essential characteristics: virtualization, multi-tenancy, real-time provisioning, pay-per-use, and delivery of computing as a service. It discusses cloud computing infrastructure, developer tools, and the impact on the consulting industry. It also provides a demo of the Apache Hadoop MapReduce framework.
Microservices Live
Microservices LiveData Driven Innovation Glynn Bird - Building the "microservices way" involves breaking monolithic IT systems into small, decoupled services that each to one job well. This talk builds a practical microservices architecture during the talk using small Node.js apps that perform storage, analytics and visualisation tasks. Learn how to orchestrate your own microservice architecture using simple, easily-tested building blocks.
Cloud Capacity Planning Tooling - South Bay SRE Meetup Aug-09-2016
Cloud Capacity Planning Tooling - South Bay SRE Meetup Aug-09-2016Coburn Watson The document summarizes Netflix's cloud capacity planning operations. It notes that Netflix tracks over 2,400 applications running on 80 instance types deployed across 30 zones and 60 accounts in AWS. To provide transparency and actionable insights at scale, Netflix built interactive dashboards and focuses on deviations and trends. Its tool called Picsou comprehensively monitors Netflix's current cloud footprint, forecasts future capacity needs, and optimizes on-demand and reserved capacity for cost and reliability. Going forward, Picsou will provide capacity alerts, forecasts, and recommendations to optimize reserved capacity usage.
Automate scalable micro services with Kubernetes on Google Cloud by Billme , ...
Automate scalable micro services with Kubernetes on Google Cloud by Billme , ...TeK Charnsilp Chinprasert The document discusses automating scalable microservices with Kubernetes on Google Cloud. It introduces Kubernetes and how it can be used to run jobs on Google Cloud with microservices split into pools for the API and jobs. This allows scaling the jobs independently while ensuring high availability of the API through Kubernetes features like node selection and node pools.
Near-real time reporting in Azure | 2017 Cloudbrew Radu Vunvulea
Near-real time reporting in Azure | 2017 Cloudbrew Radu VunvuleaRadu Vunvulea One of the most common requirement on a projects nowadays is real time monitoring and reporting. Easy to say, expensive to implement and complex to maintain. In this session we'll take a look on the Azure Services that enable us to fulfil this requirements with minimal effort and with maximum benefits. We have on our radar services like Azure Time Series Insights, Analysis Services and PowerBI. After this session you will know what are the services can be used for near-real time and what does near-real time means in the real world.
PhD Projects in Green Cloud Computing Research Guidance
PhD Projects in Green Cloud Computing Research GuidancePhD Services Pertinent Concepts in Green Cloud Computing
Typical Topics in Green Cloud Computing
Foremost Subjects in Green Cloud Computing
Automate scalable micro services with Kubernetes on Google Cloud by Billme , ...
Automate scalable micro services with Kubernetes on Google Cloud by Billme , ...TeK Charnsilp Chinprasert
Similar to The Evolving Landscape of Data Engineering (20)
The Evolving Landscape of Data Engineering
The Evolving Landscape of Data EngineeringAndrei Savu The document discusses the evolving landscape of data engineering. It provides context on the past, present, and future of data engineering. Specifically, it notes that in the past, data engineering was driven by open source communities and the early histories of AWS and Google Cloud. It describes common present-day patterns like serverless architectures and data locality. Finally, it outlines a future wish list, including data catalogs, monitoring systems, and more intelligent data infrastructure. The document concludes by offering recommendations on where to start with technologies, Google Cloud courses, and developing domain knowledge.
云计算及其应用
云计算及其应用lantianlcdx The document discusses cloud computing technology and applications. It provides an introduction to cloud computing concepts, distributed systems, MapReduce, and technologies like Google File System, BigTable and AppEngine. It then outlines the syllabus for a cloud computing course, including topics on virtualization, data centers, and guest lectures. Project presentations will account for 60% of the grading.
ESA and the Cloud
ESA and the CloudNetcetera Netcetera consultants Ronnie Brunner and Jason Brazile present the results of a year long study of existing and potential uses of cloud computing at the European Space Agency. Some unpublished internal material was removed. Queries can be directed to the contract's Technical Officer at ESA ESRIN.
Big Data Session 1.pptx
Big Data Session 1.pptxElsonPaul2 This document provides an introduction to a course on big data and analytics. It outlines the instructor and teaching assistant contact information. It then lists the main topics to be covered, including data analytics and mining techniques, Hadoop/MapReduce programming, graph databases and analytics. It defines big data and discusses the 3Vs of big data - volume, variety and velocity. It also covers big data technologies like cloud computing, Hadoop, and graph databases. Course requirements and the grading scheme are outlined.
AnilKumarT_Resume_latest
AnilKumarT_Resume_latestanil_thyagarajan Anil Kumar Thyagarajan is a senior software engineer with over 15 years of experience in areas like big data analytics, cloud computing, payment gateways, and supply chain products. He is currently a senior data engineer at Microsoft working on their Azure HDInsight platform. Previously he held roles at Nokia, Yahoo, and AOL where he led teams and worked on projects involving Hadoop, Amazon Web Services, data migration, monitoring tools, and distributed systems. He has expertise in technologies like Perl, Java, Python, Linux, Hadoop, Spark, and Amazon Web Services.
An Introduction to Data Intensive Computing
An Introduction to Data Intensive ComputingCollin Bennett This document provides an introduction to data intensive computing. It discusses how data and computing are growing exponentially due to improvements in instruments and technologies. This is creating new paradigms of data intensive science and computing. The document then discusses how cloud computing models like utility clouds (e.g. Amazon) and data clouds are facilitating data intensive computing by providing scalable resources and platforms for storing, managing and processing large amounts of data. Key concepts covered include virtualization, infrastructure as a service (IaaS), and MapReduce as a programming model for distributed computing on big data.
Scaling up with Cisco Big Data: Data + Science = Data Science
Scaling up with Cisco Big Data: Data + Science = Data ScienceeRic Choo Discover how IoT data is ingested with real-time technologies such as Apache Spark and analysed using Data Science techniques to glean insights.
Stargate, the gateway for some multi-models data API
Stargate, the gateway for some multi-models data APIData Con LA Cedrick Lunven presents on the gateway for multi-model Data APIs. The presentation discusses why data gateways are rising in popularity, the architecture and implementations of gateways like Stargate, how Apache Cassandra can be used as a multi-model database, and demos Astra which is a Cassandra-as-a-Service. The presentation aims to explain the benefits of data gateways for both developers and database administrators.
Cloud comparison - AWS vs Azure vs Google
Cloud comparison - AWS vs Azure vs GooglePatrick Pierson Slides from a talk I gave to Frederick WebTech (https://www.meetup.com/FredWebTech/) that compared the three major cloud providers.
Big data on google cloud
Big data on google cloudTu Pham The document discusses Big Data challenges at Dyno including having a multi-terabyte data warehouse with over 100 GB of new raw data daily from 65 online and unlimited offline data sources, facing daily data quality problems, and needing to derive user interests and intentions from user information, behavior, and other data while managing a high performance and cost effective system. It also advertises job openings at Dyno for frontend and backend developers.
Deep dive into Google Cloud for Big Data
Deep dive into Google Cloud for Big DataTu Le Dinh The document discusses Big Data challenges at Dyno including having a multi-terabyte data warehouse with over 100 GB of new raw data daily from 65 online and unlimited offline data sources, facing daily data quality problems, and needing to derive user interests and intentions from user information, behavior, and other data while managing a high performance and cost effective system. It also advertises job openings at Dyno for frontend and backend developers.
Cloud Computing Fundamentals
Cloud Computing FundamentalsAravind Udayashankara The document provides an introduction to cloud computing. It discusses:
- The author's background working with cloud strategies for over a decade.
- Early predictions that computing would become organized as a public utility.
- The perils of corporate computing like high costs and overcapacity.
- Definitions of cloud computing including the NIST definition that it is on-demand access to shared configurable resources over the internet.
- The types of cloud including SaaS, PaaS, and IaaS delivery models.
- Characteristics of cloud computing problems like massive scale, data intensity, and accessibility.
Cloud Computing ...changes everything
Cloud Computing ...changes everythingLew Tucker Lew Tucker discusses the rise of cloud computing and its impact. He defines various cloud service models like SaaS, PaaS, and IaaS. Tucker analogizes the shift to cloud computing from individual data centers generating their own power to today's electrical grid. Major drivers of cloud computing include the growth of web APIs and massive amounts of user-generated data. Tucker outlines how cloud computing changes what developers can access and how applications are designed and scaled.
President Election of Korea in 2017
President Election of Korea in 2017Jongwook Woo Data Analysis and Visualization of Twitter data for Korean President Election in 2017 using ElasticSearch and Hadoop Spark
Cloud Computing
Cloud Computingwebscale The document provides a comparison of Amazon AWS, Google App Engine, and Sun Project Caroline cloud computing platforms. It discusses their offerings such as hardware as a service, platform as a service, and software as a service. Amazon AWS provides extensive infrastructure services while Google App Engine focuses on its APIs and big data capabilities. Project Caroline is a research project aiming to provide programmatic control of distributed resources on a large shared grid.
Cloud Computing
Cloud ComputingCompare Infobase Limited Cloud computing allows users to access computing resources and applications over the Internet. It provides scalability, lower costs, and more efficient utilization of resources compared to local computing. There are different types of cloud services including Software as a Service (SaaS), Infrastructure as a Service (IaaS), and Platform as a Service (PaaS). Cloud computing delivers significant business benefits like reduced costs, more agile resource usage, and robust scalability.
AWS Cloud Technology And Future of Faster Modern Architecture
AWS Cloud Technology And Future of Faster Modern ArchitectureTMME - TECH MEETUP FOR MYANMAR ENGINEERS IN JP The document discusses cloud computing and AWS cloud technology. It provides an overview of cloud computing benefits like agility, scalability, and cost savings. It then describes AWS as the most comprehensive cloud platform offering over 175 services. It lists several popular AWS products and services like EC2, S3, RDS, DynamoDB, and Lambda. It discusses the future of faster modern architectures using cloud-first approaches like serverless, microservices, and DevOps to build applications. It concludes with tips on getting started with AWS like using free tiers and learning basic services first.
Big Data on OpenStack
Big Data on OpenStackNati Shalom The massive computing and storage resources that are needed to support big data applications make cloud environments an ideal fit. In this session, you'll learn how to build your big data "database on-demand" using MongoDB, Cassandra, Solr, MySQL, or any other big data solution, as well as manage your big data application using a new open source framework called “Cloudify.” All this, on top of the OpenStack cloud.
Big data.ppt
Big data.pptIdontKnow66967 This document provides an introduction to a course on big data and analytics. It outlines the following key points:
- The instructor and TA contact information and course homepage.
- The course will cover foundational data analytics, Hadoop/MapReduce programming, graph databases, and other big data topics.
- Big data is defined as data that is too large or complex for traditional database tools to process. It is characterized by high volume, velocity, and variety.
- Examples of big data sources and the exponential growth of data volumes are provided. Real-time analytics and fast data processing are also discussed.
Introduction to Cloud Computing
Introduction to Cloud ComputingAnimesh Chaturvedi Introduction to the Cloud Computing
Cloud introduction and overview
Cloud Computing Technology
Hardware & software Infrastructure
Different clouds
Risks
Cloud Services
Applications
Regulatory Issues & Limitations
AWS Cloud Technology And Future of Faster Modern Architecture
AWS Cloud Technology And Future of Faster Modern ArchitectureTMME - TECH MEETUP FOR MYANMAR ENGINEERS IN JP
Ad
More from Andrei Savu (20)
Recap on AWS Lambda after re:Invent 2015
Recap on AWS Lambda after re:Invent 2015Andrei Savu A quick presentation on what AWS Lambda is about and what was announced at AWS re:Invent 2015 Las Vegas. In see Lambda as a easy to define event handles that glue different AWS services together at a surprising scale.
One Hadoop, Multiple Clouds - NYC Big Data Meetup
One Hadoop, Multiple Clouds - NYC Big Data MeetupAndrei Savu The slide deck I presented at NYC Big Data Meetup just before Strata + Hadoop World 2015. It goes into details on what's different about running Hadoop in the cloud, main use case and some lessons learned from working with customers.
Introducing Cloudera Director at Big Data Bash
Introducing Cloudera Director at Big Data BashAndrei Savu My slide deck for Big Data Bash. This is a quick introduction on Cloudera Director and it ends with a list of open questions around some interesting future problems we are planning to work on.
APIs & Underlying Protocols #APICraftSF
APIs & Underlying Protocols #APICraftSFAndrei Savu My slides from a talk about APIs and their relationship to various network protocols, older and new ones and how that defines some of the characteristics that describe high quality implementation.
Challenges for running Hadoop on AWS - AdvancedAWS Meetup
Challenges for running Hadoop on AWS - AdvancedAWS MeetupAndrei Savu Nowadays we've got all the tools we need to spin-up and tear-down clusters with hundreds of nodes in minutes and this puts more pressure on the tools we use to configure and monitor our applications. This challenge is even more interesting when we have to deal with long running distributed data storage and processing systems like Hadoop. In this talk we will look into some of the challenges we need to deal with when creating and managing Hadoop clusters in AWS, we will discuss improvement opportunities in monitoring (e.g. detecting and dealing with instance failure, resource contention & noisy neighbors) and a bit about the future and how we should go about disconnecting workload dispatch from cluster lifecycle.
Cloud as a Data Platform
Cloud as a Data PlatformAndrei Savu My slides on how to use cloud as a data platform at BigDataWeek 2013 Romania
http://www.eurocloud.ro/en/events/all-there-is-to-know-about-big-data/#.UXZFaUDvlVI
Apache Provisionr (incubating) - Bucharest JUG 10
Apache Provisionr (incubating) - Bucharest JUG 10Andrei Savu My slides on Apache Provisionr (incubating) - a service that can be used to create and manage pools of virtual machines on multiple clouds.
http://provisionr.incubator.apache.org/
Creating pools of Virtual Machines - ApacheCon NA 2013
Creating pools of Virtual Machines - ApacheCon NA 2013Andrei Savu My slides on creating pools of virtual machines for ApacheCon NA 2013 in Portland.
Provisionr Source code:
https://github.com/axemblr/axemblr-provisionr
Apache Incubator proposal:
https://github.com/axemblr/axemblr-provisionr/wiki/Provisionr-Proposal
Data Scientist Toolbox
Data Scientist ToolboxAndrei Savu This document provides an overview of the data science process and tools for a data science project. It discusses identifying important business questions to answer with data, extracting relevant data from sources, cleaning and sampling the data, analyzing samples to create models and check hypotheses, applying results to full data sets, visualizing findings, automating and deploying solutions, and continuously learning and improving through an iterative process. Key tools mentioned include Hadoop, R, Python, Excel, and various data wrangling, analysis, and visualization tools.
Axemblr Provisionr 0.3.x Overview
Axemblr Provisionr 0.3.x OverviewAndrei Savu Simple Service for Managing Pools of 10s or 100s of Virtual Machines
With Provisionr we want to solve the problem of cloud portability by hiding completely the API and only focusing on building a cluster that matches the same set of assumptions on all clouds, assumptions like: running a specific operating system (e.g. Ubuntu LTS), having the same set of pre-installed packages and binaries, sane dns settings (forward & reverse ip resolution - as needed for Hadoop), ntp settings, networking settings, ssh admin access, vpn access etc.
2012 in Review - Bucharest JUG
2012 in Review - Bucharest JUGAndrei Savu The document summarizes the Bucharest JUG (Java User Group) meetings from May 2012 to November 2012. It provides details on the monthly meetings including average attendance of 25-80 people, topics covered such as Guava, Maven, JavaScript UI for REST, and thanks to speakers and sponsors. It outlines future plans such as live streaming international speakers, a donations/job board, and potential future topics around alternative build systems, deployment options, and monitoring tools. Contact details are provided.
Counters with Riak on Amazon EC2 at Hackover
Counters with Riak on Amazon EC2 at HackoverAndrei Savu The document discusses using distributed counters with Riak on Amazon EC2. It introduces Riak as a distributed key-value database focused on availability, fault tolerance, simplicity and scalability. It describes using Riak features like consistent hashing, replication and automatic load balancing to implement a REST API for counters with eventual consistency. The demo shows implementing and accessing counters across local machines and EC2 regions to demonstrate the architecture.
Simple REST with Dropwizard
Simple REST with DropwizardAndrei Savu This document provides an overview of using Dropwizard, an open-source Java framework, to build RESTful web services. It discusses REST concepts like resources and representations, REST verbs like GET and POST, and architectures for REST APIs. It then introduces Dropwizard and its components for building HTTP services with features like Jetty, Jersey, Jackson, and metrics support. The document demonstrates a sample Dropwizard TODO list application with REST endpoints and resources and discusses considerations for development, testing, and deployment.
Guava Overview Part 2 Bucharest JUG #2 
Guava Overview Part 2 Bucharest JUG #2 Andrei Savu This document provides an overview of Guava and discusses caches and services. Guava is Google's core Java library that contains utilities like caches, primitives, collections, and concurrency libraries. Caches can improve performance by storing values to avoid expensive re-computation. Services in Guava define lifecycles for objects with operational state and allow asynchronous starting and stopping. The document describes cache eviction strategies, service implementations, and where to find more information on Guava features like functional idioms and concurrency.
Guava Overview. Part 1 @ Bucharest JUG #1 
Guava Overview. Part 1 @ Bucharest JUG #1 Andrei Savu Guava is a Java library developed by Google that includes common libraries such as collections, caching, primitives support, concurrency libraries, and generalized utility classes. The talk covered the basic utilities in Guava including using and avoiding null, preconditions, common object utilities, ordering, and primitive array utilities. It also discussed the collections in Guava including immutable collections, new collection types like multisets and multimaps, collection utilities, and ways to extend the collections framework with decorators.
Polyglot Persistence & Big Data in the Cloud
Polyglot Persistence & Big Data in the CloudAndrei Savu This document discusses polyglot persistence and deploying big data technologies in the cloud. It introduces databases like HBase, Cassandra, MongoDB, CouchDB, and Riak for storing large amounts of data. Technologies like Elasticsearch and Solr are presented for search. Hadoop is described as a framework for distributed processing of large datasets. The document concludes by discussing how the Apache Whirr project can be used to deploy these technologies on cloud infrastructure.
Building a Great Team in Open Source - Open Agile 2011
Building a Great Team in Open Source - Open Agile 2011Andrei Savu Lessons I learned over the last 2 years as an open source contributor at The Apache Software Foundation.
Apache Whirr
Apache WhirrAndrei Savu Apache Whirr is a set of libraries for running cloud services on-demand in a cloud-neutral way. It provides common APIs and defaults for deploying clusters running Hadoop, Cassandra, HBase, and Zookeeper on EC2 and Rackspace. Whirr configurations allow deploying typical clusters with a single command. It is being developed further to support private clouds and new services, and to integrate with Hudson for testing fault injection scenarios on small test clusters.
Automated Testing for Web Applications - Wurbe #36
Automated Testing for Web Applications - Wurbe #36Andrei Savu This document discusses automated testing for web applications. It defines software testing and different types of testing like unit, integration, functional, and end-to-end testing. Automated testing is recommended because it saves time and money, improves accuracy, allows testing at scale, and improves team morale. Low hanging fruits for automation include unit, integration and functional testing, and setting up continuous integration. The document provides examples of testing a typical application with MongoDB, Tornado and SMTP and concludes with advice to think about testing from the beginning and automate where possible.
Apache ZooKeeper TechTuesday
Apache ZooKeeper TechTuesdayAndrei Savu ZooKeeper is a highly available, scalable, distributed configuration, consensus, group membership, leader election, naming and coordination service. It provides a hierarchical namespace and basic operations like create, delete, and read data. It is useful for building distributed applications and services like queues. Future releases will focus on monitoring improvements, read-only mode, and failure detection models. The community is working on features like children for ephemeral nodes and viewing session information.
Ad
Recently uploaded (20)
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep Dive
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep DiveScyllaDB Want to learn practical tips for designing systems that can scale efficiently without compromising speed?
Join us for a workshop where we’ll address these challenges head-on and explore how to architect low-latency systems using Rust. During this free interactive workshop oriented for developers, engineers, and architects, we’ll cover how Rust’s unique language features and the Tokio async runtime enable high-performance application development.
As you explore key principles of designing low-latency systems with Rust, you will learn how to:
- Create and compile a real-world app with Rust
- Connect the application to ScyllaDB (NoSQL data store)
- Negotiate tradeoffs related to data modeling and querying
- Manage and monitor the database for consistently low latencies
Cyber Awareness overview for 2025 month of security
Cyber Awareness overview for 2025 month of securityriccardosl1 Cyber awareness training educates employees on risk associated with internet and malicious emails
Mobile App Development Company in Saudi Arabia
Mobile App Development Company in Saudi ArabiaSteve Jonas EmizenTech is a globally recognized software development company, proudly serving businesses since 2013. With over 11+ years of industry experience and a team of 200+ skilled professionals, we have successfully delivered 1200+ projects across various sectors. As a leading Mobile App Development Company In Saudi Arabia we offer end-to-end solutions for iOS, Android, and cross-platform applications. Our apps are known for their user-friendly interfaces, scalability, high performance, and strong security features. We tailor each mobile application to meet the unique needs of different industries, ensuring a seamless user experience. EmizenTech is committed to turning your vision into a powerful digital product that drives growth, innovation, and long-term success in the competitive mobile landscape of Saudi Arabia.
TrsLabs - Fintech Product & Business Consulting
TrsLabs - Fintech Product & Business ConsultingTrs Labs Hybrid Growth Mandate Model with TrsLabs
Strategic Investments, Inorganic Growth, Business Model Pivoting are critical activities that business don't do/change everyday. In cases like this, it may benefit your business to choose a temporary external consultant.
An unbiased plan driven by clearcut deliverables, market dynamics and without the influence of your internal office equations empower business leaders to make right choices.
Getting things done within a budget within a timeframe is key to Growing Business - No matter whether you are a start-up or a big company
Talk to us & Unlock the competitive advantage
Rusty Waters: Elevating Lakehouses Beyond Spark
Rusty Waters: Elevating Lakehouses Beyond Sparkcarlyakerly1 Spark is a powerhouse for large datasets, but when it comes to smaller data workloads, its overhead can sometimes slow things down. What if you could achieve high performance and efficiency without the need for Spark?
At S&P Global Commodity Insights, having a complete view of global energy and commodities markets enables customers to make data-driven decisions with confidence and create long-term, sustainable value. 🌍
Explore delta-rs + CDC and how these open-source innovations power lightweight, high-performance data applications beyond Spark! 🚀
Manifest Pre-Seed Update | A Humanoid OEM Deeptech In France
Manifest Pre-Seed Update | A Humanoid OEM Deeptech In Francechb3 The latest updates on Manifest's pre-seed stage progress.
AI and Data Privacy in 2025: Global Trends
AI and Data Privacy in 2025: Global TrendsInData Labs In this infographic, we explore how businesses can implement effective governance frameworks to address AI data privacy. Understanding it is crucial for developing effective strategies that ensure compliance, safeguard customer trust, and leverage AI responsibly. Equip yourself with insights that can drive informed decision-making and position your organization for success in the future of data privacy.
This infographic contains:
-AI and data privacy: Key findings
-Statistics on AI data privacy in the today’s world
-Tips on how to overcome data privacy challenges
-Benefits of AI data security investments.
Keep up-to-date on how AI is reshaping privacy standards and what this entails for both individuals and organizations.
Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...
Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...Noah Loul Artificial intelligence is changing how businesses operate. Companies are using AI agents to automate tasks, reduce time spent on repetitive work, and focus more on high-value activities. Noah Loul, an AI strategist and entrepreneur, has helped dozens of companies streamline their operations using smart automation. He believes AI agents aren't just tools—they're workers that take on repeatable tasks so your human team can focus on what matters. If you want to reduce time waste and increase output, AI agents are the next move.
Drupalcamp Finland – Measuring Front-end Energy Consumption
Drupalcamp Finland – Measuring Front-end Energy ConsumptionExove How to measure web front-end energy consumption using Firefox Profiler. Presented in DrupalCamp Finland on April 25th, 2025.
#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025
#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025BookNet Canada Book industry standards are evolving rapidly. In the first part of this session, we’ll share an overview of key developments from 2024 and the early months of 2025. Then, BookNet’s resident standards expert, Tom Richardson, and CEO, Lauren Stewart, have a forward-looking conversation about what’s next.
Link to recording, transcript, and accompanying resource: https://bnctechforum.ca/sessions/standardsgoals-for-2025-standards-certification-roundup/
Presented by BookNet Canada on May 6, 2025 with support from the Department of Canadian Heritage.
AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...
AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...Alan Dix Talk at the final event of Data Fusion Dynamics: A Collaborative UK-Saudi Initiative in Cybersecurity and Artificial Intelligence funded by the British Council UK-Saudi Challenge Fund 2024, Cardiff Metropolitan University, 29th April 2025
https://alandix.com/academic/talks/CMet2025-AI-Changes-Everything/
Is AI just another technology, or does it fundamentally change the way we live and think?
Every technology has a direct impact with micro-ethical consequences, some good, some bad. However more profound are the ways in which some technologies reshape the very fabric of society with macro-ethical impacts. The invention of the stirrup revolutionised mounted combat, but as a side effect gave rise to the feudal system, which still shapes politics today. The internal combustion engine offers personal freedom and creates pollution, but has also transformed the nature of urban planning and international trade. When we look at AI the micro-ethical issues, such as bias, are most obvious, but the macro-ethical challenges may be greater.
At a micro-ethical level AI has the potential to deepen social, ethnic and gender bias, issues I have warned about since the early 1990s! It is also being used increasingly on the battlefield. However, it also offers amazing opportunities in health and educations, as the recent Nobel prizes for the developers of AlphaFold illustrate. More radically, the need to encode ethics acts as a mirror to surface essential ethical problems and conflicts.
At the macro-ethical level, by the early 2000s digital technology had already begun to undermine sovereignty (e.g. gambling), market economics (through network effects and emergent monopolies), and the very meaning of money. Modern AI is the child of big data, big computation and ultimately big business, intensifying the inherent tendency of digital technology to concentrate power. AI is already unravelling the fundamentals of the social, political and economic world around us, but this is a world that needs radical reimagining to overcome the global environmental and human challenges that confront us. Our challenge is whether to let the threads fall as they may, or to use them to weave a better future.
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungenpanagenda Webinar Recording: https://www.panagenda.com/webinars/hcl-nomad-web-best-practices-und-verwaltung-von-multiuser-umgebungen/
HCL Nomad Web wird als die nächste Generation des HCL Notes-Clients gefeiert und bietet zahlreiche Vorteile, wie die Beseitigung des Bedarfs an Paketierung, Verteilung und Installation. Nomad Web-Client-Updates werden “automatisch” im Hintergrund installiert, was den administrativen Aufwand im Vergleich zu traditionellen HCL Notes-Clients erheblich reduziert. Allerdings stellt die Fehlerbehebung in Nomad Web im Vergleich zum Notes-Client einzigartige Herausforderungen dar.
Begleiten Sie Christoph und Marc, während sie demonstrieren, wie der Fehlerbehebungsprozess in HCL Nomad Web vereinfacht werden kann, um eine reibungslose und effiziente Benutzererfahrung zu gewährleisten.
In diesem Webinar werden wir effektive Strategien zur Diagnose und Lösung häufiger Probleme in HCL Nomad Web untersuchen, einschließlich
- Zugriff auf die Konsole
- Auffinden und Interpretieren von Protokolldateien
- Zugriff auf den Datenordner im Cache des Browsers (unter Verwendung von OPFS)
- Verständnis der Unterschiede zwischen Einzel- und Mehrbenutzerszenarien
- Nutzung der Client Clocking-Funktion
How analogue intelligence complements AI
How analogue intelligence complements AIPaul Rowe
Artificial Intelligence is providing benefits in many areas of work within the heritage sector, from image analysis, to ideas generation, and new research tools. However, it is more critical than ever for people, with analogue intelligence, to ensure the integrity and ethical use of AI. Including real people can improve the use of AI by identifying potential biases, cross-checking results, refining workflows, and providing contextual relevance to AI-driven results.
News about the impact of AI often paints a rosy picture. In practice, there are many potential pitfalls. This presentation discusses these issues and looks at the role of analogue intelligence and analogue interfaces in providing the best results to our audiences. How do we deal with factually incorrect results? How do we get content generated that better reflects the diversity of our communities? What roles are there for physical, in-person experiences in the digital world?
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...Aqusag Technologies In late April 2025, a significant portion of Europe, particularly Spain, Portugal, and parts of southern France, experienced widespread, rolling power outages that continue to affect millions of residents, businesses, and infrastructure systems.
Generative Artificial Intelligence (GenAI) in Business
Generative Artificial Intelligence (GenAI) in BusinessDr. Tathagat Varma My talk for the Indian School of Business (ISB) Emerging Leaders Program Cohort 9. In this talk, I discussed key issues around adoption of GenAI in business - benefits, opportunities and limitations. I also discussed how my research on Theory of Cognitive Chasms helps address some of these issues
Splunk Security Update | Public Sector Summit Germany 2025
Splunk Security Update | Public Sector Summit Germany 2025Splunk Splunk Security Update
Sprecher: Marcel Tanuatmadja
IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...
IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...organizerofv IEDM 2024 Tutorial2
Quantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur MorganArthur Morgan This is a Quick Research Guide (QRG).
QRGs include the following:
- A brief, high-level overview of the QRG topic.
- A milestone timeline for the QRG topic.
- Links to various free online resource materials to provide a deeper dive into the QRG topic.
- Conclusion and a recommendation for at least two books available in the SJPL system on the QRG topic.
QRGs planned for the series:
- Artificial Intelligence QRG
- Quantum Computing QRG
- Big Data Analytics QRG
- Spacecraft Guidance, Navigation & Control QRG (coming 2026)
- UK Home Computing & The Birth of ARM QRG (coming 2027)
Any questions or comments?
- Please contact Arthur Morgan at [email protected].
100% human made.
Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...
Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...BookNet Canada Book industry standards are evolving rapidly. In the first part of this session, we’ll share an overview of key developments from 2024 and the early months of 2025. Then, BookNet’s resident standards expert, Tom Richardson, and CEO, Lauren Stewart, have a forward-looking conversation about what’s next.
Link to recording, presentation slides, and accompanying resource: https://bnctechforum.ca/sessions/standardsgoals-for-2025-standards-certification-roundup/
Presented by BookNet Canada on May 6, 2025 with support from the Department of Canadian Heritage.
The Evolving Landscape of Data Engineering
- 1. The Evolving Landscape of Data Engineering Andrei Savu - Event Co-organizer Staff Engineer @ Twitter Follow me @andreisavu
- 2. Andrei Savu Staff Engineer @ Twitter: * MoPub Backend & Data Pipelines * Mobile App Monetization Co-organizer of the Data Engineering Club. Previously Tech Lead at Cloudera via the Axemblr acquisition. Started the Cloud engineering team.
- 3. The Past: ● OSS communities ● AWS history ● Google Cloud history The Present: Patterns The Future: Wish List Topics
- 4. Weeks of Provisioning Static Infrastructure Commodity Hardware Commodity Networking Data Locality Important Running in the Public Cloud was unusual CAPEX The Past - OSS
- 5. Visionary Business Fast iterations Data Management as a key platform use case Incredible Scale Transition to “serverless” OPEX & Elastic The Past - AWS
- 6. Visionary Products Fast iterations Machine Learning as a key use case State of the Art data platform Last 3 years on fast forward Intelligent Billing OPEX & Elastic The Past - Google Cloud
- 7. The Present: Patterns Weeks to Minutes to Seconds Hadoop/Spark ecosystem is mature. We have a broad set of options. Big Data is much Bigger (e.g. x1e.32xlarge: 3TB mem, 128 vCPUs, 14Gbps network) Scale continues to be hard. Cloud economics can be very disruptive (especially for data workloads) High-performance networks are common. Storage can be decoupled from compute. Cluster locality is important. Service Endpoints (not clusters, aka serverless, aka managed etc.). Sophisticated Auto-scaling (batch & streaming, spot vs. on-demand, multi-az). Multi-DC and Multi-Region from Day 1. Various flavors of containers.
- 8. The Future: Wish List A Data Catalog product as the center of the universe. Data Monitoring Systems: * statistical properties, anomaly detection, schema changes, consumption patterns etc. More intelligence at the data infrastructure level: * data format migrations, intelligent caching based on access patterns. Declarative data transformation vs. explicit ETL. Intelligent data sampling products. Scalability has a cost.
- 9. Thanks! Join the community on Meetup.com! www.meetup.com/Data-Engineering-Club www.dataeng.club Do you want to present? Get in touch. Feedback #dataengclub