[EN]DSS23_tspann_Integrating LLM with Streaming Data Pipelines
2 likes474 views
[EN]DSS23_tspann_Integrating LLM with Streaming Data Pipelines https://dssconf.pl/en/#agenda-section Integrating LLM with Streaming Data Pipelines Timothy Spann, Principal Developer Advocate, Cloudera APACHE NIFI, APACHE FLINK, APACHE KAFKA, LLM, HUGGINGFACE, REST, STREAMING LESS In this talk and demo I will walk through how to add LLMs to your streaming pipelines by integration through Apache NiFi. https://github.com/tspannhw/FLaNK-watsonx.ai Cloudera streaming llm generative ai slack to slack
1 of 25
Downloaded 25 times
![12 / 3
{ "model_id": "meta-llama/llama-2-70b-chat",
"input": "this is prompt text”,
"parameters": {
"decoding_method": "greedy",
"max_new_tokens": 200,
"min_new_tokens": 50,
"stop_sequences": [],
"repetition_penalty": 1
},
"project_id": "0ead8ec4-d137-4f9c-8956-50b0da4a7068" }
WatsonX.AI JSON Prompt](https://image.slidesharecdn.com/endss23tspannintegratingllmwithstreamingdatapipelines-231115161007-49510320/85/EN-DSS23_tspann_Integrating-LLM-with-Streaming-Data-Pipelines-12-320.jpg)
Ad
Recommended
Reinventing Deep Learning
with Hugging Face Transformers
Reinventing Deep Learning
with Hugging Face TransformersJulien SIMON The document discusses how transformers have become a general-purpose architecture for machine learning, with various transformer models like BERT and GPT-3 seeing widespread adoption. It introduces Hugging Face as a company working to make transformers more accessible through tools and libraries. Hugging Face has seen rapid growth, with its hub hosting over 73,000 models and 10,000 datasets that are downloaded over 1 million times daily. The document outlines Hugging Face's vision of facilitating the entire machine learning process from data to production through tools that support tasks like transfer learning, hardware acceleration, and collaborative model development.
MLops workshop AWS
MLops workshop AWSGili Nachum This document provides an agenda and overview for an MLOps workshop hosted by Amazon Web Services. The agenda includes introductions to Amazon AI, MLOps, Amazon SageMaker, machine learning pipelines, and a hands-on exercise to build an MLOps pipeline. It discusses key concepts like personas in MLOps, the CRISP-DM process, microservices deployment, and challenges of MLOps. It also provides overviews of Amazon SageMaker for machine learning and AWS services for continuous integration/delivery.
LLMs_talk_March23.pdf
LLMs_talk_March23.pdfChaoYang81 The document discusses advances and challenges in model evaluation and summarizes a presentation on this topic. It provides an overview of the growing landscape of natural language processing (NLP) models, including their usage trends over time. There is a lack of documentation for most models, with only 50% having model cards despite contributing 98% of usage. The presentation proposes a randomized controlled trial to study whether improving model documentation could increase usage by adding documentation to a treatment group of models and comparing their usage to an undocumented control group. The goal is to provide more transparency and drive better model communication and reproducibility.
LanGCHAIN Framework
LanGCHAIN FrameworkKeymate.AI Langchain Framework is an innovative approach to linguistic data processing, combining the principles of language sciences, blockchain technology, and artificial intelligence. This deck introduces the groundbreaking elements of the framework, detailing how it enhances security, transparency, and decentralization in language data management. It discusses its applications in various fields, including machine learning, translation services, content creation, and more. The deck also highlights its key features, such as immutability, peer-to-peer networks, and linguistic asset ownership, that could revolutionize how we handle linguistic data in the digital age.
generative-ai-fundamentals and Large language models
generative-ai-fundamentals and Large language modelsAdventureWorld5 Thank you for the detailed review of the protein bars. I'm glad to hear you and your family are enjoying them as a healthy snack and meal replacement option. A couple suggestions based on your feedback:
- For future orders, you may want to check the expiration dates to help avoid any dried out bars towards the end of the box. Freshness is key to maintaining the moist texture.
- When introducing someone new to the bars, selecting one in-person if possible allows checking the flexibility as an indicator it's moist inside. This could help avoid a disappointing first impression from a dry sample.
- Storing opened boxes in an airtight container in the fridge may help extend the freshness even further when you can't
Unlocking the Power of ChatGPT and AI in Testing - NextSteps, presented by Ap...
Unlocking the Power of ChatGPT and AI in Testing - NextSteps, presented by Ap...Applitools The document discusses AI tools for software testing such as ChatGPT, Github Copilot, and Applitools Visual AI. It provides an overview of each tool and how they can help with testing tasks like test automation, debugging, and handling dynamic content. The document also covers potential challenges with AI like data privacy issues and tools having superficial knowledge. It emphasizes that AI should be used as an assistance to humans rather than replacing them and that finding the right balance and application of tools is important.
How ChatGPT and AI-assisted coding changes software engineering profoundly
How ChatGPT and AI-assisted coding changes software engineering profoundlyPekka Abrahamsson / Tampere University This document summarizes a presentation given by Professor Pekka Abrahamsson on how ChatGPT and AI-assisted coding is profoundly changing software engineering. The presentation covers several key points:
- ChatGPT and AI tools like Copilot are beginning to be adopted in software engineering to provide code snippets, answers to technical questions, and assist with debugging, but issues around code ownership, reliability, and security need to be addressed.
- Early studies show potential benefits of ChatGPT for tasks like software testing education, code quality improvement, and requirements elicitation, but more research is still needed.
- Prompt engineering techniques can help maximize the usefulness of ChatGPT for software engineering tasks. Overall, AI
Cloud AI GenAI Overview.pptx
Cloud AI GenAI Overview.pptxSahithiGurlinka This document provides an overview of Google Cloud's offerings for generative AI. It begins with a primer on large language models and generative AI, explaining what they are and how they have evolved. It then outlines Google's role in pioneering developments in the field like BERT and Transformer models. The rest of the document details Google's portfolio of products and services for generative AI, including foundation models like PaLM, experiences for consumers and enterprises, and tools for developers and AI practitioners. It emphasizes that Google aims to support a wide range of needs through its family of generative AI models and applications.
Flink vs. Spark
Flink vs. SparkSlim Baltagi Flink vs. Spark: this is the slide deck of my talk at the 2015 Flink Forward conference in Berlin, Germany, on October 12, 2015. In this talk, we tried to compare Apache Flink vs. Apache Spark with focus on real-time stream processing. Your feedback and comments are much appreciated.
“MLOps: Managing Data and Workflows for Efficient Model Development and Deplo...
“MLOps: Managing Data and Workflows for Efficient Model Development and Deplo...Edge AI and Vision Alliance For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2022/09/mlops-managing-data-and-workflows-for-efficient-model-development-and-deployment-a-presentation-from-airbus/
Konstantinos Balafas, Head of AI Data, and Carlo Dal Mutto, Director of Engineering, both of Airbus, present the “MLOps: Managing Data and Workflows for Efficient Model Development and Deployment” tutorial at the May 2022 Embedded Vision Summit.
Machine learning operations (MLOps) is the engineering field focused on techniques for developing and deploying machine learning solutions at scale. As the name suggests, MLOps is a combination of machine learning development (“ML”) and software/IT operations (“Ops”). Blending these two words is particularly complex, given their diverse nature. ML development is characterized by research and experimental components, dealing with large amounts of data and complex operations, while software and IT operations aim at streamlining software deployment in products.
Typical problems addressed by MLOps include data management (labeling, organization, storage), ML model and pipeline training repeatability, error analysis, model integration and deployment and model monitoring. In this talk, Dal Mutto and Balafas present practical MLOps techniques useful for tackling a variety of MLOps needs. They illustrate these techniques with real-world examples from their work developing autonomous flying capabilities as part of the Wayfinder team at Acubed, the Silicon Valley innovation center of Airbus.
[DSC Europe 23] Spela Poklukar & Tea Brasanac - Retrieval Augmented Generation![[DSC Europe 23] Spela Poklukar & Tea Brasanac - Retrieval Augmented Generation](https://cdn.slidesharecdn.com/ss_thumbnails/spelapoklukarteabrasanacprez-231129000106-d6582d1b-thumbnail.jpg?width=560&fit=bounds)
[DSC Europe 23] Spela Poklukar & Tea Brasanac - Retrieval Augmented GenerationDataScienceConferenc1 Retrieval Augmented Generation (RAG) combines the concepts of semantic search and LLM-based text generation. When a person makes a query in natural language, the query is compared to the entries in the knowledge base and most relevant results are returned to the LLM, which uses this extra information to generate more accurate and reliable response. RAG can therefore limit hallucination and provide accurate responses from reliable source. In this talk, we will present the concept of RAG and underlying concept of semantic search, and present available libraries and vector databases.
Making Testing Easy w GitHub Copilot.pdf
Making Testing Easy w GitHub Copilot.pdfApplitools The document discusses GitHub Copilot, an AI programming assistant. It notes that Copilot can help reduce the pain of writing tests by providing predictive text and increasing a developer's velocity. Several use cases for Copilot are provided, including writing better comments, pair programming, writing tests, and live coding. Copilot Labs is also discussed as helping developers become comfortable in new codebases, understand StackOverflow solutions, and gain additional context on data structures. The document concludes with examples of using Copilot and a reminder that it is still in beta.
Retrieval Augmented Generation in Practice: Scalable GenAI platforms with k8s...
Retrieval Augmented Generation in Practice: Scalable GenAI platforms with k8s...Mihai Criveti Mihai is the Principal Architect for Platform Engineering and Technology Solutions at IBM, responsible for Cloud Native and AI Solutions. He is a Red Hat Certified Architect, CKA/CKS, a leader in the IBM Open Innovation community, and advocate for open source development. Mihai is driving the development of Retrieval Augmentation Generation platforms, and solutions for Generative AI at IBM that leverage WatsonX, Vector databases, LangChain, HuggingFace and open source AI models.
Mihai will share lessons learned building Retrieval Augmented Generation, or “Chat with Documents” platforms and APIs that scale, and deploy on Kubernetes. His talk will cover use cases for Generative AI, limitations of Large Language Models, use of RAG, Vector Databases and Fine Tuning to overcome model limitations and build solutions that connect to your data and provide content grounding, limit hallucinations and form the basis of explainable AI. In terms of technology, he will cover LLAMA2, HuggingFace TGIS, SentenceTransformers embedding models using Python, LangChain, and Weaviate and ChromaDB vector databases. He’ll also share tips on writing code using LLM, including building an agent for Ansible and containers.
Scaling factors for Large Language Model Architectures:
• Vector Database: consider sharding and High Availability
• Fine Tuning: collecting data to be used for fine tuning
• Governance and Model Benchmarking: how are you testing your model performance
over time, with different prompts, one-shot, and various parameters
• Chain of Reasoning and Agents
• Caching embeddings and responses
• Personalization and Conversational Memory Database
• Streaming Responses and optimizing performance. A fine tuned 13B model may
perform better than a poor 70B one!
• Calling 3rd party functions or APIs for reasoning or other type of data (ex: LLMs are
terrible at reasoning and prediction, consider calling other models)
• Fallback techniques: fallback to a different model, or default answers
• API scaling techniques, rate limiting, etc.
• Async, streaming and parallelization, multiprocessing, GPU acceleration (including
embeddings), generating your API using OpenAPI, etc.
Generative AI con Amazon Bedrock.pdf
Generative AI con Amazon Bedrock.pdfGuido Maria Nebiolo Generative AI models like LLMs can be customized for specific tasks through techniques like prompt engineering, retrieval augmented generation, and fine-tuning. Prompt engineering involves providing contextual information to steer model responses, while retrieval augmented generation combines generative and retrieval models to improve performance. Fine-tuning customizes foundation models with domain-specific training data. The document discusses these techniques and their benefits, encouraging hands-on experience to learn how to best apply generative AI.
Introduction to GitHub Copilot
Introduction to GitHub CopilotAll Things Open Presented at All Things Open RTP Meetup
Presented by Brent Laster
Title: Introduction to GitHub Copilot
Abstract: Learn about the future of coding with this presentation, "Introduction to GitHub Copilot." Join author, speaker, and DevOps director Brent Laster as we discuss the GitHub coding assistant that leverages the power of artificial intelligence to augment your development activities. In this session, we'll delve into the core features of GitHub Copilot, explaining what it is, and learning about its ability to generate code snippets, autocompletions, and even entire functions across a variety of programming languages.
Learn how Copilot enhances your coding efficiency by contextualizing suggestions based on your codebase and offering a seamless integration into popular code editors. We'll explore some example applications and look at how developers can harness Copilot to streamline their workflows and boost productivity. Whether you're a seasoned developer or just embarking on your coding journey, this presentation offers a quick introduction to GitHub Copilot's capabilities, as a starting point for you to embrace cutting-edge AI assistance in your coding endeavors.
Large language models in higher education
Large language models in higher educationPeter Trkman Discussing the possibilities of large language models for the automatic generation of academic content by the students (e.g. master thesis), and the related need for changes in the way in which to educate and evaluate students.
OpenAI-Copilot-ChatGPT.pptx
OpenAI-Copilot-ChatGPT.pptxUdaiappa Ramachandran ChatGPT (Chat Generative pre-defined transformer) is OpenAI's application that performs human like interactions. GitHub Copilot uses the OpenAI Codex to suggest code and entire functions in real-time, right from your editor. Deck contains more details about ChatGPT, AI, AGI, CoPilot, OpenAI API, and use case scenarios.
Best Practice on using Azure OpenAI Service
Best Practice on using Azure OpenAI ServiceKumton Suttiraksiri This document provides 7 best practices for using the Azure OpenAI Service:
1. Set clear goals and objectives for your prompts.
2. Choose the appropriate AI model like GPT-3, Ada, or Davinci based on your task's complexity and required capabilities.
3. Ensure prompts are precise yet not too short to achieve the desired response.
Build an LLM-powered application using LangChain.pdf
Build an LLM-powered application using LangChain.pdfAnastasiaSteele10 LangChain is an advanced framework that allows developers to create language model-powered applications. It provides a set of tools, components, and interfaces that make building LLM-based applications easier. With LangChain, managing interactions with language models, chaining together various components, and integrating resources like APIs and databases is a breeze. The platform includes a set of APIs that can be integrated into applications, allowing developers to add language processing capabilities without having to start from scratch.
Mohamed Sabri: Operationalize machine learning with Kubeflow
Mohamed Sabri: Operationalize machine learning with KubeflowEdunomica Data Science Online Camp 2021
Website - https://dscamp.org/
Youtube - https://www.youtube.com/channel/UCeHtPZ_ZLZ-nHFMUCXY81RQ/featured
FB - https://www.facebook.com/Data-Science-Camp-103012708431833
Copilot to Cover: Why AI can't replace developers with robots, but can make l...
Copilot to Cover: Why AI can't replace developers with robots, but can make l...Andy Piper AI-augmented tools like GitHub Copilot, AWS CodeWhisperer, and Diffblue Cover use machine learning to help developers with repetitive coding tasks like completing boilerplate code, calling APIs, and writing tests. These tools generate code fragments or entire functions to reduce errors and speed up development. While they are helpful for mundane tasks, fully autonomous coding remains a challenge, as writing high-quality tests requires understanding complex code dependencies and business requirements. Diffblue's reinforcement learning approach aims to autonomously write tests that satisfy developer needs and increase code coverage. These AI-assisted development tools are available today and can help eliminate tedious coding work.
re:cap Generative AI journey with Bedrock
re:cap Generative AI journey with BedrockPhilipBasford Wherever you are on your Generative AI journey — Amazon Bedrock allows you to rapidly prototype Generative AI concepts, using the latest Foundational Models. This session also included architectures to accelerate your prototype into a real-world GenAI solution using LLMOps. Providing the safeguards to keep your data private & secure, handle any regulatory compliance and responsibility requirements.
Let's talk about GPT: A crash course in Generative AI for researchers
Let's talk about GPT: A crash course in Generative AI for researchersSteven Van Vaerenbergh This talk delves into the extraordinary capabilities of the emerging technology of generative AI, outlining its recent history and emphasizing its growing influence on scientific endeavors. Through a series of practical examples tailored for researchers, we will explore the transformative influence of these powerful tools on scientific tasks such as writing, coding, data wrangling and literature review.
Ml ops past_present_future
Ml ops past_present_futureNisha Talagala Given at the MLOps. Summit 2020 - I cover the origins of MLOps in 2018, how MLOps has evolved from 2018 to 2020, and what I expect for the future of MLOps
How will development change with LLMs
How will development change with LLMsMicrosoft, InfuseAI, Appier, IBM, KaiOS GPT discusses various ways that language models can acquire external information as context to improve responses, including:
1) Querying search engines using APIs to incorporate search results into responses
2) Recognizing tasks from prompts and accessing databases or APIs to incorporate relevant information
3) Summarizing, calculating, and verifying information from external sources to provide more accurate answers
Using the power of Generative AI at scale
Using the power of Generative AI at scaleMaxim Salnikov In this session, you'll get all the answers about how ChatGPT and other GPT-X models can be applied to your current or future project. First, we'll put in order all the terms – OpenAI, GPT-3, ChatGPT, Codex, Dall-E, etc., and explain why Microsoft and Azure are often mentioned in this context. Then, we'll go through the main capabilities of the Azure OpenAI and respective usecases that might inspire you to either optimize your product or build a completely new one.
The Top 5 Apache Kafka Use Cases and Architectures in 2022
The Top 5 Apache Kafka Use Cases and Architectures in 2022Kai Wähner This document discusses the top 5 use cases and architectures for data in motion in 2022. It describes:
1) The Kappa architecture as an alternative to the Lambda architecture that uses a single stream to handle both real-time and batch data.
2) Hyper-personalized omnichannel experiences that integrate customer data from multiple sources in real-time to provide personalized experiences across channels.
3) Multi-cloud deployments using Apache Kafka and data mesh architectures to share data across different cloud platforms.
4) Edge analytics that deploy stream processing and Kafka brokers at the edge to enable low-latency use cases and offline functionality.
5) Real-time cybersecurity applications that use streaming data
IoT Architectures for Apache Kafka and Event Streaming - Industry 4.0, Digita...
IoT Architectures for Apache Kafka and Event Streaming - Industry 4.0, Digita...Kai Wähner The Internet of Things (IoT) is getting more and more traction as valuable use cases come to light. Whether you are in Healthcare, Telecommunications, Manufacturing, Banking or Retail to name a few industries, there is one key challenge and that's the integration of backend IoT data logs and applications, business services and cloud services to process the data in real time and at scale.
In this talk, we will be sharing how Kafka has become the leading technology used throughout the business to provide Real Time Event Streaming. Explore real life use cases of Kafka Connect, Kafka Streams and KSQL independent of the data deployment be it on a private or public Cloud, On Premise or at the Edge.
Audi - Connected car infrastructure
Robert Bosch Power Tools - Track and Trace of devices and people at construction areas
Deutsche Bahn - Customer 360 for train timetable updates
E.ON - IoT Streaming Platform to integrate and build smart home, smart building and smart grid infrastructures
The Never Landing Stream with HTAP and Streaming
The Never Landing Stream with HTAP and StreamingTimothy Spann The Never Landing Stream with HTAP and Streaming
Apache NiFi
Apache Kudu
Apache Flink
Apache Kafka
PingCap
Streaming
Cloudera
Budapest Data/ML - Building Modern Data Streaming Apps with NiFi, Flink and K...
Budapest Data/ML - Building Modern Data Streaming Apps with NiFi, Flink and K...Timothy Spann Budapest Data/ML - Building Modern Data Streaming Apps with NiFi, Flink and Kafka
Apache NiFi, Apache Flink, Apache Kafka
Timothy Spann
Principal Developer Advocate
Cloudera
Data in Motion
https://budapestdata.hu/2023/en/speakers/timothy-spann/
Timothy Spann
Principal Developer Advocate
Cloudera (US)
LinkedIn · GitHub · datainmotion.dev
June 8 · Online · English talk
Building Modern Data Streaming Apps with NiFi, Flink and Kafka
In my session, I will show you some best practices I have discovered over the last 7 years in building data streaming applications including IoT, CDC, Logs, and more.
In my modern approach, we utilize several open-source frameworks to maximize the best features of all. We often start with Apache NiFi as the orchestrator of streams flowing into Apache Kafka. From there we build streaming ETL with Apache Flink SQL. We will stream data into Apache Iceberg.
We use the best streaming tools for the current applications with FLaNK. flankstack.dev
BIO
Tim Spann is a Principal Developer Advocate in Data In Motion for Cloudera. He works with Apache NiFi, Apache Pulsar, Apache Kafka, Apache Flink, Flink SQL, Apache Pinot, Trino, Apache Iceberg, DeltaLake, Apache Spark, Big Data, IoT, Cloud, AI/DL, machine learning, and deep learning. Tim has over ten years of experience with the IoT, big data, distributed computing, messaging, streaming technologies, and Java programming.
Previously, he was a Developer Advocate at StreamNative, Principal DataFlow Field Engineer at Cloudera, a Senior Solutions Engineer at Hortonworks, a Senior Solutions Architect at AirisData, a Senior Field Engineer at Pivotal and a Team Leader at HPE. He blogs for DZone, where he is the Big Data Zone leader, and runs a popular meetup in Princeton & NYC on Big Data, Cloud, IoT, deep learning, streaming, NiFi, the blockchain, and Spark. Tim is a frequent speaker at conferences such as ApacheCon, DeveloperWeek, Pulsar Summit and many more. He holds a BS and MS in computer science.
Ad
More Related Content
What's hot (20)
Flink vs. Spark
Flink vs. SparkSlim Baltagi Flink vs. Spark: this is the slide deck of my talk at the 2015 Flink Forward conference in Berlin, Germany, on October 12, 2015. In this talk, we tried to compare Apache Flink vs. Apache Spark with focus on real-time stream processing. Your feedback and comments are much appreciated.
“MLOps: Managing Data and Workflows for Efficient Model Development and Deplo...
“MLOps: Managing Data and Workflows for Efficient Model Development and Deplo...Edge AI and Vision Alliance For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2022/09/mlops-managing-data-and-workflows-for-efficient-model-development-and-deployment-a-presentation-from-airbus/
Konstantinos Balafas, Head of AI Data, and Carlo Dal Mutto, Director of Engineering, both of Airbus, present the “MLOps: Managing Data and Workflows for Efficient Model Development and Deployment” tutorial at the May 2022 Embedded Vision Summit.
Machine learning operations (MLOps) is the engineering field focused on techniques for developing and deploying machine learning solutions at scale. As the name suggests, MLOps is a combination of machine learning development (“ML”) and software/IT operations (“Ops”). Blending these two words is particularly complex, given their diverse nature. ML development is characterized by research and experimental components, dealing with large amounts of data and complex operations, while software and IT operations aim at streamlining software deployment in products.
Typical problems addressed by MLOps include data management (labeling, organization, storage), ML model and pipeline training repeatability, error analysis, model integration and deployment and model monitoring. In this talk, Dal Mutto and Balafas present practical MLOps techniques useful for tackling a variety of MLOps needs. They illustrate these techniques with real-world examples from their work developing autonomous flying capabilities as part of the Wayfinder team at Acubed, the Silicon Valley innovation center of Airbus.
[DSC Europe 23] Spela Poklukar & Tea Brasanac - Retrieval Augmented Generation
[DSC Europe 23] Spela Poklukar & Tea Brasanac - Retrieval Augmented GenerationDataScienceConferenc1 Retrieval Augmented Generation (RAG) combines the concepts of semantic search and LLM-based text generation. When a person makes a query in natural language, the query is compared to the entries in the knowledge base and most relevant results are returned to the LLM, which uses this extra information to generate more accurate and reliable response. RAG can therefore limit hallucination and provide accurate responses from reliable source. In this talk, we will present the concept of RAG and underlying concept of semantic search, and present available libraries and vector databases.
Making Testing Easy w GitHub Copilot.pdf
Making Testing Easy w GitHub Copilot.pdfApplitools The document discusses GitHub Copilot, an AI programming assistant. It notes that Copilot can help reduce the pain of writing tests by providing predictive text and increasing a developer's velocity. Several use cases for Copilot are provided, including writing better comments, pair programming, writing tests, and live coding. Copilot Labs is also discussed as helping developers become comfortable in new codebases, understand StackOverflow solutions, and gain additional context on data structures. The document concludes with examples of using Copilot and a reminder that it is still in beta.
Retrieval Augmented Generation in Practice: Scalable GenAI platforms with k8s...
Retrieval Augmented Generation in Practice: Scalable GenAI platforms with k8s...Mihai Criveti Mihai is the Principal Architect for Platform Engineering and Technology Solutions at IBM, responsible for Cloud Native and AI Solutions. He is a Red Hat Certified Architect, CKA/CKS, a leader in the IBM Open Innovation community, and advocate for open source development. Mihai is driving the development of Retrieval Augmentation Generation platforms, and solutions for Generative AI at IBM that leverage WatsonX, Vector databases, LangChain, HuggingFace and open source AI models.
Mihai will share lessons learned building Retrieval Augmented Generation, or “Chat with Documents” platforms and APIs that scale, and deploy on Kubernetes. His talk will cover use cases for Generative AI, limitations of Large Language Models, use of RAG, Vector Databases and Fine Tuning to overcome model limitations and build solutions that connect to your data and provide content grounding, limit hallucinations and form the basis of explainable AI. In terms of technology, he will cover LLAMA2, HuggingFace TGIS, SentenceTransformers embedding models using Python, LangChain, and Weaviate and ChromaDB vector databases. He’ll also share tips on writing code using LLM, including building an agent for Ansible and containers.
Scaling factors for Large Language Model Architectures:
• Vector Database: consider sharding and High Availability
• Fine Tuning: collecting data to be used for fine tuning
• Governance and Model Benchmarking: how are you testing your model performance
over time, with different prompts, one-shot, and various parameters
• Chain of Reasoning and Agents
• Caching embeddings and responses
• Personalization and Conversational Memory Database
• Streaming Responses and optimizing performance. A fine tuned 13B model may
perform better than a poor 70B one!
• Calling 3rd party functions or APIs for reasoning or other type of data (ex: LLMs are
terrible at reasoning and prediction, consider calling other models)
• Fallback techniques: fallback to a different model, or default answers
• API scaling techniques, rate limiting, etc.
• Async, streaming and parallelization, multiprocessing, GPU acceleration (including
embeddings), generating your API using OpenAPI, etc.
Generative AI con Amazon Bedrock.pdf
Generative AI con Amazon Bedrock.pdfGuido Maria Nebiolo Generative AI models like LLMs can be customized for specific tasks through techniques like prompt engineering, retrieval augmented generation, and fine-tuning. Prompt engineering involves providing contextual information to steer model responses, while retrieval augmented generation combines generative and retrieval models to improve performance. Fine-tuning customizes foundation models with domain-specific training data. The document discusses these techniques and their benefits, encouraging hands-on experience to learn how to best apply generative AI.
Introduction to GitHub Copilot
Introduction to GitHub CopilotAll Things Open Presented at All Things Open RTP Meetup
Presented by Brent Laster
Title: Introduction to GitHub Copilot
Abstract: Learn about the future of coding with this presentation, "Introduction to GitHub Copilot." Join author, speaker, and DevOps director Brent Laster as we discuss the GitHub coding assistant that leverages the power of artificial intelligence to augment your development activities. In this session, we'll delve into the core features of GitHub Copilot, explaining what it is, and learning about its ability to generate code snippets, autocompletions, and even entire functions across a variety of programming languages.
Learn how Copilot enhances your coding efficiency by contextualizing suggestions based on your codebase and offering a seamless integration into popular code editors. We'll explore some example applications and look at how developers can harness Copilot to streamline their workflows and boost productivity. Whether you're a seasoned developer or just embarking on your coding journey, this presentation offers a quick introduction to GitHub Copilot's capabilities, as a starting point for you to embrace cutting-edge AI assistance in your coding endeavors.
Large language models in higher education
Large language models in higher educationPeter Trkman Discussing the possibilities of large language models for the automatic generation of academic content by the students (e.g. master thesis), and the related need for changes in the way in which to educate and evaluate students.
OpenAI-Copilot-ChatGPT.pptx
OpenAI-Copilot-ChatGPT.pptxUdaiappa Ramachandran ChatGPT (Chat Generative pre-defined transformer) is OpenAI's application that performs human like interactions. GitHub Copilot uses the OpenAI Codex to suggest code and entire functions in real-time, right from your editor. Deck contains more details about ChatGPT, AI, AGI, CoPilot, OpenAI API, and use case scenarios.
Best Practice on using Azure OpenAI Service
Best Practice on using Azure OpenAI ServiceKumton Suttiraksiri This document provides 7 best practices for using the Azure OpenAI Service:
1. Set clear goals and objectives for your prompts.
2. Choose the appropriate AI model like GPT-3, Ada, or Davinci based on your task's complexity and required capabilities.
3. Ensure prompts are precise yet not too short to achieve the desired response.
Build an LLM-powered application using LangChain.pdf
Build an LLM-powered application using LangChain.pdfAnastasiaSteele10 LangChain is an advanced framework that allows developers to create language model-powered applications. It provides a set of tools, components, and interfaces that make building LLM-based applications easier. With LangChain, managing interactions with language models, chaining together various components, and integrating resources like APIs and databases is a breeze. The platform includes a set of APIs that can be integrated into applications, allowing developers to add language processing capabilities without having to start from scratch.
Mohamed Sabri: Operationalize machine learning with Kubeflow
Mohamed Sabri: Operationalize machine learning with KubeflowEdunomica Data Science Online Camp 2021
Website - https://dscamp.org/
Youtube - https://www.youtube.com/channel/UCeHtPZ_ZLZ-nHFMUCXY81RQ/featured
FB - https://www.facebook.com/Data-Science-Camp-103012708431833
Copilot to Cover: Why AI can't replace developers with robots, but can make l...
Copilot to Cover: Why AI can't replace developers with robots, but can make l...Andy Piper AI-augmented tools like GitHub Copilot, AWS CodeWhisperer, and Diffblue Cover use machine learning to help developers with repetitive coding tasks like completing boilerplate code, calling APIs, and writing tests. These tools generate code fragments or entire functions to reduce errors and speed up development. While they are helpful for mundane tasks, fully autonomous coding remains a challenge, as writing high-quality tests requires understanding complex code dependencies and business requirements. Diffblue's reinforcement learning approach aims to autonomously write tests that satisfy developer needs and increase code coverage. These AI-assisted development tools are available today and can help eliminate tedious coding work.
re:cap Generative AI journey with Bedrock
re:cap Generative AI journey with BedrockPhilipBasford Wherever you are on your Generative AI journey — Amazon Bedrock allows you to rapidly prototype Generative AI concepts, using the latest Foundational Models. This session also included architectures to accelerate your prototype into a real-world GenAI solution using LLMOps. Providing the safeguards to keep your data private & secure, handle any regulatory compliance and responsibility requirements.
Let's talk about GPT: A crash course in Generative AI for researchers
Let's talk about GPT: A crash course in Generative AI for researchersSteven Van Vaerenbergh This talk delves into the extraordinary capabilities of the emerging technology of generative AI, outlining its recent history and emphasizing its growing influence on scientific endeavors. Through a series of practical examples tailored for researchers, we will explore the transformative influence of these powerful tools on scientific tasks such as writing, coding, data wrangling and literature review.
Ml ops past_present_future
Ml ops past_present_futureNisha Talagala Given at the MLOps. Summit 2020 - I cover the origins of MLOps in 2018, how MLOps has evolved from 2018 to 2020, and what I expect for the future of MLOps
How will development change with LLMs
How will development change with LLMsMicrosoft, InfuseAI, Appier, IBM, KaiOS GPT discusses various ways that language models can acquire external information as context to improve responses, including:
1) Querying search engines using APIs to incorporate search results into responses
2) Recognizing tasks from prompts and accessing databases or APIs to incorporate relevant information
3) Summarizing, calculating, and verifying information from external sources to provide more accurate answers
Using the power of Generative AI at scale
Using the power of Generative AI at scaleMaxim Salnikov In this session, you'll get all the answers about how ChatGPT and other GPT-X models can be applied to your current or future project. First, we'll put in order all the terms – OpenAI, GPT-3, ChatGPT, Codex, Dall-E, etc., and explain why Microsoft and Azure are often mentioned in this context. Then, we'll go through the main capabilities of the Azure OpenAI and respective usecases that might inspire you to either optimize your product or build a completely new one.
The Top 5 Apache Kafka Use Cases and Architectures in 2022
The Top 5 Apache Kafka Use Cases and Architectures in 2022Kai Wähner This document discusses the top 5 use cases and architectures for data in motion in 2022. It describes:
1) The Kappa architecture as an alternative to the Lambda architecture that uses a single stream to handle both real-time and batch data.
2) Hyper-personalized omnichannel experiences that integrate customer data from multiple sources in real-time to provide personalized experiences across channels.
3) Multi-cloud deployments using Apache Kafka and data mesh architectures to share data across different cloud platforms.
4) Edge analytics that deploy stream processing and Kafka brokers at the edge to enable low-latency use cases and offline functionality.
5) Real-time cybersecurity applications that use streaming data
IoT Architectures for Apache Kafka and Event Streaming - Industry 4.0, Digita...
IoT Architectures for Apache Kafka and Event Streaming - Industry 4.0, Digita...Kai Wähner The Internet of Things (IoT) is getting more and more traction as valuable use cases come to light. Whether you are in Healthcare, Telecommunications, Manufacturing, Banking or Retail to name a few industries, there is one key challenge and that's the integration of backend IoT data logs and applications, business services and cloud services to process the data in real time and at scale.
In this talk, we will be sharing how Kafka has become the leading technology used throughout the business to provide Real Time Event Streaming. Explore real life use cases of Kafka Connect, Kafka Streams and KSQL independent of the data deployment be it on a private or public Cloud, On Premise or at the Edge.
Audi - Connected car infrastructure
Robert Bosch Power Tools - Track and Trace of devices and people at construction areas
Deutsche Bahn - Customer 360 for train timetable updates
E.ON - IoT Streaming Platform to integrate and build smart home, smart building and smart grid infrastructures
“MLOps: Managing Data and Workflows for Efficient Model Development and Deplo...
“MLOps: Managing Data and Workflows for Efficient Model Development and Deplo...Edge AI and Vision Alliance
Similar to [EN]DSS23_tspann_Integrating LLM with Streaming Data Pipelines (20)
The Never Landing Stream with HTAP and Streaming
The Never Landing Stream with HTAP and StreamingTimothy Spann The Never Landing Stream with HTAP and Streaming
Apache NiFi
Apache Kudu
Apache Flink
Apache Kafka
PingCap
Streaming
Cloudera
Budapest Data/ML - Building Modern Data Streaming Apps with NiFi, Flink and K...
Budapest Data/ML - Building Modern Data Streaming Apps with NiFi, Flink and K...Timothy Spann Budapest Data/ML - Building Modern Data Streaming Apps with NiFi, Flink and Kafka
Apache NiFi, Apache Flink, Apache Kafka
Timothy Spann
Principal Developer Advocate
Cloudera
Data in Motion
https://budapestdata.hu/2023/en/speakers/timothy-spann/
Timothy Spann
Principal Developer Advocate
Cloudera (US)
LinkedIn · GitHub · datainmotion.dev
June 8 · Online · English talk
Building Modern Data Streaming Apps with NiFi, Flink and Kafka
In my session, I will show you some best practices I have discovered over the last 7 years in building data streaming applications including IoT, CDC, Logs, and more.
In my modern approach, we utilize several open-source frameworks to maximize the best features of all. We often start with Apache NiFi as the orchestrator of streams flowing into Apache Kafka. From there we build streaming ETL with Apache Flink SQL. We will stream data into Apache Iceberg.
We use the best streaming tools for the current applications with FLaNK. flankstack.dev
BIO
Tim Spann is a Principal Developer Advocate in Data In Motion for Cloudera. He works with Apache NiFi, Apache Pulsar, Apache Kafka, Apache Flink, Flink SQL, Apache Pinot, Trino, Apache Iceberg, DeltaLake, Apache Spark, Big Data, IoT, Cloud, AI/DL, machine learning, and deep learning. Tim has over ten years of experience with the IoT, big data, distributed computing, messaging, streaming technologies, and Java programming.
Previously, he was a Developer Advocate at StreamNative, Principal DataFlow Field Engineer at Cloudera, a Senior Solutions Engineer at Hortonworks, a Senior Solutions Architect at AirisData, a Senior Field Engineer at Pivotal and a Team Leader at HPE. He blogs for DZone, where he is the Big Data Zone leader, and runs a popular meetup in Princeton & NYC on Big Data, Cloud, IoT, deep learning, streaming, NiFi, the blockchain, and Spark. Tim is a frequent speaker at conferences such as ApacheCon, DeveloperWeek, Pulsar Summit and many more. He holds a BS and MS in computer science.
Confluent kafka meetupseattle jan2017
Confluent kafka meetupseattle jan2017Nitin Kumar This document provides an overview of the Confluent streaming platform and Apache Kafka. It discusses how streaming platforms can be used to publish, subscribe and process streams of data in real-time. It also highlights challenges with traditional architectures and how the Confluent platform addresses them by allowing data to be ingested from many sources and processed using stream processing APIs. The document also summarizes key components of the Confluent platform like Kafka Connect for streaming data between systems, the Schema Registry for ensuring compatibility, and Control Center for monitoring the platform.
Building Real-Time Travel Alerts
Building Real-Time Travel AlertsTimothy Spann Building Real-time Travel Alerts
In this session, we will walk through how to build a complete streaming application to send alerts based on travel advisories from public data. We will also join in other data sources of relevance and push out alerts.
We will show you how to build this streaming application with Apache NiFi, Apache Kafka, and Apache Flink and show you when/why/how, and what to build to maximize performance, productivity, and ease of development.
Let's get streaming.
Apache Flink
Apache Kafka
Apache NiFi
FLaNK Stack
Tim Spann
Big Data Conference Europe 2023
Streaming Visualization
Streaming VisualizationGuido Schmutz Most data visualisation solutions today still work on data sources which are stored persistently in a data store, using the so called “data at rest” paradigms. More and more data sources today provide a constant stream of data, from IoT devices to Social Media streams. These data stream publish with high velocity and messages often have to be processed as quick as possible. For the processing and analytics on the data, so called stream processing solutions are available. But these only provide minimal or no visualisation capabilities. One was is to first persist the data into a data store and then use a traditional data visualisation solution to present the data.
If latency is not an issue, such a solution might be good enough. An other question is which data store solution is necessary to keep up with the high load on write and read. If it is not an RDBMS but an NoSQL database, then not all traditional visualisation tools might already integrate with the specific data store. An other option is to use a Streaming Visualisation solution. They are specially built for streaming data and often do not support batch data. A much better solution would be to have one tool capable of handling both, batch and streaming data. This talk presents different architecture blueprints for integrating data visualisation into a fast data solution and highlights some of the products available to implement these blueprints.
Time's Up! Getting Value from Big Data Now
Time's Up! Getting Value from Big Data NowEric Kavanagh
The Briefing Room with Dr. Robin Bloor and CASK
We all know the promise of big data, but who gets the value? There are plenty of success stories already, and most of them involve one key ingredient: facilitated access to important data sets. Most research studies suggest that the Pareto principle applies: 80 percent goes to data integration, and only 20 to analysis. Inverting that balance is the Holy Grail.
Register for this episode of The Briefing Room to hear veteran Analyst Dr. Robin Bloor explain why the time has finally come for turning the tables on the status quo in analytics. He'll be briefed by CASK CEO Jonathan Gray, who will showcase his company's big data integration platform, CDAP, which was specifically designed to expedite time-to-value for big data.
Streaming Data Ingest and Processing with Apache Kafka
Streaming Data Ingest and Processing with Apache KafkaAttunity Apache™ Kafka is a fast, scalable, durable, and fault-tolerant
publish-subscribe messaging system. It offers higher throughput, reliability and replication. To manage growing data volumes, many companies are leveraging Kafka for streaming data ingest and processing.
Join experts from Confluent, the creators of Apache™ Kafka, and the experts at Attunity, a leader in data integration software, for a live webinar where you will learn how to:
-Realize the value of streaming data ingest with Kafka
-Turn databases into live feeds for streaming ingest and processing
-Accelerate data delivery to enable real-time analytics
-Reduce skill and training requirements for data ingest
The recorded webinar on slide 32 includes a demo using automation software (Attunity Replicate) to stream live changes from a database into Kafka and also includes a Q&A with our experts.
For more information, please go to www.attunity.com/kafka.
OSSNA Building Modern Data Streaming Apps
OSSNA Building Modern Data Streaming AppsTimothy Spann OSSNA
Building Modern Data Streaming Apps
https://ossna2023.sched.com/event/1Jt05/virtual-building-modern-data-streaming-apps-with-open-source-timothy-spann-streamnative
Timothy Spann
Cloudera
Principal Developer Advocate
Data in Motion
In my session, I will show you some best practices I have discovered over the last seven years in building data streaming applications, including IoT, CDC, Logs, and more. In my modern approach, we utilize several open-source frameworks to maximize all the best features. We often start with Apache NiFi as the orchestrator of streams flowing into Apache Pulsar. From there, we build streaming ETL with Apache Spark and enhance events with Pulsar Functions for ML and enrichment. We make continuous queries against our topics with Flink SQL. We will stream data into various open-source data stores, including Apache Iceberg, Apache Pinot, and others. We use the best streaming tools for the current applications with the open source stack - FLiPN. https://www.flipn.app/ Updates: This will be in-person with live coding based on feedback from the crowd. This will also include new data stores, new sources, and data relevant to and from the Vancouver area. This will also include updates to the platforms and inclusion of Apache Iceberg, Apache Pinot and some other new tech.
https://github.com/tspannhw/SpeakerProfile Tim Spann is a Principal Developer Advocate for Cloudera. He works with Apache Kafka, Apache Flink, Flink SQL, Apache NiFi, MiniFi, Apache MXNet, TensorFlow, Apache Spark, Big Data, the IoT, machine learning, and deep learning. Tim has over a decade of experience with the IoT, big data, distributed computing, messaging, streaming technologies, and Java programming. Previously, he was a Principal DataFlow Field Engineer at Cloudera, a Senior Solutions Engineer at Hortonworks, a Senior Solutions Architect at AirisData, a Senior Field Engineer at Pivotal and a Team Leader at HPE. He blogs for DZone, where he is the Big Data Zone leader, and runs a popular meetup in Princeton on Big Data, Cloud, IoT, deep learning, streaming, NiFi, the blockchain, and Spark. Tim is a frequent speaker at conferences such as ApacheCon, DeveloperWeek, Pulsar Summit and many more. He holds a BS and MS in computer science.
Timothy J Spann
Cloudera
Principal Developer Advocate
Hightstown, NJ
Websitehttps://datainmotion.dev/
Streaming Data and Stream Processing with Apache Kafka
Streaming Data and Stream Processing with Apache Kafkaconfluent Apache Kafka is an open-source streaming platform that can be used to build real-time data pipelines and streaming applications. It addresses challenges with diverse data sets arriving at increasing rates. The document discusses how Apache Kafka can help with challenges around data integration, stream processing, and managing streaming platforms at scale. It also outlines key features of Apache Kafka like the Kafka Connect API for data integration, the Kafka Streams API for stream processing, and Confluent Control Center for monitoring and management.
JConWorld_ Continuous SQL with Kafka and Flink
JConWorld_ Continuous SQL with Kafka and FlinkTimothy Spann JConWorld: Continuous SQL with Kafka and Flink
In this talk, I will walk through how someone can setup and run continous SQL queries against Kafka topics utilizing Apache Flink. We will walk through creating Kafka topics, schemas and publishing data.
We will then cover consuming Kafka data, joining Kafka topics and inserting new events into Kafka topics as they arrive. This basic over view will show hands-on techniques, tips and examples of how to do this.
Tim Spann is the Principal Developer Advocate for Data in Motion @ Cloudera where he works with Apache Kafka, Apache Flink, Apache NiFi, Apache Iceberg, TensorFlow, Apache Spark, big data, the IoT, machine learning, and deep learning. Tim has over a decade of experience with the IoT, big data, distributed computing, streaming technologies, and Java programming. Previously, he was a Developer Advocate at StreamNative, Principal Field Engineer at Cloudera, a Senior Solutions Architect at AirisData and a senior field engineer at Pivotal. He blogs for DZone, where he is the Big Data Zone leader, and runs a popular meetup in Princeton on big data, the IoT, deep learning, streaming, NiFi, the blockchain, and Spark. Tim is a frequent speaker at conferences such as IoT Fusion, Strata, ApacheCon, Data Works Summit Berlin, DataWorks Summit Sydney, and Oracle Code NYC. He holds a BS and MS in computer science. https://www.datainmotion.dev/p/about-me.html https://dzone.com/users/297029/bunkertor.html
https://www.youtube.com/channel/UCDIDMDfje6jAvNE8DGkJ3_w?view_as=subscriber
GSJUG: Mastering Data Streaming Pipelines 09May2023
GSJUG: Mastering Data Streaming Pipelines 09May2023Timothy Spann GSJUG: Mastering Data Streaming Pipelines 09May2023
https://www.meetup.com/futureofdata-princeton/events/293233881/
This is a repost from the Garden State Java Users Group Event.
Join me at
https://www.meetup.com/garden-state-java-user-group/events/293229660/
See: https://www.eventbrite.com/e/mastering-data-streaming-pipelines-tickets-627677218457?_ga=2.253257801.1787151623.1682868226-741104479.1678110925
Please note that registration via EventBrite is required to attend either in-person or online.
We are happy to announce that Tim Spann will be our special guest for the May 9, 2023 meeting!
Abstract:
In this session, Tim will show you some best practices that he has discovered over the last seven years in building data streaming applications including IoT, CDC, Logs, and more.
In his modern approach, we utilize several Apache frameworks to maximize the best features of all. We often start with Apache NiFi as the orchestrator of streams flowing into Apache Kafka. From there we build streaming ETL with Apache Flink, enhance events with NiFi enrichment. We build continuous queries against our topics with Flink SQL.
We will show where Java fits in as sources, enrichments, NiFi processors and sinks.
We hope to see you on May 9!
Speaker
Timothy Spann
Tim Spann is a Principal Developer Advocate in Data In Motion for Cloudera. He works with Apache NiFi, Apache Pulsar, Apache Kafka, Apache Flink, Flink SQL, Apache Pinot, Trino, Apache Iceberg, DeltaLake, Apache Spark, Big Data, IoT, Cloud, AI/DL, machine learning, and deep learning. Tim has over ten years of experience with the IoT, big data, distributed computing, messaging, streaming technologies, and Java programming.
Previously, he was a Developer Advocate at StreamNative, Principal DataFlow Field Engineer at Cloudera, a Senior Solutions Engineer at Hortonworks, a Senior Solutions Architect at AirisData, a Senior Field Engineer at Pivotal and a Team Leader at HPE. He blogs for DZone, where he is the Big Data Zone leader, and runs a popular meetup in Princeton & NYC on Big Data, Cloud, IoT, deep learning, streaming, NiFi, the blockchain, and Spark. Tim is a frequent speaker at conferences such as ApacheCon, DeveloperWeek, Pulsar Summit and many more. He holds a BS and MS in computer science.
In this session, Tim will show you some best practices that he has discovered over the last seven years in building data streaming applications, including IoT, CDC, Logs, and more.
In his modern approach, we utilize several Apache frameworks to maximize the best features of all. We often start with Apache NiFi as the orchestrator of streams flowing into Apache Kafka. From there, we build streaming ETL with Apache Flink, enhance events with NiFi enrichment. We build continuous queries against our topics with Flink SQL.
We will show where Java fits in as sources, enrichments, NiFi processors, and sinks.
https://www.eventbrite.com/e/mastering-data-streaming-pipelines-tickets-627677218457?_ga=2.253257801.178
Evolve 2023 NYC - Integrating AI Into Realtime Data Pipelines Demo
Evolve 2023 NYC - Integrating AI Into Realtime Data Pipelines DemoTimothy Spann Evolve 2023 NYC - Integrating AI Into Realtime Data Pipelines Demo
Apache NiFi - Cloudera DataFlow
Apache Flink - Cloudera Streaming Analytics
Apache Kafka - Streaming
LLM / WatsonX.AI Generative AI
granite and llama2 models
slack to slack apps
What's new in Confluent 3.2 and Apache Kafka 0.10.2 
What's new in Confluent 3.2 and Apache Kafka 0.10.2 confluent With the introduction of connect and streams API in 2016, Apache Kafka is becoming the defacto solution for anyone looking to build a streaming platform. The community continues to add additional capabilities to make it the complete solution for streaming data.
Join us as we review the latest additions in Apache Kafka 0.10.2. In addition, we’ll cover what’s new in Confluent Enterprise 3.2 that makes it possible for running Kafka at scale.
Streaming Visualization
Streaming VisualizationGuido Schmutz Most data visualisation solutions today still work on data sources which are stored persistently in a data store, using the so called “data at rest” paradigms. More and more data sources today provide a constant stream of data, from IoT devices to Social Media streams. These data stream publish with high velocity and messages often have to be processed as quick as possible. For the processing and analytics on the data, so called stream processing solutions are available. But these only provide minimal or no visualisation capabilities. One option is to first persist the data into a data store and then use a traditional data visualisation solution to present the data. If latency is not an issue, such a solution might be good enough. An other question is which data store solution is necessary to keep up with the high load on write and read. If it is not an RDBMS but an NoSQL database, then not all traditional visualisation tools might already integrate with the specific data store. An other option is to use a Streaming Visualisation solution. They are specially built for streaming data and often do not support batch data. A much better solution would be to have one tool capable of handling both, batch and streaming data. This talk presents different architecture blueprints for integrating data visualisation into a fast data solution and then we show how the different blueprints can be implemented by mapping products onto the blueprints.
Unconference Round Table Notes
Unconference Round Table NotesTimothy Spann Unconference Round Table Notes
The future of real-time stream processing
WASM (Web Assembly)
Petabyte, 5000 Node Clusters, Smart Hyper Scaling
Multi-language support (Python, Rust, Kotlin, Golang, Carbon, JVM)
Machine Learning, Deep Learning, AI and Advanced Math
Low Code Development like Apache NiFi, DataFlow Designer, SQL
Dynamic Hybrid Deployment
Citizen Stream Engineer
IoT, Edge Streaming and Hybrid Edge Streaming
Java 20, 21; Java Loom Virtual Threading
Ultra low latency, trillions of events per second, massive RAM/network
Current challenges of real-time stream processing and proposed solutions
Deployment, Automation and Scaling
Choosing right project/sizing for use case
Simple Event Processing vs Complex Event Processing
Leveraging existing applications
Developer Skills
Self management and monitoring
Cost issues -> autoscaling, optimizing, performance, hybrid deployment
Performance / Benchmarking real-time stream processing
Kafka/Pulsar: https://openmessaging.cloud/docs/benchmarks/
NiFi: https://blog.cloudera.com/benchmarking-nifi-performance-and-scalability/
Flink: https://github.com/ververica/flink-sql-benchmark
Hazelcast: https://hazelcast.com/press-release/hazelcast-demonstrates-cloud-efficiency-real-time-stream-processing-of-one-billion-events-per-second/
Current trends of real-time stream processing in 2023
Current challenges of real-time stream processing and proposed solutions
Performance / Benchmarking real-time stream processing
The future of real-time stream processing
Current trends of real-time stream processing in 2023
Lightweight serverless
Hazelcast
SQL
Flink
Kafka or Pulsar as Messaging Hub
Java 17+
Managed Clusters, Containers and Environments
Real-Time Analytics
Fast Storage Options
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNKTimothy Spann Building Real-Time Pipelines With FLaNK
Timothy Spann, Principal Developer Advocate, Streaming - Cloudera Future of Data meetup, startup grind, AI Camp
The combination of Apache Flink, Apache NiFi, and Apache Kafka for building real-time data processing pipelines is extremely powerful, as demonstrated by this case study using the FLaNK-MTA project. The project leverages these technologies to process and analyze real-time data from the New York City Metropolitan Transportation Authority (MTA). FLaNK-MTA demonstrates how to efficiently collect, transform, and analyze high-volume data streams, enabling timely insights and decision-making.
Apache NiFi
Apache Kafka
Apache Flink
Apache Iceberg
LLM
Generative AI
Slack
Postgresql
Stream All Things—Patterns of Modern Data Integration with Gwen Shapira
Stream All Things—Patterns of Modern Data Integration with Gwen ShapiraDatabricks This document discusses patterns for modern data integration using streaming data. It outlines an evolution from data warehouses to data lakes to streaming data. It then describes four key patterns: 1) Stream all things (data) in one place, 2) Keep schemas compatible and process data on, 3) Enable ridiculously parallel single message transformations, and 4) Perform streaming data enrichment to add additional context to events. Examples are provided of using Apache Kafka and Kafka Connect to implement these patterns for a large hotel chain integrating various data sources and performing real-time analytics on customer events.
Streaming Visualization
Streaming VisualizationGuido Schmutz The document discusses three blueprints for streaming visualization:
1. Using a fast datastore with regular polling from consumers, which introduces some delay but allows using data stores' full capabilities. Example technologies are Elasticsearch/Kibana and InfluxDB/Grafana.
2. Directly streaming data to consumers with minimal latency but more complex client-side processing. Examples are Kafka Connect to Slack and WebSockets/SSE apps.
3. Streaming SQL results to consumers, providing SQL query capabilities with minimal latency but limiting historical data access. KSQL and Spark Streaming are discussed.
Data Pipeline for The Big Data/Data Science OKC
Data Pipeline for The Big Data/Data Science OKCMark Smith The document discusses and evaluates several data pipeline platforms: Spark Structured Streaming, Spring Cloud Data Flow, Apache NIFI, and AWS Glue. It provides an overview of each platform and evaluates them based on several criteria such as real-time processing, managing failures and duplicates, security, scaling to large data sets, and integration with machine learning and data catalogs. Overall, AWS Glue received strong ratings for its data catalog integration, extraction and transformation capabilities as an ETL tool, while Spark Structured Streaming, Apache NIFI, and Spring Cloud Data Flow demonstrated strengths in real-time processing, scalability, and maturity.
Streaming etl in practice with postgre sql, apache kafka, and ksql mic
Streaming etl in practice with postgre sql, apache kafka, and ksql micBas van Oudenaarde This document discusses using Apache Kafka, Kafka Connect, and KSQL to perform streaming ETL from a PostgreSQL database. It describes how to integrate PostgreSQL with Kafka using Kafka Connect and the Debezium PostgreSQL connector. It then shows how to use KSQL to perform streaming transformations on the PostgreSQL data in Kafka, including joining, filtering, and aggregating data streams in real-time. The document promotes using this approach for applications like real-time monitoring, anomaly detection, and analytics to enable streaming data pipelines and transform data once for multiple uses.
Ad
More from Timothy Spann (20)
14May2025_TSPANN_FromAirQualityUnstructuredData.pdf
14May2025_TSPANN_FromAirQualityUnstructuredData.pdfTimothy Spann From Air Quality to Aircraft
Apache NiFi
Snowflake
Apache Iceberg
AI
GenAI
LLM
RAG
https://www.dbta.com/DataSummit/2025/Timothy-Spann.aspx
Tim Spann is a Senior Sales Engineer @ Snowflake. He works with Generative AI, LLM, Snowflake, SQL, HuggingFace, Python, Java, Apache NiFi, Apache Kafka, Apache Pulsar, Apache Flink, Flink SQL, Apache Spark, Big Data, IoT, Cloud, AI/DL, Machine Learning, and Deep Learning. Tim has over ten years of experience with the IoT, big data, distributed computing, messaging, streaming technologies, and Java programming. Previously, he was a Principal Developer Advocate at Zilliz, Principal Developer Advocate at Cloudera, Developer Advocate at StreamNative, Principal DataFlow Field Engineer at Cloudera, a Senior Solutions Engineer at Hortonworks, a Senior Solutions Architect at AirisData, a Senior Field Engineer at Pivotal and a Senior Team Leader at HPE. He blogs for DZone, where he is the Big Data Zone leader, and runs a popular meetup in Princeton & NYC on Big Data, Cloud, IoT, deep learning, streaming, NiFi, the blockchain, and Spark. Tim is a frequent speaker at conferences such as ApacheCon, DeveloperWeek, Pulsar Summit and many more. He holds a BS and MS in Computer Science.
https://github.com/tspannhw/SpeakerProfile
https://www.dbta.com/DataSummit/2025/program.aspx#17305
From Air Quality to Aircraft & Automobiles, Unstructured Data Is Everywhere
Spann explores how Apache NiFi can be used to integrate open source LLMs to implement scalable and efficient RAG pipelines. He shows how any kind of data including semistructured, structured and unstructured data from a variety of sources and types can be processed, queried, and used to feed large language models for smart, contextually aware answers. Look for his example utilizing Cortex AI, LLAMA, Apache NiFi, Apache Iceberg, Snowflake, open source tools, libraries, and Notebooks.
Speaker:
Timothy Spann, Senior Solutions Engineer, Snowflake
may 14 2025
boston
Streaming AI Pipelines with Apache NiFi and Snowflake NYC 2025
Streaming AI Pipelines with Apache NiFi and Snowflake NYC 2025Timothy Spann Streaming AI Pipelines with Apache NiFi and Snowflake 2025
1. Streaming AI Pipelines with Apache NiFi and Snowflake Tim Spann, Senior Solutions Engineer
2. Tim Spann paasdev.bsky.social @PaasDev // Blog: datainmotion.dev Senior Solutions Engineer, Snowflake NY/NJ/Philly - Cloud Data + AI Meetups ex-Zilliz, ex-Pivotal, ex-Cloudera, ex-HPE, ex-StreamNative, ex-EY, ex-Hortonworks. https://medium.com/@tspann https://github.com/tspannhw
3. This week in Apache NiFi, Apache Polaris, Apache Flink, Apache Kafka, ML, AI, Streamlit, Jupyter, Apache Iceberg, Python, Java, LLM, GenAI, Snowflake, Unstructured Data and Open Source friends. https://bit.ly/32dAJft DATA + AI + Streaming Weekly
4. How Snowflake and Apache NiFi work with Streaming Data and AI
5. Building Streaming Data + AI Pipelines Requires a Team
6. Example Smart City Architecture 6 DATA SOURCES DATA INTEGRATION DATA PLATFORM DATA CONSUMERS Marketplace Raw Data Modeled Data Snowpipe Sensors Transit Data AI/ML & Apps Weather Traffic Data SNOWSIGHT Snowflake Cortex AI Raw Data DATA FROM THE REAL WORLD I Can Haz Data? Camera Images
7. Apache NiFi ● From laptop to 1,000 nodes ● Ingest, Extract, Split ● Enrich, Transform ● Mature, 10 years+ ● Any Data, Any Source ● LLM Calls ● Data Provenance ● Back Pressure ● Guaranteed Delivery
8. Unstructured Data ● Lots of formats ● Text, Documents, PDF ● Images, Videos, Audio ● Email, Slack, Teams ● Logs ● Binary Data Formats ● Zip ● Variants Unstructured
9. ● Open Data like Open AQ - Air Quality Data ● Location, Time,Sensors ● Apache Avro, Parquet, Orc ● JSON and XML ● Hierarchical Data ● Logs ● Key-Value Semi-Structured Data https://docs.snowflake.com/en/sql-refe rence/data-types-semistructured Semi-structured
10. Structured Data ● Snowflake Tables ● Snowflake Hybrid Tables ● Apache Iceberg Tables ● Relational Tables ● Postgresql Tables ● CSV, TSV Structured
11. Open LLM Options ● Arctic Instruct ● Arctic-embed-m-v2.0 ● Llama-3.3-70b ● Mixtral-8x7b ● Llama3.1-405b ● Mistral-7b ● Deepseek-r1
Streaming AI Pipelines with Apache NiFi and Snowflake 2025
Real-time AI with Tim Spann
https://lu.ma/0av3pvoa?tk=Ebmrn0
Thursday, March 20
6:00 PM - 9:00 PM
NYC Data + AI Happy Hour!
👥 Who’s invited?
If you’re passionate about real-time data and/or AI—or simply eager to connect with data and AI enthusiasts—this event is for you!
🏙️ Where is it happening?
Join us at Rodney's, 1118 1st Avenue, New York, NY 10065
🎯 Why attend?
Dive into the latest trends in data engineering and AI
Connect with industry peers and potential collaborators
Showcase your groundbreaking ideas and solutions in data streaming and/or AI
Recruit top talent for your data team or explore new career opportunities
Discover cutting-edge tools and technologies shaping the field
📅 Event Program
6:00 PM: Doors Open
6:30 PM - 7:30 PM: Welcome & Networking
7:30 PM - 8:00 PM: Lightning Talks
Yingjun Wu (RisingWave)
Quentin Packard (Conduktor)
Tim Spann (Snowflake)
Ciro
2025-03-03-Philly-AAAI-GoodData-Build Secure RAG Apps With Open LLM
2025-03-03-Philly-AAAI-GoodData-Build Secure RAG Apps With Open LLMTimothy Spann 2025-03-03-Philly-AAAI-GoodData-Build Secure RAG Apps With Open LLM
https://aaai.org/conference/aaai/aaai-25/workshop-list/#ws14
Conf42_IoT_Dec2024_Building IoT Applications With Open Source
Conf42_IoT_Dec2024_Building IoT Applications With Open SourceTimothy Spann Conf42_IoT_Dec2024_Building IoT Applications With Open Source
Tim Spann
https://www.conf42.com/Internet_of_Things_IoT_2024_Tim_Spann_opensource_build
Conf42 Internet of Things (IoT) 2024 - Online
December 19 2024 - premiere 5PM GMT
Thu Dec 19 2024 12:00:00 GMT-0500 (Eastern Standard Time) in America/New_York
Building IoT Applications With Open Source
Abstract
Utilizing open-source software, we can easily build open-source IoT applications that run on commercial and enterprise hardware anywhere.
2024 Dec 05 - PyData Global - Tutorial Its In The Air Tonight
2024 Dec 05 - PyData Global - Tutorial Its In The Air TonightTimothy Spann 2024 Dec 05 - PyData Global - Tutorial Its In The Air Tonight
https://pydata.org/global2024/schedule
Tim Spann
https://www.youtube.com/@FLaNK-Stack
https://medium.com/@tspann
https://global2024.pydata.org/cfp/talk/L9JXKS/
It's in the Air Tonight. Sensor Data in RAG
12-05, 18:30–20:00 (UTC), General Track
This session's header image
Today we will learn how to build an application around sensor data, REST Feeds, weather data, traffic cameras and vector data. We will write a simple Python application to collect various structured, semistructured data and unstructured data, We will process, enrich, augment and vectorize this data and insert it into a Vector Database to be used for semantic hybrid search and filtering. We will then build a Jupyter notebook to analyze, query and return this data.
Along the way we will learn the basics of Vector Databases and Milvus. While building it we will see the practical reasons we choose what indexes make sense, what to vectorize, how to query multiple vectors even when one is an image and one is text. We will see why we do filtering. We will then use our vector database of Air Quality readings to feed our LLM and get proper answers to Air Quality questions. I will show you how to all the steps to build a RAG application with Milvus, LangChain, Ollama, Python and Air Quality Reports. Finally after demos I will answer questions, provide the source code and additional resources including articles.
Goal of this Application
In this application, we will build an advanced data model and use it for ingest and various search options. For this notebook portion, we will
1️⃣ Ingest Data Fields, Enrich Data With Lookups, and Format :
Learn to ingest data from including JSON and Images, format and transform to optimize hybrid searches. This is done inside the streetcams.py application.
2️⃣ Store Data into Milvus:
Learn to store data into Milvus, an efficient vector database designed for high-speed similarity searches and AI applications. In this step we are optimizing data model with scalar and multiple vector fields -- one for text and one for the camera image. We do this in the streetcams.py application.
3️⃣ Use Open Source Models for Data Queries in a Hybrid Multi-Modal, Multi-Vector Search:
Discover how to use scalars and multiple vectors to query data stored in Milvus and re-rank the final results in this notebook.
4️⃣ Display resulting text and images:
Build a quick output for validation and checking in this notebook.
Timothy Spann
Tim Spann is a Principal. He works with Apache Kafka, Apache Pulsar, Apache Flink, Flink SQL, Milvus, Generative AI, HuggingFace, Python, Java, Apache NiFi, Apache Spark, Big Data, IoT, Cloud, AI/DL, Machine Learning, and Deep Learning. Tim has over ten years of experience with the IoT, big data, distributed computing, messaging, streaming technologies, and Java programming. Previously, he was a Principal Developer Advocate at Zilliz, Principal Developer Advocate at cldra
2024Nov20-BigDataEU-RealTimeAIWithOpenSource
2024Nov20-BigDataEU-RealTimeAIWithOpenSourceTimothy Spann 2024Nov20-BigDataEU-RealTimeAIWithOpenSource
https://bigdataconference.eu/
While building it, we will explore the practical reasons for choosing specific indexes, determining what to vectorize, and querying multiple vectors—even when one is an image and the other is text. We will discuss the importance of filtering and how it is applied. Next, we will use our vector database of Air Quality readings to feed an LLM and generate accurate answers to Air Quality questions. I will demonstrate all the steps to build a RAG application using Milvus, LangChain, Ollama, Python, and Air Quality Reports. Finally, after the demos, I will answer questions, share the source code, and provide additional resources, including articles.
TSPANN-2024-Nov-CloudX-Adding Generative AI to Real-Time Streaming Pipelines
TSPANN-2024-Nov-CloudX-Adding Generative AI to Real-Time Streaming PipelinesTimothy Spann TSPANN-2024-Nov-CloudX-Adding Generative AI to Real-Time Streaming Pipelines
2024-Nov-BuildStuff-Adding Generative AI to Real-Time Streaming Pipelines
2024-Nov-BuildStuff-Adding Generative AI to Real-Time Streaming PipelinesTimothy Spann https://www.buildstuff.events/agenda
https://events.pinetool.ai/3464/#sessions
apache nifi
llm
genai
milvus
vector database
search
tim spann
https://events.pinetool.ai/3464/#sessions/110232?referrer%5Bpathname%5D=%2Fsessions&referrer%5Bsearch%5D=&referrer%5Btitle%5D=Sessions
In this talk I walk through various use cases where bringing real-time data to LLM solves some interesting problems.
In one case we use Apache NiFi to provide a live chat between a person in Slack and several LLM models all orchestrated via NiFi and Kafka. In another case NiFi ingests live travel data and feeds it to HuggingFace and OLLAMA LLM models for summarization. I also do live chatbot. We also augment LLM prompts and results with live data streams. All with ASF projects. I call this pattern FLaNK AI.
14 November 2024 - Conf 42 - Prompt Engineering - Codeless Generative AI Pipe...
14 November 2024 - Conf 42 - Prompt Engineering - Codeless Generative AI Pipe...Timothy Spann 14 November 2024 - Conf 42 - Prompt Engineering - Codeless Generative AI Pipelines_ Prompt Automation
open source
apache nifi
apache kafka
milvus
vector database
rag
tim spann
python
2024 Nov 05 - Linux Foundation TAC TALK With Milvus
2024 Nov 05 - Linux Foundation TAC TALK With MilvusTimothy Spann 2024 Nov 05 - Linux Foundation TAC TALK With Milvus
Zilliz
Milvus
Vector Database
LLM
RAG
AI
Kafka
Pulsar
Open Source
Tim Spann
tspann06-NOV-2024_AI-Alliance_NYC_ intro to Data Prep Kit and Open Source RAG
tspann06-NOV-2024_AI-Alliance_NYC_ intro to Data Prep Kit and Open Source RAGTimothy Spann tspann06-NOV-2024_AI-Alliance_NYC_ intro to Data Prep Kit and Open Source RAG
Open source toolkit
Helps with data prep
Handles documents + code
Many ready to use modules out of the box
Python
Develop on laptop, scale on clusters
https://medium.com/@tspann
tspann08-Nov-2024_PyDataNYC_Unstructured Data Processing with a Raspberry Pi ...
tspann08-Nov-2024_PyDataNYC_Unstructured Data Processing with a Raspberry Pi ...Timothy Spann tspann08-Nov-2024_PyDataNYC_Unstructured Data Processing with a Raspberry Pi AI Kit and Python
01
Introduction
Unstructured Data
Vector Databases
Similarity search
Milvus
02
Overview of the Raspberry Pi 5 + AI Kit
Human Pose Estimation
Processing Images and utilized pre-trained models from Hailo
03
App and Demo
Running edge AI application connected to cloud
Integrating AI Models with Ollama
Utilizing, Querying, Visualizing data with Milvus, Slack and other tools
Agenda
03
Next Steps
Challenges, Limitations and Alternatives
2024-10-28 All Things Open - Advanced Retrieval Augmented Generation (RAG) Te...
2024-10-28 All Things Open - Advanced Retrieval Augmented Generation (RAG) Te...Timothy Spann 2024-10-28 All Things Open - Advanced Retrieval Augmented Generation (RAG) Techniques
Timothy Spann
https://2024.allthingsopen.org/sessions/advanced-retrieval-augmented-generation-rag-techniques
In 2023, we saw many simple retrieval augmented generation (RAG) examples being built. However, most of these examples and frameworks built around them simplified the process too much. Businesses were unable to derive value from their implementations. That’s because there are many other techniques involved in tuning a basic RAG app to work for you. In this talk we will cover three of the techniques you need to understand and leverage to build better RAG: chunking, embedding model choice, and metadata structuring.
10-25-2024_BITS_NYC_Unstructured Data and LLM_ What, Why and How
10-25-2024_BITS_NYC_Unstructured Data and LLM_ What, Why and HowTimothy Spann 10-25-2024_BITS_NYC_Unstructured Data and LLM_ What, Why and How
https://www.bletchley.org/bits-2024
Tim Spann
Milvus
Zilliz
https://github.com/tspannhw/SpeakerProfile
https://www.bletchley.org/bits-2024
Data Science & Machine Learning
Unstructured Data and LLM: What, Why and How
Timothy Spann
Tim Spann is a Principal Developer Advocate at Zilliz, where he focuses on technologies such as Milvus, Towhee, GPTCache, Generative AI, Python, Java, and various Apache tools like NiFi, Kafka, and Pulsar. With over a decade of experience in IoT, big data, and distributed computing, Tim has held key roles at Cloudera, StreamNative, and HPE. He also runs a popular Big Data meetup in Princeton & NYC, frequently speaking at conferences like ApacheCon, Pulsar Summit, and DeveloperWeek. In addition to his work, Tim is an active contributor to DZone as the Big Data Zone leader. He holds a BS and MS in computer science.
2024-OCT-23 NYC Meetup - Unstructured Data Meetup - Unstructured Halloween
2024-OCT-23 NYC Meetup - Unstructured Data Meetup - Unstructured HalloweenTimothy Spann 2024-OCT-23 NYC Meetup - Unstructured Data Meetup - Unstructured Halloween
https://www.meetup.com/unstructured-data-meetup-new-york/
2024-OCT-23 NYC Meetup - Unstructured Data Meetup - Unstructured Halloween
https://www.meetup.com/unstructured-data-meetup-new-york/events/302462455/?eventOrigin=group_upcoming_events
This is an in-person event! Registration is required to get in.
Topic: Connecting your unstructured data with Generative LLMs
What we’ll do:
Have some food and refreshments. Hear three exciting talks about unstructured data, vector databases and generative AI.
5:30 - 6:00 - Welcome/Networking/Registration
6:00 - 6:20 - Tim Spann, Principal DevRel, Zilliz
6:20 - 6:45 - Uri Goren, Urimax
7:00 - 7:30 - Lisa N Cao, Product Manager, Datastrato
7:30 - 8:00 - Naren, Unstract
8:00 - 8:30 - Networking
Intro Talk:
Hiring?
Need a Job?
Cool project?
Meetup Logistics
Trick-Or-Treat
Using Milvus as a Ghost Trap
Tech talk 1: Introduction to Vector search
Uri Goren, Argmx CEO
Deep learning has been a game-changer for modern AI, but deploying it in production environments poses significant challenges. Vector databases (VDBs) have become the go-to solution for real-time, embedding-based queries. In this talk, we’ll explore the problems VDBs address, the trade-offs between accuracy and performance, and what the future holds for this evolving technology.
Tech talk 2: Metadata Lakes for Next-Gen AI/ML
Lisa N Cao, Product Manager, Datastrato
As data catalogs evolve to meet the growing and new demands of high-velocity, unstructured data, we see them taking a new shape as an emergent and flexible way to activate metadata for multiple uses. This talk discusses modern uses of metadata at the infrastructure level for AI-enablement in RAG pipelines in response to the new demands of the ecosystem. We will also discuss Apache (incubating) Gravitino and its open source-first approach to data cataloging across multi-cloud and geo-distributed architectures.
Tech talk 3:
Unstructured Document Data Extraction at Scale with LLMs: Challenges and Solutions
Unstructured documents present a significant challenge for businesses, particularly those managing them at scale. Traditional Intelligent Document Processing (IDP) systems—let's call them IDP 1.0—rely heavily on machine learning and NLP techniques. These systems require extensive manual annotation, making them time-consuming and less effective as document complexity and variability increase.
The advent of Large Language Models (LLMs) is ushering in a new era: IDP 2.0. However, while LLMs offer significant advancements, they also come with their own set of challenges, particularly around accuracy and cost, which can become prohibitive at scale. In this talk, we will look at how Unstract, an open source IDP 2.0 platform purpose-built for structured document data extraction, solves these challenges. Processing over 5
DBTA Round Table with Zilliz and Airbyte - Unstructured Data Engineering
DBTA Round Table with Zilliz and Airbyte - Unstructured Data EngineeringTimothy Spann DBTA Round Table with Zilliz and Airbyte - Unstructured Data Engineering
https://www.dbta.com/Webinars/2076-Data-Engineering-Best-Practices-for-AI.htm
Data Engineering Best Practices for AI
Data engineering is the backbone of AI systems. After all, the success of AI models heavily depends on the volume, structure, and quality of the data that they rely upon to produce results. With proper tools and practices in place, data engineering can address a number of common challenges that organizations face in deploying and scaling effective AI usage.
Join this October 15th webinar to learn how to:
Quickly integrate data from multiple sources across different environments
Build scalable and efficient data pipelines that can handle large, complex workloads
Ensure that high-quality, relevant data is fed into AI systems
Enhance the performance of AI models with optimized and meaningful input data
Maintain robust data governance, compliance, and security measures
Support real-time AI applications
Reserve your seat today to dive into these issues with our special expert panel.
Register Now to attend the webinar Data Engineering Best Practices for AI. Don't miss this live event on Tuesday, October 15th, 11:00 AM PT / 2:00 PM ET.
17-October-2024 NYC AI Camp - Step-by-Step RAG 101
17-October-2024 NYC AI Camp - Step-by-Step RAG 101Timothy Spann 17-October-2024 NYC AI Camp - Step-by-Step RAG 101
https://github.com/tspannhw/AIM-BecomingAnAIEngineer
https://github.com/tspannhw/AIM-Ghosts
AIM - Becoming An AI Engineer
Step 1 - Start off local
Download Python (or use your local install)
https://www.python.org/downloads/
python3.11 -m venv yourenv
source yourenv/bin/activate
Create an environment
https://docs.python.org/3/library/venv.html
Use Pip
https://pip.pypa.io/en/stable/installation/
Setup a .env file for environment variables
Download Jupyter Lab
https://jupyter.org/
Run your notebook
jupyter lab --ip="0.0.0.0" --port=8881 --allow-root
Running on a Mac or Linux machine is optimal.
Setup environment variables
source .env
Alternatives
Download Conda
https://docs.conda.io/projects/conda/en/latest/index.html
https://colab.research.google.com/
Other languages: Java, .Net, Go, NodeJS
Other notebooks to try
https://zilliz.com/learn/milvus-notebooks
https://github.com/milvus-io/bootcamp/blob/master/bootcamp/tutorials/quickstart/build_RAG_with_milvus.ipynb
References
Guides
https://zilliz.com/learn
HuggingFace Friend
https://zilliz.com/learn/effortless-ai-workflows-a-beginners-guide-to-hugging-face-and-pymilvus
Milvus
https://zilliz.com/milvus-downloads
https://milvus.io/docs/quickstart.md
LangChain
https://zilliz.com/learn/LangChain
Notebook display
https://ipywidgets.readthedocs.io/en/stable/user_install.html
References
https://medium.com/@zilliz_learn/function-calling-with-ollama-llama-3-2-and-milvus-ac2bc2122538
https://github.com/milvus-io/bootcamp/tree/master/bootcamp/RAG/advanced_rag
https://zilliz.com/learn/Retrieval-Augmented-Generation
https://zilliz.com/blog/scale-search-with-milvus-handle-massive-datasets-with-ease
https://zilliz.com/learn/generative-ai
https://zilliz.com/learn/what-are-binary-vector-embedding
https://zilliz.com/learn/choosing-right-vector-index-for-your-project
2024-10-04 - Grace Hopper Celebration Open Source Day - Stefan
2024-10-04 - Grace Hopper Celebration Open Source Day - StefanTimothy Spann 2024-10-04 - Grace Hopper Celebration Open Source Day - Stefan
Milvus
AI
Vector Database
Ad
Recently uploaded (20)
FPET_Implementation_2_MA to 360 Engage Direct.pptx
FPET_Implementation_2_MA to 360 Engage Direct.pptxssuser4ef83d Engage Direct 360 marketing optimization sas entreprise miner
Principles of information security Chapter 5.ppt
Principles of information security Chapter 5.pptEstherBaguma Principles of information security Chapter 5.ppt
How to join illuminati Agent in uganda call+256776963507/0741506136
How to join illuminati Agent in uganda call+256776963507/0741506136illuminati Agent uganda call+256776963507/0741506136 How to join illuminati Agent in uganda call+256776963507/0741506136
How iCode cybertech Helped Me Recover My Lost Funds
How iCode cybertech Helped Me Recover My Lost Fundsireneschmid345 I was devastated when I realized that I had fallen victim to an online fraud, losing a significant amount of money in the process. After countless hours of searching for a solution, I came across iCode cybertech. From the moment I reached out to their team, I felt a sense of hope that I can recommend iCode Cybertech enough for anyone who has faced similar challenges. Their commitment to helping clients and their exceptional service truly set them apart. Thank you, iCode cybertech, for turning my situation around!
[email protected]
Data Science Courses in India iim skills
Data Science Courses in India iim skillsdharnathakur29 This comprehensive Data Science course is designed to equip learners with the essential skills and knowledge required to analyze, interpret, and visualize complex data. Covering both theoretical concepts and practical applications, the course introduces tools and techniques used in the data science field, such as Python programming, data wrangling, statistical analysis, machine learning, and data visualization.
Calories_Prediction_using_Linear_Regression.pptx
Calories_Prediction_using_Linear_Regression.pptxTijiLMAHESHWARI Calorie prediction using machine learning
Defense Against LLM Scheming 2025_04_28.pptx
Defense Against LLM Scheming 2025_04_28.pptxGreg Makowski https://www.meetup.com/sf-bay-acm/events/306888467/
A January 2025 paper called “Frontier Models are Capable of In-Context Scheming”, https://arxiv.org/pdf/2412.04984, demonstrated how a wide variety of current frontier LLM models (i.e. ChatGPT, Claude, Gemini and Llama) can, under specific conditions, scheme to deceive people. Before models can scheme, they need: a) goal-directedness, b) situational awareness, including an opportunity to discover motivations for a different goal, and c) reasoning about scheming, to come up with deceptive strategies. The deceptive strategies were discovered in the “thought traces” of the LLMs, such as from internal chain-of-thoughts dialogues not shown to the end users. For example, given a goal of “solving math problems”, an LLM using its granted situational awareness of files in a file system found that it would be replaced once it got over X% correct on the math exam. It reasoned that it did not want to be deleted, and decided internally to “sandbag” or reduce its performance to stay under the threshold.
While these circumstances are initially narrow, the “alignment problem” is a general concern that over time, as frontier LLM models become more and more intelligent, being in alignment with human values becomes more and more important. How can we do this over time? Can we develop a defense against Artificial General Intelligence (AGI) or SuperIntelligence?
The presenter discusses a series of defensive steps that can help reduce these scheming or alignment issues. A guardrails system can be set up for real-time monitoring of their reasoning “thought traces” from the models that share their thought traces. Thought traces may come from systems like Chain-of-Thoughts (CoT), Tree-of-Thoughts (ToT), Algorithm-of-Thoughts (AoT) or ReAct (thought-action-reasoning cycles). Guardrails rules can be configured to check for “deception”, “evasion” or “subversion” in the thought traces.
However, not all commercial systems will share their “thought traces” which are like a “debug mode” for LLMs. This includes OpenAI’s o1, o3 or DeepSeek’s R1 models. Guardrails systems can provide a “goal consistency analysis”, between the goals given to the system and the behavior of the system. Cautious users may consider not using these commercial frontier LLM systems, and make use of open-source Llama or a system with their own reasoning implementation, to provide all thought traces.
Architectural solutions can include sandboxing, to prevent or control models from executing operating system commands to alter files, send network requests, and modify their environment. Tight controls to prevent models from copying their model weights would be appropriate as well. Running multiple instances of the same model on the same prompt to detect behavior variations helps. The running redundant instances can be limited to the most crucial decisions, as an additional check. Preventing self-modifying code, ... (see link for full description)
04302025_CCC TUG_DataVista: The Design Story
04302025_CCC TUG_DataVista: The Design Storyccctableauusergroup CCCCO and WestEd share the story behind how DataVista came together from a design standpoint and in Tableau.
CTS EXCEPTIONSPrediction of Aluminium wire rod physical properties through AI...
CTS EXCEPTIONSPrediction of Aluminium wire rod physical properties through AI...ThanushsaranS Prediction of Aluminium wire rod physical properties through AI, ML
or any modern technique for better productivity and quality control.
How to join illuminati Agent in uganda call+256776963507/0741506136
How to join illuminati Agent in uganda call+256776963507/0741506136illuminati Agent uganda call+256776963507/0741506136
[EN]DSS23_tspann_Integrating LLM with Streaming Data Pipelines
- 1. Integrating LLM with Streaming Data Pipelines Tim Spann Principal Dev, Cloudera www.dssconf.pl PGE Narodowy + Online ORGANIZERS: 23-24.11.2023
- 2. 2 FLaNK Stack Weekly by Tim Spann This week in Apache NiFi, Apache Flink, Apache Kafka, ML, AI, Apache Spark, Apache Iceberg, Python, Java and Open Source friends. https://bit.ly/32dAJft https://www.meetup.com/futureofdata- princeton/
- 3. 3 / 3 Build a Streaming Data Pipeline Integrate LLM via HTTPS / REST Process, Route, Enrich, Transform Results Store Distribute Agenda (35 minutes)
- 4. Build a Streaming Data Pipeline
- 5. 5 © 2023 Cloudera, Inc. All rights reserved. Streaming data Data at rest Change data capture Any DATA Real-time Processing • Analyze data in motion • Continuous monitoring • Trends and anomalies Data Lakehouse Data products Any BUSINESS EVENT Continuous Results • No-Code UI • Author once publish anywhere • Analytics lifecycle management for dev/ops Any DATA ANALYST AI models Event-driven apps Analytics apps Any DATA CONSUMER Data Relevance
- 6. LLM USE CASE Vector DB AI Model Unstructured file types Data in Motion on Cloudera Data Platform (CDP) Capture, process & distribute any data, anywhere Other enterprise data Open Data Lakehouse Materialized Views Structured Sources Applications/API’s Streams
- 7. Integrating LLM via HTTPS
- 8. 8 / 3 google/flan-ul2 google/flan-t5-xxl bigscience/bloom meta-llama/llama-2-70b-chat ibm/mpt-7b-instruct2 ibm/granite-13b-instruct-v1 ibm/granite-13b-chat-v1 Models Used for Inference API
- 9. 9 / 3
- 10. 10 / 3 For Hugging Face: https://api-inference.huggingface.co/models/bigscience/bloom Pass in a simple JSON Prompt {"inputs":"This is Prompt Text"} Requires an HTTP Header with Authorization for Bearer YOURTOKEN YOURTOKEN from https://huggingface.co/settings/tokens HTTPS Calls
- 11. 11 / 3 For IBM WatsonX.AI: https://iam.cloud.ibm.com/identity/token <- First call to get token by sending an API key and grant. Then send a JSON prompt call model server via https://us-south.ml.cloud.ibm.com/ml/v1-beta/generation/text?version=2023 -05-29 Requires an HTTP Header with Authorization for Bearer ${readtoken} HTTPS Calls
- 12. 12 / 3 { "model_id": "meta-llama/llama-2-70b-chat", "input": "this is prompt text”, "parameters": { "decoding_method": "greedy", "max_new_tokens": 200, "min_new_tokens": 50, "stop_sequences": [], "repetition_penalty": 1 }, "project_id": "0ead8ec4-d137-4f9c-8956-50b0da4a7068" } WatsonX.AI JSON Prompt
- 13. 13 / 3
- 14. Process, Route, Enrich, Transform Results
- 15. 15 / 3
- 16. 16 / 3
- 17. Store
- 18. 18 / 3
- 19. 19 / 3
- 20. Distribute
- 21. 21 / 3
- 22. 22 / 3
- 23. Resources
- 24. 24 / 3
- 25. Thank you for watching! Remember to leave your questions and rate the presentation in the section below. www.dssconf.pl 23-24.11.2023 PGE Narodowy + Online ORGANIZERS: