Fine-Tuning Large Language Models with Declarative ML Orchestration - Shivay Lamba, Couchbase
0 likes111 views
Presented at All Things Open AI 2025 Presented by Shivay Lamba - Couchbase Title: Fine-Tuning Large Language Models with Declarative ML Orchestration Abstract: Large Language Models used in tools like ChatGPT are everywhere; however, only a few organisations with massive computing resources are capable of training such large models. While eager to fine-tune these models for specific applications, the broader ML community often grapples with significant infrastructure challenges. In the session, the audience will understand how open-source ML tooling like Flyte (a Linux Foundation open-source orchestration platform) can be used to provide a declarative specification for the infrastructure required for a wide array of ML workloads, including the fine-tuning of LLMs, even with limited resources. Thus the attendee will learn how to leverage open-source ML toolings like Flyte's capabilities to streamline their ML workflows, overcome infrastructure constraints, reduce cost and unlock the full potential of LLMs in their specific use case. Thus making it easier for a larger audience to leverage and train LLMs. Find more info about All Things Open: On the web: https://www.allthingsopen.org/ Twitter: https://twitter.com/AllThingsOpen LinkedIn: https://www.linkedin.com/company/all-things-open/ Instagram: https://www.instagram.com/allthingsopen/ Facebook: https://www.facebook.com/AllThingsOpen Mastodon: https://mastodon.social/@allthingsopen Threads: https://www.threads.net/@allthingsopen Bluesky: https://bsky.app/profile/allthingsopen.bsky.social 2025 conference: https://2025.allthingsopen.org/
![Type Safety
Get errors about your execution
graph at compile-time, even before
executing your code
Input: int
Train Model
Input: List[Dict]
Output: Model
Create Dataset
Input: int
Output: DataFrame
Output: Model
Incompatible
Types
Flyte](https://image.slidesharecdn.com/atoai-fine-tuninglargelanguagemodelswithdeclarativemlorchestration-250325140344-31b5f908/85/Fine-Tuning-Large-Language-Models-with-Declarative-ML-Orchestration-Shivay-Lamba-Couchbase-23-320.jpg)
Ad
More Related Content
Similar to Fine-Tuning Large Language Models with Declarative ML Orchestration - Shivay Lamba, Couchbase (20)
More from All Things Open (20)
![How I Learned to Stop Worrying about my Infrastructure and Love [Open]Tofu](https://cdn.slidesharecdn.com/ss_thumbnails/ato-oct-2024-opentofu-241113165914-c664afc4-thumbnail.jpg?width=560&fit=bounds)
Ad
Recently uploaded (20)
Ad
Fine-Tuning Large Language Models with Declarative ML Orchestration - Shivay Lamba, Couchbase
- 1. Fine-tuning Large Language Models with Declarative ML Orchestration Shivay Lamba Developer Relations Engineer, Couchbase @howdevelop
- 2. Outline ● 🤔 Why Fine-tune LLMs? ● 🔧 How to Fine-tune LLMs ● 🔀 Why declarative ML orchestration? ● 🚀 Fine-tuning LLMs with Flyte Flyte
- 5. Prompt Engineering Holding the model weights fixed and updating the input prompt to obtain the desired output. Flyte LLM Input / Prompt Output / Response 🔧
- 6. Flyte LLM Input / Prompt Output / Response Fine-tuning Updating model weights using a specific data distribution to obtain the desired behavior from the model. 🔧
- 7. 👀 data privacy ? ✍ prompt engineering get desired output? 🚀 ship it! tried really hard? 🔧 fine-tuning ML skills? ☁ low-code fine-tuning 💻 “high”-code fine-tuning get desired output? ⏱ wait for R&D to try again… no yes yes yes yes no no yes no no downstream application…
- 8. Flyte How to Fine-tune LLMs
- 9. 👉 Supervised Fine Tuning SFT 📖 Continued Pre-training CPT Flyte 🔄 RL from Human Feedback RLHF Choose a pretrained model Create “pile of tokens” dataset Minimum quality: low Pick an optimization method Mixed precision, ZeRO Result: text completer Choose a pretrained model Create “prompt-response” dataset Minimum quality: high Pick an optimization method Mixed precision, ZeRO, PEFT Result: a prompt responder Choose an SFT model Create “prompt-multiresponse” dataset with human preferences Minimum quality: high Train a reward model (RM) Initialize from SFT Train a policy on the reward model Initialize from SFT Loss based on RM Result: a prompt responder Focus of this talk Types of Fine-tuning
- 10. Flyte Methods to train large models Source: https://aman.ai/primers/ai/grad-accum-checkpoint/ Gradient Accumulation Model Parallelism Source: https://xiandong79.github.io/Intro-Distributed-Deep-Learning
- 11. Flyte Methods to train models faster Optimizers Data Parallelism Source: https://xiandong79.github.io/Intro-Distributed-Deep-Learning Source: https://www.fast.ai/posts/2018-07-02-adam-weight-decay.html Schedulers Source: https://github.com/sgugger/Deep-Learning/blob/master/Cy clical%20LR%20and%20momentums.ipynb
- 12. ZeRO takes ideas from data and model parallelism, sharding model weights across workers in a distributed system and proceeds from forward and backward passes in a layer-wise fashion Flyte Zero Redundancy Optimization Source: https://www.microsoft.com/en-us/research/blog/zero-deepspeed-new-system-optimizations-enable-training-models-with-over-100-billion-parameters/
- 13. Quantization reduces memory requirements but trades off precision. You can fit larger models into your GPUs, but it can lead to training instability. Flyte Intro to 8-bit Quantization
- 14. LoRA is a technique that allows us to fine-tune large models like Stable Diffusion without the need to retrain them entirely, saving both time and computational resources. LoRA stands for Low-Rank Adaptation, a method designed to fine-tune large-scale models in a more efficient manner. The key idea behind LoRA is to update only a small part of the model's weights, specifically targeting those that have the most significant impact on the task at hand. This approach contrasts with traditional fine-tuning methods, where a large portion of the model's weights might be updated, requiring substantial computational power and time. Flyte Low-Rank Adaptation (LoRA) Source: https://sebastianraschka.com/blog/2023/llm-finetuning-lora.html
- 15. Flyte Why Declarative ML Orchestration?
- 16. Flyte Orchestrators coordinate the logical flow of computations needed to get data from its raw state 🧱 into a desired state 🏠
- 17. 📦 The units of computation you need for your workload 🔀 How data flows between those units ✅ What the types and state of your data is at any given point 🐳 What dependencies each unit relies on to do its computation 🌳 What resources each unit has available to it Orchestrators help you reason about: Flyte
- 18. Flyte Flyte is a production-grade orchestrator that unifies data, ML, and analytics stacks.
- 19. Kubernetes Cluster Flyte Cluster Compiled Workflow Config ⚙ 💻 Create Tasks and Workflows Workflow Execution K8s Pod Container 🐳 Package & Register Flyte
- 20. Tasks The smallest unit of work in Flyte. Task inputs outputs 🐳 Containerized Strongly Typed Versioned Flyte
- 21. Workflows Compositions of Tasks to achieve complex computations inputs Task inputs outputs Workflow inputs outputs Task inputs outputs Task inputs outputs outputs Strongly Typed Data Flow is 1st Class Citizen Versioned Flyte
- 22. Projects and Domains Logical groupings of tasks and workflows for built-in multi-tenancy and isolation. Development Staging Production Data ETL Classification Models Forecasting Models Domains Projects Flyte
- 23. Type Safety Get errors about your execution graph at compile-time, even before executing your code Input: int Train Model Input: List[Dict] Output: Model Create Dataset Input: int Output: DataFrame Output: Model Incompatible Types Flyte
- 24. Declarative Infrastructure Declaratively provisions ephemeral cluster, CPU/GPU, and memory resources. inputs inputs outputs inputs outputs outputs Spark Cluster Setup Teardown Ray Cluster Setup Teardown Flyte Backend inputs outputs GPUs Setup Teardown Flyte Workflow
- 25. Declarative Dependencies Specify your package dependencies as code. Flyte
- 26. Abstracted Data Persistence Don’t worry about how data is serialized/deserialized as your execution graph runs inputs inputs outputs outputs Raw Data Raw Data Store inputs outputs Flyte Raw data Raw Data Flyte Access Boundary Metadata Store Metadata Metadata Metadata Workflow pointer pointer pointer
- 27. Flyte Fine-tuning LLMs with Flyte
- 28. Flyte 📖 Wikipedia 🔧 RedPajama-3B ZeRO w/ DeepSpeed 🤗 Publish to HF Hub 📊 Evaluate Model 🔮 Interactive Inference CPT Workflow 🔧 RedPajama-7B 8-bit LoRA 🤗 Publish to HF Hub SFT Workflow Fine-tuning Workflows 📖 Alpaca 📖 8bit quantize 🤗 Publish to HF Hub
- 29. Flyte 📖 Wikipedia 🔧 RedPajama-3B ZeRO w/ DeepSpeed 🤗 Publish to HF Hub 📊 Evaluate Model 🔮 Interactive Inference CPT Workflow Demo with FLyte 📖 8bit quantize 🤗 Publish to HF Hub
- 30. Flyte Website: https://flyte.org/ Flyte Docs: https://docs.flyte.org/en/latest/ LLM Fine-tuning Repo: https://github.com/unionai-oss/llm-fine-tuning LLM Evaluation Notebook: https://go.flyte.org/tmls-2023-llm-eval LLM Inference Notebook: https://go.flyte.org/tmls-2023-llm-inference 8-bit Quantization Notebook: https://go.flyte.org/tmls-2023-8bit-quantization Basic LoRA Notebook: https://go.flyte.org/tmls-2023-lora Flyte Resources
- 31. 👉 If you care about building applications from LLMs, start off with prompt engineering. Only if you can’t get the desired behavior after a lot of effort should you embark on fine-tuning. If you care about research, go ahead and do fine-tuning! 👉 The space of architectures, optimizers, and fine-tuning techniques is exploding, not just in LLMs. These range from data type representations (8-bit quantization), distributed training setups (ZeRO), and parameter efficiency tricks (LoRA). 👉 Flyte provides a orchestration platform that allows you to write Python code to compose workflows for modern ML workloads, allowing you to reason about resource requirements, infrastructure, and data flow in a declarative way. 👉 Flyte integrates well with cutting edge ML tools like transformers, peft, bitsandbytes, deepspeed, pytorch elasticand the entire Python data / ML ecosystem of libraries (pandas, pandera, numpy, etc.) while giving you reliable and reproducible workflows. Flyte Summary
- 32. Thank you for attending Connect with me here: X: @HowDevelop Github: @ShivayLamba