Wix meetup Feb 26 2020 - How to build 1000 microservices with kafka and thrive

Feb 26, 20200 likes127 views

This talk is about the Wix ecosystem for event driven architecture on top of Kafka. The best practices, SDKs and tools we have created in order to be able to scale our distributed system to more than 1000 microservices.

Wix meetup Feb 26 2020 - How to build 1000 microservices with kafka and thrive

$message HtmlSitePublished { int64 revision = 1 ; // Revision string site_id = 2; } REST$

MessageHandler<HtmlSitePublished >
handler =
...
REST

DC A
(Kafka Cluster A)
DC B
(Kafka Cluster A)

DC B
(Kafka Cluster A)
DC A
(Kafka Cluster A)
COPIES
IN REAL
TIME

This document discusses dynamically managing Kubernetes. It suggests separating different customers by scaling them differently, using different image versions, and customizing environments specifically for each customer. Kubernetes allows creating pods, replication controllers and services during runtime. A management pod could be used to manage Kubernetes resources through REST calls and long polling watches using the Fabric8 Kubernetes client, which provides a fluent API for interacting with the Kubernetes API. Managing ports was tricky due to Kubernetes not allowing port changes and issues with load balancing and connection failures. Testing was also complicated due to the asynchronous nature of container creation and needing to watch for resource version changes and orphaned containers.

Kubernetes + netflix ossCristiano Altmann

TDC2017 | São Paulo - Trilha Containers How we figured out we had a SRE team ...tdc-globalcode

KubernetesMartin Podval

This document provides an overview of Kubernetes, an open-source system for managing containerized applications across multiple hosts. It discusses how Kubernetes can transform physical resources into logical resources for customers, developers and operations teams. It then outlines the three main steps to using Kubernetes - installation, modeling your app, and running and managing it. Key aspects covered include service discovery, load balancing, and examples of running third party components on Kubernetes.

HLayer / Kubernetes for CI/CDAymen EL Amri

Kubernetes can schedule workloads for CI/CD pipelines by running containerized workloads across Kubernetes clusters. It provides features like self-healing, horizontal pod autoscaling, automated rollouts and rollbacks, service discovery, storage orchestration, and secret management. Kubernetes was born out of Borg at Google and is now a top-level Apache project donated to the CNCF. It uses a controller manager to reconcile the desired state with the actual cluster state. Key Kubernetes objects include deployments, replicasets, pods, services, and secrets. Kubernetes is well-suited for scheduling CI/CD workloads across clusters.

CD, docker and kubernetesMartin Podval

This document discusses CD, Docker, and Kubernetes. It provides information on building pipelines with Docker images, deploying small changes frequently to verify them in production, and using a branching strategy where any change can reach production. Kubernetes is introduced as a tool that manages Docker containers across a cluster, simplifying development, deployment, and operations. It ensures a consensual environment and can treat a cluster as separate machines or a single resource pool. Kubernetes helps with CD through features like pods, replication controllers, services, and rolling updates that replace containers one by one while maintaining service availability.

Docker FundamentalsAnshul Patel

The document provides an overview of Docker fundamentals, including what Docker is, its architecture, common operations, and advantages. It begins by explaining that Docker is a container technology that packages code and dependencies together. Containers run atop the same OS kernel and utilize fewer system resources than virtual machines. The document then describes Docker's client-server architecture and underlying technologies like namespaces and cgroups that enable isolation. It also outlines common Docker commands and explains that Dockerfiles define container configurations.

Developing Serverless Microservice in RustAdityo Pratomo

Deploying WSO2 Middleware on KubernetesWSO2

Aptira presents OpenStack Load Balancing as a Service at Banglore India OSUG ...OpenStack

This document discusses Load Balancer as a Service (LBaaS) for OpenStack. It provides an overview of load balancing and its benefits like optimizing resource use, scalability, and avoiding overload. The document outlines LBaaS features like RESTful APIs, CLI, and Horizon dashboard integration. It describes the LBaaS architecture and its modular structure. Implementation details for the Grizzly and Havana releases are provided, along with future roadmaps like multi-vendor support, auto-scaling, and software load balancers on demand.

Bucketbench: Benchmarking Container Runtime PerformancePhil Estes

A talk presented at the Moby Summit, Los Angeles (a co-located event with the Open Source Summit North America) on Thursday, September 14, 2017. In this talk, an open source tool, bucketbench, was presented as a way to benchmark container runtimes to compare performance impacts of changes in the runtime or changes to the configuration of Docker, runC, or containerd, the three runtimes currently supported in the bucketbench project.

Everything you want to know about IngressJanakiram MSV

Serverless architecture: introduction & first stepsThe Software House

Mariusz Richtscheid: Are you wondering about using serverless computing in your project? In this presentation, I will explain the basics of serverless architecture. I'm using an example of a real-life serverless project done at The Software House called Babelsheet (translation-management solution), and include many practical tips for any beginner starting their adventure with a serverless architecture.

LXD Container HypervisorDanial Behzadi

LXD is a container "hypervisor" and a new user experience for LXC. The daemon exports a REST API both locally and if enabled, over the network. The command line tool is designed to be a very simple, yet very powerful tool to manage all your containers. It can handle connect to multiple container hosts and easily give you an overview of all the containers on your network, let you create some more where you want them and even move them around while they're running.

Kubernetes introPravin Magdum

Containerd internals: building a core container runtimeDocker, Inc.

In this talk, we’ll briefly overview of the OpenWhisk serverless (function-as-a-service) framework that initially used the full Docker container engine as the execution vehicle for invoking user functions via containers. After several performance and stability challenges, this project decided to assess the various layers of the Docker engine (containerd and runC) as potential options for the function invoker. Out of that work came an open source project, bucketbench, that can be used to generate benchmarks of container lifecycle operations (e.g., start, stop, kill, remove, pause, unpause) and compare multithreaded operation throughput and stability of each optional engine. This talk will provide details on the bucketbench project, explain how it has been used to generate performance data for these container runtimes, and shares lessons learned along the way that greatly impact container runtime performance, including bottlenecks in the Linux kernel. In this talk you’ll learn how you can use bucketbench for your own performance tuning or assessment of container runtimes and how you can collaborate on improvements to the bucketbench project.

Migration of an Enterprise UI Microservice System from Cloud Foundry to Kuber...Tony Erwin

Presented at Open Source Summit Japan with Jonathan Schweikhart on June 21, 2018. Abstract: The 40 Node.js microservices making up the IBM Cloud UI historically have been deployed as apps on Cloud Foundry (CF), an open source PaaS. But, recently, this enterprise microservice system has been migrated to run on Kubernetes to take advantage of improved orchestration, higher availability, and better performance. Tony Erwin & Jonathan Schweikhart will discuss their team's journey and provide you with insights into the advantages of Kube over CF. Even more importantly, they will describe approaches to solving new problems that took the place of old ones, such as: 1) adapting PaaS apps to run as containers on Kube, 2) enabling geo load balancing between the different runtimes (to vette Kube before completely turning off CF), 3) integrating tools like Prometheus into existing monitoring systems, and more! Their team's first-hand experiences will help you avoid pitfalls as you prepare your own migrations to Kube! Link to Info on Talk: https://ossalsjp18.sched.com/event/EaYj/migration-of-an-enterprise-ui-microservice-system-from-cloud-foundry-to-kubernetes-tony-erwin-jonathan-schweikhart-ibm?iframe=no NOTE: CF is always evolving and the limitations on private networking and private host names mentioned in the slides are no longer current. If you have access to CF API 2.115.0 or higher (released on June 25, 2018), you can leverage CF's service discovery feature (see https://docs.cloudfoundry.org/devguide/deploy-apps/cf-networking.html#discovery ).

OpenEBS - Containerized Storage for ContainersUmasankar Mukkara

Swarm migrationJanakiram MSV

Windows server and dockergirish goudar

Kubernetes Webinar - Using ConfigMaps & Secrets Janakiram MSV

Many applications require configuration using some combination of configuration files, command line arguments, and environment variables. ConfigMaps in Kubernetes provide mechanisms to inject containers with configuration data while keeping them portable. Secrets decouple sensitive content from the pods using a volume plug-in. This webinar will discuss the use cases and scenarios for using ConfigMaps and Secrets.

Persistent Storage with Containers with Kubernetes & OpenShiftRed Hat Events

Manually configuring mounts for containers to various network storage platforms and services is tedious and time consuming. OpenShift and Kubernetes provides a rich library of volume plugins that allow authors of containerized applications (Pods) to declaratively specify what the storage requirements for the containers are so that OpenShift can dynamically provision and allocate the storage assets for the specified containers. As the author of the Kubernetes Persistent Volume specification, I will provide an overview of how Persistent Volume plugins work in OpenShift, demo block storage and file storage volume plugins and close with the Red Hat storage roadmap. Presented at LinuxCon/ContainerCon by Mark Turansky, Principal Software Engineer, Red Hat Mark Turansky is a Principal Software Engineer at Red Hat and a full-time contributor to the Kubernetes Project. Mark is the author of the Kubernetes Persistent Volume specification and a member of the Red Hat OpenShift Engineering team.

Loadbalancers: The fabric for your micro servicesChiradeep Vittal

The document discusses the importance of load balancers for microservices and container-based architectures. It covers how load balancing needs have evolved from traditional monolithic applications to modern approaches that support dynamic microservices topologies, zero-downtime deployments, client-side load balancing, and advanced patterns like circuit breakers. It also introduces NetScaler CPX as a cloud-native load balancer solution for container environments.

Kuberenetes - From Zero to HeroOri Stoliar

Migrating our micro services from Java to Kotlin (Code.Talks 2018)Björn Wendland

This talk elaborates our awesome journey migrate all our micro services from Java to Kotlin. We will take a closer look at the challenges we faced as a team and the impact on our day to day work in an agile environment and how we overcame technical hurdles integrating with Spring Boot and introducing async workflows with Kotlin coroutines. Finally we discuss at the actual benefits that we gained by moving all our micro services to Kotlin and do a critical review of our journey and an outlook at things to come.

Containers in the CloudJames Darrell Pratt

This document discusses containers in cloud computing. It begins with explaining why containers are useful for complex systems that need to deploy the same payloads across multiple environments like QA, staging and production servers. It then defines containers as isolated environments similar to virtual machines but with less overhead. The document provides a history of container technologies over time like chroot, LXC and Docker. It describes Docker as open source software that builds on LXC and provides a portable format and runtime for deploying applications in containers with optimized layers and dependency management. Finally, it introduces container orchestration tools like Docker Swarm for managing and provisioning multi-container applications across clusters with services like discovery, upgrades and load balancing.

Live Container Migration: OpenStack Summit Barcelona 2016Phil Estes

A talk presented by Phil Estes & Shaun Murakami, IBM Cloud Open Technologies, at the Barcelona OpenStack Summit on October 25, 2016. This talk covers a new feature that will be available in the Docker 1.13 engine for using the CRIU project to checkpoint and restore container processes on Linux. Phil & Shaun present details of this new capability and then demonstrate a proof-of-concept "live migration" of containers across nova compute hosts.

Async Excellence Unlocking Scalability with Kafka - Devoxx GreeceNatan Silnitsky

How do you scale 4,000 microservices while tackling latency, bottlenecks, and fault tolerance? At Wix, Kafka powers our event-driven architecture with practical patterns that enhance scalability and developer velocity. This talk explores four key patterns for asynchronous programming: 1. Integration Events: Reduce latency by pre-fetching instead of synchronous calls. 2. Task Queue: Streamline workflows by offloading non-critical tasks. 3. Task Scheduler: Enable precise, scalable scheduling for delayed or recurring tasks. 4. Iterator: Handle long-running jobs in chunks for resilience and scalability. Learn how to balance benefits and trade-offs, with actionable insights to optimize your own microservices architecture using these proven patterns.

Wix Single-Runtime - Conquering the multi-service challengeNatan Silnitsky

In pursuit of a seamless development experience, Wix has harnessed Nile, a powerful suite of tools that accelerates the creation of platformized microservices. While highly effective, the growth of thousands of microservices has introduced a few challenges: increasing infrastructure expenses, more complex upgrades, and increased inter-service communication. Enter Wix Single-Runtime—a Server Guild Infra initiative designed to combine the Nile experience, the agility of serverless architectures and the cost-effectiveness of monolithic deployments. Our solution leverages Kubernetes daemonsets for highly efficient host deployment, working in harmony with numerous lightweight guest services. We will explore the transformation of our current infrastructure to support multi-tenancy, with a keen focus on SDL (Nile's data layer) and its substantial impact on resource usage. Attendees will walk away with actionable insights on reducing production costs, simplifying infrastructure management, and fostering innovations in a polyglot environment. Join us as we navigate the triumphs and trials of building a robust, cost-effective development ecosystem.

More Related Content

What's hot (20)

Developing Serverless Microservice in RustAdityo Pratomo

Deploying WSO2 Middleware on KubernetesWSO2

Aptira presents OpenStack Load Balancing as a Service at Banglore India OSUG ...OpenStack

Bucketbench: Benchmarking Container Runtime PerformancePhil Estes

Everything you want to know about IngressJanakiram MSV

Serverless architecture: introduction & first stepsThe Software House

LXD Container HypervisorDanial Behzadi

Kubernetes introPravin Magdum

Containerd internals: building a core container runtimeDocker, Inc.

Migration of an Enterprise UI Microservice System from Cloud Foundry to Kuber...Tony Erwin

OpenEBS - Containerized Storage for ContainersUmasankar Mukkara

Swarm migrationJanakiram MSV

Windows server and dockergirish goudar

Kubernetes Webinar - Using ConfigMaps & Secrets Janakiram MSV

Persistent Storage with Containers with Kubernetes & OpenShiftRed Hat Events

Loadbalancers: The fabric for your micro servicesChiradeep Vittal

Kuberenetes - From Zero to HeroOri Stoliar

Migrating our micro services from Java to Kotlin (Code.Talks 2018)Björn Wendland

Containers in the CloudJames Darrell Pratt

Live Container Migration: OpenStack Summit Barcelona 2016Phil Estes

Developing Serverless Microservice in RustAdityo Pratomo

Deploying WSO2 Middleware on KubernetesWSO2

Aptira presents OpenStack Load Balancing as a Service at Banglore India OSUG ...OpenStack

Bucketbench: Benchmarking Container Runtime PerformancePhil Estes

Everything you want to know about IngressJanakiram MSV

Serverless architecture: introduction & first stepsThe Software House

LXD Container HypervisorDanial Behzadi

Kubernetes introPravin Magdum

Containerd internals: building a core container runtimeDocker, Inc.

Migration of an Enterprise UI Microservice System from Cloud Foundry to Kuber...Tony Erwin

OpenEBS - Containerized Storage for ContainersUmasankar Mukkara

Swarm migrationJanakiram MSV

Windows server and dockergirish goudar

Kubernetes Webinar - Using ConfigMaps & Secrets Janakiram MSV

Persistent Storage with Containers with Kubernetes & OpenShiftRed Hat Events

Loadbalancers: The fabric for your micro servicesChiradeep Vittal

Kuberenetes - From Zero to HeroOri Stoliar

Migrating our micro services from Java to Kotlin (Code.Talks 2018)Björn Wendland

Containers in the CloudJames Darrell Pratt

Live Container Migration: OpenStack Summit Barcelona 2016Phil Estes

More from Natan Silnitsky (20)

Async Excellence Unlocking Scalability with Kafka - Devoxx GreeceNatan Silnitsky

Wix Single-Runtime - Conquering the multi-service challengeNatan Silnitsky

WeAreDevs - Supercharge Your Developer Journey with Tiny Atomic HabitsNatan Silnitsky

Discover the transformative power of atomic habits in your journey as a developer. Join us in this captivating talk to unlock the secrets of becoming a remarkable developer through small, achievable changes. In this session, we will delve into the Four Laws of Behavior Change, empowering you to adopt new habits that will propel your coding skills to new heights. Learn effective strategies to enter the coding flow state while minimizing distractions, and master the art of acquiring new tech skills with ease. Don't miss this opportunity to gain practical insights and actionable takeaways that will revolutionize your development process. Embrace the power of tiny atomic habits and unlock your true potential as a great developer.

Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.ILNatan Silnitsky

In software engineering, the right architecture is essential for robust, scalable platforms. Wix has undergone a pivotal shift from event sourcing to a CRUD-based model for its microservices. This talk will chart the course of this pivotal journey. Event sourcing, which records state changes as immutable events, provided robust auditing and "time travel" debugging for Wix Stores' microservices. Despite its benefits, the complexity it introduced in state management slowed development. Wix responded by adopting a simpler, unified CRUD model. This talk will explore the challenges of event sourcing and the advantages of Wix's new "CRUD on steroids" approach, which streamlines API integration and domain event management while preserving data integrity and system resilience. Participants will gain valuable insights into Wix's strategies for ensuring atomicity in database updates and event production, as well as caching, materialization, and performance optimization techniques within a distributed system. Join us to discover how Wix has mastered the art of balancing simplicity and extensibility, and learn how the re-adoption of the modest CRUD has turbocharged their development velocity, resilience, and scalability in a high-growth environment.

Effective Strategies for Wix's Scaling challenges - GeeConNatan Silnitsky

This session unveils the multifaceted horizontal scaling strategies that power Wix's robust infrastructure. From Kafka consumer scaling and dynamic traffic routing for site segments to DynamoDB sharding and MySQL clusters custom routing with ProxySQL, we dissect the mechanisms that ensure scalability and performance at Wix. Attendees will learn about the art of sharding and routing key selection across different systems, and how to apply these strategies to their own infrastructure. We'll share insights into choosing the right scaling strategy for various scenarios, balancing between managed services and custom solutions. Key Takeaways: - Grasp various sharding techniques and routing strategies used at Wix. - Understand key considerations for sharding key and routing rule selection. - Learn when and why to choose specific horizontal scaling strategies. - Gain practical knowledge for applying these strategies to achieve scalability and high availability. Join us to gain a blueprint for scaling your systems horizontally, drawing from Wix's proven practices.

Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...Natan Silnitsky

Discover how Wix transitioned from complex event sourcing and CQRS to streamlined CRUD services, optimizing their vast platform for better scalability, performance, and resiliency. Wix's platform, designed to accommodate diverse business needs, boasts: * 3.5 Billion daily HTTP transactions * 70 Billion Kafka messages per day * Roughly 4000 microservices in production This session will highlight the simplification of Wix's architecture through domain events, resilient Kafka messaging, and advanced techniques like materialization and caching. By standardizing APIs and employing tools like protobuf and gRPC, Wix has enhanced the developer experience, both internally and externally, and fostered an open, integrative platform. Attendees will gain insights into Wix's strategies for microservice coordination, ensuring system resilience and data consistency, as well as query performance optimization through innovative 2-level caching solutions.

Workflow Engines & Event Streaming Brokers - Can they work together? [Current...Natan Silnitsky

The document discusses how workflow engines and event streaming brokers can work together. It describes some gaps in using only event streaming at scale for complex business flows and long-running tasks. It then provides an overview of how a workflow engine like Temporal could be integrated to address these gaps, including examples of implementing workflows and activities. Specific opportunities discussed for using Temporal at Wix include long-running processes, cron jobs, and internal microservice tasks.

DevSum - Lessons Learned from 2000 microservicesNatan Silnitsky

Wix has a huge scale of event driven traffic. More than 70 billion Kafka business events per day. Over the past few years Wix has made a gradual transition to an event-driven architecture for its 2000 microservices. We have made mistakes along the way but have improved and learned a lot about how to make sure our production is still maintainable, performant and resilient. In this talk you will hear about the lessons we learned including: 1. The importance of atomic operations for databases and events 2. avoiding data consistency issues due to out-of-order and duplicate processing 3. Having essential events debugging and quick-fix tools in production and a few more

GeeCon - Lessons Learned from 2000 microservicesNatan Silnitsky

Migrating to Multi Cluster Managed Kafka - ApacheKafkaILNatan Silnitsky

As Wix Kafka usage grew to 2.5B messages per day, >20K topics and >100K leader partitions serving 2000 microservices, we decided to migrate from self-operated single cluster per data-center to a managed cloud service (Like Amazon MSK or Confluent Cloud) with a multi-cluster setup. The classic approach would be to perform this transition when all incoming traffic is removed from the data center. But draining an entire data-center for an undetermined period of time, until all 2000 services complete the switch was too risky for us. This talk is about how we gradually migrated all of our Kafka consumers and producers with 0 downtime while they continued to handle regular traffic. You will learn practical steps you can take to greatly reduce the risks and speed up the migration timeline.

Wix+Confluent Meetup - Lessons Learned from 2000 Event Driven MicroservicesNatan Silnitsky

BuildStuff - Lessons Learned from 2000 Event Driven MicroservicesNatan Silnitsky

Lessons Learned from 2000 Event Driven Microservices - ReversimNatan Silnitsky

Devoxx Ukraine - Kafka based Global Data MeshNatan Silnitsky

As your organization rapidly grows in scale, so do the amount of challenges. Growing scale comes in multiple dimensions - traffic, geographic presence, products portfolio, various technologies, amount of developers, etc. Coming up with an architecture that can handle all of the data flows in a universal, simple way is key. This talk is about Wix's Kafka based global data architecture and platform. How we made it very easy for Wix 2000 microservices to publish and subscribe to data, no matter where they are deployed in the world, or what technological stack they use. All the while offering various SDKs (some of them open-source), tools, and features for adapting to growing scale and insuring high resilience.

Devoxx UK - Migrating to Multi Cluster Managed KafkaNatan Silnitsky

Dev Days Europe - Kafka based Global Data Mesh at WixNatan Silnitsky

Kafka Summit London - Kafka based Global Data Mesh at WixNatan Silnitsky

Migrating to Multi Cluster Managed Kafka - Conf42 - CloudNative Natan Silnitsky

5 Takeaways from Migrating a Library to Scala 3 - Scala LoveNatan Silnitsky

Scala 3 is going to make Scala more easy to write, and especially read. more power features like enums and less misleading keywords like implicit. But first we need to migrate our old Scala 2.12 / 2.13 codebase to Scala 3. This talk tells the story of how I tried to migrate greyhound open-source library to Scala 3 with partial success. You will hear about what works, what doesn't and about a few pitfalls to avoid Migration takeaways include: 1. use migration tools, don't do it manually 2. which popular 3rd party libraries can and can't be used by Scala 3 code and many more

Migrating to Multi Cluster Managed Kafka - DevopStars 2022Natan Silnitsky