This is a short introduction to the Apache Software Foundation and Apache CloudStack running on VMware vSphere. It will be presented at the vBrownBag sessions of VMworld 2013.
Apache CloudStack (ACS) is an open source cloud computing platform that can manage virtual machines hosted on various hypervisors including VMware vSphere, XenServer, KVM, and Open Virtualization. ACS integrates with vSphere to provide dynamic scaling of CPU and RAM for virtual machines, hot plug additional NICs, CPU/RAM overcommit, distributed resource scheduling, full clone support, VM and volume snapshots, storage migration, and zone-wide block storage. The integration does not currently support features like storage distributed resource scheduling, Cisco 1000v virtual switches, or private VLANs.
This document discusses Nomad, a distributed, highly available, datacenter-aware cluster scheduler developed by HashiCorp. Nomad schedules work (tasks) across available resources (hosts) to optimize utilization. It allows defining jobs through a declarative job specification language and handles scheduling work to available resources. Nomad aims to provide flexibility for different workloads through pluggable drivers, schedulers and fingerprinting while also being operationally simple to use with a single binary, no dependencies, and high availability.
Setup Kubernetes Cluster On AWS Using KOPSSivaprakash
This document provides steps to set up a Kubernetes cluster on AWS using KOPS. It describes installing required tools like kubectl and KOPS, creating an SSH key and Route53 host on AWS, generating a Kubernetes cluster across multiple Availability Zones, and exposing sample applications. Finally, it covers deleting the cluster.
Ceph Performance and Optimization - Ceph Day Frankfurt Ceph Community
This document summarizes a presentation about Ceph performance given at CephDays Frankfurt 2014. It discusses the good aspects of Ceph's performance including its deterministic object placement strategy and ability to aggregate IOs at the cluster and OSD levels. It also examines the bad, including issues caused by journaling, having the journal and OSD data on the same disk, filesystem fragmentation over time, lack of parallelized reads, and the impact of scrubbing. The ugly is described as multiple objects mapping to the same physical disks, causing sequential streams to get mixed and the disks to seek frequently. The document concludes with suggestions on how to properly build a Ceph cluster and considerations like hardware, data growth planning, and failure tolerance.
Openstack Summit HK - Ceph defacto - eNovanceeNovance
Sébastien Han presented on Ceph and its integration with OpenStack. Ceph is an open source distributed storage system that is well-suited for OpenStack deployments due to its self-managing capabilities and ability to scale storage resources easily. The integration between Ceph and OpenStack has improved significantly in recent OpenStack releases like Havana, with features like Cinder backup to Ceph and the ability to boot Nova instances using RBD images. Further integration work is planned for upcoming releases to fully leverage Ceph's capabilities.
Why learn jenkins via nomad_ci (nomad/consul/docker/jenkins) Dave Pitts
This document discusses learning Jenkins and provides context about the author's background. It summarizes a Nomad CI project that uses Nomad, Consul, Docker, and Jenkins. Finally, it mentions providing a live demo of Jenkins jobs that start various services like name, age, and Redis using Docker.
High Availability from the DevOps side - OpenStack Summit PortlandeNovance
This document summarizes Emilien Macchi and Sébastien Han's work on improving high availability in OpenStack. It discusses their contributions of Pacemaker resource agents and documentation updates. It also describes their experiences implementing OpenStack in a medium public cloud, noting challenges of scalability and split brains risks. Lastly, it outlines work to improve networking high availability and testing the cell architecture for horizontal scaling.
This document discusses Nomad and Consul, two products from HashiCorp that help with deploying and discovering services at scale. Nomad is a cluster scheduler that allows specifying jobs to deploy applications across datacenters. It provides advantages like higher resource utilization, decoupling work from resources, and better quality of service through features like bin packing and priorities. Consul is a service discovery and configuration tool that supports querying across datacenters and regions. It uses Raft consensus and gossip protocols to maintain high availability and scalability.
Containerization allows for standardized and isolated application deployment across infrastructure through container runtimes like Docker. Key features that enable containers include namespaces which isolate processes and their views of the system, cgroups which control resource usage, and copy-on-write storage for efficient application packaging. A container orchestration system like Mesos provides scalability, fault tolerance, and unified resource management across clusters. This allows maximizing infrastructure utilization through flexible scheduling of containerized applications and services on shared clusters.
Amazon Cloud Hands-On provides an overview of Amazon Web Services including Elastic Compute Cloud (EC2) for computing resources, Simple Storage Service (S3) for storage, and Simple Queue Service (SQS) for message queuing. EC2 allows scaling computing power up or down within minutes and starts at 10 cents per hour for 1 EC unit equal to 1.7 GHz x86 processor and 1.7 GB RAM. S3 stores objects up to 5GB in size with SOAP and REST interfaces and affordable pricing, while SQS retains messages for 4 days with a maximum size of 8KB and supports SOAP and queries.
Stateful set in kubernetes implementation & usecases Krishna-Kumar
This document summarizes a presentation on StatefulSets in Kubernetes. It discusses why StatefulSets are useful for running stateful applications in containers, the differences between stateful and stateless applications, how volumes are used in StatefulSets, examples of running single-instance and multi-instance stateful applications like Zookeeper, and the current status and future roadmap of StatefulSets in Kubernetes.
The document discusses various strategies for achieving high availability of web applications and databases. It covers evaluating business requirements, DNS configuration, using cloud infrastructure or owning hardware, basic setups with application and database servers, database replication and clustering options, load balancing tools for Linux and cloud environments, auto scaling features, and monitoring. The key strategies presented include replicating databases, load balancing web traffic, auto-scaling cloud resources, and configuring failover between redundant application and database servers.
San Francisco HashiCorp User Group at GitHubJon Benson
This document discusses Nomad and Consul, two products from HashiCorp that help with deploying and discovering services at scale. Nomad is a cluster scheduler that allows specifying jobs to deploy applications across datacenters. It provides advantages like higher resource utilization, decoupling work from resources, and better quality of service. Consul is a service discovery and configuration tool that supports service registration, health checking, and queries at scale across datacenters. The presentation covers the architectures and advantages of both Nomad and Consul for operating large clusters in a multi-region environment.
How can we deploy & connect a database into multiple locations? What advantages and disadvantages will it have? And most importantly, what kind of performance will it provide?
- Just Eat is a leading digital marketplace for takeaway food delivery founded in 2001 operating in 13 markets globally. It has processed up to 2,500 orders per minute at peak times.
- Just Eat migrated to AWS 5 years ago and runs hundreds of EC2 instances at peak dinner times using scheduled scaling, CloudFormation, and other AWS services.
- AWS Lambda was introduced in 2014 and Just Eat started using it for micro tasks like resetting delivery times, publishing SNS messages, and provisioning instance access to reduce infrastructure costs and management compared to running EC2 fleets.
Making Ceph awesome on Kubernetes with Rook - Bassam TabbaraCeph Community
Rook makes running Ceph storage on Kubernetes easy by extending Kubernetes with custom controllers and types. It automates the deployment, configuration, scaling, upgrading and management of Ceph clusters running within Kubernetes. Rook defines the desired state of the storage cluster and uses an operator to reconcile the actual cluster state with the desired state. This allows Ceph to leverage the full power of Kubernetes' services, deployments and APIs for managing stateful applications at scale.
This document discusses Terraform, an open-source tool that allows users to define and provision infrastructure resources in a declarative configuration file. It summarizes that Terraform allows users to build, change, and destroy infrastructure components like compute instances, storage buckets, and networking through declarative configuration files, enabling an infrastructure-as-code approach that is easy to version, track changes for, and integrate with continuous delivery practices.
beSharp a serverless approach to big data on awsClaudio Pontili
Claudio Pontili, a senior cloud solution architect at beSharp, presented on using serverless architectures for big data on AWS. He discussed using Lambda for ETL processes and Glue for managed ETL jobs. He also covered CI/CD for deploying Lambda and Glue code, data warehousing on Aurora Serverless v1, and a fully serverless big data architecture. Some key learnings included using serverless for high availability and scalability with no effort, pausing Aurora Serverless v1 clusters when not in use, and using infrastructure as code to deploy architectures.
AWS to Bare Metal: Motivation, Pitfalls, and ResultsMongoDB
Like many startups, Wish grew up on AWS. As our cluster grew and the price of SSDs fell, we started exploring bare metal. Fast-forward 2 years and we have hundreds of MongoDB instances on bare metal fully integrated with our AWS infrastructure. It wasn't all smooth sailing, but the performance & cost improvements were worth it! Hear the story of how we did it and gain a framework for thinking about how to make the leap from cloud-centric architecture to a hybrid model.
Leveraging Amzon EC2 Container Services for Container OrchestrationNeeraj Shah
This is the slides for a talk I gave on AWS EC2 Container Services at AWS + Docker Meetup held @ LinkedIn Bangalore. The video of my presentation can be found at https://youtu.be/GbMGJbDTj-A
This presentation covers how to use CloudFormation for deploying customized services on AWS. It goes through the background and advantages, as well as some commonly-used functions. Presented by Joseph Maxwell: lead developer at SwiftOtter Studios.
NDBench is a pluggable, cloud-enabled benchmarking tool developed by Netflix to test the performance and honesty of data stores. It can benchmark systems like Cassandra, Elasticsearch, and DynamoDB. NDBench has a REST API and UI, and allows dynamic configuration of load patterns, client drivers, and other parameters. Netflix uses NDBench for integration testing, deployment validation, and to benchmark upgrades like Cassandra version changes and hardware migrations. The presenter discusses NDBench's achievements in benchmarking Netflix's cloud infrastructure and databases. Future roadmap items include performance profiling, automated metrics analysis, and dynamic load generation.
El documento expresa gratitud a Dios por las bendiciones de salud, felicidad y prosperidad; por las lecciones duras que ayudan a conocerse mejor a uno mismo y a los demás; y por las oportunidades para cultivar la paciencia, tolerancia y esperanza. También agradece por los descubrimientos, oportunidades, capacidades desarrolladas, victorias y experiencias maravillosas de la vida. Finalmente, expresa gratitud por la conciencia de que un ser especial vela y protege a pesar de los errores y debilidades.
Association of Independent Kentucky Colleges and Universities (AIKCU) presentation to Kentucky House Postsecondary Budget Review Subcommittee, February 25, 2010.
ESEconf2011 - Schwaber Ken: "Scrum: Necessary but not sufficient for agility"Aberla
The document discusses the importance of agility and empiricism in software development. It defines agility as the ability to rapidly adapt to change and take advantage of opportunities while controlling risk. Empiricism is derived from experience and experimentation. An empirical process frequently inspects and adapts based on actual results rather than predictive planning. Transparency is also important for empiricism, requiring all aspects to be commonly understood. Scrum is presented as a tool to help teams become more agile through its emphasis on empiricism, transparency, and frequent inspection and adaptation of working software increments.
ESEconf2011 - Kaiser Traian: "How to measure productivity in software develop...Aberla
- The document summarizes a presentation given by Traian Kaiser of XING AG on software development productivity and effectiveness.
- It discusses balancing effectiveness, which is doing the right things, and efficiency, which is doing things right. Effectiveness and efficiency cannot be optimized independently and their interaction is complex.
- Metrics can provide feedback but should not replace trust, and certain metrics used incorrectly can be dysfunctional. Productivity metrics need to be used carefully and coupled with effectiveness metrics.
This document discusses Nomad and Consul, two products from HashiCorp that help with deploying and discovering services at scale. Nomad is a cluster scheduler that allows specifying jobs to deploy applications across datacenters. It provides advantages like higher resource utilization, decoupling work from resources, and better quality of service through features like bin packing and priorities. Consul is a service discovery and configuration tool that supports querying across datacenters and regions. It uses Raft consensus and gossip protocols to maintain high availability and scalability.
Containerization allows for standardized and isolated application deployment across infrastructure through container runtimes like Docker. Key features that enable containers include namespaces which isolate processes and their views of the system, cgroups which control resource usage, and copy-on-write storage for efficient application packaging. A container orchestration system like Mesos provides scalability, fault tolerance, and unified resource management across clusters. This allows maximizing infrastructure utilization through flexible scheduling of containerized applications and services on shared clusters.
Amazon Cloud Hands-On provides an overview of Amazon Web Services including Elastic Compute Cloud (EC2) for computing resources, Simple Storage Service (S3) for storage, and Simple Queue Service (SQS) for message queuing. EC2 allows scaling computing power up or down within minutes and starts at 10 cents per hour for 1 EC unit equal to 1.7 GHz x86 processor and 1.7 GB RAM. S3 stores objects up to 5GB in size with SOAP and REST interfaces and affordable pricing, while SQS retains messages for 4 days with a maximum size of 8KB and supports SOAP and queries.
Stateful set in kubernetes implementation & usecases Krishna-Kumar
This document summarizes a presentation on StatefulSets in Kubernetes. It discusses why StatefulSets are useful for running stateful applications in containers, the differences between stateful and stateless applications, how volumes are used in StatefulSets, examples of running single-instance and multi-instance stateful applications like Zookeeper, and the current status and future roadmap of StatefulSets in Kubernetes.
The document discusses various strategies for achieving high availability of web applications and databases. It covers evaluating business requirements, DNS configuration, using cloud infrastructure or owning hardware, basic setups with application and database servers, database replication and clustering options, load balancing tools for Linux and cloud environments, auto scaling features, and monitoring. The key strategies presented include replicating databases, load balancing web traffic, auto-scaling cloud resources, and configuring failover between redundant application and database servers.
San Francisco HashiCorp User Group at GitHubJon Benson
This document discusses Nomad and Consul, two products from HashiCorp that help with deploying and discovering services at scale. Nomad is a cluster scheduler that allows specifying jobs to deploy applications across datacenters. It provides advantages like higher resource utilization, decoupling work from resources, and better quality of service. Consul is a service discovery and configuration tool that supports service registration, health checking, and queries at scale across datacenters. The presentation covers the architectures and advantages of both Nomad and Consul for operating large clusters in a multi-region environment.
How can we deploy & connect a database into multiple locations? What advantages and disadvantages will it have? And most importantly, what kind of performance will it provide?
- Just Eat is a leading digital marketplace for takeaway food delivery founded in 2001 operating in 13 markets globally. It has processed up to 2,500 orders per minute at peak times.
- Just Eat migrated to AWS 5 years ago and runs hundreds of EC2 instances at peak dinner times using scheduled scaling, CloudFormation, and other AWS services.
- AWS Lambda was introduced in 2014 and Just Eat started using it for micro tasks like resetting delivery times, publishing SNS messages, and provisioning instance access to reduce infrastructure costs and management compared to running EC2 fleets.
Making Ceph awesome on Kubernetes with Rook - Bassam TabbaraCeph Community
Rook makes running Ceph storage on Kubernetes easy by extending Kubernetes with custom controllers and types. It automates the deployment, configuration, scaling, upgrading and management of Ceph clusters running within Kubernetes. Rook defines the desired state of the storage cluster and uses an operator to reconcile the actual cluster state with the desired state. This allows Ceph to leverage the full power of Kubernetes' services, deployments and APIs for managing stateful applications at scale.
This document discusses Terraform, an open-source tool that allows users to define and provision infrastructure resources in a declarative configuration file. It summarizes that Terraform allows users to build, change, and destroy infrastructure components like compute instances, storage buckets, and networking through declarative configuration files, enabling an infrastructure-as-code approach that is easy to version, track changes for, and integrate with continuous delivery practices.
beSharp a serverless approach to big data on awsClaudio Pontili
Claudio Pontili, a senior cloud solution architect at beSharp, presented on using serverless architectures for big data on AWS. He discussed using Lambda for ETL processes and Glue for managed ETL jobs. He also covered CI/CD for deploying Lambda and Glue code, data warehousing on Aurora Serverless v1, and a fully serverless big data architecture. Some key learnings included using serverless for high availability and scalability with no effort, pausing Aurora Serverless v1 clusters when not in use, and using infrastructure as code to deploy architectures.
AWS to Bare Metal: Motivation, Pitfalls, and ResultsMongoDB
Like many startups, Wish grew up on AWS. As our cluster grew and the price of SSDs fell, we started exploring bare metal. Fast-forward 2 years and we have hundreds of MongoDB instances on bare metal fully integrated with our AWS infrastructure. It wasn't all smooth sailing, but the performance & cost improvements were worth it! Hear the story of how we did it and gain a framework for thinking about how to make the leap from cloud-centric architecture to a hybrid model.
Leveraging Amzon EC2 Container Services for Container OrchestrationNeeraj Shah
This is the slides for a talk I gave on AWS EC2 Container Services at AWS + Docker Meetup held @ LinkedIn Bangalore. The video of my presentation can be found at https://youtu.be/GbMGJbDTj-A
This presentation covers how to use CloudFormation for deploying customized services on AWS. It goes through the background and advantages, as well as some commonly-used functions. Presented by Joseph Maxwell: lead developer at SwiftOtter Studios.
NDBench is a pluggable, cloud-enabled benchmarking tool developed by Netflix to test the performance and honesty of data stores. It can benchmark systems like Cassandra, Elasticsearch, and DynamoDB. NDBench has a REST API and UI, and allows dynamic configuration of load patterns, client drivers, and other parameters. Netflix uses NDBench for integration testing, deployment validation, and to benchmark upgrades like Cassandra version changes and hardware migrations. The presenter discusses NDBench's achievements in benchmarking Netflix's cloud infrastructure and databases. Future roadmap items include performance profiling, automated metrics analysis, and dynamic load generation.
El documento expresa gratitud a Dios por las bendiciones de salud, felicidad y prosperidad; por las lecciones duras que ayudan a conocerse mejor a uno mismo y a los demás; y por las oportunidades para cultivar la paciencia, tolerancia y esperanza. También agradece por los descubrimientos, oportunidades, capacidades desarrolladas, victorias y experiencias maravillosas de la vida. Finalmente, expresa gratitud por la conciencia de que un ser especial vela y protege a pesar de los errores y debilidades.
Association of Independent Kentucky Colleges and Universities (AIKCU) presentation to Kentucky House Postsecondary Budget Review Subcommittee, February 25, 2010.
ESEconf2011 - Schwaber Ken: "Scrum: Necessary but not sufficient for agility"Aberla
The document discusses the importance of agility and empiricism in software development. It defines agility as the ability to rapidly adapt to change and take advantage of opportunities while controlling risk. Empiricism is derived from experience and experimentation. An empirical process frequently inspects and adapts based on actual results rather than predictive planning. Transparency is also important for empiricism, requiring all aspects to be commonly understood. Scrum is presented as a tool to help teams become more agile through its emphasis on empiricism, transparency, and frequent inspection and adaptation of working software increments.
ESEconf2011 - Kaiser Traian: "How to measure productivity in software develop...Aberla
- The document summarizes a presentation given by Traian Kaiser of XING AG on software development productivity and effectiveness.
- It discusses balancing effectiveness, which is doing the right things, and efficiency, which is doing things right. Effectiveness and efficiency cannot be optimized independently and their interaction is complex.
- Metrics can provide feedback but should not replace trust, and certain metrics used incorrectly can be dysfunctional. Productivity metrics need to be used carefully and coupled with effectiveness metrics.
The document outlines the history of ancient Rome from its founding as a monarchy around 500 BC to the fall of the Western Roman Empire in 476 AD. It describes Rome's transition to a republic and its expansion through military conquests and the Punic Wars. The summary also notes the transition from republic to empire under Augustus and the period of peace and stability known as the Pax Romana.
How to Measure the Impact of Employee Soft Skills Training | Webinar 06.02.15BizLibrary
Why do organizations struggle with measuring the impact of their soft skills training? We know that improved communications skills make our colleagues better sales professionals, HR professionals, customer service professionals, etc. But getting to actual proof of this “knowledge” isn’t always easy.
In this session, we will explore a range of soft skills training and development challenges, and we will provide a framework for building a measurement and metrics methodology to capture the business benefits and impact of improved performance in soft skills.
www.bizlibrary.com
The document discusses key trends in mobile banking and benchmarks for innovation in Europe and the rest of the world (ROW). It covers how some banks have made mobile a standalone channel or distribution channel, focusing on account access, payments, loans, and driving broader digital strategies. Specific examples highlighted include La Caixa making mobile a full channel in Europe, Akbank distributing loans via mobile banners, and Wells Fargo using mobile as a standalone channel in the ROW. The document analyzes how mobile banking is evolving from early notification services to a critical part of digital transformation.
CloudStack is an open-source cloud computing platform that provides infrastructure as a service. It supports various hypervisors and storage types, and allows for multi-tenancy and isolation between users/organizations. CloudStack provides tools for provisioning, managing, and monitoring virtual machines and cloud infrastructure resources.
Building complete private cloud architectures that support both traditional application virtualization and emerging distributed cloud systems requires compute and object storage storages. Together Apache CloudStack and Basho's Riak CS provide the infrastructure to support both of workloads. The presentation covers the following topics:
1. Why Private Cloud?
2. Anatomy of a Private Cloud
3. Building a Apache CloudStack Compute Offering
4. Large Object Storage using Riak CS
5. Your Own Private Cloud: The Riak CS Apache CloudStack Integration Roadmap
This presentation was originally delivered to the Silicon Valley Riak group on 27 June 2013.
A study and practice of OpenStack release Kilo HA deployment. The Kilo document has some errors, and it's hardly find a detailed document to describe how to deploy a HA cloud based on Kilo release. Hope this slides can provide some clues.
Clouds are made of on-demand, scalable computing resources that are accessed as a service via the internet. There are different cloud deployment models (public, private, hybrid) and service models (IaaS, PaaS, SaaS). Infrastructure as a service (IaaS) clouds provide fundamental computing resources like storage, networking and virtual machines, while platform as a service (PaaS) clouds provide additional services like databases, messaging queues and development tools. Choosing between IaaS and PaaS involves considering factors like lock-in to the cloud vendor, control over the infrastructure, and application requirements.
This document provides an overview and introduction to VMware Virtual SAN (VSAN). It discusses the VSAN architecture which uses SSDs for caching and HDDs for storage. It also covers how VSAN can be configured through storage policies assigned at the VM level. The document outlines how VSAN provides a software-defined storage solution that is hardware agnostic and can elastically scale storage performance and capacity by adding servers and disks.
Cloud leaders such as Rackspace and Internap are building their next generation cloud using OpenStack and Xen+XenAPI, not everyone uses OpenStack with KVM. Lets take a look at how OpenStack and Xen work together, and look at how you can get more involved.
Red Hat Storage Day New York - What's New in Red Hat Ceph StorageRed_Hat_Storage
Red Hat Ceph Storage 2.0 will focus on usability improvements like the Red Hat Storage Console management tool, improved object storage features such as global clusters and authentication support, and performance enhancements from the new BlueStore backend. Future releases will add capabilities like block volume mirroring, CephFS support, and integration with OpenStack services. Upstream development will continue expanding Ceph's object, block, and file functionality.
Deploying Apache CloudStack from API to UIJoe Brockmeier
For most organizations with a large computing footprint, it's not a matter of if you'll need a private cloud - it's when, and what kind. One of the most mature and widely deployed options is Apache CloudStack, a robust, turnkey cloud that includes everything you need to set up a private, public, or hybrid cloud. We'll cover Apache CloudStack from API to UI, and a little of everything in between.
The document discusses OpenStack high availability (HA), performance tuning, and troubleshooting techniques. It covers HA concepts in OpenStack, including compute and controller node HA. It then discusses performance tuning and analyzing OpenStack logs for troubleshooting. It provides details on HA solutions for various OpenStack components like Nova, Glance, Keystone, Swift, Cinder and Neutron. It also covers techniques for optimizing performance in OpenStack like kernel tuning, huge pages, and KSM. Finally, it lists some common log locations for troubleshooting various OpenStack services.
This document provides an overview of the Xen Cloud Platform (XCP) and its history and features. XCP is an open source virtualization platform based on the Xen hypervisor. It includes the Xen hypervisor, the Xen Cloud Platform distribution of XenServer from Citrix, and the XAPI management interface. XAPI acts as the backbone of XCP management and provides an API for orchestration with other cloud platforms. The document discusses XCP's features like live VM migration and storage, management options through XAPI and graphical tools, integration with cloud orchestration stacks, and new developments including improved support for Linux distributions and live storage migration capabilities.
The ability to create global namespace, high availability, and volume economics enables unique capabilities when Red Hat Storage Server is used on Amazon Web Services (AWS). In this session, you’ll learn about the capabilities and the potential of using AWS ephemeral storage, availability zones, and regions to deliver a truly distributed cloud storage solution that is geographically diverse, cost effective, and highly available.
VMware - Virtual SAN - IT Changes EverythingVMUG IT
Virtual SAN is a hyper-converged storage platform that is built into the ESXi hypervisor. It aggregates locally attached flash and disk drives from each ESXi host in a cluster to provide a shared datastore. Virtual SAN provides dynamic capacity and performance scaling. It utilizes storage policies to provide per-VM storage service levels from the single shared datastore. Virtual SAN simplifies storage management by automating control of storage capacity, performance, and availability based on application needs.
Storage Requirements and Options for Running Spark on KubernetesDataWorks Summit
In a world of serverless computing users tend to be frugal when it comes to expenditure on compute, storage and other resources. Paying for the same when they aren’t in use becomes a significant factor. Offering Spark as service on cloud presents very unique challenges. Running Spark on Kubernetes presents a lot of challenges especially around storage and persistence. Spark workloads have very unique requirements of Storage for intermediate data, long time persistence, Share file system and requirements become very tight when it same need to be offered as a service for enterprise to mange GDPR and other compliance like ISO 27001 and HIPAA certifications.
This talk covers challenges involved in providing Serverless Spark Clusters share the specific issues one can encounter when running large Kubernetes clusters in production especially covering the scenarios related to persistence.
This talk will help people using Kubernetes or docker runtime in production and help them understand various storage options available and which is more suitable for running Spark workloads on Kubernetes and what more can be done
The document discusses Xen Cloud Platform (XCP), an open source virtualization platform based on Citrix's XenServer. It provides an overview of XCP, including what it is, its core components like XAPI and Xen hypervisor, available versions and releases, installation methods, and ways to get involved with the project. The document encourages users to download and use XCP, and asks how readers want to help make XCP more open through initiatives like an open roadmap, bug tracker, and code contributions.
The document provides a technical overview of the CLIMB OpenStack cloud including hardware, software, and configuration details. The key components are IBM servers and storage, xCAT for provisioning, SaltStack for configuration management, OpenStack for cloud services, and IBM Spectrum Scale (formerly GPFS) for parallel file storage. Spectrum Scale is integrated with OpenStack components like Cinder, Glance, and Swift to provide scalable block and object storage.
Revolutionary Storage for Modern Databases, Applications and Infrastrcturesabnees
Sanjay Sabnis presented on next generation storage solutions for modern big data applications. He discussed how NVMe storage provides significantly higher performance than SATA, with speeds over 6x faster for reads and over 40x faster for writes. Pavilion Data offers an all-NVMe rack scale storage array that provides 120GB/s of throughput with DAS-level latency. This solution can meet the performance and scalability demands of big data workloads like MongoDB, Splunk, and containerized applications.
This document discusses storage requirements for running Spark workloads on Kubernetes. It recommends using a distributed file system like HDFS or DBFS for distributed storage and emptyDir or NFS for local temp scratch space. Logs can be stored in emptyDir or pushed to object storage. Features that would improve Spark on Kubernetes include image volumes, flexible PV to PVC mappings, encrypted volumes, and clean deletion for compliance. The document provides an overview of Spark, Kubernetes benefits, and typical Spark deployments.
Better, faster, cheaper infrastructure with apache cloud stack and riak cs reduxJohn Burwell
Software is eating infrastructure. Migrating reliability and
scalability responsibilities up the stack from specialized hardware to software, cloud orchestration platforms such as Apache CloudStack (ACS) and object stores such as Riak CS increase the utilization and density of compute and storage resources by dynamically shifting workloads based on demand. Together, these platform can saturate compute and storage of 1000s of commodity hosts with strong operational visibility and end-user self-service.
This presentation explores cloud design strategies to achieve high availability and reliability using commodity components. It then applies these strategies using Apache CloudStack and Riak CS.
- vSphere 5.0 introduces several new platform enhancements including support for 2TB of host memory, 160 logical CPUs, and 512 VMs per host. ESXi now runs exclusively as the hypervisor.
- Storage features are improved with VMFS-5, which supports volumes over 2TB and faster operations. Storage DRS allows for initial placement and load balancing of VMs across datastores.
- Networking features include support for multiple vMotion NICs for faster migration. The new web client allows remote administration from any browser.
OpenStack is an open source cloud computing platform that can manage large networks of virtual machines and physical servers. It uses a distributed architecture with components like Nova (compute), Swift (object storage), Cinder (block storage), and Quantum (networking). OpenStack has been successful due to its scalability, support for multiple hypervisors including Hyper-V, and compatibility with popular programming languages like Python. While OpenStack is best suited for large public and private clouds, its complex installation and lack of unified deployment tools can present challenges, especially for small to mid-sized clouds.
HCL Nomad Web – Best Practices and Managing Multiuser Environmentspanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-nomad-web-best-practices-and-managing-multiuser-environments/
HCL Nomad Web is heralded as the next generation of the HCL Notes client, offering numerous advantages such as eliminating the need for packaging, distribution, and installation. Nomad Web client upgrades will be installed “automatically” in the background. This significantly reduces the administrative footprint compared to traditional HCL Notes clients. However, troubleshooting issues in Nomad Web present unique challenges compared to the Notes client.
Join Christoph and Marc as they demonstrate how to simplify the troubleshooting process in HCL Nomad Web, ensuring a smoother and more efficient user experience.
In this webinar, we will explore effective strategies for diagnosing and resolving common problems in HCL Nomad Web, including
- Accessing the console
- Locating and interpreting log files
- Accessing the data folder within the browser’s cache (using OPFS)
- Understand the difference between single- and multi-user scenarios
- Utilizing Client Clocking
AI and Data Privacy in 2025: Global TrendsInData Labs
In this infographic, we explore how businesses can implement effective governance frameworks to address AI data privacy. Understanding it is crucial for developing effective strategies that ensure compliance, safeguard customer trust, and leverage AI responsibly. Equip yourself with insights that can drive informed decision-making and position your organization for success in the future of data privacy.
This infographic contains:
-AI and data privacy: Key findings
-Statistics on AI data privacy in the today’s world
-Tips on how to overcome data privacy challenges
-Benefits of AI data security investments.
Keep up-to-date on how AI is reshaping privacy standards and what this entails for both individuals and organizations.
How Can I use the AI Hype in my Business Context?Daniel Lehner
𝙄𝙨 𝘼𝙄 𝙟𝙪𝙨𝙩 𝙝𝙮𝙥𝙚? 𝙊𝙧 𝙞𝙨 𝙞𝙩 𝙩𝙝𝙚 𝙜𝙖𝙢𝙚 𝙘𝙝𝙖𝙣𝙜𝙚𝙧 𝙮𝙤𝙪𝙧 𝙗𝙪𝙨𝙞𝙣𝙚𝙨𝙨 𝙣𝙚𝙚𝙙𝙨?
Everyone’s talking about AI but is anyone really using it to create real value?
Most companies want to leverage AI. Few know 𝗵𝗼𝘄.
✅ What exactly should you ask to find real AI opportunities?
✅ Which AI techniques actually fit your business?
✅ Is your data even ready for AI?
If you’re not sure, you’re not alone. This is a condensed version of the slides I presented at a Linkedin webinar for Tecnovy on 28.04.2025.
Quantum Computing Quick Research Guide by Arthur MorganArthur Morgan
This is a Quick Research Guide (QRG).
QRGs include the following:
- A brief, high-level overview of the QRG topic.
- A milestone timeline for the QRG topic.
- Links to various free online resource materials to provide a deeper dive into the QRG topic.
- Conclusion and a recommendation for at least two books available in the SJPL system on the QRG topic.
QRGs planned for the series:
- Artificial Intelligence QRG
- Quantum Computing QRG
- Big Data Analytics QRG
- Spacecraft Guidance, Navigation & Control QRG (coming 2026)
- UK Home Computing & The Birth of ARM QRG (coming 2027)
Any questions or comments?
- Please contact Arthur Morgan at [email protected].
100% human made.
Dev Dives: Automate and orchestrate your processes with UiPath MaestroUiPathCommunity
This session is designed to equip developers with the skills needed to build mission-critical, end-to-end processes that seamlessly orchestrate agents, people, and robots.
📕 Here's what you can expect:
- Modeling: Build end-to-end processes using BPMN.
- Implementing: Integrate agentic tasks, RPA, APIs, and advanced decisioning into processes.
- Operating: Control process instances with rewind, replay, pause, and stop functions.
- Monitoring: Use dashboards and embedded analytics for real-time insights into process instances.
This webinar is a must-attend for developers looking to enhance their agentic automation skills and orchestrate robust, mission-critical processes.
👨🏫 Speaker:
Andrei Vintila, Principal Product Manager @UiPath
This session streamed live on April 29, 2025, 16:00 CET.
Check out all our upcoming Dev Dives sessions at https://community.uipath.com/dev-dives-automation-developer-2025/.
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep DiveScyllaDB
Want to learn practical tips for designing systems that can scale efficiently without compromising speed?
Join us for a workshop where we’ll address these challenges head-on and explore how to architect low-latency systems using Rust. During this free interactive workshop oriented for developers, engineers, and architects, we’ll cover how Rust’s unique language features and the Tokio async runtime enable high-performance application development.
As you explore key principles of designing low-latency systems with Rust, you will learn how to:
- Create and compile a real-world app with Rust
- Connect the application to ScyllaDB (NoSQL data store)
- Negotiate tradeoffs related to data modeling and querying
- Manage and monitor the database for consistently low latencies
Special Meetup Edition - TDX Bengaluru Meetup #52.pptxshyamraj55
We’re bringing the TDX energy to our community with 2 power-packed sessions:
🛠️ Workshop: MuleSoft for Agentforce
Explore the new version of our hands-on workshop featuring the latest Topic Center and API Catalog updates.
📄 Talk: Power Up Document Processing
Dive into smart automation with MuleSoft IDP, NLP, and Einstein AI for intelligent document workflows.
Procurement Insights Cost To Value Guide.pptxJon Hansen
Procurement Insights integrated Historic Procurement Industry Archives, serves as a powerful complement — not a competitor — to other procurement industry firms. It fills critical gaps in depth, agility, and contextual insight that most traditional analyst and association models overlook.
Learn more about this value- driven proprietary service offering here.
Spark is a powerhouse for large datasets, but when it comes to smaller data workloads, its overhead can sometimes slow things down. What if you could achieve high performance and efficiency without the need for Spark?
At S&P Global Commodity Insights, having a complete view of global energy and commodities markets enables customers to make data-driven decisions with confidence and create long-term, sustainable value. 🌍
Explore delta-rs + CDC and how these open-source innovations power lightweight, high-performance data applications beyond Spark! 🚀
Big Data Analytics Quick Research Guide by Arthur MorganArthur Morgan
This is a Quick Research Guide (QRG).
QRGs include the following:
- A brief, high-level overview of the QRG topic.
- A milestone timeline for the QRG topic.
- Links to various free online resource materials to provide a deeper dive into the QRG topic.
- Conclusion and a recommendation for at least two books available in the SJPL system on the QRG topic.
QRGs planned for the series:
- Artificial Intelligence QRG
- Quantum Computing QRG
- Big Data Analytics QRG
- Spacecraft Guidance, Navigation & Control QRG (coming 2026)
- UK Home Computing & The Birth of ARM QRG (coming 2027)
Any questions or comments?
- Please contact Arthur Morgan at [email protected].
100% human made.
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...Aqusag Technologies
In late April 2025, a significant portion of Europe, particularly Spain, Portugal, and parts of southern France, experienced widespread, rolling power outages that continue to affect millions of residents, businesses, and infrastructure systems.
6. Pod 1
Cluster 1
Host 2
ACS Architecture
Host 1
….
L3 switch
Secondary
Storage
Pod N
Zone 1
FirewallLoad Balancer
Primary
Storage
L2 switch
….
Cluster N
New in ACS 4.2 – Regions
• Regions hold multiple zones
• Geographic Scalability &
Resilience
• Management Server per Region
• Object Storage Region Wide
7. ACS Management Server
Back Up DB
Management Server
MySQL DB
Replication
Load
Balancer
Infrastructure
Resources
Management Server
8. ACS Hypervisor Interaction
ACS Management
Server
XenServer
XAPI
• XenServer 5.6 and
higher
• Incremental
Snapshots
• VHD
• NFS, iSCSI, FC and
Local disk
• Storage over-
provisioning: NFS
vSphere
vCenter
HTTP
• vSphere 4.1 and
5.X
• Full Snapshots
• VMDK
• NFS, iSCSI, FC and
Local disk
• Storage over-
provisioning: NFS,
iSCSI
KVM
Agent
• RHEL 6.0, 6.1 and
Ubuntu 10.04
• Full Snapshots (not
live)
• QCOW2
• NFS, iSCSI and FC
• Storage over-
provisioning: NFS
OVM
Agent
• OVM 2.2
• No Snapshots
• RAW
• NFS and iSCSi
• No storage over-
provisioning
9. ACS / VMware Integration
• Dynamic scaling of CPU &
RAM
• Hot plug additional NICs to
VM
• CPU/RAM overcommit
• HA & DRS
• vSphere VM snapshots &
thin provisioning
• Improved ACS volume
snapshots
• Storage live migration
• Zone-wide block storage
• Not Supported: Storage
DRS – planned for 4.3
• Nicira NVP support (Layer 2
& 3 today)
• DVS support
• Cisco 1000v and ASA
1000v
Compute Storage Networking
ACS Management Server
vCenter
Editor's Notes
#3: Incorporated in 1999Governed solely by members – not companiesNon-profit corporation & public charity statusVolunteer OrganizationExists to “Let Developers Develop”Over 100 Top Level ProjectsSupplies established framework environment for projects to collaborate & growSuccess of the project is up to the projectIndependent legal entity for companies and individuals to donate resources with assurance of protection by the foundation
#6: Source: https://www.ohloh.net/orgs/apache/projects – Apache CloudStack is a very active project with a healthy amount of contributorsand commits
#7: A Host is the basic unit of scale.A Cluster groups compatible hostsAll hosts in a cluster have access to shared (primary) storageA Pod is one or more clusters, usually with a L2 switch. Typically a pod is a rack.Zones contain one or more pods, and have access to secondary storage for templates Firewall and Load balancers separate public and private networksAn availability zone is typically a Data CenterThe core components of a Citrix CloudPlatform implementation are:Hosts – Hosts are servers from at least one of the supported virtualization providers. Citrix CloudPlatform fully supports hosts from multiple providers, but does not convert VM images from one hypervisor type to another. Depending on the hypervisor, a “host” may be a higher level concept. For example, in XenServer a Citrix CloudPlatform “host” is equivalent to a XenServer resource pool and the “host” entry is the pool master.Primary Storage – Primary storage is the hypervisor level storage containing the deployed VM storage. Primary storage options will vary by hypervisor, and depending upon the hypervisor selected, CloudPlatform may impose requirements upon it.Cluster – Host groups are combined into Clusters which contain the primary storage options for the Cluster. Primary storage isn’t shared outside of a Cluster. In the case of CloudPlatform, a Cluster in of itself does not imply modification of any clustering concept within the hypervisor. For example, in XenServer a resource pool is a host to CloudPlatform, and CloudPlatform does not create a super set of Cluster functionality for XenServer. Pod -- Host groups are combined first into Clusters and then into Pods. For many customers, a pod represents a high level physical concept like a server rackNetwork – Network is the logical and physical network associated with service offerings. Multiple concurrent network service offerings and topologies can be supported within CloudPlatformSecondary Storage – Secondary storage is the storage system used for template and ISO management. It also is where snapshot events occur.Zone – A zone is a collection pods to form some level of service availability. While Amazon EC2 defines an availability zone as a data center, CloudPlatform keeps the concept more abstract allowing cloud operators to have multiple availability zones within a given data center.Management Server Farm – The CloudPlatform management server farm is a grouping of CentOS/RHEL CloudPlatform servers forming a web farm, with an underlying MySQL cluster database. The management server farm can manage multiple Zones, and can be virtualized.
#8: Management Server is statelessSingle Management Server per 5000 hostsDeploy using RHEL 5.4 + or CentOS 5.4 + MySQL database as data storeManagement server and database support clustering for redundancy and scale
#9: See hypervisor notes for full detailsXenServerCitrix CloudPlatform includes and entitlement for XenServer Advanced edition. When using XenServer, you will first add the XenServer pool master to CloudPlatform as a host, and CloudPlatform will transparently add all slave hosts to CloudPlatform. Oracle VMLimitations: No snapshot because OVM is using raw format for volumeNo system VM because OVM won’t support Debian guestNeed a helper cluster(xenserver/kvm/vmware)Advantage:Oracle provides lots of templates which have Oracle DB frameworks, applications built in, customer can quickly deploy Oracle serviceCreate templateCreate template from root volume of VMStart system VMAdd a helper cluster(XenServer/KVM/Vmware) before creating any OVM VmThe domain router will automatically be created in helper cluster when creating first OVM instanceNo OVM manager and CloudPlatform mixedOvs-agent will store data in local database on hostSupported OS typeAll Linux/Solaris templates must be from Oracle siteWindows can be installed from ISOOracle Cluster File SystemOracle recommendation solution for using ISCSIUser responsibilitySetup ISCSI device on every hostCreate OCFS2 file system on every deviceCloudPlatform responsibilityConfigure every ocfs2 nodeAdd/Remove node on demandKVMFor KVM, Support is only for RHEL 6 based KVM and Ubuntu 10.04. No other flavors of KVM are supported, including RHEV.vCenter cluster/hostA vCenter cluster is mapped directly to a CloudPlatform cluster under PodA vCenter cluster for CloudPlatform can only belong to one vCenter datacenterWhy?vCenterDatastore used by vCenter cluster is at scope of vCenter datacentervCentervSwitch used by vCenter cluster is at scope of vCenter datacenterSharing vCenter datacenter resource outside of CloudPlatform will be problematicSystem VM bootstrapFirst generation is done by CloudPlatform management serverSecond/beyond generations is done through a running SSVMSSVM (Secondary Storage VM)SSVM for template processingSSVM for VMware volume/snapshot/template operationCommand delegationSystem VM, extension of CloudPlatform management serverResource manager can be running in context of a system VMCommand delegation in CloudPlatform management serverSnapshotsCloudPlatform snapshot is taken at volume basisSnapshot in vCenter is take at VM basisFill the gapTake a VM snapshot, if it is for a detached volume in CloudPlatform, create a worker VMParse VM snapshot meta data, build up disk chain information at volume basisCreate intermediate VM on top of a selected disk chainExport VM (full backup) to secondary storageCleanupsvCentervSwitchvSwitch setup is done through vCenterNIC-bonding is done through vCenterCloudPlatform creates networks (portgroups) dynamicallyCloudPlatform propagates networks across clusterWhy? To support independent VM live migration both in CloudPlatform and vCenterDefault vSwitch portsNot enough, usually needs to extend