OpenStack in Action 4! Doug hellman - Intersection of OpenStack and python communities

Dec 16, 20131 like1,815 views

Paris, 5th December 2013 : OpenStack in Action 4! organized by eNovance, brings together members of the OpenStack community.

The Intersection of the OpenStack
and Python Communities
Doug Hellmann
DreamHost

OpenStack in Action 4 ! - Paris – 5th December 2013

1.3+ Million Lines
OpenStack in Action 4 ! - Paris – 5th December 2013

Give and Take

OpenStack in Action 4 ! - Paris – 5th December 2013

alembic amqplib anyjson argparse Babel boto cff cliff cliff-tablib confgobj coverage ddt
discover diskimage-builder Django django-bootstrap-form django_compressor django-nose
django_openstack_auth dnspython docutils dogpile.cache eventlet extras falcon feedparser
fxtures fake8 Flask greenlet hacking happybase hgtools hp3parclient httplib2 httpretty
iso8601 Jinja2 jsonpatch jsonpath-rw jsonrpclib jsonschema kazoo keyring kombu lesscpy
lockfle lxml mock mox mox3 msgpack-python MySQL-python netaddr netifaces nose noseexclude nosehtmloutput nosexcover oauth2 ordereddict pam paramiko passlib Paste
PasteDeploy pbr pecan pep8 pip PrettyTable proboscis psutil psycopg2 pyasn1 pycadf
pycrypto pyfakes pyghmi pylint pymongo pyOpenSSL pyparsing pysendfle pysnmp pysqlite
pystache python-ldap python-memcached python-subunit pytz pyudev PyYAML pyzmq qpidpython redis requests Routes rtslib-fb selenium simplejson six sockjs-tornado sphinx
sphinxcontrib-httpdomain sphinxcontrib-pecanwsme SQLAlchemy sqlalchemy-migrate
stevedore suds swift taskfow testrepository testresources testscenarios testtools tripleoimage-elements unittest2 warlock WebOb websockify WebTest wheel wsgiref WSME xattr

120

OpenStack in Action 4 ! - Paris – 5th December 2013

~5
OpenStack in Action 4 ! - Paris – 5th December 2013

Oslo Mission
To produce a set of python libraries containing code
shared by OpenStack projects. The APIs provided by
these libraries should be high quality, stable,
consistent, documented and generally applicable.

OpenStack in Action 4 ! - Paris – 5th December 2013

How general
is this code?
OpenStack in Action 4 ! - Paris – 5th December 2013

Does it require
incubation?
OpenStack in Action 4 ! - Paris – 5th December 2013

Should it exist at all?
OpenStack in Action 4 ! - Paris – 5th December 2013

Contribute Upstream
OpenStack in Action 4 ! - Paris – 5th December 2013

Python 3
OpenStack in Action 4 ! - Paris – 5th December 2013

stackforge
OpenStack in Action 4 ! - Paris – 5th December 2013

WSME & Pecan
OpenStack in Action 4 ! - Paris – 5th December 2013

Beyond Code
OpenStack in Action 4 ! - Paris – 5th December 2013

planet.openstack.org
planet.python.org
OpenStack in Action 4 ! - Paris – 5th December 2013

OpenStack in Action 4 ! - Paris – 5th December 2013

Conferences
PyCon US
EuroPython
PyCon FR
PyCon IE

PyCon PL
PyCon AU
Kiwi PyCon
PyCon India

OpenStack in Action 4 ! - Paris – 5th December 2013

Distributed Applications
OpenStack in Action 4 ! - Paris – 5th December 2013

Eventual Consistency
OpenStack in Action 4 ! - Paris – 5th December 2013

Concurrency
OpenStack in Action 4 ! - Paris – 5th December 2013

Managing Large
Open Source Projects
OpenStack in Action 4 ! - Paris – 5th December 2013

Python Software
Foundation
OpenStack in Action 4 ! - Paris – 5th December 2013

Feedback
Twitter: @doughellmann
Freenode: dhellmann
doug.hellmann@dreamhost.com

OpenStack in Action 4 ! - Paris – 5th December 2013

Vincent Untz discusses running multiple hypervisors in an OpenStack cloud. OpenStack supports various hypervisors like KVM, Xen, Hyper-V, and VMware through different drivers. Running multiple hypervisors allows using each hypervisor's strengths and migrating workloads. It presents challenges for integration with OpenStack components and feature parity. SUSE Cloud provides tools to deploy and manage multiple hypervisors in a single OpenStack cloud.

OpenStack in Action! 5 - Dell - OpenStack powered solutions - Patrick HamoneNovance

This document discusses Dell/Intel OpenStack-powered solutions and provides the following key points: 1) OpenStack is an open-source cloud operating system that is growing rapidly in adoption with over 10,000 individual members and contributors from over 70 countries. 2) Dell offers OpenStack reference architectures, hardware, software, services, and support to help customers accelerate their adoption of private and hybrid cloud solutions based on OpenStack. 3) Case studies show how Dell OpenStack solutions have helped customers like a research university and web hosting provider build scalable, cost-effective private clouds to meet their infrastructure and data storage needs.

Ceph de facto storage backend for OpenStack eNovance

The document summarizes a presentation about Ceph storage and its integration with OpenStack. It discusses: - Ceph is an open source distributed storage system that is self-managing, self-healing, and scales easily. It uses a pseudo-random placement algorithm called CRUSH to distribute data. - Ceph has become the de facto storage backend for OpenStack. The presentation discusses the current status of Ceph integration in OpenStack Havana and upcoming improvements planned for Icehouse. - Future releases like Firefly will add new features to Ceph like tiering, erasure coding, ZFS support, and multi-backend filestore capabilities.

Open stack in action enovance-quantum in actioneNovance

This document summarizes a presentation about Quantum, the OpenStack networking project. It introduces the speakers and their backgrounds working on OpenStack. It describes Quantum's role in providing networking for IaaS components like Nova, Swift, Cinder, and its transformation of clouds into SDN with the right plugins. Common Quantum plugins are presented along with their supported networking features. The document outlines limitations in the Folsom release and expected improvements in Grizzly, and notes early production uses of Quantum. It concludes by discussing the future of Quantum and an upcoming OpenStack workshop.

OpenStack in Action 4! Thierry Carrez - From Havana to IcehouseeNovance

This document provides an overview of OpenStack development from the Havana release to the upcoming Icehouse release from the perspective of the OpenStack Technical Committee chair. It discusses key accomplishments in the Havana cycle, governance changes, the introduction of programs, infrastructure improvements, new integrated projects like Trove, planned features for Icehouse, and new projects entering incubation like Ironic, Marconi, and Savanna.

OpenStack in Action 4! Alan Clark - The fundation for openstack CloudeNovance

The document discusses OpenStack, an open source cloud computing platform. It provides OpenStack's vision to create an ubiquitous open source cloud computing platform that can meet the needs of public and private clouds regardless of size. It outlines some of OpenStack's key features like compute, storage, networking, and dashboard. It also mentions OpenStack's proven release record, worldwide deployments, and how to get involved with the OpenStack community.

Containers and CloudStackShapeBlue

OpenStack in Action 4! Sebastien Han - Ceph: de facto storage backend for Ope...eNovance

Openstack Summit HK - Ceph defacto - eNovanceeNovance

Sébastien Han presented on Ceph and its integration with OpenStack. Ceph is an open source distributed storage system that is well-suited for OpenStack deployments due to its self-managing capabilities and ability to scale storage resources easily. The integration between Ceph and OpenStack has improved significantly in recent OpenStack releases like Havana, with features like Cinder backup to Ceph and the ability to boot Nova instances using RBD images. Further integration work is planned for upcoming releases to fully leverage Ceph's capabilities.

Become An OpenStack TripleO ATC - Easy As ABCK Rain Leander

The goal of today’s talk will be about becoming an RDO / OpenStack / TripleO active technical contributor, but what does that mean? It means that you are a developer or designer or support engineer or tester or project manager or writer or whatnot and you’d like to become involved in an open source project. It means that you want contribute to TripleO, which is a project within the umbrella space called OpenStack, and that you can join the RDO community for guidance, feedback, and support.

kollaAndrew Widddersheim

Kolla allows running OpenStack in containers using Docker and Ansible for simplified and repeatable deployments. It builds container images for OpenStack components that can be customized and then deployed through Ansible playbooks. Key features include opinionated out-of-the-box configurations, customizability, and integration with tools like Docker, Kubernetes, and ELK for logging. However, caution is advised as Docker, Kolla and Kubernetes are new technologies with active development.

High Availability from the DevOps side - OpenStack Summit PortlandeNovance

This document summarizes Emilien Macchi and Sébastien Han's work on improving high availability in OpenStack. It discusses their contributions of Pacemaker resource agents and documentation updates. It also describes their experiences implementing OpenStack in a medium public cloud, noting challenges of scalability and split brains risks. Lastly, it outlines work to improve networking high availability and testing the cell architecture for horizontal scaling.

OpenStack in an Ever Expanding World of Possibilities - Vancouver 2015 SummitLew Tucker

Over the past several years we have seen the continued adoption of OpenStack and it’s expansion into new areas: from cloud service providers, enterprise private clouds to large media companies, telecommunication giants, and big science. At the same time, open source based platforms for network functions virtualization (NFV) are fueling a movement toward cloud computing in almost all major telco’s. In the developer world, open source projects, such as Docker, Mesos, Kubernetes, and Spark are gaining a lot of attention and being integrated into OpenStack through projects Kolla and Magnum. This session will cover how these projects and activities relate to each other and further expand the utility and adoption OpenStack.

Build a Basic Cloud Using RDO-managerK Rain Leander

Docker Meetup Bangalore - Docker + OpenstackAshish Billore

OpenStack Kolla IntroductionDaneyon Hansen

Kolla - containerizing the cloud itselfMichal Rostecki

Kolla is a project that uses Docker containers to deploy OpenStack cloud software and services. It addresses issues with separating and upgrading OpenStack components by providing Docker images for common services like Nova, Glance, Cinder and more. Kolla utilizes technologies like Docker, Ansible and Jinja2 templates to generate configuration files and deploy containerized OpenStack. It aims to standardize OpenStack deployments and simplify upgrading components.

Containers and OpenStack: Marc Van Hoof, Kumulus: Containers and OpenStackOpenStack

Containers and OpenStack Audience: Intermediate Topic: Infrastructure Abstract: Containers are the new darling of the development world, and many are calling for an end of the IaaS world. But there are still key reasons that IaaS is important even as Container based development becomes the desired path for the development community. We will review containers in the context of their growth in popularity, and look at how OpenStack both continues to support and enable Container solutions, and the latest developments in OpenStack as a containerized solution directly. Speaker Bio: Marc Van Hoof, Kumulus Marc van Hoof has been in the technology industry for over 20 years, focused on developing, deploying, and scaling internet applications. He was part of a team that built the first internet data centre in Australia, has worked on some of the largest online real-time events, and advises companies on how to take advantage of the true benefits of migrating to the cloud. OpenStack Australia Day Government - Canberra 2016 https://events.aptira.com/openstack-australia-day-canberra-2016/

Deploying openstack using ansibleopenstackindia

Openstack-Ansible is a Rackspace initiative that provides an automated way to deploy OpenStack using Ansible playbooks and roles. It pulls services from Git repositories and uses LXC containers and Ansible to deploy OpenStack on single to thousands of nodes in a scalable way. The document discusses why OpenStack deployment is difficult, outlines the OSAD architecture, configuration, and usage, and how OpenStack services are deployed and scaled out to additional compute nodes using Openstack-Ansible.

Openstack devops challenges openstackindia

Openstack DevOps Challenges outlines the journey of CloudRX, a fictitious company, to setup a production-grade Openstack cloud using DevOps practices. It discusses challenges faced in implementing continuous integration/delivery pipelines for Openstack and its heterogeneous components, managing configurations, automated testing of environments, packaging applications, and baremetal server management.

Deploying OpenStack Services with Linux Containers - Brisbane OpenStack Meetu...Ken Thompson

PaaS on top of CloudStackbuildacloud

This document discusses building a Platform as a Service (PaaS) on top of the CloudStack cloud computing platform. It outlines several open source PaaS solutions like AppScale and Cloud Foundry that can be used on CloudStack and discusses challenges in areas like auto-scaling, deployment, and network configuration. The document also highlights a demonstration of AppScale running successfully on CloudStack and the potential to run the proprietary Stackato PaaS, which is based on Cloud Foundry, on CloudStack by supporting its required languages and frameworks.

OpenStack: Changing the Face of Service DeliveryMirantis

Keynote by Lew Tucker, VP and CTO of Cloud Computing at Cisco, at OpenStack Silicon Valley 2015. As more companies move to software-driven infrastructures, OpenStack opens up new possibilities for traditional network service providers, media production, and content providers. Micro-services, and carrier-grade service delivery become the new watchwords for those companies looking to disrupt traditional players with virtualized services running on OpenStack.

Ceph & OpenStack talk given @ OpenStack Meetup @ Bangalore, June 2015Deepak Shetty

Mirantis v OpenStack Ansible Dawn of ProductionTeK Charnsilp Chinprasert

This document discusses different options for deploying OpenStack including Packstack, TripleO, Fuel, and OpenStack Ansible. Packstack provides an all-in-one installation but lacks support for high availability and complex configurations. TripleO uses OpenStack native projects like Ironic but has a very high learning curve and is mostly CLI-driven. Fuel provides a wizard-driven deployment but has a non-flexible architecture. OpenStack Ansible is flexible, container-based, and easy to customize but network and OS installation must be done manually with no vendor support. The document recommends OpenStack Ansible for development/proof-of-concept and Fuel or TripleO for production-ready deployments.

OpenShift Overviewroundman

Devstack On DemandBarak Merimovich

Ceph Performance and Optimization - Ceph Day Frankfurt Ceph Community

This document summarizes a presentation about Ceph performance given at CephDays Frankfurt 2014. It discusses the good aspects of Ceph's performance including its deterministic object placement strategy and ability to aggregate IOs at the cluster and OSD levels. It also examines the bad, including issues caused by journaling, having the journal and OSD data on the same disk, filesystem fragmentation over time, lack of parallelized reads, and the impact of scrubbing. The ugly is described as multiple objects mapping to the same physical disks, causing sequential streams to get mixed and the disks to seek frequently. The document concludes with suggestions on how to properly build a Ceph cluster and considerations like hardware, data growth planning, and failure tolerance.

Deep dive into Verdaccio - NodeTLV 2022 - IsraelJuan Picado

carrow - Go bindings to Apache Arrow via C++-APIYoni Davidson

Apache Arrow is a cross-language development platform for in-memory data that specifies a standardized columnar memory format. It provides libraries and messaging for moving data between languages and services without serialization. The presenter discusses their motivation for creating Go bindings for Apache Arrow via C++ to share data between Go and Python programs using the same memory format. They explain several challenges of this approach, such as different memory managers in Go and C++, and solutions like generating wrapper code and handling memory with finalizers.

More Related Content

What's hot (20)

Openstack Summit HK - Ceph defacto - eNovanceeNovance

Become An OpenStack TripleO ATC - Easy As ABCK Rain Leander

kollaAndrew Widddersheim

High Availability from the DevOps side - OpenStack Summit PortlandeNovance

OpenStack in an Ever Expanding World of Possibilities - Vancouver 2015 SummitLew Tucker

Build a Basic Cloud Using RDO-managerK Rain Leander

Docker Meetup Bangalore - Docker + OpenstackAshish Billore

OpenStack Kolla IntroductionDaneyon Hansen

Kolla - containerizing the cloud itselfMichal Rostecki

Containers and OpenStack: Marc Van Hoof, Kumulus: Containers and OpenStackOpenStack

Deploying openstack using ansibleopenstackindia

Openstack devops challenges openstackindia

Deploying OpenStack Services with Linux Containers - Brisbane OpenStack Meetu...Ken Thompson

PaaS on top of CloudStackbuildacloud

OpenStack: Changing the Face of Service DeliveryMirantis

Ceph & OpenStack talk given @ OpenStack Meetup @ Bangalore, June 2015Deepak Shetty

Mirantis v OpenStack Ansible Dawn of ProductionTeK Charnsilp Chinprasert

OpenShift Overviewroundman

Devstack On DemandBarak Merimovich

Ceph Performance and Optimization - Ceph Day Frankfurt Ceph Community

Openstack Summit HK - Ceph defacto - eNovanceeNovance

Become An OpenStack TripleO ATC - Easy As ABCK Rain Leander

kollaAndrew Widddersheim

High Availability from the DevOps side - OpenStack Summit PortlandeNovance

OpenStack in an Ever Expanding World of Possibilities - Vancouver 2015 SummitLew Tucker

Build a Basic Cloud Using RDO-managerK Rain Leander

Docker Meetup Bangalore - Docker + OpenstackAshish Billore

OpenStack Kolla IntroductionDaneyon Hansen

Kolla - containerizing the cloud itselfMichal Rostecki

Containers and OpenStack: Marc Van Hoof, Kumulus: Containers and OpenStackOpenStack

Deploying openstack using ansibleopenstackindia

Openstack devops challenges openstackindia

Deploying OpenStack Services with Linux Containers - Brisbane OpenStack Meetu...Ken Thompson

PaaS on top of CloudStackbuildacloud

OpenStack: Changing the Face of Service DeliveryMirantis

Ceph & OpenStack talk given @ OpenStack Meetup @ Bangalore, June 2015Deepak Shetty

Mirantis v OpenStack Ansible Dawn of ProductionTeK Charnsilp Chinprasert

OpenShift Overviewroundman

Devstack On DemandBarak Merimovich

Ceph Performance and Optimization - Ceph Day Frankfurt Ceph Community

Similar to OpenStack in Action 4! Doug hellman - Intersection of OpenStack and python communities (20)

Deep dive into Verdaccio - NodeTLV 2022 - IsraelJuan Picado

carrow - Go bindings to Apache Arrow via C++-APIYoni Davidson

Package Management via Spack on SJTU π SupercomputerJianwen Wei

OpenStack in Action 4! Rachid Boularas - Pragmatic Storage Solutions for Hava...eNovance

CoreOS @ gluecon 2015ifup

This document discusses CoreOS, an open source operating system for container-based applications. It describes CoreOS as focusing on automatic updates, using containerization and clustering to deploy and manage applications, and providing APIs to manage machines, containers, storage and networking at scale. Key CoreOS technologies mentioned include Linux containers, etcd for distributed configuration, and fleet for cluster management.

Look beyond PHPFabien Potencier

This document discusses looking beyond PHP to improve web development skills. It recommends mastering PHP fundamentals first before exploring other languages and tools. Examples given include learning JavaScript, using tools like Twig or Jinja templates, learning from code in other languages like Python, and understanding documentation standards beyond PHP like PEP and PSR. The overall message is for developers to remain curious, passionate and willing to learn from others outside of PHP.

Learn how to build decentralized and serverless html5 applications with embar...Alessandro Confetti

Codemotion, Rome 13/04/2018 Do you have an idea for a startup and don't want to pay for scaling it up? Forget about bandwidth problems, servers to install and pay for, with the power of IPFS, DAT and the blockchain. In this talk, we will explore how to build an HTML5 DAPP (distributed application) with EmbarkJS, and figure out how to rethink servers, storage, messaging, data and payments in a distributed and decentralised way with the help of Ethereum's smart contracts, IPFS and DAT distributed storage. Discover more at http://blog.zigolab.it

Learn how to build decentralized and serverless html5 applications with Embar...Codemotion

Do you have an idea for a startup and don't want to pay for scaling it up? Forget about bandwidth problems, servers to install and pay for, with the power of IPFS, DAT and the blockchain. In this talk, we will explore how to build an HTML5 DAPP (distributed application) with EmbarkJS, and figure out how to rethink servers, storage, messaging, data and payments in a distributed and decentralised way with the help of Ethereum's smart contracts, IPFS and DAT distributed storage.

Adios hadoop, Hola Spark! T3chfest 2015dhiguero

Practical Chaos EngineeringSIGHUP

Docker for data scienceCalvin Giles

ApacheCon 2021 - Apache NiFi Deep Dive 300Timothy Spann

21-September-2021 - ApacheCon - Tuesday 17:10 UTC Apache NIFi Deep Dive 300 * https://github.com/tspannhw/EverythingApacheNiFi * https://github.com/tspannhw/FLiP-ApacheCon2021 * https://www.datainmotion.dev/2020/06/no-more-spaghetti-flows.html * https://github.com/tspannhw/FLiP-IoT * https://github.com/tspannhw/FLiP-Energy * https://github.com/tspannhw/FLiP-SOLR * https://github.com/tspannhw/FLiP-EdgeAI * https://github.com/tspannhw/FLiP-CloudQueries * https://github.com/tspannhw/FLiP-Jetson * https://www.linkedin.com/pulse/2021-schedule-tim-spann/ Tuesday 17:10 UTC Apache NIFi Deep Dive 300 Timothy Spann For Data Engineers who have flows already in production, I will dive deep into best practices, advanced use cases, performance optimizations, tips, tricks, edge cases, and interesting examples. This is a master class for those looking to learn quickly things I have picked up after years in the field with Apache NiFi in production. This will be interactive and I encourage questions and discussions. You will take away examples and tips in slides, github, and articles. This talk will cover: Load Balancing Parameters and Parameter Contexts Stateless vs Stateful NiFi Reporting Tasks NiFi CLI NiFi REST Interface DevOps Advanced Record Processing Schemas RetryFlowFile Lookup Services RecordPath Expression Language Advanced Error Handling Techniques Tim Spann is a Developer Advocate @ StreamNative where he works with Apache NiFi, Apache Pulsar, Apache Flink, Apache MXNet, TensorFlow, Apache Spark, big data, the IoT, machine learning, and deep learning. Tim has over a decade of experience with the IoT, big data, distributed computing, streaming technologies, and Java programming. Previously, he was a Principal Field Engineer at Cloudera, a senior solutions architect at AirisData and a senior field engineer at Pivotal. He blogs for DZone, where he is the Big Data Zone leader, and runs a popular meetup in Princeton on big data, the IoT, deep learning, streaming, NiFi, the blockchain, and Spark. Tim is a frequent speaker at conferences such as IoT Fusion, Strata, ApacheCon, Data Works Summit Berlin, DataWorks Summit Sydney, and Oracle Code NYC. He holds a BS and MS in computer science.

NBIS ChIP-seq coursePhil Ewels

CNCF Québec Meetup du 16 Novembre 2023Anthony Dahanne

This document discusses buildpacks, which are tools that detect and build container images for applications. Specifically, it covers cloud native buildpacks, which are open source and allow building OCI images. It provides an overview of where buildpacks originated from projects like Heroku and Cloud Foundry. It also describes Paketo buildpacks, which are maintained buildpacks that provide benefits like adding CA certificates automatically and generating security reports. Integrating buildpacks into existing pipelines can be done in various ways like using Kubernetes, GitHub Actions, or platform services that support buildpacks natively.

Integrating Applications: the Reactive WayNicola Ferraro

This document discusses integrating applications in a reactive way. It begins by defining reactive programming and reactive systems, emphasizing asynchronous, non-blocking architectures. It then discusses application integration patterns like those in Apache Camel, which supports over 200 components and many integration patterns. The document ends by discussing how to build larger reactive systems through microservice integration and communication across reactive and non-reactive applications.

OSS EU: Deep Dive into Building Streaming Applications with Apache PulsarTimothy Spann

OSS EU: Deep Dive into Building Streaming Applications with Apache Pulsar In this session I will get you started with real-time cloud native streaming programming with Java, Golang, Python and Apache NiFi. If there’s a preferred language that the attendees pick, we will focus only on that one. I will start off with an introduction to Apache Pulsar and setting up your first easy standalone cluster in docker. We will then go into terms and architecture so you have an idea of what is going on with your events. I will then show you how to produce and consume messages to and from Pulsar topics. As well as using some of the command line and REST interfaces to monitor, manage and do CRUD on things like tenants, namespaces and topics. We will discuss Functions, Sinks, Sources, Pulsar SQL, Flink SQL and Spark SQL interfaces. We also discuss why you may want to add protocols such as MoP (MQTT), AoP (AMQP/RabbitMQ) or KoP (Kafka) to your cluster. We will also look at WebSockets as a producer and consumer. I will demonstrate a simple web page that sends and receives Pulsar messages with basic JavaScript. After this session you will be able to build simple real-time streaming and messaging applications with your chosen language or tool of your choice. apache pulsar

Fast and cost effective geospatial analysis pipeline with AWS lambdaMila Frerichs

Let's break apache spark workshopGrzegorz Gawron

Have you ever been annoyed at a new toy being broken by your 3-year old? I have. But guess what - every time I pick up some new technology I start behaving like a 3-year-old myself. I want to see what's going to happen if I press here or bend it there. Or indeed - how much the thing can be squeezed before it cracks or worse! Call it vandalism if you will, but it can be a very instructive process. By succumbing to this urge you start 'feeling' the different components of the technology. What it can do, what it's limits are. There's a term coined for this sort of technology 'feeling' - mechanical sympathy. Look it up it makes for a great reading. Unlike the 3-year-old's toys that you need to throw away once broken, we're dealing with software here, so with tools like Docker under your belt the crash is, well, soft and easy to fix. So let's inflict some pain on Apache Spark and see how and where it cracks. After attending the workshop you'll be able to: - deploy Apache Spark to run locally on your laptop using Docker - run a simple distributed Python spark app in Jupyter Notebook - see how it's all connected using Spark UIs - follow the cycle: break it, feel it broken, fix it

Christian Strappazzon - Presentazione Python Milano - Codemotion Milano 2017Codemotion

PyParis2018 - Python tooling for continuous deploymentArthur Lutz

How we migrated the build and deploy processes to a continuous delivery model, and the implications of such a change in terms of technology but also team changes and the project management with the client. This talk will focus on the Python tooling that enabled to conduct such a change, but also on the human changes it requires. * changes in infrastucture, in particular, the use of python softare : docker-compose and saltstack * tools for collecting errors as soon as possible : sentry (django based) and raven (its python library on the client-side) * tools for continous integration and review : jenkins with the python tool "jenkins-job-builder", and the python based version control "mercurial" * tools for metrics and supervision: graphite-api (python rewrite of graphite which ships with django), and saltstack for collecting custom business-oriented metrics from python script * integrating the projects with cloud infrastructure, using python-nova, python-openstack and salt-cloud (openstack and AWS) * change management in the team of developpers and the project management with the final users and project managers

Deep dive into Verdaccio - NodeTLV 2022 - IsraelJuan Picado

carrow - Go bindings to Apache Arrow via C++-APIYoni Davidson

Package Management via Spack on SJTU π SupercomputerJianwen Wei

OpenStack in Action 4! Rachid Boularas - Pragmatic Storage Solutions for Hava...eNovance

CoreOS @ gluecon 2015ifup

Look beyond PHPFabien Potencier

Learn how to build decentralized and serverless html5 applications with embar...Alessandro Confetti

Learn how to build decentralized and serverless html5 applications with Embar...Codemotion

Adios hadoop, Hola Spark! T3chfest 2015dhiguero

Practical Chaos EngineeringSIGHUP

Docker for data scienceCalvin Giles

ApacheCon 2021 - Apache NiFi Deep Dive 300Timothy Spann

NBIS ChIP-seq coursePhil Ewels

CNCF Québec Meetup du 16 Novembre 2023Anthony Dahanne

Integrating Applications: the Reactive WayNicola Ferraro

OSS EU: Deep Dive into Building Streaming Applications with Apache PulsarTimothy Spann

Fast and cost effective geospatial analysis pipeline with AWS lambdaMila Frerichs

Let's break apache spark workshopGrzegorz Gawron

Christian Strappazzon - Presentazione Python Milano - Codemotion Milano 2017Codemotion

PyParis2018 - Python tooling for continuous deploymentArthur Lutz

More from eNovance (20)

OpenStack in Action! 5 - OpenStack Fundation - Behind the scenes: How we prod...eNovance

The document discusses the challenges of producing OpenStack, an open-source cloud computing platform. It notes that OpenStack has over 130 code repositories and 1.8 million lines of code, making coordination and leadership difficult. It also receives contributions from various companies and countries with different cultures and no traditional management structure. The document outlines the processes used to manage releases through milestones every six weeks and gate-centric development, with principles of automated testing, code reviews, time-based releases, and automating processes. It notes the large scale of changes, reviews, and tests occurring weekly in producing OpenStack releases.

OpenStack in Action! 5 - Red Hat - Accelerate Your Business in the Open Hybri...eNovance

This document discusses how OpenStack can help accelerate businesses in the open hybrid cloud by addressing the IT innovation gap faced by many organizations. It notes that while business demands on IT are increasing, IT budgets remain flat, limiting innovation dollars. OpenStack, integrated with Red Hat Enterprise Linux and Red Hat's OpenStack platform, provides enterprises with open, hybrid cloud technologies to help IT adopt cloud architectures and close this innovation gap. The document outlines Red Hat's portfolio of solutions that optimize OpenStack for the open hybrid cloud.

OpenStack in Action 4! Serge Frezefond - Database Clusters as a Service in O...eNovance

OpenStack in Action 4! Ryan Lane - Keep your service oriented reflexes with O...eNovance

OpenStack in action 4! Paul Cormier - Evolve to an Open Hybrid Cloud. Unleash...eNovance

OpenStack in Action 4! Franz Meyer - What Use Case does Red Hat Enterprise ...eNovance

Red Hat Enterprise Linux OpenStack Platform and Red Hat Cloud Infrastructure provide cloud solutions for both traditional and cloud-enabled workloads. Red Hat brings support, stability, certification of hardware and software partners, and long lifecycles to OpenStack. Red Hat OpenStack Platform delivers the benefits of OpenStack with the enterprise qualities expected from Red Hat. It provides a path for customers to adopt OpenStack and hybrid cloud while continuing to run traditional applications on their existing infrastructure.

OpenStack in action 4! Alessandro Pilotti - OpenStack, Hyper-V and WindowseNovance

Windows can run as a guest operating system on OpenStack. Cloudbase-Init allows customization of Windows instances at boot time through plugins. Hyper-V can be used as the hypervisor for OpenStack, and the Hyper-V Nova compute driver and Neutron plugin support integration with OpenStack. Templates in Heat allow automated provisioning of multi-server Windows environments. Puppet, Chef, Crowbar, and SUSE Cloud can be used to deploy and manage Hyper-V compute nodes running OpenStack.

OpenStack in Action 4! Emilien Macchi & Sylvain Afchain - What's new in neutr...eNovance

OpenStack in Action 4! Mark McCLain - From Segments to Services a Dive into O...eNovance

OpenStack in Action 4! Julien Niedergang - SUSE Cloud: Openstack Ready for En...eNovance

OpenStack in Action 4! Jean-Louis Lezaun - Re-architecturing the datacenter :...eNovance

OpenStack in Action 4! Heidi Bretz - State of OpenStack, what's new, the tech...eNovance

OpenStack in Action 4! Nick Barcet & Julien Danjou - From ceilometer to telem...eNovance

OpenStack in Action 4! Raphaël Ferreira & Laurent Letourmy - Build your Cloud...eNovance

OpenStack in Action 4! Patrick Hamon - Architectures of reference for OpenSta...eNovance

OpenStack in Action 4! Daniel Pays & Régis Allègre - Cloudwatt Public Cloud: ...eNovance

OpenStack in Action 4! Jérémie Bourdoncle - Keep your service oriented reflex...eNovance

OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale...eNovance

OpenStack in productioneNovance

Open stack meetup june 2013 - what's new in neutroneNovance