Axemblr Provisionr 0.3.x Overview

Jan 28, 20130 likes614 views

Simple Service for Managing Pools of 10s or 100s of Virtual Machines With Provisionr we want to solve the problem of cloud portability by hiding completely the API and only focusing on building a cluster that matches the same set of assumptions on all clouds, assumptions like: running a specific operating system (e.g. Ubuntu LTS), having the same set of pre-installed packages and binaries, sane dns settings (forward & reverse ip resolution - as needed for Hadoop), ntp settings, networking settings, ssh admin access, vpn access etc.

Axemblr Provisionr
0.3.x Overview
Why. Objectives. Demo. Internals.
asavu@axemblr.com

What?

• A service that can create and manage
pools of 10s or 100s of virtual machines
• A pool should match the same set of
assumptions on all clouds (os, dns,
packages, networking, ssh, vpn etc.)

Why?
• To create clusters with 50+ nodes for
Hadoop with pre-installed binaries
• ... and use tools like Cloudera Manager or
Ambari for conﬁguration (leverage API)
• Why? Environment normalisation. Template
caching (auto-ami). Hidden behind an API.
Persistence. Traceability. Easy to restart.

How #3
Project Structure. Building. Testing

Plans for 0.3.x

• Simple REST API
• Pool Templates as simple .yml or .json ﬁles
• Generic support for Puppet scripts
• Spot instances on EC2

Join us!

• Source code on GitHub.com/Axemblr
• IRC: #axemblr on freenode

• provisionr-users@googlegroups.com
• provisionr-dev@googlegroups.com

- Terraform allows infrastructure teams to more efficiently and agilely provision resources at scale across multiple production datacenters and regions. - Key benefits include auto-scaling, self-service provisioning of services like Elasticsearch and Cassandra, and reducing new datacenter provisioning from over 12 months to just 2 months. - Debugging and managing complex Terraform configurations, especially across modules, can currently be challenging due to limitations in Terraform's data handling and interpolation features.

[AWSKRUG&JAWS-UG Meetup #1] 70% Cost Reduction with On-demand resizingAWSKRUG - AWS한국사용자모임

This document discusses how Between, a mobile app for couples, migrated their photo architecture to reduce storage costs by 70%. The old architecture generated and stored multiple thumbnail sizes for each photo, using 6.6 billion thumbnails and 738 TB of storage. The new architecture resizes thumbnails on demand using the fast Skia library, saving the original high resolution photos as smaller WebP files. Migration of 1.1 billion existing photos to the new system took 4 days using Spot instances. This reduced storage usage by 75% to 184 TB and number of S3 objects by 82%, cutting total photo costs by 68%.

AWS Lambda at JUST EATAndrew Brown

- Just Eat is a leading digital marketplace for takeaway food delivery founded in 2001 operating in 13 markets globally. It has processed up to 2,500 orders per minute at peak times. - Just Eat migrated to AWS 5 years ago and runs hundreds of EC2 instances at peak dinner times using scheduled scaling, CloudFormation, and other AWS services. - AWS Lambda was introduced in 2014 and Just Eat started using it for micro tasks like resetting delivery times, publishing SNS messages, and provisioning instance access to reduce infrastructure costs and management compared to running EC2 fleets.

Hashicorp @ JUST EAT - Part 2Andrew Brown

초기 스타트업의 AWS - 김지훈(투어라이브) :: AWS Community Day Online 2020AWSKRUG - AWS한국사용자모임

1. The document discusses AWS services for container orchestration including EKS, Fargate, Docker Swarm, and Mesos. It also discusses using CloudFront and S3 for hosting static assets. 2. Details are provided on migrating from Docker Swarm to EKS on AWS, including using EC2 and Route53. Options for hosting databases like MySQL and PostgreSQL across regions are explored. 3. The benefits of services like EKS, Lambda, and CloudFront are summarized and pricing models for CloudFront are referenced. Moving infrastructure to different regions for latency or redundancy purposes is also briefly discussed.

Extending ansibleYan Kurniawan

This document discusses how to create custom Ansible modules to extend its functionality. It provides examples of Ansible and Puppet built-in modules, and describes how to write a custom module using Python. Key steps include choosing a language, setting up the module file structure and boilerplate code, handling arguments and exceptions, and returning JSON output. The document demonstrates a simple module to look up AWS EC2 instances by tags and return their IDs.

20명 규모의 팀에서 Vault 사용하기Doyoon Kim

From HashiCorp Korea User Group Meetup 발표자: 김민규(데브시스터즈, 인프라 관리, https://github.com/synthdnb) 발표자: 김도윤(데브시스터즈, 플랫폼 API 서버 개발, https://github.com/solmonk) 발표내용: 팀의 규모가 커지면서 Secret 관리 문제가 조금씩 부각되었습니다. 예를 들면 코드에 커밋되거나, 구전으로 전해지는 Secret들, SSH Key Rotation 등의 문제를 처리하기 위해 많은 노력과 삽질이 필요했습니다. 저희 팀에서 Vault를 통해 이런 문제들을 어떻게 해결했는지 소개하려 합니다.

Storage Is Not Virtualized Enough - part 1Zhipeng Huang

This document discusses storage function virtualization (SFV) and why storage is not yet virtualized enough. It provides an overview of SFV and how it can enhance NFV by virtualizing storage functions. It also discusses OpenStack SFV and related blueprints to improve storage scheduling and functionality. Finally, it encourages collaboration on SFV-related work beyond just OpenStack through various communication channels and open source projects.

Let's Get Acquainted: Ansible! / Symfony Camp UA 2014Igor Brovchenko

This document introduces Ansible, an IT automation tool. It discusses what Ansible is, its key features and uses cases like server provisioning, deployment, and configuration management. It also provides information on installing and using Ansible, including requirements, modules, inventory, playbooks and roles. Finally, it discusses real world examples and resources for learning more about Ansible.

Using Aws As A Game Server (AWS UG Bandung)Aswin Juari

The future of nodeSarunyhot Suwannachoti

Node.js is a JavaScript runtime built on Chrome's V8 JavaScript engine that allows JavaScript to run on the server. It is commonly used for building web servers and networking applications. Node.js uses an asynchronous event-driven model that makes it lightweight and efficient for data-intensive real-time applications. The ecosystem of Node.js modules and frameworks like Express, Restify, and Hapi support building many types of applications beyond just web servers, including desktop apps, IoT devices, and command line tools. Node.js development is module-driven using the npm registry, which has over 150,000 packages and sees high growth in usage. The future of Node.js involves continued improvements to asynchronous programming, support

EKS에서 Opentelemetry로 코드실행 모니터링하기 - 신재현 (인덴트코퍼레이션) :: AWS Community Day Online...AWSKRUG - AWS한국사용자모임

AWS Connect 2017 - Container (feat. AWS)smalltown

전 세계 팬들이 모일 수 있는 플랫폼 만들기 - 강진우 (beNX) :: AWS Community Day 2020 AWSKRUG - AWS한국사용자모임

Infrastructure as code with TerraformSam Bashton

The document discusses using Terraform to implement infrastructure as code. It describes how Terraform allows building multiple environments like development, test, staging and production in an automated and repeatable way. It also provides code examples to demonstrate how to build a VPC, security group and EC2 instance using Terraform modules to reuse infrastructure components and simplify configuration.

Automating aws infrastructure and code deployments using Ansible @WebEngageVishal Uderani

In this talk , we’ll cover how and why Ansible was leveraged to automate routine management of EC2 instances/EBS/EIP/ELB etc and why the Ansible approach towards automation is key for code and system deployments across 100’s of nodes and how we achieved this at Webengage. We will provide an overview of the deployment process and give a demonstration as an example Outlines : How ansible is a straightforward , easy way to manage multiple cloud resources Intended Audience : Previous experience with configuration management systems Previous experience with AWS and Ansible

ACS & vSphere DraftAaron Delp

Apache CloudStack (ACS) is an open source cloud computing platform that can manage virtual machines hosted on various hypervisors including VMware vSphere, XenServer, KVM, and Open Virtualization. ACS integrates with vSphere to provide dynamic scaling of CPU and RAM for virtual machines, hot plug additional NICs, CPU/RAM overcommit, distributed resource scheduling, full clone support, VM and volume snapshots, storage migration, and zone-wide block storage. The integration does not currently support features like storage distributed resource scheduling, Cisco 1000v virtual switches, or private VLANs.

Real World AWS Deployment With Boto, Fabric, and Cloud FormationDixon Whitmire

AWS Elastic Container Service - DockerHNNguyen Anh Tu

Amazon EC2 Container Service (ECS) is a container management service that supports running and managing Docker containers on a cluster of Amazon EC2 instances. It provides built-in scheduling, cluster management, and orchestration capabilities to help run and scale containerized applications. ECS integrates with other AWS services like Elastic Load Balancing, Elastic Block Store, Virtual Private Cloud, and more. Users define tasks and services in task definitions that specify containers, resources, dependencies, and other configuration.

Advance discussion on Ansible - Rahul IntiSahil Davawala

This document discusses Ansible and dynamic inventory. It explains that dynamic inventory retrieves host information from scripts instead of static inventory files, reducing human error. Ansible has inventory scripts for AWS EC2, OpenStack, Google Cloud. It also covers Ansible concepts like roles, tasks, variables, modules, templates and Ansible Vault for encrypting sensitive data. Overall it provides an overview of Ansible and demonstrates how to spawn AWS EC2 instances using dynamic inventory scripts.

Day of Cloud: Amazon EC2cmcavoy

Cloudformation vs terraform_vs_ansibleGeert Theys

Ansible, PHP and EC2Craig Marvelley

This document discusses using Ansible to provision and manage infrastructure and applications on AWS. It describes how Ansible playbooks are used to provision Vagrant VMs locally for development as well as provision production systems on AWS. The playbooks define tasks like installing packages and configure servers. They allow consistently provisioning servers and provide documentation. Demo sections show Ansible in action provisioning a PHP application environment on EC2.

Introduction to ansibleKasun Madura Rathnayaka

This document provides an introduction to Ansible, an IT automation tool. It discusses how Ansible uses YAML playbooks to describe the intended system state and requires no agent on managed machines. Playbooks can consolidate tasks and configuration into a single file and support templating. Real-world examples are given of using Ansible for tasks like installing packages, editing config files, and enabling services on remote servers.

Major Managed Kubernetes Platforms Comparison (AWS, GCP, Azure)GlobalLogic Ukraine

This presentation by Andriy Kopachevskyy (Engineering Consultant, GlobalLogic) was delivered at GlobalLogic Kharkiv DevOps TechTalk #1 on October 8, 2019. Andriy talked about basic Kubernetes services available "out of the box", conducted a comparative analysis of the cost of Kubernetes infrastructure. He drew the audience's attention to the ease of deployment, use and support of Kubernetes, talked about its integration with other cloud service providers, as well as monitoring and logging in to Kubernetes. Event materials: https://www.globallogic.com/ua/events/kharkiv-devops-techtalk-1/

COSCUP 2017 - infrastructure As Code smalltown

This document discusses the benefits of implementing infrastructure, pipelines, images, servers, orchestration, and chat operations as code. Key benefits include being reproducible, saving time, avoiding human error, and not becoming outdated. It outlines tools like Terraform, Jenkins, Packer, Kubernetes, Docker, configuration management, and Hubot. Code samples and demos are provided to define infrastructure with Terraform, transform jobs into pipelines with Jenkins, build images with Packer, define containers in Kubernetes, build servers with Docker, and interact with services through Hubot chat. A link is provided to access additional sample code.

The AWS DevOps combo (January 2017)Julien SIMON

The document discusses AWS services for DevOps workflows including infrastructure as code (AWS CloudFormation), container management (Amazon ECS and ECR), continuous integration and delivery (AWS CodeCommit, CodeBuild, CodeDeploy, CodePipeline). It provides an overview of each service and examples of how they can be used together in a continuous deployment pipeline to develop, build, test and deploy applications on AWS.

Application Lifecycle Management on AWSDavid Mat

More Related Content

What's hot (20)

20명 규모의 팀에서 Vault 사용하기Doyoon Kim

Storage Is Not Virtualized Enough - part 1Zhipeng Huang

Let's Get Acquainted: Ansible! / Symfony Camp UA 2014Igor Brovchenko

Using Aws As A Game Server (AWS UG Bandung)Aswin Juari

The future of nodeSarunyhot Suwannachoti

EKS에서 Opentelemetry로 코드실행 모니터링하기 - 신재현 (인덴트코퍼레이션) :: AWS Community Day Online...AWSKRUG - AWS한국사용자모임

AWS Connect 2017 - Container (feat. AWS)smalltown

전 세계 팬들이 모일 수 있는 플랫폼 만들기 - 강진우 (beNX) :: AWS Community Day 2020 AWSKRUG - AWS한국사용자모임

Infrastructure as code with TerraformSam Bashton

Automating aws infrastructure and code deployments using Ansible @WebEngageVishal Uderani

ACS & vSphere DraftAaron Delp

Real World AWS Deployment With Boto, Fabric, and Cloud FormationDixon Whitmire

AWS Elastic Container Service - DockerHNNguyen Anh Tu

Advance discussion on Ansible - Rahul IntiSahil Davawala

Day of Cloud: Amazon EC2cmcavoy

Cloudformation vs terraform_vs_ansibleGeert Theys

Ansible, PHP and EC2Craig Marvelley

Introduction to ansibleKasun Madura Rathnayaka

Major Managed Kubernetes Platforms Comparison (AWS, GCP, Azure)GlobalLogic Ukraine

COSCUP 2017 - infrastructure As Code smalltown

20명 규모의 팀에서 Vault 사용하기Doyoon Kim

Storage Is Not Virtualized Enough - part 1Zhipeng Huang

Let's Get Acquainted: Ansible! / Symfony Camp UA 2014Igor Brovchenko

Using Aws As A Game Server (AWS UG Bandung)Aswin Juari

The future of nodeSarunyhot Suwannachoti

EKS에서 Opentelemetry로 코드실행 모니터링하기 - 신재현 (인덴트코퍼레이션) :: AWS Community Day Online...AWSKRUG - AWS한국사용자모임

AWS Connect 2017 - Container (feat. AWS)smalltown

전 세계 팬들이 모일 수 있는 플랫폼 만들기 - 강진우 (beNX) :: AWS Community Day 2020 AWSKRUG - AWS한국사용자모임

Infrastructure as code with TerraformSam Bashton

Automating aws infrastructure and code deployments using Ansible @WebEngageVishal Uderani

ACS & vSphere DraftAaron Delp

Real World AWS Deployment With Boto, Fabric, and Cloud FormationDixon Whitmire

AWS Elastic Container Service - DockerHNNguyen Anh Tu

Advance discussion on Ansible - Rahul IntiSahil Davawala

Day of Cloud: Amazon EC2cmcavoy

Cloudformation vs terraform_vs_ansibleGeert Theys

Ansible, PHP and EC2Craig Marvelley

Introduction to ansibleKasun Madura Rathnayaka

Major Managed Kubernetes Platforms Comparison (AWS, GCP, Azure)GlobalLogic Ukraine

COSCUP 2017 - infrastructure As Code smalltown

Similar to Axemblr Provisionr 0.3.x Overview (20)

The AWS DevOps combo (January 2017)Julien SIMON

Application Lifecycle Management on AWSDavid Mat

Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...Emerson Eduardo Rodrigues Von Staffen

DevOps, Continuous Integration & Deployment on AWS discusses practices for software development on AWS including DevOps, continuous integration, continuous delivery, and continuous deployment. It provides an overview of AWS services that can be used at different stages of the software development lifecycle such as CodeCommit for source control, CodePipeline for release automation, and CodeDeploy for deployment. National Novel Writing Month (NaNoWriMo) maintains its websites and services on AWS to support its annual writing challenge. It migrated to AWS to improve uptime and scalability. Its future goals include porting older sites to Rails, using Amazon SES for email, load balancing with ELB, implementing auto scaling, and using services like CodeDeploy, SNS

Cloudformation & VPC, EC2, RDSCan Abacıgil

- TeamSQL AWS Architecture - VPC Introduction (Public, private subnets) and Demo - EC2 Introduction and Demo - RDS Introduction and Demo - Introduction to Cloudformation - A simple Cloudformation Script and make it live (Creating EC2 with Cloudformation) - Deleting Cloudformation Stack - More advanced Cloudformation Script and make it live (Cloudformation parameters, VPC, public, private subnets, RDS, ElasticBeanstalk, ElastiCache) - Updating Cloudformation Stack - Hands on - Advanced Cloudformation Script

Azure cli-azure devopsThi Nguyen Dinh

Azure CLI and Azure DevOps (VSTS) provide tools for managing Azure infrastructure and implementing continuous integration/delivery (CI/CD) pipelines. Azure CLI allows managing Azure resources like web apps from the command line, with commands, querying, and configurations. Azure DevOps includes Azure Boards for planning work, Azure Repos for source control, Azure Pipelines for building/deploying apps, and Azure Test Plans and Artifacts. It demonstrates setting up a project with work tracking and source control, then creating build and publish pipelines to continuously integrate and deploy a .NET sample app to Azure.

Kubernetes on awsYousun Jeong

Kubernetes on AWS allows users to deploy and manage Kubernetes clusters on the AWS cloud infrastructure. It provides tools to create clusters across multiple AWS availability zones for high availability. Users can define Kubernetes objects like pods, services, deployments etc using kubectl and utilize AWS services like EBS volumes for persistent storage. The presentation demonstrated setting up a Kubernetes cluster on AWS using kube-up.sh along with examples of using EBS volumes in pods through persistent volume claims. It also showed monitoring and managing applications running on the Kubernetes cluster deployed on AWS.

Introduction to AWSAngel Borroy López

Cloudsolutionday 2016: DevOps workflow with Docker on AWSAWS Vietnam Community

The document discusses DevOps workflow with Docker on AWS. It describes using Docker to isolate application environments, increasing team productivity and decreasing development team size. Key elements include using Gitlab for source control and CI/CD, building Docker images via Gitlab runners, and deploying to Kubernetes clusters. The workflow allows writing applications once and running them anywhere and forever through continuous integration and delivery of Docker images to private container registries on AWS.

從劍宗到氣宗－談AWS ECS與Serverless最佳實踐Pahud Hsieh

This document summarizes a presentation about best practices for AWS ECS and serverless architectures. It discusses the challenges of traditional infrastructures and benefits of containerization. It provides an overview of AWS ECS for container management and auto-scaling capabilities. It also introduces AWS Lambda and API Gateway for building serverless applications, including their advantages of being cloud-native and cost-effective with minimal infrastructure to manage. Some limitations of serverless architectures are also outlined. The conclusion encourages embracing immutable infrastructure, event-driven computing, and focusing on business logic over infrastructure when possible.

PowerPoint Presentationlalitjangra9

- Alfresco solutions can be provisioned and deployed on Amazon Web Services (AWS) infrastructure as a service (IaaS) cloud platform. This provides advantages like infinite resources, pay as you go pricing, rapid scalability, and provisioning speed. - AWS services like EC2, S3, and EBS allow deployment of virtual servers and storage. DevOps tools like Puppet can help automate configuration and provisioning. - Deploying Alfresco on AWS provides opportunities for auto scaling, historical usage statistics from monitoring, and a potential Alfresco admin console for management of the AWS environment. However, challenges include network and storage configuration and automated provisioning of new nodes

Azure Service Fabric MeshUdaiappa Ramachandran

Udai introduces Service Fabric Mesh, a new managed service from Microsoft for deploying containerized microservices applications. Some key points about SF Mesh: it focuses on the application rather than infrastructure; provides a fully managed cluster; applications can be deployed from CI/CD pipelines; and it includes auto-scaling and blue/green deployments. The presentation provides an overview of SF Mesh and its resource model, how it works, things to know in preview, and a demo of deploying an application using the CLI commands.

Devops on AWSAWS Riyadh User Group

This document provides an overview of DevOps on AWS. It discusses DevOps culture and goals of speed, reliability, and improved collaboration. It then explains why AWS is suitable for DevOps with managed services, scale, automation, and security. The document outlines components of DevOps practices including continuous integration (CI), continuous delivery (CD), infrastructure as code, and continuous monitoring. It also reviews deployment strategies and AWS developer tools to support CI/CD workflows such as CodeCommit, CodeBuild, CodeDeploy, CodePipeline, Cloud9, and CodeStar.

Azure: Docker Container orchestration, PaaS ( Service Farbic ) and High avail...Alexey Bokov

Deep dive into Azure cloud technologies including common considerations about technology choices and then going deep into some of them. First we start from Azure Container Service and Docker containers orchestration by using Mesos or Swarm. Next part is about PaaS v2 which called Azure Service Fabric - crash course and deep dive into some parts of SF. After that we going through high Availability and Disaster Recovery in Azure: - Azure DNS - cloud API for DNS records hosting - Traffic Manager – load balancing and fault-tolerance on DNS level - Azure Load Balancer – load balancing on transport level -Application Gateway – load balancing on application level Last part of deck is about IaaS based services and some updates for storage service: * Azure Batch for computational tasks * VM Scale sets * Storage - managed disks and cool storage

Building Open Source Platforms on AWS (April 2017)Julien SIMON

The document provides an overview of building open source platforms on AWS. It discusses using AWS services like EC2, RDS, ElastiCache, and S3 to run popular open source software like Linux, Docker, databases, analytics tools, and development frameworks. It provides links to AWS documentation and quick start guides on deploying solutions like Elasticsearch, Kafka, and Hadoop on AWS infrastructure.

Ansible 2.0 - How to use Ansible to automate your applications in AWS.Idan Tohami

Big datatraining.in devops-part2 (1)sKaushikNarayanan

This document provides details about a DevOps training course offered by www.company.com. It includes information on the training modes (classroom and online), topics that will be covered like AWS, Chef, Ansible, and prerequisites. Target audience for the course includes project managers, developers, and DevOps professionals. The training will have both theory and hands-on sessions using cloud servers. Upon completion, students will get an internal certificate and can also opt to get an external DevOps certification with assistance. They will also get access to cloud servers for 2 months to work on proof-of-concept projects.

Big datatraining.in devops-part2 (1)MvkZ

Big datatraining.in devops-part2 (1)sKaushikNarayanan

This document provides details about a DevOps training course offered by www.company.com. It includes information on the training modes (classroom and online), topics covered (AWS, Chef, Ansible), prerequisites, course outlines for each topic, target audience, key benefits, sample projects, and certification information. Upon completion of the training and a proof-of-concept project, students will receive an internal course completion certificate and an external DevOps Associate certificate. The training has a hands-on focus using real production cloud servers.

Big datatraining.in devops-part2 (1)MvkZ

The AWS DevOps combo (January 2017)Julien SIMON

Application Lifecycle Management on AWSDavid Mat

Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...Emerson Eduardo Rodrigues Von Staffen

Cloudformation & VPC, EC2, RDSCan Abacıgil

Azure cli-azure devopsThi Nguyen Dinh

Kubernetes on awsYousun Jeong

Introduction to AWSAngel Borroy López

Cloudsolutionday 2016: DevOps workflow with Docker on AWSAWS Vietnam Community

從劍宗到氣宗－談AWS ECS與Serverless最佳實踐Pahud Hsieh

PowerPoint Presentationlalitjangra9

Azure Service Fabric MeshUdaiappa Ramachandran

Devops on AWSAWS Riyadh User Group

Azure: Docker Container orchestration, PaaS ( Service Farbic ) and High avail...Alexey Bokov

Building Open Source Platforms on AWS (April 2017)Julien SIMON

Ansible 2.0 - How to use Ansible to automate your applications in AWS.Idan Tohami

Big datatraining.in devops-part2 (1)sKaushikNarayanan

Big datatraining.in devops-part2 (1)MvkZ

Big datatraining.in devops-part2 (1)sKaushikNarayanan

Big datatraining.in devops-part2 (1)MvkZ

More from Andrei Savu (20)

The Evolving Landscape of Data EngineeringAndrei Savu

The document discusses the evolving landscape of data engineering. It provides context on the past, present, and future of data engineering. Specifically, it notes that in the past, data engineering was driven by open source communities and the early histories of AWS and Google Cloud. It describes common present-day patterns like serverless architectures and data locality. Finally, it outlines a future wish list, including data catalogs, monitoring systems, and more intelligent data infrastructure. The document concludes by offering recommendations on where to start with technologies, Google Cloud courses, and developing domain knowledge.

The Evolving Landscape of Data EngineeringAndrei Savu

Data Engineering is a relatively new, but fast evolving discipline that spans multiple environments and technologies, from traditional data centers to hyper-scale cloud providers, a discipline that combines closed-source, homegrown and open source software to create scalable data pipelines and power incredible new product features. In this presentation, we will go over the last 5-10 years of technology trends and advancements and bring all of that together in a story about modern day Data Engineering and the magic behind it.

Recap on AWS Lambda after re:Invent 2015Andrei Savu

One Hadoop, Multiple Clouds - NYC Big Data MeetupAndrei Savu

Introducing Cloudera Director at Big Data BashAndrei Savu

APIs & Underlying Protocols #APICraftSFAndrei Savu

Challenges for running Hadoop on AWS - AdvancedAWS MeetupAndrei Savu

Nowadays we've got all the tools we need to spin-up and tear-down clusters with hundreds of nodes in minutes and this puts more pressure on the tools we use to configure and monitor our applications. This challenge is even more interesting when we have to deal with long running distributed data storage and processing systems like Hadoop. In this talk we will look into some of the challenges we need to deal with when creating and managing Hadoop clusters in AWS, we will discuss improvement opportunities in monitoring (e.g. detecting and dealing with instance failure, resource contention & noisy neighbors) and a bit about the future and how we should go about disconnecting workload dispatch from cluster lifecycle.

Cloud as a Data PlatformAndrei Savu

Apache Provisionr (incubating) - Bucharest JUG 10Andrei Savu

Creating pools of Virtual Machines - ApacheCon NA 2013Andrei Savu

Data Scientist ToolboxAndrei Savu

This document provides an overview of the data science process and tools for a data science project. It discusses identifying important business questions to answer with data, extracting relevant data from sources, cleaning and sampling the data, analyzing samples to create models and check hypotheses, applying results to full data sets, visualizing findings, automating and deploying solutions, and continuously learning and improving through an iterative process. Key tools mentioned include Hadoop, R, Python, Excel, and various data wrangling, analysis, and visualization tools.

2012 in Review - Bucharest JUGAndrei Savu

The document summarizes the Bucharest JUG (Java User Group) meetings from May 2012 to November 2012. It provides details on the monthly meetings including average attendance of 25-80 people, topics covered such as Guava, Maven, JavaScript UI for REST, and thanks to speakers and sponsors. It outlines future plans such as live streaming international speakers, a donations/job board, and potential future topics around alternative build systems, deployment options, and monitoring tools. Contact details are provided.

Metrics for Web Applications - Netcamp 2012Andrei Savu

Counters with Riak on Amazon EC2 at HackoverAndrei Savu

The document discusses using distributed counters with Riak on Amazon EC2. It introduces Riak as a distributed key-value database focused on availability, fault tolerance, simplicity and scalability. It describes using Riak features like consistent hashing, replication and automatic load balancing to implement a REST API for counters with eventual consistency. The demo shows implementing and accessing counters across local machines and EC2 regions to demonstrate the architecture.

Simple REST with DropwizardAndrei Savu

This document provides an overview of using Dropwizard, an open-source Java framework, to build RESTful web services. It discusses REST concepts like resources and representations, REST verbs like GET and POST, and architectures for REST APIs. It then introduces Dropwizard and its components for building HTTP services with features like Jetty, Jersey, Jackson, and metrics support. The document demonstrates a sample Dropwizard TODO list application with REST endpoints and resources and discusses considerations for development, testing, and deployment.

Guava Overview Part 2 Bucharest JUG #2 Andrei Savu

This document provides an overview of Guava and discusses caches and services. Guava is Google's core Java library that contains utilities like caches, primitives, collections, and concurrency libraries. Caches can improve performance by storing values to avoid expensive re-computation. Services in Guava define lifecycles for objects with operational state and allow asynchronous starting and stopping. The document describes cache eviction strategies, service implementations, and where to find more information on Guava features like functional idioms and concurrency.

Guava Overview. Part 1 @ Bucharest JUG #1 Andrei Savu

Guava is a Java library developed by Google that includes common libraries such as collections, caching, primitives support, concurrency libraries, and generalized utility classes. The talk covered the basic utilities in Guava including using and avoiding null, preconditions, common object utilities, ordering, and primitive array utilities. It also discussed the collections in Guava including immutable collections, new collection types like multisets and multimaps, collection utilities, and ways to extend the collections framework with decorators.

Polyglot Persistence & Big Data in the CloudAndrei Savu

This document discusses polyglot persistence and deploying big data technologies in the cloud. It introduces databases like HBase, Cassandra, MongoDB, CouchDB, and Riak for storing large amounts of data. Technologies like Elasticsearch and Solr are presented for search. Hadoop is described as a framework for distributed processing of large datasets. The document concludes by discussing how the Apache Whirr project can be used to deploy these technologies on cloud infrastructure.

Building a Great Team in Open Source - Open Agile 2011Andrei Savu

Apache WhirrAndrei Savu

Apache Whirr is a set of libraries for running cloud services on-demand in a cloud-neutral way. It provides common APIs and defaults for deploying clusters running Hadoop, Cassandra, HBase, and Zookeeper on EC2 and Rackspace. Whirr configurations allow deploying typical clusters with a single command. It is being developed further to support private clouds and new services, and to integrate with Hudson for testing fault injection scenarios on small test clusters.

The Evolving Landscape of Data EngineeringAndrei Savu

Recap on AWS Lambda after re:Invent 2015Andrei Savu

One Hadoop, Multiple Clouds - NYC Big Data MeetupAndrei Savu

Introducing Cloudera Director at Big Data BashAndrei Savu

APIs & Underlying Protocols #APICraftSFAndrei Savu

Challenges for running Hadoop on AWS - AdvancedAWS MeetupAndrei Savu

Cloud as a Data PlatformAndrei Savu

Apache Provisionr (incubating) - Bucharest JUG 10Andrei Savu

Creating pools of Virtual Machines - ApacheCon NA 2013Andrei Savu

Data Scientist ToolboxAndrei Savu

2012 in Review - Bucharest JUGAndrei Savu

Metrics for Web Applications - Netcamp 2012Andrei Savu

Counters with Riak on Amazon EC2 at HackoverAndrei Savu

Simple REST with DropwizardAndrei Savu

Guava Overview Part 2 Bucharest JUG #2 Andrei Savu

Guava Overview. Part 1 @ Bucharest JUG #1 Andrei Savu

Polyglot Persistence & Big Data in the CloudAndrei Savu

Building a Great Team in Open Source - Open Agile 2011Andrei Savu

Apache WhirrAndrei Savu

Recently uploaded (20)

Technology Trends in 2025: AI and Big Data AnalyticsInData Labs

At InData Labs, we have been keeping an ear to the ground, looking out for AI-enabled digital transformation trends coming our way in 2025. Our report will provide a look into the technology landscape of the future, including: -Artificial Intelligence Market Overview -Strategies for AI Adoption in 2025 -Anticipated drivers of AI adoption and transformative technologies -Benefits of AI and Big data for your business -Tips on how to prepare your business for innovation -AI and data privacy: Strategies for securing data privacy in AI models, etc. Download your free copy nowand implement the key findings to improve your business.

Cyber Awareness overview for 2025 month of securityriccardosl1

Splunk Security Update | Public Sector Summit Germany 2025Splunk

HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungenpanagenda

Webinar Recording: https://www.panagenda.com/webinars/hcl-nomad-web-best-practices-und-verwaltung-von-multiuser-umgebungen/ HCL Nomad Web wird als die nächste Generation des HCL Notes-Clients gefeiert und bietet zahlreiche Vorteile, wie die Beseitigung des Bedarfs an Paketierung, Verteilung und Installation. Nomad Web-Client-Updates werden “automatisch” im Hintergrund installiert, was den administrativen Aufwand im Vergleich zu traditionellen HCL Notes-Clients erheblich reduziert. Allerdings stellt die Fehlerbehebung in Nomad Web im Vergleich zum Notes-Client einzigartige Herausforderungen dar. Begleiten Sie Christoph und Marc, während sie demonstrieren, wie der Fehlerbehebungsprozess in HCL Nomad Web vereinfacht werden kann, um eine reibungslose und effiziente Benutzererfahrung zu gewährleisten. In diesem Webinar werden wir effektive Strategien zur Diagnose und Lösung häufiger Probleme in HCL Nomad Web untersuchen, einschließlich - Zugriff auf die Konsole - Auffinden und Interpretieren von Protokolldateien - Zugriff auf den Datenordner im Cache des Browsers (unter Verwendung von OPFS) - Verständnis der Unterschiede zwischen Einzel- und Mehrbenutzerszenarien - Nutzung der Client Clocking-Funktion

What is Model Context Protocol(MCP) - The new technology for communication bw...Vishnu Singh Chundawat

The MCP (Model Context Protocol) is a framework designed to manage context and interaction within complex systems. This SlideShare presentation will provide a detailed overview of the MCP Model, its applications, and how it plays a crucial role in improving communication and decision-making in distributed systems. We will explore the key concepts behind the protocol, including the importance of context, data management, and how this model enhances system adaptability and responsiveness. Ideal for software developers, system architects, and IT professionals, this presentation will offer valuable insights into how the MCP Model can streamline workflows, improve efficiency, and create more intuitive systems for a wide range of use cases.

AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...SOFTTECHHUB

I started my online journey with several hosting services before stumbling upon Ai EngineHost. At first, the idea of paying one fee and getting lifetime access seemed too good to pass up. The platform is built on reliable US-based servers, ensuring your projects run at high speeds and remain safe. Let me take you step by step through its benefits and features as I explain why this hosting solution is a perfect fit for digital entrepreneurs.

Andrew Marnell: Transforming Business Strategy Through Data-Driven InsightsAndrew Marnell

Manifest Pre-Seed Update | A Humanoid OEM Deeptech In Francechb3

Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...BookNet Canada

Book industry standards are evolving rapidly. In the first part of this session, we’ll share an overview of key developments from 2024 and the early months of 2025. Then, BookNet’s resident standards expert, Tom Richardson, and CEO, Lauren Stewart, have a forward-looking conversation about what’s next. Link to recording, presentation slides, and accompanying resource: https://bnctechforum.ca/sessions/standardsgoals-for-2025-standards-certification-roundup/ Presented by BookNet Canada on May 6, 2025 with support from the Department of Canadian Heritage.

2025-05-Q4-2024-Investor-Presentation.pptxSamuele Fogagnolo

SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdfPrecisely

Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep DiveScyllaDB

Want to learn practical tips for designing systems that can scale efficiently without compromising speed? Join us for a workshop where we’ll address these challenges head-on and explore how to architect low-latency systems using Rust. During this free interactive workshop oriented for developers, engineers, and architects, we’ll cover how Rust’s unique language features and the Tokio async runtime enable high-performance application development. As you explore key principles of designing low-latency systems with Rust, you will learn how to: - Create and compile a real-world app with Rust - Connect the application to ScyllaDB (NoSQL data store) - Negotiate tradeoffs related to data modeling and querying - Manage and monitor the database for consistently low latencies

IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...organizerofv

UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager APIUiPathCommunity

Join this UiPath Community Berlin meetup to explore the Orchestrator API, Swagger interface, and the Test Manager API. Learn how to leverage these tools to streamline automation, enhance testing, and integrate more efficiently with UiPath. Perfect for developers, testers, and automation enthusiasts! 📕 Agenda Welcome & Introductions Orchestrator API Overview Exploring the Swagger Interface Test Manager API Highlights Streamlining Automation & Testing with APIs (Demo) Q&A and Open Discussion Perfect for developers, testers, and automation enthusiasts! 👉 Join our UiPath Community Berlin chapter: https://community.uipath.com/berlin/ This session streamed live on April 29, 2025, 18:00 CET. Check out all our upcoming UiPath Community sessions at https://community.uipath.com/events/.

Rusty Waters: Elevating Lakehouses Beyond Sparkcarlyakerly1

Spark is a powerhouse for large datasets, but when it comes to smaller data workloads, its overhead can sometimes slow things down. What if you could achieve high performance and efficiency without the need for Spark? At S&P Global Commodity Insights, having a complete view of global energy and commodities markets enables customers to make data-driven decisions with confidence and create long-term, sustainable value. 🌍 Explore delta-rs + CDC and how these open-source innovations power lightweight, high-performance data applications beyond Spark! 🚀

HCL Nomad Web – Best Practices and Managing Multiuser Environmentspanagenda

Webinar Recording: https://www.panagenda.com/webinars/hcl-nomad-web-best-practices-and-managing-multiuser-environments/ HCL Nomad Web is heralded as the next generation of the HCL Notes client, offering numerous advantages such as eliminating the need for packaging, distribution, and installation. Nomad Web client upgrades will be installed “automatically” in the background. This significantly reduces the administrative footprint compared to traditional HCL Notes clients. However, troubleshooting issues in Nomad Web present unique challenges compared to the Notes client. Join Christoph and Marc as they demonstrate how to simplify the troubleshooting process in HCL Nomad Web, ensuring a smoother and more efficient user experience. In this webinar, we will explore effective strategies for diagnosing and resolving common problems in HCL Nomad Web, including - Accessing the console - Locating and interpreting log files - Accessing the data folder within the browser’s cache (using OPFS) - Understand the difference between single- and multi-user scenarios - Utilizing Client Clocking

#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025BookNet Canada

Book industry standards are evolving rapidly. In the first part of this session, we’ll share an overview of key developments from 2024 and the early months of 2025. Then, BookNet’s resident standards expert, Tom Richardson, and CEO, Lauren Stewart, have a forward-looking conversation about what’s next. Link to recording, transcript, and accompanying resource: https://bnctechforum.ca/sessions/standardsgoals-for-2025-standards-certification-roundup/ Presented by BookNet Canada on May 6, 2025 with support from the Department of Canadian Heritage.

Into The Box Conference Keynote Day 1 (ITB2025)Ortus Solutions, Corp

Drupalcamp Finland – Measuring Front-end Energy ConsumptionExove

ThousandEyes Partner Innovation Updates for May 2025ThousandEyes