This document describes Spack, an open source package management system for HPC environments. Spack allows building and installing software and its dependencies across different compiler versions and build configurations. It uses a Python DSL for package definitions and handles dependency resolution and installation. Recent updates include support for binary packages to improve performance through pre-built binaries while retaining flexibility of source installations.
Run the elastic stack on kubernetes with eck Daliya Spasova
This document discusses the challenges of running the Elastic Stack on Kubernetes and introduces Elastic Cloud on Kubernetes (ECK) as a solution. ECK uses operators to deploy and manage Elasticsearch, Kibana, APM Server, and other Elastic products on Kubernetes. The operator watches custom resources, and its reconciliation loop automatically handles tasks like creating pods, configmaps, secrets, services and managing upgrades/scaling without downtime or data loss. ECK provides simplified configuration, security, high availability, and automation to make running Elastic Stack on Kubernetes easier.
The document discusses P4 support in ONOS. It provides an overview of the P4 language and P4Runtime framework, and then describes ONOS's PI framework for controlling P4 programmable switches. The PI framework models P4 pipelines and allows both pipeline-agnostic and pipeline-aware applications. It translates between ONOS abstractions and P4Runtime messages using a pipeline interpreter and driver behaviors defined in a pipeconf file. The document demonstrates how ONOS can deploy and program P4 pipelines using these components.
Introduction to Chaos Engineering with Microsoft AzureAna Medina
https://www.gremlin.com/webinars/ce-on-azure/
Join us for a walkthrough on how to get started with Chaos Engineering on Azure. Learn the fundamentals of Chaos Engineering and how to build more reliable applications on Azure.
In this live session, we’ll show you how to get started running experiments on Azure’s managed Kubernetes (AKS) and how to implement continuous Chaos Engineering using Azure Pipelines. Then be sure to stay until the end for live Q&A.
AGENDA
- Learn the history, principles and practice of Chaos Engineering
- How to get started with Chaos Engineering on Azure
- Run chaos experiments to simulate common real-world failures on AKS
- How to implement Chaos Engineering Experiments on Azure Pipelines
HTTP proxies act as both servers and clients by receiving requests from web clients and forwarding the requests to web servers, while also sending requests to servers on behalf of clients. Proxies are commonly used for filtering content, access control, security firewalls, caching, load balancing, transcoding content formats, and providing anonymity. Proxies can be configured in various network architectures including private networks, ISP access points, as reverse proxies in front of servers, and at network exchange points.
Synadia/NATS Team Presentations for NATS Connect Live on April 16, 2020. To see the recorded event, go to our NATS YouTube Channel https://youtube.com/c/nats_messaging
Tsuru é um PaaS open source extensível que suporta diversas linguagens. Com ele, é possível gerenciar e deployar milhares de aplicações de forma simples e rápida. Nesta palestra, vou apresentar a arquitetura do Tsuru, os principais componentes, como ele gerencia as aplicações, os mecanismos para resiliência diante de falhas e recursos como escalabilidade automática de aplicações.
This document provides an overview of cBPF and eBPF. It discusses the history and implementation of cBPF, including how it was originally used for packet filtering. It then covers eBPF in more depth, explaining what it is, its history, implementation including different program types and maps. It also discusses several uses of eBPF including networking, firewalls, DDoS mitigation, profiling, security, and chaos engineering. Finally, it introduces XDP and DPDK, comparing XDP's benefits over DPDK.
Istio is a service mesh—a modernized service networking layer that provides a transparent and language-independent way to flexibly and easily automate application network functions. Istio is designed to run in a variety of environments: on-premise, cloud-hosted, in Kubernetes containers.
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDPThomas Graf
This talk will start with a deep dive and hands on examples of BPF, possibly the most promising low level technology to address challenges in application and network security, tracing, and visibility. We will discuss how BPF evolved from a simple bytecode language to filter raw sockets for tcpdump to the a JITable virtual machine capable of universally extending and instrumenting both the Linux kernel and user space applications. The introduction is followed by a concrete example of how the Cilium open source project applies BPF to solve networking, security, and load balancing for highly distributed applications. We will discuss and demonstrate how Cilium with the help of BPF can be combined with distributed system orchestration such as Docker to simplify security, operations, and troubleshooting of distributed applications.
This presentation provides an overview of MPEG-DASH and future developments, namely common media application format and virtual reality/360-degree streaming.
Avro Tutorial - Records with Schema for Kafka and HadoopJean-Paul Azar
Covers how to use Avro to save records to disk. This can be used later to use Avro with Kafka Schema Registry. This provides background on Avro which gets used with Hadoop and Kafka.
Porting a new architecture (NDS32) to open wrt projectMacpaul Lin
The document discusses porting a new RISC architecture called NDS32 to the OpenWrt project. It provides details about NDS32 such as the CPU cores and benefits. It then explains the framework of OpenWrt's build system and the flexibility it provides. Finally, it outlines the specific porting items needed to support NDS32 in OpenWrt, including changes to definitions, the toolchain, target, and other components.
This document provides an overview of CI/CD on Google Cloud Platform. It discusses key DevOps principles like treating infrastructure as code and automating processes. It then describes how GCP services like Cloud Build, Container Registry, Source Repositories, and Stackdriver can help achieve CI/CD. Spinnaker is mentioned as an open-source continuous delivery platform that integrates well with GCP. Overall the document outlines the benefits of CI/CD and how GCP makes CI/CD implementation easy and scalable.
OpenCV를 활용하는 영상처리 어플리케이션 개발자들은 항상 GPU 자원을 활용하고 싶을 것이다. 하지만 이기종 컴퓨팅 환경에서 CPU 이외의 다른 하드웨어 자원을 활용하는 것은 개발 환경 및 백그라운드 지식 등의 많은 어려움이 따른다.
GPGPU 활용에 가장 상용화로 성공한 대중적인 솔루션으로는 nVidia 사의 CUDA 기술이 있지만, 그 외에도 GPGPU 자원을 쉽게 활용할 수 있는 오픈 플랫폼이 있는데 이것이 OpenCL 표준이다.
최근 하드웨어와 소프트웨어 진영에서 모두 OpenCL의 지원 및 발전이 두드러지며 점점 더 확산되는 추세이다.
OpenCV 진영에서도, 3.0이 정식 릴리즈 면서 본격적으로 OpenCL을 활용하기가 한층 쉬워졌다.
SPACK: A Package Manager for Supercomputers, Linux, and MacOSinside-BigData.com
“HPC software is becoming increasingly complex. The space of possible build configurations is combinatorial, and existing package management tools do not handle these complexities well. Because of this, most HPC software is built by hand. This talk introduces “Spack“, an open-source tool for scientific package management which helps developers and cluster administrators avoid having to waste countless hours porting and rebuilding software. Spack uses concise package recipes written in Python to automate builds with arbitrary combinations of compilers, MPI versions, and dependency libraries. With Spack, users can rapidly install software without knowing how to build it; developers can efficiently manage automatic builds of tens or hundreds of dependency libraries; and HPC centers staff can deploy many versions of software for thousands of users.”
Watch the video: http://insidehpc.com/2017/04/spack-package-manager-supercomputers-linux-macos/
Learn more: https://spack.io/
and
http://www.hpcadvisorycouncil.com/events/2017/swiss-workshop/agenda.php
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Deep Dive into Building a Secure & Multi-tenant SaaS Solution with NATSNATS
Waldemar Quevedo gave a talk on building decentralized applications using NATS. He demonstrated building a chat application to showcase NATS features like streams and services. The chat app uses NATS streams for posts and direct messages, and a service for requesting access credentials. Users are isolated by account and can only subscribe to their own direct messages. The complete application code is available on GitHub.
Netronome's half-day tutorial on host data plane acceleration at ACM SIGCOMM 2018 introduced attendees to models for host data plane acceleration and provided an in-depth understanding of SmartNIC deployment models at hyperscale cloud vendors and telecom service providers.
Presenter Bios
Jakub Kicinski is a long term Linux kernel contributor, who has been leading the kernel team at Netronome for the last two years. Jakub’s major contributions include the creation of BPF hardware offload mechanisms in the kernel and bpftool user space utility, as well as work on the Linux kernel side of OVS offload.
David Beckett is a Software Engineer at Netronome with a strong technical background of computer networks including academic research with DDoS. David has expertise in the areas of Linux architecture and computer programming. David has a Masters Degree in Electrical, Electronic Engineering at Queen’s University Belfast and continues as a PhD student studying Emerging Application Layer DDoS threats.
- The document describes the Zephyr real-time operating system which is open source, has a vibrant community, and is built with safety and security in mind. It supports multiple architectures and hardware boards and has vendor-neutral governance.
- Key features include being highly configurable, modular, and product development ready using long-term supported releases that include security updates. It provides a range of OS services and supports over 100 sensors out of the box.
- The project focuses on safety and has established committees to improve security practices and work towards safety certification for applications requiring functional safety standards like IEC 61508.
CI/CD Pipeline as a Code using Jenkins 2Mayank Patel
Mayank Patel from Oildex gave a presentation on Jenkins 2 Pipelines. He discussed how pipelines allow continuous delivery through features like resilience, pausability, and efficiency. Pipelines can be configured as code in source control and provide security and reusability. The presentation covered the Jenkins environment, ideal pipeline flows, important plugins, and included a demo of a sample pipeline configured with Docker.
The 7 Deadly Sins of Packet Processing - Venky Venkatesan and Bruce Richardsonharryvanhaaren
The document summarizes seven deadly sins of packet processing that can negatively impact performance:
1) Unpredictable branches that confuse the branch predictor. Code should guide the compiler on likely/unlikely cases.
2) Incorrect prefetching that pulls in unnecessary data or data needed by other cores, adding overhead. Hardware prefetchers often help but can also share cache lines inadvertently.
3) Per-packet operations like memory I/O and atomics that have overhead magnified at the per-packet level.
This document provides an introduction to eBPF and XDP. It discusses the history of BPF and how it evolved into eBPF. Key aspects of eBPF covered include the instruction set, JIT compilation, verifier, helper functions, and maps. XDP is introduced as a way to program the data plane using eBPF programs attached early in the receive path. Example use cases and performance benchmarks for XDP are also mentioned.
The document discusses implementing client quotas in Apache Kafka to ensure fair sharing of broker resources among clients. It describes enforcing quotas based on client IDs and overriding quotas for specific clients. Metrics are used to monitor client throughput and enforce quotas by delaying requests if clients exceed their limits. The approach aims to improve performance for well-behaved clients by preventing a few misbehaving clients from impacting others. Evaluation results show improved throughput and latency for clients when quotas are enforced.
In the Cloud Native community, eBPF is gaining popularity, which can often be the best solution for solving different challenges with deep observability of system. Currently, eBPF is being embraced by major players.
Mydbops co-Founder, Kabilesh P.R (MySQL and Mongo Consultant) illustrates on debugging linux issues with eBPF. A brief about BPF & eBPF, BPF internals and the tools in actions for faster resolution.
Logs/Metrics Gathering With OpenShift EFK StackJosef Karásek
This document summarizes a presentation about logs and metrics gathering with the OpenShift EFK stack. It introduces the OpenShift logging team and their objectives of collecting distributed logs in a common data model with security and scalability. It describes the main components of Fluendt for collection and normalization and Elasticsearch for storage. It provides examples of using the logging stack with OpenShift, OpenStack, and oVirt and advice for custom application logging.
Protocol Buffers are a language-neutral, platform-neutral way of serializing structured data developed by Google to deal with problems of scale. Data formats are defined using .proto files which are then compiled to generate data access classes. This allows structured data to be serialized and transmitted efficiently across various languages and platforms while maintaining backward/forward compatibility. Protocol Buffers offer advantages over other solutions like XML in being more efficient, compact and faster. They have found many use cases at Google and other companies for messaging, data storage and transmission.
Istio is a service mesh—a modernized service networking layer that provides a transparent and language-independent way to flexibly and easily automate application network functions. Istio is designed to run in a variety of environments: on-premise, cloud-hosted, in Kubernetes containers.
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDPThomas Graf
This talk will start with a deep dive and hands on examples of BPF, possibly the most promising low level technology to address challenges in application and network security, tracing, and visibility. We will discuss how BPF evolved from a simple bytecode language to filter raw sockets for tcpdump to the a JITable virtual machine capable of universally extending and instrumenting both the Linux kernel and user space applications. The introduction is followed by a concrete example of how the Cilium open source project applies BPF to solve networking, security, and load balancing for highly distributed applications. We will discuss and demonstrate how Cilium with the help of BPF can be combined with distributed system orchestration such as Docker to simplify security, operations, and troubleshooting of distributed applications.
This presentation provides an overview of MPEG-DASH and future developments, namely common media application format and virtual reality/360-degree streaming.
Avro Tutorial - Records with Schema for Kafka and HadoopJean-Paul Azar
Covers how to use Avro to save records to disk. This can be used later to use Avro with Kafka Schema Registry. This provides background on Avro which gets used with Hadoop and Kafka.
Porting a new architecture (NDS32) to open wrt projectMacpaul Lin
The document discusses porting a new RISC architecture called NDS32 to the OpenWrt project. It provides details about NDS32 such as the CPU cores and benefits. It then explains the framework of OpenWrt's build system and the flexibility it provides. Finally, it outlines the specific porting items needed to support NDS32 in OpenWrt, including changes to definitions, the toolchain, target, and other components.
This document provides an overview of CI/CD on Google Cloud Platform. It discusses key DevOps principles like treating infrastructure as code and automating processes. It then describes how GCP services like Cloud Build, Container Registry, Source Repositories, and Stackdriver can help achieve CI/CD. Spinnaker is mentioned as an open-source continuous delivery platform that integrates well with GCP. Overall the document outlines the benefits of CI/CD and how GCP makes CI/CD implementation easy and scalable.
OpenCV를 활용하는 영상처리 어플리케이션 개발자들은 항상 GPU 자원을 활용하고 싶을 것이다. 하지만 이기종 컴퓨팅 환경에서 CPU 이외의 다른 하드웨어 자원을 활용하는 것은 개발 환경 및 백그라운드 지식 등의 많은 어려움이 따른다.
GPGPU 활용에 가장 상용화로 성공한 대중적인 솔루션으로는 nVidia 사의 CUDA 기술이 있지만, 그 외에도 GPGPU 자원을 쉽게 활용할 수 있는 오픈 플랫폼이 있는데 이것이 OpenCL 표준이다.
최근 하드웨어와 소프트웨어 진영에서 모두 OpenCL의 지원 및 발전이 두드러지며 점점 더 확산되는 추세이다.
OpenCV 진영에서도, 3.0이 정식 릴리즈 면서 본격적으로 OpenCL을 활용하기가 한층 쉬워졌다.
SPACK: A Package Manager for Supercomputers, Linux, and MacOSinside-BigData.com
“HPC software is becoming increasingly complex. The space of possible build configurations is combinatorial, and existing package management tools do not handle these complexities well. Because of this, most HPC software is built by hand. This talk introduces “Spack“, an open-source tool for scientific package management which helps developers and cluster administrators avoid having to waste countless hours porting and rebuilding software. Spack uses concise package recipes written in Python to automate builds with arbitrary combinations of compilers, MPI versions, and dependency libraries. With Spack, users can rapidly install software without knowing how to build it; developers can efficiently manage automatic builds of tens or hundreds of dependency libraries; and HPC centers staff can deploy many versions of software for thousands of users.”
Watch the video: http://insidehpc.com/2017/04/spack-package-manager-supercomputers-linux-macos/
Learn more: https://spack.io/
and
http://www.hpcadvisorycouncil.com/events/2017/swiss-workshop/agenda.php
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Deep Dive into Building a Secure & Multi-tenant SaaS Solution with NATSNATS
Waldemar Quevedo gave a talk on building decentralized applications using NATS. He demonstrated building a chat application to showcase NATS features like streams and services. The chat app uses NATS streams for posts and direct messages, and a service for requesting access credentials. Users are isolated by account and can only subscribe to their own direct messages. The complete application code is available on GitHub.
Netronome's half-day tutorial on host data plane acceleration at ACM SIGCOMM 2018 introduced attendees to models for host data plane acceleration and provided an in-depth understanding of SmartNIC deployment models at hyperscale cloud vendors and telecom service providers.
Presenter Bios
Jakub Kicinski is a long term Linux kernel contributor, who has been leading the kernel team at Netronome for the last two years. Jakub’s major contributions include the creation of BPF hardware offload mechanisms in the kernel and bpftool user space utility, as well as work on the Linux kernel side of OVS offload.
David Beckett is a Software Engineer at Netronome with a strong technical background of computer networks including academic research with DDoS. David has expertise in the areas of Linux architecture and computer programming. David has a Masters Degree in Electrical, Electronic Engineering at Queen’s University Belfast and continues as a PhD student studying Emerging Application Layer DDoS threats.
- The document describes the Zephyr real-time operating system which is open source, has a vibrant community, and is built with safety and security in mind. It supports multiple architectures and hardware boards and has vendor-neutral governance.
- Key features include being highly configurable, modular, and product development ready using long-term supported releases that include security updates. It provides a range of OS services and supports over 100 sensors out of the box.
- The project focuses on safety and has established committees to improve security practices and work towards safety certification for applications requiring functional safety standards like IEC 61508.
CI/CD Pipeline as a Code using Jenkins 2Mayank Patel
Mayank Patel from Oildex gave a presentation on Jenkins 2 Pipelines. He discussed how pipelines allow continuous delivery through features like resilience, pausability, and efficiency. Pipelines can be configured as code in source control and provide security and reusability. The presentation covered the Jenkins environment, ideal pipeline flows, important plugins, and included a demo of a sample pipeline configured with Docker.
The 7 Deadly Sins of Packet Processing - Venky Venkatesan and Bruce Richardsonharryvanhaaren
The document summarizes seven deadly sins of packet processing that can negatively impact performance:
1) Unpredictable branches that confuse the branch predictor. Code should guide the compiler on likely/unlikely cases.
2) Incorrect prefetching that pulls in unnecessary data or data needed by other cores, adding overhead. Hardware prefetchers often help but can also share cache lines inadvertently.
3) Per-packet operations like memory I/O and atomics that have overhead magnified at the per-packet level.
This document provides an introduction to eBPF and XDP. It discusses the history of BPF and how it evolved into eBPF. Key aspects of eBPF covered include the instruction set, JIT compilation, verifier, helper functions, and maps. XDP is introduced as a way to program the data plane using eBPF programs attached early in the receive path. Example use cases and performance benchmarks for XDP are also mentioned.
The document discusses implementing client quotas in Apache Kafka to ensure fair sharing of broker resources among clients. It describes enforcing quotas based on client IDs and overriding quotas for specific clients. Metrics are used to monitor client throughput and enforce quotas by delaying requests if clients exceed their limits. The approach aims to improve performance for well-behaved clients by preventing a few misbehaving clients from impacting others. Evaluation results show improved throughput and latency for clients when quotas are enforced.
In the Cloud Native community, eBPF is gaining popularity, which can often be the best solution for solving different challenges with deep observability of system. Currently, eBPF is being embraced by major players.
Mydbops co-Founder, Kabilesh P.R (MySQL and Mongo Consultant) illustrates on debugging linux issues with eBPF. A brief about BPF & eBPF, BPF internals and the tools in actions for faster resolution.
Logs/Metrics Gathering With OpenShift EFK StackJosef Karásek
This document summarizes a presentation about logs and metrics gathering with the OpenShift EFK stack. It introduces the OpenShift logging team and their objectives of collecting distributed logs in a common data model with security and scalability. It describes the main components of Fluendt for collection and normalization and Elasticsearch for storage. It provides examples of using the logging stack with OpenShift, OpenStack, and oVirt and advice for custom application logging.
Protocol Buffers are a language-neutral, platform-neutral way of serializing structured data developed by Google to deal with problems of scale. Data formats are defined using .proto files which are then compiled to generate data access classes. This allows structured data to be serialized and transmitted efficiently across various languages and platforms while maintaining backward/forward compatibility. Protocol Buffers offer advantages over other solutions like XML in being more efficient, compact and faster. They have found many use cases at Google and other companies for messaging, data storage and transmission.
Deview 2013 :: Backend PaaS, CloudFoundry 뽀개기Nanha Park
# Part 1
개발자의 주위환경에 대해 살펴보고 Cloud Foundry overview, Cloud Foundry 를 구성하는 components 마지막으로 Deploy 환경에 대해 알아보겠습니다.
# Part 2
설치부터 코드까지, 데모찍은 동영상은 추후 제공예정
부족한 부분은 [email protected] 으로 문의메일 주시면 성심성의껏 답변 드리겠습니다. 감사합니다.
Meetup tools for-cloud_native_apps_meetup20180510-vsminseok kim
마이크로서비스로 시스템을 구성하면 서비스간에 연관관계가 줄어들면서 서비스 릴리즈 속도가 높아지고 유연하게 대처할 수 있지만, 관리포인트가 늘어나게 되어 운영상에 많은 어려움을 마주치게 됩니다. 배포 될 때마다 생성되고 소멸되는 마이크로서비스를 다른 마이크로서비스가 쉽게 참조하게 하고 마이크로서비스들의 설정 정보를 일관되게 관리하는 일은 쉬운일이 아닙니다. 이러한 문제를 해결하기 위해 Spring Cloud 프로젝트와 같은 도구를 비롯하여 Pivotal Cloud Foundry와 같은 클라우드 플랫폼등이 있습니다. 이번 밋업에서는 마이크로서비스를 운영할 때의 어려운점과 도움을 주는 다양한 도구들에 대해 알아보도록 하겠습니다.
AWS에서는 다양한 언어에 대한 기계 번역(Translate) 등 AI 기능에 대한 API 서비스를 제공합니다. 본 실습에서는 이들 서비스(Serverless) 환경으로 AWS Amplify를 활용하여 소셜 모바일 앱을 안드로이드 기반으로 만들어 봅니다. 이를 위해 사용자 인증(Cognito), Graphql(Appsync) 등의 기능을 함께 활용합니다. 만들어진 앱은 AWS Device Farm을 통해서 클라우드 상에서 테스트 할 수 있습니다. 추가적으로Amazon Pinpoint를 이용하여 사용자 이벤트를 수집하고 분석하는 기능을 활용합니다.
35. 왜 Channel 인가?
수많은 네이티브 기능들..
알림, 딥링크, 센서, 카메라, 배터리, 위치, 사운드, 커넥션, 다
른앱과 정보공유, 다른앱 실행,로컬 스토리지 등등…
플랫폼이 시간이 지날수록 추가되는
각종 신규 기능들을 어떻게 다 빠르게
지원할 수 있을까?
48. Pigeon
Pigeon is a code generator tool to make communication between Flutter and the host platform type-
safe, easier, and faster.
Pigeon removes the necessity to manage strings across multiple platforms and languages. It also
improves efficiency over common method channel patterns. Most importantly though, it removes the
need to write custom platform channel code, since pigeon generates it for you.
Supported Platforms
Currently pigeon supports generating:
● Kotlin and Java code for Android
● Swift and Objective-C code for
iOS and macOS
● C++ code for Windows
Flutter Seoul 의 박제창님 비둘
73. iOS In-House Distribution
- TestFlight
- Ad-Hoc
- Firebase App Distribution
- 웹을 이용한 IPA 배포 (feat.Apple Developer Enterprise Program)
74. App Developer Enterprise Development
- Have 100 or more employees.
- Be a legal entity. We do not accept DBAs, fictitious businesses, trade names, or
branches.
- Use the program only to create proprietary, in-house apps for internal use, and to
distribute these apps privately and securely to employees within the organization.
- Have systems in place to ensure only employees can download your internal-use
apps, and to protect membership credentials and assets.
- Participate in and pass Apple’s verification interview and continuous evaluation
process.
75. Unlisted App Distribution
- 2022년 부터 제공한 비공개 AppStore 배포
- 엔터프라이즈 계정이 아닌 일반 개발자 계정으로도 배포가 가능
- 배포방식은 일반 스토어 배포와 동일하게 AppStoreConnect 에 심사
를 맡기지만 추가로 Unlisted App Distribution 이라는 배포 방식을 하
겠다고 description 란에 적어줘야함
- 또 별도의 Unlisted App Distribution 신청서를 작성