The document discusses the Circuit Breaker pattern, which protects applications from failures in external systems. It works by monitoring requests and switching between closed, open, and half-open states. When external systems are likely to fail, the circuit breaker will fail requests immediately instead of allowing them to fail and impact the application. Implementations of the pattern include Akka CircuitBreaker, Netflix Hystrix, and Apache Polygene's CircuitBreaker.
The document provides an overview of WebLogic Server topology, configuration, and administration. It describes key concepts such as domains, servers, clusters, Node Manager, and machines. It also covers configuration files, administration tools like the Administration Console and WLST, and some sample configuration schemes for development, high availability, and simplified administration.
The document discusses Oracle Database performance tuning. It begins by defining performance as the accepted throughput for a given workload. Performance tuning is defined as optimizing resource use to increase throughput and minimize contention. A performance problem occurs when database tasks do not complete in a timely manner, such as SQL running longer than usual or users facing slowness. Performance problems can be caused by contention for resources, overutilization of the system, or poorly written SQL. The document discusses various performance diagnostics tools and concepts like wait events, enqueues, I/O performance, and provides examples of how to analyze issues related to these areas.
Kafka Tutorial - Introduction to Apache Kafka (Part 1)Jean-Paul Azar
Why is Kafka so fast? Why is Kafka so popular? Why Kafka? This slide deck is a tutorial for the Kafka streaming platform. This slide deck covers Kafka Architecture with some small examples from the command line. Then we expand on this with a multi-server example to demonstrate failover of brokers as well as consumers. Then it goes through some simple Java client examples for a Kafka Producer and a Kafka Consumer. We have also expanded on the Kafka design section and added references. The tutorial covers Avro and the Schema Registry as well as advance Kafka Producers.
Watch this talk here: https://www.confluent.io/online-talks/apache-kafka-architecture-and-fundamentals-explained-on-demand
This session explains Apache Kafka’s internal design and architecture. Companies like LinkedIn are now sending more than 1 trillion messages per day to Apache Kafka. Learn about the underlying design in Kafka that leads to such high throughput.
This talk provides a comprehensive overview of Kafka architecture and internal functions, including:
-Topics, partitions and segments
-The commit log and streams
-Brokers and broker replication
-Producer basics
-Consumers, consumer groups and offsets
This session is part 2 of 4 in our Fundamentals for Apache Kafka series.
인터넷 환경에서 가장 많이 사용되는 전송 중 암호화 기술인 SSL/TLS 에 대한 설명입니다. TLS 는 2022년 현재 1.3 버전이 가장 최신버전이며 주요 기업 및 기관에서는 보안 강화 및 사용자 경험 개선을 위해 1.3 버전을 권고하고 있습니다. TLS 의 각 버전별 Handshake 방식과 적용되는 기술들이 어떻게 다른지 살펴보세요.
This document discusses real-time analytics on streaming data. It describes why real-time data streaming and analytics are important due to the perishable nature of data value over time. It then covers key components of real-time analytics systems including data sources, stream storage, stream ingestion, stream processing, and stream delivery. Finally, it discusses streaming data processing techniques like filtering, enriching, and converting streaming data.
The document discusses optimizing an Apache Pulsar deployment to handle 10 PB of data per day for a large customer. It estimates the initial cluster size needed using different storage options in Google Cloud Platform. It then describes four optimizations made - eliminating the journal, using direct I/O, compression, and improving the C++ client - and recalculates the cluster size after each optimization. The optimized deployment uses 200 VMs each with 24 local SSDs to meet the requirements.
Ceph: Open Source Storage Software Optimizations on Intel® Architecture for C...Odinot Stanislas
Après la petite intro sur le stockage distribué et la description de Ceph, Jian Zhang réalise dans cette présentation quelques benchmarks intéressants : tests séquentiels, tests random et surtout comparaison des résultats avant et après optimisations. Les paramètres de configuration touchés et optimisations (Large page numbers, Omap data sur un disque séparé, ...) apportent au minimum 2x de perf en plus.
This document provides an overview of Spring MVC, the model-view-controller framework for building web applications in Spring. It discusses Spring MVC's request processing workflow including the front controller and application context. It also covers controllers, mapping requests, returning views and data representation. Key topics include RESTful design, annotations like @RequestMapping and return types, and view resolvers for resolving JSP and other view technologies.
Apache Kafka is a high-throughput distributed messaging system that allows for both streaming and offline log processing. It uses Apache Zookeeper for coordination and supports activity stream processing and real-time pub/sub messaging. Kafka bridges the gaps between pure offline log processing and traditional messaging systems by providing features like batching, transactions, persistence, and support for multiple consumers.
This document provides an overview and introduction to Splunk, including:
1. It discusses the challenges of machine data including volume, velocity, variety and variability.
2. Splunk's mission is to make machine data accessible, usable and valuable to everyone.
3. It demonstrates how Splunk can unlock critical insights from machine data sources like order processing, social media, customer service systems and more.
In a world where compute is paramount, it is all too easy to overlook the importance of storage and IO in the performance and optimization of Spark jobs.
1. The document discusses microservices architecture and how Netflix transitioned from a monolithic architecture to microservices. Key aspects discussed include breaking the monolith into many small, independent services that are loosely coupled.
2. Netflix's microservices architecture is composed of hundreds of microservices running on thousands of servers. Each service focuses on doing a small, well-defined piece of work. Services communicate through well-defined APIs and share no code or databases.
3. The document provides examples of how other companies like Samsung and Vingle have also adopted microservices architectures on AWS, breaking monolithic applications into independent, scalable services. This allows for independent deployments, rapid innovation, and improved resilience.
Best Practices for Middleware and Integration Architecture Modernization with...Claus Ibsen
This document discusses best practices for middleware and integration architecture modernization using Apache Camel. It provides an overview of Apache Camel, including what it is, how it works through routes, and the different Camel projects. It then covers trends in integration architecture like microservices, cloud native, and serverless. Key aspects of Camel K and Camel Quarkus are summarized. The document concludes with a brief discussion of the Camel Kafka Connector and pointers to additional resources.
Attack monitoring using ElasticSearch Logstash and KibanaPrajal Kulkarni
This document discusses using the ELK stack (Elasticsearch, Logstash, Kibana) for attack monitoring. It provides an overview of each component, describes how to set up ELK and configure Logstash for log collection and parsing. It also demonstrates log forwarding using Logstash Forwarder, and shows how to configure alerts and dashboards in Kibana for attack monitoring. Examples are given for parsing Apache logs and syslog using Grok filters in Logstash.
The document discusses Spring Boot, a framework from the Spring Team that aims to ease the bootstrapping and development of new Spring applications. Spring Boot allows applications to start quickly with very little Spring configuration. It provides some sensible defaults to help developers get started quickly on new projects.
source : http://www.opennaru.com/cloud/msa/
마이크로서비스는 애플리케이션 구축을 위한 아키텍처 기반의 접근 방식입니다. 마이크로서비스를 전통적인 모놀리식(monolithic) 접근 방식과 구별 짓는 기준은 애플리케이션을 핵심 기능으로 세분화하는 방식입니다. 각 기능을 서비스라고 부르며, 독립적으로 구축하고 배포할 수 있습니다. 이는 개별 서비스가 다른 서비스에 부정적 영향을 주지 않으면서 작동(또는 장애가 발생)할 수 있음을 의미합니다.
AWS PrivateLink allows services running within AWS to connect to other services privately without an internet gateway, VPC peering, or EIPs. It creates private connectivity using interface or gateway endpoints within VPCs. Interface endpoints function like a network interface and support security groups, while gateway endpoints add routes to route tables. PrivateLink eliminates public access and simplifies networking management compared to traditional architectures using internet gateways or VPC peering.
This document provides an overview of the architectures and internals of Amazon DocumentDB and MongoDB. It discusses how DocumentDB separates computing and storage layers for improved scalability compared to MongoDB, which couples these layers. It also explains key differences in how each handles data reads/writes, replication, sharding, and other functions. The goal is to help users understand the pros and cons of each for their use cases and needs around performance, scalability and management.
Data Storage Tips for Optimal Spark Performance-(Vida Ha, Databricks)Spark Summit
Vida Ha presented best practices for storing and working with data in files for optimal Spark performance. Some key tips included choosing appropriate file sizes between 64 MB to 1 GB, using splittable compression formats like gzip and Snappy, enforcing schemas for structured formats like Parquet and Avro, and reusing Hadoop libraries to read various file formats. General tips involved controlling output file size through methods like coalesce and repartition, using sc.wholeTextFiles for non-splittable formats, and processing files individually by filename.
VMware ESXi - Intel and Qlogic NIC throughput difference v0.6David Pasek
We are observing different network throughputs on Intel X710 NICs and QLogic FastLinQ QL41xxx NIC. ESXi hardware supports NIC hardware offloading and queueing on 10Gb, 25Gb, 40Gb and 100Gb NIC adapters. Multiple hardware queues per NIC interface (vmnic) and multiple software threads on ESXi VMkernel is depicted and documented in this paper which may or may not be the root cause of the observed problem. The key objective of this document is to clearly document and collect NIC information on two specific Network Adapters and do a comparison to find the difference or at least root cause hypothesis for further troubleshooting.
Maven is a build tool that can manage a project's build process, dependencies, documentation and reporting. It uses a Project Object Model (POM) file to store build configuration and metadata. Maven has advantages over Ant like built-in functionality for common tasks, cross-project reuse, and support for conditional logic. It works by defining the project with a POM file then running goals bound to default phases like compile, test, package to build the project.
[오픈소스컨설팅]클라우드기반U2L마이그레이션 전략 및 고려사항Ji-Woong Choi
Cloud 기반으로 U2C(Unix to Cloud),U2L(Unix to Linux) 마이그레이션에 대한 가이드 라인과 사이징 관련 고려 사항에 대해 설명한 자료입니다.
많은 전환 프로젝트에서 추출된 경험치가 들어가 있으며, 전환별 난이도 및 고려사항이 들어가 있습니다.
This document discusses real-time analytics on streaming data. It describes why real-time data streaming and analytics are important due to the perishable nature of data value over time. It then covers key components of real-time analytics systems including data sources, stream storage, stream ingestion, stream processing, and stream delivery. Finally, it discusses streaming data processing techniques like filtering, enriching, and converting streaming data.
The document discusses optimizing an Apache Pulsar deployment to handle 10 PB of data per day for a large customer. It estimates the initial cluster size needed using different storage options in Google Cloud Platform. It then describes four optimizations made - eliminating the journal, using direct I/O, compression, and improving the C++ client - and recalculates the cluster size after each optimization. The optimized deployment uses 200 VMs each with 24 local SSDs to meet the requirements.
Ceph: Open Source Storage Software Optimizations on Intel® Architecture for C...Odinot Stanislas
Après la petite intro sur le stockage distribué et la description de Ceph, Jian Zhang réalise dans cette présentation quelques benchmarks intéressants : tests séquentiels, tests random et surtout comparaison des résultats avant et après optimisations. Les paramètres de configuration touchés et optimisations (Large page numbers, Omap data sur un disque séparé, ...) apportent au minimum 2x de perf en plus.
This document provides an overview of Spring MVC, the model-view-controller framework for building web applications in Spring. It discusses Spring MVC's request processing workflow including the front controller and application context. It also covers controllers, mapping requests, returning views and data representation. Key topics include RESTful design, annotations like @RequestMapping and return types, and view resolvers for resolving JSP and other view technologies.
Apache Kafka is a high-throughput distributed messaging system that allows for both streaming and offline log processing. It uses Apache Zookeeper for coordination and supports activity stream processing and real-time pub/sub messaging. Kafka bridges the gaps between pure offline log processing and traditional messaging systems by providing features like batching, transactions, persistence, and support for multiple consumers.
This document provides an overview and introduction to Splunk, including:
1. It discusses the challenges of machine data including volume, velocity, variety and variability.
2. Splunk's mission is to make machine data accessible, usable and valuable to everyone.
3. It demonstrates how Splunk can unlock critical insights from machine data sources like order processing, social media, customer service systems and more.
In a world where compute is paramount, it is all too easy to overlook the importance of storage and IO in the performance and optimization of Spark jobs.
1. The document discusses microservices architecture and how Netflix transitioned from a monolithic architecture to microservices. Key aspects discussed include breaking the monolith into many small, independent services that are loosely coupled.
2. Netflix's microservices architecture is composed of hundreds of microservices running on thousands of servers. Each service focuses on doing a small, well-defined piece of work. Services communicate through well-defined APIs and share no code or databases.
3. The document provides examples of how other companies like Samsung and Vingle have also adopted microservices architectures on AWS, breaking monolithic applications into independent, scalable services. This allows for independent deployments, rapid innovation, and improved resilience.
Best Practices for Middleware and Integration Architecture Modernization with...Claus Ibsen
This document discusses best practices for middleware and integration architecture modernization using Apache Camel. It provides an overview of Apache Camel, including what it is, how it works through routes, and the different Camel projects. It then covers trends in integration architecture like microservices, cloud native, and serverless. Key aspects of Camel K and Camel Quarkus are summarized. The document concludes with a brief discussion of the Camel Kafka Connector and pointers to additional resources.
Attack monitoring using ElasticSearch Logstash and KibanaPrajal Kulkarni
This document discusses using the ELK stack (Elasticsearch, Logstash, Kibana) for attack monitoring. It provides an overview of each component, describes how to set up ELK and configure Logstash for log collection and parsing. It also demonstrates log forwarding using Logstash Forwarder, and shows how to configure alerts and dashboards in Kibana for attack monitoring. Examples are given for parsing Apache logs and syslog using Grok filters in Logstash.
The document discusses Spring Boot, a framework from the Spring Team that aims to ease the bootstrapping and development of new Spring applications. Spring Boot allows applications to start quickly with very little Spring configuration. It provides some sensible defaults to help developers get started quickly on new projects.
source : http://www.opennaru.com/cloud/msa/
마이크로서비스는 애플리케이션 구축을 위한 아키텍처 기반의 접근 방식입니다. 마이크로서비스를 전통적인 모놀리식(monolithic) 접근 방식과 구별 짓는 기준은 애플리케이션을 핵심 기능으로 세분화하는 방식입니다. 각 기능을 서비스라고 부르며, 독립적으로 구축하고 배포할 수 있습니다. 이는 개별 서비스가 다른 서비스에 부정적 영향을 주지 않으면서 작동(또는 장애가 발생)할 수 있음을 의미합니다.
AWS PrivateLink allows services running within AWS to connect to other services privately without an internet gateway, VPC peering, or EIPs. It creates private connectivity using interface or gateway endpoints within VPCs. Interface endpoints function like a network interface and support security groups, while gateway endpoints add routes to route tables. PrivateLink eliminates public access and simplifies networking management compared to traditional architectures using internet gateways or VPC peering.
This document provides an overview of the architectures and internals of Amazon DocumentDB and MongoDB. It discusses how DocumentDB separates computing and storage layers for improved scalability compared to MongoDB, which couples these layers. It also explains key differences in how each handles data reads/writes, replication, sharding, and other functions. The goal is to help users understand the pros and cons of each for their use cases and needs around performance, scalability and management.
Data Storage Tips for Optimal Spark Performance-(Vida Ha, Databricks)Spark Summit
Vida Ha presented best practices for storing and working with data in files for optimal Spark performance. Some key tips included choosing appropriate file sizes between 64 MB to 1 GB, using splittable compression formats like gzip and Snappy, enforcing schemas for structured formats like Parquet and Avro, and reusing Hadoop libraries to read various file formats. General tips involved controlling output file size through methods like coalesce and repartition, using sc.wholeTextFiles for non-splittable formats, and processing files individually by filename.
VMware ESXi - Intel and Qlogic NIC throughput difference v0.6David Pasek
We are observing different network throughputs on Intel X710 NICs and QLogic FastLinQ QL41xxx NIC. ESXi hardware supports NIC hardware offloading and queueing on 10Gb, 25Gb, 40Gb and 100Gb NIC adapters. Multiple hardware queues per NIC interface (vmnic) and multiple software threads on ESXi VMkernel is depicted and documented in this paper which may or may not be the root cause of the observed problem. The key objective of this document is to clearly document and collect NIC information on two specific Network Adapters and do a comparison to find the difference or at least root cause hypothesis for further troubleshooting.
Maven is a build tool that can manage a project's build process, dependencies, documentation and reporting. It uses a Project Object Model (POM) file to store build configuration and metadata. Maven has advantages over Ant like built-in functionality for common tasks, cross-project reuse, and support for conditional logic. It works by defining the project with a POM file then running goals bound to default phases like compile, test, package to build the project.
[오픈소스컨설팅]클라우드기반U2L마이그레이션 전략 및 고려사항Ji-Woong Choi
Cloud 기반으로 U2C(Unix to Cloud),U2L(Unix to Linux) 마이그레이션에 대한 가이드 라인과 사이징 관련 고려 사항에 대해 설명한 자료입니다.
많은 전환 프로젝트에서 추출된 경험치가 들어가 있으며, 전환별 난이도 및 고려사항이 들어가 있습니다.
No trade-offs: 안전하고, 빠르고, 안정적인 네트워크 구축하기 / Building secure, fast, and reliabl...Jean Ryu
Cloudflare Magic Transit 제품에 관한 웨비나였습니다. Magic Transit은 온프레미스 데이터센터의 보안 강화를 위해 도입하실 수 있는 디도스 스크러빙 제품입니다. 거꾸로 가는 남자(?)도 아니고 2020년이 다 됐는데 이 회사는 클라우드 솔루션부터 시작했으면서 이 시점에 온프레미스용 제품을 신규 출시하십니까...라는 의문이 드셨다면 정상입니다. 웨비나에서 대답해 드립니다.
하시코프와 함께하는 알리바바 클라우드 DevSecOps 뽀개기 E01 SecOpsAlibaba Cloud Korea
해당 PPT에 대한 웨비나를 시청해보세요: https://youtu.be/Coty0etlGHU
지난 10월 6일 하시코프와 알리바바 클라우드의 조인트 웨비나 E01에서는 하시코프와 알리바바 클라우드의 인프라 보안에 대한 논의가 진행되었습니다.
하시코프에서는 Packer Image, Terraform Provisioning, Vault SSH를 알리바바 클라우드에서는 Alibaba Cloud의 인프라 보안, WAF, Anti-DDoS, Cloud Firewall, Security Center - Infra 를 다룹니다.
한대희 Web proxy_개발_2006년11월_pas_ktfDaehee Han
스마트폰이 대중화되기 직전까지 KT이동통신(KTF)의 모든 단말기가 인터넷 콘텐트 접속시에 거쳐가는 Web Proxy (PAS라 불림)를 바닥부터 새로 개발한 개발 기록. multi thread 기반으로 동작. 한 thread에서 여러 단말(client)처리. Multi-connection per thread. ACE framework사용. Reactor패턴 사용. 부하(동시접속 단말 개수)에 따라 reactor thread 개수를 동적으로 자동 조절하는 pool 방식 구현. 설계를 다시하고 밑바닥부터 새로 만듦. 200TPS 의 기존 성능을 1,000 TPS 로 올림. 5~6번의 deploy 작업 끝에 memory leak 문제 등 모든 문제 해결하고, 30일 넘게 운영해도 죽지 않는 서버로 구현함. 2006년.
Cloud-Native Architecture
MSA(Micro Service Architecture)
MDA(Micro Data Architecture)
MIA(MIcro Inference Architecture)
MSA-Service Mesh
MDA-Data Mesh
MIA-AI Inference Mesh
Kubernetes
Container
Kubeflow
Volcano
Apache Ynikorn
ChatGPT
AGI(Artificial General Intelligence)
ASI(Artificial Specialized Intelligence)
초-전환시대
초-연결시대
SQream GPU DBMS
Cloud와 Cloud Native의 목표는.. 왜? 어떻게? 뭐가 좋아지나...
1. (왜) 가속화된 초-전환, 초-연결 IT 환경변화에 대비하기 위해서
2. (어떻게-H/W) IT H/W 부분은 IaaS 서비스화하여
점유된, Over Subscription된 H/W(Server, Network, Storage)들 모아서 Pool화하고, 가상화기술을 통해 Tenant로 자원들을 분리해 서비스화해 제공하고
필요시 적시에 Pool의 가상H/W를 제공하고, 상황에 따라 확장・축소(Scale in/out, up/down)하면서, 축소된 자원을 다른 요청들을 위해 빠르게 재-할당하는 유연성을 제공하고
3. (어떻게-S/W) S/W 부문도
PaaS, SaaS 적극 활용으로 App.개발 시간을 단축하고
App.분야인 기존 MACRO Service Architecture형 Monolith Architecture(Web-WAS-DB)를 작게 쪼개서 변화에 빠르게 적응할 수 있는 MSA(Micro Service Architecture)로 변경하여 Service Mesh형으로 관리하고
Data분야도 Data Warehouse, DataLake(Bigdata), LakeHouse등 기존 MACRO Data Architecture를 MSA형식으로 MDA(Micro Data Architecture)로 전환 후 Data Mesh형태로 관리하고,
AI로 동적프로그램 생성하여 App.개발시간 단축하고, AI분야도 초-거대 AI구현(MACRO)보다는 작은|특화된 Deep Learning Network(Model)들로 작게 쪼개서 MIA(Micro Inference Architecture)로 비지니스 환경에 적용하고 Inference Mesh형태로 관리하는 시스템으로 전환하고
4. (어떻게-조직) 조직구조도 CI/CD형 DevOps환경, 데이타,트랜잭션중심업무중심, 기술중심 문제해결중심, 직능중심조직직무중심조직으로 전환하면
5. (좋아지는 것) 초-전환, 초-연결 환경에 빠르고, 지속적으로 적응할 수 IT as a Product 환경을 구현하는 것
11. Hybrid 데이터센터
사용자를 최적의 클라우드나 데이터센터로 유도
• 어플리케이션의 상태를 끊임없이 모니터링
• 비지니스 로직을 기반으로 가용한 어플리케이션쪽으로 트래픽 유도
• IP 기반 위치 인식 및 IP 차단
데이터센터
LDNS
DMZ
클라우드 환경
클라우드DMZ
BIG-IP BIG-IP
Cloud
Hosted
Apps
BIG-IP
Global Traffic Manager
BIG-IP
Local Traffic Manager
Internet