Serverless und Event-Driven Architecture

Nov 12, 2023Download as PPTX, PDF0 likes95 views

Stirbt Serverless den Kubernetes-Tod bevor es richtig geboren wurde? Es gibt zumindest eine Domäne, in der Serverless definitiv lebendig und munter ist - das Internet der Dinge (IoT). In den letzten sieben Jahren wurden IoT-Systeme vermehrt unter Verwendung von Platform as a Service (PaaS) und Serverless-Diensten der Public Cloud Provider entwickelt. Dabei kommt die Event-Driven Architecture zum Einsatz, ein Muster, das sich sowohl für Serverless als auch für die asynchrone Natur von IoT-Systemen bestens eignet. Aber wie sehen solche Architekturen konkret aus? Warum sind gerade IoT-Anwendungen prädestiniert für Serverless? Und welche wertvollen Erkenntnisse können wir aus diesem Bereich für andere Anwendungsdomänen gewinnen? In dieser Session werden wir diese Fragen anhand von Beispielen aus der Azure-Welt genauer unter die Lupe nehmen.

Serverless und Event-
Driven Architecture
Roland Krummenacher, Swisscom B2B Cloud Advisory

3
scaling not scaling
44 %
not scaling
Pets or Cattle?
44% of the AKS Node Pools for a major
Azure client are configured to operate without
auto-scaling, meaning they maintain a fixed
number of virtual machines running 24/7.

5
Serverless is more than just FaaS
Serverless is an entire stack of services that can
respond to specific events or requests, and scale to
zero when no longer in use—and for which
provisioning, management and billing are handled
by the cloud provider and invisible to developers.
Function-as-a-service, or FaaS, is a cloud computing
service that enables developers to run code or
containers in response to specific events or requests,
without specifying or managing the infrastructure
required to run to code.
FaaS is the compute model central to serverless, and
the two terms are often used interchangeably. But
serverless is much more than FaaS.
Source: https://www.ibm.com/topics/serverless
Azure Functions
Azure Logic Apps
Azure App Service (with Consumption Plan)
Azure Cosmos DB
Azure Event Grid
Azure SQL Database Serverless
Azure SignalR Service
Azure Cognitive Services (some features can be used in a serverless
manner)
Azure Key Vault
Azure Search
Azure API Management (Consumption Tier)
Azure Notification Hubs
Azure IoT Hub
Azure Machine Learning Service (in certain serverless configurations)
Azure Container Instances (ACI)
Azure Durable Functions
Azure Stream Analytics
Azure Cognitive Search
Azure Data Lake Analytics
Azure Data Factory (V2)
Azure DevOps (Azure Pipelines)
Azure Service Bus (Auto-Inflate feature for scaling)
Azure WebJobs
Azure App Configuration
Azure Static Web Apps
Azure API Apps

6
Serverless scenario: File upload and
document processing
Real-time file processing for the healthcare
scenario: The solution securely uploads patient
records as PDF files. The solution then
decomposes the data, by processes it using OCR
detection, and it adds the data to a database for
easy queries.
Source: https://learn.microsoft.com/en-us/azure/architecture/serverless-quest/reference-
architectures

8
Serverless scenario: AWS Connected Vehicle – Reference Architecture
Source: https://docs.aws.amazon.com/architecture-diagrams/latest/aws-connected-vehicle/aws-connected-

10
Challenges and solution approaches
What I have learnt from
building and running
serverless and event-
driven systems on Azure.
4
3
1 2
Security
Adding Infrastructure to re-establish
Perimeter-Security vs. implementing
SaaS-style Zero-Trust.
Operations
Complexity has moved from code to
relationships between the
components. Monitoring serverless
and event-driven systems end-to-
end is complex.
Cloud Automation
For serverless, cloud automation
(aka. IaC) needs to focus on the
application, the relationships
between component and the
configuration instead of the
infrastructure.
Architecture
Reconsider traditional Software
Design Patterns for serverless
architecture, e.g. Messaging (Pipes-
and-Filters), Hexagonal Architecture
and CQRS.

11
Roland Krummenacher
Cloud Advisor
www.linkedin.com/in/rolandkrummenacher
Questions and
Answers

This presentation delves into the journey of transitioning an on-premises monolithic real estate application to a fully serverless, event-driven microservices stack on AWS. Explore the compelling business drivers that necessitated this shift, from traffic demands and team scalability to architectural flexibility. We'll unpack the challenges, from navigating many service choices to the cost implications. At the same time, we'll celebrate the undeniable advantages: improved security through AWS's shared responsibility model, the economic flexibility of a pay-as-you-go approach, and remarkable reliability and scalability.

Azure SQL Database Managed InstanceJames Serra

Azure SQL Database Managed Instance is a new flavor of Azure SQL Database that is a game changer. It offers near-complete SQL Server compatibility and network isolation to easily lift and shift databases to Azure (you can literally backup an on-premise database and restore it into a Azure SQL Database Managed Instance). Think of it as an enhancement to Azure SQL Database that is built on the same PaaS infrastructure and maintains all it's features (i.e. active geo-replication, high availability, automatic backups, database advisor, threat detection, intelligent insights, vulnerability assessment, etc) but adds support for databases up to 35TB, VNET, SQL Agent, cross-database querying, replication, etc. So, you can migrate your databases from on-prem to Azure with very little migration effort which is a big improvement from the current Singleton or Elastic Pool flavors which can require substantial changes.

Well Architected Framework - Data Craig Milroy

Putting the Ops in DataOps: Orchestrate the Flow of Data Across Data PipelinesDATAVERSITY

With the aid of any number of data management and processing tools, data flows through multiple on-prem and cloud storage locations before it’s delivered to business users. As a result, IT teams — including IT Ops, DataOps, and DevOps — are often overwhelmed by the complexity of creating a reliable data pipeline that includes the automation and observability they require. The answer to this widespread problem is a centralized data pipeline orchestration solution. Join Stonebranch’s Scott Davis, Global Vice President and Ravi Murugesan, Sr. Solution Engineer to learn how DataOps teams orchestrate their end-to-end data pipelines with a platform approach to managing automation. Key Learnings: - Discover how to orchestrate data pipelines across a hybrid IT environment (on-prem and cloud) - Find out how DataOps teams are empowered with event-based triggers for real-time data flow - See examples of reports, dashboards, and proactive alerts designed to help you reliably keep data flowing through your business — with the observability you require - Discover how to replace clunky legacy approaches to streaming data in a multi-cloud environment - See what’s possible with the Stonebranch Universal Automation Center (UAC)

DataOps , cbuswaw April '23Jason Packer

The document provides an overview of DataOps and continuous integration/continuous delivery (CI/CD) practices for data management. It discusses: - DevOps principles like automation, collaboration and agility can be applied to data management through a DataOps approach. - CI/CD practices allow for data products and analytics to be developed, tested and released continuously through an automated pipeline. This includes orchestration of the data pipeline, testing, and monitoring. - Adopting a DataOps approach with CI/CD enables faster delivery of data and analytics, more efficient and compliant data pipelines, improved productivity, and better business outcomes through data-driven decisions.

Data Profiling: The First Step to Big Data QualityPrecisely

Big data offers the promise of a data-driven business model generating new revenue and competitive advantage fueled by new business insights, AI, and machine learning. Yet without high quality data that provides trust, confidence, and understanding, business leaders continue to rely on gut instinct to drive business decisions. The critical foundation and first step to deliver high quality data in support of a data-driven view that truly leverages the value of big data is data profiling - a proven capability to analyze the actual data content and help you understand what's really there. View this webinar on-demand to learn five core concepts to effectively apply data profiling to your big data, assess and communicate the quality issues, and take the first step to big data quality and a data-driven business.

Azure Security and ManagementAllen Brokken

1.1 Data Security Presentation.pdfChunLei(peter) Che

This document provides an overview of key concepts in data security, including data types and classification, data lifecycle management, and relevant laws and industry regulations. It discusses common data types like PII, PHI, financial information and intellectual property. It also explains the process of data classification and the data lifecycle. Finally, it outlines several important regulations and standards that govern data security, such as GDPR, HIPAA, PCI DSS, ISO 27001, and SOX.

Data Governance by stealth v0.0.2Christopher Bradley

The document provides an introduction and background on Christopher Bradley, an expert in data governance. It then discusses data governance, defining it as the design and execution of standards and policies covering the design and operation of a management system to assure that data delivers value and is not a cost, as well as who can do what to the organization. The document lists Bradley's recent presentations and publications on topics related to data governance, data modeling, master data management and information management.

Cloud 101 - What is the Cloud?RapidScale

The document discusses cloud computing and its key features. It covers the three main types of cloud services - Software as a Service (SaaS), Platform as a Service (PaaS), and Infrastructure as a Service (IaaS). It also discusses server virtualization, public and private cloud options, and the business benefits of scalability, disaster recovery, remote access, management efficiencies, and the shift from capital to operational expenditures. The cloud allows companies to access computing resources over the internet in a cost-effective and flexible manner compared to maintaining their own on-premise infrastructure.

Azure reference architecturesMasashi Narumoto

This document provides guidance on choosing the right architectures and technologies for Azure solutions. It discusses architecture styles like n-tier, microservices, CQRS, and event-driven architectures. For each style, it covers the business and technical factors to consider, common patterns, and example component mappings to Azure services. It also summarizes reference architectures for big data and IoT solutions and compares approaches like lambda and kappa architectures. The overall document aims to help readers select the optimal architecture and technologies for their specific domain and workload.

Azure Overview Arcrajramab

The document discusses challenges facing today's enterprises such as cutting costs, driving value with tight budgets, maintaining security while increasing access, and finding the right transformative capabilities. It then discusses challenges in building applications related to scaling, availability, and costs. The remainder summarizes Microsoft's Windows Azure cloud computing platform, how it addresses these challenges, example use cases, and pricing models.

Desktop virtualization customer presentationNuno Alves

Microsoft AzurePavel Ryabov

Microsoft Azure is a cloud computing platform offering various services including computing, storage, databases, and web hosting. Azure operates out of 32 regions worldwide. It offers three main service models: Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS). App Service is a PaaS offering that allows users to build, deploy, and manage web applications, REST APIs, mobile back ends, and web jobs. Traffic Manager helps distribute user traffic between different endpoints for availability and performance. It uses DNS to route clients to the optimal endpoint based on factors like location and health.

Azure HybidThomas Treml

The document discusses Microsoft's Azure multi-cloud and hybrid solutions. It describes how organizations' IT environments are becoming more complex with diverse infrastructure across locations, datacenters, edge devices, and public clouds. It introduces Azure Arc which allows organizations to connect and manage hybrid infrastructure as well as run Azure services across platforms. It provides an overview of Azure's capabilities for multi-cloud, hybrid, and edge computing including Azure Stack, Azure Arc, and security and management solutions for hybrid environments.

Azure: PaaS or IaaSShahed Chowdhuri

Modern Data architecture DesignKujambu Murugesan

The document discusses modern data architectures. It presents conceptual models for data ingestion, storage, processing, and insights/actions. It compares traditional vs modern architectures. The modern architecture uses a data lake for storage and allows for on-demand analysis. It provides an example of how this could be implemented on Microsoft Azure using services like Azure Data Lake Storage, Azure Data Bricks, and Azure Data Warehouse. It also outlines common data management functions such as data governance, architecture, development, operations, and security.

Windows Azure Active DirectoryKrunal Trivedi

Network Observability: Delivering Actionable Insights to Network OperationsEnterprise Management Associates

EMA’s market research, “Network Observability: Delivering Actionable Insights to Network Operations,” establishes a roadmap for how vendors should improve their products to deliver deep visibility and actionable insights. More importantly, this research will help IT buyers understand what network management vendors mean when they talk about network observability and how they can turn that understanding into NetOps success.

Introduction to Data EngineeringDurga Gadiraju

As part of this session, I will be giving an introduction to Data Engineering and Big Data. It covers up to date trends. * Introduction to Data Engineering * Role of Big Data in Data Engineering * Key Skills related to Data Engineering * Role of Big Data in Data Engineering * Overview of Data Engineering Certifications * Free Content and ITVersity Paid Resources Don't worry if you miss the video - you can click on the below link to go through the video after the schedule. https://youtu.be/dj565kgP1Ss * Upcoming Live Session - Overview of Big Data Certifications (Spark Based) - https://www.meetup.com/itversityin/events/271739702/ Relevant Playlists: * Apache Spark using Python for Certifications - https://www.youtube.com/playlist?list=PLf0swTFhTI8rMmW7GZv1-z4iu_-TAv3bi * Free Data Engineering Bootcamp - https://www.youtube.com/playlist?list=PLf0swTFhTI8pBe2Vr2neQV7shh9Rus8rl * Join our Meetup group - https://www.meetup.com/itversityin/ * Enroll for our labs - https://labs.itversity.com/plans * Subscribe to our YouTube Channel for Videos - http://youtube.com/itversityin/?sub_confirmation=1 * Access Content via our GitHub - https://github.com/dgadiraju/itversity-books * Lab and Content Support using Slack

Active directory deshvikas

Active Directory is a centralized hierarchical directory database that contains information about all user accounts and shared network resources. It provides user logon authentication services and organizes and manages user accounts, computers, groups and network resources. Active Directory enables authorized users to easily locate network resources. It features include fully integrated security, easy administration using group policy, scalability to large networks, and flexibility through features like cross-forest trusts and site-to-site replication.

Hyper-Converged Infrastructure: ConceptsNick Scuola

This document provides an overview of various appliance and software offerings from different vendors. It lists vendors that provide niche offerings for specific uses like VDI or video surveillance. It also outlines standard appliance options from vendors like Dell, HP, and Lenovo that are optimized for Hyper-V or KVM, as well as software-defined storage solutions and reference architectures from vendors including VMware, HP, Cisco, and Lenovo. Reference architectures provide validated designs and support and some solutions include subscription models for software. The document covers a wide range of preconfigured hardware and software options from major IT vendors.

DAS Slides: Enterprise Architecture vs. Data ArchitectureDATAVERSITY

Enterprise Architecture (EA) provides a visual blueprint of the organization, and shows key inter-relationships between data, process, applications, and more. By abstracting these assets in a graphical view, it’s possible to see key interrelationships, particularly as they relate to data and its business impact across the organization. Join us for a discussion on how Data Architecture is a key component of an overall enterprise architecture for enhanced business value and success.

AIOps: Your DevOps Co-PilotDevOps.com

Azure active directoryRaju Kumar

Azure Active Directory (AAD) is a multi-tenant cloud-based identity and access management service. It provides features like multi-factor authentication, device registration, self-service password management, role-based access control, and application usage monitoring. AAD is better suited than on-premises Active Directory for managing users across multiple platforms and cloud applications/servers. It maintains a central directory for users and applications in Microsoft cloud services like Office 365. AAD supports two types of user accounts - Microsoft personal accounts for private use and work accounts managed by an AAD administrator for organizational access.

The Microsoft Well Architected Framework For Data AnalyticsStephanie Locke

With more than a decade of organizations running large data & analytics workloads in the cloud, Microsoft have extended their architecture framework to provide best practices and guidance for businesses. In this session, we’ll introduce the 'Well Architected Framework', go into detail about effective data architectures, and give you concrete next steps you can take whether you already have a cloud data architecture or are planning your first implementation.

Databricks Platform.pptxAlex Ivy

The document provides an overview of the Databricks platform, which offers a unified environment for data engineering, analytics, and AI. It describes how Databricks addresses the complexity of managing data across siloed systems by providing a single "data lakehouse" platform where all data and analytics workloads can be run. Key features highlighted include Delta Lake for ACID transactions on data lakes, auto loader for streaming data ingestion, notebooks for interactive coding, and governance tools to securely share and catalog data and models.

Qlik-Sense-Product-Presentation.compressed.pdfhavoc2003

Microsoft Azure Technical Overviewgjuljo

Sky High With AzureClint Edmonson

Join us for a deep dive into Windows Azure. We’ll start with a developer-focused overview of this brave new platform and the cloud computing services that can be used either together or independently to build amazing applications. As the day unfolds, we’ll explore data storage, SQL Azure™, and the basics of deployment with Windows Azure. Register today for these free, live sessions in your local area.

More Related Content

What's hot (20)

Data Governance by stealth v0.0.2Christopher Bradley

Cloud 101 - What is the Cloud?RapidScale

Azure reference architecturesMasashi Narumoto

Azure Overview Arcrajramab

Desktop virtualization customer presentationNuno Alves

Microsoft AzurePavel Ryabov

Azure HybidThomas Treml

Azure: PaaS or IaaSShahed Chowdhuri

Modern Data architecture DesignKujambu Murugesan

Windows Azure Active DirectoryKrunal Trivedi

Network Observability: Delivering Actionable Insights to Network OperationsEnterprise Management Associates

Introduction to Data EngineeringDurga Gadiraju

Active directory deshvikas

Hyper-Converged Infrastructure: ConceptsNick Scuola

DAS Slides: Enterprise Architecture vs. Data ArchitectureDATAVERSITY

AIOps: Your DevOps Co-PilotDevOps.com

Azure active directoryRaju Kumar

The Microsoft Well Architected Framework For Data AnalyticsStephanie Locke

Databricks Platform.pptxAlex Ivy

Qlik-Sense-Product-Presentation.compressed.pdfhavoc2003

Data Governance by stealth v0.0.2Christopher Bradley

Cloud 101 - What is the Cloud?RapidScale

Azure reference architecturesMasashi Narumoto

Azure Overview Arcrajramab

Desktop virtualization customer presentationNuno Alves

Microsoft AzurePavel Ryabov

Azure HybidThomas Treml

Azure: PaaS or IaaSShahed Chowdhuri

Modern Data architecture DesignKujambu Murugesan

Windows Azure Active DirectoryKrunal Trivedi

Network Observability: Delivering Actionable Insights to Network OperationsEnterprise Management Associates

Introduction to Data EngineeringDurga Gadiraju

Active directory deshvikas

Hyper-Converged Infrastructure: ConceptsNick Scuola

DAS Slides: Enterprise Architecture vs. Data ArchitectureDATAVERSITY

AIOps: Your DevOps Co-PilotDevOps.com

Azure active directoryRaju Kumar

The Microsoft Well Architected Framework For Data AnalyticsStephanie Locke

Databricks Platform.pptxAlex Ivy

Qlik-Sense-Product-Presentation.compressed.pdfhavoc2003

Similar to Serverless und Event-Driven Architecture (20)

Microsoft Azure Technical Overviewgjuljo

Sky High With AzureClint Edmonson

Azure Data.pptxFedoRam1

This document provides an overview of a course on implementing a modern data platform architecture using Azure services. The course objectives are to understand cloud and big data concepts, the role of Azure data services in a modern data platform, and how to implement a reference architecture using Azure data services. The course will provide an ARM template for a data platform solution that can address most data challenges.

Cloud application architecture with sql azure and windows azureEduardo Castro

Azure diario de abordoJosé Ángel Bolaño Rucabado

Leveraging azure and cello for multi tenancyTechcello

This document discusses how Azure and Cello can be used together to deliver highly scalable multi-tenant SaaS solutions. Cello provides a framework and tools for building, deploying, and managing SaaS applications, while Azure provides scalable infrastructure. When used together, Cello handles application management and Azure handles infrastructure scaling. This allows applications to easily scale on Azure without additional coding. The document provides an overview of multi-tenancy models, why scalability is important, and how applications can be deployed and scaled on Azure using Cello.

Microsoft AzureMohab El-Shishtawy

Microsoft Windows Azure is a cloud computing platform that provides infrastructure as a service, platform as a service and software as a service. It allows developers to build and host applications and services through Microsoft's global network of data centers. Key components of Windows Azure include Web Roles, Worker Roles, Storage, SQL Azure database, Cloud Fabric for automatic scaling and failover, and development tools to facilitate building applications for the cloud. The document provides an overview of these services and components and how developers can get started using Windows Azure.

Azure fundamental -IntroductionManishK55

This document provides an overview of cloud computing concepts and Azure cloud services. It discusses cloud service models including infrastructure as a service (IaaS), platform as a service (PaaS), and software as a service (SaaS). It introduces Azure, the Microsoft cloud computing platform, and key Azure services like Azure Storage, Azure Portal, Azure Accounts, Azure Data Factory, and Azure Data Flow. Azure Data Factory allows building data integration solutions using activities, linked services, datasets and triggers without writing code. Azure Data Flow enables visually designing data transformations using a Spark optimizer without code.

Azure lessonsMufaddal Haidermota

The document provides an overview of Microsoft Azure cloud computing services. It describes Azure as a platform for building and hosting applications using infrastructure, platform and functions as a service. Key Azure services discussed include virtual machines, app service, storage, databases and content delivery network. The document also outlines Azure security measures and lists some common usage scenarios for Azure.

Ukfs Snr Dev Arch Forum Pres2 StAllyWick

This document provides an overview of the Windows Azure platform, including its core services and capabilities. The key points are: 1. Windows Azure is a cloud services platform that provides scalable compute and storage services hosted in Microsoft data centers worldwide. 2. It offers web and worker roles that can scale horizontally by replicating instances as needed to handle increasing load. Storage services allow scaling to store large amounts of data cost-effectively. 3. SQL Azure extends SQL Server capabilities to the cloud, providing a scalable and reliable database service with high availability and data replication.

SQL Azure the database in the cloud Eduardo Castro

The document provides an overview of SQL Azure, a relational database service available on the Microsoft Azure platform. Key points include: - SQL Azure allows users to build applications that use a relational database in the cloud without having to manage infrastructure. - It is based on SQL Server and provides a familiar programming model, but is designed for the cloud with high availability and scalability. - The service has limitations on database size and does not provide built-in sharding capabilities, so applications need to implement custom partitioning logic for large datasets. - Future improvements may address limitations and open up new scenarios and opportunities through integration with other Azure services. SQL Azure is part of Microsoft's broader strategy around cloud-

Azure Developer Roadmap By Scholarhat PDFScholarhat

Azure Domains.pptxVinceRichard1

Wide variety of services that cloud architectures can manage using different approaches for example Azure Active Directory Domain Services (Azure AD DS), part of Microsoft Entra, enables you to use managed domain services—such as Windows Domain Join, group policy, LDAP, and Kerberos authentication—without having to deploy, manage, or patch domain controllers. When managed domain services are enabled for virtual machines and directory-aware applications deployed in Azure with a click of a button. Reduce operational and maintenance costs associated with managing identity infrastructure for your virtual machines and legacy applications. Managed domain allows legacy applications to be run in the cloud through an easier means of migrating on-premises apps to a managed domain. Furthermore it allows streamline management of all applications from your legacy, directory-aware apps alongside your modern cloud apps with a single identity solution. It allow the achievement of highly available service with the use of Azure AD Domain Services which can includes multiple domain controllers to provide high availability for your managed domain. This approach will ensure business continuity and guaranteed service uptime and resilience to failures.

Azure intelligent edge solutions overviewCenk Ersoy

The document discusses the intelligent edge and hybrid cloud computing. It defines the intelligent edge as where data is created and processed outside traditional centralized data centers. It predicts that by 2025, 75% of enterprise data will be created and processed at the edge. It then provides an overview of different Azure products and solutions for intelligent edge computing, including Azure Sphere, IoT Edge, Stack Edge, and Stack Hub. It discusses how these products bring cloud services and capabilities to the edge through appliances, gateways, and on-premises servers to enable hybrid cloud solutions.

Leveraging azure and cello for delivering highly scalable multi tenantkanimozhin

The document discusses leveraging Azure and Cello to build scalable multi-tenant SaaS solutions. Cello provides a framework for developing SaaS applications that can be deployed on Azure. It includes features for engineering, operations, and subscription management. Using Cello with Azure allows applications to benefit from Cello's SaaS capabilities and Azure's infrastructure management and scalability. ISVs can build applications that can easily scale on Azure to serve growing numbers of customers.

Azure Day 2.pptxmasbulosoke

Modern dataintegration azuredatafactory_ssisGaurav Malhotra

This session focuses on the needs of the data integrator and data engineer whether that be for data warehousing & BI, advanced analytics of data for SaaS applications. We walk through a comprehensive set of new additions to Azure Data Factory and SSIS for moving and integrating data across on-premises and cloud. Topics and examples will include simple, scalable and reliable data pipelines in ADF using a serverless, parallel data movement service to/from the cloud, provisioning of Azure-SSIS Integration Runtime (IR) – dedicated servers for lifting & shifting SSIS packages to cloud– and its customization with your own/3rd party extensions, the execution of SSIS packages as first-class activities in ADF pipelines and their combination with other ADF activities to create modern ETL/ELT workflows all through the new code-free experience.

Azure Cloud ServicesKajal Kathrotiya

Microsoft Azure is a cloud computing platform offering a range of services including compute, analytics, storage, networking, and more. Users can choose services to develop and scale applications in the public cloud. Key Azure products include Virtual Machines, App Service, SQL Database, Storage, Backup, API Management, Cosmos DB, Machine Learning, Security and Compliance, SQL Data Warehouse, Notification Hubs, IOT Hub, and more. Azure provides scalable, reliable cloud services to build applications across platforms and devices.

New capabilities for modern data integration in the cloudMicrosoft Tech Community

This document provides an overview of how to configure an Azure-SSIS Integration Runtime (IR) within Azure Data Factory. It describes the steps to name and describe the IR, select its location, node size, cluster size, edition/license, and connect it to an existing Azure SQL Database or Managed Instance that will host the SSIS catalog database. It also covers options for using Azure Active Directory authentication or bringing your own Azure SQL credentials. The document guides users through the entire configuration process for an Azure-SSIS IR within Azure Data Factory.

New capabilities for modern data integration in the cloudGaurav Malhotra

The document describes how Azure Data Factory can be used to orchestrate data pipelines and execute SSIS packages in the cloud. It provides an overview of key capabilities like lifting and shifting existing SSIS packages to Azure, using various Azure services like SQL Data Warehouse and Databricks in data pipelines, and orchestrating workflows. It also discusses how Azure Data Factory allows scheduling SSIS package execution in a managed cloud environment using familiar SSIS authoring and monitoring tools.

Microsoft Azure Technical Overviewgjuljo

Sky High With AzureClint Edmonson

Azure Data.pptxFedoRam1

Cloud application architecture with sql azure and windows azureEduardo Castro

Azure diario de abordoJosé Ángel Bolaño Rucabado

Leveraging azure and cello for multi tenancyTechcello

Microsoft AzureMohab El-Shishtawy

Azure fundamental -IntroductionManishK55

Azure lessonsMufaddal Haidermota

Ukfs Snr Dev Arch Forum Pres2 StAllyWick

SQL Azure the database in the cloud Eduardo Castro

Azure Developer Roadmap By Scholarhat PDFScholarhat

Azure Domains.pptxVinceRichard1

Azure intelligent edge solutions overviewCenk Ersoy

Leveraging azure and cello for delivering highly scalable multi tenantkanimozhin

Azure Day 2.pptxmasbulosoke

Modern dataintegration azuredatafactory_ssisGaurav Malhotra

Azure Cloud ServicesKajal Kathrotiya

New capabilities for modern data integration in the cloudMicrosoft Tech Community

New capabilities for modern data integration in the cloudGaurav Malhotra

More from BATbern (20)

Bridging the Gap from Telco to Techco with Agile ArchitectureBATbern

The Telco industry is undergoing a major IT transformation, shifting from Telco to Techco. This shift is driven by a breakneck pace of technological change that traditional architectures simply cannot keep up with. In my presentation, I will explore the profound impact of digitization on Telco architecture. Using examples from Swisscom's Network Division, I will show why Agile Architecture is crucial for cutting complexity, accelerating time-to-market, and sparking innovation within the organization. This approach isn't just strategic; it's vital for the future success of our industry. Join me to uncover how Swisscom is navigating this transformation and what lessons can be applied to your organization.

How can TWINT be agile in an inert ecosystem?BATbern

The Swiss banking Ecosystem works as a trustworthy and reliable basis for personal and business-related financial transactions. A fintech in this field is dealing with large and globally active players who can’t (and likely shouldn’t) adapt as fast and as agile as the fintech can, and must, to secure its position in the market. In this presentation I will elaborate in which ways TWINT strives for agility and innovation in an otherwise slow-moving and slowly innovating ecosystem.

Agile Architektur und Transformation @PostfinanceBATbern

Die Architektur spielt eine zentrale Rolle in der agilen Transformation und strategischen Ausrichtung in der PostFinance. Mit diesem Vortrag, soll aufgezeigt werden, wie Architektur-Management-Prozesse gestaltet werden müssen, um flexibel auf Veränderungen zu reagieren, während strategische Leitplanken erhalten bleiben. Zudem werden der Aufbau und die Entwicklung einer agilen Architektur sowie deren Nutzen und Herausforderungen dargestellt. Ein Praxisbeispiel veranschaulicht Herausforderungen und Nutzen agiler Architektur. Abschliessend werden Lessons Learned reflektiert, um Strategien für eine zukunftsfähige Architektur abzuleiten.

BATbern54 Build & Run on the same platform, embracing Platform Engineering & ...BATbern

Michele Monforte & Fabian Nicollier, ELCA: Embracing DevOps requires a fundamental shift in team mindset, breaking down silos within the organization. Our Platform Engineering team goes beyond simply providing a platform; we actively support development teams in adopting DevOps principles, encouraging accountability for both the Build and Run phases of their projects. In an organization primarily composed of development teams, leveraging Cloud Provider services has been key factor: New projects can be quickly bootstrapped. Costs are directly allocated to the projects, and charged back accordingly. The underlying infrastructure is robust, removing the need to grasp complex, low-level concepts. There is a wealth of mature technologies available to automate and scale as needed. In this session, we will showcase how we’ve built our platform to offer a self-service portal that enables teams to be operational in less than a day. We will also demonstrate how our Platform Engineering team collaborates with development teams to empower them by fully embracing DevOps practices.

BATbern54 Plattform-Engineering für digitale Versicherungsprodukte: «Joint Ap...BATbern

BATbern54 Plattform-Engineering für digitale Versicherungsprodukte: Erfahrung...BATbern

Pascal Auderset, die Mobiliar: Ein Plattformansatz kann erfolgreich auf die Entwicklung digitaler Produkte, insbesondere im Bereich der Versicherungen im Retail-Geschäft, angewendet werden. Muster aus der Software-Entwicklung ermöglichen eine organisatorische Skalierung und Parallelisierung der Entwicklung sowie der Weiterentwicklung von Versicherungsprodukten, wodurch Unternehmen schneller auf Marktveränderungen reagieren können.

BATbern53 Post Data persistence in the business-critical and event driven env...BATbern

BATbern53 BKW Easy Migration through Clean ArchitectureBATbern

Wie kann sichergestellt werden, dass bei der Migration einer Persistenz Schicht von einer On-Premise Umgebung in die Cloud alle Anforderungen erfüllt werden und dabei keine „unvorhergesehenen Überraschungen“ auftreten ? Dieser Frage werde ich nachgehen und dabei im Deep Dive die von uns verwendete Softwarearchitektur durchleuchten. Zusätzlich werde ich deren Einfluss auf das Ergebnis des durchgeführten PoC in Bezug auf die Migration aufzeigen. Dabei werde ich auch auf solch spannende neue Begriffe wie Code Bricks Architecture oder auch Single Point of Configuration eingehen. Kommen Sie mit mir auf eine architektonische Reise durch die moderne Softwareentwicklung.

BATbern53 ETHZ Rethinking Cluster State Management for Lightweight Function a...BATbern

Serverless computing raises the level abstraction to the cloud, making the cloud easier to use and enabling the cloud platform to optimize for performance and energy efficiency under the hood. Although the serverless paradigm holds great promise, the system software that serverless platforms are built on today is still rooted in the very different, more traditional execution model of long-running containers or virtual machines. Today’s Functions as a Service (FaaS) platforms orchestrate function sanboxes with systems based on conventional cluster managers like Kubnernetes. We find that Kubernetes-based cluster managers add significant overhead in FaaS environments because the way that they manage, persist, and update cluster state is not designed for high churn or short-lived sandboxes. Rather than retrofitting existing cluster managers, we propose Dirigent, a clean-slate system tailored for FaaS that supports orders of magnitude higher scheduling throughput than Kubernetes-based systems. Dirigent simplifies cluster manger abstractions to optimize state management, eliminates persistent updates on the critical path of function invocations by recognizing that serverless allows for relaxing state reconstruction guarantees, and runs monolithic control and data planes to minimize internal communication between cluster manager components to maximize throughput.

BATbern53 SBB Wieso in jeder Zugfahrt der SBB ein Stück MongoDB drinstecktBATbern

BATBern53 - EPFL - Blue Brain and related technical challengesBATbern

Scientific exploration has increasingly shifted from traditional storage-centric approaches to compute-centric methods, leveraging advanced software for data integration, analysis, and interactive visualization of vast scientific datasets. To address this need, Blue Brain developed and open-sourced Blue Brain Brayns, a technical platform for the visualization of in-silico datasets, which later gave birth to the Blue Brain BioExplorer. Initially created to tackle key scientific questions related to the Coronavirus and to deliver a powerful visualization tool, the BioExplorer now excels in reconstructing, visualizing, exploring, and detailing the structure and function of biological systems, from molecular systems up to the full brain. In this highly visual presentation, I will demonstrate how I architected and built the application that EPFL's Blue Brain Project utilizes to generate high-quality, high-fidelity images and movies, along with interactive and immersive experiences of the digital reconstruction of the rodent brain. By prioritizing computational efficiency over traditional storage, we enable real-time manipulation and exploration of large datasets. Additionally, I will present a few examples of how AI is a game changer, solving problems that were almost impossible to solve with classic techniques. These examples will highlight the transformative impact of AI in enhancing image quality, speeding up data processing, and uncovering new scientific insights that were previously beyond our reach

BATbern53 Die Mobiliar Bring die Algorithmen zu den Daten – nicht umgekehrtBATbern

SQL-Embedded Machine Learning symbolisiert den nächsten Schritt in der Analyse von Transaktionsdaten und folgt dem Paradigma 'Bring die Algorithmen zu den Daten - nicht umgekehrt'. Die Präsentation beginnt mit einem Beispiel aus der Welt des Fußballs, gefolgt von einer kurzen Einführung in SQL-Embedded Machine Learning und einer Diskussion der Ziele und Ergebnisse einer ersten produktiven Anwendung. Wir werden auch kurz darauf eingehen, wie die Predictions auf einfache und verständliche Weise transparent gemacht werden können (Explainable AI).

BATbern53 ELCA Analyticsdatenhaltung in der CloudBATbern

Die Evolution von DWHs und Data Lakes zu Cloud Analytics Plattformen und neuen Technologien hat das Datenmanagement und die Datenhaltung grundlegend verändert. Dateiformate wie Avro und Parquet auf Blob-Storage in der Cloud sind schon länger zu Standards über Analytics hinaus geworden, haben aber auch ihre Einschränkungen. Datenhaltung mit Open Table Formats wie Iceberg, Delta Lake und Hudi bieten wichtige Erweiterungen wie Metadaten Management, Transaktionen, Versionierung und Change Data Capture. Nach einem Blick auf die Geschichte schauen wir uns Open Table Formats genauer an und diskutieren Erfahrungen anhand von zwei Projekt Beispielen.

BATber53 AWS Modernize your applications with purpose-built AWS databasesBATbern

Modernize your applications with purpose-built AWS databases What are the key attributes of a modern database application? A modern database application should be designed for innovation and agility, it should perform without any limits on performance and scalability, and it should be highly available, easily managed, and cost effective. A modern database should then support all data types and access patterns. It should implement ACID transactions and strong consistency, it should provide limitless scalability and tolerate load variability with high performance regardless of the query complexity. It should run without downtime at a predictable and cost-effective price. And, obviously it should be simple and easy to use, tune and maintain. The problem is that we know, since the CAP theorem, that such a database does not exist. That's why a modern application strategy has to rely on specific purpose-built database engines for the persistence layer. This session will cover the databases AWS offers to meet application needs for in-memory workloads, key-value stores, document collections as well as time-series, graph and ledger databases. We will illustrate the presentation with several customer architecture use-cases.

BATbern52 Moderation Berner Architekten Treffen zu Data MeshBATbern

Das BATbern52 fand wie folgt statt Datum: 8. März 2024, 16:00h - 19:00h Thema: "Data Mesh - Modernes Datenmanagement" Ort: Forum, Zentrum Paul Klee, Bern (Anfahrt) Beiträge: "Data Mesh: 2019, 2023 und 2024++", Simon Harrer, INNOQ "Skalierte Datenwertschöpfung mit Data Mesh", Bruno Russiniello, die Mobiliar, Lukas Feuz, die Mobiliar "Schöne neue Data Products und deren nicht nur technischen Knacknüsse", André Rogger, SBB "Swisscom's Journey into Data Mesh", Mirela Navodaru, Swisscom

BATbern52 Swisscom's Journey into Data MeshBATbern

Swisscom is taking one bold step after another to become a data-driven company. The approach is always business-first: how to find data&AI-driven solutions to enable the business to make the best decisions to offer a great experience to our customers. Our journey with Data Mesh is no different. Together with the business, we looked at the current challenges of quickly transforming data into information and insights while having a crucial regard for data management and governance. I invite you to this session to go through our transformation from data to data products, how to foster co-creation between data producers and data consumers, and what it takes to create the right balance between central governance and decentralizing the accountability for its implementation.

BATbern52 SBB zu Data Products und KnacknüsseBATbern

Mit den Data Products und dem Data Mesh weht ein neuer Wind durch die Data Analytics-Stuben. Konzepte, die im Bereich des Software Engineerings bereits etabliert sind, werden für neue Analytics-Vorhaben entsprechend adaptiert. Dies Nähe zur Software Entwicklung ist nicht erstaunlich, stammen doch die Prinzipien des Data Mesh aus dem Umfeld von Martin Fowler. Die Grundgedanken sind bestechend einfach: das Data Product als eine klar abgegrenzte Einheit mit klaren Regeln und Daten, die einen Mehrwert für den Datennutzer bieten. Diese Data Products können in einer Mesh-Architektur miteinander verknüpft und zu einem funktionierenden Ganzen zusammengefügt werden. Damit soll der «Ad-Hoc»-Charakter, der vielen Analytics-Lösungen innewohnt, überwunden werden. Bei der Adaption von Data Products in eine bestehende Analytics-Landschaft müssen technische und organisatorische Hürden genommen und viel Überzeugungsarbeit in der Analytics-Gemeinde geleistet werden. Wir nehmen Euch mit auf diese Reise, die noch nicht zu Ende ist.

BATbern52 Mobiliar zu Skalierte Datenprodukte mit Data MeshBATbern

Die Gruppe Mobiliar hat sich das Ziel gesetzt, 'unter jedem Dach daheim' zu sein und dabei ein optimales Kundenerlebnis zu bieten. Um dieses Ziel zu erreichen, setzt sie unter anderem auf eine skalierte Datenwertschöpfung als strategische Unternehmensfähigkeit. Bruno Russiniello und Lukas Feuz geben in dieser Präsentation Einblicke in die genaue Bedeutung dieses Ansatzes. Erfahren Sie, wie die Mobiliar durch die Nutzung von Datenprodukten die Digitalisierung aktiv vorantreibt.

BATbern52 InnoQ on Data Mesh 2019 2023 2024++BATbern

Data Mesh ist ein soziotechnischer Ansatz für die Erstellung einer dezentralen Datenarchitektur. Zhamak Dehghani hat den Begriff Data Mesh und die Theorie dahinter mit ihren Blog-Beiträgen in 2019 bereits geprägt und dann mit ihrem Buch noch etwas ausführlicher beschrieben. Diese Theorie möchte ich kurz vorstellen, um für den Abend ein Begriffsfundament zu giessen. Seit 2019 ist jedoch viel passiert. Viele haben das Konzept gelesen und einige haben ihre Data Mesh Reise bereits begonnen. Ich möchte auf 2023 zurückblicken und den aktuellen Stand der Praxis beschreiben. Enden möchte ich mit einem Ausblick über die Top 5 Entwicklungen in Data Mesh, die dich total überraschen werden.

Serverless Dev(Ops) in der PraxisBATbern

Die Schweiz, das Land der vielen Sprachen. Das Translation Portal der Raiffeisen Schweiz bietet verschiedenste Möglichkeite Übersetzungsprozesse zu standardisieren und automatisieren. Bei der Entwicklung des Portals wurden ausschliesslich cloud native Dienste und Methoden eingesetzt. Wir zeigen auf, wie Serverless Code durch vollautomatisierte Pipelines integriert, getestet und ausgerollt wird. Dabei erfahren Sie unter anderem was Serverless aus Sicht DevOps für die Raiffeisen Schweiz und im Allgemeinen bedeutet.