Discovering Seamless Integration: MuleSoft, AWS and Snowflake | MuleSoft Mysore Meetup #34

July 27th, 2023
Mysore MuleSoft Meetup
MuleSoft, AWS & Snowflake Integration

Safe Harbour Statement
● Both the speaker and the host are organizing this meet-up in individual capacity only.
We are not representing our companies here.
● This presentation is strictly for learning purposes only.
● Organizer/Presenter do not hold any responsibility that same solution will work for
your business requirements.
● This presentation is not meant for any promotional activities.
3

A recording of this meetup will be uploaded to events page within 24 hours
Questions can be submitted/asked at any time in the Chat/Questions and Answers Tab
Make it more Interactive!!!
Give us feedback! Rate this meetup session by filling feedback form at the end of the day
We Love Feedbacks!!! Its Bread & Butter for Meetup
Housekeeping
4

Organizers
5
Shubham Chaurasia
Billennium India
Pro Integration Developer
Giridhar Meka
Sr. Technical Architect
linkedin.com/in/giridharmeka
linkedin.com/in/shubhamchaurasia1

 Certified MuleSoft Developer & Architect
 6+ years of Total Experience, 5+ years in MuleSoft
 Working as Sr. Engineering Management Specialist at
HashedIn By Deloitte
Speaker
6

Agenda
• Introduction
• Overview of the use case and why this integration is necessary.
• Demo
o Creating an S3 bucket for file storage.
o Creating tables and stage environment in Snowflake.
o Configuring AWS & Snowflake for seamless communication.
o Exploring MuleSoft’s S3 & Snowflake Connectors for the use case.
o Testing the use case.
• Q&A
• Wrap Up
7

To design a seamless data integration process that enables real-time data
ingestion from a source system available in CSV/JSON format and efficiently
loads it into a Snowflake database.
AWS S3 will be utilized as an intermediate staging area to optimize the data
transfer and ensure data integrity, leveraging the power of MuleSoft as the
integration platform.
Use Case
8

10
Why Amazon Simple Storage Service (S3)?
 Staging Area for Data Ingestion: AWS S3 serves as a reliable and highly scalable staging area for the real-time data
ingestion process. It allows seamless storage and temporary retention of CSV/JSON files from the source system before loading
into Snowflake, ensuring efficient data transfer.
 High Durability & Availability: AWS S3 offers 99.999999999% (11 nines) of data durability, making it highly reliable for
critical data storage. The built-in redundancy ensures that data is available whenever required, providing a robust foundation for
the integration solution.
 Data Security & Encryption: S3 provides comprehensive data security features, including server-side & client-side encryption
options. This ensures that data stored in the bucket and during transit remains protected, aligning with industry best practices &
compliance requirements.
 Integration with MuleSoft: AWS S3 integrates seamlessly with MuleSoft, allowing smooth data flow between the source
system, S3 bucket & Snowflake database. MuleSoft's connectors and adaptors easily interact with S3 APIs, simplifying the
integration process.
 Scalability & Cost-Effectiveness: AWS S3's ability to automatically scale with data growth makes it a cost-effective choice
for storing and managing vast amounts of data. The pay-as-you-go pricing model allows organizations to pay only for the storage
& data transfer they consume, optimizing costs.

 Snowflake Cloud Data Platform: Snowflake is a leading cloud-based data platform designed for modern data warehousing
& data analytics. It provides a powerful and scalable solution for storing, processing and analysing data in real-time.
 Concurrent, Multi-Cluster Architecture: Snowflake's unique architecture allows multiple compute clusters to work
concurrently on the same data. This enables parallel data loading from the AWS S3 staging area and supports complex data
processing tasks, leading to faster and more efficient data integration.
 Seamless Data Ingestion from AWS S3: Snowflake natively integrates with AWS S3, making it easy to ingest data directly
from the S3 bucket. This integration ensures a streamlined data loading process, reducing latency and simplifying data
management.
 Data Sharing & Collaboration: Snowflake's data sharing capabilities enable seamless sharing of data between different
accounts, regions, or even different organizations. This feature facilitates collaboration and allows data to be securely shared
with stakeholders, partners, or other departments for further analysis and insights.
 Built-in Security & Compliance: Snowflake prioritizes data security and compliance. It provides various security features,
including encryption, access controls, and data masking, ensuring data protection at rest and in transit. Snowflake also complies
with various industry and regional regulations, making it suitable for handling sensitive data.
Why Snowflake?
11

Implementation Steps
12
Create trail accounts for AWS & Snowflake and Download Anypoint Studio. They are totally free!
1. https://aws.amazon.com/console/ – AWS Console
2. https://signup.snowflake.com/ – Snowflake
3. https://www.mulesoft.com/lp/dl/anypoint-mule-studio – Anypoint Studio
AWS Root Account Action Steps:
1. Create IAM User Account. Use this account for integration, but not the root account.
2. Create a role for Snowflake to access Amazon resources. For our use case we access S3 & SQS.
AWS IAM User Account Action Steps:
1. Create S3 bucket.
2. Snowflake Action Steps:
 Create Database & table.
 Create storage integration.
 Create stage using storage integration.
 Create snow pipe.
Implement Mule flow

Take a stand !
15
● Nominate yourself for the next meetup speaker and suggest a topic as well.

● Share:
○ Tweet using the hashtag #MuleSoftMeetups
○ Join Mysore Group: https://meetups.mulesoft.com/mysore/
● Feedback:
○ Fill out the survey feedback and suggest topics for upcoming events
○ Contact MuleSoft at meetups@mulesoft.com for ways to improve the program
○ Reach out to Mysore Meetup Leaders (Shubham/Giridhar) to suggest topics
for next Meetup
What’s next?
16

Discovering Seamless Integration: MuleSoft, AWS and Snowflake | MuleSoft Mysore Meetup #34

Recommended

More Related Content

Similar to Discovering Seamless Integration: MuleSoft, AWS and Snowflake | MuleSoft Mysore Meetup #34 (19)

More from MysoreMuleSoftMeetup (20)

Recently uploaded (20)

Discovering Seamless Integration: MuleSoft, AWS and Snowflake | MuleSoft Mysore Meetup #34