KnolX AWS Tech. Stack
0 likes319 views
Getting Start with AWS emerging technologies. Learn how to use various AWS tech stack for different use cases especially for data analytics and data engineering domain in real-time. Also quick guide on how to prepare for different AWS certifications. Quick walkthrough of frequently used AWS services in real time.
![[DSC Europe 22] Overview of the Databricks Platform - Petar Zecevic](https://cdn.slidesharecdn.com/ss_thumbnails/petarzecevic-databricksoverview-221130080703-c60d93de-thumbnail.jpg?width=560&fit=bounds)
Ad
More Related Content
What's hot (20)
Similar to KnolX AWS Tech. Stack (14)
Ad
More from Knoldus Inc. (20)
Ad
Recently uploaded (20)
KnolX AWS Tech. Stack
- 1. Presented By: Karuna Puri AWS Emerging Technologies
- 2. Lack of etiquette and manners is a huge turn o๏ฌ. KnolX Etiquettes Punctuality Join the session 5 minutes prior to the session start time. We start on time and conclude on time! Feedback Make sure to submit a constructive feedback for all sessions as it is very helpful for the presenter. Silent Mode Keep your mobile devices in silent mode, feel free to move out of session in case you need to attend an urgent call. Avoid Disturbance Avoid unwanted chit chat during the session.
- 3. Our Agenda 01 04:15 PM AWS Overview 02 04:25 PM AWS Pillars 03 04:30 PM Case Study 04 04:55 PM Certi๏ฌcation Tips & Tricks 05 05:00 PM Conclusion
- 5. Amazon Web Services Your subtitle will be here โ Global data centers โ 200 services โ Secure and robust โ Pay as you go โ Built for business needs
- 6. Why is AWS customers choice Your subtitle will be here 2013 2014 2016 2017 Agility Scale seamlessly Accelerate Time to market Reduce management complexity Agility Increase Innovation Minimize security vulnerabilities Optimize Cost Complexity & Risk
- 7. AWS Services Categories Your subtitle will be here
- 8. Connect to an AWS Services
- 10. Managed Services
- 13. AWS Applications & Use-cases 1. Storage & Backup 2. Big Data 3. Social Networking 4. Websites 5. Gaming
- 14. Case Study
- 15. Create Data Lake & Catalog Problem: Setting up & managing data lake today involves lot of manual, complicated, and time-consuming tasks. It starts from loading data from diverse sources, monitoring those data ๏ฌows, setting up partitions, turning on encryption and managing keys, de๏ฌning transformation jobs and monitoring their operation, re-organizing data into a columnar format, con๏ฌguring access control settings, deduplicating redundant data, matching linked records, granting access to data sets, and auditing access over time. This process can be simpli๏ฌed with help of di๏ฌerent AWS services with de๏ฌning data sources and what data access and security policies we want to apply. Solution: Create a data lake and catalog on cloud which can have di๏ฌerent level of ingest sources (like raw data, transformed data, ๏ฌnal processes data as per business expectation). Layers of data could be one of these: Bronze โ Silver โ Gold Letโs understand di๏ฌerent steps involved in this Data Lake & Data Catalog creation process.
- 16. Continued.. Create Data Lake & Catalog Steps: Bronze Data Layer (Raw Data) 1. Grant Permissions to IAM user 2. Con๏ฌgure S3 bucket to register data for lake formation 3. Data Ingestion to S3 4. Create Database 5. Crawl Data by Glue 6. Create Data Catalog with Glue 7. Use Athena to Query Data in Lake Silver/Gold Data Layer (Transformed-Processed Data) 1. Ingest real-time data using Kinesis Firehose 2. Create Database 3. Crawl Data by Glue 4. Data Catalog with Glue 5. Athena - Query Data in Lake 6. Transform data using Glue
- 17. AWS Certifications: Tips - Tricks
- 20. Conclusion Cloud Experience with AWS: โ In terms of services it has better prospects. โ Provides comprehensive integrated cloud services. โ Follows Pay as you go model. โ Availability is there in more than 245 countries and territories, with approx. 29 launched regions and 93 availability zones. โ More security. โ Implementation & Learning is comparatively tough as compared to other providers. โ Account management could be bit tedious & tricky sometimes as each master account has a set of users and roles de๏ฌned. Hence, it depends more on the organizationโs existing architecture and requirements to go with which cloud provider which takes into account factors - cost optimizations, security, reliability, operational excellence, maintainability, speed, and performance e๏ฌciency.
- 21. References: 1. https://blog.knoldus.com/enterprise-data-lake-cloud-aws/ 2. https://blog.knoldus.com/transform-data-on-cloud-with-aws-glue-managed-etl-platform/ 3. https://explore.skillbuilder.aws/learn/external-ecommerce;view=none;redirectURL= 4. https://aws.amazon.com/training/classroom/ 5. https://aws.amazon.com/training/events/ 6. https://aws.amazon.com/training/digital/
- 22. Thank You ! Get in touch with us: Lorem Studio, Lord Building D4456, LA, USA