Building an aws sdk for Perl - Granada Perl Workshop 2014
Download as PPTX, PDF3 likes1,310 views
This document summarizes the challenges and strategies for building an AWS SDK for Perl. It discusses that while there are many Perl modules on CPAN for individual AWS services, they are inconsistent and not always up to date. Building a comprehensive SDK is challenging due to the large number of services, API versions, signatures, and AWS's rapid pace of innovation. The document outlines approaches taken, including using roles and Moose to dynamically generate classes, introspecting official SDKs, and making the SDK data-driven. It also discusses lessons learned around namespaces, enums, request objects, and calling conventions. It encourages contributors to help on the open source GitHub project.
Building an aws sdk for Perl - Granada Perl Workshop 2014
- 1. Building an AWS SDK Granada Perl Workshop 2014 Jose Luis Martinez JLMARTIN on CPAN Twitter: @pplu_io [email protected]
- 2. About me CTO @CAPSiDE Training Partner for AWS Delivering AWS official training All four AWS certifications Real experience with real platforms Deliver consulting and continuous operation Writing Perl for +10 years And still having fun with it!
- 3. About AWS Mostly known for EC2 and S3 But lot's more services! (+30) Everything is an API “Programmable datacenter” Software Defined Everything
- 4. About AWS Mostly known for EC2 and S3 But lot's more services! (+30) Everything is an API “Programmable datacenter” Software Defined Everything It’s the your wet dream
- 5. Perl & AWS Bad news No official SDK Official ones: Ruby, PHP, Python, JS, .Net, Java Note: python wasn't official initially!
- 6. Perl & AWS Good news CPAN And an active community Lots of modules for different services (sometimes more than one)
- 7. Perl & AWS: The CPAN Inconsistencies In method naming Arbitrary defaults ('eu-west-1', 'us-east-1') Not up to date with latest APIs Keeping up with AWS is hard!!! “Not so popular” services not covered IAM, CloudFormation, SWF, RedShift, CloudSearch, etc Almost no Role support AWS::CLIWrapper is a good generic solution. Not native
- 8. So lets write an entire SDK!
- 10. Hard enough
- 11. You can end up with… or
- 12. Challenges: # of services 30+ services Some with 100+ API calls Each call with it’s parameters Want to return objects Evolution
- 13. Challenges: # of services
- 14. So lets start classifying
- 15. Challenges: Endpoints Endpoint Type Services Regional Services available in a región the rest Single Global services 6
- 16. Challenges: WebService Types One URL REST Params: Query Response: XML 17 3 Params: JSON Response: JSON 10 1
- 17. Challenges: Signatures Type Services v2 2 v3https 1 cloudfront 1 v4 the rest
- 18. Challenges: API versions More tan one API version is live That’s why not-up-to-date callers still work RDS 4 versions CloudFront 3 versions EC2 4 versions
- 19. AWS Pace of innovation Every month there are changes Changes mean APIs change (slightly) Very hard to keep up!
- 20. AWS Pace of innovation Every week there are announcements Changes mean APIs change (slightly) Very hard to keep up!
- 22. Hand write classes and methods, and parameters
- 23. Hand write classes and methods, and parameters
- 24. Parse docs After all… you would have to read them to write the classes
- 25. Parse docs After all… you would have to read them to write the classes
- 26. No published spec Introspect an official SDK! After all AWS already has read the docs for you
- 27. No published spec Introspect an official SDK! That’s your spec! After all AWS already has read the docs for you
- 28. Data driven SDKs! JS SDK has datastructures that define calls Proof of concept Win++! Boto and PHP2 too! Later migrate to using Boto
- 29. Data driven SDKs! JS SDK has datastructures that define calls Proof of concept Win++! Boto and PHP2 too! Later migrate to using Boto
- 30. The solutions
- 31. Modern Perl to the rescue Try to be very lazy Parameter validation: Create Moose classes that validate the parameters passed to the call. May change in the future. Got very quick results! Hide all possible implementation from the user Try not to leak implementation That way you can change it at will Meta lets you inspect the objects! Call objects get marshalled into datastructured via meta Result datastructures get marshalled into objects via meta
- 32. Roles
- 33. Modern Perl: Roles All functionality gets pulled in via Roles Code is basically in different roles, Classes are “serialized configurations” of how to call APIs Makes it easy to Generate service classes from datastructures Change behaviour for tests
- 34. Roles are used for One global endpoint vs endpoints per region Caller (Query, JSON) Response (XML, JSON) Signatures (V2, V3, V4) IO
- 35. Aws is a class factory
- 36. Dynamic class construction User never constructs service classes directly Aws class method uses default config my $ec2 = Aws->service(‘EC2’)->new( . . . )
- 37. Dynamic class construction User wants custom functionality my $aws = Aws->new( config => AWS::SDK::Config->new( caller => 'Net::AWS::TestCaller‘ ) ); my $ec2 = $aws->service(‘EC2’)->new( . . . ) service loads Aws::EC2, creates a new class with TestCaller
- 38. Win: Hello to the async world Still playing around in examples my $aws = Aws->new( config => AWS::SDK::Config->new( caller => 'Net::AWS::MojoCaller‘ ) ); my $ec2 = $aws->service(‘EC2’)->new( . . . ) my $asgs = $ec2->DescribeAutoScalingGroups; See examples/asyncAutoScaling.pl
- 39. Win: Also going “fully async” my $aws = Aws->new( config => AWS::SDK::Config->new(caller => 'Net::AWS::MojoAsyncCaller') ); my $delay = Mojo::IOLoop->delay(sub { my ($delay, @responses) = @_; Dumper($_) for @responses; }); foreach my $region ( "us-east-1", "ap-northeast-1", "sa-east-1", "ap-southeast-1", "ap-southeast-2", "us-west-2", "us-west-1", "eu-west-1", 'invented-region') { my $ctrail = $aws->service('CloudTrail')->new( region => $region, delay => $delay, ); print "Doing a call for $regionn"; my $list = $ctrail->DescribeTrails; } $delay->wait;
- 41. Lessons Learned
- 42. Lessons learned First shot with MooseX::Declare Error messages are HORRIBLE! Change to Moops didn't go well Finally plain old Moose
- 43. Lessons learned Namespace finding (still in the works) AWS vs Aws Thanks @clintongormley for the article!
- 44. Lessons learned Enums (still in the works) APIs sometimes return unexpected stuff
- 45. Lessons learned Abstract $request object passed around Finally marshalled into the UserAgents’ taste
- 46. Still open stuff Load time for big services (EC2) Not all classes will be used all the time Generate classes on demand Calling convention Now CamelCased Not so perlish Calling from objects (not service objects) $queue->DeleteQueue(); from a queue object Attribute Traits Access to Arrays and HashRefs via nicer methods Lots more!!!
- 47. Use AWS? Use Perl? Want to help? Fork your heart out: https://github.com/pplu/aws-sdk-perl/