Api Abstraction & Api Chaining

Editor's Notes

• #3: We all access them, we all use them… some of us even build them. A lot of talks have been presented here too… But what is it? How does it work? What are the architectural pieces and how do you build your api for now so it can work with future architecture?
• #4: Lets begin by talking about what an API is… (read definition) so basically: interface to functionality/data But is this the main purpose Original intent of api: communication was internalized Web api: communication extends out to architecture with api call flow As a system gets more and more advanced, the API needs to share functionality/data across the architecture and with the flow and thus cannot have it tied directly to any one portion of the application.
• #5: So when the concern is shared, the api’s main purpose becomes communication… not the building of a resource. Thus the we learn the Api’s concern is shared through the call flow to/from it; tools outside the application handling the api call pre-request can handle functionality for the api as well as post-response. API’s never took into consideration external concerns outside the application; never had to share concern; could bundle functionality/data with communication So the api is an interface to a separation of concern. But at the same time, we are saying it is a shared concern??? This doesn't make sense? How can it be both?
• #6: Well it can’t without entanglement and duplication. If the api flow is extended out to the architecture so that functionality and data needs to be shared in tooling, the concern will be duplicated. The adage ‘build for now but build to scale’ fails miserably here. It would be more apt to say ‘build for now and rebuild for scale’
• #7: When an api is ‘shared’ across a call flow, such as in I/O communications, functionality and data needs to be shared across all tooling The api functionality/data no longer exists just in the application; it exists across all tooling in the call flow The the api is not only part of the ‘I/O flow’ in the architecture; the data and functionality IS the I/O flow functional and data elements of the api are all shared across this flow processing, configuration,security, authorization… these elements from the application are duplicated in the FLOW through external tooling in the architecture. And as external tooling shares the api concern, it naturally becomes scattered and entangled (duplicated and with significant dependencies)
• #8: When we look at an api application, there is a lack of separation of concern because we are binding communication logic to business logic Business logic is not supposed to handle communication; it’s main purpose is building a ‘resource’ for an api call By binding logic for handling processing and data for I/O, we are binding the the I/O flow to the controller. Thus creating a lack of separation of concern. If this was centralized within the application, this would not be an issue. But as the api call flow extends out beyond the boundaries of the application in the architecture, so can the functionality and data to tooling. And so as we scale an application, this makes it impossible to move processing and data associated within the api to external tooling without duplicating or a complete rewrite.
• #9: Thus we can easily say that the api is a cross cutting concern. But what does that mean? If you are not familiar with aspect oriented programming, you may not be familiar with what this concept…
• #10: If you are not familiar with aspect oriented programming, Cross cutting concerns are a core concept. (read) Note to self: (Concerns are modularized groupings of code that share a similarity in purpose and data.) So when we talk about a applications, concerns are centralized but when we talk about a system, concerns begin to extend beyond the application to the architecture with the call flow. concerns around communication generally share functionality and data common to I/O so in an architecture, the flow of I/O needs to be able to access these in a common way Now most people only separate the concern within the application, But when the concern extends beyond the application to the architecture… what then?
• #11: When we look at the Api as a separation of concern WITHIN the application we can see there are shared concerns that get duplicated as a result of the application not abstracting the data/functionality from the api.
• #12: Once we abstract this data/functionality from the application api, this cross cutting concern can share the functionality and data of the api in the architecture. This makes it easier to build an api application that will scale as you and your organization does.
• #14: But everything is awesome, right? everything is working and scaling and doing great right? Well … no. In scaling, Api’s have become overly complex like a Katzenjammer House; constantly adding on in order to work around problems that exist at the very foundation. Lets take a look at some of the existing problems
• #15: As we already stated, API functionality is duplicated across all concerns. You can’t have separation of concern and a shared concern at the same time. These need to share and synchronize with the api concern but instead merely duplicate in most instances as the functionality is bound to the controller
• #16: Also there is a lack of separation of the concern related to the api in the application. Controllers acts as Business logic/communication layer This causes functionality to be redundant in the application as well as the external concerns (ie security); redundant methods for security, batching, etc or redundant code. And because request/response is I/O, processing directly related to it will always be redundant when handled away from a api/communication layer This is LESS THAN ideal for a scaleable architecture where api functionality needs to be shared in the architecture across tools.
• #17: When attaching security to controller, you cannot make it atomic easily without also being redundant Security checks need to be atomic Currently all uris have role checking but we also need request role checks on incoming data; this allows different roles to send different data response role checks to allow formatting of data per role; this allows roles to return different data.
• #18: Regardless of security used and how it is implemented, all api implementations can ONLY check api access and NOT request params based on ROLE or response params based on ROLE This is because api processing is built into controller and not separated as reload able data for the api layer to handle Apidocs/OPTIONS will not show correct data if this is internalized to your controller/method. (see above) This also makes it very difficult to changes access without reversion entire app. Having to make simple config change and redeploy and reversion is silly when a commonly cached object could be read in PRIOR and an api version could be maintained separate from application version to allow for changes to api config. we need to be able to reload the api config without reloading the application. The process of the ‘packaging’ of the api needs to be separate from the ‘package’ and as such, so does version thus allowing us to cache and reload config changes on the fly with new versions while deprecating old configs. (mention swagger meeting at ApiCraft…)
• #19: So the question I keep getting asked is ‘how come nobody thought of this before’? And I asked myself this same question and came to this conclusion… Well the first thing you need to understand is that the architectural call flow is not taken into consideration when building an api in an application in the past. Web api’s add a new layer of complexity by adding in this. Now we have to think about external tooling.
• #20: First you need to understand the core problem.. why are we tying the api to the controllers? The Endpoint. As it exists our current api flows is such that… url mapping handles entrypoint/uri and hands off to filter filter acts as an pre/post handlerinterceptor but few people use this controller handle request/response and the entire world binds all communications for API to controller If you look at most api libraries like JAX-RS, they use annotations for the controllers as well. Everyone assumes this is the endpoint because url eventually resolves to controller/method controller/method returns resource But is this REALLY the endpoint?
• #21: It all comes down to ‘confusion about the api endpoint’. If we look at the Wikipedia definition… (read) in other words, the endpoints exists in an abstracted communication layer away from the subsystems. But are we doing that? No because we are obviously binding it to the control and not a communication layer. The request is injected into the controller but I/O logic and data are not abstracted away for it to be handled PRE & POST business logic. So either as a convenience or a mistake, libraries were created to bind the api to the controller without taking into consideration eventual architectural needs and concerns.
• #22: wait wait… don’t panic. Everything still works. It’s just it’s not as scaleable or functional as it should be. Allow me to explain…
• #23: wait wait… don’t panic. Everything still works. It’s just it’s not as scaleable or functional as it should be. Allow me to explain…
• #24: But if we associate the configuration data with the uri, we can have an entirely different flow with a communication layer: url mapping handle entrypoint/uri and hands off to filter filter acts as an pre/post interceptor layer this allows communications detected as an api call using to have pre/post handler in front of all other business logic to handle api processing and data But how do we share the configuration data across the architecture?
• #25: With something I call an apiObject. Each apiObject is mapped to a controller which in turn SHOULD handle a common set of methods for a common set of data. The apiObject is a JSON object which can be reloadable and handles data which describes access and expected data for the request/response top level variables values : descriptors used for creating relationships and apidocs/options; use for domain data and command object, lists, maps, etc current_stable : current stable version; creates default version deprecation_date : cache deprecation date for this versioned object (can be null) uri method : rest method description : api description (sent with apidocs/options) roles : roles expected for api to be called request : data sent and separation of roles for detection response : data returned and separation of roles for detection
• #27: twinkie and wrapping metaphor
• #28: So the question I keep getting asked is ‘how come nobody thought of this before’? And I asked myself this same question and came to this conclusion… Well the first thing you need to understand is that the architectural call flow is not taken into consideration when building an api in an application in the past. Web api’s add a new layer of complexity by adding in this. Now we have to think about external tooling.
• #29: By using a cached apiObject, we can share a common reloadable configuration file for all tools/instances Api Layer works like this… urlmappings resolve uri filters intercept the uri the ‘before’ logic (ie preHandler) does… data checks early role checks on incoming data handles logic controller resolves resource model builds resource controller returns resource the ‘after’ logic (ie postHandler) handles… (see above) batches/api chains late formatting role checks on output data The Cache stores common api config object shared across architecture and application for security endpoint mapping method handling additional configuration and settings The API Object is a reloadable api definition associated to a common group of uris. In other words it contains a common set of [values,roles,settings] for commonly grouped uri’s such as GET/PUT/POST/DELETE for a controller