Accelerate DevOps/Microservices and Kubernetes

Editor's Notes

• #6: Rick Hightower Rick consults and does contract development for high-speed computing, Java-based uServices, Apache Spark, Apache Kafka, and Apache Cassandra. He also writes about microservice development and reactive streaming. Rick is a frequent speaker regarding high-speed, reactive microservice development and has spoken recently at JavaOne as well at FinTech at scale in SF. He specializes in high-speed, in-memory, non-blocking, microservices development which often includes Java EE, QBit, Reakt, Akka, Vert.x, Cassandra, Kafka, and cloud deployments. He has architected and implemented 100 million-users, in-memory content preference engines using Java reactive, streaming, and actor-based system as well as architected and implemented OAuth rate limiter (API gateway) for streaming music service to rate limit all backend services per partner/vendor/mobile app. Rick also contributed to the reference implementations enterprise caches as well as being a member of several spec. committees (JSR-347, JSR-107, etc.). He also is the author of the Boon JSON parser and parsing utilities which ships with Groovy.
• #10: Enterprise Applications flashback While it is true Enterprise Applications are often built with a three tier architecture: backend code, database code and a GUI written in HTML/JavaScript. This has more to do with the world changing than an active choice. The real drivers for Microservices Architecture are cloud/virtualization/OS containerization, EC2, proliferation of mobile devices, cheaper memory, more virtualization, more cores, SSD, trend towards fatter clients, 10 GBE, 100 GBE, etc. The big enterprise web app is becoming obsolete to a certain extent at least for large applications. It is not like we one day came up with a better idea, and then one day we were like hey what if we could make our code more modular. The ground changed under our collective feet. The way we build, deploy, and consume has changed. Hardware evolved. Virtualization evolved. Containerization happened. Cloud computing became a real thing, and so compelling that it is hard to ignore. The smart phone / tablet / mobile revolution happened. Microservice is the response to these external events. Microservices and NoSQL are two trends that are more focused on how to address software development where deployments are increasingly cloud based and clients are increasingly mobile based. Just like you can’t compare client / server development of the mid-90s to mainframe development from the 70s, you can’t compare enterprise applications from 2001 to microservice development targeting mobile and web clients and other microservices in 2015. The world changes. We adjust. Microservice trend is course correction not a new religion. History of Enterprise Applications Remember 1990s, the reason why Enterprise Applications are written with three tiers was was to avoid DLL hell, and the monolith. We just did it with the tools available at the time. Back in the day, we used to build apps that were two tiered. You had to actually go to each users machine and help them install the app. There was a damn good chance they downloaded some shareware that installed a DLL that screwed up the install, and you were in hell. It was not like we were, “Hey James!” .. “What Martin?” “Do you want to build a huge monolith?” “Sure Martin!”. We tried Applets but Java GUI development back then sucked (1999). I don't think it sucks so bad now, but it lost its window of opportunity for adoption. Then we were left with HTML/JavaScript clients and forcing everyone to use the same browser in the corporation at least for the corporate apps. I worked at many corporations (2003) that banned JavaScript due to incompatibilities of browsers. You were left with screen painting with HTML. It was a like a colorful green screen of yesteryear, but way slower, but pretty. There are many reasons why this style of “Enterprise Development” does not work in the cloud and for mobile devices. The server-side applications no longer need to handle HTTP requests, get data from a database and execute all domain logic, and draw pretty pictures in HTML. Much pain, and great expense has been incurred trying get this three tier architecture to scale in the cloud for various devices written in a polyglot of languages. Microservices exists because mobile, cloud, cheaper RAM, cheaper disks, and improved virtualization. It is really just taking the world where it is. It is not revolutionary at all. Server components, EAR files and WAR files.. may they rest in peace If you have lived through COM, DCOM, CORBA, EJBs, OSGi, J2EE, SOAP, SOA, DCE, etc. then you know the idea of services and components is not a new thing, but they are a date expired concept for the most part. One issue with enterprise components is they assume the use of hardware servers which are large monoliths and you want to run a lot of things on the same server. That is why we have WAR files and EAR files, and all sorts of nifty components and archives. Well turns out in 2015 (and less so in 2019), that makes no sense. Operating systems and servers are ephemeral, virtualized resources and can be shipped like a component. We have EC2 images AMIs, OpenStack, Vagrant and Docker. The world changed. Move on. Microservices just recognize this trend so you are not developing like you did when the hardware, cloud orchestration, multi-cores, and virtualization was not there. You did not develop code in the 90s with punch cards did you? So don’t use an EAR file or a WAR file in 2015. Now you can run a JVM in a Docker image which is just a process pretending to be an OS running in an OS that is running in the cloud which is running inside of a virtual machine which is running in Linux server that you don’t own that you share with people who you don’t know. Got a busy season? Well then, spin up 100 more server instances for a few weeks or hours. This is why you run Java microservices as standalone processes and not running inside of a Java EE container. The Java EE container is no longer needed because servers are not giant refrigerator boxes that you order from Sun and wait three months for (circa 2000). Don’t fight classpath, classloader hell of Java EE. Hell your whole damn OS is now an ephemeral container (Docker). Deliver an image with all the libs you need, don’t deploy to a Java EE server which has to be versioned and configured. You are only running one service in it anyway. Turns out you don’t have five war files running in the same Java EE container since oh about 2007. Let it go. If you are deploying a WAR file to a Java EE container then you are probably not doing microservice development. If you have more than one WAR file in the container or an EAR file, then you are definitely not doing microservice development. If you are deploying your service as an AMI or docker container and your microservice has a main method, then you might be writing a microservice. Microservices architectures opt to break software not into components but into reusable, independently release-able services which run as one or more processes. Application and other services communicate with each other. So where we might have used a server side component, we use a microservice running in independent processes. Where we might have had WAR files or EAR files now we have a Docker container or a Amazon AMI that has the entire app preloaded and configure with exactly the libraries it needs (Java and otherwise). JSON, HTTP, WebSocket … NO WSDL! Now you just have to document the Microservices HTTP/JSON interface so other developers can call into it. We could say REST, and certainly you can use concepts from REST, but hey HTTP calls are enough to be considered a Microservice. Keep this in mind: No XML. No SOAP. No WSDL. No WADL. JSON! Ok you can add some meta data and document how to talk to your service, but the idea is the docs should be documented with curl. If you are only using SOAP or XML then you are not producing a microservice. JSON is a must. Documents should sound more like: I give you this request with these headers, params and JSON body and you respond with this JSON. Keep it simple. You can provide things in addition to JSON, but JSON is the minimum requirement. If you are not delivering up JSON and consuming JSON over HTTP or HTTP WebSocket then what you wrote is not probably not a microservice.
• #13: Introduction To Microservices Microservice architecture is a method of developing software systems. Its focus is building small, reusable, scalable services. Applying Microservices becomes very important when you have to create services for polyglot devices: wearables, Internet of Things (IOT), mobile, desktop, and web. The trend towards providing services for rich, native mobile application and web applications started the trend towards Microservices adoption. This is one reason why microservices lean heavily on web technologies like HTTP/REST/WebSocket with JSON,Message Pack, and their ilk. The web technologies provide a low barrier to entry and least common denominator to communication. The closest to a standard definition of microservices is Microservices by James Lewis and Martin Fowler. … Continuous delivery: The microservices architectural approach is to create smaller services that are focused on a small business domain or crosscutting concern. Microservices adopt the Unix single purpose utility approach to service development. They are small so they can be released more often and are written to be malleable. They are easier to write. They are easier to change. Microservices go hand in hand with continuous integration and continuous delivery. The services are independent enough not to need a gigantic release train to release improvements or new features. In the Java world, this means you will be using other microservice like Jenkins to provide frequent releases.
• #14: Key ingredients to microservices architecture are: independently deployable, small, domain-driven services communication through a well-defined wire protocol usually JSON over HTTP (curl-able interfaces) well defined interfaces and minimal functionality avoiding cascading failures and synchronous calls - reactive designing for failure
• #15: This comes up a lot so let's address this now. It is very common for some to confuse Service-Oriented Architecture (SOA) and Microservice Architecture. In a sense, SOA and Microservice Architecture are related in some goals and purposes. Microservices is a refocus and refinement of some of the original goals of SOA to meet the demands of a polyglot device environment that can scale and support continuous deployments in a cloud / third generation virtualization environment. Since SOA was later muddled with BPEL, ESBs, SOAP, WSDL, and their ilk, it is easier to drop the SOA moniker and just focus on the parts that worked well. Keep the parts that work. Get rid of the rest. The focus on Microservices Architecture is web technologies to provide scalability, modular, domain-drive, small, and continuously deployable cloud-based services. Microservices Architecture is taking what perhaps started out as SOA and applying lessons learned as well as pressure to support polyglot devices, deploy more rapidly and the architecture liquidity that cloud computing and virtualization/containerization provide. You mix all that together and you can see where Microservices Architecture started. Microservices Architecture is in general less vendor driven than SOA and more needs driven by demands of application development and current cloud infrastructure. For more thoughts on SOA vs. Microservices read Microservices Architecture.
• #16: Microservices Architecture is taking what perhaps started out as SOA and applying lessons learned as well as pressure to support polyglot devices, deploy more rapidly and the architecture liquidity that cloud computing and virtualization/containerization provide. You mix all that together and you can see where Microservices Architecture started. Microservices Architecture is in general less vendor driven than SOA and more needs driven by demands of application development and current cloud infrastructure.
• #18: Cope with failure Microservices are designed to cope with failure. Since microservices tend to call each other, a downstream service that fails should not block upstream services and clients. Synchronous communication is avoided to avoid cascading failures. Thus Microservices tend to embrace async calls, streams, queues and actor systems. To handle failure, it is easier to embrace Queue theory so you can detect failure and provide alternatives or just fail gracefully without blocking clients. The ability to react to failure is a key characteristic of Microservices. In order to learn about downstream failure or learn about other functioning nodes and implement some sort of circuit breaker, one needs Service Discovery. Tools like consul, etcd and SkyDNSare perfect for Service Discovery. Adopting Microservices is not a free lunch. Steps to mitigate inherent complexity include distributed logging and MDC, microservices monitoring and stats, and reactive programming to coordinate async calls. The reactive call coordination would encompass what to do if there is a failure and what to do if a downstream service does not fail, but just times out or worse hangs.
• #19: User Experience and Microservices Monitoring With Microservices which are released more often, you can try new features and see how they impact user usage patterns. With this feedback, you can improve your application. It is not uncommon to employ A/B testing and multi-variant testing to try out new combinations of features. Monitoring is more than just watching for failure. With big data, data science, and microservices, monitoring microservices runtime stats is required to know your application users. You want to know what your users like and dislike and react. Debugging and Microservices Monitoring Runtime statistics and metrics are critical for distributed systems. Since microservices architecture use a lot of remote calls. Monitoring microservices metrics can include request per second, available memory, #threads, #connections, failed authentication, expired tokens, etc. These parameters are important for understanding and debugging your code. Working with distributed systems is hard. Working with distributed systems without reactive monitoring is crazy. Reactive monitoring allows you to react to failure conditions and ramp of services for higher loads. Circuit Breaker and Microservices Monitoring You can employ the Circuit Breaker pattern to prevent a catastrophic cascade, and reactive microservices monitoring can be the trigger. Downstream services can be registered in a service discovery so that you can mark nodes as unhealthy as well react by reroute in the case of outages. The reaction can be serving up a deprecated version of the data or service, but the key is to avoid cascading failure. You don't want your services falling over like dominoes. Cloud Orchestration and Microservices Monitoring Reactive microservices monitoring would enable you to detect heavy load, and spin up new instances with the cloud orchestration platform of your choice (EC2, CloudStack, OpenStack, Rackspace, boto, etc.).
• #21: Continuously deployable services The focus on Microservices is a focus on business capability, and a refocus on object oriented programming roots and organizing code around business domains with data and business rules co-located in the same process or set of processes. The focus is on breaking up applications into small reusable services which might be useful to other services or other applications. The services can be deployed independently. This allows each of these services to be tweaked, and then redeployed independently. This is where the "micro" part of microservices comes into play. The services are small and independent. This is where microservices have been compared to Unix philosophy, they provide services that handle requests and give responses. Ken Thompson, Unix creator, said Unix has a philosophy of one tool, one job. The Unix philosophy emphasizes building short, simple, clear, modular, and extendable code that can be easily maintained and repurposed by developers other than its creators.
• #29: Forsgren PhD, Nicole. Accelerate . IT Revolution Press. Kindle Edition.
• #30: High performers are twice as likely to exceed organizational performance goals as low performers: profitability, productivity, market share, number of customers. Forsgren PhD, Nicole. Accelerate . IT Revolution Press. Kindle Edition.
• #31: High performers are twice as likely to exceed noncommercial performance goals as low performers: quantity of products/ services, operating efficiency, customer satisfaction, quality of products/services, achieving organizational/mission goals. Forsgren PhD, Nicole. Accelerate . IT Revolution Press. Kindle Edition.
• #32: In a follow-up survey to the initial 2014 data collection effort, we gathered stock ticker data and performed additional analysis on responses from just over 1,000 respondents across 355 companies who volunteered the organization they worked for. For those who worked for publicly traded companies, we found the following (this analysis was not replicated in later years because our dataset was not large enough): ​–​High performers had 50% higher market capitalization growth over three years compared to low performers. Forsgren PhD, Nicole. Accelerate . IT Revolution Press. Kindle Edition.
• #33: The four measures of software delivery performance (deploy frequency, lead time, mean time to restore, change fail percentage) are good classifiers for the software delivery performance profile. The groups we identified—high, medium, and low performers—are all significantly different across all four measures each year. Our analysis of high, medium, and low performers provides evidence that there are no trade-offs between improving performance and achieving higher levels of tempo and stability: they move in tandem. Software delivery performance predicts organizational performance and noncommercial performance. Forsgren PhD, Nicole. Accelerate . IT Revolution Press. Kindle Edition.
• #34: Lead time is highly correlated with version control and automated testing. MTTR is highly correlated with version control and monitoring. Software delivery performance is correlated with organizational investment in DevOps. Software delivery performance is negatively correlated with deployment pain. The more painful code deployments are, the poorer the software delivery performance and culture. Forsgren PhD, Nicole. Accelerate . IT Revolution Press. Kindle Edition.
• #35: Unplanned work and rework: ​–​High performers reported spending 49% of their time on new work and 21% on unplanned work or rework. ​–​Low performers spend 38% of their time on new work and 27% on unplanned work or rework. ​–​There is evidence of the J-curve in our rework data. Medium performers spend more time on unplanned rework than low performers, with 32% of their time spent on unplanned work or rework. Manual work: ​–​High performers report the lowest amount of manual work across all practices (configuration management, testing, deployments, change approval process) at statistically significant levels. ​–​There is evidence of the J-curve again. Medium performers do more manual work than low performers when it comes to deployment and change approval processes, and these differences are statistically significant. Forsgren PhD, Nicole. Accelerate . IT Revolution Press. Kindle Edition.
• #36: BURNOUT AND DEPLOYMENT PAIN: Deployment pain is negatively correlated with software delivery performance and Westrum organizational culture. The five factors most highly correlated with burnout are Westrum organizational culture (negative), leaders (negative), organizational investment (negative), organizational performance (negative), and deployment pain (positive). Forsgren PhD, Nicole. Accelerate . IT Revolution Press. Kindle Edition.
• #38: Trunk-based development: ​–​High performers have the shortest integration times and branch lifetimes, with branch life and integration typically lasting hours or a day. ​–​Low performers have the longest integration times and branch lifetimes, with branch life and integration typically lasting days or weeks. Technical practices predict continuous delivery, Westrum organizational culture, identity, job satisfaction, software delivery performance, less burnout, less deployment pain, and less time spent on rework. High performers spend 50% less time remediating security issues than low performers. Forsgren PhD, Nicole. Accelerate . IT Revolution Press. Kindle Edition.
• #39: A loosely coupled, well-encapsulated architecture drives IT performance. In the 2017 dataset, this was the biggest contributor to continuous delivery. Forsgren PhD, Nicole. Accelerate . IT Revolution Press. Kindle Edition.
• #40: LEAN PRODUCT MANAGEMENT CAPABILITIES The ability to take an experimental approach to product development is highly correlated with the technical practices that contribute to continuous delivery. Lean product development capabilities predict Westrum organizational culture, software delivery performance, organizational performance, and less burnout. Forsgren PhD, Nicole. Accelerate . IT Revolution Press. Kindle Edition. Forsgren PhD, Nicole. Accelerate . IT Revolution Press. Kindle Edition.
• #45: Forsgren PhD, Nicole. Accelerate . IT Revolution Press. Kindle Edition.
• #46: Forsgren PhD, Nicole. Accelerate . IT Revolution Press. Kindle Edition.