Cloud Foundry Diego: Modular and Extensible Substructure for Microservices
The Diego project was originally conceived as a rewrite of the Droplet Execution Agent (DEA) component of the Cloud Foundry elastic runtime, the component responsible for scheduling, starting, stopping, and scaling applications in Linux containers. Since Diego’s inception, this development effort has been guided by core principles such as simplicity, loose coupling, high cohesion, separation of concerns, and seeking the right abstractions. These guiding principles have resulted in an extremely modular platform that provides a welcome home for your microservices. Microservices are loosely coupled, independently deployable applications whose individual scopes are guided by the concept of bounded contexts. Martin Fowler has described well the operational maturity required to employ microservices architectures, memorably stating “you must be this tall to ride the microservices ride,” with the capability to do rapid deployment and basic monitoring. Diego’s opinionated automation and health checking provide a great platform for operating microservices. At the same time, this platform has clean abstractions that support useful extension points. In this presentation we'll explore the Diego architecture, highlight Diego’s role as the new core of the Cloud Foundry elastic runtime, and illustrated how Diego is being used as a component in other platforms such as Lattice and Spring XD. We'll also look at how Diego's abstractions provided an easy road to adding alternative backends for other platforms like core Windows/.NET support to Cloud Foundry. Finally, we'll discover how Diego's abstractions are providing the Spring Cloud project with a clear road to providing tighter integration between the Netflix OSS stack of services and Cloud Foundry, with a goal of enabling support for polyglot cloud-native application architectures.
![DESIRED TASK
{
...
"rootfs": "docker:///docker-org/docker-image",
"env": [
{"name": "ENV_NAME_A", "value": "ENV_VALUE_A"},
{"name": "ENV_NAME_B", "value": "ENV_VALUE_B"}
],
"cpu_weight": 57,
"disk_mb": 1024,
"memory_mb": 128,
"privileged": true,
"action": ACTION(s) TO RUN,
...
}
18](https://image.slidesharecdn.com/diegoapachecon2015-150414193150-conversion-gate01/85/Cloud-Foundry-Diego-Modular-and-Extensible-Substructure-for-Microservices-18-320.jpg)
![DESIRED LRP
{
...
"instances": 17,
"rootfs": "VALID-ROOTFS",
"env": [
{"name": "ENV_NAME_A", "value": "ENV_VALUE_A"},
{"name": "ENV_NAME_B", "value": "ENV_VALUE_B"}
],
"cpu_weight": 57,
"disk_mb": 1024,
"memory_mb": 128,
"privileged": true,
"setup": ACTION,
"action": ACTION,
"monitor": ACTION,
"start_timeout": N seconds,
"ports": [8080, 5050],
...
}
21](https://image.slidesharecdn.com/diegoapachecon2015-150414193150-conversion-gate01/85/Cloud-Foundry-Diego-Modular-and-Extensible-Substructure-for-Microservices-21-320.jpg)
More Related Content
What's hot (20)
Viewers also liked (20)
![Carpeta Ns Chile Empresas[1]](https://cdn.slidesharecdn.com/ss_thumbnails/carpetanschileempresas1-12527130190224-phpapp01-thumbnail.jpg?width=560&fit=bounds)
Similar to Cloud Foundry Diego: Modular and Extensible Substructure for Microservices (20)
More from Matt Stine (20)
Recently uploaded (20)
![Pixologic ZBrush Crack Plus Activation Key [Latest 2025] New Version](https://cdn.slidesharecdn.com/ss_thumbnails/fashionevolution2-250322112409-f76abaa7-250428124909-b51264ff-250504160528-fc2bb1c5-thumbnail.jpg?width=560&fit=bounds)
![Download Wondershare Filmora Crack [2025] With Latest](https://cdn.slidesharecdn.com/ss_thumbnails/neo4j-howkgsareshapingthefutureofgenerativeaiatawssummitlondonapril2024-240426125209-2d9db05d-250419-250428115407-a04afffa-thumbnail.jpg?width=560&fit=bounds)
Cloud Foundry Diego: Modular and Extensible Substructure for Microservices
- 1. CLOUD FOUNDRY DIEGO MODULAR AND EXTENSIBLE SUBSTRUCTURE FOR MICROSERVICES 1
- 2. ME Matt Stine @mstine Principal Software Engineer Pivotal [email protected] 2
- 3. I WROTE A LITTLE CLOUD BOOK... FREE - Compliments of Pivotal http://bit.ly/cloud-native-book 3
- 5. Loosely coupled service oriented architecture with bounded contexts... — Adrian Cockcroft 5
- 6. YOU MUST BE THIS TALL TO USE MICROSERVICES ▸ RAPID PROVISIONING ▸ BASIC MONITORING ▸ RAPID APPLICATION DEPLOYMENT ▸ DEVOPS CULTURE http://martinfowler.com/bliki/ MicroservicePrerequisites.html 6
- 8. PLATFORM FEATURES ▸ Environment Provisioning ▸ On-Demand Scaling ▸ Failover/Resilience ▸ Routing/Load Balancing ▸ Data Service Operations (BOSH) ▸ Monitoring 8
- 9. But we need MOAR features... 9
- 10. HARD TO ADD NEW FEATURES HARDTO MAINTAIN EXISTING FEATURES10
- 11. WHY IS IT HARD? From https://www.youtube.com/watch?v=1OkmVTFhfLY 11
- 14. THE RESULT: 14
- 15. THIS IS DIEGO 15
- 18. DESIRED TASK { ... "rootfs": "docker:///docker-org/docker-image", "env": [ {"name": "ENV_NAME_A", "value": "ENV_VALUE_A"}, {"name": "ENV_NAME_B", "value": "ENV_VALUE_B"} ], "cpu_weight": 57, "disk_mb": 1024, "memory_mb": 128, "privileged": true, "action": ACTION(s) TO RUN, ... } 18
- 19. ACTIONS ▸ RunAction: run proc in container ▸ DownloadAction: fetches and extract archive ▸ UploadAction: POST file from container to URL ▸ ParallelAction: run multiple actions in parallel ▸ SerialAction: runs multiple actions in order 19
- 20. MOAR ACTIONS ▸ EmitProgressAction: wraps action and logs progress ▸ TimeoutAction: wrap action and fail if timed out ▸ TryAction: wrap action and ignore errors 20
- 21. DESIRED LRP { ... "instances": 17, "rootfs": "VALID-ROOTFS", "env": [ {"name": "ENV_NAME_A", "value": "ENV_VALUE_A"}, {"name": "ENV_NAME_B", "value": "ENV_VALUE_B"} ], "cpu_weight": 57, "disk_mb": 1024, "memory_mb": 128, "privileged": true, "setup": ACTION, "action": ACTION, "monitor": ACTION, "start_timeout": N seconds, "ports": [8080, 5050], ... } 21
- 22. MEET THE BBS 22
- 23. LET'S HAVE AN AUCTION 23
- 24. LET'S HAVE AN AUCTION 24
- 25. LET'S HAVE AN AUCTION 25
- 31. REP RESENTING THE ACTUAL WORK 31
- 32. WHAT IS THE TRUTH? 32
- 33. WHAT IS THE TRUTH? 33
- 34. WHAT IS THE TRUTH? 34
- 35. CONVERGENCE 35
- 37. DEPLOYING CLOUD FOUNDRY IS HARD 37
- 40. LATTICE + DIEGO ▸ Adds CF GoRouter (dynamic HTTP routing) ▸ Adds Doppler (log/metrics aggregation) ▸ LOCAL: vagrant up ▸ CLOUD: terraform apply (AWS, Google, DigitalOcean) 40
- 41. DEMO41
- 42. WHAT ELSE?42
- 43. MICROSERVICES 43
- 44. NO MICROSERVICE IS AN ISLAND 44
- 45. CHALLENGES OF DISTRIBUTED SYSTEMS ▸ Configuration Management ▸ Service Registration & Discovery ▸ Routing & Load Balancing ▸ Fault Tolerance (Circuit Breakers!) ▸ Monitoring ▸ Concurrent API Aggregation & Transformation 45
- 46. 46
- 47. SPRING CLOUDDISTRIBUTED SYSTEM PATTERNS FTW! 47
- 48. SPRING CLOUD LATTICE ▸ https://github.com/spring-cloud/spring-cloud- lattice ▸ https://spring.io/guides/gs/spring-cloud-and- lattice 48
- 51. DEMO51
- 54. SPRING XD DISTRIBUTED RUNTIME 54
- 57. DEMO57
- 58. LEARN MORE! ▸ Lattice: http://lattice.cf ▸ Diego Design Notes: https://github.com/ cloudfoundry-incubator/diego-design-notes 58
- 59. THANK YOU!59