Versioning challenges in micro services of Gwent
Download as PPTX, PDF•0 likes•248 views
This document discusses API and data versioning challenges for microservices in a game development context. It covers when and how to introduce versioning, supporting multiple concurrent versions, and dropping old versions. Key points include gradually introducing versioning after initial launch without it, using semantic versioning strategies, and duplicating database tables and foreign keys to support multiple data schemas concurrently. Orchestrating game servers for different versions and monitoring upgrades are also addressed.
Versioning challenges in micro services of Gwent
- 1. Georgii Korshunov & Krzysztof Sobczak Software Engineers at Gwent Webservices
- 2. Versioning challenges in micro services of
- 4. When should you begin API versioning?
- 5. How long should you avoid API versioning ?
- 6. GWENT webservices – GOG COVERED PARTS: • MICROTRANSACTIONS • LEADERBOARDS • MULTIPLAYER COMPONENTS • PLAYER’S PROFILES • CARD COLLECTIONS • REWARDS AND ACHIEVEMENTS
- 7. Our beginning • Only one game version NO VERSIONING
- 8. Introducing new game version • Logic changes synced with „fast” game update • Backwards compatibility NO VERSIONING
- 9. Supporting old game versions VERSIONING REQUIRED• Requirement to support multiple logic paths
- 11. Summary • Convenient way of rolling out new game versions REASONS FOR VERSIONING • Supporting old versions with different business logic • Protecting access to old/new features
- 12. API versioning
- 13. Microservices versioning Independent versions per service _version=3 _version=1,_version=3, … Global version
- 15. Background tasks
- 16. Interface http://service.gog.com/v2/endpoint?param=value http://service.gog.com/endpoint?param=value Accept: application/vnd.gog.service+json; version=2.0 X-Api-Version: 2 http://service.gog.com/endpoint?param=value&_version=2 Path Header Query
- 17. Microservices communication Synchronous — HTTP calls http://service.gog.com/endpoint?param=value&_version=3 Asynchronous — AMQP messages
- 18. Implementation Routing helper class UserController extends Controller { public function getAction($userId) {} _version=1 _version>=2 public function getAction_2($userId) {}
- 19. Implementation Strategy class with semanthic methods class ApiVersionStrategy { public function hasNewCardNames() { return $this->version >= 3; }
- 20. Dropping support for old versions • Drop support for old version ASAP • Forbid requests with old versions • Remove code and tests
- 21. Data versioning
- 22. Data versioning
- 23. Data versioning
- 24. Data versioning • API is not changing • There are several versions of data • Data versions are used by different versions of client
- 26. Interface http://service.gog.com/endpoint?_data_version=2 Query param
- 27. Implementation
- 28. Implementation • Duplicate tables of versioned entities for each version • Table names depend on current data version • Foreign keys are duplicated too • Except for foreign keys from unversioned to versioned data
- 29. Implementation • Data version is a request parameter • Hook into Doctrine metadata loader to change tablenames • Hook into Doctrine metadata cache loader
- 30. Other usages — Rankings
- 31. Other usages — Rankings
- 32. Supporting • New data versions: per-user privileges • Old data versions: forbidding access • Active data versions: readonly for data safety
- 35. Game servers
- 36. Game Server – Multiplayer • Data version • Game version Registered version includes: Additional server data: • Maximum number of players
- 38. Server Agent - deamon
- 39. Server Agent • Updates server state through HTTP MANAGEMENT • Registers the server in a server manager • Sends commands to server process with bash github.com: yohang/finite • Uses state machine for bullet proof state transitions • States include: starting, running, graceful-stopping, stopped, updating etc.
- 40. Server Manager • Provides current tasks for each servers • HTTP RESTful service • Analyzes current server requirements and spawns tasks for servers VERSION MANAGER COORDINATOR • Handles server registration & state updates • Background process • Notifies about not enough servers in the pool • Capable of auto-scaling based on the game servers usage by players
- 41. Server Manager • Manager spawns minimum number of servers • Human defines minimum/maximum number of servers per game version • Manager adjusts number of servers to cover current load SERVER PROVISIONING
- 43. Our stack CHARTS - GRAFANA METRICS - TELEGRAF LOGS - KIBANA CHECKS - ICINGA BOTS WITH UI HEADLESS BOTS DEV TESTS - DOCKER MULTIPLE TESTING ENVS
- 44. Game release monitoring Upgrade started Crossed half (players on game servers by game version)
- 45. Chart alerts game version A game version B Alerting state Recovery state
- 46. Questions?
- 47. Join us!
Editor's Notes
- #2: Jako GOG.com jesteśmy firmą działającą w przemyśle growym
- #5: Ogólny model komunikacji asynchronicznej
- #6: Ogólny model komunikacji asynchronicznej
- #8: Ogólny model komunikacji asynchronicznej
- #9: Ogólny model komunikacji asynchronicznej
- #10: Ogólny model komunikacji asynchronicznej
- #11: Ogólny model komunikacji asynchronicznej
- #14: Services updated independently Client AND Services only has to know one version number Introducing new version we make sure stuff is compatible Fallback allows for not updating other services
- #15: Protocol+Client composer repositories Microservices: passing API version in messages & calls Testing with integration tests (which we don’t have!)
- #16: No client request = no known version -> save last used user version Possibility to lock user version, disallow downgrading
- #17: Query because: versioning is optional, backward compatible, visible in access logs, easy to debug Not path because: path describes entity Not header because: proxies, HTTP cache, hacky, hard to debug
- #18: Microservices communication: passing API version in messages & calls (generic message envelope in async, context generation)
- #19: Automatically choose controller action with fallback
- #20: Semantic strategy class methods for enabling features, selecting code paths or for factories
- #21: Dropping support for old API versions: denying, removing versioning;
- #23: Diagram for game versions with different data
- #24: Diagram for game versions with different data
- #26: Passing around in messages & calls, background processes
- #27: Query parameter
- #28: Versioned and unversioned entities, duplicated tables, foreign keys lost and retained Doctrine mappings switch realtime, doctrine metadata cache User -> Versioned — manual consistency
- #29: Versioned and unversioned entities, duplicated tables, foreign keys lost and retained Doctrine mappings switch realtime, doctrine metadata cache Updating schema challenges
- #30: Versioned and unversioned entities, duplicated tables, foreign keys lost and retained Doctrine mappings switch realtime, doctrine metadata cache Updating schema challenges
- #31: Rankings
- #32: Rankings
- #33: Unit & integration tests for logic, only QA tests for data Privileges, locking from editing; removing old DV
- #36: Active servers on the left Servers in the pool on the right
- #37: Ogólny model komunikacji asynchronicznej
- #38: Ogólny model komunikacji asynchronicznej
- #39: PHP long running process -> Leszek Krupiński presentation
- #40: PHP long running process -> Leszek Krupiński presentation
- #44: Unit + functional tests for checking the logic on multiple api/data versions Isolated, allows to check if all the services integrate properly with each other Independent client which checks services API responses all the time Native game client configured to follow predefined scenarios and check for correct behavior of webservices As it has high hardware requirements we cannot run too much of them Native game client – just missing the UI part Thanks too much lower requirements we can run hordes of them (dozens of thousands) to even make internal stress tests
- #45: Ogólny model komunikacji asynchronicznej
- #46: Ogólny model komunikacji asynchronicznej
- #48: A lot of logos: Gwent, Cyberpunk, Witcher, etc