Smart Repository Management (openBlend 2011, Ljubljana, Slovenia)
1 like679 views
The document discusses the evolution and benefits of smart repository management using Nexus, highlighting issues like bandwidth waste, artifact discovery, and security concerns in software development. It emphasizes features such as proxy and hosted repositories, indexing, and artifact procurement to streamline processes and reduce errors. Additionally, it covers the integration of various repository types and provides guidance on effective consumption and management of repositories.
Smart Repository Management (openBlend 2011, Ljubljana, Slovenia)
- 1. Smart Repository Management With Nexus MMXI. Ljubljana, Slovenia Transforming Software Development Thursday, September 22, 2011
- 2. Smart Repository Management The Inception 2005 Transforming Software Development Thursday, September 22, 2011
- 3. Smart Repository Management The Inception Bandwidth waste and congestion Depending on network and external repository up-times Unsolved “hosting” and collaboration Exposed to mismanaged repositories Transforming Software Development Thursday, September 22, 2011
- 4. Smart Repository Management The Inception Laborious discovery of artifacts, plugins Infrastructure maintenance hell (deploying, site publishing) Non-trivial entry for new developers Key-based security – who entered the office had full access Transforming Software Development Thursday, September 22, 2011
- 5. Smart Repository Management Proxy Repositories Pluggable “remote” transport, caching locally Isolation from outer world Crossing two worlds: mirrors (content) and proxies (protocol) Get what you need, when you need it Transforming Software Development Thursday, September 22, 2011
- 6. Smart Repository Management Hosted Repositories Pluggable “local” storage Simplify the needed infrastructure Get all content treated same way Keep your artifacts properly separated Transforming Software Development Thursday, September 22, 2011
- 7. Smart Repository Management Grouping “Aggregating” multiple repositories into one Hiding the actual source and URL(s) of Repositories Order of processing in a group is important Became de facto “standard” setup Transforming Software Development Thursday, September 22, 2011
- 8. Smart Repository Management Typical Usage Open Source Ghetto Terracotta Codehaus Apache JBoss Nexus Dev Test Production QA Precious Developers Your Ghetto Transforming Software Development Thursday, September 22, 2011
- 9. Smart Repository Management Grouping Optimizations Is corollary of Grouping Based on repository policy Based on repository routes Parallel processing Or ultimate: stop using grouping! (today not quite possible) Transforming Software Development Thursday, September 22, 2011
- 10. Smart Repository Management Bridging and Mediation Do it transparently First application was to support Maven1 repositories Layout changes (Maven1 – Maven2 bidi) Content changes (GEM, OSGi recipes) Transforming Software Development Thursday, September 22, 2011
- 11. Smart Repository Management Indexing Pluggable Index Creators (POM, Classes, OSGi, ...) Incremental updating and publishing De facto standard, embraced by all IDEs, MRMs... Source donated to Apache Maven Project as Maven Indexer Transforming Software Development Thursday, September 22, 2011
- 12. Smart Repository Management Security Integrated Apache Shiro Supporting XML, LDAP, Atlassian Crowd, URL delegation... Content secured using Repository Targets Pluggable Realms Transforming Software Development Thursday, September 22, 2011
- 13. Smart Repository Management Staging and Promotion Dogma of Maven is immutable releases, no recall once deployed To not release incremental versions, or use temporary locations Single deployment URL, just set it in organization POM Notification and Automation Transforming Software Development Thursday, September 22, 2011
- 14. Smart Repository Management Artifact Procurement Before was laborious, error prone and difficult Firewall with user extensible rules Strategies: “Ask First”, “Check Later” Easy integration Transforming Software Development Thursday, September 22, 2011
- 15. Smart Repository Management Conclusions What We Learned Transforming Software Development Thursday, September 22, 2011
- 16. Smart Repository Management Maven Fact Maven is usually wrongly recognized as a “CLI build tool”, but is much more than that, it’s infrastructure. Transforming Software Development Thursday, September 22, 2011
- 17. Smart Repository Management Refreshing Diversity – Unified! Maven Artifact Repository Static Sites Repository Eclipse P2 Bundle Repository Apache Felix OSGi Bundle Repository Ruby GEM Repository OSGi Recipes Anything we missed? Transforming Software Development Thursday, September 22, 2011
- 18. Smart Repository Management Why do you need it? Hosts what your build outputs Proxies what your build consumes Bridges and mediates between different kind of repositories Has motto “no content duplication” encoded in it’s DNA Eases “every day routine” Gives tools to implement complex workflows Easy integration using REST API Transforming Software Development Thursday, September 22, 2011
- 19. Smart Repository Management What to proxy (outbound)? Do not proxy “aggregated” repositories (information loss) Do not proxy “staging” repositories (releases disappears) Do not proxy snapshot repositories, unless a must Transforming Software Development Thursday, September 22, 2011
- 20. Smart Repository Management How to consume (inbound) – today Groups in Nexus are cheap Groups of groups Per project groups Groups per developers, CI and public Transforming Software Development Thursday, September 22, 2011
- 21. Smart Repository Management How to consume (inbound) – future Mirror of all – The typical setup since Proximity, has it’s known problems, information loss Mirror per Repository – The most precise control, but not maintainable Dynamic mirror discovery – Intelligent mirror selection and proper fallback, magic on both ends, no information loss HTTP Proxy – Simple maintenance, still needs user side configuration, but magic happens on Nexus side, no information loss Transforming Software Development Thursday, September 22, 2011
- 22. Smart Repository Management Repositories in POM, yes or no? No doubt: yes Transforming Software Development Thursday, September 22, 2011
- 23. Smart Repository Management In-house Snapshots Mirko SCM CI Maven Nx Radirko Local Maven Repo Transforming Software Development Thursday, September 22, 2011
- 24. Smart Repository Management Third Party Snapshots “Freeze” them at least, but better is to build from source Transforming Software Development Thursday, September 22, 2011
- 25. Smart Repository Management Central Repository Facts and Stats Transforming Software Development Thursday, September 22, 2011
- 26. Smart Repository Management Transforming Software Development Thursday, September 22, 2011
- 27. Smart Repository Management Transforming Software Development Thursday, September 22, 2011
- 28. Smart Repository Management Transforming Software Development Thursday, September 22, 2011
- 29. Smart Repository Management Questions? Tamas Cservenak Sonatype Nexus Lead Developer http://twitter.com/cstamas Thanks, ~t~ Transforming Software Development Thursday, September 22, 2011