![MotivationYou often do the same thing with slight variationsCompile C++ code for different platformsTest with different browsersOr more generally, think of it as32for (x in [a,b,c]) { for (y in [d,e,f]) { …doSomethingWith(x,y,z,..); … }}](https://image.slidesharecdn.com/hudsonjavaone2010-100928131102-phpapp01/85/Hudson-java-one2010-32-320.jpg)
Hudson@java one2010
- 1. Doing More WithYour HudsonKohsuke Kawaguchi [email protected], Inc.
- 2. Hudson ProjectOpen-source CI serverWritten in Java but used for all kinds of languagesEmphasis on ease of installation and use“java -jar hudson.war” executionOr your OS-specific packageConfigure everything from browsers2https://hudson-labs.org/
- 3. EcosystemThe real reason why Hudson thrivesThe same reason why Eclipse thrivesExtensibility330+ community-developed public pluginsUp from 230+ last year in JavaOne3
- 4. Growing Hudson AdoptionEclipse Community Survey9.1% in 2009 -> 21.8% in 2010Polling by John Ferguson4
- 5. Install Base is Growing
- 6. Growing Hudson Adoption6Source: indeed.com
- 7. InfraDNAhttp://infradna.com/I left Oracle this April and started a company around HudsonSupport SubscriptionsOn open-source Hudson + pluginsOr on ICHCI: a commercial value-added distributionConsultingTroubleshooting, custom plugins, etc.Visit us at partner kiosk outside exhibit floor7
- 8. First steps to CIHudson notices that there is a changeCheck out the source codeDo some beefy computationBuildsRun testsRecord and publish resultsReport to peopleOnly when necessary8
- 9. But You Can Do So Much More!Once you automate, you can do a lotYet average Hudson users only use very small part of HudsonHence this talk9
- 10. Interconnecting Jobs aka Workflows10
- 11. Inter-Project InteractionsYou can host many projects on 1 HudsonLots of ways to have them interact with each otherIncreased parallelism is good11
- 12. Building Block #1: triggeringWhen a build is done, kick off something else“upstream”: the build that’s kicking off“downstream”: the build being kicked offTriggering is asynchronousA large graph forms a cascading behaviourUpstream doesn’t wait for downstreamDownstream can tell who is the upstream12Foo #10Bar #153
- 13. Test/Build SeparationDecompose test run into suite runsTrigger them all from buildtimeBuild #10Build #11Build #12…Test X Test YTest Z13
- 14. Test/Build Separation TechniqueDo not recompile the bits during testsInstead, copy the pre-built bitsUse the copy artifact pluginOr wget/curl via permalink14
- 15. Build/Deploy Separation1 job to build software, another to deploy itDeployment can be rerun without rebuildFailed build, new server, etc.Helps promote separate responsibilities15
- 16. Building Block #2: join pluginFan out, and then join back16timeBuild #10Build #11Build #12…Test X Test YReport Generation
- 18. Building Block #3: FingerprintsMD5 checksums recorded during buildsx.jar in product Z #112 had a bug. Where did it come from? Did it pass all tests?18
- 19. FingerprintsThe same mechanism works for …Tracking 3rd party dependenciesTracking the version of the build tools usedFingerprinting is critical when you have inter-related jobsUsed to show relationship in UIUsed to aggregate test reports19
- 20. Build PromotionSometimes plain-vanilla triggering is not desirableExample:Build XRun smoke test suites X1 and X2If they both pass, deploy X to QA siteRun QA testIf it pass, deploy X to staging site20
- 21. Build PromotionHudson’s solution: build promotionModelDefine a promotion level“Cleared Smoke tests”Define a promotion criteria“Successful completion of X1test and X2test”Define what happens after promotion“Tag the source tree and trigger Xqa”21
- 22. More Gimmicks in Build PromotionDefine multiple promotionsE.g., “cleared smoke test” & “cleared QA test”Convenient for exposing builds to progressively more expensive workPromotion criteria can be “manual approval”We often can’t automate away everything!Last promoted build gets a permalink22
- 23. Demo
- 24. Getting More From Your Maven Builds24Photo: http://www.flickr.com/photos/seeminglee/4556156477/
- 25. Maven Integration HighlightsPOM comprehensionSimplified configurationMore gets inferred automaticallySnapshot dependencies among builds25
- 26. Dealing With Large Maven BuildParallel module buildMost projects have short but wide dependency chainLet Hudson build on module-by-module basisWherever possible, build in parallel26
- 27. Dealing With Large Maven BuildIncremental buildsBuild changed modules and their descendantsSCM determines what has changed27
- 28. Private Maven RepositoryCause~/.m2/repository is not concurrent safeMultiple builds of the same projectDifferent branches, test environments, JDK, etc.ProblemArtifacts overwrite each otherArtifacts change under the carpetSolutionAllocate ~/.m2/repository per jobConsider a repository manager28
- 29. After-the-fact DeploymentDefault “mvn deploy” behavior harmfulThink about a failure in the middleBetter to let the build complete, then deploy29
- 30. Really After-the-fact DeploymentDeploy much after the build has finished30
- 32. MotivationYou often do the same thing with slight variationsCompile C++ code for different platformsTest with different browsersOr more generally, think of it as32for (x in [a,b,c]) { for (y in [d,e,f]) { …doSomethingWith(x,y,z,..); … }}
- 33. ModelDefine axesOne axis ≈ one for loopChoose from pre-defined types of AxisSlave axis: pick slaves by their names or their labelse.g., linux, solaris, and windowsGeneric axis: arbitrary values exposed as environment variables33
- 34. Matrix Project GimmicksFilteringOtherwise combinations increase exponentiallyNot all combinations make senseUse boolean expression to trim down the sizeOr tell Hudson to cut the workload to N%Hudson will thin out the combinations by itself34(label==“windows”).implies(browser==“iexplore”) &&(label==“mac”).implies(browser==“safari”)
- 35. Demo
- 36. 36Doing Selenium Tests on Hudson
- 37. Selenium GridAutomatically installs Selenium grid binariesOn all Hudson slavesAutomatically start Selenium RCs and connect themStarts Hub on Hudson master port 4444Hudson cluster now accessible as Selenium RC farm from anywhere37
- 38. Selenium Grid38Use Hudson slaves asSelenium RC nodesHudson slavesHudson master(selenium hub)
- 39. Choosing EnvironmentUse Hudson’s labels to choose the slave that runs the browser*firefoxRun Firefox somewhere (but no guarantee that the chosen slave has it)windows:*firefoxPick a slave that has the “windows” label, then run Firefox windows&sfbay:*firefoxPick a slave that has both “windows” and “sfbay” label, then run Firefox39
- 40. Testing with multiple browsersUse “matrix project” typeRun the same tests with different browsersAggregate the results40Firefox testsIE testsSafari tests
- 41. Selenium Grid w/Hudson BenefitsNo sysadmin overhead for setting up Selenium RCsEspecially nice on heterogeneous networkReuse your existing investment in Hudson cluster41
- 42. Launching browsers inside HudsonHudson typically runs without GUIaka “Headless”Browsers need GUISo you need to make GUI environment accessible to Hudson and its slaves42
- 43. On UnixUse Hudson xvncpluginImplements X protocol w/o a real GUINeeds xvnc binaries to be installed everywhereapt-get install vnc4serverHudson will start/stop xvnc and control $DISPLAY automatically43
- 44. On WindowsWindows has the concept called “WindowStation”Equivalent of X server in WindowsWinSta0is the only interactive desktopSo even headless processes can launch browsers. In theory.44
- 45. On Windows, in practiceGUI programs can hang with a dialogFor example, IE8 setup wizardThus it’s often easier to just let them access WinSta045
- 46. On Windows, really in practiceBut only SYSTEM user can interact with desktopDifferent user account creates different environments that often trip other thingsFallbackConfigure Windows for auto-loginUse start up script to start JNLP slaves46
- 47. On MacUser that runs Hudson slaves needs to be also logged in to the desktopThere appears to be no way to run GUI processes headlessly47
- 48. ConclusionsIf you aren’t using Hudson yet…What are you waiting for?If you are using Hudson…You probably can get more out of it48
- 49. Q&AKohsuke Kawaguchi [email protected], Inc.