Scaling up development of a modular code base

Scaling up development of
a modular code base
Robert Munteanu, Adobe Systems

Agenda
●
Modular development
●
Source control
●
Build tool and continuous integration
●
(Integrated) development environment
●
Communication
●
Wrap-up

Modular development vs modular deployment

Modular development, monolithical deployment

Modular development and deployment

Monolithical development and modular deployment

Source control: to split or not to split

Source control: single repository
$ svn checkout https://svn.apache.org↵
/repos/asf/sling/trunk sling
# change, commit atomically
$ svn ls https://svn.apache.org↵
/repos/asf/sling/tags | wc -l
1478

Source control: multiple repositories

Source control: repo (Android)
<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<remote name="origin" fetch=".." />
<default revision="master" remote="origin" />
<project path="api" name="api.git"/>
<project path="impl" name="impl.git"/>
</manifest>
$ repo init -u git@github.com:org/manifest.git
$ repo sync
$ repo start new-feature api/ impl/
$ repo forall -c 'git push ...'

Source control: repo automation
#!/bin/sh
echo '<?xml version="1.0" encoding="UTF-8"?>'
echo ""
echo '<manifest>'
echo ' <remote name="origin" fetch="."/>'
echo ' <default revision="master" remote="origin"/>'
for repo in $(curl -s https://api.github.com/users/rombert/repos?sort=updated
| jq '.[] | .name | match ("TMP-sling-org.apache.sling.*") | .string' | tr
-d '"' | sort); do
name=${repo#TMP-sling-}
echo " <project path="${name}" name="${repo}.git"/>"
done
echo '</manifest>'

Source control: migrating to individual repositories
$ git svn clone https://svn.apache.org↵
/repos/asf/sling/trunk sling/ sling
$ git clone sling org.apache.sling.engine
$ cd org.apache.sling.engine
$ git filter-branch --subdirectory-filter bundles/engine
$ git for-each-ref --format="%(refname)" refs/original/
| xargs -n1 git update-ref -d
$ ls # only data in bundles/engine

Build tool and continuous
integration

Build tool: the aggregate build

Build tool: aggregate vs individual builds

Build tool: automating creation of module builds
def manifest = new XmlParser().parse("https://github.com/.../default.xml")
/* ... boring XML parsing ... */
modules.each { module ->
jdks.each { jdkKey ->
mavenJob(jobName(module.location, jdkKey)) {
logRotator {
numToKeep(15)
}
triggers {
snapshotDependencies(true)
scm('H/15 * * * *')
}
goals("-U clean deploy")
/* other instructions skipped */
}
}
}

(Integrated) development
environment

IDE: provisioning and project discovery

IDE: shared preferences

<dependency>
<groupId>org.acme.tooling</groupId>
<artifactId>acme-eclipse-formatter</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>eclipse-formatter</packaging>
</dependency>

<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>eclipse-preferences-maven-plugin</artifactId>
<version>0.0.3-SNAPSHOT</version>
<extensions>true</extensions>
<configuration>
<repository>
<kind>jira</kind>
<url>https://issues.apache.org/jira</url>
</repository>
<commitTemplate>${task.key} - ${task.description}nn</commitTemplate>
</configuration>
</plugin>
</plugins>

Communication: multiple channels

Communication: summarize
Subject: [report] Jenkins - 2 jobs need attention
To: dev@sling.apache.org
1. sling-ide-tooling-1.8
[...!..!.!.]
More: https://builds.apache.org/job/sling-ide-tooling-1.8/
2. sling-aggregator
[........!!]
More: https://builds.apache.org/job/sling-aggregator-1.8/

Supporting tools
●
https://github.com/jenkinsci/job-dsl-plugin
●
https://gerrit.googlesource.com/git-repo/
●
https://projects.eclipse.org/projects/tools.oomph

Examples of modular development
●
https://github.com/apache/sling-tooling-jenkins
●
https://github.com/wcm-io-devops/eclipse-maven-plugin
●
https://github.com/bdelacretaz/docker-jenkins-dsl-ready
●
https://wiki.openstack.org/wiki/MailingListEtiquette

Scaling up development of a modular code base

Recommended

More Related Content

What's hot (19)

Similar to Scaling up development of a modular code base (20)

More from Robert Munteanu (20)

Recently uploaded (20)

Scaling up development of a modular code base