Introduction to Continuous integration

Editor's Notes

• #2: This presentation’s title is “Continuous Integration” and is meant for introducing the basic concepts of CI to a development team.As CI is tightly related to other concepts (such as automated builds, unit testing), these will be brought here as well.
• #3: Main agenda for this presentation.Continuous Integration is directly tied with other concepts such as unit testing, automated build and tools that perform various verifications to maintain high code quality.Although not directly related, I’d like to present Git and its advantages as a Distributed version control system (DVCS).
• #4: One problem with measuring how ‘good’ a code project is, is that often the measurement is objective and not based on solid data.Team members integrate their work not often enough, leading to complex merging and integration of their work.Lack of code that tests the codebase (unit tests) brings us to a state that we “believe” our code is working, however as we progress and the project becomes more complex, we cannot be certain that everything is works as expected.
• #5: What we would like to achieve is:Have criteria to tell us if our project is doing good.Integrate often to prevent complicated integration points between developers on a team.Create small pieces of code that test and exercise the code of the project we’re working on.
• #6: Definition of Continuous Integration, as brought by Martin Fowler’s essay on the subject.The full essay can be found here: http://martinfowler.com/articles/continuousIntegration.html
• #7: Without CI, a developer had to update to latest code before committing, and verify (locally) that all source code compiles and tests are OK.Using CI, the build server will build the project from the latest code checked in, and report if anything went wrong.Typically this is done quickly enough (recommended is less than 10 minutes).
• #8: Automated builds are what allows CI to be effective.It is essential that the entire project can be built using some script in a single command.Setting this up involves a few other steps, such as keeping ALL files related to building the project under source control (for example, external 3rd party libraries, configuration files, images, etc.).This removes the need to do manual work that only certain team members have or the risk of a human error when running any manual step.This also has an added value that new team members can directly build the project without installing any software on their environments, and build servers do not need any special prerequisite software for running a build.
• #9: Multiple technologies exist for constructing a build script (simple batch files, MSBuild, Nant).Upon check in (commit) to the source control system, the smallest subset of actions need to be performed, in order to deliver results as fast as possible.To this end, it is typical to set up multiple building scenarios that are invoked at different times. For example, a “small” build runs for every check in to source control.Every night a “Full” build runs that builds an MSI, documentation runs an extensive test suite, etc.
• #10: These are examples of tools that are very easy to integrate into the build.Some of them ship with an MSBuild task to be directly invoked from a build script.Most also integrate well into popular CI server software.
• #11: Code coverage shows in numbers how much of the code is actually covered.A typical scenario for this is when running unit tests, monitoring the actual code that runs during the tests.This generates a report of how much of the codebase was actually exercised during unit testing.A report is presented showing how many classes were used, methods were called, etc.
• #12: Source code highlighting shows which lines of code get called or not.This can also be integrated into the IDE (Visual Studio) to show what part of the code was used.Areas shown in red should be exercised by a unit test.
• #13: StyleCop is a tool that scans the code, enforcing a set of predefined rules.There are many styling rules, not all of them are mandatory for a consistent and high quality code.Once a set of rules is defined, it is stored in a StyleCop.settings file.This file can be placed at the root folder of a project, affecting all folders under it.StyleCop can be configured to output build errors instead of warnings. This can ensure all code checked into the source control repository is consistent with the team’s rules.
• #14: These are examples of tools that are very easy to integrate into the build.Some of them ship with an MSBuild task to be directly invoked from a build script.Most also integrate well into popular CI server software.
• #15: Older generation VCS tools were mostly centralized. A central server used to hold the master repository.Every action had to pass through this server, making it potentially slow, depending on the networking protocol used, and the proximity to the server.
• #16: Modern tools like Git and Hg are distributed.Every developer gets a copy of the entire repository, maintaining it locally.A central server can be setup to sync between team members.In this method, a developer can develop a new feature locally, check in changes as he makes progress.When the code reaches a certain stage where it’s stable enough, the developer can submit (push) the changes to a central server, making it visible for other team members.
• #17: Git originates from Linux.On Windows, It can be used in various different ways:Command line interface.Shell (explorer) extensionVisual Studio integration.
• #18: Central server, developers access and update it.Taken from: http://progit.org/book/ch5-1.htmlIn centralized systems, there is generally a single collaboration model—the centralized workflow. One central hub, or repository, can accept code, and everyone synchronizes their work to it. A number of developers are nodes — consumers of that hub — and synchronize to that one place (see Figure 5-1).This means that if two developers clone from the hub and both make changes, the first developer to push their changes back up can do so with no problems. The second developer must merge in the first one’s work before pushing changes up, so as not to overwrite the first developer’s changes. This concept is true in Git as it is in Subversion (or any CVCS), and this model works perfectly in Git.If you have a small team or are already comfortable with a centralized workflow in your company or team, you can easily continue using that workflow with Git. Simply set up a single repository, and give everyone on your team push access; Git won’t let users overwrite each other. If one developer clones, makes changes, and then tries to push their changes while another developer has pushed in the meantime, the server will reject that developer’s changes. They will be told that they’re trying to push non-fast-forward changes and that they won’t be able to do so until they fetch and merge. This workflow is attractive to a lot of people because it’s a paradigm that many are familiar and comfortable with.
• #19: Taken from: http://progit.org/book/ch5-1.html#integrationmanager_workflowBecause Git allows you to have multiple remote repositories, it’s possible to have a workflow where each developer has write access to their own public repository and read access to everyone else’s. This scenario often includes a canonical repository that represents the “official” project. To contribute to that project, you create your own public clone of the project and push your changes to it. Then, you can send a request to the maintainer of the main project to pull in your changes. They can add your repository as a remote, test your changes locally, merge them into their branch, and push back to their repository. The process works as follow (see Figure 5-2):The project maintainer pushes to their public repository.A contributor clones that repository and makes changes.The contributor pushes to their own public copy.The contributor sends the maintainer an e-mail asking them to pull changes.The maintainer adds the contributor’s repo as a remote and merges locally.The maintainer pushes merged changes to the main repository.This is a very common workflow with sites like GitHub, where it’s easy to fork a project and push your changes into your fork for everyone to see. One of the main advantages of this approach is that you can continue to work, and the maintainer of the main repository can pull in your changes at any time. Contributors don’t have to wait for the project to incorporate their changes — each party can work at their own pace.
• #20: For first implementation, an automated build + CI server is setup.Software used:Source Control – GitCI – TeamCity (free for small teams)Build Agent is on the same server as TeamCity. This is the actual component that runs the build. For scalability, many agents can be deployed on additional servers, making it possible to have multiple concurrent builds.Two build configurations are created:CI Build – triggered on every check in to the source control. Very quick (~ 1 minute).Nightly Build – slightly longer, also builds an installer using NSIS build script (~ 5 minutes).
• #21: Implementation #1 is not perfect yet, but it gives us a solid framework to build upon.Tasks that were manual before (such as creating an installer) are replaced with a 1-click automatic, repeatable action.A developer submitting new code can receive in ~ 1 minute a notification that his code passed some basic tests and is considered good.
• #22: Second implementation expands on the first one.In this example, more build agents are introduced, and can accommodate more concurrent builds by a larger development team.After passing all initial tests, a build is passed on to a VM server, where it is automatically deployed to virtual machines, according to the needs of the built applications.For example, a client application can be deployed to multiple client operating systems, where a set of automated integration tests occur.