When we first devised the setup for implementing our repository there were several requirements that we considered and addressed. The first requirement is that we have to tightly control all libraries that are present in repositories used for our release builds, at all times, and the introduction of any new libraries into the mix had to be a tightly controlled event. A second requirement is that for libraries that aren't 3rd party libraries the source code that generated that repository artifact must be available and mapped directly to that artifact. A third requirement is that we need multiple repositories including snapshot, development, and release repositories to adequately address our development, test, and production implementation processes. The final requirement is that the artifacts should be downloadable through a web based URL and that end development staff should be able to browse the contents of this repository in a browser.
When considering these requirements as whole, implementing this as a filesystem based repository backed up Clearcase by project view really jumped out at me. Clearcase would not be my first choice as it relates to versioning system. I would prefer to use SVN or Git as my versioning system to implement this. That being said, the standard versioning system we are required to use is Clearcase. And despite the holy wars that go on over what the best versioning system is, Clearcase will implement this just fine as it provides the functionality to address three of the four requirements above.
The first requirement is quite obviously addressed by using a Clearcase view to back a file system based repository on a server where only release management has access. The italicized bold part is a key qualifier, as you'll come to see when I get to the part about addressing http based download and navigation. Implementing your repository, or repositories as i'll call out soon, in this manner very clearly address the issue of tightly controlling what libraries go into the repository and thus tightly controlling what libraries can be consumed by a build.
The second requirement we mentioned is needing to map the exact set of source files that produced the artifact to the artifact itself. This is accomplished by bundling the source files at build time and associating these source files as a source artifact in the repository. As part of the artifact check in, the source jar is checked in along with it, and voila ... the source files that spawned this artifact are directly mapped to it without painful searching through Clearcase.
While tightly controlling what is going in and out of the repository and mapping the source files to artifact they created, using Clearcase views to implement filesystem based repositories also addresses the issue of being able to manage multiple repositories in an auditable way. We mentioned needing three repositories, a release repository, a development repository, and a snapshot repository. Clearcase can easily handle the development and release repository cases by having a release repository branch and a development repository branch.
The process of an artifact moving into the release repository starts with it first being installed into the development repository. It is then consumed in an integration build. This integration build is then passed off to test and validation is done. Once the testing is complete and the end product is validated, the libraries can be promoted to the release repository. This is accomplished by merging these artifacts from the development repository Clearcase branch into the release repository Clearcase branch. Once these artifacts are merged into the release repository branch, and the release build can be done in preparation for production acceptance testing and validation.
While Clearcase handles all of the above well, it does not handle a few things which needed to be done separately. These consisted of defining a snapshot repository, keeping the snapshot views up to date, and exposing their content through http. Fortunately, all were easily solvable.
As it relates to providing a snapshot repository, this is intended to be a latest and greatest repository that is not versioned and only holds the custom application libraries we developed. This is a filesystem repository like the others, the key difference is it's not backed by Clearcase. The libraries are built via continuous integration and published to this filesystem, and developers can pull these latest versions at their leisure. There is nothing major here.
While continuous integration is used to generate the libraries for the snapshot repositories, it is also used to solve the problem of keeping the filesystem repositories backed by Clearcase views up to date. We have continuous integration update the repository view when it detects changes, and thus that problem goes away. We opted to not implement this with Clearcase Dynamic Views, as they tend to be painful and difficult at times ... namely if the connection to the Clearcase server goes down your entire repository is hosed.
Finally, to expose this via http we simply setup an Apache instance in front of each of the repositories. We set the appropriate mappings to the appropriate filesystem locations, and all works great. You can use URL based resolution that is typically used when resolving Maven/Ivy artifacts, and everything works as it should. Your end state ends as follows....
This is a high level of how to setup a Maven/Ivy repository for your organization. I will be posting follow up articles on how the build process plays into this, how continuous integration works into this, and how to specifically build your artifacts and push them to these various different repositories and consume them in your builds.
No comments:
Post a Comment