Tuesday, July 31, 2012

In a small discipline, proxy repositories

Software builds on other software.  With a build system like gradle, once you declare how your code depends on other code, the build system checks your declaration with listed repositories,  and downloads appropriate packages as they are needed.  If you are coding in a JVM language, you can find an enormous proportion of the libraries you might want from maven central, either directly or via a proxy.

But if you routinely work with ancient Greek, or in any similarly specialized domain, the situation is different.  Hugh Cayless' epidoc transformer package is indispensable for my routine work, for example, but for a few minutes yesterday, the one repository where it's regularly hosted was down.  I was paralyzed.

The solution is as easy as it is obvious:  smaller communities, like those interested in ancient Greek, need to ensure that the collections of material they depend on are proxied and available from multiple repositories.

I'm using Nexus to host material developed for the Homer Multitext project, and yesterday configured it to proxy dev.papyri.info/maven, where the epidoc transcoder is housed.  The unified front to all the material hosted and proxied there is http://beta.hpcc.uh.edu/nexus/content/groups/public/.

Nexus is a "lazy" proxy:  it only acquires local copies of a proxied package when it is actually requested.  One way to guarantee that your favorite proxying site has all the packages you want is with a minimal build, that creates dependencies on everything you might want, and then simply lists their names.   The example below is a gradle build to do just this.  The repository URL and version strings for packages are kept in a separate properties file, but this example is otherwise complete:  running the showAll task will force the proxy server to retrieve any packages it does not already have locally stored.

repositories {
    maven {
        url "${repositoryUrl}"
configurations {
dependencies {
    classics group: 'edu.harvard.chs', name : 'cite', version: "${citeVersion}"
    classics group: 'edu.harvard.chs', name : 'greekutils', version: "${greekUtilsVersion}"
    classics group: 'edu.holycross.shot', name : 'hocuspocus', version: "${hocusPocusVersion}"
    classics group : 'edu.unc.epidoc', name: 'transcoder', version : "${transcoderVersion}"
task showAll  {
    description = "Downloads and shows a list of useful code libraries for classicists."
    doLast {
        println "Downloaded artifacts (including transitive dependencies):"
        configurations.classics.files.each { file ->
            println file.name

No comments: