Using TravisCI to deploy to Maven Central via Git tagging (aka death to commit clutter)

jbool_expressions is a small OSS library I maintain.  To make it easy to use, artifacts are published to Maven Central.  I have never been happy with my process for releasing to Maven Central; the releases were done manually (no CI) on my laptop and felt fragile.

I wanted to streamline this process to meet a few requirements:

• All deployments should happen from a CI system; nothing depends on laptop state (I don’t want to lose my encryption key when I get a new laptop)
• Every commit to master should automatically publish a SNAPSHOT artifact to Sonatype (no manual snapshot release process)
• Cutting a proper release to Maven Central, when needed, should be straightforward and hard to mess up
• Performing releases should not generate commit clutter.  Namely, no more of this:

Luckily for me, we recently set up a very similar process for our OSS projects at LiveRamp.  I don’t want to claim I figured this all out myself — others at LiveRamp (namely, Josh Kwan) were heavily involved in setting up the encryption parts for the LiveRamp OSS projects which I used as a model.  

There’s a lot of information out there about Maven deploys, but I had trouble finding a simple guide which ties it all together in a painless way, so I decided to write it up here.

tl,dr: through some TravisCI and Maven magic, jbool_expressions now publishes SNAPSHOT artifacts to Sonatype on each master commit, and I can now deploy a release to Maven Central with these commands:

$ git tag 1.23
$ git push origin 1.23

To update and publish the next SNAPSHOT version, I can just change and push the version:

$ mvn versions:set -DnewVersion=1.24-SNAPSHOT
$ git commit -am "Update to version 1.24-SNAPSHOT"
$ git push origin master

At no point is anything auto-committed by Maven; the only commits in the Git history are ones I did manually.  Obviously I could script these last few steps, but I like that these are all primitive commands, with no magic scripts which could go stale or break halfway through.

The thousand-foot view of the CI build I set up for jbool_expressions looks like this:

• jbool_expressions uses TravisCI to run tests and deploys on every commit to master
• Snapshots deploy to Sonatype; releases deploy to Maven Central (via a Sonatype staging repository)
• Tagged commits publish with a version corresponding to the git tag, by using the versions-maven-plugin mid-build.

The rest of this post walks through the setup necessary to get this all working, and points out the important files (if you’d rather just look around, you can just check out the repository itself)

Set up a Sonatype account

Sonatype generously provides OSS projects free artifact hosting and mirroring to Maven Central.  To set up an account with Sonatype OSS hosting, follow the guide here.  After creating the JIRA account for issues.sonatype.org, hold onto the credentials — we’ll use to use those later to publish artifacts.

Creating a “New Project” ticket and getting it approved isn’t an instant process; the approval is manual, because Maven Central requires artifact coordinates to reflect actual domain control.  Since I wanted to publish my artifacts under the com.bpodgursky coordinates, I needed to prove ownership of bpodgursky.com.  

Once a project is approved, we have permission to deploy artifacts two important places:

• https://oss.sonatype.org/content/repositories/snapshots, which lets us publish snapshot artifacts
• https://oss.sonatype.org/service/local/staging/deploy/maven2/, which lets us deploy artifacts we can promote to Maven Central.

We’ll use both of these later.

Set up TravisCI for your GitHub project.

TravisCI is easy to set up.  Follow the documentation here.

Configure Maven + Travis to publish to central 

Once we have our Sonatype and Travis accounts enabled, we need to configure a few files to get ourselves publishing to central:

• pom.xml
• .travis/maven-settings.xml
• .travis.yml
• .travis/gpg.asc.enc
• deploy

I’ll walk through the interesting parts of each.

pom.xml

Most of the interesting publishing configuration happens in the “publish” profile here.  The three maven plugins — maven-javadoc-plugin, maven-source-plugin, and maven-gpg-plugin — are all standard and generate artifacts necessary for a central deploy (the gpg plugin requires some configuration we’ll do later).  The last one — nexus-staging-maven-plugin — is a replacement for the maven-deploy-plugin, and tells Maven to deploy artifacts through Sonatype.

<distributionManagement>
  <snapshotRepository>
    <id>ossrh</id>
    <url>https://oss.sonatype.org/content/repositories/snapshots</url>
  </snapshotRepository>
  <repository>
    <id>ossrh</id>
    <url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
  </repository>
</distributionManagement>

Fairly straightforward — we want to publish release artifacts to Maven central, but SNAPSHOT artifacts to Sonatype (Maven central doesn’t accept SNAPSHOTs)

.travis/maven-settings.xml

The maven-settings here are configurations to be used during deploys, and inject credentials and buildserver specific configurations:

<servers>
  <server>
    <id>ossrh</id>
    <username>${env.OSSRH_USERNAME}</username>
    <password>${env.OSSRH_PASSWORD}</password>
  </server>
</servers>

The Maven server configurations let us tell Maven how to log into Sonatype so we can deploy artifacts.  Later I’ll explain how we get these variables into the build.

<profiles>
  <profile>
    <id>ossrh</id>
    <activation>
      <activeByDefault>true</activeByDefault>
    </activation>
    <properties>
      <gpg.executable>gpg</gpg.executable>
      <gpg.keyname>${env.GPG_KEY_NAME}</gpg.keyname>
      <gpg.passphrase>${env.GPG_PASSPHRASE}</gpg.passphrase>
    </properties>
  </profile>
</profiles>

The maven-gpg-plugin we configured earlier earlier has a number of user properties which need to be configured.  Here’s where we set them.

.travis.yml

.travis.yml completely defines how and when TravisCI builds a project.   I’ll walk through what we do in this one.

language: java
jdk:
- openjdk10

The header of .travis.yml configures a straightforward Java 10 build.  Note, we can (and do) still build at a Java 8 language level, we just don’t want to use a deprecated JDK.  

install: "/bin/true"
script:
- "./test"

Maven installs the dependencies it needs during building, so we can just disable the “install” phase entirely.  The test script can be inlined if you prefer; it runs a very simple suite:

mvn clean install -q -B

(by running through the “install” phase instead of just “test”, we also catch the “integration-test” and “verify” Maven phases, which are nice things to run in a PR, if they exist).

env:
  global:
  - secure: ...
  - secure: ...
  - secure: ...
  - secure: ...

The next four encrypted secrets were all added via travis.  For more details on how TravisCI handles encrypted secrets, see this article.  Here, these hold four variables we’ll use in the deploy script:

travis encrypt OSSRH_PASSWORD='<sonatype password>' --add env.global
travis encrypt OSSRH_USERNAME='<sonatype username>' --add env.global
travis encrypt GPG_KEY_NAME='<gpg key name>' --add env.global
travis encrypt GPG_PASSPHRASE='<pass>' --add env.global

The first two variables are the Sonatype credentials we created earlier; these are used to authenticate to Sonatype to publish snapshots and release artifacts to central. The last two are for the GPG key we’ll be using to sign the artifacts we publish to Maven central.  Setting up GPG keys and using them to sign artifacts is outside the scope of this post; Sonatype has documented how to set up your GPG key here, and how to use it to sign your Maven artifacts here.

Next we need to set up our GPG key.  To sign our artifacts in a Travis build, we need the GPG key available.  Again, set this up by having Travis encrypt the entire file:

travis encrypt-file .travis/gpg.asc --add

In this case, gpg.asc is the gpg key you want to use to sign artifacts.  This will create .travis/gpg.asc.enc — commit this file, but do not commit gpg.asc.   

Travis will have added a block to .travis.yml that looks something like this:

before_deploy:
- openssl aes-256-cbc -K $encrypted_f094dd62560a_key -iv $encrypted_f094dd62560a_iv -in .travis/gpg.asc.enc -out .travis/gpg.asc -d

Travis defaults to “before_install”, but in this case we don’t need gpg.asc available until the artifact is deployed.

deploy:
-
  skip_cleanup: true
  provider: script
  script: ./deploy
  on:
    branch: master
-
  skip_cleanup: true
  provider: script
  script: ./deploy
  on:
    tags: true

Here we actually set up the artifact to deploy (the whole point of this exercise).  We tell Travis to deploy the artifact under two circumstances: first, for any commit to master; second, if there were tags pushed.  We’ll use the distinction between the two later.  

deploy

The deploy script handles the actual deployment. After a bit of input validation, we get to the important parts:

gpg --fast-import .travis/gpg.asc

This imports the gpg key we decrypted earlier — we need to use this to sign artifacts.

if [ ! -z "$TRAVIS_TAG" ]
then
    echo "on a tag -> set pom.xml <version> to $TRAVIS_TAG"
    mvn --settings "${TRAVIS_BUILD_DIR}/.travis/mvn-settings.xml" org.codehaus.mojo:versions-maven-plugin:2.1:set -DnewVersion=$TRAVIS_TAG 1>/dev/null 2>/dev/null
else
    echo "not on a tag -> keep snapshot version in pom.xml"
fi

This is the important part for eliminating commit clutter.  Whenever a tag is pushed — aka, whenever $TRAVIS_TAG exists — we use the versions-maven-plugin to temporarily set the project’s version to that tag. Specifically, 

$ git tag 1.23
$ git push origin 1.23

The committed artifact version in pom.xml doesn’t change. It doesn’t matter what the version is in pom.xml in master — we want to publish this version as 1.23.

mvn deploy -P publish -DskipTests=true --settings "${TRAVIS_BUILD_DIR}/.travis/mvn-settings.xml"

Last but not least, the actual deploy.  Since we configured our distributionManagement section above with different snapshot and release repositories, we don’t need to think about the version anymore — if it’s still SNAPSHOT (like in the pom), it goes to Sonatype; if we pushed a release tag, it’s headed for central.

That’s it!

Before this setup, I was never really happy with my process for cutting a release and getting artifacts into Maven Central — the auto-generated git commits cluttered the history and it was too easy for a release to fail halfway.  With this process, it’s almost impossible for me to mess up a release. 

Hopefully this writeup helps a few people skip the trial-and-error part of getting OSS Java artifacts released. Let me know if I missed anything, or there’s a way to make this even simpler.