Releasing a new version of OSPREY involves several different steps.
This is probably the most self-explanatory part, so there isn’t much to say here.
Hopfully the software we’re about to release works correctly. There are many different philosophies for testing software, but one of the most reliable methods is some kind of automated testing.
OSPREY’s automated tests can be found in the /src/test folder.
Java code tends to use the popular JUnit framework (mostly version 4).
Kotlin code tends to use the Kotest framework.
Your code IDE can help you run tests for specific parts of the codebase. Just open one of the testing source files in your IDE and find the button that launches the selected tests. For example, in IntelliJ IDEA, you should see little green arrows just to the left of the lines of test code.
Most IDEs have built-in support for JUnit. But for Kotest, you may need to install an additional plugin.
Once you’re happy with the state of the code and its status as a correctly working program,
bump the version number for OSPREY. The main version number for OSPREY is recorded in /build.gradle.kts
in a line that looks something like:
version = "4.0"
Version numbers in software usually take the form $MAJOR.$MINOR.$REVISION where $MAJOR, $MINOR, and $REVISION
are all small integers starting with 0. When $REVISION is 0, it is often omitted entirely.
Here are some example versions:
0.4
1.0
4.10.5
There are a few different philosophies for picking version numbers for software. Historically, OSPREY has used one of the oldest ones. If stated explicitly, that philosophy might look something like:
Increment the major version after we re-write most of the software and it looks totally different from before.
Increment the minor version after we add new features, or modify old ones.
Increment the revision number after we fix a bug and want to release the fix quickly.
This philosophy works rather well for projects without formal support contracts, service level agreements, or otherwise paying customers. It’s particularly well-suited to the more relaxed academic environment.
Other philosophies exist too, like the popular SemVer. However, the focus of SemVer seems to be to increase trust with downstream consumers of libraries by using the version system to signal when updates will cause breaking changes. This is a fantastic idea in theory, but in practice, it’s often hard to predict when changes in the upstream software will cause breakages in downstream software. Getting these predictions right requires enormous amounts of discipline on the part of the library developers. This is particularly hard to achieve in the academic environment, when often the developers of the software are not always professional software engineers. Even professional software engineers get SemVer wrong much more often than you’d like, so it’s important to set realistic expectations for OSPREY.
For those reasons, semantic versioning is a nice-to-have goal, but shouldn’t be considered an essential part of the release process for OSPREY.
When in doubt, the safest way to bump the version number is to just increment the minor version by one.
And remember that the next number after 9 is 10.
The version number for the OSPREY service has a strict semantic meaning, so only change it with care.
We fully expect to have older versions of OSPREY linger in the wild long after their initial release. But the OSPREY version number for the Service hosted at the DLab will not always exactly match whatever OSPREY version users are using. To handle the version mismatches, the OSPREY Service has a separate version number, and the Docker container for the OSPREY Service can host multiple versions of the service at the same time. That way, as OSPREY evolves and the Service hosted at the DLab is updated, it can still serve users using older OSPREY versions without much hassle.
Therefore, most of the time when you update OSPREY and bump its version number, you should leave the Service version number alone. That way, older OSPREY versions will still work if you re-deploy the OSPREY Service hosted at the DLab.
However, there’s one exception. If you make a change to OSPREY that would affect any results returned by the OSPREY Service, you should bump the version number of the Service too.
When you bump the Service version number and re-deploy the OSPREY Service, your new changes will be only accessible to OSPREY versions that know about the new Service version. Older OSPREY versions using the older Service version number will still see the same old results. The Docker container for the OSPREY Service will host both service versions side-by-side and OSPREY versions can pick the Service version they want to use.
Once you’ve found a new version number that brings you joy, commit your change to the git main branch,
or merge your commits into the main branch. Then add a tag to your commit with the new joyful
version number. For example, your tag might be named v4.1.
Once your code is saved for all eternity in Git, the next step is to copy the code to the public
Git repository for OSPREY on GitHub using the usual git push
mechanism.
The code will need to be publicly available for the next steps, which use Continuous Integration (CI) services hosted in The Cloud. If truth-be-told, OSPREY development doesn’t actually follow the philosophies of continuous integration very much at all. However, CI services are usually pretty good at doing cross-platform builds, which OSPREY relies on to make life easier for developers that may only have access to the one platform that happens to be running on their laptop.
Once your code is saved for all eternity in Git, the next step is to build OSPREY and create the release files.
OSPREY uses Azure Pipelines to do cross-platform builds. To use it, you’ll need to log in.
First, log into the DLab site on Azure. When prompted to enter your username,
enter your Duke NetID followed by @duke.edu. For example, id123@duke.edu.
If you already have a Microsoft account, don’t try to use it here. It won’t work. You must use a Duke account.
You may need to request access from the Azure organization owner first though. If approval is needed, the Azure website should walk you through those steps during the login process. If you still need assistance, try asking the Azure organization owner for help.
Once logged in, you should see the OSPREY project and its pipelines. Specifically, we’ll be using the donaldlab.OSPREY4.release pipeline.
Before you can start the builds for OSPREY, you’ll need to install the Azure CLI.
Once installed, you can verify that the installation worked by running the azureCheckCli Gradle task.
If successful, you should see a message like:
Logged into Azure as: $YOUR_NETID@duke.edu
where $YOUR_NETID is your Duke NetID.
If you logged into the website in the previous steps, you should already be logged into the CLI too. If for some reason the CLI thinks you’re not logged in, then simply log in on the CLI directly with the shell command:
az login --allow-no-subscriptions
The --allow-no-subscriptions flag is needed to prevent a warning when logging in. Something something SSO.
It’s magic. Who really knows how it works.
Once the Azure CLI is installed and working, actually start the OSPREY builds with the azureBuild task.
This task uses the Azure CLI to start the build using your logged-in user.
Or, you could login to the Azure Devops website, click on OSPREY project, and manually run the donaldlab.OSPREY4.release pipeline.
These two options do the same thing.
After starting, you can watch the progress of the builds on the donaldlab.OSPREY4.release website. The logs there are particularly helpful when troubleshooting problems.
The builds take 10-20 minutes to finish, so keep working dilligently on other tasks in parallel so as not to waste precious time advancing towards your PhD. Or, you know, just amuse yourself until it’s done. I’m not the boss of you.
When the build finishes, Azure well helpfully send you an email notification to your Duke email address.
It might be $YOUR_NETID@duke.edu, or $YOUR.$NAME@duke.edu. Who could know for sure. Once you get the email,
feel free to check the status of the build.
If you get some kind of red error indicator, then (at least part of) the build failed. Dive into the logs to
find out why. The reason might be something wrong with your code. Or the reason might be something wrong with the
pipeline itself. If you need to modify the pipeline itself for some reason, you can do that by editing the
/release-pipelines.yml file. That file is helpfully self-documenting, so it should help you get started
making changes to the pipeline itself if that’s what you need.
On the other hand, if you got all green indicators, then the build succeeded and you can continue with the next steps!
Once the build finishes successfully, run the azureDownloadArtifacts Gradle task. This will download all the
release files from the most recent build in Azure (hopefully the one you just ran) into your /build/releases folder.
After running this task, your local git clone will look like you built all the relases locally, even the releases
for different platforms.
Run the buildDocsRelease Gradle task to build the documentation that goes with your new OSPREY release.
Run the archiveReleases task to upload all the shiny new releases to the DLab release archive.
This will archive all the releases built by Azure, as well as the new documentation you just built.
This step will make all the new release files available for download by the public too, but they won’t have links on
the website yet.
Which brings us to …
The last step is to update the OSPREY website. This is as simple as running a few more Gradle tasks (in order):
downloadDocsReleasesbuildWebsitedeployWebsiteBe sure to read updating the website before updating the website, to get an overall understanding of how the OSPREY documentation system and the website tools work.