For an administrator

We assume that you are already familiar with Vault and know how to use it, so let us skip the details of setting it up. We also assume that Vault is configured according to the official documentation.

Vault

There are a couple of ways to install Vault and the trdl plugin. The easiest one is to use the ready-made Vault binary (you can get one at the Vault website or install it using your distribution’s package manager) and the ready-made trdl plugin binary.

Docker

Install Docker. Add a Vault user to the Docker group:

usermod -a -G docker vault

Setting up the project

Git repository

Create a regular public Git repository.

Bucket

Any S3-compatible bucket will do. It should be publicly available for reading.

A note about GCS (Google Cloud Storage)

To get rid of the An error occurred (AccessDenied) when calling the CreateMultipartUpload operation: Access denied error, make sure that the Service Account used to access the bucket belongs to the Storage Admin role.

Installing the plugin

Download the trdl plugin by following the instructions in the message of the selected release. Copy it to /etc/vault.d/plugins or another directory where you normally store plugins.

Configuring the plugin

Setting up Vault includes specifying the directory where the plugins are stored:

plugin_directory = "/etc/vault.d/plugins"

Restart Vault.

Register the plugin in Vault:

vault plugin register -sha256=$(sha256sum /etc/vault.d/plugins/vault-plugin-secrets-trdl | awk '{print $1}') secret vault-plugin-secrets-trdl

In our case, the plugin file is called vault-plugin-secrets-trdl, and we register it under the same name in Vault. Refer to the official documentation to learn more about registering plugins.

Enable the plugin as a secrets engine at a specific path in Vault:

vault secrets enable -path=trdl-test-project vault-plugin-secrets-trdl

You can enable the same plugin many times; however, you must use a unique path each time. For more information, refer to the official documentation.

Now let’s configure the trdl plugin itself. We will use the /configure API method to do this:

vault write trdl-test-project/configure @configuration.json

where configuration.json has the following contents:

{
  "s3_secret_access_key": "FOO",
  "s3_access_key_id": "BAR",
  "s3_bucket_name": "trdl-test-project-tuf",
  "s3_region": "europe-west1",
  "s3_endpoint": "https://storage.googleapis.com",
  "git_repo_url": "https://github.com/werf/trdl-test-project",
  "required_number_of_verified_signatures_on_commit": 2
}

When configuring the plugin, you must specify the minimum number of GPG signatures required for a commit (required_number_of_verified_signatures_on_commit). Otherwise, the release system becomes vulnerable: any user with access can tamper with it because the system is not protected by a quorum.

The minimum number of GPG signatures required (required_number_of_verified_signatures_on_commit) depends on the size and scope of the team, frequency of updates, and other factors.

Managing public parts of trusted GPG keys

The /configure/trusted_pgp_public_key group of API methods is used to handle the public parts of trusted GPG keys.

For a developer

Setting up a GPG signature in Git

Git has a mechanism for signing new tags (releasing) and individual commits (publishing). As a result, the GPG signature becomes an integral part of the Git tag or Git commit. However, this approach supports only one signature.

The signatures plugin allows you to sign Git tags and Git commits after they are created. In this case, GPG signatures are stored in Git notes. You can use as many signatures as you want, and you can also delete previously used signatures without affecting the linked Git tag or Git commit in any way.

All you need to do is set up GPG and Git correctly to create GPG signatures. This manual can help you.

Installing the signatures plugin

To use the plugin, you have to install it to an arbitrary directory in PATH (e.g., ~/bin):

git clone https://github.com/werf/third-party-git-signatures.git
cd third-party-git-signatures
install bin/git-signatures ~/bin

After running the git signatures command you should see the plugin description.

git signatures <command> [<args>]

Git Signatures is a system for adding and verifying one or more PGP
signatures to a given git reference.

Git Signatures works by appending one of more signatures of a given
ref hash to the git notes interface for that ref at 'refs/signatures'.

In addition to built in commit signing that allows -authors- to sign,
Git Signatures allows parties other than the author to issue "approval"
signatures to a ref, allowing for decentralized cryptographic proof of
code review. This is also useful for automation use cases where CI
systems to be able to add a signatures to a repo if a repo if all tests
pass successfully.

In practice Git Signatures allows for tamper evident design and brings
strong code attestations to a deployment process.

Commands
--------

* git signatures init
    Setup git to automatically include signatures on push/pull

* git signatures import
    Import all PGP keys specified in .gitsigners file to local
    GnuPG keychain allowing for verifications.

* git signatures show
    Show signatures for a given ref.

* git signatures add
    Add a signature to a given ref.

* git signatures verify
    Verify signatures for a given ref.

* git signatures pull
    Pull all signatures for all refs from origin.

* git signatures push
    Push all signatures for all refs to origin.

* git signatures version
    Report the version number.

Configuring the build process

As a basic example of creating and arranging release artifacts for multiple platforms, let’s deliver the script that outputs a release tag when run.

All build parameters, such as environment and build instructions, are defined in the trdl.yaml file.

Caution. Release artifacts must have a specific directory structure to deliver to different platforms and handle the executable files efficiently when using trdl-client (learn more about using artifacts).

trdl.yaml

dockerImage: alpine:3.13.6@sha256:e15947432b813e8ffa90165da919953e2ce850bef511a0ad1287d7cb86de84b5
commands:
- ./build.sh {{ .Tag }} && cp -a release-build/{{ .Tag }}/* /result

build.sh

#!/bin/sh -e
VERSION=$1
if [ -z "$VERSION" ] ; then
    echo "Required version argument!" 1>&2
    echo 1>&2
    echo "Usage: $0 VERSION" 1>&2
    exit 1
fi
mkdir -p release-build/${VERSION}/any-any/bin
printf "echo ${VERSION}\n" > release-build/${VERSION}/any-any/bin/trdl-example.sh
mkdir -p release-build/${VERSION}/windows-any/bin
printf "@echo off\necho ${VERSION}\n" > release-build/${VERSION}/windows-any/bin/trdl-example.ps1

Add both files and commit them to Git.

Releasing a new version

Create and publish a new GPG-signed Git tag:

git tag -s v0.0.1 -m 'Signed v0.0.1 tag'
git push origin v0.0.1

The tag defines the version of the release artifacts and has a predefined format: an arbitrary semver number with the v prefix.

Once a Git tag is published, it needs to be signed by a sufficient number of trusted GPG keys. Each quorum member specified in the plugin configuration must sign the Git tag and publish their GPG signature using the Git signatures plugin:

git fetch --tags
git signatures pull
git signatures add v0.0.1
git signatures push

You can also use the following shortened command to sign Git tags: git signatures add --push v0.0.1.

Now that the tag has been created and signed by the necessary number of GPG keys, you can proceed to the release.

Use the /release API method to create a release. You can also use the following API methods for checking, controlling, and logging: /task/:uuid, /task/:uuid/cancel, and /task/:uuid/log.

A simplified version of the release process is available in the release.sh script in the server/examples directory of the project repository.

Four environment variables must be set before running the script:

  • VAULT_ADDR — Vault address;
  • VAULT_TOKEN — Vault token with permissions to access the endpoint at which the plugin is registered;
  • PROJECT_NAME — project name. In our case, this is the path at which the plugin is registered (see the -path parameter in the “Configuring the plugin” section);
  • GIT_TAG — Git tag.

Note that you can use our ready-made set of Vault actions for GitHub Actions.

Publishing the release channels

You must publish the release for the user to access it. To do this, switch to the main branch and add to the repository the trdl_channels.yaml file that describes the release channels.

trdl_channels.yaml:

groups:
- name: "0"
  channels:
  - name: alpha
    version: 0.0.1
  - name: stable
    version: 0.0.1

Next, add this file to Git, sign it with a GPG key, and commit it to the repository:

git add trdl_channels.yaml
git commit -S -m 'Signed release channels'
git push

Once a Git commit is published, it needs to be signed by a sufficient number of trusted GPG keys. Each quorum member specified in the plugin configuration must sign the Git commit and publish their GPG signature using the Git signatures plugin:

git fetch
git signatures pull
git signatures add origin/main
git signatures push

You can also use the following shortened command: git signatures add --push origin/main.

Now that the commit has the required number of GPG signatures, you can publish the release channels.

Use the /publish API method to do this. You can also use the following API methods for checking, controlling, and logging: /task/:uuid, /task/:uuid/cancel, and /task/:uuid/log.

A streamlined version of the publishing process is available in the publish.sh script in the server/examples directory of the project repository.

As with release.sh, the publish.sh script requires setting some environment variables:

  • VAULT_ADDR — Vault address;
  • VAULT_TOKEN — Vault token with permissions to access the endpoint at which the plugin is registered;
  • PROJECT_NAME — project name. In our case, this is the path at which the plugin is registered (see the -path parameter in the “Configuring the plugin” section).

Note that you can use our ready-made set of actions for GitHub Actions.

For a user

The instructions below are valid for Linux, macOS, and Windows. Commands can be executed in any Unix shell or in Windows PowerShell.

Installing the client

Download the trdl client by following the instructions in the message for the chosen release. Copy it to the directory in the user’s PATH.

Using the client

When adding the repository, the user has to provide details to verify the TUF repository during initial access: the TUF repository address (URL), the trusted version number (ROOT_VERSION), and the hash of the corresponding <VERSION>.root.json (ROOT_SHA512) file. The user receives these from the vendor.

In our case, the user gets the following data from the vendor:

URL=https://storage.googleapis.com/trdl-test-project-tuf
ROOT_VERSION=1
ROOT_SHA512=$(curl -Ls ${URL}/${ROOT_VERSION}.root.json | sha512sum)

Next, the user adds a repository with an arbitrary name:

REPO=test
trdl add $REPO $URL $ROOT_VERSION $ROOT_SHA512

You can then use artifacts within the desired update channel:

. $(trdl use test 0 stable)

The script is now available in the PATH of the current shell session.

trdl-example.sh
v0.0.1
trdl-example.ps1
v0.0.1