Quickstart | trdl

For an administrator

We assume that you are already familiar with Vault and know how to use it, so let us skip the details of setting it up. We also assume that Vault is configured according to the official documentation.

Vault

There are a couple of ways to install Vault and the trdl plugin. The easiest one is to use the ready-made Vault binary (you can get one at the Vault website or install it using your distribution’s package manager) and the ready-made trdl plugin binary.

Docker

Install Docker. Add a Vault user to the Docker group:

usermod -a -G docker vault

Setting up the project

Git repository

Create a regular public Git repository.

Bucket

Any S3-compatible bucket will do. It should be publicly available for reading.

A note about GCS (Google Cloud Storage)

To get rid of the An error occurred (AccessDenied) when calling the CreateMultipartUpload operation: Access denied error, make sure that the Service Account used to access the bucket belongs to the Storage Admin role.

Installing the plugin

Download the trdl plugin by following the instructions in the message of the selected release. Copy it to /etc/vault.d/plugins or another directory where you normally store plugins.

Configuring the plugin

Setting up Vault includes specifying the directory where the plugins are stored:

plugin_directory = "/etc/vault.d/plugins"

Restart Vault.

vault plugin register -sha256=$(sha256sum /etc/vault.d/plugins/vault-plugin-secrets-trdl | awk '{print $1}') secret vault-plugin-secrets-trdl

In our case, the plugin file is called vault-plugin-secrets-trdl, and we register it under the same name in Vault. Refer to the official documentation to learn more about registering plugins.

Enable the plugin as a secrets engine at a specific path in Vault:

vault secrets enable -path=trdl-test-project vault-plugin-secrets-trdl

You can enable the same plugin many times; however, you must use a unique path each time. For more information, refer to the official documentation.

Now let’s configure the trdl plugin itself. We will use the /configure API method to do this:

vault write trdl-test-project/configure @configuration.json

where configuration.json has the following contents:

{
  "s3_secret_access_key": "FOO",
  "s3_access_key_id": "BAR",
  "s3_bucket_name": "trdl-test-project-tuf",
  "s3_region": "europe-west1",
  "s3_endpoint": "https://storage.googleapis.com",
  "git_repo_url": "https://github.com/werf/trdl-test-project",
  "required_number_of_verified_signatures_on_commit": 2
}

When configuring the plugin, you must specify the minimum number of GPG signatures required for a commit (required_number_of_verified_signatures_on_commit). Otherwise, the release system becomes vulnerable: any user with access can tamper with it because the system is not protected by a quorum.

The minimum number of GPG signatures required (required_number_of_verified_signatures_on_commit) depends on the size and scope of the team, frequency of updates, and other factors.

Managing public parts of trusted GPG keys

The /configure/trusted_pgp_public_key group of API methods is used to handle the public parts of trusted GPG keys.

Adding a key

vault write werf/configure/trusted_pgp_public_key name=developer public_key=@developer.pgp

where developer.pgp is the file with the public PGP key generated by the gpg --armor --output developer.pgp --export developer@trdl.dev command.

Contents of the developer.pgp file

Please refer to the gpg documentation for more information about exporting.

Getting a list of keys

vault read werf/configure/trusted_pgp_public_key

Key     Value
---     -----
keys    [developer]

Displaying key contents

vault read werf/configure/trusted_pgp_public_key/developer

Command output

Key           Value
---           -----
name          developer

public_key    -----BEGIN PGP PUBLIC KEY BLOCK-----

mQGNBGH8PiQBDAClie5jZHKIEDUw14+UJB+knS+X5SQg8lOlZqdiizMcYBdhnEEM
OLhtvvMfTTY+ikREuvEVUBVXYMrAGSCA+291ngbKIlU5YyC75mHxV6IDvEX91UEc
5o2OXnNFlTHj3jXAJytUd6IXfv6Wx06aHI8xeFzhYxW8CHD/NaJd+XfX3gr5pmUp
U2N8T0dTIM9QZ4o8fdrpWfMcp6Q8LwO1ConFJnEPIvR0etdqNiIu+6/33ImWrYuu
09XHUQ+LZAkjP9YJS8ITK38qboEtFsflO06NMeaPH+TgLFmBi4Ov42aSJCJ5x1HS
5qB18V99oEVFE82DVjy7Eflw4oCJayue405X1mgW0uc/225n+9JwoV2ZyRG5s/aE
gQjxqaVIDr7a6RtfqRK8AAPHkSOhaP2l0PhO9voZ/y2sFqtuWq8y+I+O78Gxq85O
ejuf0U/KYcQKjg4CE1eAVxakBz24VWkSHuBvdhjvzQydSe0KEKV/uE4g5ihk8olD
tf+cAf2jFLrlBDEAEQEAAbQZVGVhbSBMZWFkZXIgPHRsQHRyZGwuZGV2PokBzgQT
AQoAOBYhBCulX9gVgDTuvpKqntnXm2Ovwwx6BQJh/D4kAhsDBQsJCAcCBhUKCQgL
AgQWAgMBAh4BAheAAAoJENnXm2Ovwwx6Ng0L+wWkj/P5QINyids8iLoNnYGdKx46
ayLzi7HquOC2ckQiazcli5KSq9/4uJn9ff2Ri4wQmwNMOuLBUSFxyfibR73ZAFtS
xHfbYFgUoQHWOH//y5QzEkHSNZXFhsSKuy3Xgmr7o3BtVtmR33qYUpbVrRVCYIdN
qKVlpBxQnObq995993eIUUKTheUfFF9Bh91mdbU4usZf1uQH0I5vhTS7Xd45U9Wd
m2g7NoMQVgM8lAmwaDWlKzv+P4XiQFUUbSGbXt7yQtqXUhVXOQ5xLh/i0mDVrSlt
tZD+F6tFYgEJphlgWkEFXpcWI9xxpGv6UCuCnhm5B9SbV83pJUp1Dr/Btw/OASUW
PzcvN54LwXX2SwTP83qxS2qpvHK4SNtHrn7+icgBi2ZLqCv+8iWNPvl3G9pF/Zzs
E8bQh0lmdvHIoJd2ZeBKfBOOMLqHEPae9DYcaW9VUkLr+GRFHJzh1WHF9f1Dd+A+
INJqsb1KawfsJwDXcZM8si1PUhoxI+YbFXgn8rkBjQRh/D4kAQwAqudoseQ/O6WU
NdE9XSCvJAhnUYKhLadTyN8pd70ibWONav4M+B71rg+BFNTTB5eRHEgGzPDJmxex
ba4Zhvt+2TAbmnF1SAcSciCEIx57239L1ERkLXpHwNLmCEjbiR3k9xOZ4wMDQEHC
1qswbf0XvO1UjYsw/L6uL253anqP8IxMSPuCG9TkZuZ4A1qrCxQ2Y8JO+XEtM764
5OqWGU90I+6PXl0hgPgg+VeFpkAXr67fwaa94aISJq/rIzfxf76N8YcJeldMlFyp
vytz7BqsdGYmVigKSjWCllIVTCyFV3oggnDJn6Gmbwhp8+lj9MuZRyBn3nFzZbZT
Mo9TAgIFy6UQ80yW2M9MnIOPMHmtRzoSjUlEgTzjwT8L/YQGQ9GnFxIINk4PUFPj
fFEvmP+y+8cb+EhrgQ770LtQEd+E6zXexrh9mvGxIj87XP5Jl6Kz8goMcPp3+jTR
vggepxU/6pmFonRMcbmwjZ1M9JpibjPX49Pb1nAkUvE6szgwMItPABEBAAGJAbYE
GAEKACAWIQQrpV/YFYA07r6Sqp7Z15tjr8MMegUCYfw+JAIbDAAKCRDZ15tjr8MM
eiJ5DACCga9PnpyVHIltDXb5UC3OEsfNLI8PCVnBnMMco2Iedea0E3pyKniMHxHS
TW/+4RT9KzdOqOEQBzIdmsL/Vq0dnh3j+UDrVhp6ppVi5dBXgrgYx1RL+4EoipOS
pVKJdmOqA/b5O8LNnN761MP3n5gJWURr5k2seKhxgjTQ27qRPi3Gq6mtj0xWRkXZ
ivia1mefDpIif0TjSCrEMy4y8Zj+4fyy6AbMGYvSkUDaCwzk0shiAwAhW+9w8V6f
2fDuY18OXvTNwW8anU7XMM12mdyNdzvVPTfe23HdboJ5dDwKH8p8E+f1B+ozXosb
qvdhnCCdTNCww95K+Nq5zy0CQ2+mGB929dmOIJCo7BTM4/vxQT100P/FnShCu/Ji
UVlFWU0M1u8czX5la8AXimkAdmO9HIiPD6Qs/X+VaLuqvgIO0OrmytC1jVXzn9HH
1GYIrC8WdSo7ATE/gI5BftJq+WXDzXwLCA1Ze2QP8GffQKkuRjHRiv3spnFAXjiZ
TJ9EZRY=
=vkcq
-----END PGP PUBLIC KEY BLOCK-----

Deleting a key

vault delete werf/configure/trusted_pgp_public_key/developer

Success! Data deleted (if it existed) at: werf/configure/trusted_pgp_public_key/developer

For a developer

Setting up a GPG signature in Git

Git has a mechanism for signing new tags (releasing) and individual commits (publishing). As a result, the GPG signature becomes an integral part of the Git tag or Git commit. However, this approach supports only one signature.

The signatures plugin allows you to sign Git tags and Git commits after they are created. In this case, GPG signatures are stored in Git notes. You can use as many signatures as you want, and you can also delete previously used signatures without affecting the linked Git tag or Git commit in any way.

All you need to do is set up GPG and Git correctly to create GPG signatures. This manual can help you.

Installing the signatures plugin

To use the plugin, you have to install it to an arbitrary directory in PATH (e.g., ~/bin):

git clone https://github.com/werf/third-party-git-signatures.git
cd third-party-git-signatures
install bin/git-signatures ~/bin

After running the git signatures command you should see the plugin description.

git signatures <command> [<args>]

Git Signatures is a system for adding and verifying one or more PGP
signatures to a given git reference.

Git Signatures works by appending one of more signatures of a given
ref hash to the git notes interface for that ref at 'refs/signatures'.

In addition to built in commit signing that allows -authors- to sign,
Git Signatures allows parties other than the author to issue "approval"
signatures to a ref, allowing for decentralized cryptographic proof of
code review. This is also useful for automation use cases where CI
systems to be able to add a signatures to a repo if a repo if all tests
pass successfully.

In practice Git Signatures allows for tamper evident design and brings
strong code attestations to a deployment process.

Commands
--------

* git signatures init
    Setup git to automatically include signatures on push/pull

* git signatures import
    Import all PGP keys specified in .gitsigners file to local
    GnuPG keychain allowing for verifications.

* git signatures show
    Show signatures for a given ref.

* git signatures add
    Add a signature to a given ref.

* git signatures verify
    Verify signatures for a given ref.

* git signatures pull
    Pull all signatures for all refs from origin.

* git signatures push
    Push all signatures for all refs to origin.

* git signatures version
    Report the version number.

Configuring the build process

As a basic example of creating and arranging release artifacts for multiple platforms, let’s deliver the script that outputs a release tag when run.

All build parameters, such as environment and build instructions, are defined in the trdl.yaml file.

Caution. Release artifacts must have a specific directory structure to deliver to different platforms and handle the executable files efficiently when using trdl-client (learn more about using artifacts).

trdl.yaml

dockerImage: alpine:3.13.6@sha256:e15947432b813e8ffa90165da919953e2ce850bef511a0ad1287d7cb86de84b5
commands:
- ./build.sh {{ .Tag }} && cp -a release-build/{{ .Tag }}/* /result

build.sh

#!/bin/sh -e
VERSION=$1
if [ -z "$VERSION" ] ; then
    echo "Required version argument!" 1>&2
    echo 1>&2
    echo "Usage: $0 VERSION" 1>&2
    exit 1
fi
mkdir -p release-build/${VERSION}/any-any/bin
printf "echo ${VERSION}\n" > release-build/${VERSION}/any-any/bin/trdl-example.sh
mkdir -p release-build/${VERSION}/windows-any/bin
printf "@echo off\necho ${VERSION}\n" > release-build/${VERSION}/windows-any/bin/trdl-example.ps1

Add both files and commit them to Git.

Releasing a new version

Create and publish a new GPG-signed Git tag:

git tag -s v0.0.1 -m 'Signed v0.0.1 tag'
git push origin v0.0.1

The tag defines the version of the release artifacts and has a predefined format: an arbitrary semver number with the v prefix.

Once a Git tag is published, it needs to be signed by a sufficient number of trusted GPG keys. Each quorum member specified in the plugin configuration must sign the Git tag and publish their GPG signature using the Git signatures plugin:

git fetch --tags
git signatures pull
git signatures add v0.0.1
git signatures push

You can also use the following shortened command to sign Git tags: git signatures add --push v0.0.1.

Now that the tag has been created and signed by the necessary number of GPG keys, you can proceed to the release.

Use the /release API method to create a release. You can also use the following API methods for checking, controlling, and logging: /task/:uuid, /task/:uuid/cancel, and /task/:uuid/log.

A simplified version of the release process is available in the release.sh script in the server/examples directory of the project repository.

Four environment variables must be set before running the script:

VAULT_ADDR — Vault address;
VAULT_TOKEN — Vault token with permissions to access the endpoint at which the plugin is registered;
PROJECT_NAME — project name. In our case, this is the path at which the plugin is registered (see the -path parameter in the “Configuring the plugin” section);
GIT_TAG — Git tag.

Note that you can use our ready-made set of Vault actions for GitHub Actions.

Publishing the release channels

You must publish the release for the user to access it. To do this, switch to the main branch and add to the repository the trdl_channels.yaml file that describes the release channels.

trdl_channels.yaml:

groups:
- name: "0"
  channels:
  - name: alpha
    version: 0.0.1
  - name: stable
    version: 0.0.1

Next, add this file to Git, sign it with a GPG key, and commit it to the repository:

git add trdl_channels.yaml
git commit -S -m 'Signed release channels'
git push

Once a Git commit is published, it needs to be signed by a sufficient number of trusted GPG keys. Each quorum member specified in the plugin configuration must sign the Git commit and publish their GPG signature using the Git signatures plugin:

git fetch
git signatures pull
git signatures add origin/main
git signatures push

You can also use the following shortened command: git signatures add --push origin/main.

Now that the commit has the required number of GPG signatures, you can publish the release channels.

Use the /publish API method to do this. You can also use the following API methods for checking, controlling, and logging: /task/:uuid, /task/:uuid/cancel, and /task/:uuid/log.

A streamlined version of the publishing process is available in the publish.sh script in the server/examples directory of the project repository.

As with release.sh, the publish.sh script requires setting some environment variables:

VAULT_ADDR — Vault address;
VAULT_TOKEN — Vault token with permissions to access the endpoint at which the plugin is registered;
PROJECT_NAME — project name. In our case, this is the path at which the plugin is registered (see the -path parameter in the “Configuring the plugin” section).

Note that you can use our ready-made set of actions for GitHub Actions.

For a user

The instructions below are valid for Linux, macOS, and Windows. Commands can be executed in any Unix shell or in Windows PowerShell.

Installing the client

Download the trdl client by following the instructions in the message for the chosen release. Copy it to the directory in the user’s PATH.

Using the client

When adding the repository, the user has to provide details to verify the TUF repository during initial access: the TUF repository address (URL), the trusted version number (ROOT_VERSION), and the hash of the corresponding <VERSION>.root.json (ROOT_SHA512) file. The user receives these from the vendor.

In our case, the user gets the following data from the vendor:

URL=https://storage.googleapis.com/trdl-test-project-tuf
ROOT_VERSION=1
ROOT_SHA512=$(curl -Ls ${URL}/${ROOT_VERSION}.root.json | sha512sum)

Next, the user adds a repository with an arbitrary name:

REPO=test
trdl add $REPO $URL $ROOT_VERSION $ROOT_SHA512

You can then use artifacts within the desired update channel:

. $(trdl use test 0 stable)

The script is now available in the PATH of the current shell session.

Linux / macOS Windows

trdl-example.sh
v0.0.1

trdl-example.ps1
v0.0.1