Docker Compose-Based Deployment of a Validator Node

This section describes how to deploy a standalone validator node on a VM or a local machine using Docker Compose. The deployment consists of the validator node along with associated wallet and CNS UIs, and onboards the validator node to the target network.

This deployment is useful for:

  • Application development, where one needs an ephemeral validator that is easy to deploy.

  • Production validators, with the following caveats:

    • The default deployment is highly insecure. Authentication should be enabled as described in the authentication section.

    • There is no support for ingress from outside your machine, nor is there support for TLS. The deployment should be kept local to your machine only and not exposed externally.

    • Reliability & scalability: docker-compose will restart containers that crash, and the deployment supports backup&restore as detailed below, but a docker-compose deployment is inherently more limited than a cloud-based Kubernetes one.

    • Monitoring: The deployment, as opposed to a Kubernetes-based one, does not include monitoring.

    • For production settings, you should aim to keep your validator up and running constantly, in order to avoid losing out on rewards, and avoid issues with catching up on ledger state after significant downtime.

Requirements

  1. A linux/MacOS machine with the following:

    1. docker compose - at least version 2.26.0 or newer

    2. curl

    3. jq

    Note that both AMD64 and ARM64 architectures are supported.

To validate that the dependencies are set up correctly, run the following commands. All commands should succeed and print out the version. Note that the exact versions you see may be different from the example here. As long as you have docker-compose 2.26.0 or newer you should be fine.

> docker compose version
Docker Compose version 2.32.1
> curl --version
curl 8.11.0 (x86_64-pc-linux-gnu) libcurl/8.11.0 OpenSSL/3.3.2 zlib/1.3.1 brotli/1.1.0 zstd/1.5.6 libidn2/2.3.7 libpsl/0.21.5 libssh2/1.11.1 nghttp2/1.64.0
Release-Date: 2024-11-06
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd
> jq --version
jq-1.7.1

  1. Your machine should either be connected to a VPN that is whitelisted on the network (contact your sponsor SV to obtain access), or have a static egress IP address. In the latter case, please provide that IP address to your sponsor SV to add it to the firewall rules.

  2. Please download the release artifacts containing the docker-compose files, from here: Download Bundle, and extract the bundle:

tar xzvf 0.4.4_splice-node.tar.gz

Warning

If you lose your keys, you lose access to your coins. While regular backups are not necessary to run your node, they are strongly recommended for recovery purposes. You should regularly back up all databases in your deployment and ensure you always have an up-to-date identity backup. Super Validators retain the information necessary to allow you to recover your Canton Coin from an identity backup. On the other hand, Super Validators do not retain transaction details from applications they are not involved in. This means that if you have other applications installed, the Super Validators cannot help you recover data from those apps; you can only rely on your own backups. (More information in Backups section for Validators or Backups section for SVs)

Required Network Parameters

To initialize your validator node, you need the following parameters that define the network you’re onboarding to and the secret required for doing so.

MIGRATION_ID

The current migration id of the network (dev/test/mainnet) you are trying to connect to. You can find this on https://sync.global/sv-network/.

SPONSOR_SV_URL

The URL of the SV app of your SV sponsor. This should be of the form https://sv.sv-1.unknown_cluster.global.canton.network.YOUR_SV_SPONSOR, e.g., if the Global Synchronizer Foundation is your sponsor use https://sv.sv-1.unknown_cluster.global.canton.network.sync.global.

ONBOARDING_SECRET

The onboarding secret provided by your sponsor. If you don’t already have one, ask your sponsor. Note that onboarding secrets are one-time use and expire after 48 hours. If you don’t join before it expires, you need to request a new secret from your SV sponsor.

DevNet-only

On DevNet, you can obtain an onboarding secret automatically by calling the following endpoint on any SV (replace SPONSOR_SV_URL with the URL of your SV sponsor):

curl -X POST SPONSOR_SV_URL/api/sv/v0/devnet/onboard/validator/prepare

Note that this self-served secret is only valid for 1 hour.

Additional parameters describing your own setup as opposed to the connection to the network are described below.

Deployment

  1. Change to the docker-compose directory inside the extracted bundle:

cd splice-node/docker-compose/validator

  1. Export the current version to an environment variable: export IMAGE_TAG=0.4.4

  2. Run the following command to start the validator node, and wait for it to become ready (could take a few minutes):

./start.sh -s "<SPONSOR_SV_URL>" -o "<ONBOARDING_SECRET>" -p "<party_hint>" -m "<MIGRATION_ID>" -w

Where:

<party_hint> will be used as the prefix of the Party ID of your validator’s administrator.

This must be of format <organization>-<function>-<enumerator>, e.g. myCompany-myWallet-1.

Note that the validator may be stopped with the command ./stop.sh and restarted again with the same start.sh command as above. Its data will be retained between invocations. In subseqent invocations, the secret itself may be left empty, but the -o is still mandatory, so a -o "" argument should be provided.

Logging into the wallet UI

Note

Docker Compose-based validator deployments use .localhost subdomains for addressing, such as wallet.localhost. .localhost URLs reportedly do not work on some browsers. If you encounter issues please try using a different browser such as Firefox or Chrome. If you’re encountering issues with reaching APIs from a custom program or script, you may need to set the HOST header on HTTP requests explicitly to the target .localhost address.

The wallet UI is accessible at http://wallet.localhost in your browser. The validator administrator’s username is administrator. Insert that name into the username field and click Log in, and you should see the wallet of the administrator of your wallet.

You can also logout of the administrator account and login as any other username. The first time a user logs in, they will be prompted with a message asking them to confirm whether they wish to be onboarded to the validator node.

Logging into the CNS UI

You can open your browser at http://ans.localhost (note that this is currently by default ans and not cns), and login using the same administrator user, or any other user that has been onboarded via the wallet, in order to purchase a CNS entry for that user.

Configuring Authentication

Warning

The default deployment uses highly insecure self-signed tokens. Anyone with access to the wallet UI (or the machine and/or its network interface) may log in to your wallet as a user of their choice. For any production use, you should configure proper authentication as described in this section.

Please refer to the authentication section for instructions on how to set up an OAuth provider for your validator. The URLs to configure for callbacks are http://wallet.localhost and http://ans.localhost.

Once you have set up your OAuth provider, you need to configure it by setting the following environment variables in the .env file:

Name

Value

AUTH_URL

The URL of your OIDC provider for obtaining the openid-configuration and jwks.json.

AUTH_JWKS_URL

The URL of your OIDC provider for obtaining the jwks.json, will typically be ${AUTH_URL}/.well-known/jwks.json.

AUTH_WELLKNOWN_URL

The URL of your OIDC provider for obtaining the openid-configuration, will typically be ${AUTH_URL}/.well-known/openid-configuration.

LEDGER_API_AUTH_AUDIENCE

The audience for the participant ledger API. e.g. https://ledger_api.example.com.

LEDGER_API_AUTH_SCOPE

The scope for the participant ledger API. Optional

VALIDATOR_AUTH_AUDIENCE

The audience for the validator backend API. e.g. https://validator.example.com.

VALIDATOR_AUTH_CLIENT_ID

The client id of the OAuth app for the validator app backend.

VALIDATOR_AUTH_CLIENT_SECRET

The client secret of the OAuth app for the validator app backend.

LEDGER_API_ADMIN_USER

Should match the sub field of JWTs issued for the validator app. For some auth providers, this would be formed as CLIENT_ID@clients.

WALLET_ADMIN_USER

The user ID of the user which should login as the wallet administrator. Note that this should be the full user id, e.g., auth0|43b68e1e4978b000cefba352, not only the suffix 43b68e1e4978b000cefba352.

WALLET_UI_CLIENT_ID

The client id of the OAuth app for the wallet UI.

ANS_UI_CLIENT_ID

The client id of the OAuth app for the CNS UI.

In order to enable auth in the deployment, add the -a flag to the start.sh command, as follows:

./start.sh -s "<SPONSOR_SV_URL>" -o "<ONBOARDING_SECRET>" -p "<party_hint>" -m "<MIGRATION_ID>" -w -a

If you have already deployed a non-authenticated validator on your machine, you can migrate it to an authenticated one by stopping the validator with ./stop.sh and restarting it with the -a flag as above. The validator operator user will be automatically migrated, and the user indicated by the WALLET_ADMIN_USER variable will be associated with the validator operator party. If you have also onboarded other users onto your validator, those will not be automatically migrated, and you need to manually associate the OAuth users with their corresponding parties. In order to do that, first take note of the party IDs of all relevant users (do this before stopping the unauthenticated validator), e.g. by copying them from the top-right corner of their wallet UIs. Now for every user that you wish to migrate, follow the instructions for associating a user with a party in the Users, Parties and Wallets in the Splice Wallet section, but replace the admin party ID with the party ID which you wish to associate with each user.

Integration with systemd and other init systems

If you want to manage the validator through systemd or a similar init system, create a service that calls the start.sh script with the right arguments. However, note that start.sh invokes docker compose up with the -d/--detach option so the script exits after the containers are up instead of continuing running.

You need to make sure that your service does not stop docker compose at that point. To accomplish this with systemd set RemainAfterExit=true. Refer to the systemd documentation for more details. If you are using another init system, look for similar options to ensure that docker compose continues running after the script exits.

Alternatively, you can edit the script to remove the -d option so the script continues running.

Comments