Secrets and sealing
How NebulaCtrl encrypts stored secrets with envelope encryption, what the master key protects, how rotation and resealing work, and which values are never logged.
Sealing is how the control plane encrypts a secret before it writes it to the database. Every sealed value is encrypted under its own key, and those keys are in turn protected by one master key that lives outside the database. A stolen database or backup is unreadable without that key.
How sealing works
NebulaCtrl uses envelope encryption:
- The control plane generates a random 256-bit data key for the value and encrypts the value with it, using AES-256-GCM.
- It derives a wrapping key from the master key with HKDF-SHA256, and encrypts the data key with it, again with AES-256-GCM.
- It stores the wrapped data key, the ciphertext, their nonces and a key id together as one string.
Both encryption layers authenticate a context string that names the row the value belongs to, such as one variable, one registry credential or one session. A ciphertext copied from one row to another therefore fails to open, even if the plaintext is the same. The control plane reports every failure to open as the same error, without saying which check failed.
The key id is a fingerprint of the master key. It identifies which key sealed a value and does not reveal the key.
What is sealed, hashed or neither
| Kind | Values |
|---|---|
| Sealed (reversible with the master key) | Service variable values, whether marked secret or not. External resource variables. Registry passwords. Git connection secrets: GitHub App client secret, private key and webhook secret, and Forgejo and GitLab tokens. Per-service and per-project webhook secrets. Object store secret keys. Cloudflare API tokens. Tailscale client secrets. Certificate private keys and ACME account keys. Notification channel secrets. Cluster SSH keys. Database point-in-time recovery credentials. Pending change set values. The ID token stored with each session. |
| Hashed (SHA-256, not reversible) | API token secrets. Session tokens. SCIM tokens. Invitation tokens. Cluster enrollment tokens and agent bearer tokens. |
In .env, not in the database | The master key, the OIDC client secret, the database password and the updater token. |
| Plain text | Everything else, including variable names, service configuration, member emails and audit events. |
The master key
The master key is the base64 of 32 random bytes. The installer generates it on the first install and keeps it in .env as NEBULA_MASTER_KEY. It is read at start-up and held in memory. The installer never replaces it.
Losing the key makes every sealed value unrecoverable, while the rest of the database stays readable. Nobody can regenerate it. See Back up and restore the control plane.
Rotation and resealing
Rotation is possible because the control plane can hold several keys at once:
- Current key.
NEBULA_MASTER_KEY. Every new or changed secret is sealed with it. - Previous keys.
NEBULA_PREVIOUS_MASTER_KEYS. They open older values, and never seal new ones.
Because each value carries the key id of the key that sealed it, the control plane knows which configured key to open it with. nebula secrets reseal walks every table that holds sealed values, opens each value that is not under the current key, and seals it again under the current key. It works in locked batches and writes a value only if it has not changed since it was read. When a second run reports zero re-encrypted values in every table, no stored value depends on a previous key and you can remove them.
A database backup taken before a rotation still holds values sealed under the old key. Keep the old key for as long as you keep that backup. The procedure is in Rotate the master key.
Reveal and masking
Variables marked secret stay masked in the console and the API. Revealing one needs the deployer role or higher; a viewer cannot. A reveal is an audit event that names the variable and never contains the value.
What is never logged
- Audit events record which variable, token or credential changed, and never its value. Creating an API token records its id, role and project scope, not the token.
- The access log records the method, the matched route pattern, the status, the duration and a request id, with the ids of the organization, person or token and cluster involved. It records no emails, token secrets, variable values or raw paths.
- Build logs replace Git and registry credentials with
[REDACTED]. - Release command logs replace the values of the release's secret variables, when they are at least six bytes long, with
[redacted]before the agent reports the tail of a failed release command. - Database connection errors have the password stripped from the URL.
One exception: the audit event for a query run in a database service's query panel records the query text. Do not put secrets in queries.
Where the values go next
A release sends the service's variable values to the cluster's agent over the agent link. The agent stores them as Kubernetes Secrets in the environment's namespace, and the workload reads them as environment variables. Clusters installed by NebulaCtrl run K3s with secrets encryption on. Sealing ends at the control plane's boundary: protecting the values inside the cluster is part of securing the cluster.
Boundaries
- Sealing protects data at rest. A process that holds the master key, such as the running control plane, can open every value.
- Anyone with database access and the master key can read every secret.
- Sealing does not hide the existence or the names of secrets.
Related
Harden a production install
A checklist for locking down a self-hosted NebulaCtrl control plane, its identity and access settings, its integrations and the clusters it manages.
Audit log
The audit events NebulaCtrl records, their fields, how to filter and export them from the Activity page or the API, and what is never written to them.