Add and back up a volume
Add a persistent volume to a process, grow it, and back it up to an object store on demand or on a schedule. Download backups and restore them onto the volume.
A service loses its disk when it restarts unless a process mounts a volume. This guide adds a volume, connects an object store, and backs up and restores the volume's data. Database services follow the same steps on their Backups tab; see Create and connect a database.
Before you begin
- You have the deployer role to add, grow and delete volumes, to verify an object store and to take backups. Admin is needed to restore, to connect or change an object store, and to take, schedule or download a backup, or delete a succeeded one, in a production environment.
- The service has an HTTP or worker process. The console adds the volume to its first one; a proxy process cannot mount a volume.
- The storage class decides what a size means. On the default K3s class,
local-path, a volume is a directory on the node's disk: its size is a request that nothing enforces, its usage shows Usage not reported, and it cannot grow in place. A class with its own filesystem enforces the size and reports usage.
Add a volume
Open the service, then its Settings tab. In the Storage section, select + Add volume.
In Add a volume, enter a Name, an absolute Mount path such as /data, and a Size (GiB) (default 1). Select Add volume.
The volume appears under Storage and exists in every environment. The process mounts it from its next release. Deploy the service; if the console reports that the release needs downtime, accept it (see downtime).
Through the API, createVolume takes name, mountPath and sizeBytes on POST /api/v1/processes/PROCESS_ID/volumes. Use it for a process other than the one the console picks.
Accept downtime
A release that mounts a volume stops the running release first, so the service is briefly unavailable. Accept this once per deployment: Deploy with downtime in the confirmation, Accept downtime and deploy anyway in the deploy dialog, or Accept downtime when you review a change set. A deployment started by a Git build accepts it automatically. The first deployment of a service has nothing to stop.
Grow a volume
A volume only grows, and only on a storage class that allows expansion. Where it does, the volume row shows +10 GiB, +50 GiB and +100 GiB; otherwise it explains that the class cannot expand claims.
Under Storage, select a growth button. The change is staged and the row shows staged · grows to the new size.
Review the change set, select Accept downtime, then select the apply button (Deploy N changes, or Request approval in a production environment). Growing always needs downtime acknowledged.
The API refuses a smaller size ("a volume can only grow"). The size is per environment. If the class cannot expand claims, the cluster keeps the old size; to get a larger volume, back it up and restore it onto a larger one.
Connect an object store
Backups go to an organization object store: an S3-compatible bucket.
Open Settings, then Integrations. In the Storage group, select Connect on Object storage, or Another bucket for a further store.
In Add object store, fill in Name, Endpoint, Region (default auto), Bucket, optional Prefix, Access key ID and Secret access key. The endpoint is an https:// URL naming only a host, with no bucket or path. Select Force path style if the provider addresses the bucket as ENDPOINT/BUCKET. Select Add object store.
The store shows Not verified. Select Manage, then Verify. NebulaCtrl writes, reads back and deletes a probe object; the status becomes Connected, or Failing with the error.
The secret key is write-only: it is never shown again, and leaving Secret access key blank on edit keeps the stored one. Adding or changing a store needs admin. A store cannot be deleted while a backup schedule uses it, or while backups, restores or a point-in-time recovery archive refer to it.
Back up a volume
Under Storage, select Back up now on the volume's backups panel. In Back up VOLUME, choose the Object store and select Back up now. The backup is a zstd-compressed tar of the live files and moves from Queued to Running to Complete or Failed. A failed row shows its message.
To get a consistent copy, select Quiesce — stop every workload mounting this volume for a consistent archive, then Accept downtime and back up anyway. The workloads scale to zero for the backup, then scale back up.
Only one backup or restore runs per volume and environment; a second request is refused with "a backup or restore is already in progress for this volume in this environment".
Schedule backups
On the volume's backups panel, select Schedule… (Edit schedule… if one exists). In the schedule dialog, choose the Object store, enter Cron and Keep last, then select Create schedule (Save changes on edit).
- Cron is five fields (minute, hour, day of month, month, day of week) evaluated in UTC. The default is
0 3 * * *. Descriptors such as@dailyandTZ=prefixes are refused. - Keep last is at least 1; the default is 7. After each scheduled backup succeeds, older succeeded backups from the schedule beyond that count are deleted. Backups you take by hand are not counted.
- Scheduled backups never quiesce.
To stop future backups, select Edit schedule…, then Delete schedule, and confirm Delete schedule. Existing backups stay and no longer fall under the schedule's retention. The API operation is putVolumeBackupSchedule; deleteVolumeBackupSchedule removes the schedule.
Download a backup
Select Download on a Complete backup. The console opens a signed link that works for 15 minutes and records the download in the activity log. The API operation is downloadVolumeBackup.
Restore a volume
A restore stops every workload on the volume, wipes its current contents, then extracts the archive. If it fails, the volume is left however far the wipe and extract got and the service stays stopped. NebulaCtrl does not return to the previous contents; start a new restore.
- On the volume's backups panel, select Restore on a backup, or Restore… to choose.
- Under Restore from, choose From a backup and pick one, or From an object key and enter an Object store and Object key of an archive. A backup you took is checked against its checksum; an object key is not.
- Type the service name to confirm, then select Restore and stop SERVICE.
Restore history appears under Restore history on the panel.
Delete a backup or a volume
Select the trash icon on a Complete or Failed backup, then Delete backup. This removes its archive from the object store and cannot be undone.
To delete a volume, select Delete beside it under Storage, type its name and select Delete volume. The API refuses with "volume NAME is still mounted by a retained release; remove that release before deleting the volume" while the active release, a release in flight or awaiting approval, or a retained earlier release in any environment still mounts it.
Verify
The backup row reads Complete with a size, and Automatic backups on the panel shows the schedule, for example "Daily at 03:00 UTC · keeps last 7". After a restore, the history row reads Complete and the service runs again with the restored files.
Next steps
Control deploy order
Make a service's deployments wait for the services and databases it depends on, with the Deploy after list or depends_on in nebula.toml, and read what you see while a deployment waits.
Add a custom domain
Route a hostname you own to one HTTP or reverse-proxy process of a service, point its DNS at the cluster and get a certificate. DNS is either yours (manual) or managed in a connected Cloudflare account.