dokku/docs/advanced-usage/persistent-storage.md

Persistent Storage

Important

New as of 0.5.0. Named storage entries new as of 0.38.0.

The preferred method to attach persistent storage to a Dokku-managed container is the Dokku storage plugin.

storage:annotations:report [<name>] [<flag>]           # Display annotations for one or more storage entries
storage:annotations:set <name> <key> [<value>]         # Set or clear a single annotation on a storage entry
storage:create <name> [<path>] [flags]                 # Register a named storage entry
storage:destroy <name> [--force] [--destroy-host-dir]  # Remove a named storage entry (must be unmounted from every app first)
storage:ensure-directory [--chown option] <directory>  # [DEPRECATED] use storage:create instead
storage:exec <name> [-- <cmd>...]                      # Run a command (or shell) in a temporary container that mounts the entry
storage:info <name> [--format text|json]               # Show details for one storage entry
storage:labels:report [<name>] [<flag>]                # Display labels for one or more storage entries
storage:labels:set <name> <key> [<value>]              # Set or clear a single label on a storage entry
storage:list <app> [--format text|json]                # List bind mounts for an app's container(s) (legacy host:container view)
storage:list-entries [--scheduler s] [--format text|json]  # List registered storage entries
storage:mount <app> <name> --container-dir <path> [flags]  # Mount a named entry into an app
storage:mount <app> <host-dir:container-dir>           # [LEGACY] colon-form mount, docker-local only
storage:report [<app>] [<flag>]                        # Display a storage report for one or more apps
storage:report --global                                # Display a cluster-wide entry inventory
storage:set <name> <property> [<value>]                # Update a storage entry in place
storage:unmount <app> <name> [--container-dir <path>]  # Remove an attachment
storage:wait <name>                                    # Block until a k3s entry's PVC is bound

A storage entry is the source of truth for the underlying volume - a host directory on docker-local, or a PersistentVolumeClaim on k3s. Multiple apps can mount the same entry, and an attachment carries the per-app details (container path, phases, subpath, readonly, process type). Names are globally unique across the install and must be DNS-1123 labels (lowercase letters, digits, dashes) of 45 characters or less so they can be used verbatim as Helm release and PVC names.

The legacy storage:mount <app> <host>:<container> form continues to work on docker-local. On a k3s app it is rejected; create a named entry with storage:create --scheduler k3s and mount it instead. Existing colon-form mounts are migrated automatically the first time the new code runs - they show up as legacy-<hash> entries in storage:list-entries.

The storage plugin supports the following mount points:

• explicit paths that exist on the host (docker-local)
• docker volumes (docker-local)
• PersistentVolumeClaims provisioned via a StorageClass (k3s)
• hostPath-backed PVs (k3s)

Usage

Listing persistent storage

Persistent storage bind mounts are specified on a per-app basis, and can be listed with the storage:list command:

dokku storage:list node-js-app

-----> node-js-app volume bind-mounts:
       /var/lib/dokku/data/storage/node-js-app:/app/storage

The output format can also be set to json for programmatic access:

dokku storage:list node-js-app --format json

[
  {
    "entry_name": "node-js-app",
    "host_path": "/var/lib/dokku/data/storage/node-js-app",
    "container_path": "/app/storage"
  }
]

Each entry mirrors the underlying attachment. readonly (boolean) and volume_options (string) reflect Attachment.Readonly and Attachment.VolumeOptions directly and are only present when set, so external tooling can drift-detect attachments against the raw attachment fields. For example, a mount created with --volume-options noexec,nosuid --volume-readonly renders as:

[
  {
    "entry_name": "node-js-data",
    "host_path": "/var/lib/dokku/data/storage/node-js-data",
    "container_path": "/app/storage",
    "readonly": true,
    "volume_options": "noexec,nosuid"
  }
]

Creating storage directories

Important

New as of 0.25.5

A storage directory can be created with the storage:ensure-directory command. This command will create a subdirectory in the recommended /var/lib/dokku/data/storage path - created during Dokku installation - and prepare it for use with an app.

dokku storage:ensure-directory node-js-app

-----> Ensuring /var/lib/dokku/data/storage/node-js-app exists
       Setting directory ownership to 32767:32767
       Directory ready for mounting

By default, permissions are set for usage with Herokuish buildpacks. These permissions can be changed via the --chown option according to the following table:

• --chown herokuish (default): Use 32767:32767 as the folder permissions.
  • This is used for apps deployed with Buildpacks via Herokuish.
• --chown heroku: Use 1000:1000 as the folder permissions.
  • This is used for apps deployed with Cloud Native Buildpacks using the heroku/builder builder.
• --chown paketo: Use 2000:2000 as the folder permissions.
  • This is used for apps deployed with Cloud Native Buildpacks using the cloudfoundry/cnb or paketo builders.
• --chown root: Use 0:0 as the folder permissions.
  • This is used for containers that run their processes as root, as is typical for most Dockerfile or Docker image deploys.
• --chown false: Skips the chown call.
• --chown <uid>: Use <uid>:<uid> as the folder permissions, where <uid> is a custom numeric user/group id.
  • This is used for containers that run their processes as a uid/gid that doesn't correspond to any of the above named options.

Users deploying via Dockerfile will want to specify --chown false and manually chown the created directory if the user and/or group id of the runnning process in the deployed container do not correspond to any of the above options.

The --chown flag - whether on storage:create or storage:ensure-directory - only manages the default /var/lib/dokku/data/storage/<name> location. If a custom <path> is passed to storage:create, the chown call is refused and the operator must chown the path themselves.

Warning

Failing to set the correct directory ownership may result in issues in persisting files written to the mounted storage directory.

Setting directory permissions

Important

New as of 0.38.27

Where --chown states who owns the host directory, --mode states its permission bits. It takes a 3 or 4 digit octal mode, and is a --mode flag on storage:create and a mode property on storage:set:

dokku storage:create node-js-data --mode 0777

dokku storage:set node-js-data mode 0770

Omitting the value clears the mode, leaving the directory's permissions alone on subsequent runs:

dokku storage:set node-js-data mode

Without a mode, a newly created directory keeps the 0755 default and a pre-existing directory keeps whatever permissions it already had. The value is stored on the entry and re-applied every time storage:create or storage:set runs against it, so a declarative caller converges the directory by re-running the same command rather than reaching for chmod over SSH. The mode is shown by storage:info:

dokku storage:info node-js-data

-----> Storage entry node-js-data
       Scheduler:        docker-local
       Host path:        /var/lib/dokku/data/storage/node-js-data
       Mode:             0777

The mode is applied to the directory itself and does not recurse into its contents. Like --chown, it is docker-local only and only manages the default /var/lib/dokku/data/storage/<name> location - it is refused for k3s entries and for entries created with a custom <path>. That refusal also covers migrated legacy-* entries, whose host paths come from the original colon-form mount rather than the default location.

Updating a storage entry

Important

The property form is new as of 0.38.27. Prior versions used flags, which still work but emit a deprecation warning.

An existing entry is edited with storage:set, which takes a property and a value. Omitting the value unsets the property, restoring whatever the entry defaults to:

dokku storage:set node-js-data chown herokuish
dokku storage:set node-js-data chown

The following properties can be set:

Property	Description	Unsetting it means
chown	Ownership preset or numeric uid for the host directory	no chown is performed
mode	Octal permissions for the host directory	permissions are left alone
namespace	Namespace holding the PVC (k3s)	the default namespace
reclaim-policy	Whether the underlying volume survives storage:destroy	Retain
size	PVC size (k3s)	rejected, since k3s entries require a size
access-mode	PVC access mode (k3s)	rejected, see below
storage-class-name	PVC storage class (k3s)	rejected, see below

access-mode and storage-class-name cannot be changed on an entry that already exists, because Kubernetes cannot apply either to a bound PVC. Both a different value and an empty one are refused, since clearing is equally a change:

dokku storage:set node-js-data access-mode ReadWriteMany

 !     storage:set cannot change access-mode in place; recreate the entry

Setting chown or mode on a docker-local entry applies the change to the host directory immediately. Every other property is a metadata write, and k3s entries re-apply their helm release so the cluster picks the change up.

The older flag form - dokku storage:set node-js-data --mode 0770 - continues to work and warns. It gained unset semantics too, so --mode "" clears the mode the same way omitting the positional value does.

Annotations and labels

Important

New as of 0.38.27

Annotations and labels are attached to a storage entry one key at a time, matching the scheduler-k3s equivalents. On k3s they propagate to both the PersistentVolumeClaim and the PersistentVolume, so backup tools like Velero and Longhorn can find the volume.

dokku storage:annotations:set node-js-data backup.velero.io/backup-volumes node-js-data
dokku storage:labels:set node-js-data app.kubernetes.io/part-of billing

Keys may contain /, as the Kubernetes-style keys above do, and are stored verbatim. To clear a single key, omit the value. Other keys are left untouched, so a declarative caller does not need to re-send the whole set on every call:

dokku storage:annotations:set node-js-data backup.velero.io/backup-volumes

Configured annotations and labels can be inspected with the matching report commands. Without an entry name they cover every registered entry:

dokku storage:annotations:report
dokku storage:annotations:report node-js-data
dokku storage:labels:report node-js-data

=====> node-js-data annotations information
       Annotation backup.velero.io/backup-volumes: node-js-data

JSON output emits the keys flat, and a single value can be read directly with a flag of the form --storage-annotations.<key> (or --storage-labels.<key>), which requires an entry name:

dokku storage:annotations:report node-js-data --format json
dokku storage:annotations:report node-js-data --storage-annotations.backup.velero.io/backup-volumes

storage:create still accepts repeatable --annotation key=value and --label key=value flags for setting the initial set at creation time. The same flags on storage:set are deprecated in favor of these commands, because they replace the entire map rather than a single key.

Mounting storage into apps

Dokku supports mounting both explicit host paths as well as docker volumes via the storage:mount command. This takes two arguments, an app name and a host-path:container-path or docker-volume:container-path combination.

# mount the directory into your container's /app/storage directory, relative to the container root (/)
# explicit host paths _must_ exist prior to usage.
dokku storage:mount node-js-app /var/lib/dokku/data/storage/node-js-app:/app/storage

# mount the docker volume into your container's /app/storage directory, relative to the container root (/)
# docker volumes _must_ exist prior to usage.
dokku storage:mount node-js-app some-docker-volume:/app/storage

In the first example, Dokku will then mount the shared contents of /var/lib/dokku/data/storage/node-js-app to /app/storage inside the container. The mount point is not relative to your app's working directory, and is instead relative to the root (/) of the container. Mounts are only available for containers created via run and by the deploy process, and not during the build process. In addition, the host path is never auto-created by either Dokku or Docker, and should be an explicit path, not one relative to the current working directory.

If the /storage path within the container had pre-existing content, the container files will be over-written. This may be an issue for users that create assets at build time but then mount a directory at the same place during runtime. Files are not merged.

For named storage entries, additional Docker mount options can be passed via --volume-options. The value is a comma-separated mount-options string stored verbatim on the attachment and rendered into the -v flag at deploy time. This is useful for SELinux labels (Z, z) or hardening flags (noexec,nosuid):

dokku storage:create node-js-data
dokku storage:mount node-js-app node-js-data --container-dir /app/storage --volume-options Z

When combined with --volume-readonly, the rendered options become ro,<volume-options> - for example, --volume-options noexec,nosuid --volume-readonly renders as :ro,noexec,nosuid.

Re-running storage:mount against a named entry with the same --container-dir and --process-type updates the existing attachment's mount-time attributes (--phase, --volume-subpath, --volume-readonly, --volume-chown, --volume-options) in place rather than appending a duplicate. This is the idempotent equivalent of storage:set for entries, and lets declarative tooling change a mount-time attribute without an unmount-then-remount dance that would briefly drop the volume from storage:report. Mount-time fields are rewritten wholesale, not merged - omitting a flag on a re-mount clears any previously-set value. The legacy host:container[:opts] form still rejects duplicates with Mount path already exists..

Once persistent storage is mounted, the app requires a restart. See the process scaling documentation for more information.

dokku ps:restart app-name

Unmounting storage

If an app no longer requires a mounted volume or directory, the storage:unmount command can be called. This takes the same arguments as the storage:mount command, an app name and a host-path:container-path or docker-volume:container-path combination.

# unmount the directory from your container's /app/storage directory, relative to the container root (/)
dokku storage:unmount node-js-app /var/lib/dokku/data/storage/node-js-app:/app/storage

# unmount the docker volume from your container's /app/storage directory, relative to the container root (/)
dokku storage:unmount node-js-app some-docker-volume:/app/storage

Once persistent storage is unmounted, the app requires a restart. See the process scaling documentation for more information.

dokku ps:restart app-name

Destroying storage entries

A named storage entry can be removed with the storage:destroy command. The entry must first be unmounted from every app that mounts it.

dokku storage:destroy rdmtest-entry

As the command is destructive - removing the registry entry and, depending on the scheduler and reclaim policy, the underlying volume - it will default to asking for confirmation before executing the removal.

 !     WARNING: Potentially Destructive Action
 !     This command will destroy storage entry rdmtest-entry.
 !     To proceed, type "rdmtest-entry"
> rdmtest-entry
-----> Storage entry rdmtest-entry destroyed

The confirmation may be avoided by providing the --force flag, which is useful for non-interactive or automated callers:

dokku storage:destroy rdmtest-entry --force

The global --force flag is also supported:

dokku --force storage:destroy rdmtest-entry

Removing the host directory

Important

New as of 0.38.27

By default a docker-local entry's host directory survives storage:destroy - the entry is deregistered but the data stays on disk. The --destroy-host-dir flag removes the directory and everything in it:

dokku storage:destroy node-js-data --destroy-host-dir

 !     Storage entry node-js-data is backed by /var/lib/dokku/data/storage/node-js-data, which will be removed along with its contents.
 !     WARNING: Potentially Destructive Action
 !     This command will destroy storage entry node-js-data.
 !     To proceed, type "node-js-data"

The removal is recursive, so it succeeds whether or not the directory is empty. It is only permitted for entries at the default /var/lib/dokku/data/storage/<name> location; an entry created with a custom <path> is refused, and the operator removes the path themselves.

The same removal can be declared ahead of time with --reclaim-policy, which behaves for a docker-local host directory the way it behaves for a k3s PersistentVolume. An entry created with Delete has its host directory removed on storage:destroy without any extra flag, while Retain - the default when unset - keeps it:

dokku storage:create node-js-data --reclaim-policy Delete
dokku storage:destroy node-js-data --force

--destroy-host-dir is docker-local only. On a k3s entry the underlying volume is already governed by the reclaim policy recorded on the entry, so passing the flag is an error.

Displaying storage reports for an app

Important

New as of 0.8.1

You can get a report about the app's storage status using the storage:report command:

dokku storage:report

=====> node-js-app storage information
       Storage build mounts:
       Storage deploy mounts: -v /var/lib/dokku/data/storage/node-js-app:/app/storage
       Storage run mounts:  -v /var/lib/dokku/data/storage/node-js-app:/app/storage
=====> python-sample storage information
       Storage build mounts:
       Storage deploy mounts:
       Storage run mounts:
=====> ruby-sample storage information
       Storage build mounts:
       Storage deploy mounts:
       Storage run mounts:

You can run the command for a specific app also.

dokku storage:report node-js-app

=====> node-js-app storage information
       Storage build mounts:
       Storage deploy mounts: -v /var/lib/dokku/data/storage/node-js-app:/app/storage
       Storage run mounts:  -v /var/lib/dokku/data/storage/node-js-app:/app/storage

You can pass flags which will output only the value of the specific information you want. For example:

dokku storage:report node-js-app --storage-deploy-mounts

In addition to the aggregated Storage build/deploy/run mounts: lines, the report emits one flat dotted key per attachment field, indexed from 1. The key shape is --storage-attachment.<index>.<field> for each of entry-name, host-path, container-path, phases, process-type, subpath, readonly, volume-options, and volume-chown. Fields render as empty strings when unset, and attachments are ordered by lex-sort of the index (so 10 sorts before 2):

dokku storage:create node-js-data
dokku storage:mount node-js-app node-js-data --container-dir /app/storage --volume-options Z --volume-chown herokuish --volume-subpath uploads
dokku storage:report node-js-app

=====> node-js-app storage information
       Storage attachment 1 container path:  /app/storage
       Storage attachment 1 entry name:      node-js-data
       Storage attachment 1 host path:       /var/lib/dokku/data/storage/node-js-data
       Storage attachment 1 phases:          deploy,run
       Storage attachment 1 process type:    _default_
       Storage attachment 1 readonly:        false
       Storage attachment 1 subpath:         uploads
       Storage attachment 1 volume chown:    herokuish
       Storage attachment 1 volume options:  Z
       Storage build mounts:
       Storage deploy mounts: -v /var/lib/dokku/data/storage/node-js-data:/app/storage:Z
       Storage run mounts:  -v /var/lib/dokku/data/storage/node-js-data:/app/storage:Z

The same keys are exposed in JSON output, both in the stripped (attachment.1.volume-options) and legacy (storage-attachment.1.volume-options) forms:

dokku storage:report node-js-app --format json | jq '. | with_entries(select(.key | startswith("attachment.")))'

{
  "attachment.1.container-path": "/app/storage",
  "attachment.1.entry-name": "node-js-data",
  "attachment.1.host-path": "/var/lib/dokku/data/storage/node-js-data",
  "attachment.1.phases": "deploy,run",
  "attachment.1.process-type": "_default_",
  "attachment.1.readonly": "false",
  "attachment.1.subpath": "uploads",
  "attachment.1.volume-chown": "herokuish",
  "attachment.1.volume-options": "Z"
}

A single attachment field can be fetched directly via the info-flag form:

dokku storage:report node-js-app --storage-attachment.1.volume-options

Z

Use Cases

Sharing storage across deploys

Dokku is powered by Docker containers, which recommends in their best practices that containers be treated as ephemeral. In order to manage persistent storage for web apps, like user uploads or large binary assets like images, a directory outside the container should be mounted.

Shared storage between containers

When scaling your app, you may require a common location to access shared assets between containers, a storage mount can be used in this situation.

Shared storage across environments

Your app may be used in a cluster that requires containers or resources not running on the same host access your data. Mounting a shared file service (like S3FS or EFS) inside your container will give you great flexibility.

Backing up

Your app may have services that are running in memory and need to be backed up locally (like a key store). Mount a non ephemeral storage mount will allow backups that are not lost when the app is shut down.

Build phase

By default, Dokku will only bind storage mounts during the deploy and run phases. Under certain conditions, one might want to bind a storage mount during the build phase. This can be accomplished by using the docker-options plugin directly.

dokku docker-options:add node-js-app build "-v /tmp/python-test:/opt/test"

You cannot use mounted volumes during the build phase of a Dockerfile deploy. This is because Docker does not support volumes when executing docker build.

Warning

This can cause data loss if you bind a mount under /app in buildpack apps as herokuish will attempt to remove the original app path during the build phase.

App User and Persistent Storage file ownership (buildpack apps only)

Important

New as of 0.7.1

By default, Dokku will execute your buildpack app processes as the herokuishuser user. You may override this by setting the DOKKU_APP_USER config variable.

Note

this user must exist in your herokuish image.

Additionally, the default docker-local scheduler that comes with Dokku will ensure your storage mounts are owned by either herokuishuser or the overridden value you have set in DOKKU_APP_USER. See the docker-local scheduler documentation docs for more information.

Properties

Internal properties

The following property is recorded internally by the storage plugin and is not exposed via storage:report:

Property	Scope	Description	Source
legacy-mounts-migrated	per-app	Per-app marker recording that the app's legacy -v docker-options entries were drained into named storage entries plus attachments. Only set when at least one -v line was actually migrated; apps that have never had legacy mounts never receive this marker	plugins/storage/migrate.go writes "true" after a successful drain