Cron containers were never reaped once they exceeded their active deadline, so a hung cron task ran indefinitely instead of being retired after 24 hours as documented. `cron:run` now also accepts a `--ttl-seconds` argument, matching the one `dokku run` already takes.
# History
## 0.38.27
Install/update via the bootstrap script:
```shell
wget -NP . https://dokku.com/install/v0.38.27/bootstrap.sh
sudo DOKKU_TAG=v0.38.27 bash bootstrap.sh
```
### Bug Fixes
- #8933: @josegonzalez Do not require a local image for k3s deploys
- #8930: @josegonzalez Do not import an already-migrated ENV file
- #8919: @josegonzalez Regenerate vector config on app lifecycle changes
- #8917: @josegonzalez Apply app-label-alias to shipped events
### New Features
- #8934: @josegonzalez Report traefik dns-provider env vars as global keys
- #8920: @josegonzalez Add storage directory mode and removal flags
- #8914: @josegonzalez Add vector-cron-sink for scheduled cron task output
### Tests
- #8931: @dependabot[bot] chore(deps): bump djangorestframework from 3.17.2 to 3.18.0 in /tests/apps/dockerfile-release
- #8927: @dependabot[bot] chore(deps): bump djangorestframework from 3.17.1 to 3.17.2 in /tests/apps/dockerfile-release
- #8923: @dependabot[bot] chore(deps): bump python from 3.15.0b4-bookworm to 3.15.0rc1-bookworm in /tests/apps/dockerfile-release
### Dependencies
- #8926: @dependabot[bot] chore(deps): bump github.com/gliderlabs/sigil from 0.12.0 to 0.12.1 in /plugins/nginx-vhosts
- #8925: @dependabot[bot] chore(deps): bump packaging from 26.2 to 26.3 in /docs/_build
- #8921: @dokku-bot chore: bump pack to 0.40.9
- #8924: @dependabot[bot] chore(deps): bump soupsieve from 2.9.1 to 2.9.2 in /docs/_build
- #8922: @dependabot[bot] chore(deps): bump python from 3.15.0b4-alpine to 3.15.0rc1-alpine in /docs/_build
- #8915: @dokku-bot chore: bump herokuish to 0.11.15
The dynamic `dns-provider-*` properties were reported as `--traefik-dns-provider-<env_var>`, outside the `global`/`computed` namespace every other traefik property uses, so a consumer reading the json report could not tell which of the two a key represented. They are now reported as `--traefik-global-dns-provider-<env_var>`. The `basic-auth-password` property is a credential as well and is now masked in the default report output on the same terms as the dns provider values, meaning the raw value is returned only for `--format json` or when the flag is queried by name.
Kubernetes pulls the app image itself, so a k3s host is free to reap its local copy while the workload keeps running, which the `registry` plugin already does on its own. Deploys, restarts, `dokku run`, and in-cluster cron no longer assert that the image is present locally, falling back to the metadata recorded in the app's current Helm release. A `ps:restart` naming a single process type now rolls only that process type's pods rather than silently redeploying every one. Apps with an `app.json` postdeploy task still require the image locally, as that task runs on the Dokku host.
The preserved copy holds everything the legacy file was not allowed to import, which includes keys that had been `config:unset` on purpose, so a revoked secret can outlive its revocation in a file `dokku config:*` no longer reads. Nothing said the copy was the operator's to remove once reviewed.
Releases 0.38.0 through 0.38.25 recorded the ENV file migration and deliberately left the file at the old path, so 0.38.26 treating a leftover file as a hand-edit replayed the environment as it stood at that upgrade over every `config:set` and `config:unset` made since, reinstating variables and secrets that had been deliberately unset. A file found at the old path once its migration is on record is no longer imported: it is removed when it agrees with the current config, and is otherwise moved aside to `ENV.migrated` with the keys it disagrees on named in a warning. That warning now reaches the operator during an upgrade rather than being swallowed, which is why the overwrite went unreported.
`storage:set` now takes `<name> <property> [<value>]` like every other `:set` command, where omitting the value unsets the property. Previously it took flags and could not distinguish an empty value from an omitted one, so nothing it set could ever be cleared. The flag form keeps working and emits a deprecation warning.
Annotations and labels move to `storage:annotations:set`, `storage:annotations:report`, `storage:labels:set`, and `storage:labels:report`, matching the `scheduler-k3s` equivalents. These operate on a single key, so clearing one leaves the rest in place rather than replacing the whole map as the `--annotation` and `--label` flags do.
`storage:create` and `storage:set` accept a `--mode` flag that sets the octal permissions of a docker-local host directory, and `storage:destroy` accepts a `--destroy-host-dir` flag that removes the directory along with its contents. A docker-local entry also honors `--reclaim-policy Delete` at destroy time now, matching how that policy governs a k3s PersistentVolume. Both are limited to the default `/var/lib/dokku/data/storage/<name>` location, the same restriction `--chown` already carries. `storage:set` applies `--chown` and `--mode` to the directory rather than only recording them.
The generated vector config is a snapshot of the app list and their sink properties, but it was only ever written by `logs:set` and `logs:vector-start`. Renaming an app left a source filtering on a label no container carries and gave the new name no source at all, so the app kept a sink with nothing feeding it. Cloning produced the same result for the clone, and destroying an app left its source and sink behind, the latter still pointing at an endpoint decommissioned along with the app. The global relabel transform embeds app names directly in generated VRL, so a rename also left behind a branch naming an app that no longer existed. Every case was silent, and the only repair was an operator running `logs:vector-start`. The `post-app-clone-setup`, `post-app-rename-setup` and `post-delete` triggers now rewrite the config, warning rather than failing so that a config write cannot abort the app operation whose state it is derived from.
Closes#8918.
The alias was only ever used to build the `include_labels` filter on the generated vector source, while dokku labels containers with `com.dokku.app-name` unconditionally. Setting the property therefore pointed the source at a label no container carries, and log collection stopped without any error. The source now always filters the label dokku applies, and a generated remap renames the field on its way to the sink, which is what the property was documented to do. An app whose own alias differs from the global one gets a branch in the global pipeline, so a per-app value is honored even when the app ships through the global sink.
Closes#8916.
# History
## 0.38.26
Install/update via the bootstrap script:
```shell
wget -NP . https://dokku.com/install/v0.38.26/bootstrap.sh
sudo DOKKU_TAG=v0.38.26 bash bootstrap.sh
```
### Bug Fixes
- #8906: @josegonzalez Match docker options by shell word when removing
### New Features
- #8911: @josegonzalez Route wildcard domains through traefik on k3s
- #8909: @josegonzalez Support manually managed cert issuers on k3s
- #8903: @josegonzalez Support kernel sysctls on the k3s scheduler
- #8856: @youdie006 Add pre-parsed port_mappings to ports:report json
### Refactors
- #8863: @josegonzalez Move host-crontab generation into cron plugin
### Documentation
- #8908: @josegonzalez Document --global on scheduler-k3s report and set
- #8858: @bakatz Added instructions for restoring backups on different CPU architectures.
### Tests
- #8893: @dependabot[bot] chore(deps): bump django from 5.2.16 to 5.2.17 in /tests/apps/dockerfile-release
- #8891: @dependabot[bot] chore(deps-dev): bump heroku/heroku-buildpack-php from 293 to 294 in /tests/apps/php
- #8877: @dependabot[bot] chore(deps): bump google.golang.org/grpc from 1.82.1 to 1.83.0 in /tests/apps/gogrpc
- #8874: @dependabot[bot] chore(deps): bump sass from 1.101.7 to 1.102.0 in /tests/apps/multi
- #8870: @dependabot[bot] chore(deps): bump sass from 1.101.6 to 1.101.7 in /tests/apps/multi
- #8866: @dependabot[bot] chore(deps): bump sass from 1.101.3 to 1.101.6 in /tests/apps/multi
- #8860: @dependabot[bot] chore(deps): bump setuptools from 78.1.1 to 83.0.0 in /tests/apps/dockerfile-release
- #8859: @dependabot[bot] chore(deps): bump immutable from 5.1.5 to 5.1.9 in /tests/apps/multi
- #8855: @dependabot[bot] chore(deps): bump sass from 1.101.0 to 1.101.3 in /tests/apps/multi
- #8857: @dependabot[bot] chore(deps): bump body-parser from 2.2.1 to 2.3.0 in /tests/apps/checks-root
- #8853: @dependabot[bot] chore(deps): bump python from 3.15.0b3-bookworm to 3.15.0b4-bookworm in /tests/apps/dockerfile-release
### Dependencies
- #8905: @dependabot[bot] chore(deps): bump oras.land/oras-go/v2 from 2.6.1 to 2.6.2 in /plugins/scheduler-k3s
- #8869: @dependabot[bot] chore(deps): bump github.com/mattn/go-isatty from 0.0.23 to 0.0.24 in /plugins/app-json
- #8887: @dependabot[bot] chore(deps): bump github.com/kedacore/keda/v2 from 2.20.1 to 2.20.2 in /plugins/scheduler-k3s
- #8883: @dokku-bot chore: bump docker-container-healthchecker to 0.16.0
- #8886: @dependabot[bot] chore(deps): update markdown requirement from <3.11,>=3.10.2 to >=3.10.3,<3.11 in /docs/_build
- #8892: @dependabot[bot] chore(deps): bump traefik from v3.7.9 to v3.7.10 in /plugins/traefik-vhosts
- #8885: @dokku-bot chore: bump dokku-update to 0.10.0
- #8884: @dokku-bot chore: bump procfile-util to 0.20.8
- #8882: @dokku-bot chore: bump docker-image-labeler to 0.10.0
- #8888: @dependabot[bot] chore(deps): bump github.com/traefik/traefik/v2 from 2.11.53 to 2.11.54 in /plugins/scheduler-k3s
- #8900: @dokku-bot chore: bump gliderlabs-sigil to 0.12.1
- #8899: @dokku-bot chore: bump herokuish to 0.11.14
- #8898: @dokku-bot chore: bump netrc to 0.11.1
- #8897: @dokku-bot chore: bump dokku-event-listener to 0.20.1
- #8896: @dokku-bot chore: bump sshcommand to 0.20.2
- #8895: @dokku-bot chore: bump lambda-builder to 0.9.4
- #8894: @dokku-bot chore: bump plugn to 0.17.1
- #8876: @dependabot[bot] chore(deps): bump github.com/cert-manager/cert-manager from 1.21.0 to 1.21.1 in /plugins/scheduler-k3s
- #8873: @dependabot[bot] chore(deps): bump traefik from v3.7.8 to v3.7.9 in /plugins/traefik-vhosts
- #8872: @dependabot[bot] chore(deps): bump github.com/traefik/traefik/v2 from 2.11.52 to 2.11.53 in /plugins/scheduler-k3s
- #8871: @dependabot[bot] chore(deps): bump k8s.io/kubernetes from 1.36.2 to 1.36.3 in /plugins/scheduler-k3s
- #8868: @dependabot[bot] chore(deps): bump k8s.io/kubectl from 0.36.2 to 0.36.3 in /plugins/scheduler-k3s
- #8867: @dependabot[bot] chore(deps): bump k8s.io/client-go from 0.36.2 to 0.36.3 in /plugins/scheduler-k3s
- #8865: @dependabot[bot] chore(deps): bump github.com/fluxcd/pkg/kustomize from 1.38.0 to 1.39.0 in /plugins/scheduler-k3s
- #8864: @dependabot[bot] chore(deps): bump soupsieve from 2.9 to 2.9.1 in /docs/_build
- #8854: @dependabot[bot] chore(deps): bump python from 3.15.0b3-alpine to 3.15.0b4-alpine in /docs/_build
- #8852: @dependabot[bot] chore(deps): bump soupsieve from 2.8.4 to 2.9 in /docs/_build
- #8851: @dependabot[bot] chore(deps): bump mkdocs-material from 9.7.6 to 9.7.7 in /docs/_build
- #8850: @dependabot[bot] chore(deps): bump actions/setup-python from 6 to 7
### Other
- #8907: @josegonzalez fix: migrate env files before reading deprecated vars
- #8881: @josegonzalez Ignore minor and patch updates for github actions
Scheduled cron task output previously reached only the `dokku` user's cron mail, and could not be redirected because `app.json` rejects bare shell operators in a cron `command`. Setting `vector-cron-sink` on an app or globally routes that output to a dedicated sink instead, on both the `docker-local` and `k3s` schedulers, which keeps log destinations under operator control rather than in a deployed repository. Cron events carry `dokku_app` and `dokku_cron_id` fields so a sink can give each task its own destination. This also fixes a `k3s` bug where configuring a global `vector-sink` silently removed the vector prometheus exporter sink.
Traefik matches hosts exactly, so an app serving a wildcard domain under the `traefik` ingress class had a valid certificate but silently 404d on every request. Wildcard domains now render as a `HostRegexp` rule that matches a single label, the same semantics as a Kubernetes wildcard host, so both ingress classes behave the same. Those routes carry an explicit low priority so an exact domain on any app still wins over another app's wildcard, mirroring ingress-nginx.
The `cert-issuer-name` and `cert-issuer-kind` properties point an app's generated `Certificate` at a cert-manager issuer created outside of Dokku, allowing certificates to be issued through solvers the built-in letsencrypt integration cannot use, such as `dns01` for wildcard certificates. Setting an issuer enables https on its own, as a manually managed issuer has no email for Dokku to configure. An imported certificate still takes precedence, and `letsencrypt-server false` remains the single off switch. Dokku warns before a build starts when the referenced issuer is absent from the cluster, without blocking the deploy. Wildcard domains no longer collide with their apex domain when generating ingress names, and `letsencrypt-server` values are now validated when set rather than at deploy time.
The usage strings for `scheduler-k3s:report` and `scheduler-k3s:set` omitted the `--global` option, which is the only way to reach the scheduler-wide report since a bare `scheduler-k3s:report` iterates every app, and `:report` also omitted `--format stdout|json`. The command listing in the k3s documentation is resynced with the help output, which additionally restores flags that had been dropped from `scheduler-k3s:cluster:add`, `scheduler-k3s:cluster:list`, and `scheduler-k3s:initialize`.
Install steps run in alphabetical order of the enabled plugin directory, so `apps`, `builder`, and `checks` read an app's environment before the `config` plugin had moved the `ENV` file to its new location. The read came back empty, so their deprecated `DOKKU_*` variables were never migrated to the matching plugin property and were never unset, with nothing reported either way: `dokku config:show` kept listing the variable while the plugin behaved as though it were unset. The relocation now runs before any deprecated variable is read, whatever the install order, and each old file is removed as soon as it has been drained rather than on a later install, which also covers the global file that was never removed at all. A file that reappears at the old path can only have been written by hand, so it is merged in with a warning naming its keys instead of being discarded.
Options drained out of the pre-0.38.0 `DOCKER_OPTIONS_<PHASE>` files were copied verbatim rather than re-serialized the way `docker-options:add` stores them, so `docker-options:remove` compared the canonical string it builds against a stored value that could never match it and exited successfully without removing anything. Removal now matches stored options by shell word, and stored options are rewritten into the canonical form once on upgrade, which additionally splits an entry that carried several flags on a single line into one entry per flag so a single flag can be removed and so the readers that match on a flag prefix see one value per entry. The leftover `.migrated` sentinel drain is restored to running ahead of the global short-circuit that had made it unreachable, and the plugin's Go tests are added to the test target that had never run them.
Sysctls the kernel does not namespace, such as `vm.max_map_count`, cannot be set from a pod spec and previously had no answer beyond editing `/etc/sysctl.d` on each host by hand. `scheduler-k3s:node-sysctls:set` now applies them through a privileged daemonset, which reaches nodes joined later and reapplies after a reboot. Sysctls may be scoped to a node profile, with a profile scope inheriting the global values and overriding them on conflict so that every node is covered by exactly one daemonset. Clearing a sysctl stops dokku managing it but does not restore the previous value, which persists until the node reboots.
The `docker-local` scheduler supports `--sysctl` for free because docker options are passed verbatim to `docker run`, but the k3s scheduler silently dropped it. Namespaced sysctls now render into the pod's `securityContext.sysctls` for deployments, cron jobs, and one-off runs. A sysctl the kernel does not namespace fails the deploy instead of being dropped, since it cannot take effect within a pod regardless of what was requested.
Node profiles controlled how a node joined the cluster but left no trace on the node afterwards, so a profile could not be selected against with `kubectl`, a `nodeSelector`, or a node affinity rule. Nodes joined without a profile are left unlabeled, and the server node created by `scheduler-k3s:initialize` never carries the label since it does not pass through `scheduler-k3s:cluster:add`.
The server node created by `scheduler-k3s:initialize` had no way to receive kubelet arguments, unlike nodes joined through `scheduler-k3s:cluster:add` or configured via `scheduler-k3s:profiles:add`. This meant settings such as `allowed-unsafe-sysctls` were unreachable on a single-node install.
Host-crontab generation for `app.json` cron tasks now lives in the `cron` plugin, gated by a new `scheduler-uses-host-cron` trigger that the `docker-local` scheduler answers true while self-managed schedulers such as `k3s` answer false. This lets any host-cron scheduler participate in normal `app.json` cron without coupling to `scheduler-docker-local` or duplicating the crontab writer, while the `k3s` scheduler continues to manage its own in-cluster cron jobs.
Closes#8862.
The `letsencrypt-email-prod` and `letsencrypt-email-stag` properties can now be set per app in addition to globally, resolving app-level before the global value for the app's selected `letsencrypt-server`. An app that sets its own email renders a namespaced cert-manager `Issuer` using that email, while apps without an override continue to use the shared `ClusterIssuer` with the global email.
