feat: add vector-cron-sink for scheduled cron task output

Scheduled cron task output previously reached only the `dokku` user's cron mail, and could not be redirected because `app.json` rejects bare shell operators in a cron `command`. Setting `vector-cron-sink` on an app or globally routes that output to a dedicated sink instead, on both the `docker-local` and `k3s` schedulers, which keeps log destinations under operator control rather than in a deployed repository. Cron events carry `dokku_app` and `dokku_cron_id` fields so a sink can give each task its own destination. This also fixes a `k3s` bug where configuring a global `vector-sink` silently removed the vector prometheus exporter sink.
This commit is contained in:
Jose Diaz-Gonzalez
2026-08-09 00:54:09 -04:00
parent 047be485a2
commit 52b26a3760
14 changed files with 1346 additions and 112 deletions

View File

@@ -124,6 +124,21 @@ The `/etc/vector` mount includes the `vector.json` configuration file, but also
The final volume mount - `/var/log/dokku/apps` - may be used for users that wish to ship logs to a file on disk that may be later logrotated. This directory is owned by the `dokku` user and group, with permissions set to `0755`. At this time, log-rotation is not configured for this directory.
Operators using a `file` sink are encouraged to configure rotation themselves, as Dokku will not truncate these files. A minimal `/etc/logrotate.d/dokku-app-logs` might look like:
```
/var/log/dokku/apps/*/*.log {
daily
rotate 14
compress
missingok
notifempty
copytruncate
}
```
`copytruncate` is used because Vector holds the file open between writes.
#### Stopping the Vector container
Vector may be stopped via the `logs:vector-stop` command.
@@ -282,6 +297,63 @@ This will transform the value to it's encoded form when configuring Vector sinks
Please read the [sink documentation](https://vector.dev/docs/reference/configuration/sinks/) for your sink of choice to configure the sink as desired.
#### Configuring a cron task log sink
Scheduled cron tasks run in one-off containers that carry the app's usual labels, so their output is already collected by the `vector-sink` configured for the app or globally. To send that output somewhere separate, set a `vector-cron-sink`.
```shell
dokku logs:set node-js-app vector-cron-sink "console://?encoding[codec]=text"
```
As with `vector-sink`, the value may be cleared by setting an empty value, and may also be set globally:
```shell
dokku logs:set --global vector-cron-sink "console://?encoding[codec]=text"
```
Setting a cron sink **moves** cron task output rather than copying it. Vector routes each log line to exactly one of the two sinks:
| Configuration | Where cron output goes | Where all other output goes |
|---|---|---|
| `vector-sink` only | `vector-sink` | `vector-sink` |
| `vector-cron-sink` only | `vector-cron-sink` | nowhere |
| both | `vector-cron-sink` | `vector-sink` |
If an app is already shipping to a metered service via `vector-sink`, adding a cron sink will stop cron output from arriving there.
Events on the cron branch have two extra fields added to them, so that they can be used in sink options that support templating:
- `dokku_app`: the name of the app the task belongs to
- `dokku_cron_id`: the cron task ID, as shown by `dokku cron:list`
> [!WARNING]
> Cron task containers are removed as soon as the task exits. Vector attaches to a container after it starts, so output from tasks that finish almost immediately - a bare `echo`, for instance - may be missed. Log shipping should not be relied on as the sole record that a task ran; use an external check for that.
##### Writing cron output to a file on disk
The `file` sink writes to a path within the vector container. The `/var/log/dokku/apps` directory is mounted into that container from the host at the same path, so it is the correct destination for output that should survive on the host.
```shell
dokku logs:set node-js-app vector-cron-sink "file://?path=/var/log/dokku/apps/node-js-app/cron.log&encoding[codec]=text"
```
Because `path` supports templating, `dokku_cron_id` can be used to give each task its own file:
```shell
dokku logs:set node-js-app vector-cron-sink "file://?path=/var/log/dokku/apps/node-js-app/cron-{{ dokku_cron_id }}.log&encoding[codec]=text"
```
Quoting the value is required, both for the `&` separators and for the spaces inside the template.
> [!WARNING]
> Vector drops any event whose templated `path` references a field it cannot resolve. Only `dokku_app` and `dokku_cron_id` are guaranteed to exist on cron events - referencing anything else risks silently discarding log lines.
Vector creates missing parent directories, and buffers writes before flushing. Set `idle_timeout_secs` to shorten that delay for infrequent tasks:
```shell
dokku logs:set node-js-app vector-cron-sink "file://?path=/var/log/dokku/apps/node-js-app/cron.log&encoding[codec]=text&idle_timeout_secs=5"
```
##### Configuring the app label
Logs shipped by vector include the label `com.dokku.app-name`, which is an alias for the app name. This can be changed via the `app-label-alias` logs property with the `logs:set` command. Specifying a new alias will reload any running vector container.
@@ -325,4 +397,5 @@ dokku logs:set --global app-label-alias
| `max-size` | app + global | `10m` | `--logs-max-size`, `--logs-global-max-size`, `--logs-computed-max-size` | Maximum size of an individual log file before rotation |
| `vector-image` | global only | _parsed from `plugins/logs/Dockerfile`_ | `--logs-global-vector-image`, `--logs-computed-vector-image` | Docker image used to run the vector log-shipper container |
| `vector-networks` | global only | none | `--logs-global-vector-networks`, `--logs-computed-vector-networks` | Comma-separated list of docker networks the vector container is attached to |
| `vector-cron-sink` | app + global | none | `--logs-vector-cron-sink`, `--logs-global-vector-cron-sink`, `--logs-computed-vector-cron-sink` | DSN-style sink configuration for scheduled cron task output; when set, cron output is routed here instead of to `vector-sink` |
| `vector-sink` | app + global | none | `--logs-vector-sink`, `--logs-global-vector-sink`, `--logs-computed-vector-sink` | DSN-style sink configuration for vector (e.g. `console://` or `loki://...`) |

View File

@@ -1113,6 +1113,24 @@ dokku scheduler-k3s:ensure-charts --charts vector
Please see the [vector logs documentation](/docs/deployment/logs.md#configuring-a-log-sink) for more information on specifying vector sinks.
#### Shipping cron task logs
The global `vector-cron-sink` property is also respected. When set, logs from cron task pods are routed to that sink instead of the sink configured for everything else, matching the behavior described in the [cron task log sink documentation](/docs/deployment/logs.md#configuring-a-cron-task-log-sink).
```shell
dokku logs:set --global vector-cron-sink "console://?encoding[codec]=text"
dokku scheduler-k3s:ensure-charts --charts vector
```
As with `vector-sink`, only the global property is respected - a per-app `vector-cron-sink` has no effect on the `k3s` scheduler.
Cron events carry the same `dokku_app` and `dokku_cron_id` fields as they do on the `docker-local` scheduler, so sink configuration referencing them is portable between the two.
Two differences are worth noting:
- Templated values must be base64 encoded. Sink values containing `{{ }}` are interpreted by Helm at chart install time rather than by Vector, so the `base64enc:` form documented under [log sink DSN format](/docs/deployment/logs.md#log-sink-dsn-format) is required.
- The `file` sink is not useful here. Vector runs as a DaemonSet agent, so a file path resolves to whichever node the agent is running on rather than to durable shared storage. Use a network sink and reference `dokku_cron_id` as a field instead of as a path component.
### Supported Resource Management Properties
The `k3s` scheduler supports a minimal list of resource _limits_ and _reservations_:

View File

@@ -60,7 +60,32 @@ When running scheduled cron tasks, there are a few items to be aware of:
- Scheduled cron tasks are supported on a per-scheduler basis. Schedulers that use the host crontab - such as `docker-local` - have their `app.json` cron tasks written to the `dokku` user crontab, while schedulers that manage their own cron backend - such as `k3s` - schedule them natively.
- Tasks for _all_ apps managed by a host-crontab scheduler such as `docker-local` are written to a single crontab file owned by the `dokku` user. The `dokku` user's crontab should be considered reserved for this purpose.
- The `command` is tokenized and exec'd directly inside the container. Shell features such as `;`, `&&`, `|`, and `>` are _not_ interpreted. Commands that contain a bare shell operator are rejected when `app.json` is validated at deploy time, so a malformed cron command will fail the deploy rather than silently fail to run. If shell semantics are required, wrap the command explicitly, for example `"sh -c 'do-thing > /var/log/x.log'"`.
- Task output is written to the container's stdout and stderr, and can be persisted via Dokku's [vector integration](/docs/deployment/logs.md#configuring-a-cron-task-log-sink). See [persisting cron task output](#persisting-cron-task-output) below.
- A cron task cannot declare a log file path in `app.json`. The crontab written for the `dokku` user contains only `dokku cron:run <app> <cron_id>` lines, and no path from a deployed repository is ever interpolated into it.
#### Persisting cron task output
Without further configuration, a task's output is only delivered to the `MAILTO` address configured for cron. To retain it, configure a sink via Dokku's [vector integration](/docs/deployment/logs.md#vector-logging-shipping).
Any sink configured for the app already receives cron task output alongside the app's other logs:
```shell
dokku logs:set node-js-app vector-sink "console://?encoding[codec]=json"
```
To keep cron output separate, set a `vector-cron-sink` instead. Cron output is then routed there rather than to the app's sink:
```shell
dokku logs:set node-js-app vector-cron-sink "console://?encoding[codec]=text"
```
To write it to a file on the host, target the `/var/log/dokku/apps` directory, which is mounted into the vector container. The `dokku_cron_id` field is available for templating, so each task can be given its own file:
```shell
dokku logs:set node-js-app vector-cron-sink "file://?path=/var/log/dokku/apps/node-js-app/cron-{{ dokku_cron_id }}.log&encoding[codec]=text"
```
See [configuring a cron task log sink](/docs/deployment/logs.md#configuring-a-cron-task-log-sink) for the routing rules, the available fields, and the caveat around very short-lived tasks.
### Changing cron management settings