mirror of
https://github.com/dokku/dokku.git
synced 2026-08-29 10:08:53 +02:00
feat: add vector-cron-sink for scheduled cron task output
Scheduled cron task output previously reached only the `dokku` user's cron mail, and could not be redirected because `app.json` rejects bare shell operators in a cron `command`. Setting `vector-cron-sink` on an app or globally routes that output to a dedicated sink instead, on both the `docker-local` and `k3s` schedulers, which keeps log destinations under operator control rather than in a deployed repository. Cron events carry `dokku_app` and `dokku_cron_id` fields so a sink can give each task its own destination. This also fixes a `k3s` bug where configuring a global `vector-sink` silently removed the vector prometheus exporter sink.
This commit is contained in:
@@ -124,6 +124,21 @@ The `/etc/vector` mount includes the `vector.json` configuration file, but also
|
||||
|
||||
The final volume mount - `/var/log/dokku/apps` - may be used for users that wish to ship logs to a file on disk that may be later logrotated. This directory is owned by the `dokku` user and group, with permissions set to `0755`. At this time, log-rotation is not configured for this directory.
|
||||
|
||||
Operators using a `file` sink are encouraged to configure rotation themselves, as Dokku will not truncate these files. A minimal `/etc/logrotate.d/dokku-app-logs` might look like:
|
||||
|
||||
```
|
||||
/var/log/dokku/apps/*/*.log {
|
||||
daily
|
||||
rotate 14
|
||||
compress
|
||||
missingok
|
||||
notifempty
|
||||
copytruncate
|
||||
}
|
||||
```
|
||||
|
||||
`copytruncate` is used because Vector holds the file open between writes.
|
||||
|
||||
#### Stopping the Vector container
|
||||
|
||||
Vector may be stopped via the `logs:vector-stop` command.
|
||||
@@ -282,6 +297,63 @@ This will transform the value to it's encoded form when configuring Vector sinks
|
||||
|
||||
Please read the [sink documentation](https://vector.dev/docs/reference/configuration/sinks/) for your sink of choice to configure the sink as desired.
|
||||
|
||||
#### Configuring a cron task log sink
|
||||
|
||||
Scheduled cron tasks run in one-off containers that carry the app's usual labels, so their output is already collected by the `vector-sink` configured for the app or globally. To send that output somewhere separate, set a `vector-cron-sink`.
|
||||
|
||||
```shell
|
||||
dokku logs:set node-js-app vector-cron-sink "console://?encoding[codec]=text"
|
||||
```
|
||||
|
||||
As with `vector-sink`, the value may be cleared by setting an empty value, and may also be set globally:
|
||||
|
||||
```shell
|
||||
dokku logs:set --global vector-cron-sink "console://?encoding[codec]=text"
|
||||
```
|
||||
|
||||
Setting a cron sink **moves** cron task output rather than copying it. Vector routes each log line to exactly one of the two sinks:
|
||||
|
||||
| Configuration | Where cron output goes | Where all other output goes |
|
||||
|---|---|---|
|
||||
| `vector-sink` only | `vector-sink` | `vector-sink` |
|
||||
| `vector-cron-sink` only | `vector-cron-sink` | nowhere |
|
||||
| both | `vector-cron-sink` | `vector-sink` |
|
||||
|
||||
If an app is already shipping to a metered service via `vector-sink`, adding a cron sink will stop cron output from arriving there.
|
||||
|
||||
Events on the cron branch have two extra fields added to them, so that they can be used in sink options that support templating:
|
||||
|
||||
- `dokku_app`: the name of the app the task belongs to
|
||||
- `dokku_cron_id`: the cron task ID, as shown by `dokku cron:list`
|
||||
|
||||
> [!WARNING]
|
||||
> Cron task containers are removed as soon as the task exits. Vector attaches to a container after it starts, so output from tasks that finish almost immediately - a bare `echo`, for instance - may be missed. Log shipping should not be relied on as the sole record that a task ran; use an external check for that.
|
||||
|
||||
##### Writing cron output to a file on disk
|
||||
|
||||
The `file` sink writes to a path within the vector container. The `/var/log/dokku/apps` directory is mounted into that container from the host at the same path, so it is the correct destination for output that should survive on the host.
|
||||
|
||||
```shell
|
||||
dokku logs:set node-js-app vector-cron-sink "file://?path=/var/log/dokku/apps/node-js-app/cron.log&encoding[codec]=text"
|
||||
```
|
||||
|
||||
Because `path` supports templating, `dokku_cron_id` can be used to give each task its own file:
|
||||
|
||||
```shell
|
||||
dokku logs:set node-js-app vector-cron-sink "file://?path=/var/log/dokku/apps/node-js-app/cron-{{ dokku_cron_id }}.log&encoding[codec]=text"
|
||||
```
|
||||
|
||||
Quoting the value is required, both for the `&` separators and for the spaces inside the template.
|
||||
|
||||
> [!WARNING]
|
||||
> Vector drops any event whose templated `path` references a field it cannot resolve. Only `dokku_app` and `dokku_cron_id` are guaranteed to exist on cron events - referencing anything else risks silently discarding log lines.
|
||||
|
||||
Vector creates missing parent directories, and buffers writes before flushing. Set `idle_timeout_secs` to shorten that delay for infrequent tasks:
|
||||
|
||||
```shell
|
||||
dokku logs:set node-js-app vector-cron-sink "file://?path=/var/log/dokku/apps/node-js-app/cron.log&encoding[codec]=text&idle_timeout_secs=5"
|
||||
```
|
||||
|
||||
##### Configuring the app label
|
||||
|
||||
Logs shipped by vector include the label `com.dokku.app-name`, which is an alias for the app name. This can be changed via the `app-label-alias` logs property with the `logs:set` command. Specifying a new alias will reload any running vector container.
|
||||
@@ -325,4 +397,5 @@ dokku logs:set --global app-label-alias
|
||||
| `max-size` | app + global | `10m` | `--logs-max-size`, `--logs-global-max-size`, `--logs-computed-max-size` | Maximum size of an individual log file before rotation |
|
||||
| `vector-image` | global only | _parsed from `plugins/logs/Dockerfile`_ | `--logs-global-vector-image`, `--logs-computed-vector-image` | Docker image used to run the vector log-shipper container |
|
||||
| `vector-networks` | global only | none | `--logs-global-vector-networks`, `--logs-computed-vector-networks` | Comma-separated list of docker networks the vector container is attached to |
|
||||
| `vector-cron-sink` | app + global | none | `--logs-vector-cron-sink`, `--logs-global-vector-cron-sink`, `--logs-computed-vector-cron-sink` | DSN-style sink configuration for scheduled cron task output; when set, cron output is routed here instead of to `vector-sink` |
|
||||
| `vector-sink` | app + global | none | `--logs-vector-sink`, `--logs-global-vector-sink`, `--logs-computed-vector-sink` | DSN-style sink configuration for vector (e.g. `console://` or `loki://...`) |
|
||||
|
||||
@@ -1113,6 +1113,24 @@ dokku scheduler-k3s:ensure-charts --charts vector
|
||||
|
||||
Please see the [vector logs documentation](/docs/deployment/logs.md#configuring-a-log-sink) for more information on specifying vector sinks.
|
||||
|
||||
#### Shipping cron task logs
|
||||
|
||||
The global `vector-cron-sink` property is also respected. When set, logs from cron task pods are routed to that sink instead of the sink configured for everything else, matching the behavior described in the [cron task log sink documentation](/docs/deployment/logs.md#configuring-a-cron-task-log-sink).
|
||||
|
||||
```shell
|
||||
dokku logs:set --global vector-cron-sink "console://?encoding[codec]=text"
|
||||
dokku scheduler-k3s:ensure-charts --charts vector
|
||||
```
|
||||
|
||||
As with `vector-sink`, only the global property is respected - a per-app `vector-cron-sink` has no effect on the `k3s` scheduler.
|
||||
|
||||
Cron events carry the same `dokku_app` and `dokku_cron_id` fields as they do on the `docker-local` scheduler, so sink configuration referencing them is portable between the two.
|
||||
|
||||
Two differences are worth noting:
|
||||
|
||||
- Templated values must be base64 encoded. Sink values containing `{{ }}` are interpreted by Helm at chart install time rather than by Vector, so the `base64enc:` form documented under [log sink DSN format](/docs/deployment/logs.md#log-sink-dsn-format) is required.
|
||||
- The `file` sink is not useful here. Vector runs as a DaemonSet agent, so a file path resolves to whichever node the agent is running on rather than to durable shared storage. Use a network sink and reference `dokku_cron_id` as a field instead of as a path component.
|
||||
|
||||
### Supported Resource Management Properties
|
||||
|
||||
The `k3s` scheduler supports a minimal list of resource _limits_ and _reservations_:
|
||||
|
||||
@@ -60,7 +60,32 @@ When running scheduled cron tasks, there are a few items to be aware of:
|
||||
- Scheduled cron tasks are supported on a per-scheduler basis. Schedulers that use the host crontab - such as `docker-local` - have their `app.json` cron tasks written to the `dokku` user crontab, while schedulers that manage their own cron backend - such as `k3s` - schedule them natively.
|
||||
- Tasks for _all_ apps managed by a host-crontab scheduler such as `docker-local` are written to a single crontab file owned by the `dokku` user. The `dokku` user's crontab should be considered reserved for this purpose.
|
||||
- The `command` is tokenized and exec'd directly inside the container. Shell features such as `;`, `&&`, `|`, and `>` are _not_ interpreted. Commands that contain a bare shell operator are rejected when `app.json` is validated at deploy time, so a malformed cron command will fail the deploy rather than silently fail to run. If shell semantics are required, wrap the command explicitly, for example `"sh -c 'do-thing > /var/log/x.log'"`.
|
||||
- Task output is written to the container's stdout and stderr, and can be persisted via Dokku's [vector integration](/docs/deployment/logs.md#configuring-a-cron-task-log-sink). See [persisting cron task output](#persisting-cron-task-output) below.
|
||||
- A cron task cannot declare a log file path in `app.json`. The crontab written for the `dokku` user contains only `dokku cron:run <app> <cron_id>` lines, and no path from a deployed repository is ever interpolated into it.
|
||||
|
||||
#### Persisting cron task output
|
||||
|
||||
Without further configuration, a task's output is only delivered to the `MAILTO` address configured for cron. To retain it, configure a sink via Dokku's [vector integration](/docs/deployment/logs.md#vector-logging-shipping).
|
||||
|
||||
Any sink configured for the app already receives cron task output alongside the app's other logs:
|
||||
|
||||
```shell
|
||||
dokku logs:set node-js-app vector-sink "console://?encoding[codec]=json"
|
||||
```
|
||||
|
||||
To keep cron output separate, set a `vector-cron-sink` instead. Cron output is then routed there rather than to the app's sink:
|
||||
|
||||
```shell
|
||||
dokku logs:set node-js-app vector-cron-sink "console://?encoding[codec]=text"
|
||||
```
|
||||
|
||||
To write it to a file on the host, target the `/var/log/dokku/apps` directory, which is mounted into the vector container. The `dokku_cron_id` field is available for templating, so each task can be given its own file:
|
||||
|
||||
```shell
|
||||
dokku logs:set node-js-app vector-cron-sink "file://?path=/var/log/dokku/apps/node-js-app/cron-{{ dokku_cron_id }}.log&encoding[codec]=text"
|
||||
```
|
||||
|
||||
See [configuring a cron task log sink](/docs/deployment/logs.md#configuring-a-cron-task-log-sink) for the routing rules, the available fields, and the caveat around very short-lived tasks.
|
||||
|
||||
### Changing cron management settings
|
||||
|
||||
|
||||
Reference in New Issue
Block a user