Merge pull request #3381 from dokku/3136-release-support

Add support for the Procfile release command
This commit is contained in:
Jose Diaz-Gonzalez
2019-01-08 08:01:02 -05:00
committed by GitHub
11 changed files with 253 additions and 173 deletions

View File

@@ -2,25 +2,52 @@
> New as of 0.5.0
Sometimes you need to run a command on at deployment time, but before an app is completely deployed.
Common use cases include:
Sometimes you need to run a command on at deployment time, but before an app is completely deployed. Common use cases include:
* Checking a database is initialized
* Running database migrations
* Any commands required to set up the server (e.g. something like a Django `collectstatic`)
## `app.json` and `scripts.dokku`
To support this, Dokku provides support for a special `release` command within your app's `Procfile`, as well as a special `scripts.dokku` key inside of your app's `app.json` file. Be aware that all commands are run within the context of the built docker image - no commands affect the host unless there are volume mounts attached to your app.
Dokku accomplishes this by using an `app.json` file. The format in use is similar to format of Heroku's [app.json](https://devcenter.heroku.com/articles/app-json-schema).
However, Dokku currently only supports the nodes `scripts.dokku.predeploy` and `scripts.dokku.postdeploy`.
For buildpack apps, simply place an `app.json` file in the root of your repository.
For dockerfile apps, place `app.json` in the configured `WORKDIR` directory; otherwise Dokku defaults to the buildpack app behavior of looking in `/app`.
Each "phase" has different expectations and limitations:
> NOTE: postdeploy changes are *NOT* committed to the app image.
- `app.json`: `scripts.dokku.predeploy`
- When to use: This should be used if your app does not support arbitrary build commands and you need to make changes to the built image.
- Are changes committed to the image at this phase: Yes
- Example use-cases
- Bundling assets in a slightly different way
- Installing a custom package from source or copying a binary into place
- `app.json`: `scripts.dokku.postdeploy`
- When to use: This should be used in conjunction with external systems to signal the completion of your deploy.
- Are changes committed to the image at this phase: No
- Example use-cases
- Notifying slack that your app is deployed
- Coordinating traffic routing with a central load balancer
- `Procfile`: `release`
- When to use: This should be used in conjunction with external systems to signal the completion of your app image build.
- Are changes committed to the image at this phase: No
- Example use-cases
- Sending CSS, JS, and other assets from your apps slug to a CDN or S3 bucket
- Priming or invalidating cache stores
- Running database migrations
### Example app.json
Please keep the above in mind when utilizing deployment tasks.
> NOTE: Only the `scripts.dokku.predeploy` and `scripts.dokku.postdeploy` tasks are supported by Dokku at this time. All other fields will be ignored and can be omitted.
> To execute commands on the host during a release phase, see the [plugin creation documentation](/docs/development/plugin-creation) docs for more information on building your own custom plugin.
## `app.json` deployment tasks
Dokku provides limited support for the `app.json` manifest from Heroku (documentation available [here](https://devcenter.heroku.com/articles/app-json-schema)). The keys available for use with Deployment Tasks are:
- `scripts.dokku.predeploy`: This is run _after_ an app's docker image is built, but _before_ any containers are scheduled. Changes made to your image are committed at this phase.
- `scripts.dokku.postdeploy`: This is run _after_ an app's containers are scheduled. Changes made to your image are *not* committed at this phase.
For buildpack-based deployments, the location of the `app.json` file should be at the root of your repository. Dockefile-based app deploys should have the `app.json` in the configured `WORKDIR` directory; otherwise Dokku defaults to the buildpack app behavior of looking in `/app`.
> Warning: Any failed `app.json` deployment task will fail the deploy. In the case of either phase, a failure will not affect any running containers.
The following is an example `app.json` file. Please note that only the `scripts.dokku.predeploy` and `scripts.dokku.postdeploy` tasks are supported by Dokku at this time. All other fields will be ignored and can be omitted.
```json
{
@@ -32,3 +59,19 @@ For dockerfile apps, place `app.json` in the configured `WORKDIR` directory; oth
}
}
```
## Procfile Release command
> New as of 0.14.0
The `Procfile` also supports a special `release` command which acts in a similar way to the [Heroku Release Phase](https://devcenter.heroku.com/articles/release-phase). This command is executed _after_ an app's docker image is built, but _before_ any containers are scheduled. This is also run _after_ any command executed by `scripts.dokku.predeploy`.
To use the `release` command, simply add a `release` stanza to your Procfile.
```Procfile
release: curl https://some.external.api.service.com/deployment?state=built
```
Unlike the `scripts.dokku.predeploy` command, changes made during by the `release` command are *not* persisted to disk.
> Warning: scaling the release command up will likely result in unspecified issues within your deployment, and is highly discouraged.

View File

@@ -1,106 +0,0 @@
#!/usr/bin/env bash
set -eo pipefail
[[ $DOKKU_TRACE ]] && set -x
source "$PLUGIN_CORE_AVAILABLE_PATH/common/functions"
source "$PLUGIN_AVAILABLE_PATH/config/functions"
get_phase_script() {
declare desc="extracts app.json from app image and returns the appropriate json key/value"
local IMAGE="$1"
local PHASE_SCRIPT_KEY="$2"
local GET_PHASE_SCRIPT_TMP_WORK_DIR=$(mktemp -d "/tmp/dokku_get_phase_script.XXXX")
local APP_JSON_FILE="$GET_PHASE_SCRIPT_TMP_WORK_DIR/app.json"
trap 'rm -rf "$GET_PHASE_SCRIPT_TMP_WORK_DIR" > /dev/null' RETURN INT TERM
copy_from_image "$IMAGE" "app.json" "$GET_PHASE_SCRIPT_TMP_WORK_DIR" 2>/dev/null || true
if [[ -f "$APP_JSON_FILE" ]]; then
local VALUE=$(get_json_value "scripts.dokku.${PHASE_SCRIPT_KEY}" <"$APP_JSON_FILE")
else
return 0
fi
echo "$VALUE"
}
execute_script() {
declare desc="executes appropriate phase script key from app.json"
local APP="$1"
local IMAGE_TAG="$2"
local PHASE_SCRIPT_KEY="$3"
local IMAGE id
IMAGE=$(get_deploying_app_image_name "$APP" "$IMAGE_TAG")
local SCRIPT_CMD=$(get_phase_script "$IMAGE" "$PHASE_SCRIPT_KEY" 2>/dev/null)
if [[ -n "$SCRIPT_CMD" ]]; then
dokku_log_info1 "Running '$SCRIPT_CMD' in app container"
local COMMAND
COMMAND="set -eo pipefail; [[ \$DOKKU_TRACE ]] && set -x ; "
COMMAND+=" if [[ -d '/app' ]]; then "
COMMAND+=" export HOME=/app ; "
COMMAND+=" cd \$HOME ; "
COMMAND+=" fi ; "
COMMAND+=" if [[ -d '/app/.profile.d' ]]; then "
COMMAND+=" for file in /app/.profile.d/*; do source \$file; done ; "
COMMAND+=" fi ; "
COMMAND+=" if [[ -d '/cache' ]]; then "
COMMAND+=" echo restoring installation cache... ; "
COMMAND+=" rm -rf /tmp/cache ; "
COMMAND+=" ln -sf /cache /tmp/cache ; "
COMMAND+=" fi ; "
if [[ "$SCRIPT_CMD" == /* ]]; then
local SCRIPT_BIN="$(echo "$SCRIPT_CMD" | cut -d' ' -f1)"
COMMAND+=" if [[ ! -x \"$SCRIPT_BIN\" ]]; then "
COMMAND+=" echo specified binary is not executable ; "
COMMAND+=" exit 1 ; "
COMMAND+=" fi "
fi
COMMAND+=" $SCRIPT_CMD || exit 1;"
COMMAND+=" if [[ -d '/cache' ]]; then "
COMMAND+=" echo removing installation cache... ; "
COMMAND+=" rm -f /tmp/cache ; "
COMMAND+=" fi "
local CACHE_DIR="$DOKKU_ROOT/$APP/cache"
local CACHE_HOST_DIR="$DOKKU_HOST_ROOT/$APP/cache"
[[ -d $CACHE_DIR ]] || mkdir -p "$CACHE_DIR"
local DOCKER_ARGS=$(: | plugn trigger docker-args-deploy "$APP" "$IMAGE_TAG")
# strip --restart args from DOCKER_ARGS
local DOCKER_ARGS=$(sed -e "s/--restart=[[:graph:]]\+[[:blank:]]\?//g" <<<"$DOCKER_ARGS")
# eval args as array to respect escapes
declare -a ARG_ARRAY
eval "ARG_ARRAY=($DOCKER_ARGS)"
local DOKKU_APP_SHELL="/bin/bash"
DOKKU_APP_SHELL="$(config_get --global DOKKU_APP_SHELL || echo "$DOKKU_APP_SHELL")"
DOKKU_APP_SHELL="$(config_get "$APP" DOKKU_APP_SHELL || echo "$DOKKU_APP_SHELL")"
[[ -z "$DOKKU_APP_SHELL" ]] && DOKKU_APP_SHELL="/bin/bash"
id=$(docker run "$DOKKU_GLOBAL_RUN_ARGS" -e DOKKU_TRACE="$DOKKU_TRACE" --label=dokku_phase_script="${PHASE_SCRIPT_KEY}" -d -v "$CACHE_HOST_DIR:/cache" "${ARG_ARRAY[@]}" "$IMAGE" "$DOKKU_APP_SHELL" -c "$COMMAND")
if test "$(docker wait "$id")" -eq 0; then
dokku_container_log_verbose_quiet "$id"
if [[ "$PHASE_SCRIPT_KEY" != "postdeploy" ]]; then
if ! is_image_herokuish_based "$IMAGE"; then
local DOKKU_DOCKERFILE_ENTRYPOINT=$(config_get "$APP" DOKKU_DOCKERFILE_ENTRYPOINT)
local DOKKU_DOCKERFILE_CMD=$(config_get "$APP" DOKKU_DOCKERFILE_CMD)
[[ -z "$DOKKU_DOCKERFILE_ENTRYPOINT" ]] && DOKKU_DOCKERFILE_ENTRYPOINT="$(get_entrypoint_from_image "$IMAGE")"
[[ -z "$DOKKU_DOCKERFILE_CMD" ]] && DOKKU_DOCKERFILE_CMD="$(get_cmd_from_image "$IMAGE")"
[[ -n "$DOKKU_DOCKERFILE_ENTRYPOINT" ]] && local DOCKER_COMMIT_ENTRYPOINT_CHANGE_ARG="--change='$DOKKU_DOCKERFILE_ENTRYPOINT'"
[[ -n "$DOKKU_DOCKERFILE_CMD" ]] && local DOCKER_COMMIT_CMD_CHANGE_ARG="--change='$DOKKU_DOCKERFILE_CMD'"
local DOCKER_COMMIT_ARGS="$DOCKER_COMMIT_ENTRYPOINT_CHANGE_ARG $DOCKER_COMMIT_CMD_CHANGE_ARG"
fi
# shellcheck disable=SC2086
eval docker commit $DOCKER_COMMIT_ARGS "$id" "$IMAGE" >/dev/null
fi
else
dokku_container_log_verbose_quiet "$id"
dokku_log_fail "execution of '$SCRIPT_CMD' failed!"
fi
fi
}

View File

@@ -1,18 +0,0 @@
#!/usr/bin/env bash
set -eo pipefail
[[ $DOKKU_TRACE ]] && set -x
source "$PLUGIN_CORE_AVAILABLE_PATH/common/functions"
source "$PLUGIN_CORE_AVAILABLE_PATH/00_dokku-standard/exec-app-json-scripts"
exec_app_json_scripts() {
declare desc="core app.json scripts execution"
local trigger="post-deploy app_json_scripts"
local APP="$1"
local IMAGE_TAG="$4"
local PHASE_SCRIPT_KEY="postdeploy"
dokku_log_info1 "Attempting to run scripts.dokku.$PHASE_SCRIPT_KEY from app.json (if defined)"
execute_script "$APP" "$IMAGE_TAG" "$PHASE_SCRIPT_KEY"
}
exec_app_json_scripts "$@"

View File

@@ -1,18 +0,0 @@
#!/usr/bin/env bash
set -eo pipefail
[[ $DOKKU_TRACE ]] && set -x
source "$PLUGIN_CORE_AVAILABLE_PATH/common/functions"
source "$PLUGIN_CORE_AVAILABLE_PATH/00_dokku-standard/exec-app-json-scripts"
exec_app_json_scripts() {
declare desc="core app.json scripts execution"
local trigger="pre-deploy app_json_scripts"
local APP="$1"
local IMAGE_TAG="$2"
local PHASE_SCRIPT_KEY="predeploy"
dokku_log_info1 "Attempting to run scripts.dokku.$PHASE_SCRIPT_KEY from app.json (if defined)"
execute_script "$APP" "$IMAGE_TAG" "$PHASE_SCRIPT_KEY"
}
exec_app_json_scripts "$@"

View File

@@ -0,0 +1,122 @@
#!/usr/bin/env bash
set -eo pipefail
[[ $DOKKU_TRACE ]] && set -x
source "$PLUGIN_CORE_AVAILABLE_PATH/common/functions"
source "$PLUGIN_AVAILABLE_PATH/config/functions"
source "$PLUGIN_AVAILABLE_PATH/ps/functions"
get_phase_script() {
declare desc="extracts app.json from app image and returns the appropriate json key/value"
declare IMAGE_TAG="$1" PHASE_SCRIPT_KEY="$2"
local GET_PHASE_SCRIPT_TMP_WORK_DIR=$(mktemp -d "/tmp/dokku_get_phase_script.XXXX")
local APP_JSON_FILE="$GET_PHASE_SCRIPT_TMP_WORK_DIR/app.json"
trap 'rm -rf "$GET_PHASE_SCRIPT_TMP_WORK_DIR" > /dev/null' RETURN INT TERM
copy_from_image "$IMAGE" "app.json" "$GET_PHASE_SCRIPT_TMP_WORK_DIR" 2>/dev/null || true
if [[ ! -f "$APP_JSON_FILE" ]]; then
return 0
fi
get_json_value "scripts.dokku.${PHASE_SCRIPT_KEY}" <"$APP_JSON_FILE"
}
get_release_cmd() {
declare desc="extracts the release command from a given app's procfile"
declare APP="$1" IMAGE_TAG="$2"
extract_procfile "$APP" "$IMAGE_TAG" >/dev/null
trap 'remove_procfile $APP' RETURN INT TERM EXIT
get_cmd_from_procfile "$APP" "release" "5000" || true
}
execute_script() {
declare desc="executes appropriate phase script key from app.json"
declare APP="$1" IMAGE_TAG="$2" PHASE_SCRIPT_KEY="$3"
local IMAGE id SCRIPT_CMD
IMAGE=$(get_deploying_app_image_name "$APP" "$IMAGE_TAG")
if [[ "$PHASE_SCRIPT_KEY" == "release" ]]; then
SCRIPT_CMD=$(get_release_cmd "$APP" "$IMAGE" 2>/dev/null)
else
SCRIPT_CMD=$(get_phase_script "$IMAGE" "$PHASE_SCRIPT_KEY" 2>/dev/null)
fi
if [[ -z "$SCRIPT_CMD" ]]; then
return
fi
dokku_log_info1 "Running '$SCRIPT_CMD' in app container"
local COMMAND
COMMAND="set -eo pipefail; [[ \$DOKKU_TRACE ]] && set -x ; "
COMMAND+=" if [[ -d '/app' ]]; then "
COMMAND+=" export HOME=/app ; "
COMMAND+=" cd \$HOME ; "
COMMAND+=" fi ; "
COMMAND+=" if [[ -d '/app/.profile.d' ]]; then "
COMMAND+=" for file in /app/.profile.d/*; do source \$file; done ; "
COMMAND+=" fi ; "
COMMAND+=" if [[ -d '/cache' ]]; then "
COMMAND+=" echo restoring installation cache... ; "
COMMAND+=" rm -rf /tmp/cache ; "
COMMAND+=" ln -sf /cache /tmp/cache ; "
COMMAND+=" fi ; "
if [[ "$SCRIPT_CMD" == /* ]]; then
local SCRIPT_BIN="$(echo "$SCRIPT_CMD" | cut -d' ' -f1)"
COMMAND+=" if [[ ! -x \"$SCRIPT_BIN\" ]]; then "
COMMAND+=" echo specified binary is not executable ; "
COMMAND+=" exit 1 ; "
COMMAND+=" fi "
fi
COMMAND+=" $SCRIPT_CMD || exit 1;"
COMMAND+=" if [[ -d '/cache' ]]; then "
COMMAND+=" echo removing installation cache... ; "
COMMAND+=" rm -f /tmp/cache ; "
COMMAND+=" fi "
local CACHE_DIR="$DOKKU_ROOT/$APP/cache"
local CACHE_HOST_DIR="$DOKKU_HOST_ROOT/$APP/cache"
[[ -d $CACHE_DIR ]] || mkdir -p "$CACHE_DIR"
local DOCKER_ARGS=$(: | plugn trigger docker-args-deploy "$APP" "$IMAGE_TAG")
# strip --restart args from DOCKER_ARGS
local DOCKER_ARGS=$(sed -e "s/--restart=[[:graph:]]\+[[:blank:]]\?//g" <<<"$DOCKER_ARGS")
# eval args as array to respect escapes
declare -a ARG_ARRAY
eval "ARG_ARRAY=($DOCKER_ARGS)"
local DOKKU_APP_SHELL="/bin/bash"
DOKKU_APP_SHELL="$(config_get --global DOKKU_APP_SHELL || echo "$DOKKU_APP_SHELL")"
DOKKU_APP_SHELL="$(config_get "$APP" DOKKU_APP_SHELL || echo "$DOKKU_APP_SHELL")"
[[ -z "$DOKKU_APP_SHELL" ]] && DOKKU_APP_SHELL="/bin/bash"
id=$(docker run "$DOKKU_GLOBAL_RUN_ARGS" -e DOKKU_TRACE="$DOKKU_TRACE" --label=dokku_phase_script="${PHASE_SCRIPT_KEY}" -d -v "$CACHE_HOST_DIR:/cache" "${ARG_ARRAY[@]}" "$IMAGE" "$DOKKU_APP_SHELL" -c "$COMMAND")
if test "$(docker wait "$id")" -ne 0; then
dokku_container_log_verbose_quiet "$id"
dokku_log_fail "execution of '$SCRIPT_CMD' failed!"
fi
dokku_container_log_verbose_quiet "$id"
if [[ "$PHASE_SCRIPT_KEY" != "predeploy" ]]; then
return
fi
if ! is_image_herokuish_based "$IMAGE"; then
local DOKKU_DOCKERFILE_ENTRYPOINT=$(config_get "$APP" DOKKU_DOCKERFILE_ENTRYPOINT)
local DOKKU_DOCKERFILE_CMD=$(config_get "$APP" DOKKU_DOCKERFILE_CMD)
[[ -z "$DOKKU_DOCKERFILE_ENTRYPOINT" ]] && DOKKU_DOCKERFILE_ENTRYPOINT="$(get_entrypoint_from_image "$IMAGE")"
[[ -z "$DOKKU_DOCKERFILE_CMD" ]] && DOKKU_DOCKERFILE_CMD="$(get_cmd_from_image "$IMAGE")"
[[ -n "$DOKKU_DOCKERFILE_ENTRYPOINT" ]] && local DOCKER_COMMIT_ENTRYPOINT_CHANGE_ARG="--change='$DOKKU_DOCKERFILE_ENTRYPOINT'"
[[ -n "$DOKKU_DOCKERFILE_CMD" ]] && local DOCKER_COMMIT_CMD_CHANGE_ARG="--change='$DOKKU_DOCKERFILE_CMD'"
local DOCKER_COMMIT_ARGS="$DOCKER_COMMIT_ENTRYPOINT_CHANGE_ARG $DOCKER_COMMIT_CMD_CHANGE_ARG"
fi
# shellcheck disable=SC2086
eval docker commit $DOCKER_COMMIT_ARGS "$id" "$IMAGE" >/dev/null
}

View File

@@ -0,0 +1,4 @@
[plugin]
description = "dokku core app-json plugin"
version = "0.13.4"
[plugin.config]

17
plugins/app-json/post-deploy Executable file
View File

@@ -0,0 +1,17 @@
#!/usr/bin/env bash
set -eo pipefail
[[ $DOKKU_TRACE ]] && set -x
source "$PLUGIN_CORE_AVAILABLE_PATH/common/functions"
source "$PLUGIN_CORE_AVAILABLE_PATH/app-json/internal-functions"
app_json_post_deploy() {
declare desc="app-json scripts execution"
local trigger="app_json_post_deploy"
local APP="$1"
local IMAGE_TAG="$4"
local PHASE_SCRIPT_KEY="postdeploy"
execute_script "$APP" "$IMAGE_TAG" "$PHASE_SCRIPT_KEY"
}
app_json_post_deploy "$@"

18
plugins/app-json/pre-deploy Executable file
View File

@@ -0,0 +1,18 @@
#!/usr/bin/env bash
set -eo pipefail
[[ $DOKKU_TRACE ]] && set -x
source "$PLUGIN_CORE_AVAILABLE_PATH/common/functions"
source "$PLUGIN_CORE_AVAILABLE_PATH/app-json/internal-functions"
app_json_pre_deploy() {
declare desc="app-json scripts execution"
local trigger="app_json_pre_deploy"
local APP="$1"
local IMAGE_TAG="$2"
local PHASE_SCRIPT_KEY="predeploy"
execute_script "$APP" "$IMAGE_TAG" "$PHASE_SCRIPT_KEY"
execute_script "$APP" "$IMAGE_TAG" "release"
}
app_json_pre_deploy "$@"

View File

@@ -7,6 +7,7 @@ cron: node worker.js
web: node web.js # testing inline comment
worker: node worker.js
custom: echo -n
release: touch /app/release.test
# Old version with separate processes (use this if you have issues with the threaded version)

View File

@@ -180,24 +180,3 @@ EOF
echo "status: $status"
assert_output "3001/udp 3000/tcp 3003"
}
@test "(core) app.json scripts" {
deploy_app
run /bin/bash -c "dokku run $TEST_APP ls /app/prebuild.test"
echo "output: $output"
echo "status: $status"
assert_failure
run /bin/bash -c "dokku run $TEST_APP ls /app/predeploy.test"
echo "output: $output"
echo "status: $status"
assert_success
CID=$(docker ps -a -q -f "ancestor=dokku/${TEST_APP}" -f "label=dokku_phase_script=postdeploy")
IMAGE_ID=$(docker commit $CID dokku-test/${TEST_APP})
run /bin/bash -c "docker run -ti $IMAGE_ID ls /app/postdeploy.test"
echo "output: $output"
echo "status: $status"
assert_success
}

View File

@@ -0,0 +1,38 @@
#!/usr/bin/env bats
load test_helper
setup() {
global_setup
create_app
DOCKERFILE="$BATS_TMPDIR/Dockerfile"
}
teardown() {
rm -rf /home/dokku/$TEST_APP/tls
destroy_app
dokku config:unset --global DOKKU_RM_CONTAINER
rm -f "$DOCKERFILE"
global_teardown
}
@test "(app-json) app.json scripts" {
deploy_app
run /bin/bash -c "dokku run $TEST_APP ls /app/prebuild.test"
echo "output: $output"
echo "status: $status"
assert_failure
run /bin/bash -c "dokku run $TEST_APP ls /app/predeploy.test"
echo "output: $output"
echo "status: $status"
assert_success
CID=$(docker ps -a -q -f "ancestor=dokku/${TEST_APP}" -f "label=dokku_phase_script=postdeploy")
IMAGE_ID=$(docker commit $CID dokku-test/${TEST_APP})
run /bin/bash -c "docker run -ti $IMAGE_ID ls /app/postdeploy.test"
echo "output: $output"
echo "status: $status"
assert_success
}