mirror of
https://github.com/bahdotsh/wrkflw.git
synced 2026-05-18 05:05:35 +02:00
feat(executor): easy GHA emulation fixes for better compatibility (#82)
* feat(executor): add easy GHA emulation fixes for better compatibility
- Expand github.* context with 13 missing env vars (CI, GITHUB_ACTIONS,
GITHUB_REF_NAME, GITHUB_REF_TYPE, GITHUB_REPOSITORY_OWNER, etc.) and
improve GITHUB_ACTOR to use git config / $USER instead of hardcoded value
- Enforce timeout-minutes at both job level (default 360m per GHA spec)
and step level via tokio::time::timeout
- Implement defaults.run.shell and defaults.run.working-directory with
proper fallback chain: step > job defaults > workflow defaults > bash
- Implement hashFiles() expression function with glob matching, sorted
file hashing (SHA-256), and integration into the substitution pipeline
* fix(executor): harden hashFiles, working-directory, and shell -e
Three issues from code review, all in the "we got the GHA emulation
*almost* right" category:
1. hashFiles() was returning an empty string when no files matched.
GHA returns the SHA-256 of empty input (e3b0c44...), not nothing.
An empty string as a cache key component is the kind of thing
that silently ruins your day. Also, unreadable files were being
skipped without a peep — now we at least warn about it.
2. The working-directory default resolution was doing a naive
Path::join with user-controlled input. If someone writes
`working-directory: ../../../etc` or an absolute path, join
happily replaces the base. Inside a container this is *somewhat*
contained, but in emulation mode it's a real path traversal.
Normalize the path and reject anything that escapes the
workspace.
3. The bash -e flag change (correct per GHA spec) was undocumented.
Scripts that relied on intermediate commands failing without
aborting the step will now break. Document it in
BREAKING_CHANGES.md so users aren't left guessing.
* fix(executor): complete the GHA shell invocation and harden hashFiles
The previous commit added `-e` to bash but stopped there, even
though the BREAKING_CHANGES.md *literally documented* the full GHA
invocation as `bash --noprofile --norc -e -o pipefail {0}`. So we
were advertising behavior we weren't actually implementing. This is
not great.
Without `-o pipefail`, piped commands like `false | echo ok` would
silently succeed, which is exactly the kind of divergence that makes
you distrust an emulator. And without `--noprofile --norc`, user
profile scripts can interfere with reproducibility.
While at it, fix hashFiles error handling — it was silently
swallowing read errors and producing a partial hash, which is worse
than failing because you get a *wrong* cache key with no indication
anything went sideways. preprocess_hash_files and
preprocess_expressions now return Result and the engine surfaces
failures as step errors.
Also add the tests that should have been there from the start:
shell invocation flags, working-directory path traversal rejection,
and defaults cascade (step > job > workflow).
* fix(executor): harden hashFiles, timeout, and shell edge cases
The previous round of GHA emulation fixes left a few holes that
would bite you in production:
hashFiles() would happily glob '../../etc/passwd' and hash whatever
it found outside the workspace. It also loaded entire file contents
into memory before hashing, which is *not great* when someone points
it at a large binary artifact. The glob patterns now reject '..'
traversal, and file contents are streamed into the SHA-256 hasher
via io::copy instead.
timeout-minutes accepted any f64 from YAML, including negative
values, NaN, and infinity — all of which make Duration::from_secs_f64
panic. Non-finite and non-positive values now fall back to the GHA
default of 360 minutes.
Unknown shell values were silently accepted with a '-c' fallback.
Now they emit a warning so you at least *know* something is off.
While at it, replaced the hash_files_read_error_returns_err test
that was testing two Ok paths (despite its name) with proper
path-traversal rejection tests.
* fix(executor): fix shadowed timeout_mins and extract sanitization helper
It turns out the job timeout error path was re-reading the *raw*
timeout_minutes value instead of using the already-sanitized one.
If someone set timeout-minutes to NaN or a negative number, the
sanitization would correctly fall back to 360, but the error
message would happily print "Job exceeded timeout of NaN minutes."
Not great.
Extract sanitize_timeout_minutes() so both the job and step
timeout paths use the same logic instead of duplicating the
is_finite/positive/clamp dance. While at it, add proper tests
for NaN, Infinity, negative, zero, and the max clamp — plus a
test that actually exercises the job-level timeout expiry branch,
which previously had zero coverage.
This commit is contained in:
@@ -33,7 +33,9 @@ num_cpus.workspace = true
|
||||
once_cell.workspace = true
|
||||
regex.workspace = true
|
||||
reqwest.workspace = true
|
||||
glob.workspace = true
|
||||
serde.workspace = true
|
||||
sha2.workspace = true
|
||||
serde_json.workspace = true
|
||||
serde_yaml.workspace = true
|
||||
shlex.workspace = true
|
||||
|
||||
@@ -236,6 +236,8 @@ mod tests {
|
||||
uses: None,
|
||||
with: None,
|
||||
secrets: None,
|
||||
timeout_minutes: None,
|
||||
defaults: None,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1695,6 +1695,9 @@ async fn execute_job(ctx: JobExecutionContext<'_>) -> Result<JobResult, Executio
|
||||
job_env.insert(key.clone(), value.clone());
|
||||
}
|
||||
|
||||
// Add job-specific context
|
||||
environment::add_job_context(&mut job_env, ctx.job_name);
|
||||
|
||||
// Execute job steps
|
||||
let mut step_results = Vec::new();
|
||||
let mut job_logs = String::new();
|
||||
@@ -1716,55 +1719,82 @@ async fn execute_job(ctx: JobExecutionContext<'_>) -> Result<JobResult, Executio
|
||||
// Determine runner image: prefer job container, then detect setup actions, fall back to runs-on
|
||||
let runner_image_value = resolve_runner_image(job, ctx.runtime).await?;
|
||||
|
||||
for (idx, step) in job.steps.iter().enumerate() {
|
||||
let outcome = run_step_with_guards(
|
||||
step,
|
||||
idx,
|
||||
&job_env,
|
||||
ctx.workflow,
|
||||
StepExecutionContext {
|
||||
// GHA default job timeout is 360 minutes; sanitize to avoid panic on negative/NaN
|
||||
let timeout_mins = sanitize_timeout_minutes(job.timeout_minutes, 360.0);
|
||||
let job_timeout = std::time::Duration::from_secs_f64(timeout_mins * 60.0);
|
||||
|
||||
let step_loop = async {
|
||||
for (idx, step) in job.steps.iter().enumerate() {
|
||||
let outcome = run_step_with_guards(
|
||||
step,
|
||||
step_idx: idx,
|
||||
job_env: &job_env,
|
||||
working_dir: job_dir.path(),
|
||||
runtime: ctx.runtime,
|
||||
workflow: ctx.workflow,
|
||||
runner_image: &runner_image_value,
|
||||
verbose: ctx.verbose,
|
||||
matrix_combination: &None,
|
||||
secret_manager: ctx.secret_manager,
|
||||
secret_masker: ctx.secret_masker,
|
||||
container_config: job.container.as_ref(),
|
||||
},
|
||||
)
|
||||
.await?;
|
||||
idx,
|
||||
&job_env,
|
||||
ctx.workflow,
|
||||
StepExecutionContext {
|
||||
step,
|
||||
step_idx: idx,
|
||||
job_env: &job_env,
|
||||
working_dir: job_dir.path(),
|
||||
runtime: ctx.runtime,
|
||||
workflow: ctx.workflow,
|
||||
runner_image: &runner_image_value,
|
||||
verbose: ctx.verbose,
|
||||
matrix_combination: &None,
|
||||
secret_manager: ctx.secret_manager,
|
||||
secret_masker: ctx.secret_masker,
|
||||
container_config: job.container.as_ref(),
|
||||
workflow_defaults: ctx.workflow.defaults.as_ref(),
|
||||
job_defaults: job.defaults.as_ref(),
|
||||
},
|
||||
)
|
||||
.await?;
|
||||
|
||||
match outcome {
|
||||
StepOutcome::Skipped(result) => {
|
||||
step_results.push(result);
|
||||
}
|
||||
StepOutcome::Completed { result, abort_job } => {
|
||||
// Add step output to logs only in verbose mode or if there's an error
|
||||
if ctx.verbose || result.status == StepStatus::Failure {
|
||||
job_logs.push_str(&format!(
|
||||
"\n=== Output from step '{}' ===\n{}\n=== End output ===\n\n",
|
||||
result.name, result.output
|
||||
));
|
||||
} else {
|
||||
job_logs.push_str(&format!(
|
||||
"Step '{}' completed with status: {:?}\n",
|
||||
result.name, result.status
|
||||
));
|
||||
match outcome {
|
||||
StepOutcome::Skipped(result) => {
|
||||
step_results.push(result);
|
||||
}
|
||||
StepOutcome::Completed { result, abort_job } => {
|
||||
// Add step output to logs only in verbose mode or if there's an error
|
||||
if ctx.verbose || result.status == StepStatus::Failure {
|
||||
job_logs.push_str(&format!(
|
||||
"\n=== Output from step '{}' ===\n{}\n=== End output ===\n\n",
|
||||
result.name, result.output
|
||||
));
|
||||
} else {
|
||||
job_logs.push_str(&format!(
|
||||
"Step '{}' completed with status: {:?}\n",
|
||||
result.name, result.status
|
||||
));
|
||||
}
|
||||
|
||||
step_results.push(result);
|
||||
step_results.push(result);
|
||||
|
||||
if abort_job {
|
||||
job_success = false;
|
||||
break;
|
||||
if abort_job {
|
||||
job_success = false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Ok::<(), ExecutionError>(())
|
||||
};
|
||||
|
||||
match tokio::time::timeout(job_timeout, step_loop).await {
|
||||
Ok(Ok(())) => {}
|
||||
Ok(Err(e)) => return Err(e),
|
||||
Err(_) => {
|
||||
wrkflw_logging::error(&format!(
|
||||
"Job '{}' exceeded timeout of {} minutes",
|
||||
ctx.job_name, timeout_mins
|
||||
));
|
||||
return Ok(JobResult {
|
||||
name: ctx.job_name.to_string(),
|
||||
status: JobStatus::Failure,
|
||||
steps: step_results,
|
||||
logs: format!("{}\nJob timed out after {} minutes", job_logs, timeout_mins),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
Ok(JobResult {
|
||||
@@ -1935,6 +1965,8 @@ async fn execute_matrix_job(
|
||||
secret_manager: None,
|
||||
secret_masker: None,
|
||||
container_config: job_template.container.as_ref(),
|
||||
workflow_defaults: workflow.defaults.as_ref(),
|
||||
job_defaults: job_template.defaults.as_ref(),
|
||||
},
|
||||
)
|
||||
.await?;
|
||||
@@ -2019,7 +2051,29 @@ async fn run_step_with_guards(
|
||||
}
|
||||
}
|
||||
|
||||
match execute_step(step_exec_ctx).await {
|
||||
// Wrap step execution with optional timeout; sanitize to avoid panic on negative/NaN
|
||||
let step_result = if let Some(minutes) = step.timeout_minutes {
|
||||
let safe_mins = sanitize_timeout_minutes(Some(minutes), 360.0);
|
||||
let dur = std::time::Duration::from_secs_f64(safe_mins * 60.0);
|
||||
match tokio::time::timeout(dur, execute_step(step_exec_ctx)).await {
|
||||
Ok(result) => result,
|
||||
Err(_) => {
|
||||
wrkflw_logging::error(&format!(
|
||||
" Step '{}' exceeded timeout of {} minutes",
|
||||
step_name, minutes
|
||||
));
|
||||
Ok(StepResult {
|
||||
name: step_name.clone(),
|
||||
status: StepStatus::Failure,
|
||||
output: format!("Step timed out after {} minutes", minutes),
|
||||
})
|
||||
}
|
||||
}
|
||||
} else {
|
||||
execute_step(step_exec_ctx).await
|
||||
};
|
||||
|
||||
match step_result {
|
||||
Ok(result) => {
|
||||
let abort_job = if result.status == StepStatus::Failure {
|
||||
if step.continue_on_error == Some(true) {
|
||||
@@ -2064,6 +2118,18 @@ async fn run_step_with_guards(
|
||||
}
|
||||
}
|
||||
|
||||
/// Sanitize a timeout-minutes value, returning a safe positive finite number.
|
||||
/// Falls back to `default` for `None`, `NaN`, `Infinity`, zero, or negative values.
|
||||
/// Clamps to a maximum of 8640 minutes (6 days).
|
||||
fn sanitize_timeout_minutes(raw: Option<f64>, default: f64) -> f64 {
|
||||
let mins = raw.unwrap_or(default);
|
||||
if mins.is_finite() && mins > 0.0 {
|
||||
mins.min(360.0 * 24.0)
|
||||
} else {
|
||||
default
|
||||
}
|
||||
}
|
||||
|
||||
// Before the execute_step function, add this struct
|
||||
struct StepExecutionContext<'a> {
|
||||
step: &'a workflow::Step,
|
||||
@@ -2080,6 +2146,8 @@ struct StepExecutionContext<'a> {
|
||||
#[allow(dead_code)] // Planned for future implementation
|
||||
secret_masker: Option<&'a SecretMasker>,
|
||||
container_config: Option<&'a JobContainer>,
|
||||
workflow_defaults: Option<&'a workflow::Defaults>,
|
||||
job_defaults: Option<&'a workflow::Defaults>,
|
||||
}
|
||||
|
||||
async fn execute_step(ctx: StepExecutionContext<'_>) -> Result<StepResult, ExecutionError> {
|
||||
@@ -2651,15 +2719,105 @@ async fn execute_step(ctx: StepExecutionContext<'_>) -> Result<StepResult, Execu
|
||||
run.clone()
|
||||
};
|
||||
|
||||
// Resolve expression substitutions (hashFiles, matrix vars)
|
||||
let resolved_run = match crate::substitution::preprocess_expressions(
|
||||
&resolved_run,
|
||||
ctx.working_dir,
|
||||
ctx.matrix_combination,
|
||||
) {
|
||||
Ok(r) => r,
|
||||
Err(e) => {
|
||||
return Ok(StepResult {
|
||||
name: step_name,
|
||||
status: StepStatus::Failure,
|
||||
output: format!("Expression substitution failed: {}", e),
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
// Check if this is a cargo command
|
||||
let is_cargo_cmd = resolved_run.trim().starts_with("cargo");
|
||||
|
||||
// For complex shell commands, use bash to execute them properly
|
||||
// This handles quotes, pipes, redirections, and command substitutions correctly
|
||||
let cmd_parts = vec!["bash", "-c", &resolved_run];
|
||||
// Resolve effective shell: step > job defaults > workflow defaults > "bash"
|
||||
let effective_shell = ctx
|
||||
.step
|
||||
.shell
|
||||
.as_deref()
|
||||
.or_else(|| {
|
||||
ctx.job_defaults
|
||||
.and_then(|d| d.run.as_ref()?.shell.as_deref())
|
||||
})
|
||||
.or_else(|| {
|
||||
ctx.workflow_defaults
|
||||
.and_then(|d| d.run.as_ref()?.shell.as_deref())
|
||||
})
|
||||
.unwrap_or("bash");
|
||||
|
||||
let cmd_parts = match effective_shell {
|
||||
"bash" => vec![
|
||||
"bash",
|
||||
"--noprofile",
|
||||
"--norc",
|
||||
"-e",
|
||||
"-o",
|
||||
"pipefail",
|
||||
"-c",
|
||||
&resolved_run,
|
||||
],
|
||||
"sh" => vec!["sh", "-e", "-c", &resolved_run],
|
||||
"python" => vec!["python", "-c", &resolved_run],
|
||||
"pwsh" | "powershell" => vec!["pwsh", "-command", &resolved_run],
|
||||
other => {
|
||||
wrkflw_logging::warning(&format!(
|
||||
" Unrecognized shell '{}', falling back to '{} -c'",
|
||||
other, other
|
||||
));
|
||||
vec![other, "-c", &resolved_run]
|
||||
}
|
||||
};
|
||||
|
||||
// Resolve effective working directory: step > job defaults > workflow defaults
|
||||
let effective_wd = ctx
|
||||
.step
|
||||
.working_directory
|
||||
.as_deref()
|
||||
.or_else(|| {
|
||||
ctx.job_defaults
|
||||
.and_then(|d| d.run.as_ref()?.working_directory.as_deref())
|
||||
})
|
||||
.or_else(|| {
|
||||
ctx.workflow_defaults
|
||||
.and_then(|d| d.run.as_ref()?.working_directory.as_deref())
|
||||
});
|
||||
|
||||
// Define the standard workspace path inside the container
|
||||
let container_workspace = Path::new("/github/workspace");
|
||||
let final_workspace = if let Some(wd) = effective_wd {
|
||||
let joined = container_workspace.join(wd);
|
||||
// Canonicalize logically to catch ".." traversal and absolute path replacement
|
||||
let mut normalized = std::path::PathBuf::new();
|
||||
for component in joined.components() {
|
||||
match component {
|
||||
std::path::Component::ParentDir => {
|
||||
normalized.pop();
|
||||
}
|
||||
c => normalized.push(c.as_os_str()),
|
||||
}
|
||||
}
|
||||
if !normalized.starts_with(container_workspace) {
|
||||
return Ok(StepResult {
|
||||
name: step_name,
|
||||
status: StepStatus::Failure,
|
||||
output: format!(
|
||||
"Invalid working-directory '{}': must be within workspace",
|
||||
wd
|
||||
),
|
||||
});
|
||||
}
|
||||
normalized
|
||||
} else {
|
||||
container_workspace.to_path_buf()
|
||||
};
|
||||
|
||||
let mount_ctx =
|
||||
prepare_step_container_context(&mut step_env, ctx.job_env, ctx.container_config);
|
||||
@@ -2676,7 +2834,7 @@ async fn execute_step(ctx: StepExecutionContext<'_>) -> Result<StepResult, Execu
|
||||
ctx.runner_image,
|
||||
&cmd_parts,
|
||||
&env_vars,
|
||||
container_workspace,
|
||||
&final_workspace,
|
||||
&volumes,
|
||||
None,
|
||||
)
|
||||
@@ -3565,6 +3723,7 @@ async fn execute_composite_action(
|
||||
on: vec![],
|
||||
on_raw: serde_yaml::Value::Null,
|
||||
jobs: HashMap::new(),
|
||||
defaults: None,
|
||||
},
|
||||
runner_image,
|
||||
verbose,
|
||||
@@ -3572,6 +3731,8 @@ async fn execute_composite_action(
|
||||
secret_manager: None, // Composite actions don't have secrets yet
|
||||
secret_masker: None,
|
||||
container_config: None, // Composite actions don't use job containers
|
||||
workflow_defaults: None,
|
||||
job_defaults: None,
|
||||
}))
|
||||
.await?;
|
||||
|
||||
@@ -3883,6 +4044,8 @@ mod tests {
|
||||
uses: None,
|
||||
with: None,
|
||||
secrets: None,
|
||||
timeout_minutes: None,
|
||||
defaults: None,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -4176,6 +4339,7 @@ mod tests {
|
||||
on: Vec::new(),
|
||||
on_raw: serde_yaml::Value::Null,
|
||||
jobs: HashMap::new(),
|
||||
defaults: None,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -4693,6 +4857,7 @@ runs:
|
||||
on: vec![],
|
||||
on_raw: serde_yaml::Value::Null,
|
||||
jobs: Default::default(),
|
||||
defaults: None,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -4743,6 +4908,8 @@ runs:
|
||||
secret_manager: None,
|
||||
secret_masker: None,
|
||||
container_config: None,
|
||||
workflow_defaults: None,
|
||||
job_defaults: None,
|
||||
};
|
||||
|
||||
let result = execute_step(ctx).await.unwrap();
|
||||
@@ -4789,6 +4956,8 @@ runs:
|
||||
secret_manager: None,
|
||||
secret_masker: None,
|
||||
container_config: None,
|
||||
workflow_defaults: None,
|
||||
job_defaults: None,
|
||||
};
|
||||
|
||||
let result = execute_step(ctx).await.unwrap();
|
||||
@@ -4840,6 +5009,8 @@ runs:
|
||||
secret_manager: None,
|
||||
secret_masker: None,
|
||||
container_config: None,
|
||||
workflow_defaults: None,
|
||||
job_defaults: None,
|
||||
};
|
||||
|
||||
let result = execute_step(ctx).await.unwrap();
|
||||
@@ -4884,6 +5055,8 @@ runs:
|
||||
secret_manager: None,
|
||||
secret_masker: None,
|
||||
container_config: None,
|
||||
workflow_defaults: None,
|
||||
job_defaults: None,
|
||||
};
|
||||
|
||||
let result = execute_step(ctx).await.unwrap();
|
||||
@@ -4929,6 +5102,8 @@ runs:
|
||||
secret_manager: None,
|
||||
secret_masker: None,
|
||||
container_config: None,
|
||||
workflow_defaults: None,
|
||||
job_defaults: None,
|
||||
};
|
||||
|
||||
let result = execute_step(ctx).await;
|
||||
@@ -4973,6 +5148,8 @@ runs:
|
||||
secret_manager: None,
|
||||
secret_masker: None,
|
||||
container_config: None,
|
||||
workflow_defaults: None,
|
||||
job_defaults: None,
|
||||
};
|
||||
|
||||
let result = execute_step(ctx).await.unwrap();
|
||||
@@ -5015,6 +5192,8 @@ runs:
|
||||
secret_manager: None,
|
||||
secret_masker: None,
|
||||
container_config: None,
|
||||
workflow_defaults: None,
|
||||
job_defaults: None,
|
||||
};
|
||||
|
||||
let result = execute_step(ctx).await.unwrap();
|
||||
@@ -5508,4 +5687,441 @@ runs:
|
||||
// Both should produce the same sorted prefix (a1-b2)
|
||||
assert_eq!(tag_ab, tag_ba);
|
||||
}
|
||||
|
||||
// --- Shell invocation tests ---
|
||||
|
||||
fn make_run_step(run: &str) -> Step {
|
||||
Step {
|
||||
name: Some("run-step".to_string()),
|
||||
uses: None,
|
||||
run: Some(run.to_string()),
|
||||
with: None,
|
||||
env: HashMap::new(),
|
||||
continue_on_error: None,
|
||||
if_condition: None,
|
||||
id: None,
|
||||
working_directory: None,
|
||||
shell: None,
|
||||
timeout_minutes: None,
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn bash_shell_uses_errexit_and_pipefail() {
|
||||
let runtime = MockContainerRuntime::default();
|
||||
let workflow = minimal_workflow();
|
||||
let job_env = HashMap::new();
|
||||
let working_dir = std::env::current_dir().unwrap();
|
||||
|
||||
let step = make_run_step("echo hello");
|
||||
|
||||
let ctx = StepExecutionContext {
|
||||
step: &step,
|
||||
step_idx: 0,
|
||||
job_env: &job_env,
|
||||
working_dir: &working_dir,
|
||||
runtime: &runtime,
|
||||
workflow: &workflow,
|
||||
runner_image: "ubuntu:latest",
|
||||
verbose: false,
|
||||
matrix_combination: &None,
|
||||
secret_manager: None,
|
||||
secret_masker: None,
|
||||
container_config: None,
|
||||
workflow_defaults: None,
|
||||
job_defaults: None,
|
||||
};
|
||||
|
||||
let result = execute_step(ctx).await.unwrap();
|
||||
assert_eq!(result.status, StepStatus::Success);
|
||||
|
||||
let calls = runtime.run_calls.lock().unwrap();
|
||||
assert_eq!(calls.len(), 1);
|
||||
let cmd = &calls[0].cmd;
|
||||
// Should be: bash --noprofile --norc -e -o pipefail -c <script>
|
||||
assert_eq!(cmd[0], "bash");
|
||||
assert_eq!(cmd[1], "--noprofile");
|
||||
assert_eq!(cmd[2], "--norc");
|
||||
assert_eq!(cmd[3], "-e");
|
||||
assert_eq!(cmd[4], "-o");
|
||||
assert_eq!(cmd[5], "pipefail");
|
||||
assert_eq!(cmd[6], "-c");
|
||||
assert_eq!(cmd[7], "echo hello");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn sh_shell_uses_errexit() {
|
||||
let runtime = MockContainerRuntime::default();
|
||||
let workflow = minimal_workflow();
|
||||
let job_env = HashMap::new();
|
||||
let working_dir = std::env::current_dir().unwrap();
|
||||
|
||||
let mut step = make_run_step("echo hello");
|
||||
step.shell = Some("sh".to_string());
|
||||
|
||||
let ctx = StepExecutionContext {
|
||||
step: &step,
|
||||
step_idx: 0,
|
||||
job_env: &job_env,
|
||||
working_dir: &working_dir,
|
||||
runtime: &runtime,
|
||||
workflow: &workflow,
|
||||
runner_image: "ubuntu:latest",
|
||||
verbose: false,
|
||||
matrix_combination: &None,
|
||||
secret_manager: None,
|
||||
secret_masker: None,
|
||||
container_config: None,
|
||||
workflow_defaults: None,
|
||||
job_defaults: None,
|
||||
};
|
||||
|
||||
let result = execute_step(ctx).await.unwrap();
|
||||
assert_eq!(result.status, StepStatus::Success);
|
||||
|
||||
let calls = runtime.run_calls.lock().unwrap();
|
||||
let cmd = &calls[0].cmd;
|
||||
assert_eq!(cmd[0], "sh");
|
||||
assert_eq!(cmd[1], "-e");
|
||||
assert_eq!(cmd[2], "-c");
|
||||
assert_eq!(cmd[3], "echo hello");
|
||||
}
|
||||
|
||||
// --- Working-directory path traversal tests ---
|
||||
|
||||
#[tokio::test]
|
||||
async fn working_directory_rejects_parent_traversal() {
|
||||
let runtime = MockContainerRuntime::default();
|
||||
let workflow = minimal_workflow();
|
||||
let job_env = HashMap::new();
|
||||
let working_dir = std::env::current_dir().unwrap();
|
||||
|
||||
let mut step = make_run_step("echo pwned");
|
||||
step.working_directory = Some("../../etc".to_string());
|
||||
|
||||
let ctx = StepExecutionContext {
|
||||
step: &step,
|
||||
step_idx: 0,
|
||||
job_env: &job_env,
|
||||
working_dir: &working_dir,
|
||||
runtime: &runtime,
|
||||
workflow: &workflow,
|
||||
runner_image: "ubuntu:latest",
|
||||
verbose: false,
|
||||
matrix_combination: &None,
|
||||
secret_manager: None,
|
||||
secret_masker: None,
|
||||
container_config: None,
|
||||
workflow_defaults: None,
|
||||
job_defaults: None,
|
||||
};
|
||||
|
||||
let result = execute_step(ctx).await.unwrap();
|
||||
assert_eq!(result.status, StepStatus::Failure);
|
||||
assert!(result.output.contains("Invalid working-directory"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn working_directory_allows_subdirectory() {
|
||||
let runtime = MockContainerRuntime::default();
|
||||
let workflow = minimal_workflow();
|
||||
let job_env = HashMap::new();
|
||||
let working_dir = std::env::current_dir().unwrap();
|
||||
|
||||
let mut step = make_run_step("echo ok");
|
||||
step.working_directory = Some("src/app".to_string());
|
||||
|
||||
let ctx = StepExecutionContext {
|
||||
step: &step,
|
||||
step_idx: 0,
|
||||
job_env: &job_env,
|
||||
working_dir: &working_dir,
|
||||
runtime: &runtime,
|
||||
workflow: &workflow,
|
||||
runner_image: "ubuntu:latest",
|
||||
verbose: false,
|
||||
matrix_combination: &None,
|
||||
secret_manager: None,
|
||||
secret_masker: None,
|
||||
container_config: None,
|
||||
workflow_defaults: None,
|
||||
job_defaults: None,
|
||||
};
|
||||
|
||||
let result = execute_step(ctx).await.unwrap();
|
||||
assert_eq!(result.status, StepStatus::Success);
|
||||
// No container calls should have failed
|
||||
let calls = runtime.run_calls.lock().unwrap();
|
||||
assert_eq!(calls.len(), 1);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn working_directory_rejects_absolute_path() {
|
||||
let runtime = MockContainerRuntime::default();
|
||||
let workflow = minimal_workflow();
|
||||
let job_env = HashMap::new();
|
||||
let working_dir = std::env::current_dir().unwrap();
|
||||
|
||||
let mut step = make_run_step("echo pwned");
|
||||
step.working_directory = Some("/tmp/evil".to_string());
|
||||
|
||||
let ctx = StepExecutionContext {
|
||||
step: &step,
|
||||
step_idx: 0,
|
||||
job_env: &job_env,
|
||||
working_dir: &working_dir,
|
||||
runtime: &runtime,
|
||||
workflow: &workflow,
|
||||
runner_image: "ubuntu:latest",
|
||||
verbose: false,
|
||||
matrix_combination: &None,
|
||||
secret_manager: None,
|
||||
secret_masker: None,
|
||||
container_config: None,
|
||||
workflow_defaults: None,
|
||||
job_defaults: None,
|
||||
};
|
||||
|
||||
let result = execute_step(ctx).await.unwrap();
|
||||
assert_eq!(result.status, StepStatus::Failure);
|
||||
assert!(result.output.contains("Invalid working-directory"));
|
||||
}
|
||||
|
||||
// --- Defaults cascade tests ---
|
||||
|
||||
#[tokio::test]
|
||||
async fn defaults_cascade_job_overrides_workflow() {
|
||||
let runtime = MockContainerRuntime::default();
|
||||
let workflow_defaults = workflow::Defaults {
|
||||
run: Some(workflow::DefaultsRun {
|
||||
shell: Some("sh".to_string()),
|
||||
working_directory: None,
|
||||
}),
|
||||
};
|
||||
let job_defaults = workflow::Defaults {
|
||||
run: Some(workflow::DefaultsRun {
|
||||
shell: Some("python".to_string()),
|
||||
working_directory: None,
|
||||
}),
|
||||
};
|
||||
let workflow = minimal_workflow();
|
||||
let job_env = HashMap::new();
|
||||
let working_dir = std::env::current_dir().unwrap();
|
||||
|
||||
let step = make_run_step("print('hello')");
|
||||
|
||||
let ctx = StepExecutionContext {
|
||||
step: &step,
|
||||
step_idx: 0,
|
||||
job_env: &job_env,
|
||||
working_dir: &working_dir,
|
||||
runtime: &runtime,
|
||||
workflow: &workflow,
|
||||
runner_image: "ubuntu:latest",
|
||||
verbose: false,
|
||||
matrix_combination: &None,
|
||||
secret_manager: None,
|
||||
secret_masker: None,
|
||||
container_config: None,
|
||||
workflow_defaults: Some(&workflow_defaults),
|
||||
job_defaults: Some(&job_defaults),
|
||||
};
|
||||
|
||||
let result = execute_step(ctx).await.unwrap();
|
||||
assert_eq!(result.status, StepStatus::Success);
|
||||
|
||||
let calls = runtime.run_calls.lock().unwrap();
|
||||
let cmd = &calls[0].cmd;
|
||||
// Job defaults (python) should override workflow defaults (sh)
|
||||
assert_eq!(cmd[0], "python");
|
||||
assert_eq!(cmd[1], "-c");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn defaults_cascade_step_overrides_job() {
|
||||
let runtime = MockContainerRuntime::default();
|
||||
let job_defaults = workflow::Defaults {
|
||||
run: Some(workflow::DefaultsRun {
|
||||
shell: Some("python".to_string()),
|
||||
working_directory: None,
|
||||
}),
|
||||
};
|
||||
let workflow = minimal_workflow();
|
||||
let job_env = HashMap::new();
|
||||
let working_dir = std::env::current_dir().unwrap();
|
||||
|
||||
let mut step = make_run_step("echo hello");
|
||||
step.shell = Some("sh".to_string());
|
||||
|
||||
let ctx = StepExecutionContext {
|
||||
step: &step,
|
||||
step_idx: 0,
|
||||
job_env: &job_env,
|
||||
working_dir: &working_dir,
|
||||
runtime: &runtime,
|
||||
workflow: &workflow,
|
||||
runner_image: "ubuntu:latest",
|
||||
verbose: false,
|
||||
matrix_combination: &None,
|
||||
secret_manager: None,
|
||||
secret_masker: None,
|
||||
container_config: None,
|
||||
workflow_defaults: None,
|
||||
job_defaults: Some(&job_defaults),
|
||||
};
|
||||
|
||||
let result = execute_step(ctx).await.unwrap();
|
||||
assert_eq!(result.status, StepStatus::Success);
|
||||
|
||||
let calls = runtime.run_calls.lock().unwrap();
|
||||
let cmd = &calls[0].cmd;
|
||||
// Step shell (sh) should override job defaults (python)
|
||||
assert_eq!(cmd[0], "sh");
|
||||
assert_eq!(cmd[1], "-e");
|
||||
assert_eq!(cmd[2], "-c");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn defaults_cascade_workflow_used_when_no_job_or_step() {
|
||||
let runtime = MockContainerRuntime::default();
|
||||
let workflow_defaults = workflow::Defaults {
|
||||
run: Some(workflow::DefaultsRun {
|
||||
shell: Some("sh".to_string()),
|
||||
working_directory: None,
|
||||
}),
|
||||
};
|
||||
let workflow = minimal_workflow();
|
||||
let job_env = HashMap::new();
|
||||
let working_dir = std::env::current_dir().unwrap();
|
||||
|
||||
let step = make_run_step("echo hello");
|
||||
|
||||
let ctx = StepExecutionContext {
|
||||
step: &step,
|
||||
step_idx: 0,
|
||||
job_env: &job_env,
|
||||
working_dir: &working_dir,
|
||||
runtime: &runtime,
|
||||
workflow: &workflow,
|
||||
runner_image: "ubuntu:latest",
|
||||
verbose: false,
|
||||
matrix_combination: &None,
|
||||
secret_manager: None,
|
||||
secret_masker: None,
|
||||
container_config: None,
|
||||
workflow_defaults: Some(&workflow_defaults),
|
||||
job_defaults: None,
|
||||
};
|
||||
|
||||
let result = execute_step(ctx).await.unwrap();
|
||||
assert_eq!(result.status, StepStatus::Success);
|
||||
|
||||
let calls = runtime.run_calls.lock().unwrap();
|
||||
let cmd = &calls[0].cmd;
|
||||
// Workflow defaults (sh) should be used
|
||||
assert_eq!(cmd[0], "sh");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn defaults_cascade_working_directory_from_job() {
|
||||
let runtime = MockContainerRuntime::default();
|
||||
let job_defaults = workflow::Defaults {
|
||||
run: Some(workflow::DefaultsRun {
|
||||
shell: None,
|
||||
working_directory: Some("src".to_string()),
|
||||
}),
|
||||
};
|
||||
let workflow = minimal_workflow();
|
||||
let job_env = HashMap::new();
|
||||
let working_dir = std::env::current_dir().unwrap();
|
||||
|
||||
let step = make_run_step("echo ok");
|
||||
|
||||
let ctx = StepExecutionContext {
|
||||
step: &step,
|
||||
step_idx: 0,
|
||||
job_env: &job_env,
|
||||
working_dir: &working_dir,
|
||||
runtime: &runtime,
|
||||
workflow: &workflow,
|
||||
runner_image: "ubuntu:latest",
|
||||
verbose: false,
|
||||
matrix_combination: &None,
|
||||
secret_manager: None,
|
||||
secret_masker: None,
|
||||
container_config: None,
|
||||
workflow_defaults: None,
|
||||
job_defaults: Some(&job_defaults),
|
||||
};
|
||||
|
||||
let result = execute_step(ctx).await.unwrap();
|
||||
assert_eq!(result.status, StepStatus::Success);
|
||||
// Should succeed — "src" is a valid subdirectory path
|
||||
}
|
||||
|
||||
// --- sanitize_timeout_minutes tests ---
|
||||
|
||||
#[test]
|
||||
fn sanitize_timeout_none_returns_default() {
|
||||
assert_eq!(sanitize_timeout_minutes(None, 360.0), 360.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sanitize_timeout_positive_value_returned() {
|
||||
assert_eq!(sanitize_timeout_minutes(Some(30.0), 360.0), 30.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sanitize_timeout_nan_returns_default() {
|
||||
assert_eq!(sanitize_timeout_minutes(Some(f64::NAN), 360.0), 360.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sanitize_timeout_infinity_returns_default() {
|
||||
assert_eq!(sanitize_timeout_minutes(Some(f64::INFINITY), 360.0), 360.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sanitize_timeout_neg_infinity_returns_default() {
|
||||
assert_eq!(
|
||||
sanitize_timeout_minutes(Some(f64::NEG_INFINITY), 360.0),
|
||||
360.0
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sanitize_timeout_zero_returns_default() {
|
||||
assert_eq!(sanitize_timeout_minutes(Some(0.0), 360.0), 360.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sanitize_timeout_negative_returns_default() {
|
||||
assert_eq!(sanitize_timeout_minutes(Some(-5.0), 360.0), 360.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sanitize_timeout_clamps_to_max() {
|
||||
// 360 * 24 = 8640
|
||||
assert_eq!(sanitize_timeout_minutes(Some(99999.0), 360.0), 8640.0);
|
||||
}
|
||||
|
||||
// --- Job-level timeout test ---
|
||||
|
||||
#[tokio::test]
|
||||
async fn job_timeout_produces_failure_result() {
|
||||
// Use a very short timeout wrapping a step that sleeps longer
|
||||
let timeout_mins = 0.0001; // ~6ms
|
||||
let dur = std::time::Duration::from_secs_f64(
|
||||
sanitize_timeout_minutes(Some(timeout_mins), 360.0) * 60.0,
|
||||
);
|
||||
|
||||
let step_loop = async {
|
||||
// Simulate a step that takes longer than the timeout
|
||||
tokio::time::sleep(std::time::Duration::from_millis(500)).await;
|
||||
Ok::<(), ExecutionError>(())
|
||||
};
|
||||
|
||||
let result = tokio::time::timeout(dur, step_loop).await;
|
||||
assert!(result.is_err(), "Expected timeout but step completed");
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
use chrono::Utc;
|
||||
use serde_json;
|
||||
use serde_yaml::Value;
|
||||
use std::{collections::HashMap, fs, io, path::Path};
|
||||
use wrkflw_matrix::MatrixCombination;
|
||||
@@ -33,7 +34,6 @@ pub fn create_github_context(
|
||||
// Basic GitHub environment variables
|
||||
env.insert("GITHUB_WORKFLOW".to_string(), workflow.name.clone());
|
||||
env.insert("GITHUB_ACTION".to_string(), "run".to_string());
|
||||
env.insert("GITHUB_ACTOR".to_string(), "wrkflw".to_string());
|
||||
env.insert("GITHUB_REPOSITORY".to_string(), get_repo_name());
|
||||
env.insert("GITHUB_EVENT_NAME".to_string(), get_event_name(workflow));
|
||||
env.insert("GITHUB_WORKSPACE".to_string(), get_workspace_path());
|
||||
@@ -78,6 +78,49 @@ pub fn create_github_context(
|
||||
let now = Utc::now();
|
||||
env.insert("GITHUB_RUN_ID".to_string(), format!("{}", now.timestamp()));
|
||||
env.insert("GITHUB_RUN_NUMBER".to_string(), "1".to_string());
|
||||
env.insert("GITHUB_RUN_ATTEMPT".to_string(), "1".to_string());
|
||||
|
||||
// CI detection variables
|
||||
env.insert("GITHUB_ACTIONS".to_string(), "true".to_string());
|
||||
env.insert("CI".to_string(), "true".to_string());
|
||||
|
||||
// GitHub URLs
|
||||
env.insert(
|
||||
"GITHUB_SERVER_URL".to_string(),
|
||||
"https://github.com".to_string(),
|
||||
);
|
||||
env.insert(
|
||||
"GITHUB_API_URL".to_string(),
|
||||
"https://api.github.com".to_string(),
|
||||
);
|
||||
env.insert(
|
||||
"GITHUB_GRAPHQL_URL".to_string(),
|
||||
"https://api.github.com/graphql".to_string(),
|
||||
);
|
||||
|
||||
// Ref-derived variables
|
||||
let full_ref = env.get("GITHUB_REF").cloned().unwrap_or_default();
|
||||
env.insert("GITHUB_REF_NAME".to_string(), get_ref_name(&full_ref));
|
||||
env.insert("GITHUB_REF_TYPE".to_string(), get_ref_type(&full_ref));
|
||||
|
||||
// PR-related variables (empty for local runs)
|
||||
env.insert("GITHUB_HEAD_REF".to_string(), String::new());
|
||||
env.insert("GITHUB_BASE_REF".to_string(), String::new());
|
||||
|
||||
// Actor-related variables
|
||||
let actor = get_actor();
|
||||
env.insert("GITHUB_ACTOR".to_string(), actor.clone());
|
||||
env.insert("GITHUB_TRIGGERING_ACTOR".to_string(), actor);
|
||||
|
||||
// Repository owner
|
||||
let repo = env.get("GITHUB_REPOSITORY").cloned().unwrap_or_default();
|
||||
env.insert(
|
||||
"GITHUB_REPOSITORY_OWNER".to_string(),
|
||||
get_repository_owner(&repo),
|
||||
);
|
||||
|
||||
// Miscellaneous
|
||||
env.insert("GITHUB_RETENTION_DAYS".to_string(), "90".to_string());
|
||||
|
||||
// Path-related variables
|
||||
env.insert("RUNNER_TEMP".to_string(), get_temp_dir());
|
||||
@@ -86,6 +129,11 @@ pub fn create_github_context(
|
||||
env
|
||||
}
|
||||
|
||||
/// Add job-specific context variables to the environment
|
||||
pub fn add_job_context(env: &mut HashMap<String, String>, job_name: &str) {
|
||||
env.insert("GITHUB_JOB".to_string(), job_name.to_string());
|
||||
}
|
||||
|
||||
/// Add matrix context variables to the environment
|
||||
pub fn add_matrix_context(
|
||||
env: &mut HashMap<String, String>,
|
||||
@@ -240,3 +288,124 @@ fn get_tool_cache_dir() -> String {
|
||||
.to_string_lossy()
|
||||
.to_string()
|
||||
}
|
||||
|
||||
fn get_ref_name(full_ref: &str) -> String {
|
||||
if let Some(name) = full_ref.strip_prefix("refs/heads/") {
|
||||
name.to_string()
|
||||
} else if let Some(name) = full_ref.strip_prefix("refs/tags/") {
|
||||
name.to_string()
|
||||
} else if let Some(name) = full_ref.strip_prefix("refs/pull/") {
|
||||
name.to_string()
|
||||
} else {
|
||||
full_ref.to_string()
|
||||
}
|
||||
}
|
||||
|
||||
fn get_ref_type(full_ref: &str) -> String {
|
||||
if full_ref.starts_with("refs/tags/") {
|
||||
"tag".to_string()
|
||||
} else {
|
||||
"branch".to_string()
|
||||
}
|
||||
}
|
||||
|
||||
fn get_repository_owner(repo: &str) -> String {
|
||||
repo.split('/').next().unwrap_or("").to_string()
|
||||
}
|
||||
|
||||
fn get_actor() -> String {
|
||||
// Try git config user.name first
|
||||
if let Ok(output) = std::process::Command::new("git")
|
||||
.args(["config", "user.name"])
|
||||
.output()
|
||||
{
|
||||
if output.status.success() {
|
||||
let name = String::from_utf8_lossy(&output.stdout).trim().to_string();
|
||||
if !name.is_empty() {
|
||||
return name;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Fall back to $USER or $USERNAME
|
||||
if let Ok(user) = std::env::var("USER") {
|
||||
if !user.is_empty() {
|
||||
return user;
|
||||
}
|
||||
}
|
||||
if let Ok(user) = std::env::var("USERNAME") {
|
||||
if !user.is_empty() {
|
||||
return user;
|
||||
}
|
||||
}
|
||||
|
||||
"wrkflw".to_string()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn ref_name_strips_heads_prefix() {
|
||||
assert_eq!(get_ref_name("refs/heads/main"), "main");
|
||||
assert_eq!(get_ref_name("refs/heads/feature/foo"), "feature/foo");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ref_name_strips_tags_prefix() {
|
||||
assert_eq!(get_ref_name("refs/tags/v1.0.0"), "v1.0.0");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ref_name_returns_input_for_unknown_prefix() {
|
||||
assert_eq!(get_ref_name("some/other/ref"), "some/other/ref");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ref_type_detects_tag() {
|
||||
assert_eq!(get_ref_type("refs/tags/v1.0.0"), "tag");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ref_type_defaults_to_branch() {
|
||||
assert_eq!(get_ref_type("refs/heads/main"), "branch");
|
||||
assert_eq!(get_ref_type("something-else"), "branch");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn repository_owner_extracts_owner() {
|
||||
assert_eq!(get_repository_owner("octocat/hello-world"), "octocat");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn repository_owner_handles_no_slash() {
|
||||
assert_eq!(get_repository_owner("myrepo"), "myrepo");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn repository_owner_handles_empty() {
|
||||
assert_eq!(get_repository_owner(""), "");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn extract_repo_from_ssh_url() {
|
||||
assert_eq!(
|
||||
extract_repo_from_url("git@github.com:owner/repo.git"),
|
||||
Some("owner/repo".to_string())
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn extract_repo_from_https_url() {
|
||||
assert_eq!(
|
||||
extract_repo_from_url("https://github.com/owner/repo.git"),
|
||||
Some("owner/repo".to_string())
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn extract_repo_from_invalid_url() {
|
||||
assert_eq!(extract_repo_from_url("not-a-url"), None);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,11 +1,15 @@
|
||||
use lazy_static::lazy_static;
|
||||
use regex::Regex;
|
||||
use serde_yaml::Value;
|
||||
use sha2::{Digest, Sha256};
|
||||
use std::collections::HashMap;
|
||||
use std::path::Path;
|
||||
|
||||
lazy_static! {
|
||||
static ref MATRIX_PATTERN: Regex =
|
||||
Regex::new(r"\$\{\{\s*matrix\.([a-zA-Z0-9_]+)\s*\}\}").unwrap();
|
||||
static ref HASH_FILES_PATTERN: Regex =
|
||||
Regex::new(r"\$\{\{\s*hashFiles\(([^)]+)\)\s*\}\}").unwrap();
|
||||
}
|
||||
|
||||
/// Preprocesses a command string to replace GitHub-style matrix variable references
|
||||
@@ -48,9 +52,126 @@ pub fn process_step_run(run: &str, matrix_combination: &Option<HashMap<String, V
|
||||
}
|
||||
}
|
||||
|
||||
/// Replace `${{ hashFiles(...) }}` expressions with the SHA-256 hash of matched files.
|
||||
///
|
||||
/// Accepts one or more comma-separated, quoted glob patterns. Files are matched
|
||||
/// relative to `workspace`, sorted lexicographically, and hashed in order to
|
||||
/// produce a deterministic digest — matching GitHub Actions behavior.
|
||||
///
|
||||
/// Returns `Err` if any matched file cannot be read.
|
||||
pub fn preprocess_hash_files(text: &str, workspace: &Path) -> Result<String, String> {
|
||||
let mut error: Option<String> = None;
|
||||
let result = HASH_FILES_PATTERN
|
||||
.replace_all(text, |caps: ®ex::Captures| {
|
||||
if error.is_some() {
|
||||
return String::new();
|
||||
}
|
||||
let args_raw = &caps[1];
|
||||
match compute_hash_files(args_raw, workspace) {
|
||||
Ok(hash) => hash,
|
||||
Err(e) => {
|
||||
error = Some(e);
|
||||
String::new()
|
||||
}
|
||||
}
|
||||
})
|
||||
.into_owned();
|
||||
match error {
|
||||
Some(e) => Err(e),
|
||||
None => Ok(result),
|
||||
}
|
||||
}
|
||||
|
||||
/// Compute a SHA-256 hash of the contents of all files matching the given glob patterns.
|
||||
///
|
||||
/// `args_raw` is the raw argument string inside `hashFiles(...)`, e.g.
|
||||
/// `'**/package-lock.json', '**/yarn.lock'`.
|
||||
///
|
||||
/// Returns `Ok(hash)` on success or `Err(message)` if any matched file cannot be read.
|
||||
fn compute_hash_files(args_raw: &str, workspace: &Path) -> Result<String, String> {
|
||||
// Parse comma-separated, quoted patterns
|
||||
let patterns: Vec<&str> = args_raw
|
||||
.split(',')
|
||||
.map(|s| s.trim().trim_matches('\'').trim_matches('"'))
|
||||
.filter(|s| !s.is_empty())
|
||||
.collect();
|
||||
|
||||
if patterns.is_empty() {
|
||||
return Ok(String::new());
|
||||
}
|
||||
|
||||
// Reject patterns containing path traversal components
|
||||
for pattern in &patterns {
|
||||
if pattern.split('/').any(|seg| seg == "..") {
|
||||
return Err(format!(
|
||||
"hashFiles: pattern '{}' contains '..' path traversal",
|
||||
pattern
|
||||
));
|
||||
}
|
||||
}
|
||||
|
||||
// Collect all matching files
|
||||
let mut matched_files = Vec::new();
|
||||
for pattern in &patterns {
|
||||
let full_pattern = workspace.join(pattern).to_string_lossy().to_string();
|
||||
if let Ok(entries) = glob::glob(&full_pattern) {
|
||||
for entry in entries.flatten() {
|
||||
if entry.is_file() {
|
||||
matched_files.push(entry);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if matched_files.is_empty() {
|
||||
// GHA returns the SHA-256 of empty input when no files match
|
||||
return Ok(format!("{:x}", Sha256::new().finalize()));
|
||||
}
|
||||
|
||||
// Sort for deterministic output (GHA sorts lexicographically)
|
||||
matched_files.sort();
|
||||
matched_files.dedup();
|
||||
|
||||
// Hash all file contents (stream to avoid loading large files into memory)
|
||||
let mut hasher = Sha256::new();
|
||||
for path in &matched_files {
|
||||
let mut file = std::fs::File::open(path)
|
||||
.map_err(|e| format!("hashFiles: could not read '{}': {}", path.display(), e))?;
|
||||
std::io::copy(&mut file, &mut hasher)
|
||||
.map_err(|e| format!("hashFiles: could not read '{}': {}", path.display(), e))?;
|
||||
}
|
||||
|
||||
Ok(format!("{:x}", hasher.finalize()))
|
||||
}
|
||||
|
||||
/// Apply all expression substitutions: hashFiles, matrix variables.
|
||||
///
|
||||
/// Returns `Err` if a `hashFiles()` expression fails (e.g. unreadable file).
|
||||
pub fn preprocess_expressions(
|
||||
text: &str,
|
||||
workspace: &Path,
|
||||
matrix_combination: &Option<HashMap<String, Value>>,
|
||||
) -> Result<String, String> {
|
||||
// First resolve hashFiles
|
||||
let result = preprocess_hash_files(text, workspace)?;
|
||||
// Then resolve matrix variables
|
||||
Ok(if let Some(matrix) = matrix_combination {
|
||||
preprocess_command(&result, matrix)
|
||||
} else {
|
||||
MATRIX_PATTERN
|
||||
.replace_all(&result, |caps: ®ex::Captures| {
|
||||
let var_name = &caps[1];
|
||||
format!("\\${{{{ matrix.{} }}}}", var_name)
|
||||
})
|
||||
.to_string()
|
||||
})
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use std::fs;
|
||||
use tempfile::tempdir;
|
||||
|
||||
#[test]
|
||||
fn test_preprocess_simple_matrix_vars() {
|
||||
@@ -103,4 +224,119 @@ mod tests {
|
||||
|
||||
assert_eq!(processed, "echo \"Value: \\${{ matrix.value }}\"");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hash_files_single_pattern() {
|
||||
let dir = tempdir().unwrap();
|
||||
fs::write(dir.path().join("package-lock.json"), "lock-content").unwrap();
|
||||
fs::write(dir.path().join("other.txt"), "other").unwrap();
|
||||
|
||||
let text = "${{ hashFiles('package-lock.json') }}";
|
||||
let result = preprocess_hash_files(text, dir.path()).unwrap();
|
||||
|
||||
assert!(!result.is_empty());
|
||||
assert!(!result.contains("hashFiles"));
|
||||
// Hash should be 64 hex chars (SHA-256)
|
||||
assert_eq!(result.len(), 64);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hash_files_multiple_patterns() {
|
||||
let dir = tempdir().unwrap();
|
||||
fs::write(dir.path().join("a.lock"), "aaa").unwrap();
|
||||
fs::write(dir.path().join("b.json"), "bbb").unwrap();
|
||||
|
||||
let text = "${{ hashFiles('*.lock', '*.json') }}";
|
||||
let result = preprocess_hash_files(text, dir.path()).unwrap();
|
||||
|
||||
assert_eq!(result.len(), 64);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hash_files_no_matches_returns_hash_of_empty() {
|
||||
let dir = tempdir().unwrap();
|
||||
|
||||
let text = "${{ hashFiles('nonexistent-*.xyz') }}";
|
||||
let result = preprocess_hash_files(text, dir.path()).unwrap();
|
||||
|
||||
// GHA returns SHA-256 of empty input when no files match
|
||||
let expected = format!("{:x}", Sha256::new().finalize());
|
||||
assert_eq!(result, expected);
|
||||
assert_eq!(result.len(), 64);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hash_files_deterministic() {
|
||||
let dir = tempdir().unwrap();
|
||||
fs::write(dir.path().join("a.txt"), "hello").unwrap();
|
||||
fs::write(dir.path().join("b.txt"), "world").unwrap();
|
||||
|
||||
let text = "${{ hashFiles('*.txt') }}";
|
||||
let r1 = preprocess_hash_files(text, dir.path()).unwrap();
|
||||
let r2 = preprocess_hash_files(text, dir.path()).unwrap();
|
||||
|
||||
assert_eq!(r1, r2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hash_files_glob_recursive() {
|
||||
let dir = tempdir().unwrap();
|
||||
let sub = dir.path().join("sub");
|
||||
fs::create_dir(&sub).unwrap();
|
||||
fs::write(sub.join("deep.lock"), "deep-content").unwrap();
|
||||
|
||||
let text = "${{ hashFiles('**/deep.lock') }}";
|
||||
let result = preprocess_hash_files(text, dir.path()).unwrap();
|
||||
|
||||
assert_eq!(result.len(), 64);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hash_files_inline_with_other_text() {
|
||||
let dir = tempdir().unwrap();
|
||||
fs::write(dir.path().join("Cargo.lock"), "lockfile").unwrap();
|
||||
|
||||
let text = "cache-key-${{ hashFiles('Cargo.lock') }}-suffix";
|
||||
let result = preprocess_hash_files(text, dir.path()).unwrap();
|
||||
|
||||
assert!(result.starts_with("cache-key-"));
|
||||
assert!(result.ends_with("-suffix"));
|
||||
assert!(!result.contains("hashFiles"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn preprocess_expressions_combines_hash_and_matrix() {
|
||||
let dir = tempdir().unwrap();
|
||||
fs::write(dir.path().join("Cargo.lock"), "lockfile").unwrap();
|
||||
|
||||
let mut matrix = HashMap::new();
|
||||
matrix.insert("os".to_string(), Value::String("ubuntu".to_string()));
|
||||
|
||||
let text = "${{ matrix.os }}-${{ hashFiles('Cargo.lock') }}";
|
||||
let result = preprocess_expressions(text, dir.path(), &Some(matrix)).unwrap();
|
||||
|
||||
assert!(result.starts_with("ubuntu-"));
|
||||
assert!(!result.contains("hashFiles"));
|
||||
assert!(!result.contains("matrix"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hash_files_rejects_path_traversal() {
|
||||
let dir = tempdir().unwrap();
|
||||
fs::write(dir.path().join("legit.txt"), "content").unwrap();
|
||||
|
||||
let text = "${{ hashFiles('../../etc/passwd') }}";
|
||||
let result = preprocess_hash_files(text, dir.path());
|
||||
assert!(result.is_err());
|
||||
assert!(result.unwrap_err().contains("path traversal"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hash_files_rejects_mid_path_traversal() {
|
||||
let dir = tempdir().unwrap();
|
||||
|
||||
let result = compute_hash_files("'subdir/../../etc/passwd'", dir.path());
|
||||
assert!(result.is_err());
|
||||
assert!(result.unwrap_err().contains("path traversal"));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -119,6 +119,7 @@ pub fn convert_to_workflow_format(pipeline: &Pipeline) -> workflow::WorkflowDefi
|
||||
on: vec!["push".to_string()], // Default trigger
|
||||
on_raw: serde_yaml::Value::String("push".to_string()),
|
||||
jobs: HashMap::new(),
|
||||
defaults: None,
|
||||
};
|
||||
|
||||
// Convert each GitLab job to a GitHub Actions job
|
||||
@@ -143,6 +144,8 @@ pub fn convert_to_workflow_format(pipeline: &Pipeline) -> workflow::WorkflowDefi
|
||||
uses: None,
|
||||
with: None,
|
||||
secrets: None,
|
||||
timeout_minutes: None,
|
||||
defaults: None,
|
||||
};
|
||||
|
||||
// Add job-specific environment variables
|
||||
|
||||
@@ -122,6 +122,20 @@ pub struct JobContainer {
|
||||
pub options: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize, Serialize, Clone, Default)]
|
||||
pub struct DefaultsRun {
|
||||
#[serde(default)]
|
||||
pub shell: Option<String>,
|
||||
#[serde(default, rename = "working-directory")]
|
||||
pub working_directory: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize, Serialize, Clone, Default)]
|
||||
pub struct Defaults {
|
||||
#[serde(default)]
|
||||
pub run: Option<DefaultsRun>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize, Serialize)]
|
||||
pub struct WorkflowDefinition {
|
||||
pub name: String,
|
||||
@@ -130,6 +144,8 @@ pub struct WorkflowDefinition {
|
||||
#[serde(rename = "on")] // Raw access to the 'on' field for custom handling
|
||||
pub on_raw: serde_yaml::Value,
|
||||
pub jobs: HashMap<String, Job>,
|
||||
#[serde(default)]
|
||||
pub defaults: Option<Defaults>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize, Serialize, Default)]
|
||||
@@ -171,6 +187,10 @@ pub struct Job {
|
||||
pub with: Option<HashMap<String, String>>,
|
||||
#[serde(default)]
|
||||
pub secrets: Option<serde_yaml::Value>,
|
||||
#[serde(default, rename = "timeout-minutes")]
|
||||
pub timeout_minutes: Option<f64>,
|
||||
#[serde(default)]
|
||||
pub defaults: Option<Defaults>,
|
||||
}
|
||||
|
||||
impl Job {
|
||||
@@ -391,6 +411,7 @@ mod tests {
|
||||
on: vec![],
|
||||
on_raw: serde_yaml::Value::Null,
|
||||
jobs: Default::default(),
|
||||
defaults: None,
|
||||
};
|
||||
let info = wd.resolve_action("actions/checkout@v4");
|
||||
assert_eq!(info.repository, "actions/checkout");
|
||||
@@ -407,6 +428,7 @@ mod tests {
|
||||
on: vec![],
|
||||
on_raw: serde_yaml::Value::Null,
|
||||
jobs: Default::default(),
|
||||
defaults: None,
|
||||
};
|
||||
let info = wd.resolve_action("owner/repo");
|
||||
assert_eq!(info.repository, "owner/repo");
|
||||
@@ -421,6 +443,7 @@ mod tests {
|
||||
on: vec![],
|
||||
on_raw: serde_yaml::Value::Null,
|
||||
jobs: Default::default(),
|
||||
defaults: None,
|
||||
};
|
||||
let info = wd.resolve_action("docker://alpine:3.18");
|
||||
assert_eq!(info.repository, "docker://alpine:3.18");
|
||||
@@ -436,6 +459,7 @@ mod tests {
|
||||
on: vec![],
|
||||
on_raw: serde_yaml::Value::Null,
|
||||
jobs: Default::default(),
|
||||
defaults: None,
|
||||
};
|
||||
let info = wd.resolve_action("./my-action");
|
||||
assert_eq!(info.repository, "./my-action");
|
||||
@@ -451,6 +475,7 @@ mod tests {
|
||||
on: vec![],
|
||||
on_raw: serde_yaml::Value::Null,
|
||||
jobs: Default::default(),
|
||||
defaults: None,
|
||||
};
|
||||
// Docker image references can use @sha256:digest — the full string is the image ref
|
||||
let info = wd.resolve_action("docker://alpine@sha256:abcdef1234567890");
|
||||
@@ -467,6 +492,7 @@ mod tests {
|
||||
on: vec![],
|
||||
on_raw: serde_yaml::Value::Null,
|
||||
jobs: Default::default(),
|
||||
defaults: None,
|
||||
};
|
||||
let info = wd.resolve_action("actions/checkout@a81bbbf8298c0fa03ea29cdc473d45769f953675");
|
||||
assert_eq!(info.repository, "actions/checkout");
|
||||
@@ -481,6 +507,7 @@ mod tests {
|
||||
on: vec![],
|
||||
on_raw: serde_yaml::Value::Null,
|
||||
jobs: Default::default(),
|
||||
defaults: None,
|
||||
};
|
||||
let info = wd.resolve_action("owner/repo/path/to/action@v2");
|
||||
assert_eq!(info.repository, "owner/repo");
|
||||
@@ -497,6 +524,7 @@ mod tests {
|
||||
on: vec![],
|
||||
on_raw: serde_yaml::Value::Null,
|
||||
jobs: Default::default(),
|
||||
defaults: None,
|
||||
};
|
||||
let info = wd.resolve_action("github/codeql-action/init@v3");
|
||||
assert_eq!(info.repository, "github/codeql-action");
|
||||
@@ -716,6 +744,83 @@ jobs:
|
||||
assert_eq!(step.continue_on_error, Some(true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_job_timeout_minutes() {
|
||||
let temp_dir = tempdir().unwrap();
|
||||
let workflow_path = temp_dir.path().join("workflow.yml");
|
||||
|
||||
let content = r#"
|
||||
name: timeout-test
|
||||
on: push
|
||||
jobs:
|
||||
build:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
steps:
|
||||
- run: echo hello
|
||||
no-timeout:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- run: echo world
|
||||
"#;
|
||||
fs::write(&workflow_path, content).unwrap();
|
||||
|
||||
let parsed = parse_workflow(&workflow_path).unwrap();
|
||||
let build_job = parsed.jobs.get("build").unwrap();
|
||||
assert_eq!(build_job.timeout_minutes, Some(30.0));
|
||||
|
||||
let no_timeout_job = parsed.jobs.get("no-timeout").unwrap();
|
||||
assert_eq!(no_timeout_job.timeout_minutes, None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_workflow_defaults() {
|
||||
let temp_dir = tempdir().unwrap();
|
||||
let workflow_path = temp_dir.path().join("workflow.yml");
|
||||
|
||||
let content = r#"
|
||||
name: defaults-test
|
||||
on: push
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
working-directory: ./src
|
||||
jobs:
|
||||
build:
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
shell: sh
|
||||
working-directory: ./app
|
||||
steps:
|
||||
- run: echo hello
|
||||
no-defaults:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- run: echo world
|
||||
"#;
|
||||
fs::write(&workflow_path, content).unwrap();
|
||||
|
||||
let parsed = parse_workflow(&workflow_path).unwrap();
|
||||
|
||||
// Workflow-level defaults
|
||||
let wf_defaults = parsed.defaults.as_ref().unwrap();
|
||||
let wf_run = wf_defaults.run.as_ref().unwrap();
|
||||
assert_eq!(wf_run.shell.as_deref(), Some("bash"));
|
||||
assert_eq!(wf_run.working_directory.as_deref(), Some("./src"));
|
||||
|
||||
// Job-level defaults override workflow defaults
|
||||
let build_job = parsed.jobs.get("build").unwrap();
|
||||
let job_defaults = build_job.defaults.as_ref().unwrap();
|
||||
let job_run = job_defaults.run.as_ref().unwrap();
|
||||
assert_eq!(job_run.shell.as_deref(), Some("sh"));
|
||||
assert_eq!(job_run.working_directory.as_deref(), Some("./app"));
|
||||
|
||||
// Job without defaults
|
||||
let no_defaults_job = parsed.jobs.get("no-defaults").unwrap();
|
||||
assert!(no_defaults_job.defaults.is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_strategy_matrix() {
|
||||
let temp_dir = tempdir().unwrap();
|
||||
|
||||
Reference in New Issue
Block a user