diff --git a/crates/evaluator/README.md b/crates/evaluator/README.md index fa4b931..d23c4c0 100644 --- a/crates/evaluator/README.md +++ b/crates/evaluator/README.md @@ -2,7 +2,7 @@ Small, focused helper for statically evaluating GitHub Actions workflow files. -- **Purpose**: Fast structural checks (e.g., `name`, `on`, `jobs`) before deeper validation/execution +- **Purpose**: Fast structural checks (e.g., `name`, `on`, `jobs`) and composite action input cross-checking before deeper validation/execution - **Used by**: `wrkflw` CLI and TUI during validation flows ### Example diff --git a/crates/executor/README.md b/crates/executor/README.md index c2a4e10..9c170d5 100644 --- a/crates/executor/README.md +++ b/crates/executor/README.md @@ -2,10 +2,12 @@ The execution engine that runs GitHub Actions workflows locally (Docker, Podman, or emulation). -- **Features**: - - Job graph execution with `needs` ordering and parallelism - - Docker/Podman container steps and emulation mode - - Basic environment/context wiring compatible with Actions +- Job graph execution with `needs` ordering and parallel independent jobs +- Docker/Podman container steps and emulation mode +- Run individual jobs via `target_job` / `--job` flag +- GitHub Actions environment file support (`GITHUB_OUTPUT`, `GITHUB_ENV`, `GITHUB_PATH`, `GITHUB_STEP_SUMMARY`) with read-back +- Docker-based action resolution (container, JavaScript, composite, local) +- Job-level `container:` directive support - **Used by**: `wrkflw` CLI and TUI ### API sketch @@ -17,12 +19,10 @@ let cfg = ExecutionConfig { runtime: RuntimeType::Docker, verbose: true, preserve_containers_on_failure: false, - target_job: None, + target_job: Some("build".to_string()), // run a single job }; -// Path to a workflow YAML let workflow_path = std::path::Path::new(".github/workflows/ci.yml"); - let result = execute_workflow(workflow_path, cfg).await?; println!("workflow status: {:?}", result.summary_status); ``` diff --git a/crates/secrets/README.md b/crates/secrets/README.md index 7d435c2..8dda76c 100644 --- a/crates/secrets/README.md +++ b/crates/secrets/README.md @@ -1,20 +1,16 @@ # wrkflw-secrets -Comprehensive secrets management for wrkflw workflow execution. This crate provides secure handling of secrets with support for multiple providers, encryption, masking, and GitHub Actions-compatible variable substitution. +Secrets management for wrkflw workflow execution. Provides secure handling of secrets with multiple providers, encryption, masking, and GitHub Actions-compatible `${{ secrets.* }}` substitution. ## Features -- **Multiple Secret Providers**: Environment variables, files, HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, Google Cloud Secret Manager -- **Secure Storage**: AES-256-GCM encryption for secrets at rest -- **Variable Substitution**: GitHub Actions-compatible `${{ secrets.* }}` syntax -- **Secret Masking**: Automatic masking of secrets in logs and output with pattern detection -- **Caching**: Optional caching with TTL for performance optimization -- **Rate Limiting**: Built-in protection against secret access abuse -- **Input Validation**: Comprehensive validation of secret names and values -- **Health Checks**: Provider health monitoring and diagnostics -- **Configuration**: Flexible YAML/JSON configuration with environment variable support -- **Thread Safety**: Full async/await support with concurrent access -- **Performance Optimized**: Compiled regex patterns and caching for high-throughput scenarios +- **Providers**: environment variables, files (JSON/YAML/.env), HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, GCP Secret Manager +- **Encryption**: AES-256-GCM encrypted storage for secrets at rest +- **Masking**: automatic masking of secrets in logs (GitHub tokens, AWS keys, JWTs, etc.) +- **Substitution**: GitHub Actions-compatible `${{ secrets.* }}` and `${{ secrets.provider:name }}` syntax +- **Caching**: optional TTL-based cache for frequently accessed secrets +- **Rate limiting**: built-in protection against secret access abuse +- **Validation**: comprehensive input validation for secret names and values ## Quick Start @@ -23,48 +19,26 @@ use wrkflw_secrets::prelude::*; #[tokio::main] async fn main() -> SecretResult<()> { - // Create a secret manager with default configuration let manager = SecretManager::default().await?; - - // Set an environment variable + std::env::set_var("GITHUB_TOKEN", "ghp_your_token_here"); - - // Get a secret let secret = manager.get_secret("GITHUB_TOKEN").await?; - println!("Token: {}", secret.value()); - - // Use secret substitution - let mut substitution = SecretSubstitution::new(&manager); - let template = "curl -H 'Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}' https://api.github.com"; - let resolved = substitution.substitute(template).await?; - + + // Substitute in templates + let mut sub = SecretSubstitution::new(&manager); + let resolved = sub.substitute("Bearer ${{ secrets.GITHUB_TOKEN }}").await?; + // Mask secrets in logs let mut masker = SecretMasker::new(); masker.add_secret(secret.value()); - let safe_log = masker.mask(&resolved); - println!("Safe log: {}", safe_log); - + println!("{}", masker.mask(&resolved)); + Ok(()) } ``` ## Configuration -### Environment Variables - -```bash -# Set default provider -export WRKFLW_DEFAULT_SECRET_PROVIDER=env - -# Enable/disable secret masking -export WRKFLW_SECRET_MASKING=true - -# Set operation timeout -export WRKFLW_SECRET_TIMEOUT=30 -``` - -### Configuration File - Create `~/.wrkflw/secrets.yml`: ```yaml @@ -78,11 +52,9 @@ providers: env: type: environment prefix: "WRKFLW_SECRET_" - file: type: file path: "~/.wrkflw/secrets.json" - vault: type: vault url: "https://vault.example.com" @@ -92,296 +64,13 @@ providers: mount_path: "secret" ``` -## Secret Providers - -### Environment Variables - -The simplest provider reads secrets from environment variables: - -```rust -// With prefix -std::env::set_var("WRKFLW_SECRET_API_KEY", "secret_value"); -let secret = manager.get_secret_from_provider("env", "API_KEY").await?; - -// Without prefix -std::env::set_var("GITHUB_TOKEN", "ghp_token"); -let secret = manager.get_secret_from_provider("env", "GITHUB_TOKEN").await?; -``` - -### File-based Storage - -Store secrets in JSON, YAML, or environment files: - -**JSON format** (`secrets.json`): -```json -{ - "API_KEY": "secret_api_key", - "DB_PASSWORD": "secret_password" -} -``` - -**Environment format** (`secrets.env`): -```bash -API_KEY=secret_api_key -DB_PASSWORD="quoted password" -GITHUB_TOKEN='single quoted token' -``` - -**YAML format** (`secrets.yml`): -```yaml -API_KEY: secret_api_key -DB_PASSWORD: secret_password -``` - -### HashiCorp Vault - -```yaml -providers: - vault: - type: vault - url: "https://vault.example.com" - auth: - method: token - token: "${VAULT_TOKEN}" - mount_path: "secret" -``` - -### AWS Secrets Manager - -```yaml -providers: - aws: - type: aws_secrets_manager - region: "us-east-1" - role_arn: "arn:aws:iam::123456789012:role/SecretRole" # optional -``` - -### Azure Key Vault - -```yaml -providers: - azure: - type: azure_key_vault - vault_url: "https://myvault.vault.azure.net/" - auth: - method: service_principal - client_id: "${AZURE_CLIENT_ID}" - client_secret: "${AZURE_CLIENT_SECRET}" - tenant_id: "${AZURE_TENANT_ID}" -``` - -### Google Cloud Secret Manager - -```yaml -providers: - gcp: - type: gcp_secret_manager - project_id: "my-project" - key_file: "/path/to/service-account.json" # optional -``` - -## Variable Substitution - -Support for GitHub Actions-compatible secret references: - -```rust -let mut substitution = SecretSubstitution::new(&manager); - -// Default provider -let template = "TOKEN=${{ secrets.GITHUB_TOKEN }}"; -let resolved = substitution.substitute(template).await?; - -// Specific provider -let template = "API_KEY=${{ secrets.vault:API_KEY }}"; -let resolved = substitution.substitute(template).await?; -``` - -## Secret Masking - -Automatically mask secrets in logs and output: - -```rust -let mut masker = SecretMasker::new(); - -// Add specific secrets -masker.add_secret("secret_value"); - -// Automatic pattern detection for common secret types -let log = "Token: ghp_1234567890123456789012345678901234567890"; -let masked = masker.mask(log); -// Output: "Token: ghp_***" -``` - -Supported patterns: -- GitHub Personal Access Tokens (`ghp_*`) -- GitHub App tokens (`ghs_*`) -- GitHub OAuth tokens (`gho_*`) -- AWS Access Keys (`AKIA*`) -- JWT tokens -- Generic API keys - -## Encrypted Storage - -For sensitive environments, use encrypted storage: - -```rust -use wrkflw_secrets::storage::{EncryptedSecretStore, KeyDerivation}; - -// Create encrypted store -let (mut store, key) = EncryptedSecretStore::new()?; - -// Add secrets -store.add_secret(&key, "API_KEY", "secret_value")?; - -// Save to file -store.save_to_file("secrets.encrypted").await?; - -// Load from file -let loaded_store = EncryptedSecretStore::load_from_file("secrets.encrypted").await?; -let secret = loaded_store.get_secret(&key, "API_KEY")?; -``` - -## Error Handling - -All operations return `SecretResult` with comprehensive error types: - -```rust -match manager.get_secret("MISSING_SECRET").await { - Ok(secret) => println!("Secret: {}", secret.value()), - Err(SecretError::NotFound { name }) => { - eprintln!("Secret '{}' not found", name); - } - Err(SecretError::ProviderNotFound { provider }) => { - eprintln!("Provider '{}' not configured", provider); - } - Err(SecretError::AuthenticationFailed { provider, reason }) => { - eprintln!("Auth failed for {}: {}", provider, reason); - } - Err(e) => eprintln!("Error: {}", e), -} -``` - -## Health Checks - -Monitor provider health: - -```rust -let health_results = manager.health_check().await; -for (provider, result) in health_results { - match result { - Ok(()) => println!("✓ {} is healthy", provider), - Err(e) => println!("✗ {} failed: {}", provider, e), - } -} -``` - -## Security Best Practices - -1. **Use encryption** for secrets at rest -2. **Enable masking** to prevent secrets in logs -3. **Rotate secrets** regularly -4. **Use least privilege** access for secret providers -5. **Monitor access** through health checks and logging -6. **Use provider-specific authentication** (IAM roles, service principals) -7. **Configure rate limiting** to prevent abuse -8. **Validate input** - the system automatically validates secret names and values - -## Rate Limiting - -Protect against abuse with built-in rate limiting: - -```rust -use wrkflw_secrets::rate_limit::RateLimitConfig; -use std::time::Duration; - -let mut config = SecretConfig::default(); -config.rate_limit = RateLimitConfig { - max_requests: 100, // Max requests per window - window_duration: Duration::from_secs(60), // 1 minute window - enabled: true, -}; - -let manager = SecretManager::new(config).await?; - -// Rate limiting is automatically applied to all secret access operations -match manager.get_secret("API_KEY").await { - Ok(secret) => println!("Success: {}", secret.value()), - Err(SecretError::RateLimitExceeded(msg)) => { - println!("Rate limited: {}", msg); - } - Err(e) => println!("Other error: {}", e), -} -``` - -## Input Validation - -All inputs are automatically validated: - -```rust -// Secret names must: -// - Be 1-255 characters long -// - Contain only letters, numbers, underscores, hyphens, and dots -// - Not start or end with dots -// - Not contain consecutive dots -// - Not be reserved system names - -// Secret values must: -// - Be under 1MB in size -// - Not contain null bytes -// - Be valid UTF-8 - -// Invalid examples that will be rejected: -manager.get_secret("").await; // Empty name -manager.get_secret("invalid/name").await; // Invalid characters -manager.get_secret(".hidden").await; // Starts with dot -manager.get_secret("CON").await; // Reserved name -``` - -## Performance Features - -### Caching - -```rust -let config = SecretConfig { - enable_caching: true, - cache_ttl_seconds: 300, // 5 minutes - ..Default::default() -}; -``` - -### Optimized Pattern Matching - -- Pre-compiled regex patterns for secret detection -- Global pattern cache using `OnceLock` -- Efficient string replacement algorithms -- Cached mask generation - -### Benchmarking - -Run performance benchmarks: - -```bash -cargo bench -p wrkflw-secrets -``` - ## Feature Flags -Enable optional providers: - ```toml [dependencies] -wrkflw-secrets = { version = "0.1", features = ["vault-provider", "aws-provider"] } +wrkflw-secrets = { version = "0.7", features = ["vault-provider", "aws-provider"] } ``` -Available features: -- `env-provider` (default) -- `file-provider` (default) -- `vault-provider` -- `aws-provider` -- `azure-provider` -- `gcp-provider` -- `all-providers` +Available: `env-provider` (default), `file-provider` (default), `vault-provider`, `aws-provider`, `azure-provider`, `gcp-provider`, `all-providers`. -## License - -MIT License - see LICENSE file for details. +See the [secrets demo](../../examples/secrets-demo/) for end-to-end usage examples. diff --git a/crates/ui/README.md b/crates/ui/README.md index 3ef3c59..7b2ccc7 100644 --- a/crates/ui/README.md +++ b/crates/ui/README.md @@ -3,7 +3,9 @@ Terminal user interface for browsing workflows, running them, and viewing logs. - Tabs: Workflows, Execution, Logs, Help +- Job selection mode: pick and run individual jobs within a workflow - Hotkeys: `1-4`, `Tab`, `Enter`, `r`, `R`, `t`, `v`, `e`, `q`, etc. +- Optional: enabled via the `tui` cargo feature flag - Integrates with `wrkflw-executor` and `wrkflw-logging` ### Example diff --git a/crates/wrkflw/README.md b/crates/wrkflw/README.md index e2b5dec..9950b1a 100644 --- a/crates/wrkflw/README.md +++ b/crates/wrkflw/README.md @@ -52,7 +52,7 @@ wrkflw tui --runtime podman - **run**: Execute a workflow or pipeline locally - Runtimes: `docker` (default), `podman`, `emulation` - - Flags: `--runtime`, `--preserve-containers-on-failure`, `--gitlab`, `--verbose` + - Flags: `--runtime`, `--job` (run a single job), `--jobs` (list jobs), `--preserve-containers-on-failure`, `--gitlab`, `--verbose` - **tui**: Interactive terminal interface - Browse workflows, execute, and inspect logs and job details