Mirrors/wrkflw
1
0
Fork 0
mirror of https://github.com/bahdotsh/wrkflw.git synced 2026-09-01 19:50:23 +02:00
Code Issues Packages Projects Releases Wiki Activity

docs: update crate READMEs for latest features and trim secrets

Browse Source 
The crate READMEs were quietly falling behind the actual code. The
executor README didn't mention --job, environment file read-back,
or job-level container directives. The UI README didn't mention job
selection mode or the tui feature flag. The evaluator README didn't
mention composite action input cross-checking.

Meanwhile, the secrets README was 387 lines of documentation for a
crate whose siblings average 25. It had full provider configuration
examples, rate limiting docs, input validation specs, and
benchmarking instructions — all of which belong in rustdoc, not a
README that's supposed to give you a quick overview.

Trim secrets to ~80 lines. Update executor, ui, evaluator, and
wrkflw READMEs to reflect features from PRs #77-#83.
This commit is contained in:
bahdotsh
2026-04-02 23:57:40 +05:30
parent c67e4bfd20
commit af5e80e97e
5 changed files with 30 additions and 339 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
crates/evaluator/README.md
View File

@@ -2,7 +2,7 @@
Small, focused helper for statically evaluating GitHub Actions workflow files.
- **Purpose**: Fast structural checks (e.g., `name`, `on`, `jobs`) before deeper validation/execution
- **Purpose**: Fast structural checks (e.g., `name`, `on`, `jobs`) and composite action input cross-checking before deeper validation/execution
- **Used by**: `wrkflw` CLI and TUI during validation flows
### Example

14
crates/executor/README.md
View File

@@ -2,10 +2,12 @@
The execution engine that runs GitHub Actions workflows locally (Docker, Podman, or emulation).
- **Features**:
- Job graph execution with `needs` ordering and parallelism
- Docker/Podman container steps and emulation mode
- Basic environment/context wiring compatible with Actions
- Job graph execution with `needs` ordering and parallel independent jobs
- Docker/Podman container steps and emulation mode
- Run individual jobs via `target_job` / `--job` flag
- GitHub Actions environment file support (`GITHUB_OUTPUT`, `GITHUB_ENV`, `GITHUB_PATH`, `GITHUB_STEP_SUMMARY`) with read-back
- Docker-based action resolution (container, JavaScript, composite, local)
- Job-level `container:` directive support
- **Used by**: `wrkflw` CLI and TUI
### API sketch
@@ -17,12 +19,10 @@ let cfg = ExecutionConfig {
runtime: RuntimeType::Docker,
verbose: true,
preserve_containers_on_failure: false,
target_job: None,
target_job: Some("build".to_string()), // run a single job
};
// Path to a workflow YAML
let workflow_path = std::path::Path::new(".github/workflows/ci.yml");
let result = execute_workflow(workflow_path, cfg).await?;
println!("workflow status: {:?}", result.summary_status);
```

349
crates/secrets/README.md
View File

@@ -1,20 +1,16 @@
# wrkflw-secrets
Comprehensive secrets management for wrkflw workflow execution. This crate provides secure handling of secrets with support for multiple providers, encryption, masking, and GitHub Actions-compatible variable substitution.
Secrets management for wrkflw workflow execution. Provides secure handling of secrets with multiple providers, encryption, masking, and GitHub Actions-compatible `${{ secrets.* }}` substitution.
## Features
- **Multiple Secret Providers**: Environment variables, files, HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, Google Cloud Secret Manager
- **Secure Storage**: AES-256-GCM encryption for secrets at rest
- **Variable Substitution**: GitHub Actions-compatible `${{ secrets.* }}` syntax
- **Secret Masking**: Automatic masking of secrets in logs and output with pattern detection
- **Caching**: Optional caching with TTL for performance optimization
- **Rate Limiting**: Built-in protection against secret access abuse
- **Input Validation**: Comprehensive validation of secret names and values
- **Health Checks**: Provider health monitoring and diagnostics
- **Configuration**: Flexible YAML/JSON configuration with environment variable support
- **Thread Safety**: Full async/await support with concurrent access
- **Performance Optimized**: Compiled regex patterns and caching for high-throughput scenarios
- **Providers**: environment variables, files (JSON/YAML/.env), HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, GCP Secret Manager
- **Encryption**: AES-256-GCM encrypted storage for secrets at rest
- **Masking**: automatic masking of secrets in logs (GitHub tokens, AWS keys, JWTs, etc.)
- **Substitution**: GitHub Actions-compatible `${{ secrets.* }}` and `${{ secrets.provider:name }}` syntax
- **Caching**: optional TTL-based cache for frequently accessed secrets
- **Rate limiting**: built-in protection against secret access abuse
- **Validation**: comprehensive input validation for secret names and values
## Quick Start
@@ -23,48 +19,26 @@ use wrkflw_secrets::prelude::*;
#[tokio::main]
async fn main() -> SecretResult<()> {
// Create a secret manager with default configuration
let manager = SecretManager::default().await?;
// Set an environment variable
std::env::set_var("GITHUB_TOKEN", "ghp_your_token_here");
// Get a secret
let secret = manager.get_secret("GITHUB_TOKEN").await?;
println!("Token: {}", secret.value());
// Use secret substitution
let mut substitution = SecretSubstitution::new(&manager);
let template = "curl -H 'Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}' https://api.github.com";
let resolved = substitution.substitute(template).await?;
// Substitute in templates
let mut sub = SecretSubstitution::new(&manager);
let resolved = sub.substitute("Bearer ${{ secrets.GITHUB_TOKEN }}").await?;
// Mask secrets in logs
let mut masker = SecretMasker::new();
masker.add_secret(secret.value());
let safe_log = masker.mask(&resolved);
println!("Safe log: {}", safe_log);
println!("{}", masker.mask(&resolved));
Ok(())
}
```
## Configuration
### Environment Variables
```bash
# Set default provider
export WRKFLW_DEFAULT_SECRET_PROVIDER=env
# Enable/disable secret masking
export WRKFLW_SECRET_MASKING=true
# Set operation timeout
export WRKFLW_SECRET_TIMEOUT=30
```
### Configuration File
Create `~/.wrkflw/secrets.yml`:
```yaml
@@ -78,11 +52,9 @@ providers:
env:
type: environment
prefix: "WRKFLW_SECRET_"
file:
type: file
path: "~/.wrkflw/secrets.json"
vault:
type: vault
url: "https://vault.example.com"
@@ -92,296 +64,13 @@ providers:
mount_path: "secret"
```
## Secret Providers
### Environment Variables
The simplest provider reads secrets from environment variables:
```rust
// With prefix
std::env::set_var("WRKFLW_SECRET_API_KEY", "secret_value");
let secret = manager.get_secret_from_provider("env", "API_KEY").await?;
// Without prefix
std::env::set_var("GITHUB_TOKEN", "ghp_token");
let secret = manager.get_secret_from_provider("env", "GITHUB_TOKEN").await?;
```
### File-based Storage
Store secrets in JSON, YAML, or environment files:
**JSON format** (`secrets.json`):
```json
{
"API_KEY": "secret_api_key",
"DB_PASSWORD": "secret_password"
}
```
**Environment format** (`secrets.env`):
```bash
API_KEY=secret_api_key
DB_PASSWORD="quoted password"
GITHUB_TOKEN='single quoted token'
```
**YAML format** (`secrets.yml`):
```yaml
API_KEY: secret_api_key
DB_PASSWORD: secret_password
```
### HashiCorp Vault
```yaml
providers:
vault:
type: vault
url: "https://vault.example.com"
auth:
method: token
token: "${VAULT_TOKEN}"
mount_path: "secret"
```
### AWS Secrets Manager
```yaml
providers:
aws:
type: aws_secrets_manager
region: "us-east-1"
role_arn: "arn:aws:iam::123456789012:role/SecretRole" # optional
```
### Azure Key Vault
```yaml
providers:
azure:
type: azure_key_vault
vault_url: "https://myvault.vault.azure.net/"
auth:
method: service_principal
client_id: "${AZURE_CLIENT_ID}"
client_secret: "${AZURE_CLIENT_SECRET}"
tenant_id: "${AZURE_TENANT_ID}"
```
### Google Cloud Secret Manager
```yaml
providers:
gcp:
type: gcp_secret_manager
project_id: "my-project"
key_file: "/path/to/service-account.json" # optional
```
## Variable Substitution
Support for GitHub Actions-compatible secret references:
```rust
let mut substitution = SecretSubstitution::new(&manager);
// Default provider
let template = "TOKEN=${{ secrets.GITHUB_TOKEN }}";
let resolved = substitution.substitute(template).await?;
// Specific provider
let template = "API_KEY=${{ secrets.vault:API_KEY }}";
let resolved = substitution.substitute(template).await?;
```
## Secret Masking
Automatically mask secrets in logs and output:
```rust
let mut masker = SecretMasker::new();
// Add specific secrets
masker.add_secret("secret_value");
// Automatic pattern detection for common secret types
let log = "Token: ghp_1234567890123456789012345678901234567890";
let masked = masker.mask(log);
// Output: "Token: ghp_***"
```
Supported patterns:
- GitHub Personal Access Tokens (`ghp_*`)
- GitHub App tokens (`ghs_*`)
- GitHub OAuth tokens (`gho_*`)
- AWS Access Keys (`AKIA*`)
- JWT tokens
- Generic API keys
## Encrypted Storage
For sensitive environments, use encrypted storage:
```rust
use wrkflw_secrets::storage::{EncryptedSecretStore, KeyDerivation};
// Create encrypted store
let (mut store, key) = EncryptedSecretStore::new()?;
// Add secrets
store.add_secret(&key, "API_KEY", "secret_value")?;
// Save to file
store.save_to_file("secrets.encrypted").await?;
// Load from file
let loaded_store = EncryptedSecretStore::load_from_file("secrets.encrypted").await?;
let secret = loaded_store.get_secret(&key, "API_KEY")?;
```
## Error Handling
All operations return `SecretResult<T>` with comprehensive error types:
```rust
match manager.get_secret("MISSING_SECRET").await {
Ok(secret) => println!("Secret: {}", secret.value()),
Err(SecretError::NotFound { name }) => {
eprintln!("Secret '{}' not found", name);
}
Err(SecretError::ProviderNotFound { provider }) => {
eprintln!("Provider '{}' not configured", provider);
}
Err(SecretError::AuthenticationFailed { provider, reason }) => {
eprintln!("Auth failed for {}: {}", provider, reason);
}
Err(e) => eprintln!("Error: {}", e),
}
```
## Health Checks
Monitor provider health:
```rust
let health_results = manager.health_check().await;
for (provider, result) in health_results {
match result {
Ok(()) => println!("✓ {} is healthy", provider),
Err(e) => println!("✗ {} failed: {}", provider, e),
}
}
```
## Security Best Practices
1. **Use encryption** for secrets at rest
2. **Enable masking** to prevent secrets in logs
3. **Rotate secrets** regularly
4. **Use least privilege** access for secret providers
5. **Monitor access** through health checks and logging
6. **Use provider-specific authentication** (IAM roles, service principals)
7. **Configure rate limiting** to prevent abuse
8. **Validate input** - the system automatically validates secret names and values
## Rate Limiting
Protect against abuse with built-in rate limiting:
```rust
use wrkflw_secrets::rate_limit::RateLimitConfig;
use std::time::Duration;
let mut config = SecretConfig::default();
config.rate_limit = RateLimitConfig {
max_requests: 100, // Max requests per window
window_duration: Duration::from_secs(60), // 1 minute window
enabled: true,
};
let manager = SecretManager::new(config).await?;
// Rate limiting is automatically applied to all secret access operations
match manager.get_secret("API_KEY").await {
Ok(secret) => println!("Success: {}", secret.value()),
Err(SecretError::RateLimitExceeded(msg)) => {
println!("Rate limited: {}", msg);
}
Err(e) => println!("Other error: {}", e),
}
```
## Input Validation
All inputs are automatically validated:
```rust
// Secret names must:
// - Be 1-255 characters long
// - Contain only letters, numbers, underscores, hyphens, and dots
// - Not start or end with dots
// - Not contain consecutive dots
// - Not be reserved system names
// Secret values must:
// - Be under 1MB in size
// - Not contain null bytes
// - Be valid UTF-8
// Invalid examples that will be rejected:
manager.get_secret("").await; // Empty name
manager.get_secret("invalid/name").await; // Invalid characters
manager.get_secret(".hidden").await; // Starts with dot
manager.get_secret("CON").await; // Reserved name
```
## Performance Features
### Caching
```rust
let config = SecretConfig {
enable_caching: true,
cache_ttl_seconds: 300, // 5 minutes
..Default::default()
};
```
### Optimized Pattern Matching
- Pre-compiled regex patterns for secret detection
- Global pattern cache using `OnceLock`
- Efficient string replacement algorithms
- Cached mask generation
### Benchmarking
Run performance benchmarks:
```bash
cargo bench -p wrkflw-secrets
```
## Feature Flags
Enable optional providers:
```toml
[dependencies]
wrkflw-secrets = { version = "0.1", features = ["vault-provider", "aws-provider"] }
wrkflw-secrets = { version = "0.7", features = ["vault-provider", "aws-provider"] }
```
Available features:
- `env-provider` (default)
- `file-provider` (default)
- `vault-provider`
- `aws-provider`
- `azure-provider`
- `gcp-provider`
- `all-providers`
Available: `env-provider` (default), `file-provider` (default), `vault-provider`, `aws-provider`, `azure-provider`, `gcp-provider`, `all-providers`.
## License
MIT License - see LICENSE file for details.
See the [secrets demo](../../examples/secrets-demo/) for end-to-end usage examples.

2
crates/ui/README.md
View File

@@ -3,7 +3,9 @@
Terminal user interface for browsing workflows, running them, and viewing logs.
- Tabs: Workflows, Execution, Logs, Help
- Job selection mode: pick and run individual jobs within a workflow
- Hotkeys: `1-4`, `Tab`, `Enter`, `r`, `R`, `t`, `v`, `e`, `q`, etc.
- Optional: enabled via the `tui` cargo feature flag
- Integrates with `wrkflw-executor` and `wrkflw-logging`
### Example

2
crates/wrkflw/README.md
View File

@@ -52,7 +52,7 @@ wrkflw tui --runtime podman
- **run**: Execute a workflow or pipeline locally
- Runtimes: `docker` (default), `podman`, `emulation`
- Flags: `--runtime`, `--preserve-containers-on-failure`, `--gitlab`, `--verbose`
- Flags: `--runtime`, `--job` (run a single job), `--jobs` (list jobs), `--preserve-containers-on-failure`, `--gitlab`, `--verbose`
- **tui**: Interactive terminal interface
- Browse workflows, execute, and inspect logs and job details