Files
PowerToys/.github/skills/powertoys-verification
Boliang Zhang 4395b1d0ca [skill] Generalize verification skill into 2 scenarios (module checklist / PR validation) (#48848)
## Summary of the Pull Request

Generalizes the `powertoys-verification` agent skill from "one module
checklist" into a **single shared engine that serves two verification
scenarios**, using the *shared engine + per-scenario reference docs*
design (no duplication of the winapp mechanics):

- **A — Module checklist**: verify a supplied module checklist against
the installed build.
- **B — PR validation**: derive each PR's checklist from its description
+ diff, then drive it. A single **bits sub-decision** picks the target —
drive the **installed** build when the PR's code is already shipped (a
merged PR, or a whole release/hotfix set, e.g. the 0.100.1 14-PR
sign-off), or **build + sideload** the affected module when it isn't (an
unmerged or not-yet-released PR, e.g. #45242).

Both scenarios share the same `winapp ui` drive techniques, helper
scripts, per-module profiles, taxonomy, and report format. They differ
mainly on **what bits you run**, so the only new content is a thin
scenario layer.

## PR Checklist

- [ ] **Closes:** N/A (follow-up to the merged `powertoys-verification`
skill)
- [x] **Communication:** design discussed and agreed before
implementation
- [x] **Tests:** N/A — docs/skill-only change; validated link/constraint
checks (see below)
- [x] **Localization:** N/A
- [x] **Dev docs:** this PR *is* skill documentation
- [x] **New binaries:** none

## Detailed Description of the Pull Request / Additional comments

**`SKILL.md` (shared engine)** — broaden `description` + "When to use"
to cover both scenarios, add a **Step-0 scenario router** (plus, for B,
the bits sub-decision "is the PR's code in the build under test?"), keep
the supplied-checklist read **Scenario-A-only**, and make the
bootstrap/placeholders/"What NOT to do" scenario-aware.

**`references/scenarios/`:**
- `index.md` — router table (A/B); the **"bits under test" contract**
(the one real conflict: installed-and-immutable, vs build-and-sideload
when the PR's code isn't in the build, echoed in the report header so
the evidence chain is trustworthy); and a verdict-vocabulary mapping
(engine `PASS/FAIL/BLOCKED` <-> legacy labels).
- `module-checklist.md` (A) — supplied checklist, installed bits.
- `pr-validation.md` (B, **new — merges the former release-PR and
active-PR docs**) — derive each PR's checklist from `gh pr view/diff`;
the **bits sub-decision**; a **PR-discovery model with a size gate**
(auto-verify hotfix-sized sets <=25; for full ~100-PR releases,
scope-and-confirm instead of blind-looping) and a non-runtime-PR
pre-filter; per-PR folders + roll-up; and the **build + sideload**
front-end (in-repo worktree helper + build commands), unpackaged (run
the built runner, dismiss the expected partial-build dialogs) vs
packaged/CmdPal (`Add-AppxPackage -Register`) deploy recipes,
prove-your-bits, and restore-to-shipped cleanup.

Removed the separate `release-pr-signoff.md` and
`active-pr-validation.md` (folded into `pr-validation.md`). The existing
engine docs (`winapp-ui-testing.md`, `pre-flight.md`,
`reporting-format.md`), per-module profiles, and helper scripts are
reused unchanged — single source of truth. The `BITS:` header version is
a `<version>` placeholder (not pinned to a release).

## Validation Steps Performed

- `SKILL.md` within authoring limits: **331 lines** (< 500) and
**description 999 chars** (< 1024).
- All relative markdown links in `references/scenarios/*.md` resolve,
and every engine doc they cite exists.
- Scenario B exercised end-to-end both ways: the 0.100.1 14-PR sign-off
(8 PASS / 6 BLOCKED, installed bits) and PR #45242 (Advanced Paste, 3/3
PASS, build + sideload).

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-07-08 14:53:38 -07:00
..