mirror of
https://github.com/microsoft/PowerToys.git
synced 2026-08-29 10:09:43 +02:00
Add issue triage actions (#49828)
## Summary of the Pull Request Replaces the retired GitHub Models-based automatic issue triage and deduplication flows with the GitHub Agentic Workflow proven in the `niels9001/powertoys-ai-triage-sandbox`. This PR also: - aligns `Needs-Author-Feedback` closure to 7 days for issues and PRs; - removes the automatic GitHub Models issue/PR labeler; - removes the automatic GitHub Models new-issue deduplicator; - removes the Azure Pipelines XAML Styler verification step while retaining the local styling script. This is a draft because production rollout still requires the appropriate privacy and Responsible AI reviews. ## Issue triage rules ### Triggers and refresh behavior - Runs when an issue is opened, edited, or reopened. - Runs when the issue author attaches a `PowerToysReport_*.zip` in a comment. - Maintainers can force regeneration with `/triage refresh`. - Ignores unrelated comments, unchanged issue edits, PR comments, and bot-initiated reopens. - Uses per-issue concurrency so a newer run supersedes an older run. - Maintains one canonical triage comment instead of adding repeated bot comments. ### Comment format - Separates **For the issue author** from **For the PowerToys team**. - Mentions the author once and lists each requested action as a bullet. - Distinguishes blocking **Needed** actions from non-blocking **Recommended** actions. - Shows the product, issue kind, reported PowerToys version, concise summary, diagnostic findings, possible duplicates, and collapsed investigation checks. - Ends with a short disclosure that triage is AI-assisted and maintainers make final decisions. ### Classification and labels - Detects PowerToys bug-template issues deterministically. - Reads the selected product area and adds a matching primary `Product-*` label. - Handles production aliases such as FancyZones Editor and File Explorer preview/thumbnail areas. - Product labeling is additive: existing product and maintainer labels are never removed. - Normalizes the reported PowerToys version and adds a matching version label when one exists. - Applies `Needs-Author-Feedback` only when blocking information or an English translation is required. - Removes `Needs-Author-Feedback` when the issue becomes actionable. ### PowerToys version rule - Compares the reported version with the latest stable PowerToys GitHub release. - Older versions receive a recommended update-and-retest action. - Current versions, newer preview/dev versions, missing versions, and release lookup failures are not flagged as outdated. - Updating is advisory and does not block triage by itself. ### Reproduction rule - Concrete actions plus an observed result are sufficient. - Concise steps can use the separate Actual Behavior section as the observed result. - Passive or intermittent failures are sufficient when the timing/trigger and observed failure are clear. - Vague statements without an actionable scenario remain insufficient. - Clearly non-English steps are not treated as missing; reproduction is reassessed after the author translates the issue. ### Language rule - Classifies author-written prose as English, non-English, or uncertain. - Ignores template headings, code, logs, filenames, URLs, hidden comments, and quoted text. - Clearly non-English issues ask the author to translate the title and description to English. - Short, mixed, code-heavy, or uncertain text is not flagged. ### Diagnostic report rule - A report is **required** for diagnostic-heavy failures: crashes, hangs, startup/load failures, installation/update failures, performance failures, and service/driver/shell-integration failures. - A report is **optional** for clear reproducible UI/visual defects. - A report is **recommended**, but not blocking, for other actionable bugs. - Missing or rejected reports block only when the deterministic requirement is `REQUIRED`. ### Diagnostic report privacy and safety - Accepts only PowerToys report attachment URLs matching the expected pattern. - Enforces archive size, decompressed size, file-count, per-file, path traversal, and encryption limits. - Selects only bounded relevant metadata and product-log evidence. - Redacts email addresses, IP addresses, user paths, URLs, GUIDs, SIDs, identity fields, tokens, secrets, and passwords. - Sends only the sanitized evidence to Copilot. - Never sends the raw ZIP or extracted files to Copilot, logs, artifacts, or repository storage. - Deletes the temporary archive after processing. ### Duplicate rule - Searches only older issues using focused product, title/body, and exact technical-signal queries. - Ranks candidates deterministically before Copilot runs. - Copilot judges only the supplied candidates and returns at most five high-confidence matches. - Similar product area alone is not enough; the underlying request or failure must match. - The model never closes an issue directly. - The workflow submits the strongest match as a native GitHub duplicate-close suggestion. - **When a maintainer accepts the suggestion, GitHub automatically closes the issue as a duplicate and links it to the selected canonical issue.** - Declining the suggestion leaves the issue open. - A defensive safeguard reopens the issue and fails the run if GitHub applies the close without holding it for review. ### AI cost and permission controls - Uses the `small` model alias. - Maximum 5 turns and 10 AI credits per run. - Maximum 300 AI credits per day. - Maximum 5 runs per user per 60-minute window. - Content hashing skips unchanged work before inference. - The agent receives only `contents: read`, `issues: read`, and `copilot-requests: write`. - A separate validated safe-output job receives `issues: write`. ## Seven-day author-feedback lifecycle The existing Microsoft GitHub Policy Service configuration remains responsible for stale closure: - Open issues with `Needs-Author-Feedback` and no activity for 7 days are closed with an explanatory comment. - Open PRs with `Needs-Author-Feedback` and no activity for 7 days are closed with an explanatory comment. - An author comment removes `Needs-Author-Feedback` and returns the issue/PR to team triage. - An author push removes `Needs-Author-Feedback` from a PR. - Manually removing the label immediately makes the issue or PR ineligible for scheduled closure. ## Deprecated automation - Deletes `.github/workflows/automatic-issue-deduplication.yml`. - Deletes `.github/workflows/auto-labeler.yml`. - Automatic PR product labeling from the old Models workflow is intentionally not replaced in this PR; a production PR ownership/path map should be agreed separately. - Keeps the manual batch deduplication workflow unchanged. - Removes the passive XAML Styler verification step from `.pipelines/v2/templates/job-build-project.yml`. - Keeps `.pipelines/applyXamlStyling.ps1` available for local developer use. ## Validation Steps Performed - Compiled `.github/workflows/issue-triage.md` with `gh aw compile`. - Ran 32 focused Python tests for issue parsing, duplicate retrieval, version checks, reproduction rules, language signals, archive validation, report selection, redaction, and output privacy. - Parsed the changed workflow and resource-management YAML. - Verified the required production labels exist. - Tested the workflow against the latest 20 PowerToys issues in the sandbox; all 20 produced one canonical comment. - Verified live variants for outdated versions, intermittent/passive reproduction, non-English issues, rejected and analyzed reports, optional UI reports, and title-only issues. ## PR Checklist - [ ] **Communication:** Discussed with core contributors. - [x] **Tests:** Added/updated and all focused tests pass. - [ ] **Privacy / Responsible AI:** Complete required production reviews before enabling. - [x] **Localization:** No product UI strings are added. - [x] **Dev docs:** Updated repository automation documentation. - [x] **New binaries:** None. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 18a9b8ad-fd7e-4b9d-a06c-5e350bcde9d7 Copilot-Session: fd512b9b-db6f-4004-a65b-aa49404d568d
This commit is contained in:
78
.github/scripts/issue-triage/README.md
vendored
Normal file
78
.github/scripts/issue-triage/README.md
vendored
Normal file
@@ -0,0 +1,78 @@
|
||||
# AI-assisted issue triage
|
||||
|
||||
The workflow in `.github/workflows/issue-triage.md` maintains one canonical
|
||||
triage comment for newly opened, edited, or reopened issues. It combines
|
||||
deterministic preprocessing with one bounded GitHub Copilot pass.
|
||||
|
||||
## Rules
|
||||
|
||||
- Parse the issue template, PowerToys version, product area, reproduction
|
||||
quality, language, and diagnostic-report requirement before AI runs.
|
||||
- Compare the reported version with the latest stable PowerToys release.
|
||||
Older versions receive a non-blocking update-and-retest recommendation.
|
||||
- Retrieve a bounded set of older candidate issues using product labels,
|
||||
technical identifiers, and focused title/body searches.
|
||||
- Ask Copilot only to summarize the issue, judge supplied duplicate candidates,
|
||||
interpret sanitized diagnostics, and classify the author-written language.
|
||||
- Maintain one marked comment with separate sections for the issue author and
|
||||
the PowerToys team.
|
||||
- Mention the author once and list only needed or recommended actions.
|
||||
- Apply `Needs-Author-Feedback` when blocking information or an English
|
||||
translation is required. Removing the label disables scheduled closure.
|
||||
- Add a matching primary `Product-*` label and the reported version label
|
||||
without removing existing product or maintainer labels.
|
||||
- Submit duplicate closure as a native GitHub suggestion. A maintainer must
|
||||
accept or decline it; acceptance closes the issue as a duplicate and links
|
||||
it to the selected canonical issue.
|
||||
- Never close an issue directly from the model output.
|
||||
|
||||
## Reproduction and diagnostics
|
||||
|
||||
- Concrete actions plus an observed result are sufficient.
|
||||
- Passive or intermittent failures can be sufficient when the timing/trigger
|
||||
and observed failure are clear.
|
||||
- Non-English reproduction steps are reassessed after translation rather than
|
||||
treated as missing.
|
||||
- Diagnostic reports are required for crashes, hangs, startup/load failures,
|
||||
installation/update failures, performance failures, and system-integration
|
||||
failures.
|
||||
- Reports are optional for clear UI/visual defects and recommended for other
|
||||
actionable bugs.
|
||||
- Report ZIP files are validated for path traversal, encryption, archive size,
|
||||
file count, and decompressed size. Only bounded redacted evidence reaches
|
||||
Copilot; raw archives are deleted and never uploaded as artifacts.
|
||||
|
||||
## Author-feedback lifecycle
|
||||
|
||||
`.github/policies/resourceManagement.yml` closes open issues and pull requests
|
||||
that retain `Needs-Author-Feedback` for seven days without activity.
|
||||
|
||||
- An author comment removes `Needs-Author-Feedback` and returns the item to
|
||||
team triage.
|
||||
- A PR push removes `Needs-Author-Feedback`.
|
||||
- Manual label removal immediately makes the item ineligible for scheduled
|
||||
closure.
|
||||
|
||||
## Cost and safety controls
|
||||
|
||||
- The `small` model alias is limited to five turns and 10 AI credits per run.
|
||||
- A content hash skips unchanged edits and unrelated comments.
|
||||
- Per-user rate limits, daily AI-credit limits, and per-issue concurrency bound
|
||||
repeated execution.
|
||||
- The agent has read-only issue/repository access. A separate validated
|
||||
safe-output job owns comment, label, and duplicate-suggestion writes.
|
||||
|
||||
## Retired automation
|
||||
|
||||
- The GitHub Models-based automatic issue deduplicator is removed.
|
||||
- The GitHub Models-based issue/PR area labeler is removed. This workflow
|
||||
replaces issue labeling only; automatic PR product labeling is intentionally
|
||||
not replaced here.
|
||||
- The Azure Pipelines XAML Styler verification step is removed. The local
|
||||
`.pipelines/applyXamlStyling.ps1` developer tool remains available.
|
||||
|
||||
Run the focused tests with:
|
||||
|
||||
```console
|
||||
python -m unittest discover .github\scripts\issue-triage\tests -v
|
||||
```
|
||||
Reference in New Issue
Block a user