mirror of
https://github.com/microsoft/PowerToys.git
synced 2026-08-29 10:09:43 +02:00
Improve issue triage product-label detection (#49905)
## Summary Issues that put the module in a `[Module]` title prefix (a common PowerToys convention) but omit the bug template's **"Area(s) with issue?"** section were left **Unclassified** with no `Product-*` label — e.g. #49899 *"[Screen Ruler] Settings crashes ..."* got no `Product-Screen Ruler` label despite the title. Root cause: product-label detection was purely deterministic and narrow. `parse_area` (`.github/scripts/issue-triage/issue-context.py`) only read the template area section or a 6-entry keyword map, and the agent prompt instructed the model to copy that candidate verbatim (and send `None` otherwise). The `[Module]` title convention was never consulted. ## Change (two layers) **1. Deterministic title-prefix matching (primary).** Parse the leading `[Module]` bracket(s) in the title and match against existing `Product-*` labels; upgrade the detected area when the body has no area signal. Fully deterministic and auditable — this alone fixes Screen Ruler and every other bracketed title. **2. Constrained AI fallback (secondary).** Expose the repo's `Product-*` labels as `Available product labels` in the deterministic evidence, and allow the agent — **only when the deterministic candidate is `None`** — to select the single best-matching existing label. This is safe because the publisher already validates the agent's `product_label` against the real label set, so the agent can only ever **add a valid existing label**, never invent one or remove/change others. The workflow prompt is `{{#runtime-import}}`-ed from `issue-triage.md`, so the lock file changes only by its `body_hash` (sync check); recompiled with the repo's current gh-aw `v0.84.3` to avoid unrelated version drift. ## Tests New unit tests in `tests/test_issue_context.py`: - `test_title_prefix_maps_to_existing_product_label` - `test_available_product_labels_are_sorted_and_filtered` - `test_prepare_labels_bracketed_title_without_area_section` All 30 tests pass (`python -m unittest tests.test_issue_context`). ## Files - `.github/scripts/issue-triage/issue-context.py` — title-prefix detection, available-label list, wiring - `.github/scripts/issue-triage/tests/test_issue_context.py` — new tests - `.github/workflows/issue-triage.md` — prompt allows constrained fallback - `.github/workflows/issue-triage.lock.yml` — recompiled (`body_hash` only) Generated with the GitHub Copilot CLI. Co-authored-by: niels9001 <niels9001@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 26025067-259e-43e3-9dc7-a9fc4b5ba58b
This commit is contained in:
5
.github/aw/actions-lock.json
vendored
5
.github/aw/actions-lock.json
vendored
@@ -1,5 +1,10 @@
|
||||
{
|
||||
"entries": {
|
||||
"actions/checkout@v6.0.2": {
|
||||
"repo": "actions/checkout",
|
||||
"version": "v6.0.2",
|
||||
"sha": "de0fac2e4500dabe0009e67214ff5f5447ce83dd"
|
||||
},
|
||||
"github/gh-aw-actions/setup@v0.86.2": {
|
||||
"repo": "github/gh-aw-actions/setup",
|
||||
"version": "v0.86.2",
|
||||
|
||||
11
.github/scripts/issue-triage/README.md
vendored
11
.github/scripts/issue-triage/README.md
vendored
@@ -59,8 +59,15 @@ that retain `Needs-Author-Feedback` for seven days without activity.
|
||||
- A content hash skips unchanged edits and unrelated comments.
|
||||
- Per-user rate limits, daily AI-credit limits, and per-issue concurrency bound
|
||||
repeated execution.
|
||||
- The agent has read-only issue/repository access. A separate validated
|
||||
safe-output job owns comment, label, and duplicate-suggestion writes.
|
||||
- The agent has no shell or GitHub API tools. It can only read the checked-out
|
||||
repository and call the structured safe-output tool.
|
||||
- Threat detection fails closed; publication requires an explicit successful
|
||||
detection result.
|
||||
- The publishing job rebuilds evidence from the current issue and accepts only
|
||||
deterministic product-label candidates, duplicate candidates, hashes, and
|
||||
classifications. Stale or manipulated model output fails before any write.
|
||||
- A separate validated safe-output job owns comment, label, and
|
||||
duplicate-suggestion writes.
|
||||
|
||||
## Retired automation
|
||||
|
||||
|
||||
200
.github/scripts/issue-triage/issue-context.py
vendored
200
.github/scripts/issue-triage/issue-context.py
vendored
@@ -146,6 +146,9 @@ class GitHubApi:
|
||||
break
|
||||
return comments
|
||||
|
||||
def get_issue(self, issue_number):
|
||||
return self.request(f"/repos/{self.repository}/issues/{issue_number}")
|
||||
|
||||
def list_labels(self):
|
||||
labels = []
|
||||
for page in range(1, 5):
|
||||
@@ -405,6 +408,55 @@ def product_label(area, labels):
|
||||
return "None"
|
||||
|
||||
|
||||
def title_bracket_segments(title):
|
||||
match = re.match(r"\s*((?:\[[^\]]*\]\s*)+)", title or "")
|
||||
if not match:
|
||||
return []
|
||||
return [
|
||||
segment.strip()
|
||||
for segment in re.findall(r"\[([^\]]*)\]", match.group(1))
|
||||
if segment.strip()
|
||||
]
|
||||
|
||||
|
||||
def title_product_label(title, labels):
|
||||
for segment in title_bracket_segments(title):
|
||||
candidate = product_label(segment, labels)
|
||||
if candidate != "None":
|
||||
return candidate
|
||||
return "None"
|
||||
|
||||
|
||||
def available_product_labels(labels):
|
||||
names = {
|
||||
label.get("name", "") if isinstance(label, dict) else str(label)
|
||||
for label in labels
|
||||
}
|
||||
return sorted(name for name in names if name.startswith("Product-"))
|
||||
|
||||
|
||||
def allowed_product_labels(title, body, labels, deterministic_label):
|
||||
if deterministic_label != "None":
|
||||
return [deterministic_label]
|
||||
|
||||
normalized_text = " " + re.sub(
|
||||
r"[^a-z0-9]+",
|
||||
" ",
|
||||
f"{title or ''}\n{body or ''}".lower(),
|
||||
).strip() + " "
|
||||
candidates = []
|
||||
for label in available_product_labels(labels):
|
||||
product_name = label[len("Product-"):]
|
||||
normalized_name = re.sub(r"[^a-z0-9]+", " ", product_name.lower()).strip()
|
||||
if (
|
||||
normalized_name
|
||||
and normalized_name != "general"
|
||||
and f" {normalized_name} " in normalized_text
|
||||
):
|
||||
candidates.append(label)
|
||||
return candidates[:5]
|
||||
|
||||
|
||||
def build_queries(repository, title, body, label):
|
||||
technical, concepts = search_terms(title, body)
|
||||
scope = f"repo:{repository} is:issue"
|
||||
@@ -604,6 +656,8 @@ def render_context(issue, facts, queries, candidates, digest):
|
||||
f"Issue kind: {facts['issue_kind']}",
|
||||
f"Detected area: {facts['area']}",
|
||||
f"Candidate product label: {facts['product_label']}",
|
||||
"Allowed product label candidates: "
|
||||
+ (", ".join(facts.get("allowed_product_labels", [])) or "None"),
|
||||
f"PowerToys version: {facts['version']}",
|
||||
"Latest stable PowerToys version: "
|
||||
f"{facts.get('latest_stable_version', 'Unavailable')}",
|
||||
@@ -633,40 +687,28 @@ def render_context(issue, facts, queries, candidates, digest):
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def prepare(event, api):
|
||||
issue = event.get("issue")
|
||||
if not isinstance(issue, dict) or not isinstance(issue.get("number"), int):
|
||||
raise ValueError("Event does not contain an issue")
|
||||
if issue.get("pull_request"):
|
||||
write_noop("Pull request comments are handled by the deterministic PR intake workflow")
|
||||
return (
|
||||
"# Deterministic issue evidence\n\nAgent execution was skipped.\n",
|
||||
event,
|
||||
False,
|
||||
)
|
||||
comments = api.list_comments(issue["number"])
|
||||
author = issue.get("user", {}).get("login")
|
||||
report_comment = latest_author_report_comment(comments, author)
|
||||
process, force = should_process(event, report_comment)
|
||||
digest = input_hash(issue, report_comment)
|
||||
if not process:
|
||||
write_noop("No relevant issue content, author report, or maintainer refresh command changed")
|
||||
return (
|
||||
"# Deterministic issue evidence\n\nAgent execution was skipped.\n",
|
||||
event,
|
||||
False,
|
||||
)
|
||||
if not force and existing_input_hash(comments) == digest:
|
||||
write_noop("The triage-relevant issue content has not changed")
|
||||
return (
|
||||
"# Deterministic issue evidence\n\nAgent execution was skipped.\n",
|
||||
event,
|
||||
False,
|
||||
)
|
||||
|
||||
def collect_evidence(issue, report_comment, api):
|
||||
labels = api.list_labels()
|
||||
area = parse_area(issue.get("body", ""), issue.get("title", ""))
|
||||
body = issue.get("body", "")
|
||||
title = issue.get("title", "")
|
||||
area_section = extract_section(body, "Area(s) with issue?")
|
||||
has_explicit_area = bool(
|
||||
area_section
|
||||
and area_section.lower() not in {"_no response_", "no response", "n/a"}
|
||||
)
|
||||
area = parse_area(body, title)
|
||||
desired_label = product_label(area, labels)
|
||||
title_label = title_product_label(title, labels)
|
||||
if title_label != "None" and (desired_label == "None" or not has_explicit_area):
|
||||
desired_label = title_label
|
||||
if area == "Unknown" or not has_explicit_area:
|
||||
area = title_label[len("Product-"):]
|
||||
product_candidates = allowed_product_labels(
|
||||
title,
|
||||
body,
|
||||
labels,
|
||||
desired_label,
|
||||
)
|
||||
issue_for_ranking = dict(issue)
|
||||
issue_for_ranking["labels"] = list(issue.get("labels") or [])
|
||||
if desired_label != "None":
|
||||
@@ -678,6 +720,7 @@ def prepare(event, api):
|
||||
"issue_kind": "BUG" if is_bug_template(issue.get("body", "")) else "OTHER",
|
||||
"area": area,
|
||||
"product_label": desired_label,
|
||||
"allowed_product_labels": product_candidates,
|
||||
"version": reported_version,
|
||||
"latest_stable_version": stable_version,
|
||||
"version_status": version_status(reported_version, stable_version),
|
||||
@@ -689,6 +732,71 @@ def prepare(event, api):
|
||||
),
|
||||
"author_body_status": author_body_status(issue.get("body", "")),
|
||||
}
|
||||
digest = input_hash(issue, report_comment)
|
||||
evidence = {
|
||||
"input_sha256": digest,
|
||||
"suggested_area": area,
|
||||
"candidate_product_label": desired_label,
|
||||
"allowed_product_labels": product_candidates,
|
||||
"powertoys_version": reported_version,
|
||||
"issue_kind": facts["issue_kind"],
|
||||
"reproduction_quality": facts["reproduction_quality"],
|
||||
"bug_report_requirement": facts["bug_report_requirement"],
|
||||
"duplicate_candidate_numbers": [
|
||||
candidate["number"] for candidate in candidates
|
||||
],
|
||||
"issue_author": issue.get("user", {}).get("login"),
|
||||
"current_labels": [
|
||||
label.get("name", "") if isinstance(label, dict) else str(label)
|
||||
for label in issue.get("labels", [])
|
||||
],
|
||||
}
|
||||
return facts, queries, candidates, evidence
|
||||
|
||||
|
||||
def prepare_with_evidence(event, api, force_evidence=False):
|
||||
issue = event.get("issue")
|
||||
if not isinstance(issue, dict) or not isinstance(issue.get("number"), int):
|
||||
raise ValueError("Event does not contain an issue")
|
||||
if issue.get("pull_request"):
|
||||
write_noop("Pull request comments are handled by the deterministic PR intake workflow")
|
||||
return (
|
||||
"# Deterministic issue evidence\n\nAgent execution was skipped.\n",
|
||||
event,
|
||||
False,
|
||||
None,
|
||||
)
|
||||
if force_evidence:
|
||||
issue = api.get_issue(issue["number"])
|
||||
event = dict(event)
|
||||
event["issue"] = issue
|
||||
comments = api.list_comments(issue["number"])
|
||||
author = issue.get("user", {}).get("login")
|
||||
report_comment = latest_author_report_comment(comments, author)
|
||||
process, force = should_process(event, report_comment)
|
||||
digest = input_hash(issue, report_comment)
|
||||
if not force_evidence and not process:
|
||||
write_noop("No relevant issue content, author report, or maintainer refresh command changed")
|
||||
return (
|
||||
"# Deterministic issue evidence\n\nAgent execution was skipped.\n",
|
||||
event,
|
||||
False,
|
||||
None,
|
||||
)
|
||||
if not force_evidence and not force and existing_input_hash(comments) == digest:
|
||||
write_noop("The triage-relevant issue content has not changed")
|
||||
return (
|
||||
"# Deterministic issue evidence\n\nAgent execution was skipped.\n",
|
||||
event,
|
||||
False,
|
||||
None,
|
||||
)
|
||||
|
||||
facts, queries, candidates, evidence = collect_evidence(
|
||||
issue,
|
||||
report_comment,
|
||||
api,
|
||||
)
|
||||
normalized_event = dict(event)
|
||||
if report_comment:
|
||||
normalized_event["comment"] = report_comment
|
||||
@@ -696,24 +804,39 @@ def prepare(event, api):
|
||||
render_context(issue, facts, queries, candidates, digest),
|
||||
normalized_event,
|
||||
True,
|
||||
evidence,
|
||||
)
|
||||
|
||||
|
||||
def prepare(event, api):
|
||||
context, normalized_event, should_process_event, _ = prepare_with_evidence(
|
||||
event,
|
||||
api,
|
||||
)
|
||||
return context, normalized_event, should_process_event
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) != 4:
|
||||
if len(sys.argv) not in {4, 5}:
|
||||
print(
|
||||
"Usage: issue-context.py EVENT_JSON OUTPUT_MARKDOWN NORMALIZED_EVENT_JSON",
|
||||
"Usage: issue-context.py EVENT_JSON OUTPUT_MARKDOWN "
|
||||
"NORMALIZED_EVENT_JSON [EVIDENCE_JSON]",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 2
|
||||
event_path, context_path, normalized_event_path = sys.argv[1:4]
|
||||
evidence_path = sys.argv[4] if len(sys.argv) == 5 else None
|
||||
with open(event_path, "r", encoding="utf-8") as event_file:
|
||||
event = json.load(event_file)
|
||||
api = GitHubApi(
|
||||
os.environ.get("GITHUB_TOKEN") or os.environ.get("GH_TOKEN"),
|
||||
os.environ.get("GITHUB_REPOSITORY"),
|
||||
)
|
||||
context, normalized_event, should_process_event = prepare(event, api)
|
||||
context, normalized_event, should_process_event, evidence = prepare_with_evidence(
|
||||
event,
|
||||
api,
|
||||
force_evidence=os.environ.get("ISSUE_TRIAGE_FORCE_EVIDENCE") == "true",
|
||||
)
|
||||
for output_path, payload in (
|
||||
(context_path, context),
|
||||
(normalized_event_path, json.dumps(normalized_event, ensure_ascii=True)),
|
||||
@@ -723,6 +846,13 @@ def main():
|
||||
output_file.write(payload)
|
||||
if not payload.endswith("\n"):
|
||||
output_file.write("\n")
|
||||
if evidence_path:
|
||||
if evidence is None:
|
||||
raise ValueError("Deterministic evidence was not generated")
|
||||
os.makedirs(os.path.dirname(os.path.abspath(evidence_path)), exist_ok=True)
|
||||
with open(evidence_path, "w", encoding="utf-8", newline="\n") as output_file:
|
||||
json.dump(evidence, output_file, ensure_ascii=True, sort_keys=True)
|
||||
output_file.write("\n")
|
||||
write_step_output("should_process", "true" if should_process_event else "false")
|
||||
print("Prepared deterministic issue evidence and duplicate candidates")
|
||||
return 0
|
||||
|
||||
@@ -45,9 +45,16 @@ _No response_
|
||||
class FakeApi:
|
||||
repository = "owner/repo"
|
||||
|
||||
def __init__(self, results=None, comments=None, latest_release=None):
|
||||
def __init__(
|
||||
self,
|
||||
results=None,
|
||||
comments=None,
|
||||
latest_release=None,
|
||||
current_issue=None,
|
||||
):
|
||||
self.results = results or []
|
||||
self.comments = comments or []
|
||||
self.current_issue = current_issue
|
||||
self.latest_release = latest_release or {
|
||||
"tag_name": "v0.100.2",
|
||||
"prerelease": False,
|
||||
@@ -57,6 +64,9 @@ class FakeApi:
|
||||
def list_comments(self, _issue_number):
|
||||
return self.comments
|
||||
|
||||
def get_issue(self, _issue_number):
|
||||
return self.current_issue
|
||||
|
||||
def list_labels(self):
|
||||
return [{"name": "Product-Keyboard Manager"}]
|
||||
|
||||
@@ -272,6 +282,130 @@ class IssueContextTests(unittest.TestCase):
|
||||
"Product-General",
|
||||
)
|
||||
|
||||
def test_title_prefix_maps_to_existing_product_label(self):
|
||||
labels = [{"name": "Product-Screen Ruler"}, {"name": "Product-FancyZones"}]
|
||||
self.assertEqual(
|
||||
CONTEXT.title_bracket_segments("[Screen Ruler] Settings crash"),
|
||||
["Screen Ruler"],
|
||||
)
|
||||
self.assertEqual(
|
||||
CONTEXT.title_bracket_segments("[Screen Ruler][Settings] crash"),
|
||||
["Screen Ruler", "Settings"],
|
||||
)
|
||||
self.assertEqual(CONTEXT.title_bracket_segments("No brackets here"), [])
|
||||
self.assertEqual(
|
||||
CONTEXT.title_product_label("[Screen Ruler] Settings crash", labels),
|
||||
"Product-Screen Ruler",
|
||||
)
|
||||
self.assertEqual(
|
||||
CONTEXT.title_product_label("[Bug] something broke", labels),
|
||||
"None",
|
||||
)
|
||||
|
||||
def test_available_product_labels_are_sorted_and_filtered(self):
|
||||
labels = [
|
||||
{"name": "Product-Screen Ruler"},
|
||||
{"name": "Needs-Triage"},
|
||||
{"name": "Product-Awake"},
|
||||
]
|
||||
self.assertEqual(
|
||||
CONTEXT.available_product_labels(labels),
|
||||
["Product-Awake", "Product-Screen Ruler"],
|
||||
)
|
||||
|
||||
def test_allowed_product_labels_require_explicit_issue_text(self):
|
||||
labels = [
|
||||
{"name": "Product-Screen Ruler"},
|
||||
{"name": "Product-FancyZones"},
|
||||
{"name": "Product-General"},
|
||||
]
|
||||
self.assertEqual(
|
||||
CONTEXT.allowed_product_labels(
|
||||
"Ruler settings problem",
|
||||
"The Screen Ruler overlay is misplaced.",
|
||||
labels,
|
||||
"None",
|
||||
),
|
||||
["Product-Screen Ruler"],
|
||||
)
|
||||
|
||||
def test_prepare_labels_bracketed_title_without_area_section(self):
|
||||
issue = {
|
||||
"number": 42,
|
||||
"title": "[Screen Ruler] Settings crashes on legacy units value",
|
||||
"body": (
|
||||
"## Description\n\nSettings crashes when leaving the Screen "
|
||||
"Ruler page with a legacy measurement-unit value."
|
||||
),
|
||||
"user": {"login": "alice"},
|
||||
"labels": [],
|
||||
}
|
||||
|
||||
class ScreenRulerApi(FakeApi):
|
||||
def list_labels(self):
|
||||
return [{"name": "Product-Screen Ruler"}]
|
||||
|
||||
context, _, should_process = CONTEXT.prepare(
|
||||
{"action": "opened", "issue": issue},
|
||||
ScreenRulerApi(),
|
||||
)
|
||||
self.assertTrue(should_process)
|
||||
self.assertIn("Detected area: Screen Ruler", context)
|
||||
self.assertIn("Candidate product label: Product-Screen Ruler", context)
|
||||
self.assertIn(
|
||||
"Allowed product label candidates: Product-Screen Ruler",
|
||||
context,
|
||||
)
|
||||
|
||||
def test_title_prefix_overrides_inferred_area_without_template_selection(self):
|
||||
issue = {
|
||||
"number": 42,
|
||||
"title": "[Screen Ruler] FancyZones-style overlay is misplaced",
|
||||
"body": "The FancyZones overlay comparison shows the ruler is misplaced.",
|
||||
"user": {"login": "alice"},
|
||||
"labels": [],
|
||||
}
|
||||
|
||||
class ProductApi(FakeApi):
|
||||
def list_labels(self):
|
||||
return [
|
||||
{"name": "Product-FancyZones"},
|
||||
{"name": "Product-Screen Ruler"},
|
||||
]
|
||||
|
||||
context, _, _ = CONTEXT.prepare(
|
||||
{"action": "opened", "issue": issue},
|
||||
ProductApi(),
|
||||
)
|
||||
self.assertIn("Detected area: Screen Ruler", context)
|
||||
self.assertIn("Candidate product label: Product-Screen Ruler", context)
|
||||
|
||||
def test_explicit_template_area_takes_precedence_over_title_prefix(self):
|
||||
issue = {
|
||||
"number": 42,
|
||||
"title": "[Screen Ruler] FancyZones editor issue",
|
||||
"body": (
|
||||
"### Area(s) with issue?\n\nFancyZones\n\n"
|
||||
"### Description\n\nThe FancyZones editor is misplaced."
|
||||
),
|
||||
"user": {"login": "alice"},
|
||||
"labels": [],
|
||||
}
|
||||
|
||||
class ProductApi(FakeApi):
|
||||
def list_labels(self):
|
||||
return [
|
||||
{"name": "Product-FancyZones"},
|
||||
{"name": "Product-Screen Ruler"},
|
||||
]
|
||||
|
||||
context, _, _ = CONTEXT.prepare(
|
||||
{"action": "opened", "issue": issue},
|
||||
ProductApi(),
|
||||
)
|
||||
self.assertIn("Detected area: FancyZones", context)
|
||||
self.assertIn("Candidate product label: Product-FancyZones", context)
|
||||
|
||||
def test_context_redacts_report_attachment_urls(self):
|
||||
context = CONTEXT.render_context(
|
||||
{
|
||||
@@ -413,6 +547,38 @@ class IssueContextTests(unittest.TestCase):
|
||||
self.assertEqual(normalized["issue"]["number"], 10)
|
||||
self.assertTrue(should_process)
|
||||
|
||||
def test_force_evidence_uses_current_issue_and_emits_allowlists(self):
|
||||
stale_issue = {
|
||||
"number": 10,
|
||||
"title": "Old title",
|
||||
"body": "",
|
||||
"user": {"login": "alice"},
|
||||
"labels": [],
|
||||
}
|
||||
current_issue = {
|
||||
"number": 10,
|
||||
"title": "Screen Ruler overlay problem",
|
||||
"body": "The Screen Ruler overlay is misplaced.",
|
||||
"user": {"login": "alice"},
|
||||
"labels": [],
|
||||
}
|
||||
|
||||
class VerificationApi(FakeApi):
|
||||
def list_labels(self):
|
||||
return [{"name": "Product-Screen Ruler"}]
|
||||
|
||||
_, normalized, should_process, evidence = CONTEXT.prepare_with_evidence(
|
||||
{"action": "edited", "issue": stale_issue},
|
||||
VerificationApi(current_issue=current_issue),
|
||||
force_evidence=True,
|
||||
)
|
||||
self.assertTrue(should_process)
|
||||
self.assertEqual(normalized["issue"]["title"], current_issue["title"])
|
||||
self.assertEqual(
|
||||
evidence["allowed_product_labels"],
|
||||
["Product-Screen Ruler"],
|
||||
)
|
||||
|
||||
def test_candidate_retrieval_only_returns_older_issues(self):
|
||||
issue = {
|
||||
"number": 10,
|
||||
|
||||
69
.github/scripts/issue-triage/tests/test_verify_agent_output.py
vendored
Normal file
69
.github/scripts/issue-triage/tests/test_verify_agent_output.py
vendored
Normal file
@@ -0,0 +1,69 @@
|
||||
import importlib.util
|
||||
import pathlib
|
||||
import unittest
|
||||
|
||||
|
||||
SCRIPT = pathlib.Path(__file__).parents[1] / "verify-agent-output.py"
|
||||
SPEC = importlib.util.spec_from_file_location("verify_agent_output", SCRIPT)
|
||||
VERIFY = importlib.util.module_from_spec(SPEC)
|
||||
SPEC.loader.exec_module(VERIFY)
|
||||
|
||||
|
||||
class VerifyAgentOutputTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.evidence = {
|
||||
"input_sha256": "a" * 64,
|
||||
"suggested_area": "Screen Ruler",
|
||||
"candidate_product_label": "None",
|
||||
"allowed_product_labels": ["Product-Screen Ruler"],
|
||||
"powertoys_version": "0.100.0",
|
||||
"issue_kind": "BUG",
|
||||
"reproduction_quality": "SUFFICIENT",
|
||||
"bug_report_requirement": "OPTIONAL",
|
||||
"duplicate_candidate_numbers": [10, 20],
|
||||
"issue_author": "alice",
|
||||
"current_labels": [],
|
||||
}
|
||||
self.item = {
|
||||
"input_sha256": "a" * 64,
|
||||
"suggested_area": "Screen Ruler",
|
||||
"product_label": "Product-Screen Ruler",
|
||||
"powertoys_version": "0.100.0",
|
||||
"issue_kind": "BUG",
|
||||
"reproduction_quality": "SUFFICIENT",
|
||||
"bug_report_requirement": "OPTIONAL",
|
||||
"bug_report_status": "NOT_FOUND",
|
||||
"duplicate_candidates_json": (
|
||||
'[{"number":10,"reason":"Same failure","confidence":"HIGH"}]'
|
||||
),
|
||||
}
|
||||
|
||||
def test_accepts_only_deterministic_candidates(self):
|
||||
verified = VERIFY.verify(
|
||||
self.item,
|
||||
self.evidence,
|
||||
"Status: NOT_FOUND\n",
|
||||
)
|
||||
self.assertEqual(verified["product_label"], "Product-Screen Ruler")
|
||||
self.assertEqual(verified["requested_duplicate_numbers"], [10])
|
||||
|
||||
def test_rejects_product_label_outside_candidate_set(self):
|
||||
self.item["product_label"] = "Product-FancyZones"
|
||||
with self.assertRaisesRegex(ValueError, "candidate set"):
|
||||
VERIFY.verify(self.item, self.evidence, "Status: NOT_FOUND\n")
|
||||
|
||||
def test_rejects_duplicate_outside_retrieval_results(self):
|
||||
self.item["duplicate_candidates_json"] = (
|
||||
'[{"number":99,"reason":"Injected","confidence":"HIGH"}]'
|
||||
)
|
||||
with self.assertRaisesRegex(ValueError, "retrieval results"):
|
||||
VERIFY.verify(self.item, self.evidence, "Status: NOT_FOUND\n")
|
||||
|
||||
def test_rejects_stale_input_hash(self):
|
||||
self.item["input_sha256"] = "b" * 64
|
||||
with self.assertRaisesRegex(ValueError, "current issue content"):
|
||||
VERIFY.verify(self.item, self.evidence, "Status: NOT_FOUND\n")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
133
.github/scripts/issue-triage/verify-agent-output.py
vendored
Normal file
133
.github/scripts/issue-triage/verify-agent-output.py
vendored
Normal file
@@ -0,0 +1,133 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
|
||||
|
||||
def load_triage_item(agent_output):
|
||||
items = agent_output.get("items")
|
||||
if not isinstance(items, list):
|
||||
raise ValueError("Agent output does not contain an items array")
|
||||
matches = [
|
||||
item
|
||||
for item in items
|
||||
if isinstance(item, dict) and item.get("type") == "publish_triage_summary"
|
||||
]
|
||||
if len(matches) != 1:
|
||||
raise ValueError("Agent output must contain exactly one triage summary")
|
||||
return matches[0]
|
||||
|
||||
|
||||
def exact_string(item, name, expected):
|
||||
actual = item.get(name)
|
||||
if actual != expected:
|
||||
raise ValueError(f"{name} does not match deterministic evidence")
|
||||
return actual
|
||||
|
||||
|
||||
def verify(item, evidence, bug_report_context):
|
||||
input_sha256 = str(item.get("input_sha256") or "").lower()
|
||||
if input_sha256 != evidence.get("input_sha256"):
|
||||
raise ValueError("input_sha256 does not match current issue content")
|
||||
|
||||
suggested_area = exact_string(
|
||||
item,
|
||||
"suggested_area",
|
||||
evidence.get("suggested_area"),
|
||||
)
|
||||
powertoys_version = exact_string(
|
||||
item,
|
||||
"powertoys_version",
|
||||
evidence.get("powertoys_version"),
|
||||
)
|
||||
issue_kind = exact_string(item, "issue_kind", evidence.get("issue_kind"))
|
||||
reproduction_quality = exact_string(
|
||||
item,
|
||||
"reproduction_quality",
|
||||
evidence.get("reproduction_quality"),
|
||||
)
|
||||
bug_report_requirement = exact_string(
|
||||
item,
|
||||
"bug_report_requirement",
|
||||
evidence.get("bug_report_requirement"),
|
||||
)
|
||||
|
||||
product_label = item.get("product_label")
|
||||
allowed_labels = set(evidence.get("allowed_product_labels") or [])
|
||||
deterministic_label = evidence.get("candidate_product_label")
|
||||
if deterministic_label != "None":
|
||||
if product_label != deterministic_label:
|
||||
raise ValueError("product_label does not match deterministic evidence")
|
||||
elif product_label != "None" and product_label not in allowed_labels:
|
||||
raise ValueError("product_label is outside the deterministic candidate set")
|
||||
|
||||
try:
|
||||
duplicate_candidates = json.loads(item.get("duplicate_candidates_json"))
|
||||
except (TypeError, json.JSONDecodeError) as error:
|
||||
raise ValueError("duplicate_candidates_json is invalid") from error
|
||||
if not isinstance(duplicate_candidates, list) or len(duplicate_candidates) > 5:
|
||||
raise ValueError("duplicate_candidates_json must contain at most five items")
|
||||
allowed_duplicates = set(evidence.get("duplicate_candidate_numbers") or [])
|
||||
requested_duplicate_numbers = []
|
||||
for candidate in duplicate_candidates:
|
||||
if not isinstance(candidate, dict) or not isinstance(candidate.get("number"), int):
|
||||
raise ValueError("Each duplicate candidate must contain an integer number")
|
||||
number = candidate["number"]
|
||||
if number not in allowed_duplicates:
|
||||
raise ValueError("Duplicate candidate is outside deterministic retrieval results")
|
||||
requested_duplicate_numbers.append(number)
|
||||
|
||||
status_match = re.search(
|
||||
r"^Status: (ANALYZED|NOT_FOUND|REJECTED)$",
|
||||
bug_report_context,
|
||||
re.MULTILINE,
|
||||
)
|
||||
bug_report_status = (
|
||||
status_match.group(1)
|
||||
if issue_kind == "BUG" and status_match
|
||||
else "NOT_APPLICABLE"
|
||||
)
|
||||
if item.get("bug_report_status") != bug_report_status:
|
||||
raise ValueError("bug_report_status does not match sanitized diagnostics")
|
||||
|
||||
return {
|
||||
"input_sha256": input_sha256,
|
||||
"suggested_area": suggested_area,
|
||||
"product_label": product_label,
|
||||
"powertoys_version": powertoys_version,
|
||||
"issue_kind": issue_kind,
|
||||
"reproduction_quality": reproduction_quality,
|
||||
"bug_report_requirement": bug_report_requirement,
|
||||
"bug_report_status": bug_report_status,
|
||||
"requested_duplicate_numbers": requested_duplicate_numbers,
|
||||
"issue_author": evidence.get("issue_author"),
|
||||
"current_labels": evidence.get("current_labels") or [],
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) != 5:
|
||||
print(
|
||||
"Usage: verify-agent-output.py AGENT_OUTPUT EVIDENCE_JSON "
|
||||
"BUG_REPORT_CONTEXT VERIFIED_OUTPUT",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 2
|
||||
agent_output_path, evidence_path, bug_context_path, output_path = sys.argv[1:]
|
||||
with open(agent_output_path, "r", encoding="utf-8") as input_file:
|
||||
item = load_triage_item(json.load(input_file))
|
||||
with open(evidence_path, "r", encoding="utf-8") as input_file:
|
||||
evidence = json.load(input_file)
|
||||
with open(bug_context_path, "r", encoding="utf-8") as input_file:
|
||||
bug_report_context = input_file.read()
|
||||
verified = verify(item, evidence, bug_report_context)
|
||||
with open(output_path, "w", encoding="utf-8", newline="\n") as output_file:
|
||||
json.dump(verified, output_file, ensure_ascii=True, sort_keys=True)
|
||||
output_file.write("\n")
|
||||
print("Verified agent output against fresh deterministic evidence")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
197
.github/workflows/issue-triage.lock.yml
generated
vendored
197
.github/workflows/issue-triage.lock.yml
generated
vendored
@@ -1,5 +1,5 @@
|
||||
# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"3d7f2e38745aec34c92ab7cbe75ccc0c3741e4409c2bc771f611cae6dc053ef6","body_hash":"1bbf2e8b3d952c071c04a97b22e7dd8325e4d9caa843355882e192df85464be5","compiler_version":"v0.86.2","strict":true,"agent_id":"copilot","agent_model":"small","engine_versions":{"copilot":"1.0.79"}}
|
||||
# gh-aw-manifest: {"version":1,"secrets":["GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/cache/restore","sha":"55cc8345863c7cc4c66a329aec7e433d2d1c52a9","version":"v6.1.0"},{"repo":"actions/cache/save","sha":"55cc8345863c7cc4c66a329aec7e433d2d1c52a9","version":"v6.1.0"},{"repo":"actions/checkout","sha":"3d3c42e5aac5ba805825da76410c181273ba90b1","version":"v7.0.1"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"820762786026740c76f36085b0efc47a31fe5020","version":"v7.0.0"},{"repo":"actions/setup-python","sha":"5fda3b95a4ea91299a34e894583c3862153e4b97","version":"v7.0.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"6aab9e5b5c91c615506061f09bedd81a23babe3c","version":"v0.86.2"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.44","digest":"sha256:0d727725c737b58c7bdf51f640cffb928385ec46517e0917c7f1a02f1bada8b4","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.44@sha256:0d727725c737b58c7bdf51f640cffb928385ec46517e0917c7f1a02f1bada8b4"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.44","digest":"sha256:b50fbadba138f6e9aba94aca09711335c489bb3b15861220cb66f6092e042dc7","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.44@sha256:b50fbadba138f6e9aba94aca09711335c489bb3b15861220cb66f6092e042dc7"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.44","digest":"sha256:83e48bbe12c634be8c228a576832fe45f66c529ac3659db92bddbcf2eeb6d627","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.44@sha256:83e48bbe12c634be8c228a576832fe45f66c529ac3659db92bddbcf2eeb6d627"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.4.9","digest":"sha256:e5a1569aeaf41820fa7bdee3e94468cae448133cdbf00119ad24f5b74db1ab9f","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.4.9@sha256:e5a1569aeaf41820fa7bdee3e94468cae448133cdbf00119ad24f5b74db1ab9f"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:0d9f1fb5fd6610c0ac1f5194a38e45a8a1e81f8a390d5142d8e4e6f26a4b3196","pinned_image":"ghcr.io/github/gh-aw-node@sha256:0d9f1fb5fd6610c0ac1f5194a38e45a8a1e81f8a390d5142d8e4e6f26a4b3196"},{"image":"ghcr.io/github/github-mcp-server:v1.9.0","digest":"sha256:881b53d6f75f69bdbc1b5b10fc2f1361717c19054143b3a8529fb5c32061a50e","pinned_image":"ghcr.io/github/github-mcp-server:v1.9.0@sha256:881b53d6f75f69bdbc1b5b10fc2f1361717c19054143b3a8529fb5c32061a50e"}]}
|
||||
# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"e9a454298d120d615b774f8a43454eb1cd0fd2851f93470b6b4098a9afceed23","body_hash":"45ed48558c4efc2979f9f25a79d4c69805e91a16b47b1c9297bdccf5ca883163","compiler_version":"v0.86.2","strict":true,"agent_id":"copilot","agent_model":"small","engine_versions":{"copilot":"1.0.79"}}
|
||||
# gh-aw-manifest: {"version":1,"secrets":["GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/cache/restore","sha":"55cc8345863c7cc4c66a329aec7e433d2d1c52a9","version":"v6.1.0"},{"repo":"actions/cache/save","sha":"55cc8345863c7cc4c66a329aec7e433d2d1c52a9","version":"v6.1.0"},{"repo":"actions/checkout","sha":"3d3c42e5aac5ba805825da76410c181273ba90b1","version":"v7.0.1"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"820762786026740c76f36085b0efc47a31fe5020","version":"v7.0.0"},{"repo":"actions/setup-python","sha":"5fda3b95a4ea91299a34e894583c3862153e4b97","version":"v7.0.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"6aab9e5b5c91c615506061f09bedd81a23babe3c","version":"v0.86.2"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.44","digest":"sha256:0d727725c737b58c7bdf51f640cffb928385ec46517e0917c7f1a02f1bada8b4","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.44@sha256:0d727725c737b58c7bdf51f640cffb928385ec46517e0917c7f1a02f1bada8b4"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.44","digest":"sha256:b50fbadba138f6e9aba94aca09711335c489bb3b15861220cb66f6092e042dc7","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.44@sha256:b50fbadba138f6e9aba94aca09711335c489bb3b15861220cb66f6092e042dc7"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.44","digest":"sha256:83e48bbe12c634be8c228a576832fe45f66c529ac3659db92bddbcf2eeb6d627","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.44@sha256:83e48bbe12c634be8c228a576832fe45f66c529ac3659db92bddbcf2eeb6d627"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.4.9","digest":"sha256:e5a1569aeaf41820fa7bdee3e94468cae448133cdbf00119ad24f5b74db1ab9f","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.4.9@sha256:e5a1569aeaf41820fa7bdee3e94468cae448133cdbf00119ad24f5b74db1ab9f"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:0d9f1fb5fd6610c0ac1f5194a38e45a8a1e81f8a390d5142d8e4e6f26a4b3196","pinned_image":"ghcr.io/github/gh-aw-node@sha256:0d9f1fb5fd6610c0ac1f5194a38e45a8a1e81f8a390d5142d8e4e6f26a4b3196"}]}
|
||||
# This file was automatically generated by gh-aw (v0.86.2). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md
|
||||
#
|
||||
# ___ _ _
|
||||
@@ -36,7 +36,6 @@
|
||||
# - actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
# - actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
|
||||
# - actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
|
||||
# - actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 (source v9)
|
||||
# - actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
|
||||
# - actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
|
||||
# - actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
||||
@@ -48,7 +47,6 @@
|
||||
# - ghcr.io/github/gh-aw-firewall/squid:0.27.44@sha256:83e48bbe12c634be8c228a576832fe45f66c529ac3659db92bddbcf2eeb6d627
|
||||
# - ghcr.io/github/gh-aw-mcpg:v0.4.9@sha256:e5a1569aeaf41820fa7bdee3e94468cae448133cdbf00119ad24f5b74db1ab9f
|
||||
# - ghcr.io/github/gh-aw-node@sha256:0d9f1fb5fd6610c0ac1f5194a38e45a8a1e81f8a390d5142d8e4e6f26a4b3196
|
||||
# - ghcr.io/github/github-mcp-server:v1.9.0@sha256:881b53d6f75f69bdbc1b5b10fc2f1361717c19054143b3a8529fb5c32061a50e
|
||||
|
||||
name: "AI Issue Triage"
|
||||
on:
|
||||
@@ -255,22 +253,13 @@ jobs:
|
||||
GH_AW_ACTIONS_DIR: ${{ runner.temp }}/gh-aw/actions
|
||||
GH_AW_PROMPT: ${{ runner.temp }}/gh-aw/aw-prompts/prompt.txt
|
||||
GH_AW_SAFE_OUTPUTS: ${{ runner.temp }}/gh-aw/safeoutputs/outputs.jsonl
|
||||
GH_AW_PROMPT_CONFIG: "{\"items\":[{\"content_env\":\"GH_AW_PROMPT_CONTENT_0000\"},{\"file\":\"xpia.md\"},{\"file\":\"temp_folder_prompt.md\"},{\"file\":\"markdown.md\"},{\"file\":\"safe_outputs_prompt.md\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0001\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0002\"},{\"file\":\"mcp_cli_tools_prompt.md\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0003\"},{\"file\":\"github_mcp_tools_with_safeoutputs_prompt.md\"},{\"file\":\"pr_context_prompt.md\",\"condition_env\":\"GH_AW_INCLUDE_PR_CONTEXT\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0004\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0005\"}]}"
|
||||
GH_AW_EXPR_1A3A194A: ${{ github.event.discussion.number || (fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_type == 'discussion' && fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_number) }}
|
||||
GH_AW_EXPR_463A214A: ${{ github.event.pull_request.number || (fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_type == 'pull_request' && fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_number) }}
|
||||
GH_AW_EXPR_802A9F6A: ${{ github.event.issue.number || (fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_type == 'issue' && fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_number) }}
|
||||
GH_AW_EXPR_FF1D34CE: ${{ github.event.comment.id || fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').comment_id }}
|
||||
GH_AW_GITHUB_ACTOR: ${{ github.actor }}
|
||||
GH_AW_GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GH_AW_GITHUB_RUN_ID: ${{ github.run_id }}
|
||||
GH_AW_GITHUB_WORKSPACE: ${{ github.workspace }}
|
||||
GH_AW_PROMPT_CONFIG: "{\"items\":[{\"content_env\":\"GH_AW_PROMPT_CONTENT_0000\"},{\"file\":\"xpia.md\"},{\"file\":\"temp_folder_prompt.md\"},{\"file\":\"markdown.md\"},{\"file\":\"safe_outputs_prompt.md\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0001\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0002\"},{\"file\":\"mcp_cli_tools_prompt.md\"},{\"file\":\"pr_context_prompt.md\",\"condition_env\":\"GH_AW_INCLUDE_PR_CONTEXT\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0003\"},{\"content_env\":\"GH_AW_PROMPT_CONTENT_0004\"}]}"
|
||||
GH_AW_INCLUDE_PR_CONTEXT: ${{ (github.event_name == 'issue_comment' && github.event.issue.pull_request != null) || github.event_name == 'pull_request_review_comment' || github.event_name == 'pull_request_review' }}
|
||||
GH_AW_PROMPT_CONTENT_0000: "<system>\n"
|
||||
GH_AW_PROMPT_CONTENT_0001: "<safe-output-tools>\nTools: missing_tool, missing_data, noop, publish_triage_summary\n"
|
||||
GH_AW_PROMPT_CONTENT_0002: "</safe-output-tools>\n"
|
||||
GH_AW_PROMPT_CONTENT_0003: "<github-context>\nThe following GitHub context information is available for this workflow:\n{{#if github.actor}}\n- **actor**: __GH_AW_GITHUB_ACTOR__\n{{/if}}\n{{#if github.repository}}\n- **repository**: __GH_AW_GITHUB_REPOSITORY__\n{{/if}}\n{{#if github.workspace}}\n- **workspace**: __GH_AW_GITHUB_WORKSPACE__\n{{/if}}\n{{#if github.event.issue.number || (github.aw.context.item_type == 'issue' && github.aw.context.item_number)}}\n- **issue-number**: #__GH_AW_EXPR_802A9F6A__\n{{/if}}\n{{#if github.event.discussion.number || (github.aw.context.item_type == 'discussion' && github.aw.context.item_number)}}\n- **discussion-number**: #__GH_AW_EXPR_1A3A194A__\n{{/if}}\n{{#if github.event.pull_request.number || (github.aw.context.item_type == 'pull_request' && github.aw.context.item_number)}}\n- **pull-request-number**: #__GH_AW_EXPR_463A214A__\n{{/if}}\n{{#if github.event.comment.id || github.aw.context.comment_id}}\n- **comment-id**: __GH_AW_EXPR_FF1D34CE__\n{{/if}}\n{{#if github.run_id}}\n- **workflow-run-id**: __GH_AW_GITHUB_RUN_ID__\n{{/if}}\n</github-context>\n\n"
|
||||
GH_AW_PROMPT_CONTENT_0004: "</system>\n"
|
||||
GH_AW_PROMPT_CONTENT_0005: "{{#runtime-import .github/workflows/issue-triage.md}}\n"
|
||||
GH_AW_PROMPT_CONTENT_0003: "</system>\n"
|
||||
GH_AW_PROMPT_CONTENT_0004: "{{#runtime-import .github/workflows/issue-triage.md}}\n"
|
||||
with:
|
||||
script: |
|
||||
const { setupGlobals } = require(process.env.GH_AW_ACTIONS_DIR + '/setup_globals.cjs');
|
||||
@@ -292,16 +281,8 @@ jobs:
|
||||
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
|
||||
env:
|
||||
GH_AW_PROMPT: ${{ runner.temp }}/gh-aw/aw-prompts/prompt.txt
|
||||
GH_AW_EXPR_1A3A194A: ${{ github.event.discussion.number || (fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_type == 'discussion' && fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_number) }}
|
||||
GH_AW_EXPR_463A214A: ${{ github.event.pull_request.number || (fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_type == 'pull_request' && fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_number) }}
|
||||
GH_AW_EXPR_802A9F6A: ${{ github.event.issue.number || (fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_type == 'issue' && fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_number) }}
|
||||
GH_AW_EXPR_FF1D34CE: ${{ github.event.comment.id || fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').comment_id }}
|
||||
GH_AW_GITHUB_ACTOR: ${{ github.actor }}
|
||||
GH_AW_GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GH_AW_GITHUB_RUN_ID: ${{ github.run_id }}
|
||||
GH_AW_GITHUB_WORKSPACE: ${{ github.workspace }}
|
||||
GH_AW_INCLUDE_PR_CONTEXT: ${{ (github.event_name == 'issue_comment' && github.event.issue.pull_request != null) || github.event_name == 'pull_request_review_comment' || github.event_name == 'pull_request_review' }}
|
||||
GH_AW_MCP_CLI_SERVERS_LIST: "- `github` — run `github --help` to see available tools\n- `safeoutputs` — run `safeoutputs --help` to see available tools"
|
||||
GH_AW_MCP_CLI_SERVERS_LIST: '- `safeoutputs` — run `safeoutputs --help` to see available tools'
|
||||
GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_ACTIVATED: ${{ needs.pre_activation.outputs.activated }}
|
||||
with:
|
||||
script: |
|
||||
@@ -314,14 +295,6 @@ jobs:
|
||||
return await substitutePlaceholders({
|
||||
file: process.env.GH_AW_PROMPT,
|
||||
substitutions: {
|
||||
GH_AW_EXPR_1A3A194A: process.env.GH_AW_EXPR_1A3A194A,
|
||||
GH_AW_EXPR_463A214A: process.env.GH_AW_EXPR_463A214A,
|
||||
GH_AW_EXPR_802A9F6A: process.env.GH_AW_EXPR_802A9F6A,
|
||||
GH_AW_EXPR_FF1D34CE: process.env.GH_AW_EXPR_FF1D34CE,
|
||||
GH_AW_GITHUB_ACTOR: process.env.GH_AW_GITHUB_ACTOR,
|
||||
GH_AW_GITHUB_REPOSITORY: process.env.GH_AW_GITHUB_REPOSITORY,
|
||||
GH_AW_GITHUB_RUN_ID: process.env.GH_AW_GITHUB_RUN_ID,
|
||||
GH_AW_GITHUB_WORKSPACE: process.env.GH_AW_GITHUB_WORKSPACE,
|
||||
GH_AW_INCLUDE_PR_CONTEXT: process.env.GH_AW_INCLUDE_PR_CONTEXT,
|
||||
GH_AW_MCP_CLI_SERVERS_LIST: process.env.GH_AW_MCP_CLI_SERVERS_LIST,
|
||||
GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_ACTIVATED: process.env.GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_ACTIVATED
|
||||
@@ -480,16 +453,6 @@ jobs:
|
||||
GH_AW_COMPILED_VERSION: v0.86.2
|
||||
- name: Install AWF binary
|
||||
run: bash "${RUNNER_TEMP}/gh-aw/actions/install_awf_binary.sh" v0.27.44 --rootless
|
||||
- name: Determine automatic lockdown mode for GitHub MCP Server
|
||||
id: determine-automatic-lockdown
|
||||
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 (source v9)
|
||||
env:
|
||||
GH_AW_GITHUB_TOKEN: ${{ secrets.GH_AW_GITHUB_TOKEN }}
|
||||
GH_AW_GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN }}
|
||||
with:
|
||||
script: |
|
||||
const determineAutomaticLockdown = require('${{ runner.temp }}/gh-aw/actions/determine_automatic_lockdown.cjs');
|
||||
await determineAutomaticLockdown(github, context, core);
|
||||
- name: Restore agent config folders from base branch
|
||||
if: steps.checkout-pr.outcome == 'success'
|
||||
env:
|
||||
@@ -506,7 +469,7 @@ jobs:
|
||||
GH_AW_SKILL_DIR: ".github/skills"
|
||||
run: bash "${RUNNER_TEMP}/gh-aw/actions/restore_inline_skills.sh"
|
||||
- name: Download container images
|
||||
run: bash "${RUNNER_TEMP}/gh-aw/actions/download_docker_images.sh" ghcr.io/github/gh-aw-firewall/agent:0.27.44@sha256:0d727725c737b58c7bdf51f640cffb928385ec46517e0917c7f1a02f1bada8b4 ghcr.io/github/gh-aw-firewall/api-proxy:0.27.44@sha256:b50fbadba138f6e9aba94aca09711335c489bb3b15861220cb66f6092e042dc7 ghcr.io/github/gh-aw-firewall/squid:0.27.44@sha256:83e48bbe12c634be8c228a576832fe45f66c529ac3659db92bddbcf2eeb6d627 ghcr.io/github/gh-aw-mcpg:v0.4.9@sha256:e5a1569aeaf41820fa7bdee3e94468cae448133cdbf00119ad24f5b74db1ab9f ghcr.io/github/gh-aw-node@sha256:0d9f1fb5fd6610c0ac1f5194a38e45a8a1e81f8a390d5142d8e4e6f26a4b3196 ghcr.io/github/github-mcp-server:v1.9.0@sha256:881b53d6f75f69bdbc1b5b10fc2f1361717c19054143b3a8529fb5c32061a50e
|
||||
run: bash "${RUNNER_TEMP}/gh-aw/actions/download_docker_images.sh" ghcr.io/github/gh-aw-firewall/agent:0.27.44@sha256:0d727725c737b58c7bdf51f640cffb928385ec46517e0917c7f1a02f1bada8b4 ghcr.io/github/gh-aw-firewall/api-proxy:0.27.44@sha256:b50fbadba138f6e9aba94aca09711335c489bb3b15861220cb66f6092e042dc7 ghcr.io/github/gh-aw-firewall/squid:0.27.44@sha256:83e48bbe12c634be8c228a576832fe45f66c529ac3659db92bddbcf2eeb6d627 ghcr.io/github/gh-aw-mcpg:v0.4.9@sha256:e5a1569aeaf41820fa7bdee3e94468cae448133cdbf00119ad24f5b74db1ab9f ghcr.io/github/gh-aw-node@sha256:0d9f1fb5fd6610c0ac1f5194a38e45a8a1e81f8a390d5142d8e4e6f26a4b3196
|
||||
- name: Generate Safe Outputs Config
|
||||
run: |
|
||||
mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs"
|
||||
@@ -733,10 +696,6 @@ jobs:
|
||||
GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }}
|
||||
GH_AW_SAFE_OUTPUTS_CONFIG_PATH: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS_CONFIG_PATH }}
|
||||
GH_AW_SAFE_OUTPUTS_TOOLS_PATH: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS_TOOLS_PATH }}
|
||||
GH_AW_SINK_VISIBILITY: ${{ steps.determine-automatic-lockdown.outputs.visibility }}
|
||||
GITHUB_MCP_GUARD_MIN_INTEGRITY: ${{ steps.determine-automatic-lockdown.outputs.min_integrity }}
|
||||
GITHUB_MCP_GUARD_REPOS: ${{ steps.determine-automatic-lockdown.outputs.repos }}
|
||||
GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: |
|
||||
set -eo pipefail
|
||||
@@ -763,26 +722,9 @@ jobs:
|
||||
|
||||
mkdir -p "$HOME/.copilot"
|
||||
GH_AW_NODE=$(which node 2>/dev/null || command -v node 2>/dev/null || echo node)
|
||||
cat << GH_AW_MCP_CONFIG_e26f23cc8d0ae67e_EOF | "$GH_AW_NODE" "${RUNNER_TEMP}/gh-aw/actions/start_mcp_gateway.cjs"
|
||||
cat << GH_AW_MCP_CONFIG_948d9cbdbcfa4bfe_EOF | "$GH_AW_NODE" "${RUNNER_TEMP}/gh-aw/actions/start_mcp_gateway.cjs"
|
||||
{
|
||||
"mcpServers": {
|
||||
"github": {
|
||||
"type": "stdio",
|
||||
"container": "ghcr.io/github/github-mcp-server:v1.9.0",
|
||||
"env": {
|
||||
"GITHUB_FEATURES": "fields_param",
|
||||
"GITHUB_HOST": "${GITHUB_SERVER_URL}",
|
||||
"GITHUB_PERSONAL_ACCESS_TOKEN": "${GITHUB_MCP_SERVER_TOKEN}",
|
||||
"GITHUB_READ_ONLY": "1",
|
||||
"GITHUB_TOOLSETS": "context,repos,issues,pull_requests"
|
||||
},
|
||||
"guard-policies": {
|
||||
"allow-only": {
|
||||
"min-integrity": "$GITHUB_MCP_GUARD_MIN_INTEGRITY",
|
||||
"repos": "$GITHUB_MCP_GUARD_REPOS"
|
||||
}
|
||||
}
|
||||
},
|
||||
"safeoutputs": {
|
||||
"type": "stdio",
|
||||
"container": "ghcr.io/github/gh-aw-node",
|
||||
@@ -806,14 +748,6 @@ jobs:
|
||||
"GITHUB_TOKEN": "\${GITHUB_TOKEN}",
|
||||
"GITHUB_WORKSPACE": "\${GITHUB_WORKSPACE}",
|
||||
"RUNNER_TEMP": "\${RUNNER_TEMP}"
|
||||
},
|
||||
"guard-policies": {
|
||||
"write-sink": {
|
||||
"accept": [
|
||||
"*"
|
||||
],
|
||||
"sink-visibility": "${GH_AW_SINK_VISIBILITY}"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
@@ -825,7 +759,7 @@ jobs:
|
||||
"startupTimeout": 120
|
||||
}
|
||||
}
|
||||
GH_AW_MCP_CONFIG_e26f23cc8d0ae67e_EOF
|
||||
GH_AW_MCP_CONFIG_948d9cbdbcfa4bfe_EOF
|
||||
- name: Mount MCP servers as CLIs
|
||||
id: mount-mcp-clis
|
||||
continue-on-error: true
|
||||
@@ -850,6 +784,8 @@ jobs:
|
||||
- name: Execute GitHub Copilot CLI
|
||||
id: agentic_execution
|
||||
# Copilot CLI tool arguments (sorted):
|
||||
# --allow-tool safeoutputs
|
||||
# --allow-tool write
|
||||
timeout-minutes: 20
|
||||
run: |
|
||||
set -o pipefail
|
||||
@@ -895,8 +831,8 @@ jobs:
|
||||
fi
|
||||
fi
|
||||
# shellcheck disable=SC1003,SC2016,SC2086
|
||||
awf --config "${RUNNER_TEMP}/gh-aw/awf-config.json" --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" ${GH_AW_TOOL_CACHE_MOUNT:+--mount "$GH_AW_TOOL_CACHE_MOUNT"} ${GH_AW_DOCKER_HOST:+--docker-host "$GH_AW_DOCKER_HOST"} --env-all --exclude-env ACTIONS_ID_TOKEN_REQUEST_TOKEN --exclude-env ACTIONS_ID_TOKEN_REQUEST_URL --exclude-env COPILOT_GITHUB_TOKEN --exclude-env GITHUB_MCP_SERVER_TOKEN --exclude-env MCP_GATEWAY_API_KEY --mount /tmp/gh-aw:/tmp/gh-aw:rw --log-level info --skip-pull \
|
||||
-- /bin/bash -c 'set +o histexpand; export PATH="${RUNNER_TEMP}/gh-aw/mcp-cli/bin:$PATH" && : "${RUNNER_TOOL_CACHE:?RUNNER_TOOL_CACHE must be set}"; GH_AW_TOOL_CACHE="$RUNNER_TOOL_CACHE"; export PATH="$(find "$GH_AW_TOOL_CACHE" -maxdepth 5 -type d -name bin 2>/dev/null | tr '\''\n'\'' '\'':'\'')$PATH"; [ -n "$GOROOT" ] && export PATH="$GOROOT/bin:$PATH" || true; [ -n "$ERLANG_HOME" ] && export PATH="$ERLANG_HOME/bin:$PATH" || true && GH_AW_NODE_EXEC="${GH_AW_NODE_BIN:-}"; if [ -z "$GH_AW_NODE_EXEC" ] || [ ! -x "$GH_AW_NODE_EXEC" ]; then GH_AW_NODE_EXEC="$(command -v node 2>/dev/null || true)"; fi; if [ -z "$GH_AW_NODE_EXEC" ]; then echo "node runtime missing on this runner — check runtimes.node in workflow YAML" >&2; exit 127; fi; GH_AW_NPM_GLOBAL_ROOT="$(npm root -g 2>/dev/null || true)"; if [ -n "$GH_AW_NPM_GLOBAL_ROOT" ]; then export NODE_PATH="${GH_AW_NPM_GLOBAL_ROOT}${NODE_PATH:+:${NODE_PATH}}"; fi; "$GH_AW_NODE_EXEC" "${RUNNER_TEMP}/gh-aw/actions/copilot_harness.cjs" "${RUNNER_TEMP}/gh-aw/bin/copilot" --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --disable-builtin-mcps --no-ask-user --allow-all-tools --allow-all-paths --add-dir "${GITHUB_WORKSPACE}" --prompt-file /tmp/gh-aw/aw-prompts/prompt.txt' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log
|
||||
awf --config "${RUNNER_TEMP}/gh-aw/awf-config.json" --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" ${GH_AW_TOOL_CACHE_MOUNT:+--mount "$GH_AW_TOOL_CACHE_MOUNT"} ${GH_AW_DOCKER_HOST:+--docker-host "$GH_AW_DOCKER_HOST"} --env-all --exclude-env ACTIONS_ID_TOKEN_REQUEST_TOKEN --exclude-env ACTIONS_ID_TOKEN_REQUEST_URL --exclude-env COPILOT_GITHUB_TOKEN --exclude-env MCP_GATEWAY_API_KEY --mount /tmp/gh-aw:/tmp/gh-aw:rw --log-level info --skip-pull \
|
||||
-- /bin/bash -c 'set +o histexpand; export PATH="${RUNNER_TEMP}/gh-aw/mcp-cli/bin:$PATH" && : "${RUNNER_TOOL_CACHE:?RUNNER_TOOL_CACHE must be set}"; GH_AW_TOOL_CACHE="$RUNNER_TOOL_CACHE"; export PATH="$(find "$GH_AW_TOOL_CACHE" -maxdepth 5 -type d -name bin 2>/dev/null | tr '\''\n'\'' '\'':'\'')$PATH"; [ -n "$GOROOT" ] && export PATH="$GOROOT/bin:$PATH" || true; [ -n "$ERLANG_HOME" ] && export PATH="$ERLANG_HOME/bin:$PATH" || true && GH_AW_NODE_EXEC="${GH_AW_NODE_BIN:-}"; if [ -z "$GH_AW_NODE_EXEC" ] || [ ! -x "$GH_AW_NODE_EXEC" ]; then GH_AW_NODE_EXEC="$(command -v node 2>/dev/null || true)"; fi; if [ -z "$GH_AW_NODE_EXEC" ]; then echo "node runtime missing on this runner — check runtimes.node in workflow YAML" >&2; exit 127; fi; GH_AW_NPM_GLOBAL_ROOT="$(npm root -g 2>/dev/null || true)"; if [ -n "$GH_AW_NPM_GLOBAL_ROOT" ]; then export NODE_PATH="${GH_AW_NPM_GLOBAL_ROOT}${NODE_PATH:+:${NODE_PATH}}"; fi; "$GH_AW_NODE_EXEC" "${RUNNER_TEMP}/gh-aw/actions/copilot_harness.cjs" "${RUNNER_TEMP}/gh-aw/bin/copilot" --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --disable-builtin-mcps --no-ask-user --allow-tool safeoutputs --allow-tool write --allow-all-paths --add-dir "${GITHUB_WORKSPACE}" --prompt-file /tmp/gh-aw/aw-prompts/prompt.txt' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log
|
||||
env:
|
||||
AWF_REFLECT_ENABLED: 1
|
||||
COPILOT_AGENT_RUNNER_TYPE: STANDALONE
|
||||
@@ -914,7 +850,6 @@ jobs:
|
||||
GITHUB_AW: true
|
||||
GITHUB_COPILOT_INTEGRATION_ID: agentic-workflows
|
||||
GITHUB_HEAD_REF: ${{ github.head_ref }}
|
||||
GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
GITHUB_REF_NAME: ${{ github.ref_name }}
|
||||
GITHUB_SERVER_URL: ${{ github.server_url }}
|
||||
GITHUB_STEP_SUMMARY: /tmp/gh-aw/agent-step-summary.md
|
||||
@@ -1641,9 +1576,10 @@ jobs:
|
||||
- detection
|
||||
if: >
|
||||
(!cancelled()) && needs.agent.result != 'skipped' && contains(needs.agent.outputs.output_types, 'publish_triage_summary') &&
|
||||
(needs.detection.result == 'success')
|
||||
(needs.detection.result == 'success' && needs.detection.outputs.detection_success == 'true')
|
||||
runs-on: ubuntu-slim
|
||||
permissions:
|
||||
contents: read
|
||||
issues: write
|
||||
steps:
|
||||
- name: Download agent output artifact
|
||||
@@ -1652,10 +1588,38 @@ jobs:
|
||||
with:
|
||||
name: agent
|
||||
path: ${{ runner.temp }}/gh-aw/safe-jobs/
|
||||
- name: Check out trusted workflow source
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
env:
|
||||
GH_AW_AGENT_OUTPUT: ${{ runner.temp }}/gh-aw/safe-jobs/agent_output.json
|
||||
with:
|
||||
persist-credentials: false
|
||||
ref: ${{ github.sha }}
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
|
||||
env:
|
||||
GH_AW_AGENT_OUTPUT: ${{ runner.temp }}/gh-aw/safe-jobs/agent_output.json
|
||||
with:
|
||||
python-version: "3.12"
|
||||
- name: Rebuild current deterministic evidence
|
||||
run: python .github/scripts/issue-triage/issue-context.py "$GITHUB_EVENT_PATH" "$RUNNER_TEMP/verified-issue-context.md" "$RUNNER_TEMP/verified-triage-event.json" "$RUNNER_TEMP/verified-evidence.json"
|
||||
env:
|
||||
GH_AW_AGENT_OUTPUT: ${{ runner.temp }}/gh-aw/safe-jobs/agent_output.json
|
||||
GITHUB_TOKEN: ${{ github.token }}
|
||||
ISSUE_TRIAGE_FORCE_EVIDENCE: "true"
|
||||
- name: Rebuild sanitized bug report context
|
||||
run: python .github/scripts/issue-triage/bug-report-analyzer.py "$RUNNER_TEMP/verified-triage-event.json" "$RUNNER_TEMP/verified-bug-report-context.md"
|
||||
env:
|
||||
GH_AW_AGENT_OUTPUT: ${{ runner.temp }}/gh-aw/safe-jobs/agent_output.json
|
||||
- name: Verify agent output against current evidence
|
||||
run: python .github/scripts/issue-triage/verify-agent-output.py "$GH_AW_AGENT_OUTPUT" "$RUNNER_TEMP/verified-evidence.json" "$RUNNER_TEMP/verified-bug-report-context.md" "$RUNNER_TEMP/verified-triage-output.json"
|
||||
env:
|
||||
GH_AW_AGENT_OUTPUT: ${{ runner.temp }}/gh-aw/safe-jobs/agent_output.json
|
||||
- name: Upsert canonical triage summary
|
||||
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
|
||||
env:
|
||||
GH_AW_AGENT_OUTPUT: ${{ runner.temp }}/gh-aw/safe-jobs/agent_output.json
|
||||
ISSUE_TRIAGE_VERIFIED_OUTPUT: ${{ runner.temp }}/verified-triage-output.json
|
||||
with:
|
||||
script: |
|
||||
const fs = require('fs');
|
||||
@@ -1675,6 +1639,12 @@ jobs:
|
||||
core.setFailed('The agent did not provide a triage summary');
|
||||
return;
|
||||
}
|
||||
const verifiedPath = process.env.ISSUE_TRIAGE_VERIFIED_OUTPUT;
|
||||
if (!verifiedPath || !fs.existsSync(verifiedPath)) {
|
||||
core.setFailed('Verified triage output is unavailable');
|
||||
return;
|
||||
}
|
||||
const verified = JSON.parse(fs.readFileSync(verifiedPath, 'utf8'));
|
||||
|
||||
const bounded = (value, max, fallback) => {
|
||||
if (typeof value !== 'string') return fallback;
|
||||
@@ -1723,25 +1693,12 @@ jobs:
|
||||
.replace(/\b(?:machine|computer|user)(?:name)?\b\s*[:=]\s*[^\s,;]+/gi, '<identity>=<redacted>');
|
||||
|
||||
const summary = bounded(item.summary, 800, 'The issue needs maintainer review.');
|
||||
const inputSha256 = String(item.input_sha256 || '').toLowerCase();
|
||||
if (!/^[0-9a-f]{64}$/.test(inputSha256)) {
|
||||
core.setFailed('input_sha256 is invalid');
|
||||
return;
|
||||
}
|
||||
const area = bounded(item.suggested_area, 100, 'Unknown');
|
||||
const requestedProductLabel = bounded(item.product_label, 100, 'None');
|
||||
const issueBody = String(context.payload.issue?.body || '');
|
||||
const versionSection = issueBody.match(
|
||||
/###\s+Microsoft PowerToys version\s*\r?\n+([\s\S]*?)(?=\r?\n###|\s*$)/i
|
||||
)?.[1] || '';
|
||||
const reportedVersion = versionSection.match(
|
||||
/\b(?:v)?(\d+(?:\.\d+){1,3}(?:-[A-Za-z0-9.-]+)?)\b/
|
||||
)?.[1];
|
||||
const powertoysVersion = reportedVersion || bounded(
|
||||
item.powertoys_version,
|
||||
50,
|
||||
'Not provided'
|
||||
);
|
||||
const inputSha256 = verified.input_sha256;
|
||||
const area = verified.suggested_area;
|
||||
const requestedProductLabel = verified.product_label;
|
||||
const powertoysVersion = verified.powertoys_version;
|
||||
const reportedVersion =
|
||||
powertoysVersion === 'Not provided' ? null : powertoysVersion;
|
||||
const numericVersion = value => {
|
||||
const match = String(value || '').match(
|
||||
/\b(?:v)?(\d+(?:\.\d+){1,3})(?:-[A-Za-z0-9.-]+)?\b/
|
||||
@@ -1798,10 +1755,7 @@ jobs:
|
||||
800,
|
||||
hasMissingInformation ? 'Additional investigation details are needed.' : 'None'
|
||||
);
|
||||
const requestedIssueKind = String(item.issue_kind || '').toUpperCase();
|
||||
const issueKind = ['BUG', 'OTHER'].includes(requestedIssueKind)
|
||||
? requestedIssueKind
|
||||
: 'OTHER';
|
||||
const issueKind = verified.issue_kind;
|
||||
if (
|
||||
issueKind === 'OTHER' &&
|
||||
/\b(?:bug report|report ZIP|ZIP file)\b/i.test(missingInformation)
|
||||
@@ -1809,30 +1763,9 @@ jobs:
|
||||
hasMissingInformation = false;
|
||||
missingInformation = 'None';
|
||||
}
|
||||
const requestedReproductionQuality = String(
|
||||
item.reproduction_quality || ''
|
||||
).toUpperCase();
|
||||
const reproductionQuality = [
|
||||
'SUFFICIENT', 'INSUFFICIENT', 'NOT_APPLICABLE'
|
||||
].includes(requestedReproductionQuality)
|
||||
? requestedReproductionQuality
|
||||
: 'NOT_APPLICABLE';
|
||||
const requestedBugReportRequirement = String(
|
||||
item.bug_report_requirement || ''
|
||||
).toUpperCase();
|
||||
const bugReportRequirement = [
|
||||
'REQUIRED', 'RECOMMENDED', 'OPTIONAL', 'NOT_APPLICABLE'
|
||||
].includes(requestedBugReportRequirement)
|
||||
? requestedBugReportRequirement
|
||||
: issueKind === 'BUG' ? 'RECOMMENDED' : 'NOT_APPLICABLE';
|
||||
const requestedBugReportStatus = String(
|
||||
item.bug_report_status || ''
|
||||
).toUpperCase();
|
||||
const bugReportStatus = [
|
||||
'ANALYZED', 'NOT_FOUND', 'REJECTED', 'NOT_APPLICABLE'
|
||||
].includes(requestedBugReportStatus)
|
||||
? requestedBugReportStatus
|
||||
: 'NOT_APPLICABLE';
|
||||
const reproductionQuality = verified.reproduction_quality;
|
||||
const bugReportRequirement = verified.bug_report_requirement;
|
||||
const bugReportStatus = verified.bug_report_status;
|
||||
const requestedBugReportConfidence = String(
|
||||
item.bug_report_confidence || ''
|
||||
).toUpperCase();
|
||||
@@ -1893,12 +1826,16 @@ jobs:
|
||||
}
|
||||
|
||||
const issueNumber = context.issue.number;
|
||||
const allowedDuplicateNumbers = new Set(
|
||||
verified.requested_duplicate_numbers
|
||||
);
|
||||
const verifiedDuplicates = [];
|
||||
const seen = new Set();
|
||||
for (const candidate of requestedDuplicates) {
|
||||
const number = Number(candidate?.number);
|
||||
if (!Number.isSafeInteger(number) || number <= 0 ||
|
||||
number === issueNumber || seen.has(number)) {
|
||||
number === issueNumber || seen.has(number) ||
|
||||
!allowedDuplicateNumbers.has(number)) {
|
||||
continue;
|
||||
}
|
||||
seen.add(number);
|
||||
@@ -1939,7 +1876,7 @@ jobs:
|
||||
].join('\n')
|
||||
).join('\n\n')
|
||||
: '';
|
||||
const author = context.payload.issue?.user?.login;
|
||||
const author = verified.issue_author;
|
||||
const authorActions = [];
|
||||
if (hasMissingInformation) {
|
||||
authorActions.push(
|
||||
@@ -2124,9 +2061,7 @@ jobs:
|
||||
const needsAuthorFeedback =
|
||||
needsEnglishTranslation || hasMissingInformation;
|
||||
const currentLabels = new Set(
|
||||
(context.payload.issue?.labels || [])
|
||||
.map(label => typeof label === 'string' ? label : label?.name)
|
||||
.filter(Boolean)
|
||||
verified.current_labels
|
||||
);
|
||||
if (needsAuthorFeedback && !currentLabels.has('Needs-Author-Feedback')) {
|
||||
await github.rest.issues.addLabels({
|
||||
|
||||
121
.github/workflows/issue-triage.md
vendored
121
.github/workflows/issue-triage.md
vendored
@@ -25,6 +25,10 @@ permissions:
|
||||
contents: read
|
||||
issues: read
|
||||
copilot-requests: write
|
||||
tools:
|
||||
bash: false
|
||||
edit: true
|
||||
github: false
|
||||
steps:
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v7.0.0
|
||||
@@ -52,9 +56,12 @@ safe-outputs:
|
||||
publish-triage-summary:
|
||||
description: Create or update the canonical triage summary for the triggering issue.
|
||||
runs-on: ubuntu-slim
|
||||
if: needs.detection.result == 'success'
|
||||
if: >-
|
||||
needs.detection.result == 'success' &&
|
||||
needs.detection.outputs.detection_success == 'true'
|
||||
output: The canonical triage summary was published.
|
||||
permissions:
|
||||
contents: read
|
||||
issues: write
|
||||
inputs:
|
||||
summary:
|
||||
@@ -124,8 +131,41 @@ safe-outputs:
|
||||
type: choice
|
||||
options: [ENGLISH, NON_ENGLISH, UNCERTAIN]
|
||||
steps:
|
||||
- name: Check out trusted workflow source
|
||||
uses: actions/checkout@v7.0.1
|
||||
with:
|
||||
ref: ${{ github.sha }}
|
||||
persist-credentials: false
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v7.0.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
- name: Rebuild current deterministic evidence
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ github.token }}
|
||||
ISSUE_TRIAGE_FORCE_EVIDENCE: "true"
|
||||
run: >-
|
||||
python .github/scripts/issue-triage/issue-context.py
|
||||
"$GITHUB_EVENT_PATH"
|
||||
"$RUNNER_TEMP/verified-issue-context.md"
|
||||
"$RUNNER_TEMP/verified-triage-event.json"
|
||||
"$RUNNER_TEMP/verified-evidence.json"
|
||||
- name: Rebuild sanitized bug report context
|
||||
run: >-
|
||||
python .github/scripts/issue-triage/bug-report-analyzer.py
|
||||
"$RUNNER_TEMP/verified-triage-event.json"
|
||||
"$RUNNER_TEMP/verified-bug-report-context.md"
|
||||
- name: Verify agent output against current evidence
|
||||
run: >-
|
||||
python .github/scripts/issue-triage/verify-agent-output.py
|
||||
"$GH_AW_AGENT_OUTPUT"
|
||||
"$RUNNER_TEMP/verified-evidence.json"
|
||||
"$RUNNER_TEMP/verified-bug-report-context.md"
|
||||
"$RUNNER_TEMP/verified-triage-output.json"
|
||||
- name: Upsert canonical triage summary
|
||||
uses: actions/github-script@v9.0.0
|
||||
env:
|
||||
ISSUE_TRIAGE_VERIFIED_OUTPUT: ${{ runner.temp }}/verified-triage-output.json
|
||||
with:
|
||||
script: |
|
||||
const fs = require('fs');
|
||||
@@ -145,6 +185,12 @@ safe-outputs:
|
||||
core.setFailed('The agent did not provide a triage summary');
|
||||
return;
|
||||
}
|
||||
const verifiedPath = process.env.ISSUE_TRIAGE_VERIFIED_OUTPUT;
|
||||
if (!verifiedPath || !fs.existsSync(verifiedPath)) {
|
||||
core.setFailed('Verified triage output is unavailable');
|
||||
return;
|
||||
}
|
||||
const verified = JSON.parse(fs.readFileSync(verifiedPath, 'utf8'));
|
||||
|
||||
const bounded = (value, max, fallback) => {
|
||||
if (typeof value !== 'string') return fallback;
|
||||
@@ -193,25 +239,12 @@ safe-outputs:
|
||||
.replace(/\b(?:machine|computer|user)(?:name)?\b\s*[:=]\s*[^\s,;]+/gi, '<identity>=<redacted>');
|
||||
|
||||
const summary = bounded(item.summary, 800, 'The issue needs maintainer review.');
|
||||
const inputSha256 = String(item.input_sha256 || '').toLowerCase();
|
||||
if (!/^[0-9a-f]{64}$/.test(inputSha256)) {
|
||||
core.setFailed('input_sha256 is invalid');
|
||||
return;
|
||||
}
|
||||
const area = bounded(item.suggested_area, 100, 'Unknown');
|
||||
const requestedProductLabel = bounded(item.product_label, 100, 'None');
|
||||
const issueBody = String(context.payload.issue?.body || '');
|
||||
const versionSection = issueBody.match(
|
||||
/###\s+Microsoft PowerToys version\s*\r?\n+([\s\S]*?)(?=\r?\n###|\s*$)/i
|
||||
)?.[1] || '';
|
||||
const reportedVersion = versionSection.match(
|
||||
/\b(?:v)?(\d+(?:\.\d+){1,3}(?:-[A-Za-z0-9.-]+)?)\b/
|
||||
)?.[1];
|
||||
const powertoysVersion = reportedVersion || bounded(
|
||||
item.powertoys_version,
|
||||
50,
|
||||
'Not provided'
|
||||
);
|
||||
const inputSha256 = verified.input_sha256;
|
||||
const area = verified.suggested_area;
|
||||
const requestedProductLabel = verified.product_label;
|
||||
const powertoysVersion = verified.powertoys_version;
|
||||
const reportedVersion =
|
||||
powertoysVersion === 'Not provided' ? null : powertoysVersion;
|
||||
const numericVersion = value => {
|
||||
const match = String(value || '').match(
|
||||
/\b(?:v)?(\d+(?:\.\d+){1,3})(?:-[A-Za-z0-9.-]+)?\b/
|
||||
@@ -268,10 +301,7 @@ safe-outputs:
|
||||
800,
|
||||
hasMissingInformation ? 'Additional investigation details are needed.' : 'None'
|
||||
);
|
||||
const requestedIssueKind = String(item.issue_kind || '').toUpperCase();
|
||||
const issueKind = ['BUG', 'OTHER'].includes(requestedIssueKind)
|
||||
? requestedIssueKind
|
||||
: 'OTHER';
|
||||
const issueKind = verified.issue_kind;
|
||||
if (
|
||||
issueKind === 'OTHER' &&
|
||||
/\b(?:bug report|report ZIP|ZIP file)\b/i.test(missingInformation)
|
||||
@@ -279,30 +309,9 @@ safe-outputs:
|
||||
hasMissingInformation = false;
|
||||
missingInformation = 'None';
|
||||
}
|
||||
const requestedReproductionQuality = String(
|
||||
item.reproduction_quality || ''
|
||||
).toUpperCase();
|
||||
const reproductionQuality = [
|
||||
'SUFFICIENT', 'INSUFFICIENT', 'NOT_APPLICABLE'
|
||||
].includes(requestedReproductionQuality)
|
||||
? requestedReproductionQuality
|
||||
: 'NOT_APPLICABLE';
|
||||
const requestedBugReportRequirement = String(
|
||||
item.bug_report_requirement || ''
|
||||
).toUpperCase();
|
||||
const bugReportRequirement = [
|
||||
'REQUIRED', 'RECOMMENDED', 'OPTIONAL', 'NOT_APPLICABLE'
|
||||
].includes(requestedBugReportRequirement)
|
||||
? requestedBugReportRequirement
|
||||
: issueKind === 'BUG' ? 'RECOMMENDED' : 'NOT_APPLICABLE';
|
||||
const requestedBugReportStatus = String(
|
||||
item.bug_report_status || ''
|
||||
).toUpperCase();
|
||||
const bugReportStatus = [
|
||||
'ANALYZED', 'NOT_FOUND', 'REJECTED', 'NOT_APPLICABLE'
|
||||
].includes(requestedBugReportStatus)
|
||||
? requestedBugReportStatus
|
||||
: 'NOT_APPLICABLE';
|
||||
const reproductionQuality = verified.reproduction_quality;
|
||||
const bugReportRequirement = verified.bug_report_requirement;
|
||||
const bugReportStatus = verified.bug_report_status;
|
||||
const requestedBugReportConfidence = String(
|
||||
item.bug_report_confidence || ''
|
||||
).toUpperCase();
|
||||
@@ -363,12 +372,16 @@ safe-outputs:
|
||||
}
|
||||
|
||||
const issueNumber = context.issue.number;
|
||||
const allowedDuplicateNumbers = new Set(
|
||||
verified.requested_duplicate_numbers
|
||||
);
|
||||
const verifiedDuplicates = [];
|
||||
const seen = new Set();
|
||||
for (const candidate of requestedDuplicates) {
|
||||
const number = Number(candidate?.number);
|
||||
if (!Number.isSafeInteger(number) || number <= 0 ||
|
||||
number === issueNumber || seen.has(number)) {
|
||||
number === issueNumber || seen.has(number) ||
|
||||
!allowedDuplicateNumbers.has(number)) {
|
||||
continue;
|
||||
}
|
||||
seen.add(number);
|
||||
@@ -409,7 +422,7 @@ safe-outputs:
|
||||
].join('\n')
|
||||
).join('\n\n')
|
||||
: '';
|
||||
const author = context.payload.issue?.user?.login;
|
||||
const author = verified.issue_author;
|
||||
const authorActions = [];
|
||||
if (hasMissingInformation) {
|
||||
authorActions.push(
|
||||
@@ -594,9 +607,7 @@ safe-outputs:
|
||||
const needsAuthorFeedback =
|
||||
needsEnglishTranslation || hasMissingInformation;
|
||||
const currentLabels = new Set(
|
||||
(context.payload.issue?.labels || [])
|
||||
.map(label => typeof label === 'string' ? label : label?.name)
|
||||
.filter(Boolean)
|
||||
verified.current_labels
|
||||
);
|
||||
if (needsAuthorFeedback && !currentLabels.has('Needs-Author-Feedback')) {
|
||||
await github.rest.issues.addLabels({
|
||||
@@ -772,8 +783,8 @@ Call `publish_triage_summary` exactly once with:
|
||||
- `summary`: a factual one- or two-sentence summary.
|
||||
- `suggested_area`: copy `Detected area` from the deterministic evidence.
|
||||
- `product_label`: copy `Candidate product label` from the deterministic
|
||||
evidence. When the evidence says `None`, send the literal string `None`;
|
||||
never send JSON null.
|
||||
evidence. When it says `None`, you may select one exact label from
|
||||
`Allowed product label candidates`, or send `None`. Never invent a label.
|
||||
- `powertoys_version`: copy `PowerToys version` from the deterministic evidence.
|
||||
|
||||
The deterministic publisher independently verifies the latest stable release.
|
||||
|
||||
Reference in New Issue
Block a user