Files
PowerToys/.github/review-tools/Test-IncrementalReview.ps1
Gordon Lam 534c411fd8 Add Review and Fix prompts for acceleration github issues (#43419)
## Summary of the Pull Request

This PR adds comprehensive GitHub PR review tooling infrastructure to
improve review efficiency through incremental review capabilities. The
tooling consists of PowerShell scripts that detect changes between
review iterations, enabling reviewers to focus only on modified files,
and AI prompt templates for structured PR and issue reviews.

**Key additions:**
- `.github/review-tools/`: PowerShell scripts for incremental PR review
detection
- `.github/prompts/`: AI prompt templates for PR reviews, issue reviews,
and issue fixes
- Copilot instructions documentation for review tools integration

## PR Checklist

- [ ] Closes: #xxx
- [x] **Communication:** I've discussed this with core contributors
already. If the work hasn't been agreed, this work might be rejected
- [x] **Tests:** Added/updated and all pass (N/A - tooling scripts with
manual validation)
- [x] **Localization:** All end-user-facing strings can be localized
(N/A - development tooling only)
- [x] **Dev docs:** Added/updated (comprehensive documentation in
`review-tools.instructions.md`)
- [ N/A ] **New binaries:** Added on the required places (N/A -
PowerShell scripts, no binaries)
- [ N/A ] **Documentation updated:** If checked, please file a pull
request on [our docs
repo](https://github.com/MicrosoftDocs/windows-uwp/tree/docs/hub/powertoys)
and link it here: #xxx (N/A - internal development tooling)

## Detailed Description of the Pull Request / Additional comments

### Review Tools Added (`.github/review-tools/`)

**Core Scripts:**
1. **`Get-PrIncrementalChanges.ps1`** (173 lines) - Compares last
reviewed commit SHA with current PR head to identify incremental
changes. Returns JSON with changed files, new commits, and review
recommendations. Handles scenarios: first review, no changes, force-push
detection, and incremental changes.

2. **`Test-IncrementalReview.ps1`** (170 lines) - Helper script to
preview incremental review detection before running full review.
Displays colored console output showing current vs last reviewed SHAs,
changed files, new commits, and recommended review strategy.

3. **`Migrate-ReviewToIncrementalFormat.ps1`** (206 lines) - One-time
migration script to add review metadata sections to existing
`00-OVERVIEW.md` files, enabling incremental review functionality for
legacy PR reviews.

4. **`Get-GitHubRawFile.ps1`** (91 lines) - Downloads file content from
GitHub repository at specific git reference for baseline comparison
during reviews. Supports optional line numbering.

5. **`Get-GitHubPrFilePatch.ps1`** (79 lines) - Fetches unified diff
(patch) for a specific file in a pull request using GitHub API.

6. **`review-tools.instructions.md`** (355 lines) - Comprehensive
documentation covering script usage, workflow integration, error
handling, best practices, and AI integration examples. Serves as Copilot
instructions for the review tools.

### AI Prompt Templates Added (`.github/prompts/`)

1. **`review-pr.prompt.md`** (200 lines) - Structured prompt for
comprehensive PR reviews with incremental review support
2. **`review-issue.prompt.md`** (158 lines) - Template for systematic
issue analysis and triage
3. **`fix-issue.prompt.md`** (71 lines) - Guided workflow for
implementing issue fixes
4. **Minor updates** to `create-commit-title.prompt.md` and
`create-pr-summary.prompt.md` for consistency

### Key Features

**Incremental Review Flow:**
- Initial review (iteration 1): Creates overview with metadata including
HEAD SHA
- Subsequent reviews (iteration 2+): Reads last reviewed SHA, detects
changes, reviews only modified files
- Smart step filtering: Skips irrelevant review steps based on file
types changed (e.g., skip Localization if no `.resx` files changed)

**Error Handling:**
- Detects force-push scenarios (last reviewed SHA no longer in history)
- Validates GitHub CLI authentication
- Provides detailed error messages with actionable solutions
- Exit code 1 on errors for automation compatibility

**Integration:**
- Works with GitHub CLI (`gh`) for API access
- JSON output for programmatic consumption
- Designed for AI-powered review systems (Copilot integration)
- Metadata tracking for context-aware suggestions

## Validation Steps Performed

**Manual Testing:**
1.  Verified all PowerShell scripts follow PSScriptAnalyzer rules
2.  Tested `Get-PrIncrementalChanges.ps1` with various scenarios:
   - First review (no previous SHA)
   - No changes (SHA matches current)
   - Force-push detection (SHA not in history)
   - Incremental changes (multiple commits and files)
3.  Validated `Test-IncrementalReview.ps1` output formatting and
accuracy
4.  Confirmed `Migrate-ReviewToIncrementalFormat.ps1` adds metadata
without corrupting existing reviews
5.  Tested GitHub API integration with `Get-GitHubRawFile.ps1` and
`Get-GitHubPrFilePatch.ps1`
6.  Verified documentation accuracy and completeness in
`review-tools.instructions.md`
7.  Validated prompt templates follow PowerToys repo conventions from
`.github/copilot-instructions.md`

**Dependencies Verified:**
- Requires PowerShell 7+ (or Windows PowerShell 5.1+)
- Requires GitHub CLI (`gh`) installed and authenticated
- All scripts include comprehensive help documentation (`Get-Help`
support)

**No Automated Tests:** These are development/CI tooling scripts used by
maintainers for PR reviews. Manual validation appropriate as they
interact with live GitHub API and require human judgment for review
quality assessment.
2025-12-23 16:18:22 +08:00

171 lines
7.1 KiB
PowerShell

<#
.SYNOPSIS
Tests and previews incremental review detection for a pull request.
.DESCRIPTION
This helper script validates the incremental review detection logic by analyzing an existing
PR review folder. It reads the last reviewed SHA from the overview file, compares it with
the current PR state, and displays detailed information about what has changed.
This is useful for:
- Testing the incremental review system before running a full review
- Understanding what changed since the last review iteration
- Verifying that review metadata was properly recorded
.PARAMETER PullRequestNumber
The pull request number to test incremental review detection for.
.PARAMETER RepositoryOwner
The GitHub repository owner. Defaults to "microsoft".
.PARAMETER RepositoryName
The GitHub repository name. Defaults to "PowerToys".
.OUTPUTS
Colored console output displaying:
- Current and last reviewed commit SHAs
- Whether incremental review is possible
- List of new commits since last review
- List of changed files with status indicators
- Recommended review strategy
.EXAMPLE
.\Test-IncrementalReview.ps1 -PullRequestNumber 42374
Tests incremental review detection for PR #42374.
.EXAMPLE
.\Test-IncrementalReview.ps1 -PullRequestNumber 42374 -RepositoryOwner "myorg" -RepositoryName "myrepo"
Tests incremental review for a PR in a different repository.
.NOTES
Requires GitHub CLI (gh) to be installed and authenticated.
Run 'gh auth login' if not already authenticated.
Prerequisites:
- PR review folder must exist at "Generated Files\prReview\{PRNumber}"
- 00-OVERVIEW.md must exist in the review folder
- For incremental detection, overview must contain "Last reviewed SHA" metadata
.LINK
https://cli.github.com/
#>
[CmdletBinding()]
param(
[Parameter(Mandatory = $true, HelpMessage = "Pull request number to test")]
[int]$PullRequestNumber,
[Parameter(Mandatory = $false, HelpMessage = "Repository owner")]
[string]$RepositoryOwner = "microsoft",
[Parameter(Mandatory = $false, HelpMessage = "Repository name")]
[string]$RepositoryName = "PowerToys"
)
# Resolve paths to review folder and overview file
$repositoryRoot = Split-Path (Split-Path $PSScriptRoot -Parent) -Parent
$reviewFolderPath = Join-Path $repositoryRoot "Generated Files\prReview\$PullRequestNumber"
$overviewFilePath = Join-Path $reviewFolderPath "00-OVERVIEW.md"
Write-Host "=== Testing Incremental Review for PR #$PullRequestNumber ===" -ForegroundColor Cyan
Write-Host ""
# Check if review folder exists
if (-not (Test-Path $reviewFolderPath)) {
Write-Host "❌ Review folder not found: $reviewFolderPath" -ForegroundColor Red
Write-Host "This appears to be a new review (iteration 1)" -ForegroundColor Yellow
exit 0
}
# Check if overview file exists
if (-not (Test-Path $overviewFilePath)) {
Write-Host "❌ Overview file not found: $overviewFilePath" -ForegroundColor Red
Write-Host "This appears to be an incomplete review" -ForegroundColor Yellow
exit 0
}
# Read overview file and extract last reviewed SHA
Write-Host "📄 Reading overview file..." -ForegroundColor Green
$overviewFileContent = Get-Content $overviewFilePath -Raw
if ($overviewFileContent -match '\*\*Last reviewed SHA:\*\*\s+(\w+)') {
$lastReviewedSha = $Matches[1]
Write-Host "✅ Found last reviewed SHA: $lastReviewedSha" -ForegroundColor Green
} else {
Write-Host "⚠️ No 'Last reviewed SHA' found in overview - this may be an old format" -ForegroundColor Yellow
Write-Host "Proceeding without incremental detection (full review will be needed)" -ForegroundColor Yellow
exit 0
}
Write-Host ""
Write-Host "🔍 Running incremental change detection..." -ForegroundColor Cyan
# Call the incremental changes detection script
$incrementalChangesScriptPath = Join-Path $PSScriptRoot "Get-PrIncrementalChanges.ps1"
if (-not (Test-Path $incrementalChangesScriptPath)) {
Write-Host "❌ Script not found: $incrementalChangesScriptPath" -ForegroundColor Red
exit 1
}
try {
$analysisResult = & $incrementalChangesScriptPath `
-PullRequestNumber $PullRequestNumber `
-LastReviewedCommitSha $lastReviewedSha `
-RepositoryOwner $RepositoryOwner `
-RepositoryName $RepositoryName | ConvertFrom-Json
# Display analysis results
Write-Host ""
Write-Host "=== Incremental Review Analysis ===" -ForegroundColor Cyan
Write-Host "Current HEAD SHA: $($analysisResult.CurrentHeadSha)" -ForegroundColor White
Write-Host "Last reviewed SHA: $($analysisResult.LastReviewedSha)" -ForegroundColor White
Write-Host "Base branch: $($analysisResult.BaseRefName)" -ForegroundColor White
Write-Host "Head branch: $($analysisResult.HeadRefName)" -ForegroundColor White
Write-Host ""
Write-Host "Is incremental? $($analysisResult.IsIncremental)" -ForegroundColor $(if ($analysisResult.IsIncremental) { "Green" } else { "Yellow" })
Write-Host "Need full review? $($analysisResult.NeedFullReview)" -ForegroundColor $(if ($analysisResult.NeedFullReview) { "Yellow" } else { "Green" })
Write-Host ""
Write-Host "Summary: $($analysisResult.Summary)" -ForegroundColor Cyan
Write-Host ""
# Display new commits if any
if ($analysisResult.NewCommits -and $analysisResult.NewCommits.Count -gt 0) {
Write-Host "📝 New commits ($($analysisResult.NewCommits.Count)):" -ForegroundColor Green
foreach ($commit in $analysisResult.NewCommits) {
Write-Host " - $($commit.Sha): $($commit.Message)" -ForegroundColor Gray
}
Write-Host ""
}
# Display changed files if any
if ($analysisResult.ChangedFiles -and $analysisResult.ChangedFiles.Count -gt 0) {
Write-Host "📁 Changed files ($($analysisResult.ChangedFiles.Count)):" -ForegroundColor Green
foreach ($file in $analysisResult.ChangedFiles) {
$statusDisplayColor = switch ($file.Status) {
"added" { "Green" }
"removed" { "Red" }
"modified" { "Yellow" }
"renamed" { "Cyan" }
default { "White" }
}
Write-Host " - [$($file.Status)] $($file.Filename) (+$($file.Additions)/-$($file.Deletions))" -ForegroundColor $statusDisplayColor
}
Write-Host ""
}
# Suggest review strategy based on analysis
Write-Host "=== Recommended Review Strategy ===" -ForegroundColor Cyan
if ($analysisResult.NeedFullReview) {
Write-Host "🔄 Full review recommended" -ForegroundColor Yellow
} elseif ($analysisResult.IsIncremental -and ($analysisResult.ChangedFiles.Count -eq 0)) {
Write-Host "✅ No changes detected - no review needed" -ForegroundColor Green
} elseif ($analysisResult.IsIncremental) {
Write-Host "⚡ Incremental review possible - review only changed files" -ForegroundColor Green
Write-Host "💡 Consider applying smart step filtering based on file types" -ForegroundColor Cyan
}
} catch {
Write-Host "❌ Error running incremental change detection: $_" -ForegroundColor Red
exit 1
}