mirror of
https://github.com/microsoft/PowerToys.git
synced 2026-04-03 09:46:54 +02:00
## Summary of the Pull Request This PR adds comprehensive GitHub PR review tooling infrastructure to improve review efficiency through incremental review capabilities. The tooling consists of PowerShell scripts that detect changes between review iterations, enabling reviewers to focus only on modified files, and AI prompt templates for structured PR and issue reviews. **Key additions:** - `.github/review-tools/`: PowerShell scripts for incremental PR review detection - `.github/prompts/`: AI prompt templates for PR reviews, issue reviews, and issue fixes - Copilot instructions documentation for review tools integration ## PR Checklist - [ ] Closes: #xxx - [x] **Communication:** I've discussed this with core contributors already. If the work hasn't been agreed, this work might be rejected - [x] **Tests:** Added/updated and all pass (N/A - tooling scripts with manual validation) - [x] **Localization:** All end-user-facing strings can be localized (N/A - development tooling only) - [x] **Dev docs:** Added/updated (comprehensive documentation in `review-tools.instructions.md`) - [ N/A ] **New binaries:** Added on the required places (N/A - PowerShell scripts, no binaries) - [ N/A ] **Documentation updated:** If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/windows-uwp/tree/docs/hub/powertoys) and link it here: #xxx (N/A - internal development tooling) ## Detailed Description of the Pull Request / Additional comments ### Review Tools Added (`.github/review-tools/`) **Core Scripts:** 1. **`Get-PrIncrementalChanges.ps1`** (173 lines) - Compares last reviewed commit SHA with current PR head to identify incremental changes. Returns JSON with changed files, new commits, and review recommendations. Handles scenarios: first review, no changes, force-push detection, and incremental changes. 2. **`Test-IncrementalReview.ps1`** (170 lines) - Helper script to preview incremental review detection before running full review. Displays colored console output showing current vs last reviewed SHAs, changed files, new commits, and recommended review strategy. 3. **`Migrate-ReviewToIncrementalFormat.ps1`** (206 lines) - One-time migration script to add review metadata sections to existing `00-OVERVIEW.md` files, enabling incremental review functionality for legacy PR reviews. 4. **`Get-GitHubRawFile.ps1`** (91 lines) - Downloads file content from GitHub repository at specific git reference for baseline comparison during reviews. Supports optional line numbering. 5. **`Get-GitHubPrFilePatch.ps1`** (79 lines) - Fetches unified diff (patch) for a specific file in a pull request using GitHub API. 6. **`review-tools.instructions.md`** (355 lines) - Comprehensive documentation covering script usage, workflow integration, error handling, best practices, and AI integration examples. Serves as Copilot instructions for the review tools. ### AI Prompt Templates Added (`.github/prompts/`) 1. **`review-pr.prompt.md`** (200 lines) - Structured prompt for comprehensive PR reviews with incremental review support 2. **`review-issue.prompt.md`** (158 lines) - Template for systematic issue analysis and triage 3. **`fix-issue.prompt.md`** (71 lines) - Guided workflow for implementing issue fixes 4. **Minor updates** to `create-commit-title.prompt.md` and `create-pr-summary.prompt.md` for consistency ### Key Features **Incremental Review Flow:** - Initial review (iteration 1): Creates overview with metadata including HEAD SHA - Subsequent reviews (iteration 2+): Reads last reviewed SHA, detects changes, reviews only modified files - Smart step filtering: Skips irrelevant review steps based on file types changed (e.g., skip Localization if no `.resx` files changed) **Error Handling:** - Detects force-push scenarios (last reviewed SHA no longer in history) - Validates GitHub CLI authentication - Provides detailed error messages with actionable solutions - Exit code 1 on errors for automation compatibility **Integration:** - Works with GitHub CLI (`gh`) for API access - JSON output for programmatic consumption - Designed for AI-powered review systems (Copilot integration) - Metadata tracking for context-aware suggestions ## Validation Steps Performed **Manual Testing:** 1. ✅ Verified all PowerShell scripts follow PSScriptAnalyzer rules 2. ✅ Tested `Get-PrIncrementalChanges.ps1` with various scenarios: - First review (no previous SHA) - No changes (SHA matches current) - Force-push detection (SHA not in history) - Incremental changes (multiple commits and files) 3. ✅ Validated `Test-IncrementalReview.ps1` output formatting and accuracy 4. ✅ Confirmed `Migrate-ReviewToIncrementalFormat.ps1` adds metadata without corrupting existing reviews 5. ✅ Tested GitHub API integration with `Get-GitHubRawFile.ps1` and `Get-GitHubPrFilePatch.ps1` 6. ✅ Verified documentation accuracy and completeness in `review-tools.instructions.md` 7. ✅ Validated prompt templates follow PowerToys repo conventions from `.github/copilot-instructions.md` **Dependencies Verified:** - Requires PowerShell 7+ (or Windows PowerShell 5.1+) - Requires GitHub CLI (`gh`) installed and authenticated - All scripts include comprehensive help documentation (`Get-Help` support) **No Automated Tests:** These are development/CI tooling scripts used by maintainers for PR reviews. Manual validation appropriate as they interact with live GitHub API and require human judgment for review quality assessment.
314 lines
10 KiB
Markdown
314 lines
10 KiB
Markdown
---
|
|
description: PowerShell scripts for efficient PR reviews in PowerToys repository
|
|
applyTo: '**'
|
|
---
|
|
|
|
# PR Review Tools - Reference Guide
|
|
|
|
PowerShell scripts to support efficient and incremental pull request reviews in the PowerToys repository.
|
|
|
|
## Quick Start
|
|
|
|
### Prerequisites
|
|
- PowerShell 7+ (or Windows PowerShell 5.1+)
|
|
- GitHub CLI (`gh`) installed and authenticated (`gh auth login`)
|
|
- Access to the PowerToys repository
|
|
|
|
### Testing Your Setup
|
|
|
|
Run the full test suite (recommended):
|
|
```powershell
|
|
cd "d:\PowerToys-00c1\.github\review-tools"
|
|
.\Run-ReviewToolsTests.ps1
|
|
```
|
|
|
|
Expected: 9-10 tests passing
|
|
|
|
### Individual Script Tests
|
|
|
|
**Test incremental change detection:**
|
|
```powershell
|
|
.\Get-PrIncrementalChanges.ps1 -PullRequestNumber 42374
|
|
```
|
|
Expected: JSON output showing review analysis
|
|
|
|
**Preview incremental review:**
|
|
```powershell
|
|
.\Test-IncrementalReview.ps1 -PullRequestNumber 42374
|
|
```
|
|
Expected: Analysis showing current vs last reviewed SHA
|
|
|
|
**Fetch file content:**
|
|
```powershell
|
|
.\Get-GitHubRawFile.ps1 -FilePath "README.md" -GitReference "main"
|
|
```
|
|
Expected: README content displayed
|
|
|
|
**Get PR file patch:**
|
|
```powershell
|
|
.\Get-GitHubPrFilePatch.ps1 -PullRequestNumber 42374 -FilePath ".github/actions/spell-check/expect.txt"
|
|
```
|
|
Expected: Unified diff output
|
|
|
|
## Available Scripts
|
|
|
|
### Get-GitHubRawFile.ps1
|
|
|
|
Downloads and displays file content from a GitHub repository at a specific git reference.
|
|
|
|
**Purpose:** Retrieve baseline file content for comparison during PR reviews.
|
|
|
|
**Parameters:**
|
|
- `FilePath` (required): Relative path to file in repository
|
|
- `GitReference` (optional): Git ref (branch, tag, SHA). Default: "main"
|
|
- `RepositoryOwner` (optional): Repository owner. Default: "microsoft"
|
|
- `RepositoryName` (optional): Repository name. Default: "PowerToys"
|
|
- `ShowLineNumbers` (switch): Prefix each line with line number
|
|
- `StartLineNumber` (optional): Starting line number when using `-ShowLineNumbers`. Default: 1
|
|
|
|
**Usage:**
|
|
```powershell
|
|
.\Get-GitHubRawFile.ps1 -FilePath "src/runner/main.cpp" -GitReference "main" -ShowLineNumbers
|
|
```
|
|
|
|
### Get-GitHubPrFilePatch.ps1
|
|
|
|
Fetches the unified diff (patch) for a specific file in a pull request.
|
|
|
|
**Purpose:** Get the exact changes made to a file in a PR for detailed review.
|
|
|
|
**Parameters:**
|
|
- `PullRequestNumber` (required): Pull request number
|
|
- `FilePath` (required): Relative path to file in the PR
|
|
- `RepositoryOwner` (optional): Repository owner. Default: "microsoft"
|
|
- `RepositoryName` (optional): Repository name. Default: "PowerToys"
|
|
|
|
**Usage:**
|
|
```powershell
|
|
.\Get-GitHubPrFilePatch.ps1 -PullRequestNumber 42374 -FilePath "src/modules/cmdpal/main.cpp"
|
|
```
|
|
|
|
**Output:** Unified diff showing changes made to the file.
|
|
|
|
### Get-PrIncrementalChanges.ps1
|
|
|
|
Compares the last reviewed commit with the current PR head to identify incremental changes.
|
|
|
|
**Purpose:** Enable efficient incremental reviews by detecting what changed since the last review iteration.
|
|
|
|
**Parameters:**
|
|
- `PullRequestNumber` (required): Pull request number
|
|
- `LastReviewedCommitSha` (optional): SHA of the commit that was last reviewed. If omitted, assumes first review.
|
|
- `RepositoryOwner` (optional): Repository owner. Default: "microsoft"
|
|
- `RepositoryName` (optional): Repository name. Default: "PowerToys"
|
|
|
|
**Usage:**
|
|
```powershell
|
|
.\Get-PrIncrementalChanges.ps1 -PullRequestNumber 42374 -LastReviewedCommitSha "abc123def456"
|
|
```
|
|
|
|
**Output:** JSON object with detailed change analysis:
|
|
```json
|
|
{
|
|
"PullRequestNumber": 42374,
|
|
"CurrentHeadSha": "xyz789abc123",
|
|
"LastReviewedSha": "abc123def456",
|
|
"IsIncremental": true,
|
|
"NeedFullReview": false,
|
|
"ChangedFiles": [
|
|
{
|
|
"Filename": "src/modules/cmdpal/main.cpp",
|
|
"Status": "modified",
|
|
"Additions": 15,
|
|
"Deletions": 8,
|
|
"Changes": 23
|
|
}
|
|
],
|
|
"NewCommits": [
|
|
{
|
|
"Sha": "def456",
|
|
"Message": "Fix memory leak",
|
|
"Author": "John Doe",
|
|
"Date": "2025-11-07T10:30:00Z"
|
|
}
|
|
],
|
|
"Summary": "Incremental review: 1 new commit(s), 1 file(s) changed since SHA abc123d"
|
|
}
|
|
```
|
|
|
|
**Scenarios Handled:**
|
|
- **No LastReviewedCommitSha**: Returns `NeedFullReview: true` (first review)
|
|
- **SHA matches current HEAD**: Returns empty `ChangedFiles` (no changes)
|
|
- **Force-push detected**: Returns `NeedFullReview: true` (SHA not in history)
|
|
- **Incremental changes**: Returns list of changed files and new commits
|
|
|
|
### Test-IncrementalReview.ps1
|
|
|
|
Helper script to test and preview incremental review detection before running the full review.
|
|
|
|
**Purpose:** Validate incremental review functionality and preview what changed.
|
|
|
|
**Parameters:**
|
|
- `PullRequestNumber` (required): Pull request number
|
|
- `RepositoryOwner` (optional): Repository owner. Default: "microsoft"
|
|
- `RepositoryName` (optional): Repository name. Default: "PowerToys"
|
|
|
|
**Usage:**
|
|
```powershell
|
|
.\Test-IncrementalReview.ps1 -PullRequestNumber 42374
|
|
```
|
|
|
|
**Output:** Colored console output showing:
|
|
- Current and last reviewed SHAs
|
|
- Whether incremental review is possible
|
|
- List of new commits and changed files
|
|
- Recommended review strategy
|
|
|
|
## Workflow Integration
|
|
|
|
These scripts integrate with the PR review prompt (`.github/prompts/review-pr.prompt.md`).
|
|
|
|
### Typical Review Flow
|
|
|
|
1. **Initial Review (Iteration 1)**
|
|
- Review prompt processes the PR
|
|
- Creates `Generated Files/prReview/{PR}/00-OVERVIEW.md`
|
|
- Includes review metadata section with current HEAD SHA
|
|
|
|
2. **Subsequent Reviews (Iteration 2+)**
|
|
- Review prompt reads `00-OVERVIEW.md` to get last reviewed SHA
|
|
- Calls `Get-PrIncrementalChanges.ps1` to detect what changed
|
|
- If incremental:
|
|
- Reviews only changed files
|
|
- Skips irrelevant review steps (e.g., skip Localization if no `.resx` files changed)
|
|
- Uses `Get-GitHubPrFilePatch.ps1` to get patches for changed files
|
|
- Updates `00-OVERVIEW.md` with new SHA and iteration number
|
|
|
|
### Manual Testing Workflow
|
|
|
|
Preview changes before review:
|
|
```powershell
|
|
# Check what changed in PR #42374 since last review
|
|
.\Test-IncrementalReview.ps1 -PullRequestNumber 42374
|
|
|
|
# Get incremental changes programmatically
|
|
$changes = .\Get-PrIncrementalChanges.ps1 -PullRequestNumber 42374 -LastReviewedCommitSha "abc123" | ConvertFrom-Json
|
|
|
|
if (-not $changes.NeedFullReview) {
|
|
Write-Host "Only need to review $($changes.ChangedFiles.Count) files"
|
|
|
|
# Review each changed file
|
|
foreach ($file in $changes.ChangedFiles) {
|
|
Write-Host "Reviewing $($file.Filename)..."
|
|
.\Get-GitHubPrFilePatch.ps1 -PullRequestNumber 42374 -FilePath $file.Filename
|
|
}
|
|
}
|
|
```
|
|
|
|
## Error Handling and Troubleshooting
|
|
|
|
### Common Requirements
|
|
|
|
All scripts:
|
|
- Exit with code 1 on error
|
|
- Write detailed error messages to stderr
|
|
- Require `gh` CLI to be installed and authenticated
|
|
|
|
### Common Issues
|
|
|
|
**Error: "gh not found"**
|
|
- **Solution**: Install GitHub CLI from https://cli.github.com/ and run `gh auth login`
|
|
|
|
**Error: "Failed to query GitHub API"**
|
|
- **Solution**: Verify `gh` authentication with `gh auth status`
|
|
- **Solution**: Check PR number exists and you have repository access
|
|
|
|
**Error: "PR not found"**
|
|
- **Solution**: Verify the PR number is correct and still exists
|
|
- **Solution**: Ensure repository owner and name are correct
|
|
|
|
**Error: "SHA not found" or "Force-push detected"**
|
|
- **Explanation**: Last reviewed SHA no longer exists in branch history (force-push occurred)
|
|
- **Solution**: A full review is required; incremental review not possible
|
|
|
|
**Tests show "FAIL" but functionality works**
|
|
- **Explanation**: Some tests may show exit code failures even when logic is correct
|
|
- **Solution**: Check test output message - if it says "Correctly detected", functionality is working
|
|
|
|
**Error: "Could not find insertion point"**
|
|
- **Explanation**: Overview file doesn't have expected "**Changed files:**" line
|
|
- **Solution**: Verify overview file format is correct or regenerate it
|
|
|
|
### Verification Checklist
|
|
|
|
After setup, verify:
|
|
- [ ] `Run-ReviewToolsTests.ps1` shows 9+ tests passing
|
|
- [ ] `Get-PrIncrementalChanges.ps1` returns valid JSON
|
|
- [ ] `Test-IncrementalReview.ps1` analyzes a PR without errors
|
|
- [ ] `Get-GitHubRawFile.ps1` downloads files correctly
|
|
- [ ] `Get-GitHubPrFilePatch.ps1` retrieves patches correctly
|
|
|
|
## Best Practices
|
|
|
|
### For Review Authors
|
|
|
|
1. **Test before full review**: Use `Test-IncrementalReview.ps1` to preview changes
|
|
2. **Check for force-push**: Review the analysis output - force-pushes require full reviews
|
|
3. **Smart step filtering**: Skip review steps for file types that didn't change
|
|
|
|
### For Script Users
|
|
|
|
1. **Use absolute paths**: When specifying folders, use absolute paths to avoid ambiguity
|
|
2. **Check exit codes**: Scripts exit with code 1 on error - check `$LASTEXITCODE` in automation
|
|
3. **Parse JSON output**: Use `ConvertFrom-Json` to work with structured output from `Get-PrIncrementalChanges.ps1`
|
|
4. **Handle empty results**: Check `ChangedFiles.Count` before iterating
|
|
|
|
### Performance Tips
|
|
|
|
1. **Batch operations**: When reviewing multiple PRs, collect all PR numbers and process in batch
|
|
2. **Cache raw files**: Download baseline files once and reuse for multiple comparisons
|
|
3. **Filter early**: Use incremental detection to skip unnecessary file reviews
|
|
4. **Parallel processing**: Consider processing independent PRs in parallel
|
|
|
|
## Integration with AI Review Systems
|
|
|
|
These tools are designed to work with AI-powered review systems:
|
|
|
|
1. **Copilot Instructions**: This file serves as reference documentation for GitHub Copilot
|
|
2. **Structured Output**: JSON output from scripts is easily parsed by AI systems
|
|
3. **Incremental Intelligence**: AI can focus on changed files for more efficient reviews
|
|
4. **Metadata Tracking**: Review iterations are tracked for context-aware suggestions
|
|
|
|
### Example AI Integration
|
|
|
|
```powershell
|
|
# Get incremental changes
|
|
$analysis = .\Get-PrIncrementalChanges.ps1 -PullRequestNumber $PR | ConvertFrom-Json
|
|
|
|
# Feed to AI review system
|
|
$reviewPrompt = @"
|
|
Review the following changed files in PR #$PR:
|
|
$($analysis.ChangedFiles | ForEach-Object { "- $($_.Filename) ($($_.Status))" } | Out-String)
|
|
|
|
Focus on incremental changes only. Previous review was at SHA $($analysis.LastReviewedSha).
|
|
"@
|
|
|
|
# Execute AI review with context
|
|
Invoke-AIReview -Prompt $reviewPrompt -Files $analysis.ChangedFiles
|
|
```
|
|
|
|
## Support and Further Information
|
|
|
|
For detailed script documentation, use PowerShell's help system:
|
|
```powershell
|
|
Get-Help .\Get-PrIncrementalChanges.ps1 -Full
|
|
Get-Help .\Test-IncrementalReview.ps1 -Detailed
|
|
```
|
|
|
|
Related documentation:
|
|
- `.github/prompts/review-pr.prompt.md` - Complete review workflow guide
|
|
- `doc/devdocs/` - PowerToys development documentation
|
|
- GitHub CLI documentation: https://cli.github.com/manual/
|
|
|
|
For issues or questions, refer to the PowerToys contribution guidelines.
|