Files
PowerToys/.github/skills/ui-tests-pipeline-ci/scripts/Test-AzureDevOpsSetup.ps1
Gleb Khmyznikov bea1b8e247 [UITests][PowerRename] Migrate to new .Next and add more UI tests (#50096)
## Summary of the Pull Request

Adds a `PowerRename.UITests.Next` suite powered by winappcli and
automates all 18 scenarios from #40663. The suite covers PowerRename
settings, search and replace behavior, regular expressions, formatting
and filtering options, file-list interactions, and both classic and
Windows 11 context-menu workflows.

The PR also stabilizes shared `UITestAutomation.Next` runner lifetimes
and settings restoration, adds automation IDs for the original and
renamed counters, and prepares unsigned CI builds for PowerRename shell
testing. CI now signs the sparse context-menu MSIX and the
runner/Settings IPC companions with a disposable machine-trusted test
identity.

## PR Checklist

- [x] Closes: #40663
- [x] **Communication:** I've discussed this with core contributors
already. If the work hasn't been agreed, this work might be rejected
- [x] **Tests:** Added/updated and all pass
- [x] **Localization:** All end-user-facing strings can be localized
- [x] **Dev docs:** Added/updated
- [x] **New binaries:** Not applicable; no new shipped product binaries
are added
   - [x] JSON for signing: Not applicable
   - [x] WXS for installer: Not applicable
- [x] YML for CI pipeline: The new UI-test project is discovered through
the existing `*UITest*.csproj` pipeline flow and is registered in
`PowerToys.slnx`
   - [x] YML for signed pipeline: Not applicable
- [x] **Documentation updated:** Not applicable; there are no
user-facing behavior or documentation changes

## Detailed Description of the Pull Request / Additional comments

### PowerRename UI tests

- Adds a 25-case `PowerRename.UITests.Next` executable covering all 18
checklist items from #40663.
- Exercises classic context-menu registration on Windows 10 and Windows
11.
- Exercises the signed Windows 11 tier-1 context menu, including icon
visibility and real invocation with an Explorer selection.
- Covers search/replace preview and application, text formatting,
file/folder/subfolder inclusion, filename/extension scope, enumeration,
case sensitivity, match-all behavior, regular expressions, file
timestamps, Boost syntax, MRU autocomplete, persisted values, and
file-list selection/filtering.
- Preserves the existing legacy tests.

### Test reliability and automation hooks

- Reuses one runner/Settings lifetime across the complete PowerRename
suite to avoid repeated cold launches on constrained agents.
- Retains and restores global settings for the full class lifetime and
verifies that both Settings and the runner remain healthy.
- Propagates scope and PowerRename cleanup failures instead of silently
leaking process or profile state.
- Adds `OriginalCount` and `RenamedCount` automation IDs. These are
automation-only metadata and do not change the visible UI.
- Uses stable preview samples, exact count targeting, authoritative
Explorer selection, readable classic-menu inventories, and live UIA
visibility for popup items.

### Unsigned CI build support

- Extends the existing sparse-package test signer with required
Authenticode companion files.
- PowerRename jobs sign `PowerToys.exe` and `PowerToys.Settings.exe`
with the same disposable machine-trusted test identity used for sparse
MSIX packages.
- This preserves Release IPC authentication while allowing Settings
module-toggle commands to work on unsigned PR builds.
- Windows 11 and ARM64 PowerRename jobs require a validly signed
`PowerRenameContextMenuPackage.msix` before tests start.

## Validation Steps Performed

### Builds and discovery

- `PowerRename.UITests.Next` Debug x64 build: passed
- `PowerRename.UITests.Next` Debug ARM64 cross-build: passed
- `PowerRenameUI` Release x64 build: passed
- `UITestAutomation.Next.UnitTests` Debug x64 build: passed
- `UITestAutomation.Next.UnitTests`: 16/16 passed
- Microsoft.Testing.Platform discovery: 25 unique PowerRename test cases

### Local VM matrix

All runs used the complete unfiltered `TestCategory=PowerRename` suite
and restored the standard user's settings file byte-for-byte.

| Guest | Profile | Result |
|---|---|---:|
| Windows 10 x64 | Default, 4 vCPU / 8 GB | 25/25 | 
| Windows 10 x64 | Constrained, 1 vCPU / 4 GB | 25/25 | 
| Windows 11 x64 | Default, 4 vCPU / 8 GB | 25/25 | 
| Windows 11 x64 | Constrained, 1 vCPU / 4 GB | 25/25 | 

### Azure DevOps UI Test Automation

- Final build: [155646235 /
20260824.1](https://dev.azure.com/microsoft/Dart/_build/results?buildId=155646235)
- Source revision: `4496683104aea622b30179909bd6e95a17d5500f`
- ARM64: 25/25 passed
- Windows 10 x64: 25/25 passed
- Windows 11 x64: 25/25 passed
- Total: 75/75 passed, with zero failed, skipped, not-executed, or
unanalyzed results
- ARM64 and x64 Release product builds succeeded and published their
normal artifacts
- Signing steps verified the PowerRename sparse MSIX where applicable
and both IPC companion executables on every PowerRename test job

<img width="372" height="314" alt="image"
src="https://github.com/user-attachments/assets/bc303673-0325-4f85-8d70-ef93110baf5b"
/>
2026-08-26 00:26:10 +02:00

16 KiB