2 Commits

Author	SHA1	Message	Date
Gleb Khmyznikov	bea1b8e247	[UITests][PowerRename] Migrate to new .Next and add more UI tests (#50096) ... ## Summary of the Pull Request Adds a `PowerRename.UITests.Next` suite powered by winappcli and automates all 18 scenarios from #40663. The suite covers PowerRename settings, search and replace behavior, regular expressions, formatting and filtering options, file-list interactions, and both classic and Windows 11 context-menu workflows. The PR also stabilizes shared `UITestAutomation.Next` runner lifetimes and settings restoration, adds automation IDs for the original and renamed counters, and prepares unsigned CI builds for PowerRename shell testing. CI now signs the sparse context-menu MSIX and the runner/Settings IPC companions with a disposable machine-trusted test identity. ## PR Checklist - [x] Closes: #40663 - [x] **Communication:** I've discussed this with core contributors already. If the work hasn't been agreed, this work might be rejected - [x] **Tests:** Added/updated and all pass - [x] **Localization:** All end-user-facing strings can be localized - [x] **Dev docs:** Added/updated - [x] **New binaries:** Not applicable; no new shipped product binaries are added - [x] JSON for signing: Not applicable - [x] WXS for installer: Not applicable - [x] YML for CI pipeline: The new UI-test project is discovered through the existing `*UITest*.csproj` pipeline flow and is registered in `PowerToys.slnx` - [x] YML for signed pipeline: Not applicable - [x] **Documentation updated:** Not applicable; there are no user-facing behavior or documentation changes ## Detailed Description of the Pull Request / Additional comments ### PowerRename UI tests - Adds a 25-case `PowerRename.UITests.Next` executable covering all 18 checklist items from #40663. - Exercises classic context-menu registration on Windows 10 and Windows 11. - Exercises the signed Windows 11 tier-1 context menu, including icon visibility and real invocation with an Explorer selection. - Covers search/replace preview and application, text formatting, file/folder/subfolder inclusion, filename/extension scope, enumeration, case sensitivity, match-all behavior, regular expressions, file timestamps, Boost syntax, MRU autocomplete, persisted values, and file-list selection/filtering. - Preserves the existing legacy tests. ### Test reliability and automation hooks - Reuses one runner/Settings lifetime across the complete PowerRename suite to avoid repeated cold launches on constrained agents. - Retains and restores global settings for the full class lifetime and verifies that both Settings and the runner remain healthy. - Propagates scope and PowerRename cleanup failures instead of silently leaking process or profile state. - Adds `OriginalCount` and `RenamedCount` automation IDs. These are automation-only metadata and do not change the visible UI. - Uses stable preview samples, exact count targeting, authoritative Explorer selection, readable classic-menu inventories, and live UIA visibility for popup items. ### Unsigned CI build support - Extends the existing sparse-package test signer with required Authenticode companion files. - PowerRename jobs sign `PowerToys.exe` and `PowerToys.Settings.exe` with the same disposable machine-trusted test identity used for sparse MSIX packages. - This preserves Release IPC authentication while allowing Settings module-toggle commands to work on unsigned PR builds. - Windows 11 and ARM64 PowerRename jobs require a validly signed `PowerRenameContextMenuPackage.msix` before tests start. ## Validation Steps Performed ### Builds and discovery - `PowerRename.UITests.Next` Debug x64 build: passed - `PowerRename.UITests.Next` Debug ARM64 cross-build: passed - `PowerRenameUI` Release x64 build: passed - `UITestAutomation.Next.UnitTests` Debug x64 build: passed - `UITestAutomation.Next.UnitTests`: 16/16 passed - Microsoft.Testing.Platform discovery: 25 unique PowerRename test cases ### Local VM matrix All runs used the complete unfiltered `TestCategory=PowerRename` suite and restored the standard user's settings file byte-for-byte. | Guest | Profile | Result | |---|---|---:| | Windows 10 x64 | Default, 4 vCPU / 8 GB | 25/25 | | Windows 10 x64 | Constrained, 1 vCPU / 4 GB | 25/25 | | Windows 11 x64 | Default, 4 vCPU / 8 GB | 25/25 | | Windows 11 x64 | Constrained, 1 vCPU / 4 GB | 25/25 | ### Azure DevOps UI Test Automation - Final build: [155646235 / 20260824.1](https://dev.azure.com/microsoft/Dart/_build/results?buildId=155646235) - Source revision: `4496683104aea622b30179909bd6e95a17d5500f` - ARM64: 25/25 passed - Windows 10 x64: 25/25 passed - Windows 11 x64: 25/25 passed - Total: 75/75 passed, with zero failed, skipped, not-executed, or unanalyzed results - ARM64 and x64 Release product builds succeeded and published their normal artifacts - Signing steps verified the PowerRename sparse MSIX where applicable and both IPC companion executables on every PowerRename test job <img width="372" height="314" alt="image" src="https://github.com/user-attachments/assets/bc303673-0325-4f85-8d70-ef93110baf5b" />	2026-08-26 00:26:10 +02:00
Gleb Khmyznikov	e48152c52d	[UITests] Add UITest.Next suites (Image Resizer, Peek, File Explorer add‑ons, File Locksmith) + local‑VM tooling and CI test‑signing (#49671) ... ## Summary Adds end‑to‑end UI tests on the `Microsoft.PowerToys.UITest.Next` (winappcli) framework for three modules, grows the shared `.Next` test framework with the helpers those suites needed, and adds the CI plumbing that lets shell‑extension tests exercise the **real** Windows 11 modern context menu. Also ships two agent skills that document how to write and run these tests. Product runtime behavior is **unchanged** — the only product edits are test‑observability hooks in Peek and a unit‑test project exclude. Closes: https://github.com/microsoft/PowerToys/issues/40660 https://github.com/microsoft/PowerToys/issues/49424 https://github.com/microsoft/PowerToys/issues/40661 ## What's added ### New UI test suites - **Image Resizer** — `src/modules/imageresizer/tests/ImageResizer.UITests`: context‑menu enable/disable tracking, the resize dialog, custom presets, every fit mode, every unit, filename format, keep‑date, shrink‑only, replace‑in‑place, and orientation. - **Peek** — `src/modules/peek/Peek.UITests.Next`: file‑preview coverage across image/text/archive/ markdown types with per‑arch visual baselines. - **File Explorer add‑ons** — `src/modules/previewpane/PreviewPane.UITests`: Preview Pane handlers and thumbnail providers. ### `UITestAutomation.Next` framework - New helpers: `ExplorerShell` (Shell selection/view‑mode interop), `WaitHelper` (structured stable waits), `WindowControl` (foreground/context‑menu/process control), `VisualAssert` (image compare), `WindowHelper`. - Updates to `Session`, `UITestBase`, `SettingsConfigHelper`, `WinappCli`. - New `UITestAutomation.Next.UnitTests` project covering the new wait/settings/CLI helpers. ### CI — sign sparse MSIX so the modern menu registers - **`.pipelines/signSparsePackages.ps1`** — self‑signs each sparse context‑menu MSIX with a publisher‑matching test certificate and force‑trusts it (machine stores), so `AddPackageByUriAsync` succeeds on otherwise‑unsigned PR builds. Robust `signtool` discovery with a NuGet fallback; test‑only trust that asserts no security. - Wired into **`.pipelines/v2/templates/job-test-project.yml`** as a best‑effort step covering the run‑in‑place, machine‑install, and per‑user‑install locations. Signs nothing it can't (skips already‑signed packages) and never fails the job. ### Product changes (test observability only) - **Peek `FilePreview.xaml` / `.xaml.cs`** — a named `LoadingIndicator` and a hidden automation peer that exposes the current preview state as text, so tests can read load state deterministically. No runtime behavior change. - **`ImageResizer.UnitTests.csproj`** — exclude the sibling `ImageResizer.UITests\**` folder from the unit‑test compilation. ### Agent skills & docs - **New `ui-tests-local-vm` skill** — run `.Next` suites in persistent dockur/windows VMs: setup, agentic loop, image customization, troubleshooting, the shell‑extension **signing** reference, plus controller/guest scripts and VM templates. - **Updated `ui-tests-migration` skill** — WinAppDriver/Selenium → `.Next` porting guidance (CI stability, Explorer/shell‑extension test design, patterns & pitfalls). - **`doc/devdocs/development/ui-tests.md`** — updated for the `.Next` workflow. ## Testing - All three suites pass locally and in CI across **x64 Win10**, **x64 Win11**, and **arm64** (machine and per‑user install legs). ## Reviewer notes - No product runtime behavior changes; product edits are limited to the Peek test hooks above. - The CI signing step is a **test‑only** trust anchor (self‑signed, scoped to the agent) and is best‑effort, so it can only add modern‑menu coverage and never regress the job.	2026-08-10 19:55:03 +02:00