2 Commits

Author	SHA1	Message	Date
Gleb Khmyznikov	bea1b8e247	[UITests][PowerRename] Migrate to new .Next and add more UI tests (#50096) ... ## Summary of the Pull Request Adds a `PowerRename.UITests.Next` suite powered by winappcli and automates all 18 scenarios from #40663. The suite covers PowerRename settings, search and replace behavior, regular expressions, formatting and filtering options, file-list interactions, and both classic and Windows 11 context-menu workflows. The PR also stabilizes shared `UITestAutomation.Next` runner lifetimes and settings restoration, adds automation IDs for the original and renamed counters, and prepares unsigned CI builds for PowerRename shell testing. CI now signs the sparse context-menu MSIX and the runner/Settings IPC companions with a disposable machine-trusted test identity. ## PR Checklist - [x] Closes: #40663 - [x] **Communication:** I've discussed this with core contributors already. If the work hasn't been agreed, this work might be rejected - [x] **Tests:** Added/updated and all pass - [x] **Localization:** All end-user-facing strings can be localized - [x] **Dev docs:** Added/updated - [x] **New binaries:** Not applicable; no new shipped product binaries are added - [x] JSON for signing: Not applicable - [x] WXS for installer: Not applicable - [x] YML for CI pipeline: The new UI-test project is discovered through the existing `*UITest*.csproj` pipeline flow and is registered in `PowerToys.slnx` - [x] YML for signed pipeline: Not applicable - [x] **Documentation updated:** Not applicable; there are no user-facing behavior or documentation changes ## Detailed Description of the Pull Request / Additional comments ### PowerRename UI tests - Adds a 25-case `PowerRename.UITests.Next` executable covering all 18 checklist items from #40663. - Exercises classic context-menu registration on Windows 10 and Windows 11. - Exercises the signed Windows 11 tier-1 context menu, including icon visibility and real invocation with an Explorer selection. - Covers search/replace preview and application, text formatting, file/folder/subfolder inclusion, filename/extension scope, enumeration, case sensitivity, match-all behavior, regular expressions, file timestamps, Boost syntax, MRU autocomplete, persisted values, and file-list selection/filtering. - Preserves the existing legacy tests. ### Test reliability and automation hooks - Reuses one runner/Settings lifetime across the complete PowerRename suite to avoid repeated cold launches on constrained agents. - Retains and restores global settings for the full class lifetime and verifies that both Settings and the runner remain healthy. - Propagates scope and PowerRename cleanup failures instead of silently leaking process or profile state. - Adds `OriginalCount` and `RenamedCount` automation IDs. These are automation-only metadata and do not change the visible UI. - Uses stable preview samples, exact count targeting, authoritative Explorer selection, readable classic-menu inventories, and live UIA visibility for popup items. ### Unsigned CI build support - Extends the existing sparse-package test signer with required Authenticode companion files. - PowerRename jobs sign `PowerToys.exe` and `PowerToys.Settings.exe` with the same disposable machine-trusted test identity used for sparse MSIX packages. - This preserves Release IPC authentication while allowing Settings module-toggle commands to work on unsigned PR builds. - Windows 11 and ARM64 PowerRename jobs require a validly signed `PowerRenameContextMenuPackage.msix` before tests start. ## Validation Steps Performed ### Builds and discovery - `PowerRename.UITests.Next` Debug x64 build: passed - `PowerRename.UITests.Next` Debug ARM64 cross-build: passed - `PowerRenameUI` Release x64 build: passed - `UITestAutomation.Next.UnitTests` Debug x64 build: passed - `UITestAutomation.Next.UnitTests`: 16/16 passed - Microsoft.Testing.Platform discovery: 25 unique PowerRename test cases ### Local VM matrix All runs used the complete unfiltered `TestCategory=PowerRename` suite and restored the standard user's settings file byte-for-byte. | Guest | Profile | Result | |---|---|---:| | Windows 10 x64 | Default, 4 vCPU / 8 GB | 25/25 | | Windows 10 x64 | Constrained, 1 vCPU / 4 GB | 25/25 | | Windows 11 x64 | Default, 4 vCPU / 8 GB | 25/25 | | Windows 11 x64 | Constrained, 1 vCPU / 4 GB | 25/25 | ### Azure DevOps UI Test Automation - Final build: [155646235 / 20260824.1](https://dev.azure.com/microsoft/Dart/_build/results?buildId=155646235) - Source revision: `4496683104aea622b30179909bd6e95a17d5500f` - ARM64: 25/25 passed - Windows 10 x64: 25/25 passed - Windows 11 x64: 25/25 passed - Total: 75/75 passed, with zero failed, skipped, not-executed, or unanalyzed results - ARM64 and x64 Release product builds succeeded and published their normal artifacts - Signing steps verified the PowerRename sparse MSIX where applicable and both IPC companion executables on every PowerRename test job <img width="372" height="314" alt="image" src="https://github.com/user-attachments/assets/bc303673-0325-4f85-8d70-ef93110baf5b" />	2026-08-26 00:26:10 +02:00
Gleb Khmyznikov	5759a62d4a	[UITests][Keyboard Manager] UI tests for new keyboard manager. (#50059) ... <!-- Suggested title: [UITests][Keyboard Manager] Add and stabilize UI tests for the unified editor --> ## Summary of the Pull Request Closes: #40662 Adds a new `Microsoft.PowerToys.UITest.Next` end-to-end suite for Keyboard Manager and hardens the unified editor behavior uncovered while exercising it on Windows 10, Windows 11, x64, and ARM64. The Keyboard Manager suite executes 32 test cases covering: - Unified editor create, edit, save, enable/disable, delete, restart persistence, missing-profile recovery, validation, and special actions. - Single-key, key-to-shortcut, shortcut-to-shortcut, shortcut-to-key, disabled-key, and app-specific remapping. - Modifier ordering and release behavior, including Alt+Tab and Alt+F4 targets. - Real keyboard behavior through Windows Notepad and Calculator fixtures rather than synthetic input windows. The PR also adds 26 managed unit tests around settings normalization, profile reconciliation, metadata ownership, active-state changes, and canonical modifier ordering. ## Detailed Description of the Pull Request / Additional comments ### Keyboard Manager UI tests Adds `src/modules/keyboardmanager/Tests/KeyboardManager.UITests` as a Microsoft Testing Platform executable using `UITestAutomation.Next` and winappcli. The suite includes: - `KeyboardManager.Editor.CreateEditPersistDelete` - Creates and edits mappings. - Verifies native profile and editor metadata persistence across restarts. - Exercises active-state toggling, row deletion, and missing native-profile recovery. - `KeyboardManager.Editor.InputAndValidation` - Covers key recording, dropdown input, keyboard navigation, cancellation, and app-specific validation. - `KeyboardManager.Editor.ActionPersistence` - Covers Open URL, Open app, and Insert text actions and verifies canonical readback after restart. - Parameterized runtime tests for key/shortcut combinations, disabled targets, app-specific mappings, modifier release order, Alt+Tab, and Alt+F4. The test support code provides: - A shared cross-process fixture lock and isolated Keyboard Manager settings scope. - Real Notepad documents with exact window/document ownership and cleanup. - Win10/Win11-aware Calculator window ownership for Alt+F4 assertions. - A low-level keyboard event recorder that validates injected key-down/key-up sequences and flags. - Authoritative persisted JSON assertions instead of relying only on transient UI state. ### Keyboard Manager correctness fixes found by the suite The tests exposed product races and persistence issues that are fixed in the same PR: - Native profile JSON is written through checked same-directory atomic replacement instead of a truncation-prone direct write. - Editor metadata and native mappings are normalized and reconciled per profile. - Create, edit, delete, and active-state mutations commit metadata and native state transactionally under a cross-process lock. - Startup reconciliation preserves inactive metadata owned by other profiles and repairs legacy/profileless settings. - Native Boolean return values use one-byte marshaling to match the C++ ABI. - Modifier keys and serialized targets are canonicalized consistently. - Unit-test initialization no longer starts real settings synchronization. ### CI stability hardening The editor workflows use coordinate-free UIA invocation for command buttons and authoritative-signal retries for idempotent row interactions. Window fixtures bind exact top-level HWNDs and distinguish the Win10 Calculator `ApplicationFrameWindow` from its child content window. The child-specific `UITestAutomation.Next` updates add or improve: - `WindowShowWatcher` lifecycle handling used by the Keyboard Manager window fixtures. - Exact HWND foreground, close, and process-tree cleanup helpers required by the Win10/Win11 tests. ### Pipeline workflow Adds the internal `ui-tests-pipeline-ci` skill and Azure DevOps helper used during stabilization: - Uses an existing Azure CLI session plus Azure DevOps REST APIs without per-call authentication prompts. - Supports paged branch/build discovery, preview and queue operations, exact-SHA reconciliation, stage retry/cancel, logs, test results, artifacts, and direct result-attachment downloads. - Uses build-scoped one-shot completion monitoring and retains the three-run stabilization guardrail. ## Validation Steps Performed ### Azure DevOps UI Test Automation Final verification build: - https://microsoft.visualstudio.com/Dart/_build/results?buildId=155512681&view=results - https://microsoft.visualstudio.com/Dart/_build/results?buildId=155525972&view=results ## Reviewer guide Suggested review order: 1. `src/modules/keyboardmanager/Tests/KeyboardManager.UITests/` - intended workflows and assertions. 2. `KeyboardManagerEditorUI/Settings/SettingsManager.cs` and `Pages/MainPage.xaml.cs` - transaction and reconciliation ownership. 3. `common/MappingConfiguration.cpp` and `Interop/KeyboardManagerInterop.cs` - native persistence and ABI fixes. 4. `src/common/UITestAutomation.Next/WindowControl.cs` and `WindowShowWatcher.cs` - child-specific window lifecycle hardening. 5. `.github/skills/ui-tests-pipeline-ci/` - internal Azure CLI/REST stabilization workflow. Local evidence: <img width="1237" height="854" alt="image" src="https://github.com/user-attachments/assets/ff49652e-fbc8-4581-a48f-836dbed37b6e" />	2026-08-21 17:35:13 -07:00