[UITests] AI migration skill + ScreenRuler tests example port. (#48842)

This PR is build on top of
https://github.com/microsoft/PowerToys/pull/48717 and
https://github.com/microsoft/PowerToys/pull/48467

It adds the ui-tests-migration skill which should be used to migrate the
old UITest module by module to use new .Next UI framework.

As an example, the screen ruller tests was ported here.

---------

Co-authored-by: Boliang Zhang (from Dev Box) <bozhang@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Gleb Khmyznikov
2026-07-08 20:02:54 -07:00
committed by GitHub
parent 4395b1d0ca
commit 85f698677a
43 changed files with 3964 additions and 436 deletions

View File

@@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to the Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2026 Microsoft Corporation
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

View File

@@ -0,0 +1,208 @@
---
name: ui-tests-migration
description: "Migrate PowerToys module UI tests from the legacy WinAppDriver/Selenium harness (Microsoft.PowerToys.UITest) to the new winappcli-based harness (Microsoft.PowerToys.UITest.Next). Use when asked to port/convert/rewrite/modernize a module's UI tests to the .Next framework, create a new [Module].UITests.Next project alongside existing legacy tests, or stand up brand-new winappcli UI tests for a module that has none by reading its human test sign-off markdown. Covers the API mapping (By/Element/Session/UITestBase, KeyboardHelper/MouseHelper/ClipboardHelper), project/csproj scaffolding, naming rules, common PowerToys test recipes (toggle a module, read an activation shortcut, fire a global hotkey, inspect the clipboard, discover overlay/editor windows), build/run validation, and CI-stability hardening for fewer CI iterations. Keywords: UI test, UITests, UITestAutomation.Next, winappcli, WinAppDriver, Selenium, migrate, port, modernize, .Next, MSTest, CI stability, flaky test, stabilize on CI."
license: Complete terms in LICENSE.txt
---
# PowerToys UI-Tests Migration (legacy → `.Next`)
Convert a PowerToys module's UI tests from the legacy **WinAppDriver / Selenium / Appium** harness
(`Microsoft.PowerToys.UITest`, in `src/common/UITestAutomation/`) to the new **winappcli** harness
(`Microsoft.PowerToys.UITest.Next`, in `src/common/UITestAutomation.Next/`).
The new harness shells out to `winapp.exe` and parses its JSON — **no WinAppDriver server on :4723,
no Selenium/Appium NuGet packages, no `WindowsElement`/`WindowsDriver`.** The public *shape*
(`UITestBase`, `Session`, `Find<T>`, `By`, element wrappers like `ToggleSwitch`) is deliberately
similar, so most of the work is mechanical API mapping plus reworking a few patterns that don't
translate one-to-one (XPath selectors, stateful elements, instance mouse/keyboard helpers).
## When to use this skill
Use this skill when the task is to:
- **Port** a module's existing legacy UI tests to `.Next` (e.g. "migrate the ScreenRuler UI tests to
the new framework", "convert FancyZones.UITests to winappcli").
- **Create a new** `[Module].UITests.Next` project that re-implements the legacy tests with the new
harness, leaving the old project in place.
- **Stand up brand-new** `.Next` UI tests for a module that has **no** UI tests at all, by reading the
module's human test **sign-off markdown** (e.g. `ColorPickerUITest.md`) and turning each manual
checklist item into an automated test.
This skill is the *how*: the framework differences, the API mapping, the project scaffolding, the
naming rules, the recurring PowerToys test recipes, and the build/validate loop. The *what* (which
module, which tests) comes from the calling prompt.
> **Reference implementation — read these working examples before porting anything.** They are
> the ground truth for "what good looks like" with each harness:
> - **New (`.Next`)**: [ColorPickerEndToEndTests.cs](../../../src/modules/colorPicker/ColorPicker.UITests/ColorPickerEndToEndTests.cs)
> — full end-to-end scenario (navigate Settings → toggle module → read shortcut → fire hotkey →
> read overlay → click-capture → inspect editor), driven entirely through `winappcli`.
> - **Legacy**: [TestSpacing.cs](../../../src/modules/MeasureTool/Tests/ScreenRuler.UITests/TestSpacing.cs)
> + [TestHelper.cs](../../../src/modules/MeasureTool/Tests/ScreenRuler.UITests/TestHelper.cs)
> — a `UITestBase` subclass plus a static helper that navigates, toggles, reads the shortcut, fires
> the hotkey, and validates the clipboard.
> - **Worked Scenario-A port (validated 5/5, where the legacy suite scored 0/5 locally)**: the
> ScreenRuler suite ported from the legacy project above lives in
> [ScreenRuler.UITests.Next/TestHelper.cs](../../../src/modules/MeasureTool/Tests/ScreenRuler.UITests.Next/TestHelper.cs)
> + 5 test classes. It is the canonical port reference — cross-window toolbar discovery via
> `Session.FromProcess`, a DPI-aware `app.manifest`, cursor centering, and patient hotkey
> activation are all there because real runs needed them (see
> [references/patterns-and-pitfalls.md](references/patterns-and-pitfalls.md)).
## Required reads (in order)
1. **This `SKILL.md`** — the decision tree (which scenario), the naming rules, the high-level
workflow, and the build/validate loop.
2. **[references/framework-differences.md](references/framework-differences.md)** — the conceptual
deltas you MUST internalize before writing code: winappcli engine, stateless elements, selector
grammar (no XPath/CssSelector), session scopes (window vs process), lifecycle/hygiene/module
pre-enablement, multi-window discovery, and what the new harness does NOT (yet) provide.
3. **[references/api-mapping.md](references/api-mapping.md)** — the line-by-line cheat sheet:
namespaces, `By`, `Element` actions/properties, `Session`, `UITestBase`, the static
Keyboard/Mouse/Clipboard helpers, and the element-wrapper catalog. Keep this open while editing.
4. **[references/project-setup.md](references/project-setup.md)** — csproj scaffold, naming/placement
rules, `.slnx` registration, and how to build & run a `.Next` project. Uses the
[templates/](templates/) starter files.
5. **[references/porting-workflow.md](references/porting-workflow.md)** — the two end-to-end
playbooks: **A)** port existing legacy tests, and **B)** author tests from a human sign-off
markdown when none exist.
6. **[references/patterns-and-pitfalls.md](references/patterns-and-pitfalls.md)** — adaptable recipes
for the recurring PowerToys patterns (toggle a module + verify its process, read the activation
shortcut from a `ShortcutControl`, fire a global hotkey reliably, inspect the clipboard, discover
overlay/editor windows) and the gotchas that bite during migration.
7. **[references/ci-stability.md](references/ci-stability.md)** — the CI-stability capstone: the
Win32-window vs UIA-element mental model, five design principles that keep a port green on a slow
CI agent (authoritative-signal retries over fixed sleeps, invoke-vs-physical-click, screen-capture
cold-start, toggle-state guards, on-screen/DPI/clean-profile hygiene), and a **pre-flight
checklist** to apply BEFORE the first CI push so the first run *validates* instead of *discovers*.
Read this to spend one CI iteration instead of six.
## Pick your scenario
```mermaid
flowchart TD
A[Module to migrate] --> B{Does a legacy<br/>UITests project exist?}
B -- Yes --> C["Scenario A: PORT<br/>Create [Module].UITests.Next<br/>Re-implement each legacy test"]
B -- No --> D{Is there a human test<br/>sign-off .md?}
D -- Yes --> E["Scenario B: GREENFIELD<br/>Create [Module].UITests<br/>Turn each checklist item into a test"]
D -- No --> F[Ask the user for the<br/>test spec / sign-off doc]
```
| Scenario | Trigger | New project name | Source of test cases |
|---|---|---|---|
| **A — Port** | A legacy `[Module].UITests` (or similar) project already exists and references `UITestAutomation.csproj` | **`[Module].UITests.Next`** — keep the `.Next` suffix so it lives **alongside** the legacy project | The existing legacy test methods (1:1 re-implementation) |
| **B — Greenfield** | The module has **no** UI tests at all | **`[Module].UITests`** — **drop** the `.Next` suffix; there's nothing to live alongside | The module's human sign-off markdown (manual checklist), e.g. `ColorPickerUITest.md` |
Place the new project under **`src/modules/[Module]/Tests/[Module].UITests.Next/`** (or
`…/Tests/[Module].UITests/` for Scenario B). If the module already keeps tests in a different
`Tests/` layout, match the module's existing convention rather than forcing this one — see
[references/project-setup.md](references/project-setup.md).
> **Keep it abstract.** Every PowerToys module is unique and the legacy tests were written by
> different people in different styles. Treat the recipes in this skill as *adaptable patterns*, not
> a rigid script. Re-create the **intent and assertions** of each test; do not mechanically translate
> brittle, harness-specific scaffolding (Selenium `Actions`, XPath walks, manual driver attaches) when
> the new harness has a cleaner idiom.
## High-level workflow
Create a TODO list and work top-to-bottom. Each step links to the reference that drives it.
```markdown
- [ ] 1. Identify the module + scenario (A port / B greenfield) — this SKILL.md "Pick your scenario"
- [ ] 2. Read the two reference examples (ColorPicker .Next + ScreenRuler legacy) end-to-end
- [ ] 3. Inventory the source:
• Scenario A → list every [TestMethod] + shared helper in the legacy project
• Scenario B → read the module's sign-off .md; list each manual checklist item
— references/porting-workflow.md
- [ ] 4. Internalize the deltas — references/framework-differences.md
- [ ] 5. Scaffold the new project (csproj from template, name per the table, register in .slnx)
— references/project-setup.md
- [ ] 6. Re-implement tests, mapping each API as you go — references/api-mapping.md
+ recipes from references/patterns-and-pitfalls.md
- [ ] 7. Apply the CI-stability checklist BEFORE building — references/ci-stability.md
(authoritative-signal retries not fixed sleeps, navigation via UIA invoke, Win32 window/overlay
detection, screen-capture cold-start handling, DPI manifest, single-module enable, first-run
suppression)
- [ ] 8. Build the new project to exit code 0 — this SKILL.md "Build & validate"
- [ ] 9. (If a live desktop is available) run the tests; otherwise report that they build and are
ready to run, and summarize coverage vs. the source
```
## Build & validate
The `.Next` harness needs `winapp.exe` only at **run** time, not build time — the project has zero
managed dependency on the engine. So you can always compile-verify a migration even on an agent with
no winappcli installed.
```pwsh
# 0. FIRST build of a brand-new project: restore so the assets file exists, otherwise the build
# fails with NETSDK1004 "Assets file ... project.assets.json not found".
dotnet restore src\modules\<Module>\Tests\<Module>.UITests.Next\<Module>.UITests.Next.csproj -p:Platform=x64
# (Equivalently, run tools\build\build-essentials.cmd once at the start of the session.)
# 1. Build just the new test project (fast inner loop). Prefer the repo build script.
tools\build\build.cmd -Path src\modules\<Module>\Tests\<Module>.UITests.Next -Platform x64 -Configuration Debug
# Exit code 0 = success; non-zero = failure. On failure read the errors log next to the project:
# build.<Configuration>.<Platform>.errors.log
# 2. Run (needs a live desktop). A .Next project is a Microsoft.Testing.Platform Exe — run the
# produced exe directly with a TRX report; filter to one test/category for a tight loop.
$exe = "<repo>\x64\Debug\tests\<Module>.UITests.Next\net10.0-windows10.0.26100.0\<Module>.UITests.Next.exe"
& $exe --filter "TestCategory=<Cat>" --report-trx --report-trx-filename run.trx --results-directory <dir>
# --filter accepts "TestCategory=X" or "FullyQualifiedName~Y"; omit it to run everything.
# Exit 0 = all passed. Parse the .trx for per-test outcomes + failure messages.
```
- **Design for CI stability up-front — [references/ci-stability.md](references/ci-stability.md).**
Before the first push, walk its pre-flight checklist (authoritative-signal retries instead of fixed
sleeps, navigation via UIA invoke, Win32 window/overlay detection, screen-capture cold-start
handling, DPI manifest, single-module enable, first-run suppression). Most "passes local, fails CI"
loops come from skipping one of these; applying them proactively is how you spend one CI iteration
instead of six.
- **Run it in a loop: write → build → run → diagnose → repeat.** UI tests surface environment-real
failures (DPI scaling, cursor position, hotkey-arming races) that only a live run reveals. Start
with one deterministic test (e.g. the activation/toggle test), get it green, then widen.
- **First, run the *legacy* suite once for a baseline — and run it ELEVATED.** The legacy harness
launches PowerToys via `ProcessStartInfo { Verb = "runas" }` (elevated), so a **non-elevated** test
host can't complete the launch and **every test fails at startup with a misleading `Win32Exception`
cascade** — a false 0/N that looks like "the tests are broken" but is purely the run method. (That's
why VS Test Explorer passes them: VS runs as admin.) Run from an **elevated** terminal: start
`WinAppDriver.exe` on `127.0.0.1:4723`, then run the built DLL with `vstest.console.exe` (see
[references/porting-workflow.md](references/porting-workflow.md) §A0 for the `-Verb RunAs` recipe).
A measurement failure on a scaled (non-100%) display is usually a pre-existing DPI issue (Pitfall
12), not something the port must reproduce — the ScreenRuler legacy suite scores **4/5** elevated
here (Bounds fails at 150% scale) while the `.Next` port scores **5/5**. `.Next` tests themselves
need **no** elevation (the new harness launches the runner non-elevated).
- **Always** build to exit code 0 before declaring the migration done. Fix every compile error — do
not leave `// TODO: port this` stubs that break the build.
- Running the tests requires a **live interactive desktop** plus `winapp.exe`
(`winget install Microsoft.winappcli`, or set `WINAPP_CLI_PATH`). The whole PowerToys runner is
launched by the harness (`PowerToys.exe --open-settings`) — you should see the Settings window
appear. If the environment has no desktop (headless agent), state that the project **builds clean
and is ready to run**, and list which source tests/checklist items each new `[TestMethod]` covers.
- New `.csproj` files under `src/` MUST `<Import Project="$(RepoRoot)src\Common.Dotnet.CsWinRT.props" />`
right after `<Project Sdk=...>` (CI audits this). The template already does.
## What NOT to do
- **Do NOT delete or edit the legacy `[Module].UITests` project** in Scenario A. The `.Next` project
lives alongside it; removing the old one is a separate, explicit decision for the maintainers.
- **Do NOT touch product code.** This is a test-only migration. If a test needs a UIA hook that
doesn't exist (e.g. an `AutomationId` or a hidden automation-peer TextBlock), flag it for the user
rather than silently editing the module. (The ColorPicker example's `ColorHexAutomationPeer` hook
is a documented, pre-existing exception — see its class remarks.)
- **Do NOT port the legacy plumbing literally.** No Selenium `Actions`, no `WindowsDriver`/`WindowsElement`,
no `By.XPath`/`By.CssSelector`, no `:4723`. Map them to the winappcli idioms in
[references/api-mapping.md](references/api-mapping.md).
- **Do NOT add a `ProjectReference` to `UITestAutomation.csproj`** (the legacy harness) — reference
**`UITestAutomation.Next.csproj`** only.
- **Do NOT invent assertions** for a vague sign-off item. If a checklist line has no observable
pass/fail signal, implement what you can and leave a clearly-marked `TestContext.WriteLine` note
(or skip with an explanation) rather than asserting on something you can't actually read.
- **Do NOT introduce new third-party NuGet dependencies.** The `.Next` harness is intentionally
dependency-free (MSTest only). Use the Win32-based helpers it already ships.
## What is NICE to do
- **Improve the new UT Test framework if you see such opportunity**. The new framework works only with a few modules and may lack something other requires. If you see the old test uses something that we don't have in a new framework and it's handy, don't hesiate to port it to a new one. Or you may see the test uses a bunch of extra helpers ouside of test framework, which also may be a signal.

View File

@@ -0,0 +1,171 @@
# API mapping cheat sheet (legacy → `.Next`)
Keep this open while editing. Left column is the legacy `Microsoft.PowerToys.UITest` API; right column
is the `Microsoft.PowerToys.UITest.Next` equivalent. "—" means no direct member; see the Notes.
## Namespaces & usings
| Legacy | `.Next` |
|---|---|
| `using Microsoft.PowerToys.UITest;` | `using Microsoft.PowerToys.UITest.Next;` |
| `using OpenQA.Selenium;` / `…Appium…` | *(delete — no Selenium/Appium)* |
| `[TestClass] : UITestBase` | `[TestClass] : UITestBase` *(same shape; different namespace)* |
| `using Microsoft.VisualStudio.TestTools.UnitTesting;` | *(unchanged)* |
## `UITestBase` (the base class)
| Legacy | `.Next` | Notes |
|---|---|---|
| `: base(PowerToysModule.PowerToysSettings)` | `: base(PowerToysModule.PowerToysSettings)` | Same enum name; **values differ** — see enum table below. |
| `: base(scope, WindowSize.Large)` | `: base(scope, WindowSize.Large)` | Same `WindowSize` enum. |
| `: base(scope, size, commandLineArgs: new[]{…})` | `: base(scope, size, enableModules: new[]{…})` | 3rd arg changed from launch args to a deterministic module-enable list. |
| `Session` (property) | `Session` (property) | Same name. Legacy is `required set`; `.Next` is `private set` (assigned by `TestInit`). |
| `Find<T>(by, timeoutMS, global)` | `Find<T>(by, timeoutMS)` | No `global` param (see framework-differences §4). |
| `Find(name)` / `Find<T>(name)` | `Find(name)` / `Find<T>(name)` | Same. |
| `Has<T>/HasOne<T>(by, …, global)` | `Has<T>/HasOne<T>(by, …)` | No `global`. |
| `FindByPartialName<T>(s)` | `Find<T>(By.Name(s))` | winappcli `By.Name` is already a substring match. |
| `FindByPattern<T>(regex)` | `Session.FindAll<T>(By.Name(...))` + C# `Regex` | No base helper; filter in C#. |
| `FindByClassName<T>(c)` | `Find<T>(By.Name(...))` with a typed wrapper | Wrappers pin ClassName; or `FindAll` + filter on `.ClassName`. |
| `SendKeys(Key[])` / `SendKeySequence(Key[])` | `KeyboardHelper.SendKeys(Key[])` | Static helper (also `Session.SendKeys` passthrough). |
| `MoveMouseTo(x,y)` | `MouseHelper.MoveTo(x,y)` | Static helper. |
| `GetMousePosition()``(int,int)` | `MouseHelper.GetMousePosition()``(int X,int Y)` | Static helper. |
| `IsWindowOpen(name)` | `WindowsFinder.ListByApp(proc).Count > 0` | Or `SessionHelper.IsRunning(scope)` for a process check. |
| `RestartScopeExe(enableModules?)` | `RestartScope(enableModules?)` | Returns the fresh `Session`. |
| `ExitScopeExe()` | *(automatic)* `sessionHelper.StopIfStarted()` in `TestCleanup` | Rarely needed manually. |
## `PowerToysModule` enum (values differ!)
| Legacy value | `.Next` value | Notes |
|---|---|---|
| `PowerToysSettings` | `PowerToysSettings` | Same. The default; drive most modules through it. |
| `FancyZone` | `FancyZonesEditor` | **Renamed.** |
| `Hosts` | `Hosts` | Same. |
| `Runner` | `Runner` | Same. |
| `Workspaces` | `Workspaces` | Same. |
| `PowerRename` | `PowerRename` | Same. |
| `CommandPalette` | `CommandPalette` | Same. |
| `ScreenRuler` | `ScreenRuler` | Same. |
| `LightSwitch` | `LightSwitch` | Same. |
| *(n/a)* | `ColorPicker` | New entry (overlay module — drive via the Settings scope). |
## `By` selectors
| Legacy | `.Next` | Notes |
|---|---|---|
| `By.Name("x")` | `By.Name("x")` | winappcli = case-insensitive **substring** over Name/AutomationId. |
| `By.AccessibilityId("Id")` | `By.AccessibilityId("Id")` | **Preferred.** Also `By.Id("Id")`. |
| `By.Id("Id")` | `By.Id("Id")` / `By.AccessibilityId("Id")` | Same intent. |
| `By.ClassName("C")` | *(none)* | Use a typed wrapper, or `FindAll` + filter on `.ClassName`. |
| `By.XPath("//*[contains(@Name,'x')]")` | `By.Name("x")` | Substring search covers `contains(@Name)`. |
| `By.XPath("//*[@Name='x']")` | `By.Name("x")` (+ C# exact filter if needed) | |
| `By.XPath` (structural axes) | scoped `element.Find<T>(By.…)` or `FindAll` + C# filter | No XPath engine. |
| `By.CssSelector(...)` | *(none)* | Re-express as above. |
| *(n/a)* | `By.Slug("btn-x-1a2b")` | Direct slug from `inspect`/`search` output. |
## `Element` — properties
| Legacy | `.Next` | Notes |
|---|---|---|
| `Name` | `Name` | `.Next` is cached at Find time; re-find for fresh. |
| `ClassName` | `ClassName` | Cached. |
| `ControlType` | `ControlType` | Cached. |
| `Text` | `GetValue()` | TextPattern→ValuePattern→Selection→Name fallback. |
| `Enabled` | `IsEnabled` | Live read via `get-property`. |
| `Displayed` | `Displayed` (== `!IsOffscreen`) | Live read. |
| `Selected` | `Selected` | Live read (`IsSelected`). |
| `AutomationId` | `AutomationId` | Live read. |
| `HelpText` | `HelpText` | Live read (used for `ShortcutControl` text). |
| `Rect``Rectangle?` | `X`, `Y`, `Width`, `Height` (ints) | Cached snapshot; re-find if UI moved. |
| `GetAttribute("P")` | `GetAttribute("P")` / `GetProperty("P")` | Both live-read one UIA property. |
## `Element` — actions
| Legacy | `.Next` | Notes |
|---|---|---|
| `Click(rightClick=false, msPreAction=500, msPostAction=500)` | `Click(rightClick=false, msPostAction=200)` | **No `msPreAction`.** Uses UIA invoke (falls back to toggle/select/expand); `rightClick``click --right`. Add an explicit `Thread.Sleep` before if you relied on `msPreAction`. |
| `Click()` on a non-invokable element (TextBlock/ListItem) | `MouseClick(msPostAction=200)` | Real mouse simulation — use when the click is handled by an ancestor (the ColorPicker utility-stack label pattern). |
| `DoubleClick()` | `DoubleClick(msPostAction=200)` | Real mouse double-click. |
| Selenium `Actions` drag | `Drag(offsetX, offsetY, steps=10)` / `DragTo(target)` | Win32 mouse; uses cached center. |
| `Actions` key-down + drag | `KeyDownAndDrag(key, targetX, targetY, steps)` | Modifier-drag (FancyZones merge, tab tear-off). |
| `ReleaseKey(key)` | `KeyboardHelper.ReleaseKey(key)` | |
| `SetText`/`Clear`+`SendKeys` (TextBox) | `TextBox.SetText("v")` | `winapp ui set-value`. |
| `element.Find<T>(by)` | `element.Find<T>(by)` | Scoped search under the element. |
| `ScrollIntoView()` | `ScrollIntoView()` | Same. |
| — | `Scroll(ScrollDirection)`, `ScrollToEdge(toBottom)` | New scroll verbs. |
| — | `Focus()` | `winapp ui focus`. |
| — | `WaitForProperty(p, v, t)`, `WaitForValue(v, contains, t)`, `WaitForGone(t)` | Built-in waits (replace manual poll loops). |
## `Session`
| Legacy | `.Next` | Notes |
|---|---|---|
| `Find<T>(by, t, global)` / `Find(name)` | `Find<T>(by, t)` / `Find(name)` | No `global`. |
| `FindAll<T>(by, t, global)` | `FindAll<T>(by, t)` | No `global`; polls until found or timeout. |
| `Has`/`HasOne`/`Has<T>` | `Has`/`HasOne<T>`/`Has<T>` | Same intent. |
| `Attach(PowerToysModule)` / `Attach(windowName)` | `Session.Attach(module, size?)` / `Session.FromProcess(app)` / `WindowsFinder.WaitForWindowByApp(...)` | Re-bind to another window/process. |
| `SendKeys(Key[])` / `SendKey(key, …)` | `Session.SendKeys(Key[])` or `KeyboardHelper.SendKeys(Key[])` | Prefer the static helper. |
| `MoveMouseTo(x,y, …)` | `MouseHelper.MoveTo(x,y)` | Static. |
| `PerformMouseAction(MouseActionType.LeftClick)` | `MouseHelper.LeftClick()` | See action map below. |
| `SetMainWindowSize(size)` | `WindowHelper.SetWindowSize(hwnd, size)` | `hwnd = new IntPtr(Session.WindowHandle)`. |
| `MainWindowHandler` (`IntPtr`) | `WindowHandle` (`long`) / `WindowHandleArg` (string) | |
| — | `Inspect(depth, interactive, …)``JsonElement` | `winapp ui inspect --json` tree (the ColorPicker editor walk). |
| — | `WaitForElement(by, t)`, `WaitFor(Func<bool>, t)` | Built-in waits. |
| — | `Screenshot(path, element?, captureScreen?)` / `TryScreenshot(...)` | |
### `MouseActionType` → `MouseHelper`
| Legacy `PerformMouseAction(...)` | `.Next` |
|---|---|
| `MouseActionType.LeftClick` | `MouseHelper.LeftClick()` |
| `MouseActionType.RightClick` | `MouseHelper.RightClick()` |
| `MouseActionType.LeftDown` / `LeftUp` | `MouseHelper.LeftDown()` / `LeftUp()` |
| `MouseActionType.RightDown` / `RightUp` | `MouseHelper.RightDown()` / `RightUp()` |
| (scroll) | `MouseHelper.ScrollUp()` / `ScrollDown()` / `ScrollWheel(amount)` |
| (drag) | `MouseHelper.Drag(fromX, fromY, toX, toY, steps)` |
## Static helpers (new — no instance equivalent)
| Need | `.Next` helper |
|---|---|
| Send a key chord (incl. global Win-key hotkeys) | `KeyboardHelper.SendKeys(Key.LWin, Key.Shift, Key.C)` |
| Hold/release a key | `KeyboardHelper.PressKey(key)` / `KeyboardHelper.ReleaseKey(key)` |
| Move cursor / read cursor | `MouseHelper.MoveTo(x,y)` / `MouseHelper.GetMousePosition()` |
| Click at the current/again a point | `MouseHelper.LeftClick()` / `LeftClickAt(x,y)` / `RightClick()` / `DoubleClick()` |
| Read clipboard | `ClipboardHelper.GetText()` |
| Clear clipboard | `ClipboardHelper.Clear()` |
| Set clipboard | `ClipboardHelper.SetText("v")` |
| Wait for clipboard to change | `ClipboardHelper.WaitForText(ignoredValue, timeoutMS)` |
| Seed module on/off baseline | `SettingsConfigHelper.ConfigureGlobalModuleSettings("ColorPicker", …)` |
| Edit a module's own settings.json | `SettingsConfigHelper.UpdateModuleSettings(name, default, json => {…})` |
> The legacy `TestHelper.ClearClipboard`/`GetClipboardText` STA-thread wrappers are replaced by
> `ClipboardHelper` (which already runs on an STA thread internally). Delete the hand-rolled STA code.
## Element wrappers (`Find<T>`)
| Wrapper | Legacy | `.Next` | Notes |
|---|---|---|---|
| `Element` | ✅ | ✅ | Base. |
| `Button` | ✅ | ✅ | |
| `CheckBox` | ✅ | ✅ | |
| `ComboBox` | ✅ | ✅ | `.Select(item)` / `.SelectByText(text)` / `.SelectedText`. |
| `RadioButton` | ✅ | ✅ | |
| `Slider` | ✅ | ✅ | |
| `Tab` | ✅ | ✅ | |
| `TextBlock` | ✅ | ✅ | |
| `TextBox` | ✅ | ✅ | `.SetText(v)` / `.Value`. |
| `ToggleSwitch` | ✅ | ✅ | `.IsOn` / `.Toggle(bool)`. Pins `ClassName="ToggleSwitch"`. |
| `Thumb` | ✅ | ✅ | |
| `NavigationViewItem` | ✅ | ✅ | UIA `ListItem`. |
| `Pane` | ✅ | ✅ | |
| `Custom` | ✅ | ✅ | UIA `Custom` (FancyZones zones, Workspaces canvas). |
| `Window` | ✅ | ✅ | |
| `Group` | ✅ | ❌ | Use `Find<Element>` or add a wrapper. |
| `HyperlinkButton` | ✅ | ❌ | Use `Find<Button>` (it's a Button under UIA) or add a wrapper. |
## `Key` enum
Both frameworks expose a `Key` enum. The `.Next` `Key` (in `KeyboardHelper.cs`) uses `LWin` (not
`Win`). When porting a shortcut parser, map `"win"`/`"windows"``Key.LWin`. Letters `A``Z`,
digits `Num0``Num9`, `F1``F12`, and the usual `Ctrl/Shift/Alt/Esc/Enter/Tab/Space/Arrows` are all
present.

View File

@@ -0,0 +1,188 @@
# CI stability — get a port green in fewer iterations
The expensive gap in a UI-test migration is between **"passes on my box"** and **"green on the CI
agent."** Every round-trip through that gap is a push, a queue wait, and a log dig. This document
distills the failure modes that cause those round-trips into (1) a **mental model**, (2) **design
principles to bake in from the first commit**, and (3) a **pre-flight checklist** so the first CI run
is the *validation*, not the *discovery*.
Read this **after** [patterns-and-pitfalls.md](patterns-and-pitfalls.md) (it references those recipes
and pitfalls by number). The canonical worked example for everything below is the ScreenRuler port:
[ScreenRuler.UITests.Next/TestHelper.cs](../../../../src/modules/MeasureTool/Tests/ScreenRuler.UITests.Next/TestHelper.cs).
> **Why this matters for iteration count.** Almost every "flaky on CI, fine locally" failure traces to
> one of five root causes below. A dev box hides all of them (higher-res display, warmed caches, a
> profile that already dismissed first-run windows, a human not touching the mouse). If you design for
> them up-front, the first CI run tends to be green; if you don't, you rediscover them one push at a
> time.
---
## The core mental model: Win32 **windows** vs UIA **elements**
This single distinction drives the right tool choice for almost every interaction, and picking the
wrong layer is the #1 source of slow, racy CI failures.
| Layer | Sees | Cost / risk | Use it for |
|---|---|---|---|
| **Win32** (`WindowControl`, `WindowsFinder`) | **WINDOWS** — HWNDs: handle, class, title, rect, PID, visibility | Cheap, synchronous, attaches **no** UIA client | "Is it open? where/how big? is it visible? close/focus it?" — anything **window**-level |
| **UIA / winappcli** (`Find`, `GetProperty`, `Invoke`, `Inspect`) | **ELEMENTS** — XAML controls inside one window's content island (**not** child HWNDs) | Spins up a UIA client, walks a tree; can **race** a not-yet-ready window and **disturb** a live screen-capture | Reading element **state** (ToggleState/Name/Value), and **locating** a control's rect to act on it |
The operating rules that fall out of this:
- **Window-level question → Win32.** Presence, size, visibility, close, focus. Never ask winappcli
"is this window up?" when `WindowControl.EnumerateProcessWindows(pids)` answers it synchronously
without touching UIA.
- **Act on an element → locate once via UIA, then input via Win32.** Search for the control's
rectangle **once**, then drive a real mouse/keyboard at that point. This is exactly what
`Element.Click()` does (UIA `search``MouseHelper` click at the centre; falls back to a
coordinate-free UIA invoke only when the control reports no on-screen size).
- **Read element state → UIA (unavoidable).** `ToggleState`, `Name`, `Value` have no Win32 equivalent.
- **NEVER walk a live/capturing window's UIA tree.** Attaching a UIA client and enumerating the tree
(winappcli `list-windows` / `Inspect`) **disturbs a Windows.Graphics.Capture (WGC) session** and
empties the very next frame. For a capture module, detect windows with Win32 `EnumWindows`, not UIA
(Pitfall 18).
---
## Principle 1 — Assert on an **authoritative signal**, retry until true (not a fixed sleep)
A `Thread.Sleep(500)` *guesses* how long a step takes; a CI agent can be 10× slower, so the guess is
either flaky (too short) or wasteful (too long). Instead, name the **one observable fact** that proves
the step happened and **poll for it to a deadline**:
| Step | Authoritative signal | How to read it |
|---|---|---|
| Module enabled | its process is running | `Process.GetProcessesByName(name).Length > 0` |
| Tool / overlay engaged | the overlay **window** exists | Win32 `WindowControl.EnumerateProcessWindows` (Pitfall 18) |
| Measurement taken | clipboard is non-empty | `ClipboardHelper.WaitForText` |
| Page navigated | the target control is present | `Session.Has(By.AccessibilityId(...))` |
Retry the **whole interaction** (press → check signal) until the signal is true or the deadline
elapses — don't press once and hope. Reference: `SelectToolAndVerify` presses the toolbar button and
re-checks `IsMeasureOverlayPresent()` on a 25 s deadline; `MeasureWithRetry` re-runs the gesture while
the clipboard is empty. Both adapt to a slow agent for free.
> Corollary: **fail with the signal in the message.** `Assert.Fail("overlay never appeared after N
> attempts")` tells you *which* signal missed on CI; `Assert.IsTrue(x)` tells you nothing.
---
## Principle 2 — The input-method decision: UIA **invoke** vs physical **click**
Two ways to activate a control, with different failure modes:
- **Physical click** (`Element.Click()` → UIA locates the rect, Win32 clicks its centre). Use for
**real interactions** that need genuine on-screen, foreground input: drags, clicks on a
capture surface, anything that measures cursor position. Requires the control **visible** and the
window **interactive-for-mouse-input**.
- **Coordinate-free UIA invoke** (`Element.Invoke()` → InvokePattern → Toggle → Select → Expand).
Use where a real cursor move is undesirable or the on-screen point is unreliable: **navigation
items** (they live in a scrollable pane / overflow "…" menu, so they can report a size yet sit
off-viewport), the **first interaction** right after a window appears, and zero-bounds/off-screen
controls.
**The iteration-burning trap: the first interaction after a window appears.** A window's UIA tree
exists a moment *before* the window is interactive for mouse input. A physical click that lands in that
window is **silently dropped** — flaky, and only on slower agents. Navigation is almost always the
first interaction, so it must not depend on a physical click.
The harness bakes this in so you get it for free: `NavigationViewItem.Click()` is overridden to a
coordinate-free invoke, and `Element.Click()` falls back to invoke for zero-bounds controls. **So keep
navigating with `By.AccessibilityId(...).Click()`** (Recipe 1) — it's race-safe under the hood. Only
reach for a raw `MouseClick`/manual `MouseHelper` when the interaction genuinely needs real mouse input
(and by then the window has settled).
---
## Principle 3 — Screen-capture (WGC) modules: cold-start + don't disturb the session
Any module built on **Windows.Graphics.Capture** — Screen Ruler spacing, Magnifier, Text Extractor,
Color Picker's zoom, screenshot tools — shares three facts that a warm dev box hides:
1. **First-frame cold-start.** The first captured frame is instant when warm but can take **several
seconds** on a cold/headless CI agent. A gesture that reads the result too early gets **nothing**.
2. **Per-process, no cross-test warming.** Each test spawns its own module process = its own capture
session = its own cold-start. There is no "the previous test warmed it up."
3. **A UIA tree-walk of the live window disturbs/empties it** (the mental-model rule above).
The resilient shape (see `PerformSpacingToolTest` / `MeasureWithRetry` / `ReengageTool` /
`IsMeasureOverlayPresent`, and Recipe 12):
- **Detect** the overlay/window via **Win32 EnumWindows**, never winappcli `list-windows`/`Inspect`.
- **Retry the gesture IN PLACE** (same session) to give the first frame time — do **not** close/reopen
between attempts (that *resets* the cold-start every time).
- If in-place retries still yield nothing, **RE-ENGAGE ONCE** (tear the session down and recreate it)
to recover a genuine *stall* — but only once, after a generous in-place window. Re-engaging on every
attempt is the classic mistake that never recovers.
---
## Principle 4 — Guard state-toggling controls on their current state
A `ToggleSwitch` / toolbar `ToggleButton` flips **relative to its current state**, so a blind "press
to select" can *deselect* an already-engaged control — and an innocent retry can toggle it back off.
**Read the state first** and only press when it's wrong: press when `ToggleState == "Off"` to select,
`"On"` to deselect. `ToggleSwitch.Toggle(bool)` already does this; for a raw toolbar `ToggleButton`,
guard on `GetProperty("ToggleState")` yourself (as `SelectToolAndVerify` / `ReengageTool` do). This is
also why a *retry loop* around a toggle is dangerous unless it re-reads state each pass.
---
## Principle 5 — Everything on-screen, DPI-correct, from a clean profile
The whole "passes local, fails CI" cluster is environment differences a dev box papers over. Each has
a one-time fix; do them all up-front:
| Difference (CI vs local) | Symptom | Fix (bake in once) |
|---|---|---|
| **DPI** — CI often 100%, dev often 125150% (or vice-versa) | coordinate tests off by the scale factor (`150 × 149` for a 100px drag) | `app.manifest` with `PerMonitorV2`, wired in the csproj (Pitfall 12) |
| **Off-screen** — same-size 1920×1080 agent, a resized window keeps its old top-left | gesture lands off-screen → empty result | anchor to `ScreenCenter()`, move in steps; harness centers+clamps `WindowSize` presets (Pitfall 16, Recipe 11) |
| **Fresh profile** — OOBE / "what's new" window, centered + topmost | centre-screen gesture hits *that* window | harness `PreTestHygiene` calls `SettingsConfigHelper.SuppressFirstRunExperience()` (Pitfall 17) |
| **Cursor position** — undefined at test start | gesture anchored to current cursor drifts off-screen | park at `ScreenCenter()`, never anchor to `GetMousePosition()` (Recipe 11) |
| **Cold runner** — ~15 default modules start on a clean profile | slow start, cross-module hotkey/overlay interference | enable **only** the module under test via the base ctor (Recipe 9) |
---
## Pre-flight CI-stability checklist
Tick these **before** the first CI push. Each maps to a principle/recipe above; skipping one is a
likely extra CI iteration.
```markdown
- [ ] app.manifest (PerMonitorV2) wired into the csproj — any coordinate-exact test (P5 / Pitfall 12)
- [ ] Base ctor enables ONLY the module under test (P5 / Recipe 9)
- [ ] First-run/what's-new suppression confirmed for capture & coordinate modules (P5 / Pitfall 17)
- [ ] Gestures anchored to ScreenCenter(), cursor moved in steps, never to the current cursor (P5 / Recipe 11)
- [ ] Navigation & the first interaction go through By.AccessibilityId(...).Click() (invoke under the hood) (P2 / Recipe 1)
- [ ] Window/overlay presence via WindowControl/WindowsFinder (Win32) — never a UIA walk of a live-capture window (mental model / P3 / Pitfall 18)
- [ ] Every wait polls an authoritative signal to a deadline — no bare Thread.Sleep standing in for "wait until ready" (P1)
- [ ] Capture modules: in-place gesture retry + single re-engage; overlay detected via Win32 (P3 / Recipe 12)
- [ ] Toggle/ToggleButton presses guarded on the current ToggleState (P4)
- [ ] Clipboard via ClipboardHelper (STA + retry); no hand-rolled STA wrapper (Recipe 5)
- [ ] All mutated state restored in a finally; cleanup uses WindowControl.Try* so it never masks the real failure (Pitfall 9)
- [ ] Content-dependent measurements assert on FORMAT (regex); exact values only for content-independent gestures (Pitfall 15)
```
---
## Local vs CI — why a local pass is not proof
A green local run tells you the code **compiles and the logic executes**; it does **not** tell you the
test is CI-stable, because your box differs from the agent on all four axes at once:
- **Higher-res display** → everything stays on-screen (hides off-screen gestures, Pitfall 16).
- **Warmed profile** → OOBE/what's-new already dismissed (hides Pitfall 17), caches hot (hides WGC
cold-start, P3).
- **Faster machine** → the not-yet-interactive-window race (P2) and hook-arming race (Pitfall 14)
rarely trigger.
- **A human at the keyboard** → *you might touch the mouse.* A real mouse drag (Bounds-style) is
corrupted by any physical cursor movement mid-gesture — a wrong box size that looks like a bug but is
just interference. CI has no human, so this is local-only noise; **don't touch input during a local
run.**
Practical local discipline: treat local as the fast **compile + logic** loop, and CI as the real gate.
**Don't over-run the suite locally** — for modules that kill/relaunch their process each test (e.g. the
Measure Tool), repeated runs can wedge Win32 input injection until the desktop session is reset
(`Win32Exception` on `SendInput`/`SetCursorPos`); that's an environment artifact, not a code defect,
and never happens on a fresh CI agent.

View File

@@ -0,0 +1,167 @@
# Framework differences: legacy vs `.Next`
The conceptual deltas you must internalize before porting. Read this once, end-to-end, then keep
[api-mapping.md](api-mapping.md) open for the mechanical lookups.
## At a glance
| Aspect | Legacy `Microsoft.PowerToys.UITest` | New `Microsoft.PowerToys.UITest.Next` |
|---|---|---|
| Folder | `src/common/UITestAutomation/` | `src/common/UITestAutomation.Next/` |
| Namespace | `Microsoft.PowerToys.UITest` | `Microsoft.PowerToys.UITest.Next` |
| Assembly | `Microsoft.PowerToys.UITest` | `Microsoft.PowerToys.UITest.Next` |
| Engine | WinAppDriver server on `http://127.0.0.1:4723` + Selenium/Appium | `winapp.exe` CLI (shell out, parse `--json`) |
| Driver object | `WindowsDriver<WindowsElement>`, `WindowsElement` | none — every call is a `winapp ui …` subprocess |
| 3rd-party deps | `Appium.WebDriver`, `Selenium.WebDriver`, … | none (MSTest only) |
| Element model | **stateful** — wraps a live `WindowsElement` | **stateless** — wraps a selector; every read/action re-shells out |
| Selector grammar | Selenium `By` (Name, ClassName, Id, **XPath**, **CssSelector**, AccessibilityId) | winappcli `By` (**Name=text**, **AccessibilityId**, **Slug**) — no XPath/CSS |
| Find scope flag | `bool global` parameter on every `Find` | no `global` param — session scope (`-w`/`-a`) decides reach |
| Mouse/keyboard | instance methods on `Session`/`UITestBase` (`MoveMouseTo`, `PerformMouseAction`, `SendKeys`) | **static** helpers (`MouseHelper`, `KeyboardHelper`, `ClipboardHelper`) |
| Run-time prereq | WinAppDriver installed + running | `winapp.exe` on PATH (or `WINAPP_CLI_PATH`) |
| Elevation | **Required** — harness launches the runner via `Verb="runas"`; a non-elevated host fails at launch | **Not required** — harness launches the runner non-elevated (works from a plain terminal) |
| Test runner | MSTest (VSTest) | MSTest via Microsoft.Testing.Platform (`EnableMSTestRunner`) |
## 1. The engine: winappcli, not WinAppDriver
The legacy harness spins up a WinAppDriver server and talks Selenium WebDriver to it. The `.Next`
harness has **no server and no session protocol**`WinappCli.Invoke(...)` starts `winapp.exe`,
captures stdout/stderr/exit-code, and (for `--json` verbs) parses the envelope. Every `Find`,
property read, click, and key press is an independent process invocation.
Consequences you'll feel while porting:
- There is no long-lived "driver" to attach/dispose. `Session` is a lightweight value object holding a
target flag (`-w <hwnd>` or `-a <app>`) and metadata. `Session.Cleanup()` is a no-op.
- "Is the CLI installed?" is checked once per run (`WinappCli.IsAvailable()` from `UITestBase`), and a
missing CLI fails fast with an install hint — you don't manage that.
- Errors surface as non-zero exit codes + stderr, wrapped into MSTest `Assert` failures with a
`winapp … -> exit N; stderr: …` description. There are no `WebDriverException`/`NoSuchElementException`
types to catch — use the `Has*`/`WaitFor*` probes instead of try/catch on Find.
## 2. Elements are stateless
Legacy `Element` wraps a live `WindowsElement`; properties like `Enabled`, `Text`, `Rect` read the
cached Selenium object. `.Next` `Element` wraps **only a selector** (a winappcli slug or text query)
plus the owning `Session`. The `ControlType`, `ClassName`, `Name`, `X/Y/Width/Height` fields are the
values captured **at `Find` time**; every *fresh* read (`IsEnabled`, `GetProperty(...)`, `GetValue()`)
shells out again via `winapp ui get-property`/`get-value`.
Porting implications:
- Cached geometry (`X`, `Y`, `Width`, `Height`) is a **snapshot**. If the UI moved since `Find`,
re-find before using coordinates for a `Drag`/`MouseClick`.
- There is no `element.Rect` returning a live `Rectangle`. Use the cached `X/Y/Width/Height` ints, or
re-find.
- Don't hold an `Element` across a navigation/relaunch and expect it to still resolve — re-find after
the tree changes.
## 3. Selectors: `By.Name` / `By.AccessibilityId` / `By.Slug` only
The new `By` (in `By.cs`) is **not** Selenium's `By`. It has three kinds:
| `.Next` factory | Meaning | winappcli mechanic |
|---|---|---|
| `By.Name(text)` | case-insensitive substring search over Name/AutomationId | `winapp ui search "<text>"` |
| `By.AccessibilityId(id)` / `By.Id(id)` | stable `AutomationId` | search by id |
| `By.Slug(slug)` | a semantic slug printed by `inspect`/`search` (e.g. `btn-close-d1a0`) | direct slug selector |
There is **no** `By.XPath`, `By.ClassName`, or `By.CssSelector`. To port those:
- `By.ClassName("ToggleSwitch")` → use the typed wrapper (`Find<ToggleSwitch>(By.Name(...))`), which
pins `ClassName` via `TargetClassName`. The wrapper's class filter replaces the ClassName selector.
- `By.XPath("//*[contains(@Name,'foo')]")` (the legacy `FindByPartialName`) → `By.Name("foo")` already
does substring matching in winappcli, so a partial-name XPath usually collapses to a plain
`By.Name`.
- `By.XPath("//*[@Name='exact']")``By.Name("exact")` (winappcli substring-matches; if you need to
disambiguate, `FindAll` then filter in C# on `m.Name == "exact"`).
- Complex structural XPath (parent/child axes) → there is no direct equivalent. Re-express as: find the
container by id, then `container.Find<T>(By.…)` (scoped search), or `Session.FindAll<T>` + a C#
`Where(...)` on the cached `ControlType`/`ClassName`/`Name`/coordinates. The ColorPicker example
does exactly this (`FindAll<Element>(By.Name("Color Picker"))` then `.OrderByDescending(m => m.X)`).
**Prefer `By.AccessibilityId`.** When porting, if a legacy test used a fragile `By.Name` or XPath, check
the module's XAML for an `x:Name`/`AutomationProperties.AutomationId` and switch to `By.AccessibilityId`
— it's the most stable selector and what the new examples favor.
## 4. No `global` parameter — session scope decides reach
Legacy `Find<T>(by, timeoutMS, global)` had a `global` bool to widen the search beyond the current
window. `.Next` `Find<T>(by, timeoutMS)` has **no** `global` param. Instead, the **session scope**
governs reach:
- **Window scope** (`-w <hwnd>`, the default from `UITestBase`/`SessionHelper.Init`): searches within
one window. Use when a process owns several windows and you must pin one (Settings vs. its
`PopupHost`; ColorPicker overlay vs. editor).
- **Process scope** (`-a <name|pid>`, via `Session.FromProcess(...)`): searches all of a process's
windows; every call re-resolves, so it transparently survives window replacement (re-navigation,
page swaps, dropdown popups in a separate `PopupHost`). Closest analog to the legacy `global: true`.
To reach a **different** window (e.g. an editor/overlay the module just spawned), don't pass a flag —
discover it with `WindowsFinder`/`WindowControl` (see §6) and get a new `Session` bound to it.
## 5. Lifecycle, hygiene, and module pre-enablement (`UITestBase`)
Both bases run `[TestInitialize]`/`[TestCleanup]`, but the `.Next` base centralizes things the legacy
tests often did by hand:
- **Constructor:** `UITestBase(PowerToysModule scope = PowerToysSettings, WindowSize size = UnSpecified, string[]? enableModules = null)`.
- `scope` — which module/window to drive. **Most module tests use `PowerToysModule.PowerToysSettings`**
and drive the utility *through* the Settings UI + its activation hotkey, because the **runner**
(`PowerToys.exe`) owns module toggles and the centralized keyboard hook. Launching a module's UI
exe standalone bypasses that and the hotkey never fires.
- `size` — applied after the window appears; `UnSpecified` maximizes (deterministic on CI). Maps to
the legacy `WindowSize` ctor arg.
- `enableModules` — when non-null, exactly these modules are enabled (others disabled) in the global
`settings.json` **before** launch. This is the deterministic replacement for the legacy
`commandLineArgs`/`StartExe(enableModules)` pattern. The names are the keys under `"enabled"` (e.g.
`"ColorPicker"`, `"FancyZones"`, `"Measure Tool"`).
- **Pre-test hygiene** runs automatically: `Win+M` (minimize all) → `Esc` → kill stale PowerToys
processes (`StaleProcessNames`, overridable). You usually delete the legacy test's manual
`CloseOtherApplications`/`Win+M` calls.
- **Teardown** stops only what the base launched (`StopIfStarted()`), so you rarely need a manual
process-kill in `[TestCleanup]`. (Per-test cleanup of *spawned* windows — an overlay/editor the test
popped — is still the test's job; use `WindowControl.TryCloseByApp` in a `finally`.)
- **`RestartScope(enableModules?)`** replaces the legacy `RestartScopeExe` — re-seeds modules,
kills + relaunches, reapplies size, returns the fresh `Session`.
- **Class-shared window:** override `protected bool ReuseScopeAcrossTests => true;` to launch once per
class and reuse the window across `[TestMethod]`s (skips per-test hygiene/relaunch). Use for smoke
suites with many cheap cases against one window. Default is per-test isolation.
## 6. Multi-window discovery
The legacy harness used `Session.Attach(module|windowName)` to switch the driver to another window.
`.Next` discovers windows with two static helpers:
- **`WindowsFinder`** (read/wait): `ListByApp(appNameOrPid)`, `ListAll()`,
`WaitForWindowByApp(app, predicate, timeoutMS)`, `WaitForWindowByTitle(...)`,
`WaitForWindowByProcess(...)`. Returns `WindowInfo` (hwnd/title/process/size/className) and, for the
`WaitFor*` variants, a ready-to-use `Session` bound to that window. This is how the ColorPicker test
finds the overlay (`Width<300 && Height<200`) vs. the editor (`Width>300 && Height>300`) from the
same `PowerToys.ColorPickerUI` process.
- **`WindowControl`** (tolerant cleanup): `TryCloseByApp(app[, predicate])`, `TryFocusByApp`,
`TryKillProcessByName` (exact), `TryKillProcess` (substring), `SafeCloseAndFocus`. Every method
swallows exceptions and returns a bool — designed for `finally` blocks so cleanup never masks the
real failure.
Note: unfiltered `WindowsFinder.ListAll()` drops windows with no Win32 title (e.g. the ColorPicker
editor exposes its name only via UIA). **Use `ListByApp`/`WaitForWindowByApp` with a process filter**
for those.
## 7. What `.Next` does NOT (yet) provide
When a legacy test relies on one of these, adapt rather than expecting a drop-in:
- **`By.XPath` / `By.CssSelector` / `By.ClassName`** — none exist (see §3).
- **`FindByPattern` / regex Name matching** as a base helper — re-express with `FindAll<T>(By.Name(...))`
+ a C# `Regex`/`Where` on the cached `Name` (the legacy base's `FindByNamePattern` shows the shape).
- **`Group`, `HyperlinkButton` wrappers** — the legacy `Element/` set has them; `.Next` doesn't.
Use `Find<Element>` (or `Find<Button>` for a hyperlink button, which is a Button under UIA), or add a
tiny wrapper subclass mirroring `Button.cs`/`NavigationViewItem.cs` if you need the type.
- **`element.Text` / `element.Rect` / `element.Enabled`** (legacy names) — use `GetValue()` /
`X,Y,Width,Height` / `IsEnabled` (see [api-mapping.md](api-mapping.md)).
- **Instance `Session.SendKeys`/`MoveMouseTo`/`PerformMouseAction`** — exist as a thin `Session.SendKeys`
passthrough, but prefer the static `KeyboardHelper`/`MouseHelper`.
If a genuinely missing capability blocks a port, add it to the `.Next` harness in a small, focused way
that mirrors the existing file style (one wrapper class, or one static helper method) — and call it out
to the user. Don't pull in a NuGet package.

View File

@@ -0,0 +1,410 @@
# Patterns & pitfalls
Adaptable recipes for the recurring PowerToys UI-test patterns, plus the gotchas that bite during a
`.Next` migration. **These are patterns, not a script** — every module differs; lift the shape, not
the literal strings. All snippets assume `using Microsoft.PowerToys.UITest.Next;` and a class deriving
from `UITestBase`.
## Recipe 1 — Navigate to a module's Settings page
Two common shapes. Prefer the NavigationView item by `AutomationId` when the module has one:
```csharp
// Stable: the left-nav item (a ListItem) by AutomationId. Expand the parent group first if needed.
// NavigationViewItem.Click() is a coordinate-free UIA invoke (the harness overrides it) — race-safe
// even as the FIRST interaction right after Settings opens, and immune to nav-pane scroll/overflow
// (a physical click there is silently dropped before the window is interactive; see Pitfall 19).
if (Session.Has(By.AccessibilityId("ScreenRulerNavItem"), 500) == false)
{
Session.Find<NavigationViewItem>(By.AccessibilityId("SystemToolsNavItem")).Click(msPostAction: 500);
}
Session.Find<NavigationViewItem>(By.AccessibilityId("ScreenRulerNavItem")).Click(msPostAction: 500);
```
```csharp
// Dashboard utility-stack label that has no InvokePattern (the click is handled by the ancestor
// SettingsCard). A Name search may return several elements — disambiguate, then MouseClick (real
// mouse), not Click (UIA invoke), because the label itself isn't invokable.
var matches = Session.FindAll<Element>(By.Name("Color Picker"));
var label = matches.Where(m => m.ClassName.Equals("TextBlock", StringComparison.OrdinalIgnoreCase))
.OrderByDescending(m => m.X) // rightmost = the utility-stack label
.First();
label.MouseClick(msPostAction: 800);
```
> Pitfall: a `By.Name("Color Picker")` substring search can match a quick-access tile, its label, the
> utility-stack label, and a `ToggleSwitch`. Use `FindAll` + a C# filter on `ClassName`/`ControlType`/
> coordinates instead of assuming a single hit.
## Recipe 2 — Toggle a module on/off and verify its process
```csharp
// The page-level enable switch. ToggleSwitch pins ClassName="ToggleSwitch", so the Name search
// won't grab a sibling Button with the same Name (e.g. a dashboard card).
var toggle = Find<ToggleSwitch>(By.Name("Color Picker"));
bool initial = toggle.IsOn;
toggle.Toggle(false); // flips only if currently on
Assert.IsTrue(toggle.WaitForProperty("ToggleState", "Off", 5_000), "UI didn't flip to Off.");
Assert.IsTrue(WaitForProcess("PowerToys.ColorPickerUI", false, 10_000), "Process didn't exit.");
toggle.Toggle(true);
Assert.IsTrue(toggle.WaitForProperty("ToggleState", "On", 5_000), "UI didn't flip to On.");
Assert.IsTrue(WaitForProcess("PowerToys.ColorPickerUI", true, 10_000), "Process didn't start.");
// ... restore `initial` in a finally ...
```
```csharp
// Poll for process presence — no built-in, so keep a small helper (from the ColorPicker example).
private static bool WaitForProcess(string name, bool expected, int timeoutMS)
{
var deadline = DateTime.UtcNow + TimeSpan.FromMilliseconds(timeoutMS);
while (DateTime.UtcNow < deadline)
{
if ((Process.GetProcessesByName(name).Length > 0) == expected) return true;
Thread.Sleep(250);
}
return false;
}
```
> Process names are the `-a` names (no `.exe`): `PowerToys.ColorPickerUI`, `PowerToys.ScreenRuler`
> (actually `PowerToys.MeasureToolUI`), `PowerToys.FancyZonesEditor`, etc. — see `ModuleConfigData.cs`
> in the harness for the authoritative list.
## Recipe 3 — Read the activation shortcut from a `ShortcutControl`
PowerToys' `ShortcutControl` renders the current chord on its inner `EditButton`, exposing the readable
text (e.g. `"Win + Shift + C"`) via `AutomationProperties.HelpText`. `x:Name` reflects as the
`AutomationId` in WinUI when none is set, so:
```csharp
var editButton = Find<Button>(By.AccessibilityId("EditButton"));
string shortcutText = editButton.HelpText; // "Win + Shift + C"
Key[] keys = ParseShortcutText(shortcutText); // -> [LWin, Shift, C]
```
When the page has several shortcut controls, scope the search under the specific card first:
```csharp
var card = Session.Find<Element>(By.AccessibilityId("Shortcut_ScreenRuler"));
var editButton = card.Find<Element>(By.AccessibilityId("EditButton"));
```
```csharp
// Shortcut-string parser (ports verbatim from either example; note "win" -> Key.LWin).
private static Key[] ParseShortcutText(string s)
{
var parts = s.Split(new[] { " + ", "+", " " }, StringSplitOptions.RemoveEmptyEntries);
var keys = new List<Key>();
foreach (var raw in parts)
{
var p = raw.Trim().ToLowerInvariant();
Key? k = p switch
{
"win" or "windows" => Key.LWin,
"ctrl" or "control" => Key.Ctrl,
"shift" => Key.Shift,
"alt" => Key.Alt,
_ when p.Length == 1 && p[0] >= 'a' && p[0] <= 'z' => (Key)Enum.Parse(typeof(Key), p.ToUpperInvariant()),
_ => null,
};
if (k.HasValue) keys.Add(k.Value);
}
return keys.ToArray();
}
```
## Recipe 4 — Fire a global hotkey reliably
The runner arms its low-level keyboard hook **asynchronously** after a module is enabled, so the very
first chord can be lost. Re-send with patient polling between attempts — and don't re-send too eagerly,
because for some modules re-sending hides/re-shows the target window:
```csharp
const int attempts = 3;
Session? overlay = null;
for (int i = 1; i <= attempts && overlay is null; i++)
{
KeyboardHelper.SendKeys(keys);
overlay = WindowsFinder.WaitForWindowByApp(
"PowerToys.ColorPickerUI", w => w.Width < 300 && w.Height < 200, timeoutMS: 2_500);
if (overlay is null)
{
MouseHelper.MoveTo(cx + 60, cy + 60); // recovery nudge for cursor-following overlays
overlay = WindowsFinder.WaitForWindowByApp(
"PowerToys.ColorPickerUI", w => w.Width < 300 && w.Height < 200, timeoutMS: 2_500);
}
}
Assert.IsNotNull(overlay, "Activation window did not appear after retries.");
```
> Only the runner's centralized hook can catch a global PowerToys hotkey, which is *why* tests launch
> through the Settings/runner scope. `KeyboardHelper.SendKeys` holds `LWin` via `keybd_event` while
> sending the rest through SendInput — pure injection doesn't reliably trigger `RegisterHotKey`.
## Recipe 5 — Inspect the clipboard around an action
```csharp
ClipboardHelper.Clear();
MouseHelper.LeftClick(); // the action that copies
string captured = ClipboardHelper.WaitForText(ignoredValue: string.Empty, timeoutMS: 3_000);
Assert.IsFalse(string.IsNullOrEmpty(captured), "Nothing was copied within 3s.");
```
`ClipboardHelper` already marshals to an STA thread and swallows contention errors — delete any legacy
hand-rolled STA wrapper.
## Recipe 6 — Discover overlay vs. editor windows from one process
```csharp
// Small overlay (transparent/topmost) — filter by size.
var overlay = WindowsFinder.WaitForWindowByApp(
"PowerToys.ColorPickerUI", w => w.Width < 300 && w.Height < 200, timeoutMS: 2_500);
// Larger editor window from the SAME process.
var editor = WindowsFinder.WaitForWindowByApp(
"PowerToys.ColorPickerUI", w => w.Width > 300 && w.Height > 300, timeoutMS: 10_000);
// Each returns a Session bound to that window; search within it:
var peer = overlay!.Find(By.AccessibilityId("ColorHexAutomationPeer"), timeoutMS: 2_000);
string hex = peer.Name;
```
> Use `ListByApp`/`WaitForWindowByApp` (process-filtered), **not** `ListAll`, for windows that expose
> their name only via UIA (no Win32 title) — the unfiltered list drops them.
## Recipe 7 — Walk a window's UIA tree (when there's no single selector)
```csharp
var tree = editor.Inspect(depth: 12); // JsonElement: { windows:[{ elements:[{type,name,value,children}] }] }
var values = new List<(string Type, string Name, string Value)>();
WalkElements(tree, values); // recursive walk (see ColorPicker example)
bool found = values.Any(v =>
v.Name.Contains(captured, StringComparison.OrdinalIgnoreCase) ||
v.Value.Contains(captured, StringComparison.OrdinalIgnoreCase));
Assert.IsTrue(found, $"'{captured}' not found in editor tree.");
```
Use this when a value can appear in any of several controls (e.g. ColorPicker's editor renders the
captured color in whichever format control matches) and you only need "it's somewhere in the tree".
## Recipe 8 — Read a value the UIA Name hides
When `AutomationProperties.Name` overrides the UIA Name with a friendly label (e.g. a color *name*
instead of its HEX), `GetValue()` still reads the underlying Text/Value binding:
```csharp
string displayed = Find<TextBlock>(By.AccessibilityId("SomeLabel")).GetValue(); // the real text, not the Name
```
## Recipe 9 — Enable ONLY the module under test (deterministic, faster, isolated)
Pass `enableModules` to the base ctor so exactly those modules are on before launch — and for a
single-module suite, pass **just the one you're testing**. `ConfigureGlobalModuleSettings` enables the
named modules and **disables every other one**, so the runner boots only what you need:
```csharp
// All five ScreenRuler test classes do this; ColorPicker too. The key is the settings.json
// "enabled" name (note spaces, e.g. "Measure Tool", "PowerToys Run") — see the enabled section of
// %LocalAppData%\Microsoft\PowerToys\settings.json or ModuleConfigData.
public MyTests() : base(PowerToysModule.PowerToysSettings, enableModules: new[] { "Measure Tool" }) { }
```
Why it's worth doing on every per-module suite:
- **Faster on a fresh profile (CI).** The runner's `start_enabled_powertoys` phase starts each enabled
module; on a clean CI profile that's ~15 default-on modules (~10s). Enabling one cuts that to ~1s
(~9s saved per cold start). *(The hotkey register/unregister loop runs over all modules regardless,
so it's unchanged — the win is the start phase.)* Locally it's timing-neutral.
- **Isolated + deterministic.** No other module's global hotkey, overlay, or tray behavior can
interfere with your gesture, and the test starts from a known on/off baseline instead of whatever
`settings.json` happened to hold.
It's compatible with tests that toggle the module themselves (e.g. ColorPicker toggles OFF→ON to check
the process lifecycle) — the module just starts already-enabled.
For a per-module *setting* (not just enable/disable), edit the module's own settings file before launch:
```csharp
SettingsConfigHelper.UpdateModuleSettings(
"ColorPicker",
defaultSettingsContent: "{}",
settings => settings["copiedColorRepresentation"] = "HEX");
```
## Recipe 10 — Drive controls that live in a *different* window (process-scoped session)
A module's toolbar / overlay / editor is a separate window from Settings. The legacy `global: true`
Find reached into it implicitly; in `.Next` bind a session to that **process** and search there.
`Session.FromProcess` uses the `-a` (process) scope, so it resolves a control across whichever of the
process's windows owns it — ideal for a toolbar that may be one of several windows.
```csharp
// Screen Ruler's toolbar buttons live in PowerToys.MeasureToolUI, NOT the Settings window.
var ruler = Session.FromProcess("PowerToys.MeasureToolUI", PowerToysModule.ScreenRuler, timeoutMS: 5_000);
ruler.Find<Element>(By.AccessibilityId("Button_Spacing"), 15_000).Click();
```
> **Process name ≠ window title.** The Measure Tool's window *title* is `"PowerToys.ScreenRuler"`, but
> the *process* name winappcli's `-a` flag needs is `"PowerToys.MeasureToolUI"`. The authoritative
> process names are in the harness's `ModuleConfigData.cs`.
## Recipe 11 — Center the cursor before a coordinate measurement
```csharp
var size = System.Windows.Forms.SystemInformation.PrimaryMonitorSize; // PHYSICAL px when DPI-aware (Pitfall 12)
int cx = size.Width / 2, cy = size.Height / 2;
MouseHelper.MoveTo(cx, cy); // park at a known on-screen spot
MouseHelper.Drag(cx - 50, cy - 50, cx + 49, cy + 49); // 100x100 box centred on screen
```
Never anchor a gesture to the *current* cursor (`GetMousePosition() + 200`) — the cursor can be
anywhere (often near the bottom edge after a toolbar pops up), pushing the gesture off-screen and
producing a wrong/empty measurement. `System.Windows.Forms` flows transitively from the harness
(`UseWindowsForms=true`), so you can call `SystemInformation` without adding a reference.
**Move in steps so the overlay tracks the cursor.** A coordinate gesture must land on-screen, and the
module's overlay needs to see the cursor *move* before the click — a single `SetCursorPos` can land
without a tracked move, leaving the measurement empty. Park at a known on-screen point (screen-centre)
and move in a couple of steps:
```csharp
var (cx, cy) = ScreenCenter();
MouseHelper.MoveTo(cx - 60, cy - 60); // first move...
Thread.Sleep(200);
MouseHelper.MoveTo(cx, cy); // ...then settle on the target so the overlay is tracking
Thread.Sleep(400);
MouseHelper.LeftClick(); // or Drag(...) for a free-form box
```
## Recipe 12 — Drive a screen-capture module (cold-start + Win32 overlay presence)
Modules built on Windows.Graphics.Capture (Screen Ruler spacing, Magnifier, Text Extractor, screenshot
tools) need three habits a warm dev box hides — see [ci-stability.md](ci-stability.md) Principle 3. The
ScreenRuler port is the reference (`SelectToolAndVerify` / `MeasureWithRetry` / `ReengageTool` /
`IsMeasureOverlayPresent`).
```csharp
// 1) Detect the capture overlay with Win32 EnumWindows — NOT winappcli list-windows/Inspect, which
// attaches a UIA client and empties the live capture (Pitfall 18).
private static bool IsOverlayPresent(string processName)
{
var pids = Process.GetProcessesByName(processName).Select(p => p.Id).ToList();
return WindowControl.EnumerateProcessWindows(pids)
.Any(w => w.ClassName.Contains("OverlayWindow", StringComparison.OrdinalIgnoreCase));
}
// 2) Engage the tool, then RETRY until the authoritative signal (the overlay window) confirms — the
// tool's UIA tree exists before the window is interactive, so an early press is silently dropped.
var deadline = DateTime.UtcNow.AddSeconds(25);
while (!IsOverlayPresent(proc) && DateTime.UtcNow < deadline)
{
var tool = ruler.Find<Element>(By.AccessibilityId("Button_Spacing"), 8_000);
if (tool.GetProperty("ToggleState") != "On") tool.Click(msPostAction: 300); // guard: don't toggle it off
MouseHelper.MoveTo(cx, cy); Thread.Sleep(500); // leave the toolbar onto the surface
}
// 3) Measure, retrying IN PLACE while the clipboard is empty (the WGC first frame cold-starts). If the
// in-place retries yield nothing, re-engage ONCE (toggle off+on = a fresh capture session) to
// recover a genuine stall — never close/reopen every attempt (that resets the cold-start).
string result = MeasureWithRetry(() => { MouseHelper.MoveTo(cx, cy); MouseHelper.LeftClick(); }, maxAttempts: 5);
```
> Each test spawns its own module process = its own capture session = its own cold-start; there is no
> cross-test warming, so every capture test must tolerate the first-frame delay on its own.
---
## Pitfalls
1. **`Click` has no `msPreAction` in `.Next`.** Legacy `Click(msPreAction: 1000, msPostAction: 2000)`
`Thread.Sleep(1000); el.Click(msPostAction: 2000);`. Forgetting the pre-delay causes flaky clicks
on slow-rendering pages.
2. **`Click` (invoke) vs. `MouseClick` (real mouse).** `Click` uses UIA InvokePattern (and falls back
to Toggle/Select/Expand). For elements with **no** invoke pattern (TextBlocks, list labels, headers
whose ancestor handles the click), `Click` silently does nothing useful — use `MouseClick`.
3. **`By.Name` is a substring match and may return many hits.** Always `FindAll` + filter when the
name isn't unique. Prefer `By.AccessibilityId`.
4. **No `global` parameter.** If a legacy `Find(by, t, global: true)` reached into a popup/other
window, switch the session scope (`Session.FromProcess`) or discover the window via `WindowsFinder`.
5. **`PowerToysModule.FancyZone` was renamed to `FancyZonesEditor`.** Update the enum value.
6. **Don't launch overlay/utility module exes standalone.** Drive `ColorPicker`/`LightSwitch`/etc.
through the `PowerToysSettings` scope so the runner owns the hotkey and toggles; a standalone exe
has no runner behind it.
7. **`System.Threading.Timer` is ambiguous** in this harness (WinForms is referenced and also defines
`Timer`). Fully-qualify if you add one. (Rare in tests, common if you port harness-level code.)
8. **Cached element geometry is a snapshot.** Re-`Find` before using `X/Y/Width/Height` for a
drag/mouse-click if the UI moved since the lookup.
9. **Restore state you change.** Toggles, settings.json edits, and clipboard contents must be restored
in a `finally` so a failure mid-test doesn't poison the next one. Make cleanup tolerant
(`WindowControl.Try*`) so it never masks the real failure.
10. **First-build/NuGet errors** → run `tools\build\build-essentials.cmd` once before the per-project
build (or `dotnet restore <csproj> -p:Platform=x64`). A missing `project.assets.json` shows up as
`NETSDK1004`. Missing `Common.Dotnet.CsWinRT.props` import → CI's `verifyCommonProps.ps1` fails;
the template already includes it.
11. **`winapp.exe` missing at run time** is expected on a headless agent — the project still *builds*.
Don't treat a missing-CLI run failure as a migration defect; report build-clean + ready-to-run.
12. **Coordinate-exact tests need an `app.manifest` with `PerMonitorV2`.** Without it the test host is
DPI-unaware, so `MouseHelper`'s `SetCursorPos`/`GetCursorPos` coordinates are virtualized by the
display scale and stop matching winappcli's PHYSICAL-pixel bounds. On a 150% display a 99px drag
measured as ~149px (Screen Ruler reported `150 x 149` instead of `100 x 100`). Copy the manifest
from the module's legacy UITests project (or [templates/app.manifest](../templates/app.manifest))
and add `<ApplicationManifest>app.manifest</ApplicationManifest>` to the csproj. Regex-only
assertions (e.g. `\d+ x \d+`) don't notice the scale — only exact-value tests fail, which makes
this easy to miss.
**Why the legacy project's manifest doesn't save it:** a legacy `OutputType=Library` test runs
inside `testhost.exe` (vstest), whose manifest — not the test DLL's — governs DPI awareness, so the
legacy `app.manifest` is silently ignored and its coordinate-exact tests can't be DPI-correct on a
scaled display (the ScreenRuler legacy Bounds test fails `150 x 149` even *with* its manifest). A
`.Next` project is an `OutputType=Exe` (MTP), so ITS manifest applies to its own process — which is
why adding the manifest actually fixes the port, and can make it pass where the legacy can't.
13. **Anchor coordinate gestures to the screen centre, not the current cursor** (Recipe 11). This is
the #1 cause of "measurement is wrong/empty" — the cursor drifts to the bottom edge after a
toolbar appears.
14. **Global-hotkey activation is racy right after enabling a module.** The runner arms its keyboard
hook asynchronously, so the first chord is easily lost. Settle ~1.5s after the toggle, then
re-send the chord and poll for the window, for several attempts (SKILL Recipe 4; the ScreenRuler
`SendShortcutUntilVisible` helper is the reference).
15. **Per-test cold relaunch amplifies flakiness.** By default each `[TestMethod]` kills + relaunches
the runner, so every test pays the startup + hook-arming cost. For a suite of cheap cases against
one page, consider `ReuseScopeAcrossTests => true` (one launch per class). Content-dependent
measurements (spacing edge-detection) also vary with what's under the cursor — assert on **format**
(regex) unless the gesture is content-independent (a free-form drag like Bounds), where an exact
value is safe.
16. **Coordinate gestures break when the window/cursor is off-screen — and it only shows on CI.** A
`WindowSize` preset that resized but kept its old top-left could push the Settings window (and the
measurement area) partially off a same-sized 1920×1080 CI display, so the gesture landed off-screen
and nothing was captured (empty clipboard). It passed **locally** only because a higher-res dev
display left everything on-screen — so don't trust a local pass for coordinate tests. The harness
now **centers and clamps** `WindowSize` presets to ~90% of the display, keeping the window fully
on-screen; anchor gestures to `ScreenCenter()` (always on-screen) and move in steps (Recipe 11).
You do **not** need to minimize or move the covering window — an overlay module like the Measure
Tool captures the gesture even with the Settings window underneath (verified); the failure was the
off-screen position, not the window covering the centre.
17. **The first-run "Welcome to PowerToys" / "What's new" window appears on a fresh profile (CI) and
eats centre-screen gestures.** On a clean profile the runner opens the OOBE (Welcome) or SCOOBE
(what's-new) window — **centered and topmost** — so a coordinate measurement at screen-centre lands
on it instead of the module overlay (empty clipboard). It never shows on a dev box because your
profile already marked them seen — the *same* local-passes/CI-fails trap as Pitfall 16, and the
hardest to spot because the runner log still shows the hotkey firing and the module activating. The
harness now suppresses both in `PreTestHygiene` via
`SettingsConfigHelper.SuppressFirstRunExperience()` (seeds `oobe_settings.json`
`openedAtFirstLaunch=true` + `settings.json` `show_whats_new_after_updates=false`, mirroring the
runner's own gating). If you drive coordinate gestures and see "passes local, empty result on CI",
suspect a stray fresh-run window first.
18. **Never walk a live screen-capture window's UIA tree.** winappcli `list-windows` / `Inspect` (and
any `Find` that enumerates the overlay) attaches a UI Automation client, and walking the tree
**disturbs a Windows.Graphics.Capture session** — the very next frame comes back empty, so the
measurement is blank with no error. Detect capture overlays with Win32
`WindowControl.EnumerateProcessWindows` (by class/title) and read the result from the **clipboard**,
not the overlay's UIA. (Reading UIA on a *non*-capture window is fine.) See Recipe 12 and the
Win32-window vs UIA-element mental model in [ci-stability.md](ci-stability.md).
19. **A physical click on the FIRST interaction after a window appears is racy.** A window's UIA tree
exists a moment before the window is interactive-for-mouse-input, so a real click that lands early
is silently dropped — flaky, and only on slower agents (it cost a Win10-only "NavigationViewItem
not found" until navigation was moved to UIA invoke). Navigation is almost always the first
interaction: activate a `NavigationViewItem` with `By.AccessibilityId(...).Click()` (the harness
routes it to a coordinate-free UIA invoke), not a raw `MouseHelper`/`MouseClick`. See
[ci-stability.md](ci-stability.md) Principle 2.

View File

@@ -0,0 +1,187 @@
# Porting workflow
Two end-to-end playbooks. Pick the one matching your scenario (see SKILL.md "Pick your scenario").
Both assume you've read [framework-differences.md](framework-differences.md) and have
[api-mapping.md](api-mapping.md) open.
---
## Scenario A — Port existing legacy tests
Re-implement an existing `[Module].UITests` project (which references `UITestAutomation.csproj`) as a
new `[Module].UITests.Next` project (referencing `UITestAutomation.Next.csproj`), preserving every
test's **intent and assertions**.
### A0. Baseline the legacy suite first — ELEVATED (recommended)
Before porting, run the **legacy** suite once to learn its real local pass rate. **Run it elevated:**
the legacy harness launches PowerToys via `ProcessStartInfo { Verb = "runas" }`, so a non-elevated
test host can't complete the launch and **every test fails at startup with a misleading
`Win32Exception` cascade** — a false 0/N that looks like "the tests are broken" but is just the run
method. (This is exactly why VS Test Explorer passes them: VS runs as admin.) Don't conclude the
legacy suite is broken from a non-elevated run.
```pwsh
# 1. Build the legacy project (WinAppDriver-based, OutputType=Library).
tools\build\build.cmd -Path src\modules\<Module>\Tests\<Module>.UITests -Platform x64 -Configuration Debug
# 2. Run ELEVATED. Put the run in a .ps1 and launch it with -Verb RunAs (one UAC prompt) so the
# harness's runas launch has an elevated host. The script should start WinAppDriver + run vstest:
# $dll = "$PWD\x64\Debug\tests\<Module>.UITests\net10.0-windows10.0.26100.0\<Module>.UITests.dll"
# Start-Process "C:\Program Files (x86)\Windows Application Driver\WinAppDriver.exe" -ArgumentList "127.0.0.1","4723"
# vstest.console.exe $dll /Platform:x64 /InIsolation /Logger:"trx;LogFileName=legacy.trx" /ResultsDirectory:<dir>
# Have the script write a DONE marker at the end; poll for it, then read the .trx.
Start-Process pwsh -Verb RunAs -ArgumentList "-NoProfile","-ExecutionPolicy","Bypass","-File","<runner>.ps1"
```
Knowing the baseline tells you which failures are pre-existing product/environment issues you should
NOT expect the port to fix. A measurement failure on a scaled (non-100%) display is usually DPI (see
[patterns-and-pitfalls.md](patterns-and-pitfalls.md) Pitfall 12): the ScreenRuler legacy suite scores
**4/5** elevated here (Bounds fails at 150% scale), while the `.Next` port scores **5/5** — its Exe
`app.manifest` makes it DPI-aware where the legacy `Library` project's manifest is silently ignored.
### A1. Inventory the source
- List every `[TestClass]` and `[TestMethod]` in the legacy project. Note `[TestCategory]` tags,
`DataRow`s, and the base-ctor args (`scope`, `WindowSize`, `commandLineArgs`).
- List shared helpers (a `TestHelper`/`*Helpers` static class is common — ScreenRuler's
`TestHelper.cs` is the canonical example). Decide per-helper whether to **port it**, **inline it**,
or **drop it** (Selenium-only scaffolding usually drops).
- For each test, write a one-line statement of *what it asserts* (the behavior), independent of how the
legacy harness did it. You're re-creating that behavior, not the Selenium calls.
### A2. Map the structure
| Legacy piece | `.Next` target |
|---|---|
| `[TestClass] FooTests : UITestBase` | same shape, `using Microsoft.PowerToys.UITest.Next;` |
| ctor `: base(PowerToysSettings, WindowSize.Large)` | `: base(PowerToysModule.PowerToysSettings, WindowSize.Large)` |
| ctor `commandLineArgs: new[]{ "--enable", "Foo" }` | `enableModules: new[]{ "Foo" }` (deterministic module baseline) |
| `TestHelper.InitializeTest(this, …)` | a private setup method, or rely on `UITestBase` hygiene + an explicit nav helper |
| `[TestMethod("Foo.Bar")]` | `[TestMethod]` + keep `[TestCategory("Foo")]` |
### A3. Re-implement each test, method by method
For each legacy method:
1. **Translate the selectors** first (the highest-risk part). Replace `By.XPath`/`By.ClassName` per
[framework-differences.md §3](framework-differences.md). Prefer `By.AccessibilityId` — open the
module's XAML and find the `x:Name`/`AutomationProperties.AutomationId` the control exposes.
2. **Translate the actions** with [api-mapping.md](api-mapping.md). The frequent ones:
- `element.Click(msPreAction: N, …)` → if you relied on the pre-delay, add `Thread.Sleep(N)` then
`element.Click(msPostAction: …)` (`.Next` `Click` has no `msPreAction`).
- A click on a non-invokable element (TextBlock/ListItem whose ancestor handles it) →
`element.MouseClick(...)`.
- Selenium `Actions` drags → `element.Drag(...)` / `MouseHelper.Drag(...)`.
- `testBase.SendKeys(...)` / `Session.PerformMouseAction(...)``KeyboardHelper.*` / `MouseHelper.*`.
3. **Translate the waits.** Replace hand-rolled `while (DateTime.Now < end) { … Task.Delay(...) }`
poll loops with the built-ins: `element.WaitForProperty("ToggleState","On",t)`,
`element.WaitForValue(...)`, `Session.WaitForElement(by,t)`, `Session.WaitFor(() => …, t)`, or
`ClipboardHelper.WaitForText(...)`. Keep a custom poll only when you're polling something with no
built-in (e.g. `Process.GetProcessesByName(...)` — see the ColorPicker `WaitForProcess` helper).
4. **Translate cleanup.** Delete manual `CloseOtherApplications`/`Win+M` (the base does hygiene). For
windows the *test* spawned (overlay/editor), close them in a `finally` with
`WindowControl.TryCloseByApp("PowerToys.<Module>UI")`. Restore any toggle you flipped to its
initial state in a `finally` (see the ColorPicker example's nested `finally`).
5. **Keep the assertions identical in spirit** — same things checked, same pass/fail meaning.
### A4. Port shared helpers thoughtfully
- Start from [../templates/TestHelper.cs](../templates/TestHelper.cs) — it already implements the
common building blocks (navigate, toggle + verify process, read shortcut, discover/activate/close
the module window, clipboard, screen-center) with the right `.Next` idioms; map your legacy helper's
module-specific bits onto it rather than translating Selenium scaffolding line-by-line.
- A static `TestHelper` is fine to keep, but re-point it at the new APIs. Drop members that only
existed to work around Selenium (manual `Session.Attach` dances, STA-clipboard wrappers → use
`ClipboardHelper`).
- Shortcut-string parsing helpers (`ParseShortcutText` turning `"Win + Shift + C"` into `Key[]`) port
almost verbatim — just map `"win"``Key.LWin`. Both examples include this parser; reuse it.
### A5. Validate (write → build → run → iterate)
Build to exit 0 (see [project-setup.md §5](project-setup.md)). Then map each new `[TestMethod]` back
to the legacy method it replaces and confirm none were dropped. On a live desktop, **run in a loop**:
start with one deterministic test (the activation/toggle test), get it green, then widen to the whole
suite. UI runs expose environment-real failures that only show up live — DPI scaling, cursor drift,
and hotkey-arming races (all hit during the ScreenRuler port; see
[patterns-and-pitfalls.md](patterns-and-pitfalls.md) Pitfalls 1215). Diagnose each from the TRX
failure message + the auto-captured failure screenshots, fix, and re-run — don't just re-run hoping
for a different result.
---
## Scenario B — Greenfield from a human sign-off markdown
The module has **no** automated UI tests. Build a new `[Module].UITests` project (no `.Next` suffix)
whose tests come from the module's **manual test sign-off** document — the human checklist QA runs
before a release. `ColorPickerUITest.md` is the archetype:
```text
* Enable the Color Picker in settings and ensure that the hotkey brings up Color Picker
- [] Change `Activate Color Picker shortcut` and check the new shortcut is working
- [] Try all three `Activation behavior`s
- [] Change `Color format for clipboard` and check if the correct format is copied
...
```
### B1. Find the sign-off doc
- Look in the module's folder and its `Tests/`/`UITests/` subfolders for a `*.md` describing manual
test steps (often `<Module>UITest.md`, `<Module>Test.md`, or a section in the module README).
Search the repo for the module name + "test"/"checklist" if it's not obvious.
- If there's genuinely no doc, **ask the user** for the test spec rather than inventing coverage.
### B2. Turn each checklist item into a test intent
For every bullet, write down: **trigger → observable signal → assertion**. Classify each item by how
the new harness can drive it (see [patterns-and-pitfalls.md](patterns-and-pitfalls.md) for the
recipes):
| Checklist phrasing | Drive technique | Observable signal |
|---|---|---|
| "Enable X in settings; module runs" | toggle the page switch | `WaitForProcess("PowerToys.<M>UI", true)` |
| "Hotkey brings up X" | read shortcut from `ShortcutControl`, `KeyboardHelper.SendKeys(...)` | the module's window/overlay appears (`WindowsFinder.WaitForWindowByApp`) |
| "Change shortcut and it works" | set the new shortcut (UI or settings.json), fire it | window appears for the new chord |
| "Change format/option and output matches" | flip the setting, perform the action | clipboard/value matches (`ClipboardHelper.WaitForText`) |
| "Value is shown in the UI" | read it | `element.GetValue()` / `.Name` / `.HelpText` equals expected |
| "Select/remove item from a list" | `Find`+`Click` the item | list count / selection changes |
| "Check logs for errors" | *(usually not automatable)* | note as out-of-scope; don't fake an assertion |
### B3. Group items into test methods
- One `[TestMethod]` per coherent scenario, not necessarily one per bullet — several related bullets
(enable → read shortcut → activate → capture → verify) often belong in one end-to-end flow, exactly
like `ColorPickerEndToEndTests.NavigateReadShortcutActivateAndCapture`.
- Add `[TestCategory("<Module>")]` so the suite is filterable.
- Drive **through the Settings scope** (`base(PowerToysModule.PowerToysSettings)`) for overlay/utility
modules so the runner owns the hotkey and toggles — don't launch the module exe standalone.
### B4. Make the UI observable (flag, don't fix)
Sign-off docs assume a human's eyes. Some signals aren't UIA-readable (a transparent overlay's
displayed HEX, a canvas color). If an assertion needs a hook the product doesn't expose:
- First try the existing readouts: `GetValue()` (reads the Text binding even when
`AutomationProperties.Name` overrides the UIA Name), `Inspect(...)` tree walks, clipboard, window
geometry.
- If there's truly no signal, **flag it to the user** that a small test-only UIA hook is needed (like
ColorPicker's hidden `ColorHexAutomationPeer` TextBlock — `Visibility=Visible, Opacity=0`, bound to
the same source). Do **not** add such a hook to product code yourself without sign-off; describe it
and let the user decide.
### B5. Validate
Build to exit 0. List each checklist item and the `[TestMethod]` (or `TestContext.WriteLine` note)
that covers it, and explicitly call out any items left as manual-only (e.g. "check logs for errors").
---
## Both scenarios — definition of done
- [ ] New project builds to **exit code 0**, referencing `UITestAutomation.Next.csproj` only.
- [ ] No Selenium/Appium/`WindowsDriver`/`By.XPath`/`:4723` left anywhere.
- [ ] Registered in `PowerToys.slnx` with the `*|ARM64`/`*|x64` platform block.
- [ ] (A) Every legacy `[TestMethod]` has a `.Next` counterpart; the legacy project is untouched.
- [ ] (B) Every actionable sign-off item maps to a test or is explicitly noted as manual-only.
- [ ] Toggles/settings the test changes are restored in a `finally`; spawned windows are closed.
- [ ] No product-code edits (or any needed UIA hook is flagged to the user, not silently added).

View File

@@ -0,0 +1,173 @@
# Project setup & scaffolding
How to create, place, name, register, and build the new `.Next` test project. The starter files live
in [../templates/](../templates/).
## 1. Decide the name and location
| Scenario | Project name | Folder |
|---|---|---|
| **A — Port** (legacy UI tests exist) | `[Module].UITests.Next` | `src/modules/[Module]/Tests/[Module].UITests.Next/` |
| **B — Greenfield** (no UI tests) | `[Module].UITests` | `src/modules/[Module]/Tests/[Module].UITests/` |
Rules and judgment:
- **The `.Next` suffix exists only to avoid colliding with an existing legacy project.** If there is
nothing to live alongside (Scenario B), drop it.
- **Match the module's existing test layout.** Many modules already nest tests under a `Tests/`
folder (`MeasureTool/Tests/ScreenRuler.UITests`, `LightSwitch/Tests/LightSwitch.UITests`); others
put the UI-tests project directly under the module root (`colorPicker/ColorPicker.UITests`,
`fancyzones/FancyZones.UITests`). **Mirror whatever the module already does** — don't invent a new
structure. The path-segment count only changes the relative `..\` depth to `common\` in the csproj.
- Keep the **`AssemblyName`** matching the project name (`[Module].UITests.Next`) so logs and build
artifacts are unambiguous; there's no need to strip the `.Next` from the assembly name.
- If the legacy project has an unusual file name (e.g. `HostsEditor.UITests.csproj` inside a
`Hosts.UITests/` folder), prefer a clean `[Module].UITests.Next.csproj`; consistency with the new
examples (`ColorPicker.UITests.csproj`, `Settings.UITests.csproj`) wins.
## 2. Scaffold the csproj
Copy [../templates/Module.UITests.Next.csproj](../templates/Module.UITests.Next.csproj) and replace the
`__MODULE__` placeholder (and fix the `..\` depth on the ProjectReference). The reference csproj
(ColorPicker, whose project folder sits 3 levels under `src/`) is:
```xml
<Project Sdk="Microsoft.NET.Sdk">
<!-- Look at Directory.Build.props in root for common stuff as well -->
<Import Project="$(RepoRoot)src\Common.Dotnet.CsWinRT.props" />
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net10.0-windows10.0.26100.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<IsPackable>false</IsPackable>
<TreatWarningsAsErrors>false</TreatWarningsAsErrors>
<RootNamespace>Microsoft.__MODULE__.UITests</RootNamespace>
<AssemblyName>__MODULE__.UITests.Next</AssemblyName>
<!-- Microsoft.Testing.Platform: appears in Test Explorer AND runs via dotnet test / vstest. -->
<IsTestingPlatformApplication>true</IsTestingPlatformApplication>
<EnableMSTestRunner>true</EnableMSTestRunner>
<GenerateDocumentationFile>false</GenerateDocumentationFile>
<!-- UI tests need a live desktop; never run them as part of MSBuild. -->
<RunVSTest>false</RunVSTest>
</PropertyGroup>
<!-- Stage the built test app under <Platform>\<Configuration>\tests\ so the UI-tests build
pipeline (CopyFiles glob **/<plat>/<config>/tests/**) picks it up. -->
<PropertyGroup>
<OutputPath>$(RepoRoot)$(Platform)\$(Configuration)\tests\__MODULE__.UITests.Next\</OutputPath>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="MSTest" />
</ItemGroup>
<ItemGroup>
<!-- Adjust the ..\ depth to reach src\common from THIS project's folder. -->
<ProjectReference Include="..\..\..\common\UITestAutomation.Next\UITestAutomation.Next.csproj" />
</ItemGroup>
</Project>
```
Critical, non-negotiable bits (CI audits or the build will fail without them):
1. **`<Import Project="$(RepoRoot)src\Common.Dotnet.CsWinRT.props" />`** immediately after the
`<Project Sdk=...>` line. `.pipelines/verifyCommonProps.ps1` requires it on every `src/**` csproj.
2. **`OutputType=Exe`**, **`IsTestingPlatformApplication=true`**, **`EnableMSTestRunner=true`** — the
Microsoft.Testing.Platform runner the rest of the repo uses; this is what makes the class appear in
Test Explorer and run via `dotnet test`/`vstest.console.exe`.
3. **`<OutputPath>$(RepoRoot)$(Platform)\$(Configuration)\tests\<Name>\</OutputPath>`** — stages the
build output where the UI-tests pipeline globs (`**/<plat>/<config>/tests/**`). Without it the app
builds to `bin\` and is never picked up by the test job.
4. **`RunVSTest=false`** — UI tests must not run during MSBuild.
5. **ProjectReference to `UITestAutomation.Next.csproj` only** — never the legacy
`UITestAutomation.csproj`. Fix the `..\` depth to match the folder nesting:
- `src/modules/<M>/Tests/<M>.UITests.Next/` (4 levels under `src`) → `..\..\..\..\common\UITestAutomation.Next\UITestAutomation.Next.csproj`
- `src/modules/<M>/<M>.UITests/` (3 levels under `src`) → `..\..\..\common\UITestAutomation.Next\UITestAutomation.Next.csproj`
- `src/settings-ui/<M>.UITests/` (2 levels under `src`) → `..\..\common\UITestAutomation.Next\UITestAutomation.Next.csproj`
> Use `MSTest` (the meta-package) for a test **Exe**, matching the ColorPicker/Settings examples — not
> the bare `MSTest.TestFramework` the harness library itself uses.
## 3. Register in `PowerToys.slnx`
Add the project to [../../../../PowerToys.slnx](../../../../PowerToys.slnx) inside the module's
`<Folder>`, right next to the legacy project (Scenario A) so they're visually paired:
```xml
<Project Path="src/modules/<Module>/Tests/<Module>.UITests.Next/<Module>.UITests.Next.csproj">
<Platform Solution="*|ARM64" Project="ARM64" />
<Platform Solution="*|x64" Project="x64" />
</Project>
```
Match the `<Platform>` mapping block of the sibling projects in the same folder (every UI-tests entry
uses the `*|ARM64 → ARM64` / `*|x64 → x64` pair shown above).
## 4. Add the test class(es) and shared helper
Copy [../templates/ModuleEndToEndTests.cs](../templates/ModuleEndToEndTests.cs) into the project,
rename it to `[Module]EndToEndTests.cs` (or keep the legacy test-class names in Scenario A), and start
filling in test methods.
For anything beyond a single trivial test, also copy
[../templates/TestHelper.cs](../templates/TestHelper.cs) — a static helper with the reusable building
blocks every port needs (navigate to the page, toggle + verify the process, read the activation
shortcut, discover/activate/close the module window with patient retry, clipboard, screen-center).
Fill in the `__MODULE__` / `__MODULEUI__` / AutomationId placeholders and delete what you don't use.
This mirrors how the legacy suites are organized (a `TestHelper` + thin test classes) and is exactly
the shape of the validated ScreenRuler port.
The standard file header is required on every `.cs`:
```csharp
// Copyright (c) Microsoft Corporation
// The Microsoft Corporation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.
```
## 4b. (Coordinate-exact tests only) add a DPI-aware `app.manifest`
If any test drives the mouse by **pixel coordinates** and asserts on an **exact** value (a drag that
must measure `100 x 100`, a click at a precise point), the test host MUST be per-monitor DPI aware,
otherwise `MouseHelper`'s `SetCursorPos`/`GetCursorPos` are virtualized by the display scale and stop
matching winappcli's physical-pixel bounds (a 99px drag measured ~149px on a 150% display).
Copy [../templates/app.manifest](../templates/app.manifest) into the project (or the one from the
module's legacy UITests project) and reference it in the csproj:
```xml
<PropertyGroup>
<ApplicationManifest>app.manifest</ApplicationManifest>
</PropertyGroup>
```
Tests that only assert on **format** (regex like `\d+ x \d+`) or never touch raw coordinates don't
need the manifest — which is why ColorPicker/Settings `.Next` projects omit it.
## 5. Build & run
```pwsh
# 0. FIRST build of a new project: restore so project.assets.json exists (else NETSDK1004).
dotnet restore src\modules\<Module>\Tests\<Module>.UITests.Next\<Module>.UITests.Next.csproj -p:Platform=x64
# (or run tools\build\build-essentials.cmd once at the start of the session.)
# 1. Build only this project (fast). Exit code 0 = success.
tools\build\build.cmd -Path src\modules\<Module>\Tests\<Module>.UITests.Next -Platform x64 -Configuration Debug
# 2. Run (needs a live desktop + winapp.exe). A .Next project is a Microsoft.Testing.Platform Exe,
# so run the produced exe directly (Test Explorer also works). Filter + TRX report for a tight loop:
$exe = "$PWD\x64\Debug\tests\<Module>.UITests.Next\net10.0-windows10.0.26100.0\<Module>.UITests.Next.exe"
& $exe --filter "TestCategory=<Cat>" --report-trx --report-trx-filename run.trx --results-directory .\TestResults\<Module>
# --filter accepts "TestCategory=X" or "FullyQualifiedName~Y"; omit it to run everything. Exit 0 = all passed.
```
- On build failure, read `build.<Configuration>.<Platform>.errors.log` next to the project.
- `winapp.exe` is a **run-time** prerequisite only (`winget install Microsoft.winappcli`, or set
`WINAPP_CLI_PATH`). A migration that compiles clean is valid even where the CLI/desktop is absent;
say so and list coverage.
- `dotnet test` also works for a one-shot run, but prefer the produced exe for a fast iterate loop and
do **not** run UI tests from inside an MSBuild step — they need an interactive session.

View File

@@ -0,0 +1,54 @@
<Project Sdk="Microsoft.NET.Sdk">
<!--
TEMPLATE — copy into src/modules/<Module>/Tests/<Module>.UITests.Next/ and replace:
__MODULE__ -> the module name used for the project/assembly (e.g. ColorPicker, ScreenRuler)
the ProjectReference ..\ depth -> enough ..\ to reach src\common from THIS folder
(see references/project-setup.md §2)
For a GREENFIELD project (module had no UI tests), rename the file and AssemblyName to drop
the ".Next" suffix (use <Module>.UITests).
-->
<!-- REQUIRED: must be the first line after <Project>. CI (verifyCommonProps.ps1) audits this. -->
<Import Project="$(RepoRoot)src\Common.Dotnet.CsWinRT.props" />
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net10.0-windows10.0.26100.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<IsPackable>false</IsPackable>
<TreatWarningsAsErrors>false</TreatWarningsAsErrors>
<RootNamespace>Microsoft.__MODULE__.UITests</RootNamespace>
<AssemblyName>__MODULE__.UITests.Next</AssemblyName>
<!--
Microsoft.Testing.Platform: the modern runner Directory.Build.props enables repo-wide, so this
test class appears in Test Explorer AND can be run via `dotnet test` / `vstest.console.exe`.
-->
<IsTestingPlatformApplication>true</IsTestingPlatformApplication>
<EnableMSTestRunner>true</EnableMSTestRunner>
<GenerateDocumentationFile>false</GenerateDocumentationFile>
<!-- UI tests need a live desktop; never run them as part of MSBuild. -->
<RunVSTest>false</RunVSTest>
</PropertyGroup>
<!--
Stage the built test app under <Platform>\<Configuration>\tests\ so the UI-tests build pipeline
(CopyFiles glob **/<plat>/<config>/tests/**) picks it up, matching the other *.UITests projects.
Without this it builds to bin\ and is never staged into the artifact.
-->
<PropertyGroup>
<OutputPath>$(RepoRoot)$(Platform)\$(Configuration)\tests\__MODULE__.UITests.Next\</OutputPath>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="MSTest" />
</ItemGroup>
<ItemGroup>
<!-- Reference the NEW harness only. NEVER reference ..\common\UITestAutomation\UITestAutomation.csproj.
Adjust the ..\ depth so it resolves from this project's folder to src\common. -->
<ProjectReference Include="..\..\..\..\common\UITestAutomation.Next\UITestAutomation.Next.csproj" />
</ItemGroup>
</Project>

View File

@@ -0,0 +1,144 @@
// Copyright (c) Microsoft Corporation
// The Microsoft Corporation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.
// TEMPLATE — a starting scaffold for a `.Next` UI-test class. Replace __MODULE__ / __MODULEUI__ /
// selectors with the real values for your module, delete what you don't need, and add test methods.
// See the skill's references/patterns-and-pitfalls.md for the full recipe catalog and
// ColorPickerEndToEndTests.cs for a complete worked example.
using System.Diagnostics;
using Microsoft.PowerToys.UITest.Next;
using Microsoft.VisualStudio.TestTools.UnitTesting;
namespace Microsoft.__MODULE__.UITests;
[TestClass]
public class __MODULE__EndToEndTests : UITestBase
{
// Drive overlay/utility modules through the Settings scope so the runner owns the activation
// hotkey and module toggles. `enableModules` enables ONLY the listed modules (disabling the rest)
// before launch — pass just the one under test so the runner boots a single module (faster on a
// fresh CI profile + isolated from other modules' hotkeys/overlays). The name is the settings.json
// "enabled" key (note spaces, e.g. "Measure Tool", "PowerToys Run"). Add a WindowSize if needed.
public __MODULE__EndToEndTests()
: base(PowerToysModule.PowerToysSettings, enableModules: new[] { "__MODULE_SETTINGS_KEY__" })
{
}
[TestMethod]
[TestCategory("__MODULE__")]
public void ExampleScenario()
{
try
{
RunTest();
}
finally
{
// Tolerant cleanup — close any window the test spawned, then Settings. Never throws, so it
// can't mask the real failure.
WindowControl.TryCloseByApp("__MODULEUI__");
WindowControl.TryCloseByApp("PowerToys.Settings");
}
}
private void RunTest()
{
// 1. Navigate to the module's Settings page (adjust selector / nav-item id for your module).
// Some pages use a left-nav NavigationViewItem by AutomationId; others a dashboard label.
// Session.Find<NavigationViewItem>(By.AccessibilityId("__MODULE__NavItem")).Click(msPostAction: 500);
// 2. Find the page enable toggle and verify the module process follows it.
var toggle = Find<ToggleSwitch>(By.Name("__MODULE__"));
bool initialIsOn = toggle.IsOn;
try
{
if (!toggle.IsOn)
{
toggle.Toggle(true);
Assert.IsTrue(toggle.WaitForProperty("ToggleState", "On", 5_000), "Toggle didn't turn On.");
Assert.IsTrue(WaitForProcess("__MODULEUI__", expected: true, 10_000), "Process didn't start.");
}
// 3. Read the activation shortcut from the ShortcutControl's EditButton (HelpText carries
// the readable chord, e.g. "Win + Shift + C").
var editButton = Find<Button>(By.AccessibilityId("EditButton"));
Key[] keys = ParseShortcutText(editButton.HelpText);
Assert.IsTrue(keys.Length > 0, $"Could not parse shortcut '{editButton.HelpText}'.");
// 4. Fire the hotkey (retry — the runner arms its hook asynchronously) and wait for the
// module window/overlay to appear.
Session? appWindow = null;
for (int attempt = 1; attempt <= 3 && appWindow is null; attempt++)
{
KeyboardHelper.SendKeys(keys);
appWindow = WindowsFinder.WaitForWindowByApp("__MODULEUI__", _ => true, timeoutMS: 2_500);
}
Assert.IsNotNull(appWindow, "Module window did not appear after firing the shortcut.");
// 5. ... assert on the module's UI (read values, click, inspect tree, check clipboard) ...
TestContext.WriteLine($"Module window appeared: hwnd={appWindow!.WindowHandle}");
}
finally
{
// Restore the toggle to its initial state, tolerantly.
try
{
if (toggle.IsOn != initialIsOn)
{
toggle.Toggle(initialIsOn);
}
}
catch
{
}
}
}
/// <summary>Poll for a process becoming present/absent (no built-in wait for this).</summary>
private static bool WaitForProcess(string name, bool expected, int timeoutMS)
{
var deadline = DateTime.UtcNow + TimeSpan.FromMilliseconds(timeoutMS);
while (DateTime.UtcNow < deadline)
{
if ((Process.GetProcessesByName(name).Length > 0) == expected)
{
return true;
}
Thread.Sleep(250);
}
return false;
}
/// <summary>Parse a UI shortcut string like "Win + Shift + C" into the Key chord.</summary>
private static Key[] ParseShortcutText(string shortcutText)
{
var parts = shortcutText.Split(new[] { " + ", "+", " " }, StringSplitOptions.RemoveEmptyEntries);
var keys = new List<Key>();
foreach (var raw in parts)
{
var part = raw.Trim().ToLowerInvariant();
Key? key = part switch
{
"win" or "windows" => Key.LWin,
"ctrl" or "control" => Key.Ctrl,
"shift" => Key.Shift,
"alt" => Key.Alt,
_ when part.Length == 1 && part[0] >= 'a' && part[0] <= 'z' =>
(Key)Enum.Parse(typeof(Key), part.ToUpperInvariant()),
_ => null,
};
if (key.HasValue)
{
keys.Add(key.Value);
}
}
return keys.ToArray();
}
}

View File

@@ -0,0 +1,218 @@
// Copyright (c) Microsoft Corporation
// The Microsoft Corporation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.
// TEMPLATE — a static helper for a `.Next` UI-test project, distilled from the validated ScreenRuler
// port. Copy alongside ModuleEndToEndTests.cs, then:
// • Replace __MODULE__ (project name) and __MODULEUI__ (the module's PROCESS name, e.g.
// "PowerToys.MeasureToolUI" — NOT the window title; see ModuleConfigData.cs in the harness).
// • Fill in the AutomationIds for your module's nav item(s), toggle, and shortcut card from the
// module's XAML (or discover them live: `winapp ui search "<id>" -a PowerToys.Settings --json`).
// • Delete the helpers you don't need. Keep each helper ADAPTABLE — every module is different.
// See references/patterns-and-pitfalls.md for the full recipe catalog these are based on.
using Microsoft.PowerToys.UITest.Next;
using Microsoft.VisualStudio.TestTools.UnitTesting;
namespace Microsoft.__MODULE__.UITests;
public static class TestHelper
{
// ── Customize: AutomationIds + process name ───────────────────────────────────────────────
// The module's PROCESS name (winappcli -a). Window TITLE may differ — use the process name.
public const string ModuleProcess = "__MODULEUI__";
// Left-nav item AutomationId for the module's Settings page, and its parent group (if the item
// lives under a collapsible group like "System Tools"). Set ParentNavItemId to null if there's none.
public const string NavItemId = "__MODULE__NavItem";
public const string? ParentNavItemId = "SystemToolsNavItem";
// The page enable ToggleSwitch and the ShortcutControl card AutomationIds.
public const string ToggleId = "Toggle___MODULE__";
public const string ShortcutCardId = "Shortcut___MODULE__";
// ── Navigation ────────────────────────────────────────────────────────────────────────────
/// <summary>Navigate to the module's Settings page (expanding its parent nav group if needed).</summary>
public static void NavigateToPage(UITestBase testBase)
{
// A collapsible parent group hides its children until expanded; expand only when the child
// isn't already in the tree (re-clicking an expanded group would collapse it).
if (ParentNavItemId is not null && !testBase.Session.Has(By.AccessibilityId(NavItemId), 500))
{
testBase.Session.Find<NavigationViewItem>(By.AccessibilityId(ParentNavItemId), 5000).Click(msPostAction: 500);
}
testBase.Session.Find<NavigationViewItem>(By.AccessibilityId(NavItemId), 5000).Click(msPostAction: 800);
}
// ── Toggle ────────────────────────────────────────────────────────────────────────────────
/// <summary>Set the page enable toggle and wait for the UI to reflect the new state.</summary>
public static ToggleSwitch SetToggle(UITestBase testBase, bool enable)
{
var toggle = testBase.Session.Find<ToggleSwitch>(By.AccessibilityId(ToggleId), 5000);
toggle.Toggle(enable);
toggle.WaitForProperty("ToggleState", enable ? "On" : "Off", 5000);
return toggle;
}
/// <summary>Set the toggle and assert it (and optionally the module process) reached the state.</summary>
public static void SetAndVerifyToggle(UITestBase testBase, bool enable, bool verifyProcess = false, int timeoutMs = 10_000)
{
var toggle = SetToggle(testBase, enable);
Assert.AreEqual(enable, toggle.IsOn, $"Toggle should be {(enable ? "On" : "Off")}.");
if (verifyProcess)
{
Assert.IsTrue(
WaitForProcess(ModuleProcess, expected: enable, timeoutMs),
$"Process '{ModuleProcess}' should be {(enable ? "running" : "stopped")} after toggling.");
}
}
// ── Activation shortcut ───────────────────────────────────────────────────────────────────
/// <summary>Read the activation shortcut from the ShortcutControl's EditButton HelpText.</summary>
public static Key[] ReadActivationShortcut(UITestBase testBase)
{
var card = testBase.Session.Find<Element>(By.AccessibilityId(ShortcutCardId), 5000);
var editButton = card.Find<Element>(By.AccessibilityId("EditButton"), 5000);
return ParseShortcutText(editButton.HelpText);
}
/// <summary>Parse "Win + Ctrl + Shift + M" into a Key chord (note: "win" maps to <see cref="Key.LWin"/>).</summary>
public static Key[] ParseShortcutText(string shortcutText)
{
var keys = new List<Key>();
if (string.IsNullOrEmpty(shortcutText))
{
return keys.ToArray();
}
foreach (var raw in shortcutText.Split(new[] { " + ", "+", " " }, StringSplitOptions.RemoveEmptyEntries))
{
var part = raw.Trim().ToLowerInvariant();
Key? key = part switch
{
"win" or "windows" => Key.LWin,
"ctrl" or "control" => Key.Ctrl,
"shift" => Key.Shift,
"alt" => Key.Alt,
_ when part.Length == 1 && part[0] >= 'a' && part[0] <= 'z' =>
(Key)Enum.Parse(typeof(Key), part.ToUpperInvariant()),
_ => null,
};
if (key.HasValue)
{
keys.Add(key.Value);
}
}
return keys.ToArray();
}
// ── Module window lifecycle ───────────────────────────────────────────────────────────────
/// <summary>True when at least one of the module's windows is open.</summary>
public static bool IsModuleUIOpen() => WindowsFinder.ListByApp(ModuleProcess).Count > 0;
/// <summary>Poll until the module UI reaches the requested presence.</summary>
public static bool WaitForModuleUIState(bool shouldBeOpen, int timeoutMs = 5000, int pollMs = 100)
{
var deadline = DateTime.UtcNow.AddMilliseconds(timeoutMs);
while (DateTime.UtcNow < deadline)
{
if (IsModuleUIOpen() == shouldBeOpen)
{
return true;
}
Thread.Sleep(pollMs);
}
return false;
}
public static bool WaitForModuleUI(int timeoutMs = 5000) => WaitForModuleUIState(true, timeoutMs);
public static bool WaitForModuleUIToDisappear(int timeoutMs = 5000) => WaitForModuleUIState(false, timeoutMs);
/// <summary>
/// Send the activation chord, retrying until the module UI appears. The runner arms its keyboard
/// hook asynchronously after the module is enabled, so the first chord is easily lost — settle
/// first, then retry (see Recipe 4 / Pitfall 14).
/// </summary>
public static bool SendShortcutUntilVisible(UITestBase testBase, Key[] activationKeys, int attempts = 5, int perAttemptMs = 3000)
{
Thread.Sleep(1500); // let the just-enabled module register its global hotkey
for (int i = 0; i < attempts; i++)
{
KeyboardHelper.SendKeys(activationKeys);
if (WaitForModuleUI(perAttemptMs))
{
return true;
}
}
return false;
}
/// <summary>
/// Activate the module via its shortcut and return a PROCESS-scoped session for its window(s).
/// Process scope (<see cref="Session.FromProcess"/>) resolves controls across whichever of the
/// module's windows owns them — the winappcli equivalent of the legacy <c>global: true</c> Find.
/// </summary>
public static Session ActivateModule(UITestBase testBase, Key[] activationKeys, string testName)
{
ClipboardHelper.Clear();
Assert.IsTrue(
SendShortcutUntilVisible(testBase, activationKeys),
$"Module UI should appear after the activation shortcut for {testName}: {string.Join(" + ", activationKeys)}");
return Session.FromProcess(ModuleProcess, PowerToysModule.PowerToysSettings, timeoutMS: 5000);
}
/// <summary>Close the module UI if open (best-effort, tolerant — safe in a finally).</summary>
public static void CloseModuleUI(UITestBase testBase)
{
if (!IsModuleUIOpen())
{
return;
}
// Prefer an in-UI Close button if the module has one; otherwise WM_CLOSE every window.
// try { Session.FromProcess(ModuleProcess).Find<Element>(By.AccessibilityId("Button_Close"), 2000).Click(); } catch { }
WindowControl.TryCloseByApp(ModuleProcess);
}
// ── Utilities ─────────────────────────────────────────────────────────────────────────────
/// <summary>Poll for a process becoming present/absent (no built-in wait for this).</summary>
public static bool WaitForProcess(string processName, bool expected, int timeoutMs)
{
var deadline = DateTime.UtcNow.AddMilliseconds(timeoutMs);
while (DateTime.UtcNow < deadline)
{
if ((System.Diagnostics.Process.GetProcessesByName(processName).Length > 0) == expected)
{
return true;
}
Thread.Sleep(250);
}
return false;
}
/// <summary>
/// Primary-monitor centre in PHYSICAL pixels — the right anchor for coordinate gestures (don't
/// offset from the current cursor, which can be off-screen). Correct only when the test host is
/// per-monitor DPI aware (add the app.manifest, Pitfall 12); otherwise the size is virtualized.
/// </summary>
public static (int X, int Y) ScreenCenter()
{
var size = System.Windows.Forms.SystemInformation.PrimaryMonitorSize;
return (size.Width / 2, size.Height / 2);
}
}

View File

@@ -0,0 +1,33 @@
<?xml version="1.0" encoding="utf-8"?>
<!--
TEMPLATE app.manifest for a .Next UI-test project.
ADD THIS ONLY for projects with COORDINATE-EXACT tests (mouse drag/click asserting on exact
pixel/measurement values, e.g. Screen Ruler's Bounds "100 x 100"). Without PerMonitorV2 the test
host is DPI-unaware and MouseHelper's SetCursorPos/GetCursorPos coordinates are virtualized by the
display scale factor, so they no longer match the PHYSICAL pixels winappcli reports (a 99px drag
measured ~149px on a 150% display).
Wire it into the csproj:
<PropertyGroup>
<ApplicationManifest>app.manifest</ApplicationManifest>
</PropertyGroup>
Replace __MODULE__ in the assemblyIdentity name.
-->
<assembly manifestVersion="1.0" xmlns="urn:schemas-microsoft-com:asm.v1">
<assemblyIdentity version="1.0.0.0" name="__MODULE__.UITests.Next.app"/>
<compatibility xmlns="urn:schemas-microsoft-com:compatibility.v1">
<application>
<!-- Windows 10+ feature support for unpackaged apps. -->
<supportedOS Id="{8e0f7a12-bfb3-4fe8-b9a5-48fd50a15a9a}" />
</application>
</compatibility>
<application xmlns="urn:schemas-microsoft-com:asm.v3">
<windowsSettings>
<dpiAwareness xmlns="http://schemas.microsoft.com/SMI/2016/WindowsSettings">PerMonitorV2</dpiAwareness>
</windowsSettings>
</application>
</assembly>