Mirrors/PowerToys
1
0
Fork 0
mirror of https://github.com/microsoft/PowerToys.git synced 2026-08-29 10:09:43 +02:00
Code Issues Packages Projects Releases Wiki Activity

[UITests] New framework around WinApp CLI, no WinAppDriver or Selenium. (#48467)

Browse Source 
# Add winappcli-based UI test harness (no WinAppDriver / Selenium)

## Summary

Introduces a new UI test harness — `Microsoft.PowerToys.UITest.Next` —
that drives PowerToys
modules through Microsoft's
[winappcli](https://github.com/microsoft/WinAppCli) (UI Automation
CLI) instead of WinAppDriver + Selenium. Engine is a single executable
shelled out from
C#; no third-party NuGet packages, no driver process, no Appium server.
Adds two real consumers:
a full ColorPicker end-to-end scenario and a Settings shell navigation
smoke test.

This is opt-in and additive — the existing `UITestAutomation` library
and the
WinAppDriver-based test projects are untouched. Both can coexist while
we evaluate the new
harness.

Inspired in part by
[#48414](https://github.com/microsoft/PowerToys/pull/48414), which lands
the same architectural bet (winappcli, AutomationId selectors, no
WinAppDriver) at a smaller
scope. This PR generalizes it into a reusable library.

## Why

WinAppDriver + Selenium is a legacy pre-agentic solution that is no
longer actively maintained. It's unreliable, heavyweight, and slow. To
achieve 100% UI test coverage, we should leverage modern, reliable
solutions, and WinApp CLI is a strong candidate.

## What's in this PR

### Harness library —
[`src/common/UITestAutomation.Next/`](src/common/UITestAutomation.Next/)

| File | Purpose |
|---|---|
| [`WinappCli.cs`](src/common/UITestAutomation.Next/WinappCli.cs) |
Process wrapper around `winapp.exe`. `Invoke` / `InvokeAssertSuccess` /
`InvokeJson` / `IsAvailable` / `TryResolveExecutable`. `Result` carries
the args and emits `DescribeFailure()` like `winapp ui invoke X -w 12345
-> exit 1; stderr: ...` |
| [`Session.cs`](src/common/UITestAutomation.Next/Session.cs) | Test
session, scoped by either HWND (`-w`) or process (`-a`) via
`TargetScope`. `Find<T>` / `FindAll<T>` / `Inspect` / `Screenshot` /
`SendKeys`. `Session.FromProcess(...)` factory for the
single-window-per-process case |
|
[`SessionHelper.cs`](src/common/UITestAutomation.Next/SessionHelper.cs)
| Owns the launch + window-readiness flow. Static `EnsureRunning(scope,
timeout)` returns whether the call had to launch (so cleanup only kills
what we started). Uses `UseShellExecute=true` so child handles don't
keep MSTest hanging |
| [`UITestBase.cs`](src/common/UITestAutomation.Next/UITestBase.cs) |
MSTest base class. Pre-flights `WinappCli.IsAvailable()` once per
process and fails fast with the install hint if `winapp.exe` isn't on
PATH |
| [`Element/*.cs`](src/common/UITestAutomation.Next/Element/) |
`Element`, `Button`, `ToggleSwitch`, `TextBox`, `NavigationViewItem`,
`Window`. `Click` / `MouseClick` / `Focus` / `GetProperty` / `GetValue`
/ `HelpText` / `WaitForProperty` / `WaitForGone` plus coords
(`X`/`Y`/`Width`/`Height`) |
| [`By.cs`](src/common/UITestAutomation.Next/By.cs) | `By.Name` /
`By.AccessibilityId` / `By.Id` / `By.Slug` |
| [`Windows.cs`](src/common/UITestAutomation.Next/Windows.cs) |
`WindowsFinder.ListAll` / `ListByApp` / `WaitForWindowByApp` /
`WaitForWindowByProcess`. Notes the winappcli bug where unfiltered
`list-windows` drops untitled windows |
|
[`WindowControl.cs`](src/common/UITestAutomation.Next/WindowControl.cs)
| Tolerant Win32 helpers — `TryCloseByApp` / `TryFocusByApp` /
`SafeCloseAndFocus` / `TryKillProcess` — for `finally` blocks |
|
[`KeyboardHelper.cs`](src/common/UITestAutomation.Next/KeyboardHelper.cs)
| Hybrid `keybd_event` + `SendKeys.SendWait` chord sender — required for
global PowerToys hotkeys |
| [`MouseHelper.cs`](src/common/UITestAutomation.Next/MouseHelper.cs) |
`MoveTo` / `LeftClick` / `RightClick` / `LeftClickAt` Win32 wrappers |
|
[`ClipboardHelper.cs`](src/common/UITestAutomation.Next/ClipboardHelper.cs)
| STA-thread `Clipboard` access with `WaitForText` |
|
[`ModuleConfigData.cs`](src/common/UITestAutomation.Next/ModuleConfigData.cs)
| `PowerToysModule` enum + path/process-name resolution |

### Tests


**[`src/modules/colorPicker/ColorPicker.UITests/`](src/modules/colorPicker/ColorPicker.UITests/)**
— replaces the previous empty `UITest-ColorPicker` stub. One test,
[`ColorPickerEndToEndTests.NavigateReadShortcutActivateAndCapture`](src/modules/colorPicker/ColorPicker.UITests/ColorPickerEndToEndTests.cs),
drives the full E2E:
1. Navigate to the Color Picker page via the dashboard utilities stack
2. Toggle the module OFF, verify `PowerToys.ColorPickerUI` exits; toggle
ON, verify it respawns
3. Read the activation shortcut from the page's `ShortcutControl`
(`EditButton.HelpText`)
4. Clear clipboard, park cursor, send the chord
5. Wait for the picker overlay window
6. Read the displayed HEX from a hidden XAML automation peer (see below)
7. Left-click to capture; assert the clipboard value matches the peer's
HEX
8. Wait for the editor window and assert the captured color appears in
its tree


**[`src/settings-ui/Settings.UITests/`](src/settings-ui/Settings.UITests/)**
— `SettingsNavigationSmokeTests.NavigationItem_NavigatesWithoutCrashing`
is one `[TestMethod]` parameterized with `[DynamicData]`, producing 31
discrete results — one per `NavigationViewItem` in
[`ShellPage.xaml`](src/settings-ui/Settings.UI/SettingsXAML/Views/ShellPage.xaml).
For each item: navigate, settle 250ms, assert `PowerToys.Settings` is
still alive. Catches FailFast regressions in
`ShellViewModel.Frame_NavigationFailed` that pure-logic unit tests can't
reach (the failure path needs a `NavigationFailedEventArgs` which is a
sealed WinRT projection).

### Product change


**[`src/modules/colorPicker/ColorPickerUI/Views/MainView.xaml`](src/modules/colorPicker/ColorPickerUI/Views/MainView.xaml)**
— adds a hidden `TextBlock` automation peer:

```xml
<TextBlock
    x:Name="ColorHexAutomationPeer"
    AutomationProperties.AutomationId="ColorHexAutomationPeer"
    IsHitTestVisible="False"
    Opacity="0"
    Text="{Binding ColorText}" />
```

The visible `ColorTextBlock` has `AutomationProperties.Name="{Binding
ColorName}"`, which masks the HEX value in the UIA tree (you see "White"
instead of `#FFFFFF`). This zero-impact peer mirrors `ColorText` so
tests can read the actually-displayed HEX. `Opacity=0` +
`IsHitTestVisible=False` keep it out of the visual layout and out of
accessibility focus.

### Project wiring

- [`PowerToys.slnx`](PowerToys.slnx) — registers `UITestAutomation.Next`
under `/common/`, `ColorPicker.UITests` under
`/modules/colorpicker/Tests/`, and `Settings.UITests` under
`/settings-ui/Tests/`. Original `UITest-ColorPicker` stub csproj
removed.
-
[`.github/actions/spell-check/expect.txt`](.github/actions/spell-check/expect.txt)
— adds `winapp` / `winappcli`.

### Not in this PR

- No pipeline changes. `winapp.exe` is expected to be pre-staged on the
test agent image. If it's missing, `UITestBase` fails the first test
with the install hint (`winget install Microsoft.winappcli`) rather than
producing 30 opaque per-test errors.
- No changes to the legacy `UITestAutomation` library or any of the
existing `*.UITests` projects.

## Validation

- All three projects build clean on `x64|Debug` (empty
`build.<config>.<plat>.errors.log`):
  - `src/common/UITestAutomation.Next/`
  - `src/modules/colorPicker/ColorPicker.UITests/`
  - `src/settings-ui/Settings.UITests/`
- Both tests run in Test Explorer / `dotnet test` via
Microsoft.Testing.Platform (already enabled repo-wide in
`Directory.Build.props`).
- Local runs: ColorPicker E2E green; Settings smoke green across all 31
nav items.
- `winapp 0.3.2` from `winget install Microsoft.winappcli`.

## Notes for reviewers

- **`UseShellExecute = true`** in `SessionHelper.EnsureRunning` is
intentional — `false` makes child processes inherit the test host's
stdin/stdout/stderr handles, which keeps MTP/Test Explorer marking the
run as "in progress" until the spawned PowerToys exits.
- **Process-scope (`-a`) targeting** in the Settings smoke test handles
single-instance handoff: the EXE you launch may exit with code 0
immediately after signalling an existing owner, so the alive check uses
`Process.GetProcessesByName` rather than the launcher PID.
- **AutomationId-only selectors** in the Settings smoke list keep the
test localization-independent. Parent groups have
`SelectsOnInvoked="False"` and only expand on click — `Element.Click`
tries `InvokePattern → TogglePattern → SelectionItemPattern →
ExpandCollapsePattern` so the same call works for both leaves and
groups.
- **Untitled-window discovery**: filtered `winapp ui list-windows -a
<name>` returns windows that the unfiltered call drops (e.g. ColorPicker
editor). `WindowsFinder.ListByApp` uses the filtered form. Reported
upstream.

## Before Merge

- Add the `winappcli` install step to the UI-test pipeline.
This commit is contained in:
Gleb Khmyznikov
2026-07-07 22:58:08 -07:00
committed by GitHub
parent f5c6f50e30
commit 74e6c3ad79
57 changed files with 5837 additions and 75 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
PowerToys.slnx
View File

@@ -9,11 +9,11 @@
<Platform Solution="*|ARM64" Project="ARM64" />
<Platform Solution="*|x64" Project="x64" />
</Project>
<Project Path="src/common/Common.UI/Common.UI.csproj">
<Project Path="src/common/Common.UI.Controls/Common.UI.Controls.csproj">
<Platform Solution="*|ARM64" Project="ARM64" />
<Platform Solution="*|x64" Project="x64" />
</Project>
<Project Path="src/common/Common.UI.Controls/Common.UI.Controls.csproj">
<Project Path="src/common/Common.UI/Common.UI.csproj">
<Platform Solution="*|ARM64" Project="ARM64" />
<Platform Solution="*|x64" Project="x64" />
</Project>
@@ -54,10 +54,14 @@
<Platform Solution="*|ARM64" Project="ARM64" />
<Platform Solution="*|x64" Project="x64" />
</Project>
<Project Path="src/common/UITestAutomation.Next/UITestAutomation.Next.csproj">
<Platform Solution="*|ARM64" Project="ARM64" />
<Platform Solution="*|x64" Project="x64" />
</Project>
<Project Path="src/common/UnitTests-CommonLib/UnitTests-CommonLib.vcxproj" Id="1a066c63-64b3-45f8-92fe-664e1cce8077" />
<Project Path="src/common/UnitTests-CommonUtils/UnitTests-CommonUtils.vcxproj" Id="8b5cfb38-ccba-40a8-ad7a-89c57b070884" />
<Project Path="src/common/updating/updating.vcxproj" Id="17da04df-e393-4397-9cf0-84dabe11032e" />
<Project Path="src/common/updating/UnitTests/UpdatingUnitTests.vcxproj" Id="a1b2c3d4-e5f6-7890-abcd-ef1234567890" />
<Project Path="src/common/updating/updating.vcxproj" Id="17da04df-e393-4397-9cf0-84dabe11032e" />
<Project Path="src/common/version/version.vcxproj" Id="cc6e41ac-8174-4e8a-8d22-85dd7f4851df" />
</Folder>
<Folder Name="/common/interop/">
@@ -190,6 +194,10 @@
<Platform Solution="*|ARM64" Project="ARM64" />
<Platform Solution="*|x64" Project="x64" />
</Project>
<Project Path="src/modules/colorPicker/ColorPicker.UITests/ColorPicker.UITests.csproj">
<Platform Solution="*|ARM64" Project="ARM64" />
<Platform Solution="*|x64" Project="x64" />
</Project>
</Folder>
<Folder Name="/modules/CommandPalette/">
<Project Path="src/modules/cmdpal/CmdPalKeyboardService/CmdPalKeyboardService.vcxproj" Id="5f63c743-f6ce-4dba-a200-2b3f8a14e8c2" />
@@ -200,11 +208,11 @@
</Project>
</Folder>
<Folder Name="/modules/CommandPalette/Built-in Extensions/">
<Project Path="src/modules/cmdpal/ext/Microsoft.CmdPal.Ext.Apps/Microsoft.CmdPal.Ext.Apps.csproj">
<Project Path="src/modules/cmdpal/ext/Microsoft.CmdPal.Ext.Actions/Microsoft.CmdPal.Ext.Actions.csproj">
<Platform Solution="*|ARM64" Project="ARM64" />
<Platform Solution="*|x64" Project="x64" />
</Project>
<Project Path="src/modules/cmdpal/ext/Microsoft.CmdPal.Ext.Actions/Microsoft.CmdPal.Ext.Actions.csproj">
<Project Path="src/modules/cmdpal/ext/Microsoft.CmdPal.Ext.Apps/Microsoft.CmdPal.Ext.Apps.csproj">
<Platform Solution="*|ARM64" Project="ARM64" />
<Platform Solution="*|x64" Project="x64" />
</Project>
@@ -718,11 +726,11 @@
</Project>
</Folder>
<Folder Name="/modules/PowerDisplay/">
<Project Path="src/modules/powerdisplay/PowerDisplay.Models/PowerDisplay.Models.csproj">
<Project Path="src/modules/powerdisplay/PowerDisplay.Lib/PowerDisplay.Lib.csproj">
<Platform Solution="*|ARM64" Project="ARM64" />
<Platform Solution="*|x64" Project="x64" />
</Project>
<Project Path="src/modules/powerdisplay/PowerDisplay.Lib/PowerDisplay.Lib.csproj">
<Project Path="src/modules/powerdisplay/PowerDisplay.Models/PowerDisplay.Models.csproj">
<Platform Solution="*|ARM64" Project="ARM64" />
<Platform Solution="*|x64" Project="x64" />
</Project>
@@ -1099,6 +1107,10 @@
<Platform Solution="*|ARM64" Project="ARM64" />
<Platform Solution="*|x64" Project="x64" />
</Project>
<Project Path="src/settings-ui/Settings.UITests/Settings.UITests.csproj">
<Platform Solution="*|ARM64" Project="ARM64" />
<Platform Solution="*|x64" Project="x64" />
</Project>
</Folder>
<Folder Name="/Solution Items/">
<File Path=".vsconfig" />
@@ -1130,14 +1142,14 @@
<BuildDependency Project="src/modules/fancyzones/editor/FancyZonesEditor/FancyZonesEditor.csproj" />
<BuildDependency Project="src/modules/fancyzones/FancyZonesLib/FancyZonesLib.vcxproj" />
<BuildDependency Project="src/modules/fancyzones/FancyZonesModuleInterface/FancyZonesModuleInterface.vcxproj" />
<BuildDependency Project="src/modules/GrabAndMove/GrabAndMove/GrabAndMove.vcxproj" />
<BuildDependency Project="src/modules/GrabAndMove/GrabAndMoveModuleInterface/GrabAndMoveModuleInterface.vcxproj" />
<BuildDependency Project="src/modules/imageresizer/dll/ImageResizerExt.vcxproj" />
<BuildDependency Project="src/modules/imageresizer/ui/ImageResizerUI.csproj" />
<BuildDependency Project="src/modules/keyboardmanager/dll/KeyboardManager.vcxproj" />
<BuildDependency Project="src/modules/launcher/Microsoft.Launcher/Microsoft.Launcher.vcxproj" />
<BuildDependency Project="src/modules/LightSwitch/LightSwitchModuleInterface/LightSwitchModuleInterface.vcxproj" />
<BuildDependency Project="src/modules/LightSwitch/LightSwitchService/LightSwitchService.vcxproj" />
<BuildDependency Project="src/modules/GrabAndMove/GrabAndMoveModuleInterface/GrabAndMoveModuleInterface.vcxproj" />
<BuildDependency Project="src/modules/GrabAndMove/GrabAndMove/GrabAndMove.vcxproj" />
<BuildDependency Project="src/modules/powerrename/dll/PowerRenameExt.vcxproj" />
<BuildDependency Project="src/modules/powerrename/lib/PowerRenameLib.vcxproj" />
<BuildDependency Project="src/modules/previewpane/Common/PreviewHandlerCommon.csproj" />