[pipeline] feat: Implement flexible UI test pipeline with configurable build and execution modes (#40490)

<!-- Enter a brief description/summary of your PR here. What does it
fix/what does it change/how was it tested (even manually, if necessary)?
-->
## Summary of the Pull Request
**Root Cause:**
The current pipeline builds the entire solution and runs all UI tests
every time, which takes more than 2 hours to complete.

**Fix**
Make the PowerToys UI test pipeline provides flexible options for
building and testing:

### Pipeline Options

- **useLatestOfficialBuild**: When checked, downloads the latest
official PowerToys build and installs it for testing. This skips the
full solution build and only builds UI test projects.

- **useCurrentBranchBuild**: When checked along with
`useLatestOfficialBuild`, downloads the official build from the current
branch instead of main.

- **uiTestModules**: Specify which UI test modules to build and run.
Examples:
  - `UITests-FancyZones` - Only FancyZones UI tests
  - `MouseUtils.UITests` - Only MouseUtils UI tests
- `['UITests-FancyZones', 'MouseUtils.UITests']` - Multiple specific
modules
  - Leave empty to build and run all UI test modules

### Build Modes

1. **Official Build + Selective Testing** (`useLatestOfficialBuild =
true`)
   - Downloads and installs official PowerToys build
   - Builds only specified UI test projects
   - Runs specified UI tests against installed PowerToys
   - Controlled by `uiTestModules` parameter

2. **Full Build + Testing** (`useLatestOfficialBuild = false`)
   - Builds entire PowerToys solution
   - Builds UI test projects (all or specific based on `uiTestModules`)
   - Runs UI tests (all or specific based on `uiTestModules`)
   - Uses freshly built PowerToys for testing
  
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist

- [ ] **Closes:** #xxx
- [ ] **Communication:** I've discussed this with core contributors
already. If the work hasn't been agreed, this work might be rejected
- [x] **Tests:** Added/updated and all pass
- [ ] **Localization:** All end-user-facing strings can be localized
- [x] **Dev docs:** Added/updated
- [ ] **New binaries:** Added on the required places
- [ ] [JSON for
signing](https://github.com/microsoft/PowerToys/blob/main/.pipelines/ESRPSigning_core.json)
for new binaries
- [ ] [WXS for
installer](https://github.com/microsoft/PowerToys/blob/main/installer/PowerToysSetup/Product.wxs)
for new binaries and localization folder
- [ ] [YML for CI
pipeline](https://github.com/microsoft/PowerToys/blob/main/.pipelines/ci/templates/build-powertoys-steps.yml)
for new test projects
- [ ] [YML for signed
pipeline](https://github.com/microsoft/PowerToys/blob/main/.pipelines/release.yml)
- [ ] **Documentation updated:** If checked, please file a pull request
on [our docs
repo](https://github.com/MicrosoftDocs/windows-uwp/tree/docs/hub/powertoys)
and link it here: #xxx

<!-- Provide a more detailed description of the PR, other things fixed,
or any additional comments/features here -->
## Detailed Description of the Pull Request / Additional comments

<!-- Describe how you validated the behavior. Add automated tests
wherever possible, but list manual validation steps taken as well -->
## Validation Steps Performed
This commit is contained in:
leileizhang
2025-07-10 09:26:26 +08:00
committed by GitHub
parent 608eb1e034
commit 6d29c3a2c9
10 changed files with 562 additions and 73 deletions

View File

@@ -16,6 +16,56 @@
- Run tests in the Test Explorer (`Test > Test Explorer` or `Ctrl+E, T`).
## Running tests in pipeline
The PowerToys UI test pipeline provides flexible options for building and testing:
### Pipeline Options
- **useLatestOfficialBuild**: When checked, downloads the latest official PowerToys build and installs it for testing. This skips the full solution build and only builds UI test projects.
- **useCurrentBranchBuild**: When checked along with `useLatestOfficialBuild`, downloads the official build from the current branch instead of main.
**Default value**: `false` (downloads from main branch)
**When to use this**:
- **Default scenario**: The pipeline tests against the latest signed PowerToys build from the `main` branch, regardless of which branch your test code changes are from
- **Custom branch testing**: Only specify `true` when:
- Your branch has produced its own signed PowerToys build via the official build pipeline
- You want to test against that specific branch's PowerToys build instead of main
- You are testing PowerToys functionality changes that are only available in your branch's build
**Important notes**:
- The test pipeline itself runs from your specified branch, but by default tests against the main branch's PowerToys build
- Not all branches have signed builds available - only use this if you're certain your branch has a signed build
- If enabled but no build exists for your branch, the pipeline may fail or fall back to main
- **uiTestModules**: Specify which UI test modules to build and run. This parameter controls both the `.csproj` projects to build and the `.dll` test assemblies to execute. Examples:
- `['UITests-FancyZones']` - Only FancyZones UI tests
- `['MouseUtils.UITests']` - Only MouseUtils UI tests
- `['UITests-FancyZones', 'MouseUtils.UITests']` - Multiple specific modules
- Leave empty to build and run all UI test modules
**Important**: The `uiTestModules` parameter values must match both the test project names (for `.csproj` selection during build) and the test assembly names (for `.dll` execution during testing).
### Build Modes
1. **Official Build + Selective Testing** (`useLatestOfficialBuild = true`)
- Downloads and installs official PowerToys build
- Builds only specified UI test projects
- Runs specified UI tests against installed PowerToys
- Controlled by `uiTestModules` parameter
2. **Full Build + Testing** (`useLatestOfficialBuild = false`)
- Builds entire PowerToys solution
- Builds UI test projects (all or specific based on `uiTestModules`)
- Runs UI tests (all or specific based on `uiTestModules`)
- Uses freshly built PowerToys for testing
> **Note**: Both modes support the `uiTestModules` parameter to control which specific UI test modules to build and run.
### Pipeline Access
- Pipeline: https://microsoft.visualstudio.com/Dart/_build?definitionId=161438&_a=summary
## How to add the first UI tests for your modules