mirror of
https://github.com/microsoft/PowerToys.git
synced 2025-12-16 03:37:59 +01:00
[pipeline] feat: Implement flexible UI test pipeline with configurable build and execution modes (#40490)
<!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? --> ## Summary of the Pull Request **Root Cause:** The current pipeline builds the entire solution and runs all UI tests every time, which takes more than 2 hours to complete. **Fix** Make the PowerToys UI test pipeline provides flexible options for building and testing: ### Pipeline Options - **useLatestOfficialBuild**: When checked, downloads the latest official PowerToys build and installs it for testing. This skips the full solution build and only builds UI test projects. - **useCurrentBranchBuild**: When checked along with `useLatestOfficialBuild`, downloads the official build from the current branch instead of main. - **uiTestModules**: Specify which UI test modules to build and run. Examples: - `UITests-FancyZones` - Only FancyZones UI tests - `MouseUtils.UITests` - Only MouseUtils UI tests - `['UITests-FancyZones', 'MouseUtils.UITests']` - Multiple specific modules - Leave empty to build and run all UI test modules ### Build Modes 1. **Official Build + Selective Testing** (`useLatestOfficialBuild = true`) - Downloads and installs official PowerToys build - Builds only specified UI test projects - Runs specified UI tests against installed PowerToys - Controlled by `uiTestModules` parameter 2. **Full Build + Testing** (`useLatestOfficialBuild = false`) - Builds entire PowerToys solution - Builds UI test projects (all or specific based on `uiTestModules`) - Runs UI tests (all or specific based on `uiTestModules`) - Uses freshly built PowerToys for testing <!-- Please review the items on the PR checklist before submitting--> ## PR Checklist - [ ] **Closes:** #xxx - [ ] **Communication:** I've discussed this with core contributors already. If the work hasn't been agreed, this work might be rejected - [x] **Tests:** Added/updated and all pass - [ ] **Localization:** All end-user-facing strings can be localized - [x] **Dev docs:** Added/updated - [ ] **New binaries:** Added on the required places - [ ] [JSON for signing](https://github.com/microsoft/PowerToys/blob/main/.pipelines/ESRPSigning_core.json) for new binaries - [ ] [WXS for installer](https://github.com/microsoft/PowerToys/blob/main/installer/PowerToysSetup/Product.wxs) for new binaries and localization folder - [ ] [YML for CI pipeline](https://github.com/microsoft/PowerToys/blob/main/.pipelines/ci/templates/build-powertoys-steps.yml) for new test projects - [ ] [YML for signed pipeline](https://github.com/microsoft/PowerToys/blob/main/.pipelines/release.yml) - [ ] **Documentation updated:** If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/windows-uwp/tree/docs/hub/powertoys) and link it here: #xxx <!-- Provide a more detailed description of the PR, other things fixed, or any additional comments/features here --> ## Detailed Description of the Pull Request / Additional comments <!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well --> ## Validation Steps Performed
This commit is contained in:
@@ -16,6 +16,56 @@
|
||||
|
||||
- Run tests in the Test Explorer (`Test > Test Explorer` or `Ctrl+E, T`).
|
||||
|
||||
## Running tests in pipeline
|
||||
|
||||
The PowerToys UI test pipeline provides flexible options for building and testing:
|
||||
|
||||
### Pipeline Options
|
||||
|
||||
- **useLatestOfficialBuild**: When checked, downloads the latest official PowerToys build and installs it for testing. This skips the full solution build and only builds UI test projects.
|
||||
|
||||
- **useCurrentBranchBuild**: When checked along with `useLatestOfficialBuild`, downloads the official build from the current branch instead of main.
|
||||
|
||||
**Default value**: `false` (downloads from main branch)
|
||||
|
||||
**When to use this**:
|
||||
- **Default scenario**: The pipeline tests against the latest signed PowerToys build from the `main` branch, regardless of which branch your test code changes are from
|
||||
- **Custom branch testing**: Only specify `true` when:
|
||||
- Your branch has produced its own signed PowerToys build via the official build pipeline
|
||||
- You want to test against that specific branch's PowerToys build instead of main
|
||||
- You are testing PowerToys functionality changes that are only available in your branch's build
|
||||
|
||||
**Important notes**:
|
||||
- The test pipeline itself runs from your specified branch, but by default tests against the main branch's PowerToys build
|
||||
- Not all branches have signed builds available - only use this if you're certain your branch has a signed build
|
||||
- If enabled but no build exists for your branch, the pipeline may fail or fall back to main
|
||||
|
||||
- **uiTestModules**: Specify which UI test modules to build and run. This parameter controls both the `.csproj` projects to build and the `.dll` test assemblies to execute. Examples:
|
||||
- `['UITests-FancyZones']` - Only FancyZones UI tests
|
||||
- `['MouseUtils.UITests']` - Only MouseUtils UI tests
|
||||
- `['UITests-FancyZones', 'MouseUtils.UITests']` - Multiple specific modules
|
||||
- Leave empty to build and run all UI test modules
|
||||
|
||||
**Important**: The `uiTestModules` parameter values must match both the test project names (for `.csproj` selection during build) and the test assembly names (for `.dll` execution during testing).
|
||||
|
||||
### Build Modes
|
||||
|
||||
1. **Official Build + Selective Testing** (`useLatestOfficialBuild = true`)
|
||||
- Downloads and installs official PowerToys build
|
||||
- Builds only specified UI test projects
|
||||
- Runs specified UI tests against installed PowerToys
|
||||
- Controlled by `uiTestModules` parameter
|
||||
|
||||
2. **Full Build + Testing** (`useLatestOfficialBuild = false`)
|
||||
- Builds entire PowerToys solution
|
||||
- Builds UI test projects (all or specific based on `uiTestModules`)
|
||||
- Runs UI tests (all or specific based on `uiTestModules`)
|
||||
- Uses freshly built PowerToys for testing
|
||||
|
||||
> **Note**: Both modes support the `uiTestModules` parameter to control which specific UI test modules to build and run.
|
||||
|
||||
### Pipeline Access
|
||||
- Pipeline: https://microsoft.visualstudio.com/Dart/_build?definitionId=161438&_a=summary
|
||||
|
||||
## How to add the first UI tests for your modules
|
||||
|
||||
|
||||
Reference in New Issue
Block a user