[UI Tests] Enhance UI Test Automation and Pipeline Support for CmdPal Module (#40871)

<!-- Enter a brief description/summary of your PR here. What does it
fix/what does it change/how was it tested (even manually, if necessary)?
-->
## Summary of the Pull Request
This pull request introduces enhancements to UI test automation,
improvements to pipeline configuration, and project structure updates.
The goal is to improve flexibility, maintainability, and efficiency in
PowerToys’ CI/CD processes.

### UI Test Enhancements:
Delayed Text Input Support
- UI tests now support character-by-character text input with
configurable delays.
- This serves as a workaround for a known CmdPal bug where input is
swallowed too quickly. The delay mitigates the issue until it is fixed
in CmdPal.
 
Centralized Environment Management
- Introduced a new class to centralize environment variable access for
UI test configuration.

CmdPal Launch Handling in Pipelines

- Adjusted test logic to handle CmdPal module startup specifically in CI
pipelines



### Pipeline Configuration Updates:
Build Artifact Customization

- Included test-related folders in pipeline build outputs for better
traceability.

Support for Build ID Targeting

- Added support for specifying PowerToys build IDs in test pipelines,
with conditional logic for specific or latest build selection.
<img width="264" height="44" alt="image"
src="https://github.com/user-attachments/assets/0d68a51e-e41a-4868-a1c3-f4233c56b0ee"
/>


### Project Structure Updates:
Added Peek.UITests back to the solution which removed by
https://github.com/microsoft/PowerToys/pull/40754

<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist

- [ ] Closes: #xxx
- [ ] **Communication:** I've discussed this with core contributors
already. If the work hasn't been agreed, this work might be rejected
- [ ] **Tests:** Added/updated and all pass
- [ ] **Localization:** All end-user-facing strings can be localized
- [ ] **Dev docs:** Added/updated
- [ ] **New binaries:** Added on the required places
- [ ] [JSON for
signing](https://github.com/microsoft/PowerToys/blob/main/.pipelines/ESRPSigning_core.json)
for new binaries
- [ ] [WXS for
installer](https://github.com/microsoft/PowerToys/blob/main/installer/PowerToysSetup/Product.wxs)
for new binaries and localization folder
- [ ] [YML for CI
pipeline](https://github.com/microsoft/PowerToys/blob/main/.pipelines/ci/templates/build-powertoys-steps.yml)
for new test projects
- [ ] [YML for signed
pipeline](https://github.com/microsoft/PowerToys/blob/main/.pipelines/release.yml)
- [ ] **Documentation updated:** If checked, please file a pull request
on [our docs
repo](https://github.com/MicrosoftDocs/windows-uwp/tree/docs/hub/powertoys)
and link it here: #xxx

<!-- Provide a more detailed description of the PR, other things fixed,
or any additional comments/features here -->
## Detailed Description of the Pull Request / Additional comments

<!-- Describe how you validated the behavior. Add automated tests
wherever possible, but list manual validation steps taken as well -->
## Validation Steps Performed
This commit is contained in:
leileizhang
2025-07-31 13:55:23 +08:00
committed by GitHub
parent d26ef36e31
commit 9a998b2056
14 changed files with 466 additions and 139 deletions

View File

@@ -22,23 +22,23 @@ The PowerToys UI test pipeline provides flexible options for building and testin
### Pipeline Options
- **useLatestOfficialBuild**: When checked, downloads the latest official PowerToys build and installs it for testing. This skips the full solution build and only builds UI test projects.
- **buildSource**: Select the build type for testing:
- `latestMainOfficialBuild`: Downloads and uses the latest official PowerToys build from main branch
- `buildNow`: Builds PowerToys from current source code and uses it for testing
- `specificBuildId`: Downloads a specific PowerToys build using the build ID specified in `specificBuildId` parameter
- **useCurrentBranchBuild**: When checked along with `useLatestOfficialBuild`, downloads the official build from the current branch instead of main.
**Default value**: `latestMainOfficialBuild`
**Default value**: `false` (downloads from main branch)
- **specificBuildId**: When `buildSource` is set to `specificBuildId`, specify the exact PowerToys build ID to download and test against.
**Default value**: `"xxxx"` (placeholder, enter actual build ID when using specificBuildId option)
**When to use this**:
- **Default scenario**: The pipeline tests against the latest signed PowerToys build from the `main` branch, regardless of which branch your test code changes are from
- **Custom branch testing**: Only specify `true` when:
- Your branch has produced its own signed PowerToys build via the official build pipeline
- You want to test against that specific branch's PowerToys build instead of main
- You are testing PowerToys functionality changes that are only available in your branch's build
- Testing against a specific known build for reproducibility
- Regression testing against a particular build version
- Validating fixes in a specific build before release
**Important notes**:
- The test pipeline itself runs from your specified branch, but by default tests against the main branch's PowerToys build
- Not all branches have signed builds available - only use this if you're certain your branch has a signed build
- If enabled but no build exists for your branch, the pipeline may fail or fall back to main
**Usage**: Enter the build ID number (e.g., `12345`) to download that specific build. Only used when `buildSource` is set to `specificBuildId`.
- **uiTestModules**: Specify which UI test modules to build and run. This parameter controls both the `.csproj` projects to build and the `.dll` test assemblies to execute. Examples:
- `['UITests-FancyZones']` - Only FancyZones UI tests
@@ -50,19 +50,19 @@ The PowerToys UI test pipeline provides flexible options for building and testin
### Build Modes
1. **Official Build + Selective Testing** (`useLatestOfficialBuild = true`)
- Downloads and installs official PowerToys build
- Builds only specified UI test projects
- Runs specified UI tests against installed PowerToys
- Controlled by `uiTestModules` parameter
1. **Official Build Testing** (`buildSource = latestMainOfficialBuild` or `specificBuildId`)
- Downloads and installs official PowerToys build (latest from main or specific build ID)
- Builds only UI test projects (all or specific based on `uiTestModules`)
- Runs UI tests against installed PowerToys
- Tests both machine-level and per-user installation modes automatically
2. **Full Build + Testing** (`useLatestOfficialBuild = false`)
- Builds entire PowerToys solution
2. **Current Source Build Testing** (`buildSource = buildNow`)
- Builds entire PowerToys solution from current source code
- Builds UI test projects (all or specific based on `uiTestModules`)
- Runs UI tests (all or specific based on `uiTestModules`)
- Uses freshly built PowerToys for testing
- Runs UI tests against freshly built PowerToys
- Uses artifacts from current pipeline build
> **Note**: Both modes support the `uiTestModules` parameter to control which specific UI test modules to build and run.
> **Note**: All modes support the `uiTestModules` parameter to control which specific UI test modules to build and run. Both machine-level and per-user installation modes are tested automatically when using official builds.
### Pipeline Access
- Pipeline: https://microsoft.visualstudio.com/Dart/_build?definitionId=161438&_a=summary