Files
PowerToys/.pipelines/v2/templates/pipeline-ui-tests-automation.yml
leileizhang 6d29c3a2c9 [pipeline] feat: Implement flexible UI test pipeline with configurable build and execution modes (#40490)
<!-- Enter a brief description/summary of your PR here. What does it
fix/what does it change/how was it tested (even manually, if necessary)?
-->
## Summary of the Pull Request
**Root Cause:**
The current pipeline builds the entire solution and runs all UI tests
every time, which takes more than 2 hours to complete.

**Fix**
Make the PowerToys UI test pipeline provides flexible options for
building and testing:

### Pipeline Options

- **useLatestOfficialBuild**: When checked, downloads the latest
official PowerToys build and installs it for testing. This skips the
full solution build and only builds UI test projects.

- **useCurrentBranchBuild**: When checked along with
`useLatestOfficialBuild`, downloads the official build from the current
branch instead of main.

- **uiTestModules**: Specify which UI test modules to build and run.
Examples:
  - `UITests-FancyZones` - Only FancyZones UI tests
  - `MouseUtils.UITests` - Only MouseUtils UI tests
- `['UITests-FancyZones', 'MouseUtils.UITests']` - Multiple specific
modules
  - Leave empty to build and run all UI test modules

### Build Modes

1. **Official Build + Selective Testing** (`useLatestOfficialBuild =
true`)
   - Downloads and installs official PowerToys build
   - Builds only specified UI test projects
   - Runs specified UI tests against installed PowerToys
   - Controlled by `uiTestModules` parameter

2. **Full Build + Testing** (`useLatestOfficialBuild = false`)
   - Builds entire PowerToys solution
   - Builds UI test projects (all or specific based on `uiTestModules`)
   - Runs UI tests (all or specific based on `uiTestModules`)
   - Uses freshly built PowerToys for testing
  
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist

- [ ] **Closes:** #xxx
- [ ] **Communication:** I've discussed this with core contributors
already. If the work hasn't been agreed, this work might be rejected
- [x] **Tests:** Added/updated and all pass
- [ ] **Localization:** All end-user-facing strings can be localized
- [x] **Dev docs:** Added/updated
- [ ] **New binaries:** Added on the required places
- [ ] [JSON for
signing](https://github.com/microsoft/PowerToys/blob/main/.pipelines/ESRPSigning_core.json)
for new binaries
- [ ] [WXS for
installer](https://github.com/microsoft/PowerToys/blob/main/installer/PowerToysSetup/Product.wxs)
for new binaries and localization folder
- [ ] [YML for CI
pipeline](https://github.com/microsoft/PowerToys/blob/main/.pipelines/ci/templates/build-powertoys-steps.yml)
for new test projects
- [ ] [YML for signed
pipeline](https://github.com/microsoft/PowerToys/blob/main/.pipelines/release.yml)
- [ ] **Documentation updated:** If checked, please file a pull request
on [our docs
repo](https://github.com/MicrosoftDocs/windows-uwp/tree/docs/hub/powertoys)
and link it here: #xxx

<!-- Provide a more detailed description of the PR, other things fixed,
or any additional comments/features here -->
## Detailed Description of the Pull Request / Additional comments

<!-- Describe how you validated the behavior. Add automated tests
wherever possible, but list manual validation steps taken as well -->
## Validation Steps Performed
2025-07-10 09:26:26 +08:00

175 lines
6.8 KiB
YAML

variables:
- name: runCodesignValidationInjectionBG
value: false
- name: EnablePipelineCache
value: true
- ${{ if eq(parameters.enableMsBuildCaching, true) }}:
- name: EnablePipelineCache
value: true
parameters:
- name: buildPlatforms
type: object
default:
- x64
- arm64
- name: enableMsBuildCaching
type: boolean
default: false
- name: useVSPreview
type: boolean
default: false
- name: useLatestWebView2
type: boolean
default: false
- name: useLatestOfficialBuild
type: boolean
default: true
- name: testBothInstallModes
type: boolean
default: true
- name: useCurrentBranchBuild
type: boolean
default: false
- name: uiTestModules
type: object
default: []
stages:
- ${{ each platform in parameters.buildPlatforms }}:
- ${{ if eq(parameters.useLatestOfficialBuild, false) }}:
- stage: Build_${{ platform }}
displayName: Build ${{ platform }}
dependsOn: []
jobs:
- template: job-build-project.yml
parameters:
pool:
${{ if eq(variables['System.CollectionId'], 'cb55739e-4afe-46a3-970f-1b49d8ee7564') }}:
name: SHINE-INT-L
${{ else }}:
name: SHINE-OSS-L
${{ if eq(parameters.useVSPreview, true) }}:
demands: ImageOverride -equals SHINE-VS17-Preview
buildPlatforms:
- ${{ platform }}
buildConfigurations: [Release]
enablePackageCaching: true
enableMsBuildCaching: ${{ parameters.enableMsBuildCaching }}
runTests: false
buildTests: true
useVSPreview: ${{ parameters.useVSPreview }}
- ${{ if eq(parameters.useLatestOfficialBuild, true) }}:
- stage: BuildUITests_${{ platform }}
displayName: Build UI Tests Only
dependsOn: []
jobs:
- template: job-build-ui-tests.yml
parameters:
pool:
${{ if eq(variables['System.CollectionId'], 'cb55739e-4afe-46a3-970f-1b49d8ee7564') }}:
name: SHINE-INT-L
${{ else }}:
name: SHINE-OSS-L
${{ if eq(parameters.useVSPreview, true) }}:
demands: ImageOverride -equals SHINE-VS17-Preview
buildPlatforms:
- ${{ platform }}
uiTestModules: ${{ parameters.uiTestModules }}
- ${{ if eq(platform, 'x64') }}:
- stage: Test_x64Win10
displayName: Test x64Win10
${{ if eq(parameters.useLatestOfficialBuild, true) }}:
dependsOn:
- BuildUITests_${{ platform }}
${{ else }}:
dependsOn:
- Build_${{ platform }}
jobs:
- template: job-test-project.yml
parameters:
platform: x64Win10
configuration: Release
useLatestWebView2: ${{ parameters.useLatestWebView2 }}
useLatestOfficialBuild: ${{ parameters.useLatestOfficialBuild }}
useCurrentBranchBuild: ${{ parameters.useCurrentBranchBuild }}
uiTestModules: ${{ parameters.uiTestModules }}
# Additional per-user installation test (when both modes are enabled)
- ${{ if and(eq(parameters.useLatestOfficialBuild, true), eq(parameters.testBothInstallModes, true)) }}:
- template: job-test-project.yml
parameters:
platform: x64Win10
configuration: Release
useLatestWebView2: ${{ parameters.useLatestWebView2 }}
useLatestOfficialBuild: ${{ parameters.useLatestOfficialBuild }}
useCurrentBranchBuild: ${{ parameters.useCurrentBranchBuild }}
uiTestModules: ${{ parameters.uiTestModules }}
installMode: 'peruser'
jobSuffix: '_PerUser'
- ${{ if eq(platform, 'x64') }}:
- stage: Test_x64Win11
displayName: Test x64Win11
${{ if eq(parameters.useLatestOfficialBuild, true) }}:
dependsOn:
- BuildUITests_${{ platform }}
${{ else }}:
dependsOn:
- Build_${{ platform }}
jobs:
- template: job-test-project.yml
parameters:
platform: x64Win11
configuration: Release
useLatestWebView2: ${{ parameters.useLatestWebView2 }}
useLatestOfficialBuild: ${{ parameters.useLatestOfficialBuild }}
useCurrentBranchBuild: ${{ parameters.useCurrentBranchBuild }}
uiTestModules: ${{ parameters.uiTestModules }}
# Additional per-user installation test (when both modes are enabled)
- ${{ if and(eq(parameters.useLatestOfficialBuild, true), eq(parameters.testBothInstallModes, true)) }}:
- template: job-test-project.yml
parameters:
platform: x64Win11
configuration: Release
useLatestWebView2: ${{ parameters.useLatestWebView2 }}
useLatestOfficialBuild: ${{ parameters.useLatestOfficialBuild }}
useCurrentBranchBuild: ${{ parameters.useCurrentBranchBuild }}
uiTestModules: ${{ parameters.uiTestModules }}
installMode: 'peruser'
jobSuffix: '_PerUser'
- ${{ if ne(platform, 'x64') }}:
- stage: Test_${{ platform }}
displayName: Test ${{ platform }}
${{ if eq(parameters.useLatestOfficialBuild, true) }}:
dependsOn:
- BuildUITests_${{ platform }}
${{ else }}:
dependsOn:
- Build_${{ platform }}
jobs:
- template: job-test-project.yml
parameters:
platform: ${{ platform }}
configuration: Release
useLatestWebView2: ${{ parameters.useLatestWebView2 }}
useLatestOfficialBuild: ${{ parameters.useLatestOfficialBuild }}
useCurrentBranchBuild: ${{ parameters.useCurrentBranchBuild }}
uiTestModules: ${{ parameters.uiTestModules }}
# Additional per-user installation test (when both modes are enabled)
- ${{ if and(eq(parameters.useLatestOfficialBuild, true), eq(parameters.testBothInstallModes, true)) }}:
- template: job-test-project.yml
parameters:
platform: ${{ platform }}
configuration: Release
useLatestWebView2: ${{ parameters.useLatestWebView2 }}
useLatestOfficialBuild: ${{ parameters.useLatestOfficialBuild }}
useCurrentBranchBuild: ${{ parameters.useCurrentBranchBuild }}
uiTestModules: ${{ parameters.uiTestModules }}
installMode: 'peruser'
jobSuffix: '_PerUser'