Files
PowerToys/.pipelines/v2/templates/job-test-project.yml
leileizhang 6d29c3a2c9 [pipeline] feat: Implement flexible UI test pipeline with configurable build and execution modes (#40490)
<!-- Enter a brief description/summary of your PR here. What does it
fix/what does it change/how was it tested (even manually, if necessary)?
-->
## Summary of the Pull Request
**Root Cause:**
The current pipeline builds the entire solution and runs all UI tests
every time, which takes more than 2 hours to complete.

**Fix**
Make the PowerToys UI test pipeline provides flexible options for
building and testing:

### Pipeline Options

- **useLatestOfficialBuild**: When checked, downloads the latest
official PowerToys build and installs it for testing. This skips the
full solution build and only builds UI test projects.

- **useCurrentBranchBuild**: When checked along with
`useLatestOfficialBuild`, downloads the official build from the current
branch instead of main.

- **uiTestModules**: Specify which UI test modules to build and run.
Examples:
  - `UITests-FancyZones` - Only FancyZones UI tests
  - `MouseUtils.UITests` - Only MouseUtils UI tests
- `['UITests-FancyZones', 'MouseUtils.UITests']` - Multiple specific
modules
  - Leave empty to build and run all UI test modules

### Build Modes

1. **Official Build + Selective Testing** (`useLatestOfficialBuild =
true`)
   - Downloads and installs official PowerToys build
   - Builds only specified UI test projects
   - Runs specified UI tests against installed PowerToys
   - Controlled by `uiTestModules` parameter

2. **Full Build + Testing** (`useLatestOfficialBuild = false`)
   - Builds entire PowerToys solution
   - Builds UI test projects (all or specific based on `uiTestModules`)
   - Runs UI tests (all or specific based on `uiTestModules`)
   - Uses freshly built PowerToys for testing
  
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist

- [ ] **Closes:** #xxx
- [ ] **Communication:** I've discussed this with core contributors
already. If the work hasn't been agreed, this work might be rejected
- [x] **Tests:** Added/updated and all pass
- [ ] **Localization:** All end-user-facing strings can be localized
- [x] **Dev docs:** Added/updated
- [ ] **New binaries:** Added on the required places
- [ ] [JSON for
signing](https://github.com/microsoft/PowerToys/blob/main/.pipelines/ESRPSigning_core.json)
for new binaries
- [ ] [WXS for
installer](https://github.com/microsoft/PowerToys/blob/main/installer/PowerToysSetup/Product.wxs)
for new binaries and localization folder
- [ ] [YML for CI
pipeline](https://github.com/microsoft/PowerToys/blob/main/.pipelines/ci/templates/build-powertoys-steps.yml)
for new test projects
- [ ] [YML for signed
pipeline](https://github.com/microsoft/PowerToys/blob/main/.pipelines/release.yml)
- [ ] **Documentation updated:** If checked, please file a pull request
on [our docs
repo](https://github.com/MicrosoftDocs/windows-uwp/tree/docs/hub/powertoys)
and link it here: #xxx

<!-- Provide a more detailed description of the PR, other things fixed,
or any additional comments/features here -->
## Detailed Description of the Pull Request / Additional comments

<!-- Describe how you validated the behavior. Add automated tests
wherever possible, but list manual validation steps taken as well -->
## Validation Steps Performed
2025-07-10 09:26:26 +08:00

193 lines
6.7 KiB
YAML

parameters:
- name: configuration
type: string
default: "Release"
- name: platform
type: string
default: ""
- name: inputArtifactStem
type: string
default: ""
- name: useLatestWebView2
type: boolean
default: false
- name: useLatestOfficialBuild
type: boolean
default: true
- name: useCurrentBranchBuild
type: boolean
default: false
- name: uiTestModules
type: object
default: []
- name: installMode
type: string
default: 'machine'
values:
- 'machine'
- 'peruser'
- name: jobSuffix
type: string
default: ''
jobs:
- job: Test${{ parameters.platform }}${{ parameters.configuration }}${{ parameters.jobSuffix }}
displayName: Test ${{ parameters.platform }} ${{ parameters.configuration }}${{ parameters.jobSuffix }}
timeoutInMinutes: 300
variables:
${{ if or(eq(parameters.platform, 'x64Win10'), eq(parameters.platform, 'x64Win11')) }}:
BuildPlatform: x64
${{ else }}:
BuildPlatform: ${{ parameters.platform }}
TestPlatform: ${{ parameters.platform }}
BuildConfiguration: ${{ parameters.configuration }}
SrcPath: $(Build.Repository.LocalPath)
TestArtifactsName: build-${{ variables.BuildPlatform }}-${{ parameters.configuration }}${{ parameters.inputArtifactStem }}
pool:
${{ if eq(variables['System.CollectionId'], 'cb55739e-4afe-46a3-970f-1b49d8ee7564') }}:
${{ if ne(parameters.platform, 'ARM64') }}:
name: SHINE-INT-Testing-x64
${{ if eq(parameters.platform, 'x64Win11') }}:
demands: ImageOverride -equals SHINE-W11-Testing
${{ else }}:
name: SHINE-INT-Testing-arm64
${{ else }}:
${{ if ne(parameters.platform, 'ARM64') }}:
name: SHINE-OSS-Testing-x64
${{ if eq(parameters.platform, 'x64Win11') }}:
demands: ImageOverride -equals SHINE-W11-Testing
${{ else }}:
name: SHINE-OSS-Testing-arm64
steps:
- checkout: self
submodules: false
clean: true
fetchDepth: 1
fetchTags: false
- ${{ if eq(parameters.useLatestWebView2, true) }}:
- powershell: |
$edge_url = 'https://go.microsoft.com/fwlink/?linkid=2084649&Channel=Canary&language=en'
$timeout = New-TimeSpan -Minutes 6
$timeoutSeconds = [int]$timeout.TotalSeconds
$command = {
Invoke-WebRequest -Uri $using:edge_url -OutFile $(Pipeline.Workspace)\MicrosoftEdgeSetup.exe
Write-Host "##[command]Installing Canary channel of Microsoft Edge"
Start-Process $(Pipeline.Workspace)\MicrosoftEdgeSetup.exe -ArgumentList '/silent /install' -Wait
}
$job = Start-Job -ScriptBlock $command
Wait-Job $job -Timeout $timeoutSeconds
if ($job.State -eq "Running") {
Stop-Job $job
Write-Host "##[warning]The job was stopped because it exceeded the time limit."
}
displayName: "Install the latest MSEdge Canary"
- script:
reg add "HKLM\Software\Policies\Microsoft\Edge\WebView2\ReleaseChannels" /v PowerToys.exe /t REG_SZ /d "3"
displayName: "Enable WebView2 Canary Channel"
- ${{ if ne(parameters.platform, 'arm64') }}:
- download: current
displayName: Download artifacts
artifact: $(TestArtifactsName)
patterns: |-
**
!**\*.pdb
!**\*.lib
- ${{ else }}:
- template: steps-download-artifacts-with-azure-cli.yml
parameters:
artifactName: $(TestArtifactsName)
- template: steps-ensure-dotnet-version.yml
parameters:
sdk: true
version: '9.0'
- task: VisualStudioTestPlatformInstaller@1
displayName: Ensure VSTest Platform
- pwsh: |-
& '$(build.sourcesdirectory)\.pipelines\InstallWinAppDriver.ps1'
displayName: Download and install WinAppDriver
- ${{ if eq(parameters.useLatestOfficialBuild, true) }}:
- task: DownloadPipelineArtifact@2
inputs:
buildType: 'specific'
project: 'Dart'
definition: '76541'
buildVersionToDownload: 'latestFromBranch'
${{ if eq(parameters.useCurrentBranchBuild, true) }}:
branchName: '$(Build.SourceBranch)'
${{ else }}:
branchName: 'refs/heads/main'
artifactName: 'build-$(BuildPlatform)-Release'
targetPath: '$(Build.ArtifactStagingDirectory)'
${{ if eq(parameters.installMode, 'peruser') }}:
patterns: |
**/PowerToysUserSetup*.exe
${{ else }}:
patterns: |
**/PowerToysSetup*.exe
- ${{ if eq(parameters.useLatestOfficialBuild, true) }}:
- ${{ if eq(parameters.installMode, 'peruser') }}:
- pwsh: |-
& "$(build.sourcesdirectory)\.pipelines\installPowerToys.ps1" -InstallMode "PerUser"
displayName: Install PowerToys (Per-User)
- ${{ if eq(parameters.installMode, 'machine') }}:
- pwsh: |-
& "$(build.sourcesdirectory)\.pipelines\installPowerToys.ps1" -InstallMode "Machine"
displayName: Install PowerToys (Machine-Level)
- ${{ if ne(parameters.platform, 'arm64') }}:
- task: ScreenResolutionUtility@1
inputs:
displaySettings: 'optimal'
- ${{ if eq(length(parameters.uiTestModules), 0) }}:
- task: VSTest@3
displayName: Run UI Tests
inputs:
platform: '$(BuildPlatform)'
configuration: '$(BuildConfiguration)'
testSelector: 'testAssemblies'
searchFolder: '$(Pipeline.Workspace)\$(TestArtifactsName)'
vsTestVersion: 'toolsInstaller'
uiTests: true
rerunFailedTests: true
# Since UITests-FancyZonesEditor.dll is generated in both UITests-FancyZonesEditor and UITests-FancyZones, removed one to avoid duplicate test runs
testAssemblyVer2: |
**\*UITest*.dll
!**\obj\**
!**\ref\**
!**\UITests-FancyZones\**\UITests-FancyZonesEditor.dll
env:
platform: '$(TestPlatform)'
useInstallerForTest: ${{ parameters.useLatestOfficialBuild }}
- ${{ if ne(length(parameters.uiTestModules), 0) }}:
- ${{ each module in parameters.uiTestModules }}:
- task: VSTest@3
displayName: Run UI Test - ${{ module }}
inputs:
platform: '$(BuildPlatform)'
configuration: '$(BuildConfiguration)'
testSelector: 'testAssemblies'
searchFolder: '$(Pipeline.Workspace)\$(TestArtifactsName)'
vsTestVersion: 'toolsInstaller'
uiTests: true
rerunFailedTests: true
testAssemblyVer2: |
**\*${{ module }}*.dll
!**\obj\**
!**\ref\**
!**\UITests-FancyZones\**\UITests-FancyZonesEditor.dll
env:
platform: '$(TestPlatform)'
useInstallerForTest: ${{ parameters.useLatestOfficialBuild }}