Mirrors/PowerToys
1
0
Fork 0
mirror of https://github.com/microsoft/PowerToys.git synced 2026-04-07 03:36:44 +02:00
Code Issues Packages Projects Releases Wiki Activity

[UI Tests] Enhance UI Test Automation and Pipeline Support for CmdPal Module (#40871)

Browse Source 
<!-- Enter a brief description/summary of your PR here. What does it
fix/what does it change/how was it tested (even manually, if necessary)?
-->
## Summary of the Pull Request
This pull request introduces enhancements to UI test automation,
improvements to pipeline configuration, and project structure updates.
The goal is to improve flexibility, maintainability, and efficiency in
PowerToys’ CI/CD processes.

### UI Test Enhancements:
Delayed Text Input Support
- UI tests now support character-by-character text input with
configurable delays.
- This serves as a workaround for a known CmdPal bug where input is
swallowed too quickly. The delay mitigates the issue until it is fixed
in CmdPal.
 
Centralized Environment Management
- Introduced a new class to centralize environment variable access for
UI test configuration.

CmdPal Launch Handling in Pipelines

- Adjusted test logic to handle CmdPal module startup specifically in CI
pipelines



### Pipeline Configuration Updates:
Build Artifact Customization

- Included test-related folders in pipeline build outputs for better
traceability.

Support for Build ID Targeting

- Added support for specifying PowerToys build IDs in test pipelines,
with conditional logic for specific or latest build selection.
<img width="264" height="44" alt="image"
src="https://github.com/user-attachments/assets/0d68a51e-e41a-4868-a1c3-f4233c56b0ee"
/>


### Project Structure Updates:
Added Peek.UITests back to the solution which removed by
https://github.com/microsoft/PowerToys/pull/40754

<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist

- [ ] Closes: #xxx
- [ ] **Communication:** I've discussed this with core contributors
already. If the work hasn't been agreed, this work might be rejected
- [ ] **Tests:** Added/updated and all pass
- [ ] **Localization:** All end-user-facing strings can be localized
- [ ] **Dev docs:** Added/updated
- [ ] **New binaries:** Added on the required places
- [ ] [JSON for
signing](https://github.com/microsoft/PowerToys/blob/main/.pipelines/ESRPSigning_core.json)
for new binaries
- [ ] [WXS for
installer](https://github.com/microsoft/PowerToys/blob/main/installer/PowerToysSetup/Product.wxs)
for new binaries and localization folder
- [ ] [YML for CI
pipeline](https://github.com/microsoft/PowerToys/blob/main/.pipelines/ci/templates/build-powertoys-steps.yml)
for new test projects
- [ ] [YML for signed
pipeline](https://github.com/microsoft/PowerToys/blob/main/.pipelines/release.yml)
- [ ] **Documentation updated:** If checked, please file a pull request
on [our docs
repo](https://github.com/MicrosoftDocs/windows-uwp/tree/docs/hub/powertoys)
and link it here: #xxx

<!-- Provide a more detailed description of the PR, other things fixed,
or any additional comments/features here -->
## Detailed Description of the Pull Request / Additional comments

<!-- Describe how you validated the behavior. Add automated tests
wherever possible, but list manual validation steps taken as well -->
## Validation Steps Performed
This commit is contained in:
leileizhang
2025-07-31 13:55:23 +08:00
committed by GitHub
parent d26ef36e31
commit 9a998b2056
14 changed files with 466 additions and 139 deletions
Download Patch File Download Diff File Expand all files Collapse all files

37
src/common/UITestAutomation/Element/TextBox.cs
View File

@@ -2,6 +2,8 @@
// The Microsoft Corporation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.
using System.Threading.Tasks;
namespace Microsoft.PowerToys.UITest
{
/// <summary>
@@ -25,8 +27,9 @@ namespace Microsoft.PowerToys.UITest
/// </summary>
/// <param name="value">The text to set.</param>
/// <param name="clearText">A value indicating whether to clear the text before setting it. Default value is true</param>
/// <param name="charDelayMS">Delay in milliseconds between each character. Default is 0 (no delay).</param>
/// <returns>The current TextBox instance.</returns>
public TextBox SetText(string value, bool clearText = true)
public TextBox SetText(string value, bool clearText = true, int charDelayMS = 0)
{
if (clearText)
{
@@ -39,10 +42,36 @@ namespace Microsoft.PowerToys.UITest
Task.Delay(500).Wait();
}
PerformAction((actions, windowElement) =>
// TODO: CmdPal bug – when inputting text, characters are swallowed too quickly.
// This should be fixed within CmdPal itself.
// Temporary workaround: introduce a delay between character inputs to avoid the issue
if (charDelayMS > 0 || EnvironmentConfig.IsInPipeline)
{
windowElement.SendKeys(value);
});
// Send text character by character with delay (if specified or in pipeline)
PerformAction((actions, windowElement) =>
{
foreach (char c in value)
{
windowElement.SendKeys(c.ToString());
if (charDelayMS > 0)
{
Task.Delay(charDelayMS).Wait();
}
else if (EnvironmentConfig.IsInPipeline)
{
Task.Delay(50).Wait();
}
}
});
}
else
{
// No character delay - send all text at once (original behavior)
PerformAction((actions, windowElement) =>
{
windowElement.SendKeys(value);
});
}
return this;
}

45
src/common/UITestAutomation/EnvironmentConfig.cs Normal file
View File

@@ -0,0 +1,45 @@
// Copyright (c) Microsoft Corporation
// The Microsoft Corporation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.
using System;
namespace Microsoft.PowerToys.UITest
{
/// <summary>
/// Centralized configuration for all environment variables used in UI tests.
/// </summary>
public static class EnvironmentConfig
{
private static readonly Lazy<bool> _isInPipeline = new(() =>
!string.IsNullOrEmpty(Environment.GetEnvironmentVariable("platform")));
private static readonly Lazy<bool> _useInstallerForTest = new(() =>
{
string? envValue = Environment.GetEnvironmentVariable("useInstallerForTest") ??
Environment.GetEnvironmentVariable("USEINSTALLERFORTEST");
return !string.IsNullOrEmpty(envValue) && bool.TryParse(envValue, out bool result) && result;
});
private static readonly Lazy<string?> _platform = new(() =>
Environment.GetEnvironmentVariable("platform"));
/// <summary>
/// Gets a value indicating whether the tests are running in a CI/CD pipeline.
/// Determined by the presence of the "platform" environment variable.
/// </summary>
public static bool IsInPipeline => _isInPipeline.Value;
/// <summary>
/// Gets a value indicating whether to use installer paths for testing.
/// Checks both "useInstallerForTest" and "USEINSTALLERFORTEST" environment variables.
/// </summary>
public static bool UseInstallerForTest => _useInstallerForTest.Value;
/// <summary>
/// Gets the platform name from the environment variable.
/// Typically used in CI/CD pipelines to identify the build platform.
/// </summary>
public static string? Platform => _platform.Value;
}
}

4
src/common/UITestAutomation/ModuleConfigData.cs
View File

@@ -92,9 +92,7 @@ namespace Microsoft.PowerToys.UITest
private ModuleConfigData()
{
// Check if we should use installer paths from environment variable
string? useInstallerForTestEnv =
Environment.GetEnvironmentVariable("useInstallerForTest") ?? Environment.GetEnvironmentVariable("USEINSTALLERFORTEST");
UseInstallerForTest = !string.IsNullOrEmpty(useInstallerForTestEnv) && bool.TryParse(useInstallerForTestEnv, out bool result) && result;
UseInstallerForTest = EnvironmentConfig.UseInstallerForTest;
// Module information including executable name, window name, and optional subdirectory
ModuleInfo = new Dictionary<PowerToysModule, ModuleInfo>

117
src/common/UITestAutomation/SessionHelper.cs
View File

@@ -5,6 +5,7 @@
using System;
using System.Diagnostics;
using System.Diagnostics.CodeAnalysis;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Threading.Tasks;
@@ -37,6 +38,9 @@ namespace Microsoft.PowerToys.UITest
private PowerToysModule scope;
private string[]? commandLineArgs;
/// <summary>
/// Gets a value indicating whether to use installer paths for testing.
/// </summary>
private bool UseInstallerForTest { get; }
[UnconditionalSuppressMessage("SingleFile", "IL3000:Avoid accessing Assembly file path when publishing as a single file", Justification = "<Pending>")]
@@ -45,9 +49,7 @@ namespace Microsoft.PowerToys.UITest
this.scope = scope;
this.commandLineArgs = commandLineArgs;
this.sessionPath = ModuleConfigData.Instance.GetModulePath(scope);
string? useInstallerForTestEnv =
Environment.GetEnvironmentVariable("useInstallerForTest") ?? Environment.GetEnvironmentVariable("USEINSTALLERFORTEST");
UseInstallerForTest = !string.IsNullOrEmpty(useInstallerForTestEnv) && bool.TryParse(useInstallerForTestEnv, out bool result) && result;
UseInstallerForTest = EnvironmentConfig.UseInstallerForTest;
this.locationPath = UseInstallerForTest ? string.Empty : Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
CheckWinAppDriverAndRoot();
@@ -136,6 +138,10 @@ namespace Microsoft.PowerToys.UITest
{
TryLaunchPowerToysSettings(opts);
}
else if (scope == PowerToysModule.CommandPalette && UseInstallerForTest)
{
TryLaunchCommandPalette(opts);
}
else
{
opts.AddAdditionalCapability("app", appPath);
@@ -163,48 +169,77 @@ namespace Microsoft.PowerToys.UITest
private void TryLaunchPowerToysSettings(AppiumOptions opts)
{
CheckWinAppDriverAndRoot();
var runnerProcessInfo = new ProcessStartInfo
try
{
FileName = locationPath + runnerPath,
Verb = "runas",
Arguments = "--open-settings",
};
ExitExe(runnerProcessInfo.FileName);
runner = Process.Start(runnerProcessInfo);
Thread.Sleep(5000);
// Exit CmdPal UI before launching new process if use installer for test
ExitExeByName("Microsoft.CmdPal.UI");
if (root != null)
{
const int maxRetries = 5;
const int delayMs = 5000;
var windowName = "PowerToys Settings";
for (int attempt = 1; attempt <= maxRetries; attempt++)
var runnerProcessInfo = new ProcessStartInfo
{
var settingsWindow = ApiHelper.FindDesktopWindowHandler(
[windowName, AdministratorPrefix + windowName]);
FileName = locationPath + runnerPath,
Verb = "runas",
Arguments = "--open-settings",
};
if (settingsWindow.Count > 0)
{
var hexHwnd = settingsWindow[0].HWnd.ToString("x");
opts.AddAdditionalCapability("appTopLevelWindow", hexHwnd);
return;
}
ExitExe(runnerProcessInfo.FileName);
runner = Process.Start(runnerProcessInfo);
if (attempt < maxRetries)
{
Thread.Sleep(delayMs);
}
else
{
throw new TimeoutException("Failed to find PowerToys Settings window after multiple attempts.");
}
WaitForWindowAndSetCapability(opts, "PowerToys Settings", 5000, 5);
// Exit CmdPal UI before launching new process if use installer for test
ExitExeByName("Microsoft.CmdPal.UI");
}
catch (Exception ex)
{
throw new InvalidOperationException($"Failed to launch PowerToys Settings: {ex.Message}", ex);
}
}
private void TryLaunchCommandPalette(AppiumOptions opts)
{
try
{
// Exit any existing CmdPal UI process
ExitExeByName("Microsoft.CmdPal.UI");
var processStartInfo = new ProcessStartInfo
{
FileName = "cmd.exe",
Arguments = "/c start shell:appsFolder\\Microsoft.CommandPalette_8wekyb3d8bbwe!App",
UseShellExecute = false,
CreateNoWindow = true,
WindowStyle = ProcessWindowStyle.Hidden,
};
var process = Process.Start(processStartInfo);
process?.WaitForExit();
WaitForWindowAndSetCapability(opts, "Command Palette", 5000, 10);
}
catch (Exception ex)
{
throw new InvalidOperationException($"Failed to launch Command Palette: {ex.Message}", ex);
}
}
private void WaitForWindowAndSetCapability(AppiumOptions opts, string windowName, int delayMs, int maxRetries)
{
for (int attempt = 1; attempt <= maxRetries; attempt++)
{
var window = ApiHelper.FindDesktopWindowHandler(
[windowName, AdministratorPrefix + windowName]);
if (window.Count > 0)
{
var hexHwnd = window[0].HWnd.ToString("x");
opts.AddAdditionalCapability("appTopLevelWindow", hexHwnd);
return;
}
if (attempt < maxRetries)
{
Thread.Sleep(delayMs);
}
else
{
throw new TimeoutException($"Failed to find {windowName} window after multiple attempts.");
}
}
}

177
src/common/UITestAutomation/UITestBase.cs
View File

@@ -5,6 +5,7 @@
using System.Collections.ObjectModel;
using System.Diagnostics;
using System.Runtime.InteropServices;
using System.Text.RegularExpressions;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using OpenQA.Selenium;
@@ -20,6 +21,9 @@ namespace Microsoft.PowerToys.UITest
public required Session Session { get; set; }
/// <summary>
/// Gets a value indicating whether the tests are running in a CI/CD pipeline.
/// </summary>
public bool IsInPipeline { get; }
public string? ScreenshotDirectory { get; set; }
@@ -34,8 +38,8 @@ namespace Microsoft.PowerToys.UITest
public UITestBase(PowerToysModule scope = PowerToysModule.PowerToysSettings, WindowSize size = WindowSize.UnSpecified, string[]? commandLineArgs = null)
{
this.IsInPipeline = !string.IsNullOrEmpty(Environment.GetEnvironmentVariable("platform"));
Console.WriteLine($"Running tests on platform: {Environment.GetEnvironmentVariable("platform")}");
this.IsInPipeline = EnvironmentConfig.IsInPipeline;
Console.WriteLine($"Running tests on platform: {EnvironmentConfig.Platform}");
if (IsInPipeline)
{
NativeMethods.ChangeDisplayResolution(1920, 1080);
@@ -56,6 +60,7 @@ namespace Microsoft.PowerToys.UITest
[TestInitialize]
public void TestInit()
{
KeyboardHelper.SendKeys(Key.Win, Key.M);
CloseOtherApplications();
if (IsInPipeline)
{
@@ -247,6 +252,174 @@ namespace Microsoft.PowerToys.UITest
return this.Session.Has<Element>(name, timeoutMS, global);
}
/// <summary>
/// Finds an element using partial name matching (contains).
/// Useful for finding windows with variable titles like "filename.txt - Notepad" or "filename - Notepad".
/// </summary>
/// <typeparam name="T">The class of the element, should be Element or its derived class.</typeparam>
/// <param name="partialName">Part of the name to search for.</param>
/// <param name="timeoutMS">The timeout in milliseconds (default is 5000).</param>
/// <returns>The found element.</returns>
protected T FindByPartialName<T>(string partialName, int timeoutMS = 5000, bool global = false)
where T : Element, new()
{
return Session.Find<T>(By.XPath($"//*[contains(@Name, '{partialName}')]"), timeoutMS, global);
}
/// <summary>
/// Finds an element using partial name matching (contains).
/// </summary>
/// <param name="partialName">Part of the name to search for.</param>
/// <param name="timeoutMS">The timeout in milliseconds (default is 5000).</param>
/// <returns>The found element.</returns>
protected Element FindByPartialName(string partialName, int timeoutMS = 5000, bool global = false)
{
return FindByPartialName<Element>(partialName, timeoutMS, global);
}
/// <summary>
/// Base method for finding elements by selector and filtering by name pattern.
/// </summary>
/// <typeparam name="T">The class of the element, should be Element or its derived class.</typeparam>
/// <param name="selector">The selector to find initial candidates.</param>
/// <param name="namePattern">Pattern to match against the Name attribute. Supports regex patterns.</param>
/// <param name="timeoutMS">The timeout in milliseconds (default is 5000).</param>
/// <param name="errorMessage">Custom error message when no element is found.</param>
/// <returns>The found element.</returns>
private T FindByNamePattern<T>(By selector, string namePattern, int timeoutMS = 5000, bool global = false, string? errorMessage = null)
where T : Element, new()
{
var elements = Session.FindAll<T>(selector, timeoutMS, global);
var regex = new Regex(namePattern, RegexOptions.IgnoreCase);
foreach (var element in elements)
{
var name = element.GetAttribute("Name");
if (!string.IsNullOrEmpty(name) && regex.IsMatch(name))
{
return element;
}
}
throw new NoSuchElementException(errorMessage ?? $"No element found matching pattern: {namePattern}");
}
/// <summary>
/// Finds an element using regular expression pattern matching.
/// </summary>
/// <typeparam name="T">The class of the element, should be Element or its derived class.</typeparam>
/// <param name="pattern">Regular expression pattern to match against the Name attribute.</param>
/// <param name="timeoutMS">The timeout in milliseconds (default is 5000).</param>
/// <returns>The found element.</returns>
protected T FindByPattern<T>(string pattern, int timeoutMS = 5000, bool global = false)
where T : Element, new()
{
return FindByNamePattern<T>(By.XPath("//*[@Name]"), pattern, timeoutMS, global, $"No element found matching pattern: {pattern}");
}
/// <summary>
/// Finds an element using regular expression pattern matching.
/// </summary>
/// <param name="pattern">Regular expression pattern to match against the Name attribute.</param>
/// <param name="timeoutMS">The timeout in milliseconds (default is 5000).</param>
/// <returns>The found element.</returns>
protected Element FindByPattern(string pattern, int timeoutMS = 5000, bool global = false)
{
return FindByPattern<Element>(pattern, timeoutMS, global);
}
/// <summary>
/// Finds an element by ClassName only.
/// Returns the first element found with the specified ClassName.
/// </summary>
/// <typeparam name="T">The class of the element, should be Element or its derived class.</typeparam>
/// <param name="className">The ClassName to search for (e.g., "Notepad", "CabinetWClass").</param>
/// <param name="timeoutMS">The timeout in milliseconds (default is 5000).</param>
/// <returns>The found element.</returns>
protected T FindByClassName<T>(string className, int timeoutMS = 5000, bool global = false)
where T : Element, new()
{
return Session.Find<T>(By.ClassName(className), timeoutMS, global);
}
/// <summary>
/// Finds an element by ClassName only.
/// Returns the first element found with the specified ClassName.
/// </summary>
/// <param name="className">The ClassName to search for (e.g., "Notepad", "CabinetWClass").</param>
/// <param name="timeoutMS">The timeout in milliseconds (default is 5000).</param>
/// <returns>The found element.</returns>
protected Element FindByClassName(string className, int timeoutMS = 5000, bool global = false)
{
return FindByClassName<Element>(className, timeoutMS, global);
}
/// <summary>
/// Finds an element by ClassName and matches its Name attribute using regex pattern matching.
/// </summary>
/// <typeparam name="T">The class of the element, should be Element or its derived class.</typeparam>
/// <param name="className">The ClassName to search for (e.g., "Notepad", "CabinetWClass").</param>
/// <param name="namePattern">Pattern to match against the Name attribute. Supports regex patterns.</param>
/// <param name="timeoutMS">The timeout in milliseconds (default is 5000).</param>
/// <returns>The found element.</returns>
protected T FindByClassNameAndNamePattern<T>(string className, string namePattern, int timeoutMS = 5000, bool global = false)
where T : Element, new()
{
return FindByNamePattern<T>(By.ClassName(className), namePattern, timeoutMS, global, $"No element with ClassName '{className}' found matching name pattern: {namePattern}");
}
/// <summary>
/// Finds an element by ClassName and matches its Name attribute using regex pattern matching.
/// </summary>
/// <param name="className">The ClassName to search for (e.g., "Notepad", "CabinetWClass").</param>
/// <param name="namePattern">Pattern to match against the Name attribute. Supports regex patterns.</param>
/// <param name="timeoutMS">The timeout in milliseconds (default is 5000).</param>
/// <returns>The found element.</returns>
protected Element FindByClassNameAndNamePattern(string className, string namePattern, int timeoutMS = 5000, bool global = false)
{
return FindByClassNameAndNamePattern<Element>(className, namePattern, timeoutMS, global);
}
/// <summary>
/// Finds a Notepad window regardless of whether the file extension is shown in the title.
/// Handles both "filename.txt - Notepad" and "filename - Notepad" formats.
/// Uses ClassName to efficiently find Notepad windows first, then matches the filename.
/// </summary>
/// <param name="baseFileName">The base filename without extension (e.g., "test" for "test.txt").</param>
/// <param name="timeoutMS">The timeout in milliseconds (default is 5000).</param>
/// <returns>The found Notepad window element.</returns>
protected Element FindNotepadWindow(string baseFileName, int timeoutMS = 5000, bool global = false)
{
string pattern = $@"^{Regex.Escape(baseFileName)}(\.\w+)?(\s*-\s*|\s+)Notepad$";
return FindByClassNameAndNamePattern("Notepad", pattern, timeoutMS, global);
}
/// <summary>
/// Finds an Explorer window regardless of the folder or file name display format.
/// Handles various Explorer window title formats like "FolderName", "FileName", "FolderName - File Explorer", etc.
/// Uses ClassName to efficiently find Explorer windows first, then matches the folder or file name.
/// </summary>
/// <param name="folderName">The folder or file name to search for (e.g., "Documents", "Desktop", "test.txt").</param>
/// <param name="timeoutMS">The timeout in milliseconds (default is 5000).</param>
/// <returns>The found Explorer window element.</returns>
protected Element FindExplorerWindow(string folderName, int timeoutMS = 5000, bool global = false)
{
string pattern = $@"^{Regex.Escape(folderName)}(\s*-\s*(File\s+Explorer|Windows\s+Explorer))?$";
return FindByClassNameAndNamePattern("CabinetWClass", pattern, timeoutMS, global);
}
/// <summary>
/// Finds an Explorer window by partial folder path.
/// Useful when the full path might be displayed in the title.
/// </summary>
/// <param name="partialPath">Part of the folder path to search for.</param>
/// <param name="timeoutMS">The timeout in milliseconds (default is 5000).</param>
/// <returns>The found Explorer window element.</returns>
protected Element FindExplorerByPartialPath(string partialPath, int timeoutMS = 5000, bool global = false)
{
return FindByPartialName(partialPath, timeoutMS, global);
}
/// <summary>
/// Finds all elements by selector.
/// Shortcut for this.Session.FindAll<T>(by, timeoutMS)

8
src/common/UITestAutomation/VisualAssert.cs
View File

@@ -27,10 +27,8 @@ namespace Microsoft.PowerToys.UITest
[RequiresUnreferencedCode("This method uses reflection which may not be compatible with trimming.")]
public static void AreEqual(TestContext? testContext, Element element, string scenarioSubname = "")
{
var pipelinePlatform = Environment.GetEnvironmentVariable("platform");
// Perform visual validation only in the pipeline
if (string.IsNullOrEmpty(pipelinePlatform))
if (!EnvironmentConfig.IsInPipeline)
{
Console.WriteLine("Skip visual validation in the local run.");
return;
@@ -55,11 +53,11 @@ namespace Microsoft.PowerToys.UITest
if (string.IsNullOrWhiteSpace(scenarioSubname))
{
scenarioSubname = string.Join("_", callerClassName, callerName, pipelinePlatform);
scenarioSubname = string.Join("_", callerClassName, callerName, EnvironmentConfig.Platform);
}
else
{
scenarioSubname = string.Join("_", callerClassName, callerName, scenarioSubname.Trim(), pipelinePlatform);
scenarioSubname = string.Join("_", callerClassName, callerName, scenarioSubname.Trim(), EnvironmentConfig.Platform);
}
var baselineImageResourceName = callerMethod!.DeclaringType!.Assembly.GetManifestResourceNames().Where(name => name.Contains(scenarioSubname)).FirstOrDefault();