mirror of
https://github.com/microsoft/PowerToys.git
synced 2026-09-02 12:13:27 +02:00
Advanced Paste additional customizations and PhiSilica provider (#46727)
## Summary Adds an on-device **Phi Silica (Windows AI) paste provider** to Advanced Paste and richer per-action customization, plus the **package-identity plumbing** that lets the unpackaged Advanced Paste app use Windows AI APIs. > Note: this supersedes the earlier "self-contained MSIX package installed/registered by the > WiX installer" approach. Advanced Paste is **not** packaged or installed as a standalone > MSIX. It keeps shipping as the existing unpackaged, self-contained WinUI 3 executable in > `WinUI3Apps/` and acquires **package identity at runtime from the existing PowerToys sparse > package** — there are no installer or ESRP MSIX repack changes. ## Advanced Paste features - **New Phi Silica paste provider** (`CustomActions/PhiSilicaPasteProvider.cs`): an on-device AI provider backed by the Windows AI Phi Silica language model — no cloud endpoint or API key required. Registered as a new `AIServiceType` via `PasteAIProviderFactory` / `AIServiceTypeRegistry`. - **Additional custom actions** (`AdvancedPasteAdditionalAction`): user-defined actions with their own prompt, system prompt, AI provider, and shortcut — plus an optional "coaching" prompt/system-prompt/provider/shortcut and shortcut-conflict detection. - **Built-in default prompts** (`AdvancedPasteDefaultPrompts`) and updates to `AdvancedPasteCustomAction`, `PasteFormat(s)`, `OptionsViewModel`, and `PasteFormatExecutor` to support per-action provider selection and system prompts. - **Settings UI** (`AdvancedPastePage.xaml`/`.xaml.cs`, `Resources.resw`): configure the Phi Silica provider, choose a provider per action, edit system/coaching prompts, and a Phi Silica availability/readiness experience — Settings queries the Advanced Paste executable via `--check-phi-silica`, and a **"Download model"** action triggers `--prepare-phi-silica` to fetch the model and then re-probes. ## Package identity for Windows AI (replaces the MSIX-install approach) - Phi Silica is a **Limited Access Feature (LAF)** that can only be unlocked by a process with a registered **package identity**. Advanced Paste runs unpackaged, so it obtains identity from the existing **`Microsoft.PowerToys.SparseApp`** sparse package (`src/PackageIdentity/`): a new `<Application Id="PowerToys.AdvancedPasteUI">` entry in `AppxManifest.xml` maps it to `PowerToys.AdvancedPaste.exe`, with matching updates to `BuildSparsePackage.ps1`. - **LAF unlock** at runtime via `PhiSilicaLafHelper.cs`. The token/attestation are baked at build time by the `GeneratePhiSilicaLafCredentials` MSBuild target into `PhiSilicaLafCredentials.g.cs` — local **dev defaults** live in `src/PhiSilicaLaf.props` (imported from `Directory.Build.props`) and the **production secret** is injected via `/p:` in the release pipeline. - New **`AdvancedPaste.dev.manifest` / `AdvancedPaste.prod.manifest`** application manifests (selected by `CIBuild`) declaring full-trust and the system AI models capability. ## Build & pipeline - **Windows App SDK** moved to the coherent **stable `2.2.0`** line and **added `Microsoft.WindowsAppSDK.AI` `2.2.3`** (the Phi Silica APIs). Foundation `2.1.0` carries the sparse-identity PRI fix, and the stable AI build matches the OS Windows AI runtime. - **Independent versioning** for Advanced Paste (`src/modules/AdvancedPaste/custom.props`, XES one-store versioning, `AdvancedPasteVersion`). A `steps-setup-versioning.yml` step is added for Advanced Paste in `job-build-project.yml`, ordered **before** CmdPal to avoid a version-collision installer failure (WIX0103). - `release.yml` passes `PhiSilicaLafToken`/`PhiSilicaLafAttestation` into the main build; spell-check allow-list/patterns updated. - Removed now-unneeded dependencies: the `Microsoft.Windows.Compatibility` reference and the `Common.UI` "force matching DLL versions" hack. <img width="1073" height="716" alt="image" src="https://github.com/user-attachments/assets/9364d86a-c0d1-4a08-a669-d98cbcb4b140" /> <img width="1038" height="308" alt="image" src="https://github.com/user-attachments/assets/00eef1dd-b407-4841-bd64-f54f8a145c46" /> <img width="194" height="405" alt="image" src="https://github.com/user-attachments/assets/d2f9c5bb-2507-4112-b3e0-56da681f88ea" /> <img width="489" height="453" alt="image" src="https://github.com/user-attachments/assets/811d1afd-9993-48be-824e-82bb56c5ceca" /> [Video clip internal](https://onedrive.cloud.microsoft/✌️/a@9n6nl3fp/S/cQpvAHrL5M9ZR6eUawztyfyBEgUCwCF-aKg9TbKyyGWP4c0KMA) [Build internal](https://microsoft.visualstudio.com/Dart/_build/results?buildId=149920754&view=artifacts&pathAsName=false&type=publishedArtifacts) --------- Co-authored-by: Niels Laute <niels.laute@live.nl> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
@@ -66,6 +66,8 @@ namespace
|
||||
const wchar_t JSON_KEY_PROVIDERS[] = L"providers";
|
||||
const wchar_t JSON_KEY_SERVICE_TYPE[] = L"service-type";
|
||||
const wchar_t JSON_KEY_ENABLE_ADVANCED_AI[] = L"enable-advanced-ai";
|
||||
const wchar_t JSON_KEY_COACHING_SHORTCUT[] = L"coaching-shortcut";
|
||||
const wchar_t JSON_KEY_COACHING_ENABLED[] = L"coaching-enabled";
|
||||
const wchar_t JSON_KEY_VALUE[] = L"value";
|
||||
}
|
||||
|
||||
@@ -255,6 +257,21 @@ private:
|
||||
};
|
||||
|
||||
m_additional_actions.push_back(additionalAction);
|
||||
|
||||
// Register coaching shortcut as a separate hotkey with a "-coaching" suffix ID
|
||||
if (action.HasKey(JSON_KEY_COACHING_SHORTCUT) && action.GetNamedBoolean(JSON_KEY_COACHING_ENABLED, false))
|
||||
{
|
||||
auto coachingHotkey = parse_single_hotkey(action.GetNamedObject(JSON_KEY_COACHING_SHORTCUT), actionIsShown);
|
||||
if (coachingHotkey.key != 0)
|
||||
{
|
||||
const AdditionalAction coachingAction
|
||||
{
|
||||
std::wstring(actionName.c_str()) + L"-coaching",
|
||||
coachingHotkey
|
||||
};
|
||||
m_additional_actions.push_back(coachingAction);
|
||||
}
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
@@ -407,6 +424,7 @@ private:
|
||||
// Define the expected order to ensure consistent hotkey ID assignment
|
||||
const std::vector<winrt::hstring> expectedOrder = {
|
||||
L"image-to-text",
|
||||
L"fix-spelling-and-grammar",
|
||||
L"paste-as-file",
|
||||
L"transcode"
|
||||
};
|
||||
@@ -982,6 +1000,12 @@ public:
|
||||
m_triggerEventWaiter.start(CommonSharedConstants::ADVANCED_PASTE_SHOW_UI_EVENT, [this](DWORD) {
|
||||
// Same logic as hotkeyId == 1 (m_advanced_paste_ui_hotkey)
|
||||
Logger::trace(L"AdvancedPaste ShowUI event triggered");
|
||||
|
||||
if (m_auto_copy_selection_custom_action)
|
||||
{
|
||||
send_copy_selection(); // best-effort; ignore failure
|
||||
}
|
||||
|
||||
m_process_manager.start();
|
||||
m_process_manager.bring_to_front();
|
||||
m_process_manager.send_message(CommonSharedConstants::ADVANCED_PASTE_SHOW_UI_MESSAGE);
|
||||
@@ -1032,13 +1056,11 @@ public:
|
||||
}
|
||||
}
|
||||
|
||||
if (is_custom_action_hotkey && m_auto_copy_selection_custom_action)
|
||||
// Try to capture selected text for all hotkey actions when the setting is enabled.
|
||||
// If nothing is selected (clipboard unchanged), fall through to use existing clipboard content.
|
||||
if (m_auto_copy_selection_custom_action)
|
||||
{
|
||||
if (!send_copy_selection())
|
||||
{
|
||||
Logger::warn(L"Auto-copy: failed to copy selection for custom action index {} — aborting action", custom_action_index);
|
||||
return false;
|
||||
}
|
||||
send_copy_selection(); // best-effort; ignore failure
|
||||
}
|
||||
|
||||
m_process_manager.start();
|
||||
|
||||
Reference in New Issue
Block a user