mirror of https://github.com/microsoft/PowerToys.git synced 2026-09-01 19:51:34 +02:00

Files

Gleb Khmyznikov a87f5b07a9 Advanced Paste additional customizations and PhiSilica provider (#46727 )

## Summary

Adds an on-device **Phi Silica (Windows AI) paste provider** to Advanced
Paste and richer
per-action customization, plus the **package-identity plumbing** that
lets the unpackaged
Advanced Paste app use Windows AI APIs.

> Note: this supersedes the earlier "self-contained MSIX package
installed/registered by the
> WiX installer" approach. Advanced Paste is **not** packaged or
installed as a standalone
> MSIX. It keeps shipping as the existing unpackaged, self-contained
WinUI 3 executable in
> `WinUI3Apps/` and acquires **package identity at runtime from the
existing PowerToys sparse
> package** — there are no installer or ESRP MSIX repack changes.

## Advanced Paste features

- **New Phi Silica paste provider**
(`CustomActions/PhiSilicaPasteProvider.cs`): an on-device
AI provider backed by the Windows AI Phi Silica language model — no
cloud endpoint or API
key required. Registered as a new `AIServiceType` via
`PasteAIProviderFactory` /
  `AIServiceTypeRegistry`.
- **Additional custom actions** (`AdvancedPasteAdditionalAction`):
user-defined actions with
their own prompt, system prompt, AI provider, and shortcut — plus an
optional "coaching"
prompt/system-prompt/provider/shortcut and shortcut-conflict detection.
- **Built-in default prompts** (`AdvancedPasteDefaultPrompts`) and
updates to
  `AdvancedPasteCustomAction`, `PasteFormat(s)`, `OptionsViewModel`, and
`PasteFormatExecutor` to support per-action provider selection and
system prompts.
- **Settings UI** (`AdvancedPastePage.xaml`/`.xaml.cs`,
`Resources.resw`): configure the
Phi Silica provider, choose a provider per action, edit system/coaching
prompts, and a
Phi Silica availability/readiness experience — Settings queries the
Advanced Paste
executable via `--check-phi-silica`, and a **"Download model"** action
triggers
  `--prepare-phi-silica` to fetch the model and then re-probes.

## Package identity for Windows AI (replaces the MSIX-install approach)

- Phi Silica is a **Limited Access Feature (LAF)** that can only be
unlocked by a process
with a registered **package identity**. Advanced Paste runs unpackaged,
so it obtains
identity from the existing **`Microsoft.PowerToys.SparseApp`** sparse
package
(`src/PackageIdentity/`): a new `<Application
Id="PowerToys.AdvancedPasteUI">` entry in
`AppxManifest.xml` maps it to `PowerToys.AdvancedPaste.exe`, with
matching updates to
  `BuildSparsePackage.ps1`.
- **LAF unlock** at runtime via `PhiSilicaLafHelper.cs`. The
token/attestation are baked at
build time by the `GeneratePhiSilicaLafCredentials` MSBuild target into
`PhiSilicaLafCredentials.g.cs` — local **dev defaults** live in
`src/PhiSilicaLaf.props`
(imported from `Directory.Build.props`) and the **production secret** is
injected via
  `/p:` in the release pipeline.
- New **`AdvancedPaste.dev.manifest` / `AdvancedPaste.prod.manifest`**
application manifests
(selected by `CIBuild`) declaring full-trust and the system AI models
capability.

## Build & pipeline

- **Windows App SDK** moved to the coherent **stable `2.2.0`** line and
**added
`Microsoft.WindowsAppSDK.AI` `2.2.3`** (the Phi Silica APIs). Foundation
`2.1.0` carries the
sparse-identity PRI fix, and the stable AI build matches the OS Windows
AI runtime.
- **Independent versioning** for Advanced Paste
(`src/modules/AdvancedPaste/custom.props`,
XES one-store versioning, `AdvancedPasteVersion`). A
`steps-setup-versioning.yml` step is
added for Advanced Paste in `job-build-project.yml`, ordered **before**
CmdPal to avoid a
  version-collision installer failure (WIX0103).
- `release.yml` passes `PhiSilicaLafToken`/`PhiSilicaLafAttestation`
into the main build;
  spell-check allow-list/patterns updated.
- Removed now-unneeded dependencies: the
`Microsoft.Windows.Compatibility` reference and the
  `Common.UI` "force matching DLL versions" hack.
  
<img width="1073" height="716" alt="image"
src="https://github.com/user-attachments/assets/9364d86a-c0d1-4a08-a669-d98cbcb4b140"
/>
<img width="1038" height="308" alt="image"
src="https://github.com/user-attachments/assets/00eef1dd-b407-4841-bd64-f54f8a145c46"
/>
<img width="194" height="405" alt="image"
src="https://github.com/user-attachments/assets/d2f9c5bb-2507-4112-b3e0-56da681f88ea"
/>
<img width="489" height="453" alt="image"
src="https://github.com/user-attachments/assets/811d1afd-9993-48be-824e-82bb56c5ceca"
/>

[Video clip
internal](https://onedrive.cloud.microsoft/✌️/a@9n6nl3fp/S/cQpvAHrL5M9ZR6eUawztyfyBEgUCwCF-aKg9TbKyyGWP4c0KMA)

[Build
internal](https://microsoft.visualstudio.com/Dart/_build/results?buildId=149920754&view=artifacts&pathAsName=false&type=publishedArtifacts)

---------

Co-authored-by: Niels Laute <niels.laute@live.nl>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

2026-08-08 07:43:59 +02:00

7.3 KiB

Raw Blame History

Advanced Paste

Public overview - Microsoft Learn

Quick Links

All Issues
Bugs
Pull Requests

Overview

Advanced Paste is a PowerToys module that provides enhanced clipboard pasting with formatting options and additional functionality.

Implementation Details

Source code

TODO: Add implementation details

Paste with AI Preview

The "Show preview" setting (ShowCustomPreview) controls whether AI-generated results are displayed in a preview window before pasting. The preview feature does not consume additional AI credits—the preview displays the same AI response that was already generated, cached locally from a single API call.

The implementation flow:

User initiates "Paste with AI" action
A single AI API call is made via ExecutePasteFormatAsync
The result is cached in GeneratedResponses
If preview is enabled, the cached result is displayed in the preview UI
User can paste the cached result without any additional API calls

See the ExecutePasteFormatAsync(PasteFormat, PasteActionSource) method in OptionsViewModel.cs for the implementation.

Debugging

Advanced Paste is an unpackaged, self-contained WinUI 3 app (PowerToys.AdvancedPaste.exe). To call Windows AI APIs (Phi Silica / Microsoft.Windows.AI.Text.LanguageModel) it acquires package identity at runtime via a shared sparse MSIX package (Microsoft.PowerToys.SparseApp).

Running and attaching the debugger

Set the Runner project (src/runner) as the startup project in Visual Studio.
Launch the Runner (F5). This starts the PowerToys tray icon and loads all module interfaces.
Open Settings (right-click tray icon → Settings) and enable the Advanced Paste module if it isn't already. The module launches PowerToys.AdvancedPaste.exe in the background immediately.
In Visual Studio, go to Debug → Attach to Process (Ctrl+Alt+P) and attach to PowerToys.AdvancedPaste.exe (select Managed (.NET Core) debugger).

Alternatively, use the VS Code launch configuration "Run AdvancedPaste" from .vscode/launch.json to launch the exe directly — but note that without the Runner, IPC and hotkeys won't work.

Sparse package identity (local development)

Why is this needed?

The LanguageModel API requires a Limited Access Feature (LAF) unlock, which only succeeds when the calling process has a matching package identity.
Advanced Paste is an unpackaged, self-contained WinUI 3 app. The sparse package grants it identity without converting it to a full MSIX.
The csproj uses <ProjectPriFileName>PowerToys.AdvancedPaste.pri</ProjectPriFileName> (matching the convention of other WinUI3 apps like ImageResizer). This requires WindowsAppSDK Foundation >= 2.0.22 (PR #6376) which fixes MRT PRI lookup under sparse identity so Application.LoadComponent resolves custom-named PRI files instead of hard-coding resources.pri.

One-step dev setup

pwsh src/PackageIdentity/BuildSparsePackage.ps1 -Platform ARM64 -Configuration Debug -DevRegister

-DevRegister:

Generates a dev certificate under src/PackageIdentity/.user/ (first run only).
Auto-imports that certificate into CurrentUser\TrustedPeople and CurrentUser\Root so the OS grants sparse identity to AP (without trust, GetPackageFamilyName returns APPMODEL_ERROR_NO_PACKAGE and LAF unlock silently fails).
Removes any prior registration.
Rewrites the publisher in a temp copy of AppxManifest.xml to match the dev cert subject.
Registers via Add-AppxPackage -Register … -ExternalLocation X:\…\<Platform>\<Config>\WinUI3Apps.

After registration verify:

$pkg = Get-AppxPackage -Name '*SparseApp*'
$pkg.PackageFamilyName    # Microsoft.PowerToys.SparseApp_<PublisherId>
$pkg.PublisherId          # djwsxzxb4ksa8
$pkg.IsDevelopmentMode    # True

Confirm AP picks up sparse identity at runtime:

& 'ARM64\Debug\WinUI3Apps\PowerToys.AdvancedPaste.exe' --check-phi-silica
# Exit 0 = Available, 1 = NotReady, 2 = NotSupported

Re-register after rebuilding AP, changing src/PackageIdentity/AppxManifest.xml, or switching platforms/configurations by re-running the same command. Unregister with -Unregister.

Troubleshooting

Problem	Cause	Fix
`GetPackageFamilyName` returns `APPMODEL_ERROR_NO_PACKAGE` (15700) at runtime; LAF unlock returns `Unavailable`	Dev certificate not trusted (or sparse package not registered)	Re-run `BuildSparsePackage.ps1 -DevRegister` — auto-imports the cert into `TrustedPeople` and `Root`.
`Microsoft.UI.Xaml.dll` crash with `0xC000027B` (class-not-registered) on AP or Settings startup	`<Application>` `Executable` path in `src/PackageIdentity/AppxManifest.xml` does not resolve under the registered `ExternalLocation` (`<Config>\WinUI3Apps\`)	Confirm every `Executable` is relative to `WinUI3Apps\` (per #47177) and the file exists under the build output.
AP launches but never shows a window when triggered via hotkey	Runner's pipe-server wait timed out before AP's cold-start finished bootstrapping WinAppSDK + DI host	Already mitigated by the 15 s pipe timeout in `AdvancedPasteProcessManager.cpp`; warm-start launches connect in well under 1 s.
`XamlParseException` / `ms-appx:///Microsoft.UI.Xaml/Themes/…` not found	WindowsAppSDK Foundation < 2.0.22; MRT can't resolve custom PRI name under sparse identity	Ensure `Microsoft.WindowsAppSDK.Foundation` >= 2.0.22 in `Directory.Packages.props`.

How Settings UI checks Phi Silica availability

Settings UI does not have sparse package identity. To check whether Phi Silica is available, it launches Advanced Paste as a short-lived subprocess:

PowerToys.AdvancedPaste.exe --check-phi-silica

Program.Main recognizes this flag, calls PhiSilicaLafHelper.TryUnlock() + LanguageModel.GetReadyState(), prints one of Available / NotReady / NotSupported to stdout, and exits with the matching code (0/1/2). Settings reads stdout with a 10 s wait. Because each call is a fresh process, transient Unavailable results are not cached across checks.

Settings

Setting	Description
`ShowCustomPreview`	When enabled, shows AI-generated results in a preview window before pasting. Does not affect AI credit consumption.

Future Improvements

TODO: Add potential future improvements

7.3 KiB Raw Blame History