Files
PowerToys/doc/devdocs/modules/advancedpaste-phisilica-local-testing.md
Gleb Khmyznikov a87f5b07a9 Advanced Paste additional customizations and PhiSilica provider (#46727)
## Summary

Adds an on-device **Phi Silica (Windows AI) paste provider** to Advanced
Paste and richer
per-action customization, plus the **package-identity plumbing** that
lets the unpackaged
Advanced Paste app use Windows AI APIs.

> Note: this supersedes the earlier "self-contained MSIX package
installed/registered by the
> WiX installer" approach. Advanced Paste is **not** packaged or
installed as a standalone
> MSIX. It keeps shipping as the existing unpackaged, self-contained
WinUI 3 executable in
> `WinUI3Apps/` and acquires **package identity at runtime from the
existing PowerToys sparse
> package** — there are no installer or ESRP MSIX repack changes.

## Advanced Paste features

- **New Phi Silica paste provider**
(`CustomActions/PhiSilicaPasteProvider.cs`): an on-device
AI provider backed by the Windows AI Phi Silica language model — no
cloud endpoint or API
key required. Registered as a new `AIServiceType` via
`PasteAIProviderFactory` /
  `AIServiceTypeRegistry`.
- **Additional custom actions** (`AdvancedPasteAdditionalAction`):
user-defined actions with
their own prompt, system prompt, AI provider, and shortcut — plus an
optional "coaching"
prompt/system-prompt/provider/shortcut and shortcut-conflict detection.
- **Built-in default prompts** (`AdvancedPasteDefaultPrompts`) and
updates to
  `AdvancedPasteCustomAction`, `PasteFormat(s)`, `OptionsViewModel`, and
`PasteFormatExecutor` to support per-action provider selection and
system prompts.
- **Settings UI** (`AdvancedPastePage.xaml`/`.xaml.cs`,
`Resources.resw`): configure the
Phi Silica provider, choose a provider per action, edit system/coaching
prompts, and a
Phi Silica availability/readiness experience — Settings queries the
Advanced Paste
executable via `--check-phi-silica`, and a **"Download model"** action
triggers
  `--prepare-phi-silica` to fetch the model and then re-probes.

## Package identity for Windows AI (replaces the MSIX-install approach)

- Phi Silica is a **Limited Access Feature (LAF)** that can only be
unlocked by a process
with a registered **package identity**. Advanced Paste runs unpackaged,
so it obtains
identity from the existing **`Microsoft.PowerToys.SparseApp`** sparse
package
(`src/PackageIdentity/`): a new `<Application
Id="PowerToys.AdvancedPasteUI">` entry in
`AppxManifest.xml` maps it to `PowerToys.AdvancedPaste.exe`, with
matching updates to
  `BuildSparsePackage.ps1`.
- **LAF unlock** at runtime via `PhiSilicaLafHelper.cs`. The
token/attestation are baked at
build time by the `GeneratePhiSilicaLafCredentials` MSBuild target into
`PhiSilicaLafCredentials.g.cs` — local **dev defaults** live in
`src/PhiSilicaLaf.props`
(imported from `Directory.Build.props`) and the **production secret** is
injected via
  `/p:` in the release pipeline.
- New **`AdvancedPaste.dev.manifest` / `AdvancedPaste.prod.manifest`**
application manifests
(selected by `CIBuild`) declaring full-trust and the system AI models
capability.

## Build & pipeline

- **Windows App SDK** moved to the coherent **stable `2.2.0`** line and
**added
`Microsoft.WindowsAppSDK.AI` `2.2.3`** (the Phi Silica APIs). Foundation
`2.1.0` carries the
sparse-identity PRI fix, and the stable AI build matches the OS Windows
AI runtime.
- **Independent versioning** for Advanced Paste
(`src/modules/AdvancedPaste/custom.props`,
XES one-store versioning, `AdvancedPasteVersion`). A
`steps-setup-versioning.yml` step is
added for Advanced Paste in `job-build-project.yml`, ordered **before**
CmdPal to avoid a
  version-collision installer failure (WIX0103).
- `release.yml` passes `PhiSilicaLafToken`/`PhiSilicaLafAttestation`
into the main build;
  spell-check allow-list/patterns updated.
- Removed now-unneeded dependencies: the
`Microsoft.Windows.Compatibility` reference and the
  `Common.UI` "force matching DLL versions" hack.
  
<img width="1073" height="716" alt="image"
src="https://github.com/user-attachments/assets/9364d86a-c0d1-4a08-a669-d98cbcb4b140"
/>
<img width="1038" height="308" alt="image"
src="https://github.com/user-attachments/assets/00eef1dd-b407-4841-bd64-f54f8a145c46"
/>
<img width="194" height="405" alt="image"
src="https://github.com/user-attachments/assets/d2f9c5bb-2507-4112-b3e0-56da681f88ea"
/>
<img width="489" height="453" alt="image"
src="https://github.com/user-attachments/assets/811d1afd-9993-48be-824e-82bb56c5ceca"
/>

[Video clip
internal](https://onedrive.cloud.microsoft/✌️/a@9n6nl3fp/S/cQpvAHrL5M9ZR6eUawztyfyBEgUCwCF-aKg9TbKyyGWP4c0KMA)

[Build
internal](https://microsoft.visualstudio.com/Dart/_build/results?buildId=149920754&view=artifacts&pathAsName=false&type=publishedArtifacts)

---------

Co-authored-by: Niels Laute <niels.laute@live.nl>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-08-08 07:43:59 +02:00

4.8 KiB
Raw Permalink Blame History

Advanced Paste Phi Silica local testing

How to build, register, and test Phi Silica in Advanced Paste (AP) on a dev machine, plus the few things that actually break it.

How it fits together

AP ships as an unpackaged, self-contained WinUI 3 exe (PowerToys.AdvancedPaste.exe). The Windows AI LanguageModel (Phi Silica) API is a Limited Access Feature (LAF). For it to work, all of these must line up:

  1. Package identity — AP runs with identity granted by the sparse MSIX Microsoft.PowerToys.SparseApp.
  2. Matching LAF creds — the token/attestation baked into the exe match the registered sparse package's publisher.
  3. AI metadata deployed — the Microsoft.Windows.AI*.winmd files ship next to the exe; the AI runtime resolves them at runtime.
  4. Model ready — supported hardware and the on-device model downloaded (GetReadyState() == Ready).

Two identities — the baked token must match the registered package's publisher:

Build Publisher Id LAF creds
Dev djwsxzxb4ksa8 dev default in src/PhiSilicaLaf.props
Prod 8wekyb3d8bbwe secret, injected only by .pipelines/v2/release.yml

Non-secret pairing check: the exe's baked Attestation must equal the registered package's PublisherId.

Build + register (dev loop)

$repo = "X:\GitHub\PowerToys"; $Plat = "ARM64"; $Cfg = "Debug"   # or x64 / Release

# Build AP only (C#; reuses existing C++ outputs):
dotnet restore "$repo\src\modules\AdvancedPaste\AdvancedPaste\AdvancedPaste.csproj" /p:Platform=$Plat
& "$repo\tools\build\build.cmd" -Path "$repo\src\modules\AdvancedPaste\AdvancedPaste" `
    -Platform $Plat -Configuration $Cfg /p:BuildProjectReferences=false

# Register the dev sparse package (creates + trusts a dev cert, grants identity):
pwsh -ExecutionPolicy Bypass -File "$repo\src\PackageIdentity\BuildSparsePackage.ps1" `
    -Platform $Plat -Configuration $Cfg -DevRegister
# Expect: PublisherId djwsxzxb4ksa8, IsDevelopmentMode True

Check the API

PowerToys.AdvancedPaste.exe is a GUI-subsystem app — run directly in a console it prints nothing and returns no exit code. Redirect stdout/stderr and wait:

$exe = "$repo\$Plat\$Cfg\WinUI3Apps\PowerToys.AdvancedPaste.exe"
$o = "$env:TEMP\ap.out"; $e = "$env:TEMP\ap.err"
$p = Start-Process $exe '--check-phi-silica' -Wait -PassThru -WindowStyle Hidden `
        -RedirectStandardOutput $o -RedirectStandardError $e
"exit=$($p.ExitCode)  stdout=$((Get-Content $o -Raw).Trim())"
Get-Content $e -Raw   # stderr: [phi-silica] LAF unlock status: <…>; ReadyState: <…>
--check-phi-silica --prepare-phi-silica (downloads the model)
0 Available · 1 NotReady · 2 NotSupported / unlock failed 0 Ready · 1 Failed · 2 NotSupported

--check only reads state; use --prepare to trigger the model download (EnsureReadyAsync). On failure it prints the HRESULT to stderr.

Confirm the running AP has identity:

$apPid = (Get-Process PowerToys.AdvancedPaste -EA SilentlyContinue | Select-Object -First 1).Id
if ($apPid) { & "$repo\src\PackageIdentity\Check-ProcessIdentity.ps1" -ProcessId $apPid }
# Expect a PFN ending in the publisher id that matches the baked attestation

What actually breaks it

  • Missing .winmd (most important). The Windows AI runtime resolves Microsoft.Windows.AI*.winmd from the app folder at runtime. If they aren't deployed, GetReadyState() returns NotReady and EnsureReadyAsync() fails with RO_E_METADATA_NAME_NOT_FOUND (0x8000000F) — even though identity, token, and the AI DLLs are all correct. The build emits these winmd into WinUI3Apps\; the installer must harvest them (*.winmd is in the inclusion list of generateAllFileComponents.ps1). Classic symptom: "works from the build output but not from the installer" → check that the installed WinUI3Apps\ contains Microsoft.Windows.AI*.winmd.
  • Dev/prod mismatch. A dev-cred exe running against a prod sparse package (or vice versa) makes the LAF unlock silently return Unavailable. Keep the exe and the registered package the same flavor, and verify with the attestation == publisherId check above.
  • Forgot to redirect. --check-phi-silica in a console prints nothing — that's the GUI-subsystem quirk, not a result.

Cleanup

pwsh -ExecutionPolicy Bypass -File "$repo\src\PackageIdentity\BuildSparsePackage.ps1" -Unregister

⚠️ This removes any Microsoft.PowerToys.SparseApp registration, including a prod one from an installer — reinstall/repair PowerToys to restore it.