Files
PowerToys/doc/devdocs/modules/advancedpaste-phisilica-local-testing.md
Gleb Khmyznikov a87f5b07a9 Advanced Paste additional customizations and PhiSilica provider (#46727)
## Summary

Adds an on-device **Phi Silica (Windows AI) paste provider** to Advanced
Paste and richer
per-action customization, plus the **package-identity plumbing** that
lets the unpackaged
Advanced Paste app use Windows AI APIs.

> Note: this supersedes the earlier "self-contained MSIX package
installed/registered by the
> WiX installer" approach. Advanced Paste is **not** packaged or
installed as a standalone
> MSIX. It keeps shipping as the existing unpackaged, self-contained
WinUI 3 executable in
> `WinUI3Apps/` and acquires **package identity at runtime from the
existing PowerToys sparse
> package** — there are no installer or ESRP MSIX repack changes.

## Advanced Paste features

- **New Phi Silica paste provider**
(`CustomActions/PhiSilicaPasteProvider.cs`): an on-device
AI provider backed by the Windows AI Phi Silica language model — no
cloud endpoint or API
key required. Registered as a new `AIServiceType` via
`PasteAIProviderFactory` /
  `AIServiceTypeRegistry`.
- **Additional custom actions** (`AdvancedPasteAdditionalAction`):
user-defined actions with
their own prompt, system prompt, AI provider, and shortcut — plus an
optional "coaching"
prompt/system-prompt/provider/shortcut and shortcut-conflict detection.
- **Built-in default prompts** (`AdvancedPasteDefaultPrompts`) and
updates to
  `AdvancedPasteCustomAction`, `PasteFormat(s)`, `OptionsViewModel`, and
`PasteFormatExecutor` to support per-action provider selection and
system prompts.
- **Settings UI** (`AdvancedPastePage.xaml`/`.xaml.cs`,
`Resources.resw`): configure the
Phi Silica provider, choose a provider per action, edit system/coaching
prompts, and a
Phi Silica availability/readiness experience — Settings queries the
Advanced Paste
executable via `--check-phi-silica`, and a **"Download model"** action
triggers
  `--prepare-phi-silica` to fetch the model and then re-probes.

## Package identity for Windows AI (replaces the MSIX-install approach)

- Phi Silica is a **Limited Access Feature (LAF)** that can only be
unlocked by a process
with a registered **package identity**. Advanced Paste runs unpackaged,
so it obtains
identity from the existing **`Microsoft.PowerToys.SparseApp`** sparse
package
(`src/PackageIdentity/`): a new `<Application
Id="PowerToys.AdvancedPasteUI">` entry in
`AppxManifest.xml` maps it to `PowerToys.AdvancedPaste.exe`, with
matching updates to
  `BuildSparsePackage.ps1`.
- **LAF unlock** at runtime via `PhiSilicaLafHelper.cs`. The
token/attestation are baked at
build time by the `GeneratePhiSilicaLafCredentials` MSBuild target into
`PhiSilicaLafCredentials.g.cs` — local **dev defaults** live in
`src/PhiSilicaLaf.props`
(imported from `Directory.Build.props`) and the **production secret** is
injected via
  `/p:` in the release pipeline.
- New **`AdvancedPaste.dev.manifest` / `AdvancedPaste.prod.manifest`**
application manifests
(selected by `CIBuild`) declaring full-trust and the system AI models
capability.

## Build & pipeline

- **Windows App SDK** moved to the coherent **stable `2.2.0`** line and
**added
`Microsoft.WindowsAppSDK.AI` `2.2.3`** (the Phi Silica APIs). Foundation
`2.1.0` carries the
sparse-identity PRI fix, and the stable AI build matches the OS Windows
AI runtime.
- **Independent versioning** for Advanced Paste
(`src/modules/AdvancedPaste/custom.props`,
XES one-store versioning, `AdvancedPasteVersion`). A
`steps-setup-versioning.yml` step is
added for Advanced Paste in `job-build-project.yml`, ordered **before**
CmdPal to avoid a
  version-collision installer failure (WIX0103).
- `release.yml` passes `PhiSilicaLafToken`/`PhiSilicaLafAttestation`
into the main build;
  spell-check allow-list/patterns updated.
- Removed now-unneeded dependencies: the
`Microsoft.Windows.Compatibility` reference and the
  `Common.UI` "force matching DLL versions" hack.
  
<img width="1073" height="716" alt="image"
src="https://github.com/user-attachments/assets/9364d86a-c0d1-4a08-a669-d98cbcb4b140"
/>
<img width="1038" height="308" alt="image"
src="https://github.com/user-attachments/assets/00eef1dd-b407-4841-bd64-f54f8a145c46"
/>
<img width="194" height="405" alt="image"
src="https://github.com/user-attachments/assets/d2f9c5bb-2507-4112-b3e0-56da681f88ea"
/>
<img width="489" height="453" alt="image"
src="https://github.com/user-attachments/assets/811d1afd-9993-48be-824e-82bb56c5ceca"
/>

[Video clip
internal](https://onedrive.cloud.microsoft/✌️/a@9n6nl3fp/S/cQpvAHrL5M9ZR6eUawztyfyBEgUCwCF-aKg9TbKyyGWP4c0KMA)

[Build
internal](https://microsoft.visualstudio.com/Dart/_build/results?buildId=149920754&view=artifacts&pathAsName=false&type=publishedArtifacts)

---------

Co-authored-by: Niels Laute <niels.laute@live.nl>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-08-08 07:43:59 +02:00

101 lines
4.8 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Advanced Paste Phi Silica local testing
How to build, register, and test **Phi Silica** in **Advanced Paste (AP)** on a dev machine,
plus the few things that actually break it.
## How it fits together
AP ships as an **unpackaged, self-contained WinUI 3 exe** (`PowerToys.AdvancedPaste.exe`).
The Windows AI `LanguageModel` (Phi Silica) API is a **Limited Access Feature (LAF)**. For it
to work, all of these must line up:
1. **Package identity** — AP runs with identity granted by the sparse MSIX
`Microsoft.PowerToys.SparseApp`.
2. **Matching LAF creds** — the token/attestation baked into the exe match the registered
sparse package's publisher.
3. **AI metadata deployed** — the `Microsoft.Windows.AI*.winmd` files ship next to the exe;
the AI runtime resolves them **at runtime**.
4. **Model ready** — supported hardware and the on-device model downloaded
(`GetReadyState() == Ready`).
Two identities — the baked token must match the registered package's publisher:
| Build | Publisher Id | LAF creds |
|-------|--------------|-----------|
| **Dev** | `djwsxzxb4ksa8` | dev default in [`src/PhiSilicaLaf.props`](../../../src/PhiSilicaLaf.props) |
| **Prod** | `8wekyb3d8bbwe` | secret, injected only by `.pipelines/v2/release.yml` |
Non-secret pairing check: the exe's baked **Attestation** must equal the registered package's
**PublisherId**.
## Build + register (dev loop)
```powershell
$repo = "X:\GitHub\PowerToys"; $Plat = "ARM64"; $Cfg = "Debug" # or x64 / Release
# Build AP only (C#; reuses existing C++ outputs):
dotnet restore "$repo\src\modules\AdvancedPaste\AdvancedPaste\AdvancedPaste.csproj" /p:Platform=$Plat
& "$repo\tools\build\build.cmd" -Path "$repo\src\modules\AdvancedPaste\AdvancedPaste" `
-Platform $Plat -Configuration $Cfg /p:BuildProjectReferences=false
# Register the dev sparse package (creates + trusts a dev cert, grants identity):
pwsh -ExecutionPolicy Bypass -File "$repo\src\PackageIdentity\BuildSparsePackage.ps1" `
-Platform $Plat -Configuration $Cfg -DevRegister
# Expect: PublisherId djwsxzxb4ksa8, IsDevelopmentMode True
```
## Check the API
`PowerToys.AdvancedPaste.exe` is a **GUI-subsystem** app — run directly in a console it prints
nothing and returns no exit code. **Redirect** stdout/stderr and wait:
```powershell
$exe = "$repo\$Plat\$Cfg\WinUI3Apps\PowerToys.AdvancedPaste.exe"
$o = "$env:TEMP\ap.out"; $e = "$env:TEMP\ap.err"
$p = Start-Process $exe '--check-phi-silica' -Wait -PassThru -WindowStyle Hidden `
-RedirectStandardOutput $o -RedirectStandardError $e
"exit=$($p.ExitCode) stdout=$((Get-Content $o -Raw).Trim())"
Get-Content $e -Raw # stderr: [phi-silica] LAF unlock status: <…>; ReadyState: <…>
```
| `--check-phi-silica` | `--prepare-phi-silica` (downloads the model) |
|----------------------|----------------------------------------------|
| `0` Available · `1` NotReady · `2` NotSupported / unlock failed | `0` Ready · `1` Failed · `2` NotSupported |
`--check` only reads state; use `--prepare` to trigger the model download (`EnsureReadyAsync`).
On failure it prints the `HRESULT` to stderr.
Confirm the running AP has identity:
```powershell
$apPid = (Get-Process PowerToys.AdvancedPaste -EA SilentlyContinue | Select-Object -First 1).Id
if ($apPid) { & "$repo\src\PackageIdentity\Check-ProcessIdentity.ps1" -ProcessId $apPid }
# Expect a PFN ending in the publisher id that matches the baked attestation
```
## What actually breaks it
- **Missing `.winmd` (most important).** The Windows AI runtime resolves
`Microsoft.Windows.AI*.winmd` from the app folder at runtime. If they aren't deployed,
`GetReadyState()` returns `NotReady` and `EnsureReadyAsync()` fails with
`RO_E_METADATA_NAME_NOT_FOUND` (`0x8000000F`) — even though identity, token, and the AI DLLs
are all correct. The build emits these winmd into `WinUI3Apps\`; the **installer must harvest
them** (`*.winmd` is in the inclusion list of
[`generateAllFileComponents.ps1`](../../../installer/PowerToysSetupVNext/generateAllFileComponents.ps1)).
Classic symptom: "works from the build output but not from the installer" → check that the
installed `WinUI3Apps\` contains `Microsoft.Windows.AI*.winmd`.
- **Dev/prod mismatch.** A dev-cred exe running against a prod sparse package (or vice versa)
makes the LAF unlock silently return `Unavailable`. Keep the exe and the registered package
the same flavor, and verify with the attestation == publisherId check above.
- **Forgot to redirect.** `--check-phi-silica` in a console prints nothing — that's the
GUI-subsystem quirk, not a result.
## Cleanup
```powershell
pwsh -ExecutionPolicy Bypass -File "$repo\src\PackageIdentity\BuildSparsePackage.ps1" -Unregister
```
⚠️ This removes any `Microsoft.PowerToys.SparseApp` registration, **including a prod one** from
an installer — reinstall/repair PowerToys to restore it.