Files
PowerToys/doc/devdocs/modules/advancedpaste.md

121 lines
7.3 KiB
Markdown
Raw Normal View History

# Advanced Paste
[Public overview - Microsoft Learn](https://learn.microsoft.com/en-us/windows/powertoys/advanced-paste)
## Quick Links
[All Issues](https://github.com/microsoft/PowerToys/issues?q=is%3Aopen%20label%3A%22Product-Advanced%20Paste%22)<br>
[Bugs](https://github.com/microsoft/PowerToys/issues?q=is%3Aopen%20label%3A%22Product-Advanced%20Paste%22%20label%3AIssue-Bug)<br>
[Pull Requests](https://github.com/microsoft/PowerToys/pulls?q=is%3Apr+is%3Aopen++label%3A%22Product-Advanced+Paste%22)
## Overview
Advanced Paste is a PowerToys module that provides enhanced clipboard pasting with formatting options and additional functionality.
## Implementation Details
[Source code](/src/modules/AdvancedPaste)
TODO: Add implementation details
docs(paste): add AI preview credit documentation (#45236) docs(paste): add AI preview credit documentation ```markdown ## Summary of the Pull Request Adds documentation clarifying that the "Show preview" setting for Paste with AI does not consume additional AI credits. The preview displays the same AI response that was already generated from a single API call, cached locally. ## PR Checklist - [x] Closes: #32950 - [x] **Communication:** I've discussed this with core contributors already. If the work hasn't been agreed, this work might be rejected - [ ] **Tests:** Added/updated and all pass - N/A (documentation only) - [ ] **Localization:** All end-user-facing strings can be localized - N/A (dev docs only) - [x] **Dev docs:** Added/updated - [ ] **Documentation updated:** If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/windows-uwp/tree/docs/hub/powertoys) and link it here: #xxx ## Detailed Description of the Pull Request / Additional comments This PR addresses the question raised in issue #32950 about whether enabling preview for Paste with AI costs extra AI quota. Changes to `doc/devdocs/modules/advancedpaste.md`: - Added new "Paste with AI Preview" section explaining: - The `ShowCustomPreview` setting behavior - Confirmation that preview does **not** consume additional AI credits - The implementation flow showing a single API call with local caching - Reference to `OptionsViewModel.cs` lines 702-717 - Added settings documentation table for `ShowCustomPreview` Fixes #32950 ## Validation Steps Performed - Verified documentation renders correctly in Markdown preview - Confirmed technical accuracy by referencing `OptionsViewModel.cs` implementation ``` --------- Co-authored-by: yeelam-gordon <yeelam-gordon@users.noreply.github.com>
2026-01-31 09:03:24 -08:00
### Paste with AI Preview
The "Show preview" setting (`ShowCustomPreview`) controls whether AI-generated results are displayed in a preview window before pasting. **The preview feature does not consume additional AI credits**—the preview displays the same AI response that was already generated, cached locally from a single API call.
The implementation flow:
1. User initiates "Paste with AI" action
2. A single AI API call is made via `ExecutePasteFormatAsync`
3. The result is cached in `GeneratedResponses`
4. If preview is enabled, the cached result is displayed in the preview UI
5. User can paste the cached result without any additional API calls
See the `ExecutePasteFormatAsync(PasteFormat, PasteActionSource)` method in `OptionsViewModel.cs` for the implementation.
## Debugging
Advanced Paste additional customizations and PhiSilica provider (#46727) ## Summary Adds an on-device **Phi Silica (Windows AI) paste provider** to Advanced Paste and richer per-action customization, plus the **package-identity plumbing** that lets the unpackaged Advanced Paste app use Windows AI APIs. > Note: this supersedes the earlier "self-contained MSIX package installed/registered by the > WiX installer" approach. Advanced Paste is **not** packaged or installed as a standalone > MSIX. It keeps shipping as the existing unpackaged, self-contained WinUI 3 executable in > `WinUI3Apps/` and acquires **package identity at runtime from the existing PowerToys sparse > package** — there are no installer or ESRP MSIX repack changes. ## Advanced Paste features - **New Phi Silica paste provider** (`CustomActions/PhiSilicaPasteProvider.cs`): an on-device AI provider backed by the Windows AI Phi Silica language model — no cloud endpoint or API key required. Registered as a new `AIServiceType` via `PasteAIProviderFactory` / `AIServiceTypeRegistry`. - **Additional custom actions** (`AdvancedPasteAdditionalAction`): user-defined actions with their own prompt, system prompt, AI provider, and shortcut — plus an optional "coaching" prompt/system-prompt/provider/shortcut and shortcut-conflict detection. - **Built-in default prompts** (`AdvancedPasteDefaultPrompts`) and updates to `AdvancedPasteCustomAction`, `PasteFormat(s)`, `OptionsViewModel`, and `PasteFormatExecutor` to support per-action provider selection and system prompts. - **Settings UI** (`AdvancedPastePage.xaml`/`.xaml.cs`, `Resources.resw`): configure the Phi Silica provider, choose a provider per action, edit system/coaching prompts, and a Phi Silica availability/readiness experience — Settings queries the Advanced Paste executable via `--check-phi-silica`, and a **"Download model"** action triggers `--prepare-phi-silica` to fetch the model and then re-probes. ## Package identity for Windows AI (replaces the MSIX-install approach) - Phi Silica is a **Limited Access Feature (LAF)** that can only be unlocked by a process with a registered **package identity**. Advanced Paste runs unpackaged, so it obtains identity from the existing **`Microsoft.PowerToys.SparseApp`** sparse package (`src/PackageIdentity/`): a new `<Application Id="PowerToys.AdvancedPasteUI">` entry in `AppxManifest.xml` maps it to `PowerToys.AdvancedPaste.exe`, with matching updates to `BuildSparsePackage.ps1`. - **LAF unlock** at runtime via `PhiSilicaLafHelper.cs`. The token/attestation are baked at build time by the `GeneratePhiSilicaLafCredentials` MSBuild target into `PhiSilicaLafCredentials.g.cs` — local **dev defaults** live in `src/PhiSilicaLaf.props` (imported from `Directory.Build.props`) and the **production secret** is injected via `/p:` in the release pipeline. - New **`AdvancedPaste.dev.manifest` / `AdvancedPaste.prod.manifest`** application manifests (selected by `CIBuild`) declaring full-trust and the system AI models capability. ## Build & pipeline - **Windows App SDK** moved to the coherent **stable `2.2.0`** line and **added `Microsoft.WindowsAppSDK.AI` `2.2.3`** (the Phi Silica APIs). Foundation `2.1.0` carries the sparse-identity PRI fix, and the stable AI build matches the OS Windows AI runtime. - **Independent versioning** for Advanced Paste (`src/modules/AdvancedPaste/custom.props`, XES one-store versioning, `AdvancedPasteVersion`). A `steps-setup-versioning.yml` step is added for Advanced Paste in `job-build-project.yml`, ordered **before** CmdPal to avoid a version-collision installer failure (WIX0103). - `release.yml` passes `PhiSilicaLafToken`/`PhiSilicaLafAttestation` into the main build; spell-check allow-list/patterns updated. - Removed now-unneeded dependencies: the `Microsoft.Windows.Compatibility` reference and the `Common.UI` "force matching DLL versions" hack. <img width="1073" height="716" alt="image" src="https://github.com/user-attachments/assets/9364d86a-c0d1-4a08-a669-d98cbcb4b140" /> <img width="1038" height="308" alt="image" src="https://github.com/user-attachments/assets/00eef1dd-b407-4841-bd64-f54f8a145c46" /> <img width="194" height="405" alt="image" src="https://github.com/user-attachments/assets/d2f9c5bb-2507-4112-b3e0-56da681f88ea" /> <img width="489" height="453" alt="image" src="https://github.com/user-attachments/assets/811d1afd-9993-48be-824e-82bb56c5ceca" /> [Video clip internal](https://onedrive.cloud.microsoft/:v:/a@9n6nl3fp/S/cQpvAHrL5M9ZR6eUawztyfyBEgUCwCF-aKg9TbKyyGWP4c0KMA) [Build internal](https://microsoft.visualstudio.com/Dart/_build/results?buildId=149920754&view=artifacts&pathAsName=false&type=publishedArtifacts) --------- Co-authored-by: Niels Laute <niels.laute@live.nl> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-08-07 22:43:59 -07:00
Advanced Paste is an unpackaged, self-contained WinUI 3 app (`PowerToys.AdvancedPaste.exe`). To call Windows AI APIs (Phi Silica / `Microsoft.Windows.AI.Text.LanguageModel`) it acquires **package identity** at runtime via a shared sparse MSIX package (`Microsoft.PowerToys.SparseApp`).
### Running and attaching the debugger
1. Set the **Runner** project (`src/runner`) as the startup project in Visual Studio.
2. Launch the Runner (F5). This starts the PowerToys tray icon and loads all module interfaces.
3. Open Settings (right-click tray icon → Settings) and enable the **Advanced Paste** module if it isn't already. The module launches `PowerToys.AdvancedPaste.exe` in the background immediately.
4. In Visual Studio, go to **Debug → Attach to Process** (`Ctrl+Alt+P`) and attach to `PowerToys.AdvancedPaste.exe` (select **Managed (.NET Core)** debugger).
Alternatively, use the VS Code launch configuration **"Run AdvancedPaste"** from [.vscode/launch.json](/.vscode/launch.json) to launch the exe directly — but note that without the Runner, IPC and hotkeys won't work.
### Sparse package identity (local development)
#### Why is this needed?
- The `LanguageModel` API requires a Limited Access Feature (LAF) unlock, which only succeeds when the calling process has a matching package identity.
- Advanced Paste is an unpackaged, self-contained WinUI 3 app. The sparse package grants it identity without converting it to a full MSIX.
- The csproj uses `<ProjectPriFileName>PowerToys.AdvancedPaste.pri</ProjectPriFileName>` (matching the convention of other WinUI3 apps like ImageResizer). This requires WindowsAppSDK Foundation >= 2.0.22 ([PR #6376](https://github.com/microsoft/WindowsAppSDK/pull/6376)) which fixes MRT PRI lookup under sparse identity so `Application.LoadComponent` resolves custom-named PRI files instead of hard-coding `resources.pri`.
#### One-step dev setup
```powershell
pwsh src/PackageIdentity/BuildSparsePackage.ps1 -Platform ARM64 -Configuration Debug -DevRegister
```
`-DevRegister`:
1. Generates a dev certificate under `src/PackageIdentity/.user/` (first run only).
2. Auto-imports that certificate into `CurrentUser\TrustedPeople` and `CurrentUser\Root` so the OS grants sparse identity to AP (without trust, `GetPackageFamilyName` returns `APPMODEL_ERROR_NO_PACKAGE` and LAF unlock silently fails).
3. Removes any prior registration.
4. Rewrites the publisher in a temp copy of `AppxManifest.xml` to match the dev cert subject.
5. Registers via `Add-AppxPackage -Register … -ExternalLocation X:\…\<Platform>\<Config>\WinUI3Apps`.
After registration verify:
```powershell
$pkg = Get-AppxPackage -Name '*SparseApp*'
$pkg.PackageFamilyName # Microsoft.PowerToys.SparseApp_<PublisherId>
$pkg.PublisherId # djwsxzxb4ksa8
$pkg.IsDevelopmentMode # True
```
Confirm AP picks up sparse identity at runtime:
```powershell
& 'ARM64\Debug\WinUI3Apps\PowerToys.AdvancedPaste.exe' --check-phi-silica
# Exit 0 = Available, 1 = NotReady, 2 = NotSupported
```
Re-register after rebuilding AP, changing `src/PackageIdentity/AppxManifest.xml`, or switching platforms/configurations by re-running the same command. Unregister with `-Unregister`.
#### Troubleshooting
| Problem | Cause | Fix |
|---------|-------|-----|
| `GetPackageFamilyName` returns `APPMODEL_ERROR_NO_PACKAGE` (15700) at runtime; LAF unlock returns `Unavailable` | Dev certificate not trusted (or sparse package not registered) | Re-run `BuildSparsePackage.ps1 -DevRegister` — auto-imports the cert into `TrustedPeople` and `Root`. |
| `Microsoft.UI.Xaml.dll` crash with `0xC000027B` (class-not-registered) on AP or Settings startup | `<Application>` `Executable` path in `src/PackageIdentity/AppxManifest.xml` does not resolve under the registered `ExternalLocation` (`<Config>\WinUI3Apps\`) | Confirm every `Executable` is relative to `WinUI3Apps\` (per #47177) and the file exists under the build output. |
| AP launches but never shows a window when triggered via hotkey | Runner's pipe-server wait timed out before AP's cold-start finished bootstrapping WinAppSDK + DI host | Already mitigated by the 15 s pipe timeout in `AdvancedPasteProcessManager.cpp`; warm-start launches connect in well under 1 s. |
| `XamlParseException` / `ms-appx:///Microsoft.UI.Xaml/Themes/…` not found | WindowsAppSDK Foundation < 2.0.22; MRT can't resolve custom PRI name under sparse identity | Ensure `Microsoft.WindowsAppSDK.Foundation` >= 2.0.22 in `Directory.Packages.props`. |
### How Settings UI checks Phi Silica availability
Settings UI does not have sparse package identity. To check whether Phi Silica is available, it launches Advanced Paste as a short-lived subprocess:
```
PowerToys.AdvancedPaste.exe --check-phi-silica
```
`Program.Main` recognizes this flag, calls `PhiSilicaLafHelper.TryUnlock()` + `LanguageModel.GetReadyState()`, prints one of `Available` / `NotReady` / `NotSupported` to stdout, and exits with the matching code (0/1/2). Settings reads stdout with a 10 s wait. Because each call is a fresh process, transient `Unavailable` results are not cached across checks.
### See also
- [Phi Silica local testing & troubleshooting guide](advancedpaste-phisilica-local-testing.md) — layer-by-layer diagnostics for Phi Silica availability
- [`src/PackageIdentity/readme.md`](/src/PackageIdentity/readme.md) — full sparse package documentation
- [microsoft/microsoft-ui-xaml#10856](https://github.com/microsoft/microsoft-ui-xaml/issues/10856) — original WinUI sparse-identity PRI bug
- [microsoft/WindowsAppSDK#6376](https://github.com/microsoft/WindowsAppSDK/pull/6376) — MRT sparse PRI fix (Foundation >= 2.0.22)
## Settings
docs(paste): add AI preview credit documentation (#45236) docs(paste): add AI preview credit documentation ```markdown ## Summary of the Pull Request Adds documentation clarifying that the "Show preview" setting for Paste with AI does not consume additional AI credits. The preview displays the same AI response that was already generated from a single API call, cached locally. ## PR Checklist - [x] Closes: #32950 - [x] **Communication:** I've discussed this with core contributors already. If the work hasn't been agreed, this work might be rejected - [ ] **Tests:** Added/updated and all pass - N/A (documentation only) - [ ] **Localization:** All end-user-facing strings can be localized - N/A (dev docs only) - [x] **Dev docs:** Added/updated - [ ] **Documentation updated:** If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/windows-uwp/tree/docs/hub/powertoys) and link it here: #xxx ## Detailed Description of the Pull Request / Additional comments This PR addresses the question raised in issue #32950 about whether enabling preview for Paste with AI costs extra AI quota. Changes to `doc/devdocs/modules/advancedpaste.md`: - Added new "Paste with AI Preview" section explaining: - The `ShowCustomPreview` setting behavior - Confirmation that preview does **not** consume additional AI credits - The implementation flow showing a single API call with local caching - Reference to `OptionsViewModel.cs` lines 702-717 - Added settings documentation table for `ShowCustomPreview` Fixes #32950 ## Validation Steps Performed - Verified documentation renders correctly in Markdown preview - Confirmed technical accuracy by referencing `OptionsViewModel.cs` implementation ``` --------- Co-authored-by: yeelam-gordon <yeelam-gordon@users.noreply.github.com>
2026-01-31 09:03:24 -08:00
| Setting | Description |
|---------|-------------|
| `ShowCustomPreview` | When enabled, shows AI-generated results in a preview window before pasting. Does not affect AI credit consumption. |
## Future Improvements
TODO: Add potential future improvements