Mirrors/talemate
1
0
Fork 0
mirror of https://github.com/vegu-ai/talemate.git synced 2026-09-03 04:30:14 +02:00
Code Issues Packages Projects Releases Wiki Activity
Files
main
talemate/docs/user-guide/node-editor/reference/nodes/agents-visual-generation.md

227 lines
9.5 KiB
Markdown
Raw Permalink Normal View History

0.39.0 (#274) - **Help Agent** — A new agent that answers questions about Talemate itself — settings, agents, clients, the world editor — grounded in the bundled documentation, and able to change agent and application settings on request. - **Timeline Rollback** — Scrub a slider across a scene's automatic version history, preview the message history at any revision, then roll the scene back in place or fork the revision into a new save. - **Pi Bridge Client** — A new client type that drives generations through the pi coding agent's headless RPC mode, making any model pi can reach usable in Talemate. - **Application Settings** — Settings move out of their modal into an always-available Settings tab, with a single topic-grouped sidebar, a settings search, all API keys on one page, and unsaved-change tracking. - **Scene Library** — The home screen is rebuilt as a full-page landing view with a file-tree scene library, cover thumbnails and per-save metadata, scene and save deletion, and a drag-and-drop import dropzone. - **Fast Character Creation** — Generate a character in a single request instead of one prompt per aspect, with a Consolidate multi-select, a token budget, and a Fill in misses pass. Off by default. - **Scene Backdrop** — Set any scene illustration as a backdrop filling the scene view behind the messages, saved with the scene, with configurable text legibility and an Immersive quick-toggle. - **Scene Visual Manager** — World Editor → Scene gains a Visuals tab to browse, upload, generate and delete the scene's backgrounds and illustrations, and set the cover image and backdrop. - **Visual Prompt Finalization** — Post-processing actions — match and replace, or a free-form AI instruction — rewrite image prompts right before they reach the image backend, with per-scene and per-character overrides and reusable template sets. - fixes #271 - fixes #275 Plus 25 improvements and 39 bug fixes — see [CHANGELOG.md](https://github.com/vegu-ai/talemate/blob/prep-0.39.0/CHANGELOG.md#0390) for the full list.
2026-08-29 13:21:22 +03:00
# Visualizer Agent Generation Nodes
Run visual generation: build generation requests, select the backend, generate images and analyze them. Prompts come from the [prompt nodes](agents-visual.md).
<!-- glossary:generated - everything below this line is generated by `python -m talemate.game.engine.nodes.tools glossary --write` - do not edit -->
_7 nodes._
| Node | Registry path |
| --- | --- |
| [Analyze Images](#analyze-images) | `agents/visual/AnalyzeImages` |
| [Backend Status](#backend-status) | `agents/visual/BackendStatus` |
| [Generate Image](#generate-image) | `agents/visual/GenerateImage` |
| [Visual Generation Request](#visual-generation-request) | `agents/visual/GenerationRequest` |
| [Select Backend](#select-backend) | `agents/visual/SelectBackend` |
| [Unpack Visual Generation Request](#unpack-visual-generation-request) | `agents/visual/UnpackGenerationRequest` |
| [Unpack Visual Generation Response](#unpack-visual-generation-response) | `agents/visual/UnpackGenerationResponse` |
## Analyze Images
`agents/visual/AnalyzeImages`
Analyzes images in batches using asyncio.Semaphore to limit concurrent requests.
**Inputs**
| Input | Type | Description |
| --- | --- | --- |
| `state` | `any` | graph state (required) |
| `asset_ids` | `list` | list of asset IDs to analyze (required) |
| `missing_only` | `bool` | (optional) only analyze assets without existing analysis (optional, default True) |
| `prompt` | `str` | (optional) analysis prompt to use (optional, default "Describe this image in detail.") |
| `save` | `bool` | (optional) whether to save analysis to asset meta (optional, default True) |
**Outputs**
| Output | Type | Description |
| --- | --- | --- |
| `state` | `any` | graph state (passed through) |
| `asset_ids` | `list` | original list of asset IDs (passed through) |
| `missing_only` | `bool` | missing_only flag (passed through) |
| `prompt` | `str` | analysis prompt used (passed through) |
| `save` | `bool` | save flag (passed through) |
| `analyzed_ids` | `list` | list of successfully analyzed asset IDs |
| `skipped_ids` | `list` | list of skipped asset IDs (missing assets or already analyzed) |
| `failed_ids` | `list` | list of asset IDs that failed to analyze |
**Properties**
| Property | Type | Default | Description |
| --- | --- | --- | --- |
| `missing_only` | `bool` | `True` | Only analyze assets that don't have an existing analysis |
| `prompt` | `text` | `"Describe this image in detail. (3 paragraphs max.)"` | The prompt to use for image analysis |
| `save` | `bool` | `True` | Whether to save the analysis to asset meta |
## Backend Status
`agents/visual/BackendStatus`
Reports the capabilities of the visual agent's currently configured
backends.
**Outputs**
| Output | Type | Description |
| --- | --- | --- |
| `can_generate_images` | `bool` | Whether an image generation backend is available |
| `can_edit_images` | `bool` | Whether an image editing backend is available |
| `max_references` | `int` | Maximum number of reference images the image-edit backend supports (0 if none is available) |
## Generate Image
`agents/visual/GenerateImage`
Generates an image by submitting a generation request to the visual
agent, which routes it to the appropriate backend. The request's
callback (if any) is invoked with the response, and depending on the
request's asset attachment context the resulting image may be saved
to the scene's assets.
**Inputs**
| Input | Type | Description |
| --- | --- | --- |
| `state` | `any` | The graph state |
| `generation_request` | `visual/generation_request` | The generation request to execute |
**Outputs**
| Output | Type | Description |
| --- | --- | --- |
| `state` | `any` | The state input, passed through |
| `generation_request` | `visual/generation_request` | The generation request, passed through |
| `generation_response` | `visual/generation_response` | The generation response containing the image |
## Visual Generation Request
`agents/visual/GenerationRequest`
Creates a generation request for image generation.
**Inputs**
| Input | Type | Description |
| --- | --- | --- |
| `prompt` | `visual/prompt` | visual prompt object (required) |
| `vis_type` | `str` | type of visual to generate (optional) |
| `gen_type` | `str` | type of generation (TEXT_TO_IMAGE, etc.) (optional) |
| `format` | `str` | image format/aspect ratio (optional) |
| `instructions` | `str` | additional instructions for generation (optional) |
| `character_name` | `str` | name of character for character-specific generation (optional) |
| `reference_assets` | `list` | list of reference asset IDs (optional) |
| `callback` | `function` | callback function to run after generation (optional) |
| `extra_config` | `dict` | additional configuration dict (optional) |
| `asset_attachment_context` | `asset_attachment_context` | controls automatic asset attachment behavior (optional) |
**Outputs**
| Output | Type | Description |
| --- | --- | --- |
| `generation_request` | `visual/generation_request` | the created generation request object |
| `prompt` | `visual/prompt` | the visual prompt (passed through) |
| `vis_type` | `str` | visual type (passed through) |
| `format` | `str` | format type (passed through) |
| `character_name` | `str` | character name (passed through) |
| `reference_assets` | `list` | reference assets list (passed through) |
| `gen_type` | `str` | generation type (passed through) |
| `extra_config` | `dict` | extra config dict (passed through) |
**Properties**
| Property | Type | Default | Description |
| --- | --- | --- | --- |
| `vis_type` | `str` | `"UNSPECIFIED"` | The type of visual to generate. Choices: `UNSPECIFIED`, `CHARACTER_CARD`, `CHARACTER_PORTRAIT`, `SCENE_CARD`, `SCENE_BACKGROUND`, `SCENE_ILLUSTRATION`, `OBJECT_ILLUSTRATION`. |
| `gen_type` | `str` | `"TEXT_TO_IMAGE"` | The type of generation to perform. Choices: `TEXT_TO_IMAGE`, `IMAGE_EDIT`, `UPLOAD`. |
| `format` | `str` | `"LANDSCAPE"` | The format of the visual to generate. Choices: `LANDSCAPE`, `PORTRAIT`, `SQUARE`. |
| `character_name` | `str` | `""` | The name of the character to generate |
| `instructions` | `text` | `""` | The instructions for the generation request |
| `extra_config` | `dict` | `{}` | The extra configuration for the generation request |
## Select Backend
`agents/visual/SelectBackend`
Determines which visual backend and generation type to use for a request.
Selects the image-edit backend (gen_type IMAGE_EDIT) when reference assets
are provided and image editing is available, or when image generation is
unavailable but editing is; otherwise selects the image generation backend
(gen_type TEXT_TO_IMAGE). Also resolves the prompt type the selected
backend expects and the image format implied by the visual type.
**Inputs**
| Input | Type | Description |
| --- | --- | --- |
| `state` | `any` | Graph state |
| `vis_type` | `str` | (optional) The type of visual to generate |
| `reference_assets` | `list` | (optional) List of reference asset IDs; when set, steers selection toward the image-edit backend |
**Outputs**
| Output | Type | Description |
| --- | --- | --- |
| `state` | `any` | Graph state (passed through) |
| `backend_name` | `str` | Name of the selected backend (empty if none is available) |
| `gen_type` | `str` | The selected generation type (TEXT_TO_IMAGE or IMAGE_EDIT) |
| `vis_type` | `str` | The visual type (passed through) |
| `prompt_type` | `str` | Prompt type the selected backend expects (falls back to the agent's fallback prompt type when no backend is available) |
| `format` | `str` | Image format derived from the visual type (e.g. PORTRAIT) |
| `reference_assets` | `list` | The reference assets list (passed through) |
**Properties**
| Property | Type | Default | Description |
| --- | --- | --- | --- |
| `vis_type` | `str` | `"UNSPECIFIED"` | The type of visual to generate. Choices: `UNSPECIFIED`, `CHARACTER_CARD`, `CHARACTER_PORTRAIT`, `SCENE_CARD`, `SCENE_BACKGROUND`, `SCENE_ILLUSTRATION`, `OBJECT_ILLUSTRATION`. |
## Unpack Visual Generation Request
`agents/visual/UnpackGenerationRequest`
Unpacks a visual generation request into its individual fields.
**Inputs**
| Input | Type | Description |
| --- | --- | --- |
| `generation_request` | `visual/generation_request` | The generation request to unpack |
**Outputs**
| Output | Type | Description |
| --- | --- | --- |
| `generation_request` | `visual/generation_request` | The generation request, passed through |
| `prompt` | `str` | The positive prompt string |
| `vis_type` | `str` | The visual type |
| `format` | `str` | The image format |
| `character_name` | `str` | The character name |
| `reference_assets` | `list` | The list of reference asset IDs |
| `gen_type` | `str` | The generation type |
| `extra_config` | `dict` | The extra configuration dict |
| `asset_attachment_context` | `asset_attachment_context` | The asset attachment context |
## Unpack Visual Generation Response
`agents/visual/UnpackGenerationResponse`
Unpacks a visual generation response into its individual fields.
**Inputs**
| Input | Type | Description |
| --- | --- | --- |
| `generation_response` | `visual/generation_response` | The generation response to unpack |
**Outputs**
| Output | Type | Description |
| --- | --- | --- |
| `generation_response` | `visual/generation_response` | The generation response, passed through |
| `base64` | `str` | The generated image as base64 encoded data |
| `image_data` | `str` | The generated image as a data URI |
| `id` | `str` | The generation's ID |
| `backend_name` | `str` | The name of the backend that generated the image |
| `request` | `visual/generation_request` | The generation request that produced this response |
Reference in New Issue Copy Permalink