diff --git a/CHANGELOG.yaml b/CHANGELOG.yaml index 4f280ca9..a1e3fa98 100644 --- a/CHANGELOG.yaml +++ b/CHANGELOG.yaml @@ -11,6 +11,7 @@ - "Client Auto Retry: Clients can now automatically retry on response issues before you are notified — separate 0–5 retry sliders for empty responses, API rate limiting (HTTP 429, with progressively longer waits between attempts), and reasoning models that skip their reasoning tokens. A notification shows the retry progress with an abort option, and when retries run out (or a slider is at 0, the default) the usual generation error dialog appears." - "New Node Graph Events: Node graphs can now hook into more of this release's features. `agent.visual.prompt_finalize.before` / `.after` fire around visual prompt finalization (mutable prompts, and the `.before` finalizer list can be modified — they fire even with the agent setting disabled, so a graph can act as its own finalizer). `agent.creator.dialogue_examples.before` / `.after` fire around character example dialogue generation (inject instructions, or rewrite the generated examples). `agent.help.chat.before` / `.after` fire around help chat responses. Scene asset mutations gained `asset_deleted`, `scene.backdrop_changed`, `scene.cover_image_changed` and `character.cover_image_changed` alongside the existing `asset_saved`. All are documented in the node editor's Events reference." - "Simplified Character Card Import: The character card import dialog gained an AI Generation section with individual toggles for each generation step — content context, description rewrite, attribute extraction, dialogue instructions, example dialogue, and story intent — plus Full/Minimal preset buttons to flip them all at once. Disabled steps fall back to the card's original data (the description and example dialogue import verbatim), so a minimal import needs no text generation at all and completes in a fraction of the time." + - "Scene Browser Landing Page: The home screen was rebuilt into a full-page landing view, replacing the sidebar load panel. A file-tree Scene Library lists scene projects with cover thumbnails, save counts and per-save metadata, plus an asset and node-module summary row per expanded project; long save lists collapse behind a 'Show all' row and the list is filterable with click-to-load. Individual scene files and character cards can be deleted from the tree, and an entire scene project (saves, assets, nodes, history) can be deleted behind a type-the-project-name confirmation. Character cards display their card image and only actual card files are listed. Scene and character card import moved to a prominent drag-and-drop dropzone next to a Create-new-scene card, with the quick-load recent scene cards kept on top." improvements: - "Uniform Settings Framework: Agent settings and client settings now share one field-definition schema on the backend and one field renderer on the frontend. Client-specific settings gain the full widget set previously exclusive to agents (sliders, autocompletes, selects with rich choices, per-value notes) plus conditional visibility, and choice lists are delivered to the frontend in a single normalized shape." - "MistralAI Client: Added a Concurrent Inference toggle so batch operations can dispatch multiple requests in parallel. Off by default." diff --git a/docs/getting-started/load-a-scene.md b/docs/getting-started/load-a-scene.md index fae19d85..e978e5ba 100644 --- a/docs/getting-started/load-a-scene.md +++ b/docs/getting-started/load-a-scene.md @@ -1,10 +1,10 @@ # Load a scenario -Once you've set up a client and assigned it to all the agents, you will be presented with the `Home` screen. From here, you can load talemate scenarios and upload character cards. +Once you've set up a client and assigned it to all the agents, you will be presented with the `Home` screen. From here, you can load talemate scenarios, browse the scene library, and import scenes or character cards — see [Home Screen & Scene Library](/talemate/user-guide/scene-browser) for a full tour. -To load the introductory `Infinity Quest` scenario, simply click on its entry in the `Quick Load` section. +To load the introductory `Infinity Quest` scenario, simply click on its entry in the `Quick load` section. -![Load infinity quest](/talemate/img/0.26.0/getting-started-load-screen.png) +![Load infinity quest](/talemate/img/0.39.0/getting-started-load-screen.png) !!! info "First time may take a moment" When you load the a scenario for the first time, Talemate will need to initialize the long term memory model. Which likely means a download. Just be patient and it will be ready soon. diff --git a/docs/img/0.26.0/getting-started-load-screen.png b/docs/img/0.26.0/getting-started-load-screen.png deleted file mode 100644 index 3e5f2881..00000000 Binary files a/docs/img/0.26.0/getting-started-load-screen.png and /dev/null differ diff --git a/docs/img/0.34.0/character-card-4.png b/docs/img/0.34.0/character-card-4.png deleted file mode 100644 index bfca5512..00000000 Binary files a/docs/img/0.34.0/character-card-4.png and /dev/null differ diff --git a/docs/img/0.37.0/model-testing-harness-load.png b/docs/img/0.37.0/model-testing-harness-load.png deleted file mode 100644 index 23f1b81e..00000000 Binary files a/docs/img/0.37.0/model-testing-harness-load.png and /dev/null differ diff --git a/docs/img/0.39.0/character-card-4.png b/docs/img/0.39.0/character-card-4.png new file mode 100644 index 00000000..545bbae8 Binary files /dev/null and b/docs/img/0.39.0/character-card-4.png differ diff --git a/docs/img/0.39.0/getting-started-load-screen.png b/docs/img/0.39.0/getting-started-load-screen.png new file mode 100644 index 00000000..16ef789a Binary files /dev/null and b/docs/img/0.39.0/getting-started-load-screen.png differ diff --git a/docs/img/0.39.0/model-testing-harness-load.png b/docs/img/0.39.0/model-testing-harness-load.png new file mode 100644 index 00000000..61db053a Binary files /dev/null and b/docs/img/0.39.0/model-testing-harness-load.png differ diff --git a/docs/img/0.39.0/scene-browser-delete-project.png b/docs/img/0.39.0/scene-browser-delete-project.png new file mode 100644 index 00000000..2d34f6fb Binary files /dev/null and b/docs/img/0.39.0/scene-browser-delete-project.png differ diff --git a/docs/img/0.39.0/scene-browser-library.png b/docs/img/0.39.0/scene-browser-library.png new file mode 100644 index 00000000..52b3a8d4 Binary files /dev/null and b/docs/img/0.39.0/scene-browser-library.png differ diff --git a/docs/user-guide/character-card-import.md b/docs/user-guide/character-card-import.md index f9a4f4f8..142f4e68 100644 --- a/docs/user-guide/character-card-import.md +++ b/docs/user-guide/character-card-import.md @@ -22,13 +22,15 @@ Character cards can be provided as: ## Starting an Import -Character card import is initiated from the **Load Scene** panel in the left sidebar on the home screen: +Character card import is initiated from the **Import** dropzone on the [home screen](/talemate/user-guide/scene-browser): -![Drag and Drop Upload Area](/talemate/img/0.34.0/character-card-4.png) +![Drag and Drop Upload Area](/talemate/img/0.39.0/character-card-4.png) -1. Drag and drop a character card file (image or JSON), or click to browse +1. Drag and drop a character card file (image or JSON) onto the dropzone, or click it to browse 2. If the file is detected as a character card, the **Character Card Import Options** dialog will automatically open +Cards already stored in your `scenes/characters/` directory can also be imported by clicking them in the **Character Cards** section of the [Scene Library](/talemate/user-guide/scene-browser#character-cards). + ![Character Card Import Dialog](/talemate/img/0.34.0/character-card-1.png) The import dialog will automatically analyze the character card and display detected information. diff --git a/docs/user-guide/clients/model-testing-harness.md b/docs/user-guide/clients/model-testing-harness.md index 3624959a..27345693 100644 --- a/docs/user-guide/clients/model-testing-harness.md +++ b/docs/user-guide/clients/model-testing-harness.md @@ -17,11 +17,10 @@ Passing every test does not guarantee good creative output, but any failure is a The harness is a normal Talemate scene and loads the same way as any other — see [Load a scenario](/talemate/getting-started/load-a-scene/) for the general flow. -1. From the **Home** screen, start typing `Model Testing Harness` into the **Search scenes** field. -2. Select the matching entry from the autocomplete. -3. Click **Load**. +1. From the **Home** screen, type `model` into the **Filter scenes** field of the [Scene Library](/talemate/user-guide/scene-browser). +2. Click the `model-testing-harness.json` save file in the **model-testing-harness** project. -![Searching for the Model Testing Harness from the Home screen](/talemate/img/0.37.0/model-testing-harness-load.png) +![Searching for the Model Testing Harness from the Home screen](/talemate/img/0.39.0/model-testing-harness-load.png) The tests start automatically as soon as the scene finishes loading — there is no separate "run" button to press. diff --git a/docs/user-guide/scene-browser.md b/docs/user-guide/scene-browser.md new file mode 100644 index 00000000..7146b960 --- /dev/null +++ b/docs/user-guide/scene-browser.md @@ -0,0 +1,73 @@ +# Home Screen & Scene Library + +The :material-home: **Home** screen is Talemate's landing page. It is where you load, import, create, and manage your scenes. + +![Home screen](/talemate/img/0.39.0/getting-started-load-screen.png) + +It is made up of four areas: + +- **Quick load** — cards for your most recently played scenes +- **Scene Library** — a file browser of every scene project on disk +- **Import** — drag-and-drop import for scenes and character cards +- **Create new scene** — start a blank scene + +## Quick load + +The **Quick load** row shows your most recently saved scenes as cards. Click a card to load that scene. + +Each card has a three-dot menu (⋮) with additional actions: + +- :material-history: **Timeline** — browse and restore the scene's version history, see [Restoring Scenes](/talemate/user-guide/restoring-scenes) +- **Remove from Quick Load** — remove the card without deleting any files +- **Delete** — delete the scene file itself + +## Scene Library + +The **Scene Library** lists every scene project found in your `scenes/` directory as an expandable tree. Each project row shows its cover image, the number of save files it contains, and when it was last saved. + +![Scene library](/talemate/img/0.39.0/scene-browser-library.png) + +Expanding a project reveals: + +- An information row summarizing the project's contents — how many media assets and [node modules](/talemate/user-guide/node-editor/) it holds +- The project's save files, each with its scene name, last-modified date, and file size + +Click a save file to load it. + +!!! tip "Filtering" + The **Filter scenes** field narrows the tree to matching projects, save files, and character cards. Matching projects expand automatically so you can see the hits. + +Projects with many save files show only the ten most recent — click **Show all N saves** to expand the rest. + +### Character cards + +Character card files stored in `scenes/characters/` are listed in their own **Character Cards** section at the bottom of the library. Image cards display a thumbnail of the card art. + +Clicking a card starts a [character card import](/talemate/user-guide/character-card-import), letting you create a new scene from the card. + +### Deleting scenes and projects + +Each row in the library has a delete action: + +- :material-file-remove-outline: on a **save file** or **character card** deletes that single file after a confirmation prompt. +- :material-folder-remove-outline: on a **project** deletes the entire scene project — all of its save files, assets, node modules, and version history. + +Deleting a project is irreversible, so the confirmation dialog requires typing the project name before the delete button becomes available: + +![Delete scene project dialog](/talemate/img/0.39.0/scene-browser-delete-project.png) + +!!! warning + Deleting a scene project removes its whole directory from disk — see [Scene Directory Structure](/talemate/user-guide/scene-directory) for what that includes. There is no undo. + +## Import + +The **Import** dropzone accepts: + +- **Talemate scenes** — `.json` save files, or `.zip` archives [exported from the world editor](/talemate/user-guide/world-editor/scene/export) (a complete scene including its assets and node modules) +- **Character cards** — `.png`, `.webp`, or `.json` card files + +Drag a file onto the dropzone or click it to browse. Character cards open the [import options dialog](/talemate/user-guide/character-card-import); Talemate scene files load directly. + +## Create new scene + +**Create new scene** starts a blank scene. After choosing a name and optional writing style and director persona, the scene opens in the [world editor](/talemate/user-guide/world-editor/) where you can add characters and scene details. diff --git a/docs/user-guide/scene-directory.md b/docs/user-guide/scene-directory.md index b7ebbef3..7aed9df8 100644 --- a/docs/user-guide/scene-directory.md +++ b/docs/user-guide/scene-directory.md @@ -71,3 +71,5 @@ Understanding the directory structure is helpful for: - **Manual backups** — you can copy the entire project directory to back up a scene and all its associated data - **Sharing scenes** — the project directory contains everything needed to share a scene with others - **Understanding saves** — all saves for a scene are grouped together in one directory, making it clear which files belong together + +Scene projects and their save files can be browsed — and deleted — from the [Scene Library](/talemate/user-guide/scene-browser) on the home screen. diff --git a/src/talemate/agents/help/docs-index.yaml b/src/talemate/agents/help/docs-index.yaml index bf1e224b..aed6d9ef 100644 --- a/src/talemate/agents/help/docs-index.yaml +++ b/src/talemate/agents/help/docs-index.yaml @@ -377,9 +377,10 @@ indicator, and using {{ system_prompt }} to append to the default prompt. - path: user-guide/character-card-import.md title: Character Card Import - summary: How to import TavernAI-style character cards (Chara Card V0-V3, PNG/JPG/WebP images or JSON) from the Load Scene - panel to create a new scene. Covers the import dialog, character detection/manual selection, options for character book - entries, alternate greetings, shared context setup, writing style template, player character setup, and troubleshooting + summary: How to import TavernAI-style character cards (Chara Card V0-V3, PNG/JPG/WebP images or JSON) from the home screen's + Import dropzone or the Scene Library's Character Cards section to create a new scene. Covers the import dialog, character + detection/manual selection, options for character book entries, alternate greetings, shared context setup, writing style + template, player character setup, and troubleshooting failed analysis. - path: user-guide/clients/auto-retry.md title: Auto Retry @@ -745,6 +746,12 @@ summary: The scene tools toolbar and agent activity bar. Covers actor actions, director actions, narrator actions, advancing time, world state actions, creative tools (e.g. taking characters in/out of the scene, introducing new characters with optional example dialogue generation and guidance), the visualizer menu, and saving, with Ctrl/Cmd modifier click behavior. +- path: user-guide/scene-browser.md + title: Home Screen & Scene Library + summary: 'The home screen landing page: Quick load recent-scene cards, the Scene Library file tree (projects with cover + thumbnails, save files, asset/node-module counts, filtering, Show all saves), deleting save files, character cards, and + entire scene projects (type-name confirmation), the Import dropzone for scenes and character cards, and creating a new + blank scene.' - path: user-guide/scene-directory.md title: Scene Directory Structure summary: Layout of a scene's project directory under scenes/, save .json files, and the assets/, changelog/, backups/, nodes/, diff --git a/src/talemate/files.py b/src/talemate/files.py index 6efbe9fd..4cd8794b 100644 --- a/src/talemate/files.py +++ b/src/talemate/files.py @@ -1,6 +1,29 @@ +import datetime import fnmatch +import json import os +from talemate.path import SCENES_DIR + +# top-level directories inside scenes/ that are not scene projects +RESERVED_SCENES_DIRS = {"assets", "characters"} + +# directories inside a scene project that never contain loadable saves +EXCLUDED_PROJECT_SUBDIRS = {"nodes", "changelog", "assets", "info"} + +MEDIA_TYPES = { + "png": "image/png", + "webp": "image/webp", + "jpg": "image/jpeg", + "jpeg": "image/jpeg", + "json": "application/json", +} + +CHARACTER_CARD_SPECS = {"chara_card_v1", "chara_card_v2", "chara_card_v3"} + +# scene save file metadata cache: path -> (mtime, scene_name, cover_asset_id) +_SCENE_META_CACHE: dict[str, tuple[float, str | None, str | None]] = {} + def list_scenes_directory(path: str = ".", list_images: bool = True) -> list: """ @@ -8,10 +31,8 @@ def list_scenes_directory(path: str = ".", list_images: bool = True) -> list: :param directory: Directory to list scene files from. :return: List of scene files in the given directory. """ - current_dir = os.getcwd() - scenes = _list_files_and_directories( - os.path.join(current_dir, "scenes"), path, list_images=list_images + scenes_directory(), path, list_images=list_images ) return scenes @@ -57,3 +78,231 @@ def _list_files_and_directories(root: str, path: str, list_images: bool = True) break return items + + +def scenes_directory() -> str: + """ + Absolute path to the scenes directory. + """ + return str(SCENES_DIR) + + +def list_scenes_tree() -> dict: + """ + Structured listing of the scenes directory for the scene browser. + + Returns a dict with: + - projects: scene project directories with their save files (newest first) + - characters: character card files in scenes/characters + """ + root = scenes_directory() + + projects = [] + characters = [] + + if not os.path.isdir(root): + return {"projects": [], "characters": []} + + for entry in sorted(os.listdir(root)): + full_path = os.path.join(root, entry) + if not os.path.isdir(full_path): + continue + if entry == "characters": + characters = _list_character_cards(full_path) + continue + if entry in RESERVED_SCENES_DIRS: + continue + project = _scene_project_entry(full_path) + if project["files"]: + projects.append(project) + + projects.sort(key=lambda project: project["modified"], reverse=True) + + return {"projects": projects, "characters": characters} + + +def _file_entry(path: str, base_dir: str) -> dict: + file_stat = os.stat(path) + return { + "path": path, + "filename": os.path.basename(path), + "relpath": os.path.relpath(path, base_dir), + "size": file_stat.st_size, + "modified": datetime.datetime.fromtimestamp(file_stat.st_mtime).isoformat(), + } + + +def identify_character_card_spec(data) -> str | None: + """ + The character card spec of the given data, or None if it is not a + character card. Single source of truth for card detection - the loader's + identify_import_spec() builds on this, and the frontend mirrors it. + """ + if not isinstance(data, dict): + return None + spec = data.get("spec") + if spec in CHARACTER_CARD_SPECS: + return spec + if "first_mes" in data: + # original chara card didnt specify a spec + return "chara_card_v0" + if "first_mes" in (data.get("data") or {}): + # fallback for future chara card versions, which are supposed to be + # backwards compatible + return "chara_card_v3" + return None + + +def is_character_card_data(data) -> bool: + return identify_character_card_spec(data) is not None + + +def _list_character_cards(characters_dir: str) -> list: + # cards are stored flat in scenes/characters - subdirectories hold + # upload side artifacts (assets, changelog), never cards + cards = [] + for filename in sorted(os.listdir(characters_dir)): + path = os.path.join(characters_dir, filename) + if not os.path.isfile(path): + continue + extension = filename.rsplit(".", 1)[-1].lower() + if extension not in ("png", "webp", "json"): + continue + + # json files in the characters dir are only cards if they actually + # contain character card data (scene uploads land here too) + if extension == "json": + try: + with open(path) as file: + if not is_character_card_data(json.load(file)): + continue + except (OSError, ValueError): + continue + + entry = _file_entry(path, characters_dir) + entry["media_type"] = MEDIA_TYPES[extension] + cards.append(entry) + cards.sort(key=lambda card: card["modified"], reverse=True) + return cards + + +def _load_asset_library(project_dir: str) -> dict: + library_path = os.path.join(project_dir, "assets", "library.json") + try: + with open(library_path) as file: + return json.load(file).get("assets", {}) + except (OSError, ValueError): + return {} + + +def _resolve_cover_image( + project_dir: str, asset_id: str | None, library: dict +) -> dict | None: + if not asset_id: + return None + + asset = library.get(asset_id) + if asset: + file_type = asset.get("file_type", "png") + return { + "id": asset_id, + "file_type": file_type, + "media_type": asset.get("media_type", "image/png"), + "path": os.path.join(project_dir, "assets", f"{asset_id}.{file_type}"), + } + + # asset not in the library - probe the assets directory directly + for file_type in ("png", "webp", "jpg", "jpeg"): + asset_path = os.path.join(project_dir, "assets", f"{asset_id}.{file_type}") + if os.path.exists(asset_path): + return { + "id": asset_id, + "file_type": file_type, + "media_type": MEDIA_TYPES[file_type], + "path": asset_path, + } + + return None + + +def _scene_file_meta(file_path: str) -> tuple[str | None, str | None]: + """ + Scene name and cover asset id for a save file, cached by mtime so large + projects don't re-parse every save on each tree request. + """ + mtime = os.stat(file_path).st_mtime + cached = _SCENE_META_CACHE.get(file_path) + if cached and cached[0] == mtime: + return cached[1], cached[2] + + scene_name = None + cover_asset_id = None + try: + with open(file_path) as file: + scene_data = json.load(file) + if isinstance(scene_data, dict): + scene_name = scene_data.get("name") + cover_asset_id = (scene_data.get("assets") or {}).get("cover_image") + except (OSError, ValueError): + pass + + _SCENE_META_CACHE[file_path] = (mtime, scene_name, cover_asset_id) + return scene_name, cover_asset_id + + +def _count_project_assets(project_dir: str) -> int: + assets_dir = os.path.join(project_dir, "assets") + try: + entries = os.listdir(assets_dir) + except OSError: + return 0 + return sum( + 1 + for filename in entries + if filename != "library.json" + and os.path.isfile(os.path.join(assets_dir, filename)) + ) + + +def _count_project_nodes(project_dir: str) -> int: + count = 0 + for _, _, filenames in os.walk(os.path.join(project_dir, "nodes")): + count += sum(1 for filename in filenames if filename.endswith(".json")) + return count + + +def _scene_project_entry(project_dir: str) -> dict: + library = _load_asset_library(project_dir) + files = [] + + for dirpath, dirnames, filenames in os.walk(project_dir): + dirnames[:] = [d for d in dirnames if d not in EXCLUDED_PROJECT_SUBDIRS] + for filename in filenames: + if not filename.endswith(".json"): + continue + + file_path = os.path.join(dirpath, filename) + entry = _file_entry(file_path, project_dir) + + scene_name, cover_asset_id = _scene_file_meta(file_path) + entry["scene_name"] = scene_name + entry["cover_image"] = _resolve_cover_image( + project_dir, cover_asset_id, library + ) + files.append(entry) + + files.sort(key=lambda entry: entry["modified"], reverse=True) + + cover_image = next( + (entry["cover_image"] for entry in files if entry["cover_image"]), None + ) + + return { + "name": os.path.basename(project_dir), + "path": project_dir, + "modified": files[0]["modified"] if files else None, + "cover_image": cover_image, + "num_assets": _count_project_assets(project_dir), + "num_nodes": _count_project_nodes(project_dir), + "files": files, + } diff --git a/src/talemate/load/character_card.py b/src/talemate/load/character_card.py index 1adfffbf..8d3d1ab2 100644 --- a/src/talemate/load/character_card.py +++ b/src/talemate/load/character_card.py @@ -13,6 +13,7 @@ import talemate.instance as instance from talemate import Character, Player from talemate.character import activate_character from talemate.exceptions import UnknownDataSpec +from talemate.files import identify_character_card_spec from talemate.status import LoadingStatus from talemate.config import get_config from talemate.util import extract_metadata, select_best_texts_by_keyword, count_tokens @@ -223,24 +224,9 @@ def identify_import_spec(data: dict) -> ImportSpec: f"The file may not contain valid character card metadata." ) - if data.get("spec") == "chara_card_v3": - return ImportSpec.chara_card_v3 - - if data.get("spec") == "chara_card_v2": - return ImportSpec.chara_card_v2 - - if data.get("spec") == "chara_card_v1": - return ImportSpec.chara_card_v1 - - if "first_mes" in data: - # original chara card didnt specify a spec, - # if the first_mes key exists, we can assume it's a v0 chara card - return ImportSpec.chara_card_v0 - - if "first_mes" in data.get("data", {}): - # this can also serve as a fallback for future chara card versions - # as they are supposed to be backwards compatible - return ImportSpec.chara_card_v3 + spec = identify_character_card_spec(data) + if spec is not None: + return ImportSpec(spec) # TODO: probably should actually check for valid talemate scene data return ImportSpec.talemate diff --git a/src/talemate/scene_assets.py b/src/talemate/scene_assets.py index 761edaa8..bf0d0078 100644 --- a/src/talemate/scene_assets.py +++ b/src/talemate/scene_assets.py @@ -32,6 +32,7 @@ from talemate.agents.visual.schema import ( Resolution, AssetAttachmentContext, ) +from talemate.files import MEDIA_TYPES from talemate.path import SCENES_DIR async_signals.register( @@ -122,21 +123,11 @@ def get_media_type_from_extension(file_extension: str) -> str: Raises: ValueError: If the file extension is not supported """ - # Normalize extension to lowercase and ensure it starts with a dot - ext = file_extension.lower() - if not ext.startswith("."): - ext = f".{ext}" - - if ext == ".png": - return "image/png" - elif ext in [".jpg", ".jpeg"]: - return "image/jpeg" - elif ext == ".webp": - return "image/webp" - elif ext == ".json": - return "application/json" - else: - raise ValueError(f"Unsupported file extension: {ext}") + ext = file_extension.lower().lstrip(".") + media_type = MEDIA_TYPES.get(ext) + if media_type is None: + raise ValueError(f"Unsupported file extension: .{ext}") + return media_type def get_media_type_from_file_path(file_path: str) -> str: diff --git a/src/talemate/server/api.py b/src/talemate/server/api.py index 626c7e95..d79d4316 100644 --- a/src/talemate/server/api.py +++ b/src/talemate/server/api.py @@ -198,6 +198,8 @@ async def websocket_endpoint(websocket): query = data.get("query", "") list_images = data.get("list_images", True) handler.request_scenes_list(query, list_images) + elif action_type == "request_scenes_tree": + handler.request_scenes_tree() elif action_type == "configure_clients": await update_config({"clients": data.get("clients")}) await instance.instantiate_clients() diff --git a/src/talemate/server/config.py b/src/talemate/server/config.py index 79aa41a9..ede610cd 100644 --- a/src/talemate/server/config.py +++ b/src/talemate/server/config.py @@ -1,6 +1,7 @@ import pydantic import structlog import os +import shutil from talemate import VERSION from talemate.changelog import delete_changelog_files, scene_ref_from_path @@ -18,6 +19,7 @@ from talemate.config.schema import ( ) from talemate.config import get_config, Config, update_config from talemate.emit import emit +from talemate.files import scenes_directory, RESERVED_SCENES_DIRS from talemate.instance import emit_clients_status, get_client from .websocket_plugin import Plugin @@ -65,6 +67,10 @@ class DeleteScenePayload(pydantic.BaseModel): path: str +class DeleteSceneProjectPayload(pydantic.BaseModel): + path: str + + class SaveUnifiedAPIKeyPayload(pydantic.BaseModel): config_path: str api_key: str | None @@ -446,6 +452,11 @@ class ConfigPlugin(Plugin): async def handle_delete_scene(self, data): payload = DeleteScenePayload(**data) + file_path = os.path.realpath(payload.path) + scenes_root = os.path.realpath(scenes_directory()) + if not file_path.startswith(scenes_root + os.sep): + raise ValueError(f"Not a scene file: {payload.path}") + await self.handle_remove_scene_from_recents(data) log.info("Deleting scene", path=payload.path) @@ -488,6 +499,64 @@ class ConfigPlugin(Plugin): {"type": "app_config", "data": config.model_dump(), "version": VERSION} ) + async def handle_delete_scene_project(self, data): + payload = DeleteSceneProjectPayload(**data) + + project_path = os.path.realpath(payload.path) + scenes_root = os.path.realpath(scenes_directory()) + project_name = os.path.basename(project_path) + + # only direct children of the scenes directory are scene projects + if ( + os.path.dirname(project_path) != scenes_root + or project_name in RESERVED_SCENES_DIRS + ): + raise ValueError(f"Not a scene project directory: {payload.path}") + + if not os.path.isdir(project_path): + raise ValueError(f"Scene project not found: {payload.path}") + + prefix = project_path + os.sep + + # the next save of a loaded scene would silently recreate a skeleton + # project with broken asset references, so refuse instead + active_scene = self.scene + if ( + active_scene + and active_scene.full_path + and os.path.realpath(active_scene.full_path).startswith(prefix) + ): + raise ValueError( + f"Cannot delete the project of the currently loaded scene: {project_name}" + ) + + log.info("Deleting scene project", path=project_path) + + config: Config = get_config() + + for recent_scene in list(config.recent_scenes.scenes): + if os.path.realpath(recent_scene.path).startswith(prefix): + config.recent_scenes.scenes.remove(recent_scene) + + await config.set_dirty() + + shutil.rmtree(project_path) + + self.websocket_handler.queue_put( + { + "type": "config", + "action": "delete_scene_project_complete", + "data": { + "path": payload.path, + "name": project_name, + }, + } + ) + + self.websocket_handler.queue_put( + {"type": "app_config", "data": config.model_dump(), "version": VERSION} + ) + async def handle_set_agent_action_override(self, data): payload = SetAgentActionOverridePayload(**data["data"]) diff --git a/src/talemate/server/websocket_server.py b/src/talemate/server/websocket_server.py index 0abb4abe..a136626f 100644 --- a/src/talemate/server/websocket_server.py +++ b/src/talemate/server/websocket_server.py @@ -14,7 +14,7 @@ from talemate.config import get_config, Config from talemate.context import ActiveScene from talemate.emit import Emission, Receiver, abort_wait_for_input, emit import talemate.emit.async_signals as async_signals -from talemate.files import list_scenes_directory +from talemate.files import list_scenes_directory, list_scenes_tree from talemate.load import load_scene, SceneInitialization from talemate.scene_assets import Asset, get_media_type_from_file_path, VIS_TYPE from talemate.scene_message import versions_payload_for @@ -649,6 +649,14 @@ class WebsocketHandler(SceneAssetsBatchingMixin, Receiver): } ) + def request_scenes_tree(self): + self.queue_put( + { + "type": "scenes_tree", + "data": list_scenes_tree(), + } + ) + def request_scene_history(self): history = [archived_history for archived_history in self.scene.archived_history] diff --git a/talemate_frontend/src/components/IntroView.vue b/talemate_frontend/src/components/IntroView.vue deleted file mode 100644 index 3f679f64..00000000 --- a/talemate_frontend/src/components/IntroView.vue +++ /dev/null @@ -1,45 +0,0 @@ - - - \ No newline at end of file diff --git a/talemate_frontend/src/components/LoadScene.vue b/talemate_frontend/src/components/LoadScene.vue deleted file mode 100644 index 160d6645..00000000 --- a/talemate_frontend/src/components/LoadScene.vue +++ /dev/null @@ -1,485 +0,0 @@ - - - - - - \ No newline at end of file diff --git a/talemate_frontend/src/components/SceneBrowser.vue b/talemate_frontend/src/components/SceneBrowser.vue new file mode 100644 index 00000000..03c503af --- /dev/null +++ b/talemate_frontend/src/components/SceneBrowser.vue @@ -0,0 +1,474 @@ + + + + + diff --git a/talemate_frontend/src/components/SceneLanding.vue b/talemate_frontend/src/components/SceneLanding.vue new file mode 100644 index 00000000..bde9baae --- /dev/null +++ b/talemate_frontend/src/components/SceneLanding.vue @@ -0,0 +1,502 @@ + + + + + diff --git a/talemate_frontend/src/components/TalemateApp.vue b/talemate_frontend/src/components/TalemateApp.vue index ebf9d05d..b1a2394b 100644 --- a/talemate_frontend/src/components/TalemateApp.vue +++ b/talemate_frontend/src/components/TalemateApp.vue @@ -52,7 +52,7 @@ - + Toggle sidebar mdi-arrow-collapse-left mdi-arrow-collapse-right @@ -116,7 +116,7 @@ - + Not connected to Talemate backend

@@ -125,14 +125,6 @@ There are some outstanding configuration issues, please ensure that all enabled agents are configured correctly. - - - @@ -223,12 +215,16 @@ - @@ -399,7 +395,7 @@ ref="sceneTimeline" :scene="scene" :appearance-config="effectiveAppearanceConfig" - @load-at-revision="(target) => { resetViews(); $refs.loadScene.loadJsonSceneFromPath(target.path, false, target.rev); }" + @load-at-revision="(target) => { resetViews(); $refs.sceneLanding.loadJsonSceneFromPath(target.path, false, target.rev); }" /> empty list (os.walk on missing path) assert list_scenes_directory(list_images=True) == [] diff --git a/tests/test_scenes_tree.py b/tests/test_scenes_tree.py new file mode 100644 index 00000000..fb0d7720 --- /dev/null +++ b/tests/test_scenes_tree.py @@ -0,0 +1,340 @@ +""" +Tests for the structured scenes-tree listing used by the scene browser +landing page (``list_scenes_tree`` in ``talemate.files``). +""" + +import json +import os + +import pytest + +import talemate.files +from talemate.files import list_scenes_tree + + +def _write_json(path, data): + os.makedirs(os.path.dirname(path), exist_ok=True) + with open(path, "w") as file: + json.dump(data, file) + + +def _touch(path, mtime): + os.utime(path, (mtime, mtime)) + + +@pytest.fixture +def scenes_root(tmp_path, monkeypatch): + root = tmp_path / "scenes" + root.mkdir() + monkeypatch.setattr(talemate.files, "SCENES_DIR", root) + return root + + +COVER_ID = "cover-asset-id" + + +def _make_project(root, name, saves: dict, cover_in_library: bool = True): + """ + Create a scene project directory with the given saves. + + ``saves`` maps filename -> (mtime, scene_data) + """ + project = root / name + project.mkdir(exist_ok=True) + + if cover_in_library: + _write_json( + str(project / "assets" / "library.json"), + { + "assets": { + COVER_ID: { + "id": COVER_ID, + "file_type": "webp", + "media_type": "image/webp", + } + } + }, + ) + + for filename, (mtime, scene_data) in saves.items(): + path = project / filename + _write_json(str(path), scene_data) + _touch(str(path), mtime) + + return project + + +def test_empty_scenes_dir(scenes_root): + tree = list_scenes_tree() + assert tree == {"projects": [], "characters": []} + + +def test_missing_scenes_dir(tmp_path, monkeypatch): + monkeypatch.setattr(talemate.files, "SCENES_DIR", tmp_path / "does-not-exist") + tree = list_scenes_tree() + assert tree == {"projects": [], "characters": []} + + +def test_project_listing_with_metadata(scenes_root): + _make_project( + scenes_root, + "adventure", + { + "old-save.json": ( + 1000, + {"name": "Adventure", "assets": {"cover_image": COVER_ID}}, + ), + "new-save.json": ( + 2000, + {"name": "Adventure II", "assets": {"cover_image": COVER_ID}}, + ), + }, + ) + + tree = list_scenes_tree() + + assert len(tree["projects"]) == 1 + project = tree["projects"][0] + assert project["name"] == "adventure" + assert project["path"] == str(scenes_root / "adventure") + + # files sorted newest first + filenames = [entry["filename"] for entry in project["files"]] + assert filenames == ["new-save.json", "old-save.json"] + + newest = project["files"][0] + assert newest["scene_name"] == "Adventure II" + assert newest["relpath"] == "new-save.json" + assert newest["size"] > 0 + assert newest["modified"] == project["modified"] + + # cover resolved through the asset library, path anchored at the + # project's assets dir + assert project["cover_image"] == { + "id": COVER_ID, + "file_type": "webp", + "media_type": "image/webp", + "path": str(scenes_root / "adventure" / "assets" / f"{COVER_ID}.webp"), + } + + # library.json does not count as an asset + assert project["num_assets"] == 0 + assert project["num_nodes"] == 0 + + +def test_project_asset_and_node_counts(scenes_root): + project = _make_project( + scenes_root, "counted", {"save.json": (1000, {"name": "Counted"})} + ) + + (project / "assets" / "one.png").write_bytes(b"png") + (project / "assets" / "two.webp").write_bytes(b"webp") + _write_json(str(project / "nodes" / "scene-loop.json"), {}) + _write_json(str(project / "nodes" / "sub" / "module.json"), {}) + + tree = list_scenes_tree() + + counted = next(p for p in tree["projects"] if p["name"] == "counted") + assert counted["num_assets"] == 2 + assert counted["num_nodes"] == 2 + + +def test_projects_sorted_newest_first(scenes_root): + _make_project(scenes_root, "older", {"save.json": (1000, {"name": "Older"})}) + _make_project(scenes_root, "newer", {"save.json": (2000, {"name": "Newer"})}) + + tree = list_scenes_tree() + + assert [project["name"] for project in tree["projects"]] == ["newer", "older"] + + +def test_excluded_subdirs_and_empty_projects(scenes_root): + project = _make_project( + scenes_root, "adventure", {"save.json": (1000, {"name": "Adventure"})} + ) + + # json files in nodes/changelog/assets/info must not be listed + _write_json(str(project / "nodes" / "graph.json"), {}) + _write_json(str(project / "changelog" / "save.json.base.json"), {}) + _write_json(str(project / "assets" / "library.json"), {"assets": {}}) + _write_json(str(project / "info" / "modules.json"), {}) + + # a project directory without any save files is not listed + (scenes_root / "empty-project").mkdir() + + # a reserved top-level assets dir is not a project + _write_json(str(scenes_root / "assets" / "some.json"), {}) + + tree = list_scenes_tree() + + assert [project["name"] for project in tree["projects"]] == ["adventure"] + assert [entry["filename"] for entry in tree["projects"][0]["files"]] == [ + "save.json" + ] + + +def test_unparseable_and_non_dict_saves_are_still_listed(scenes_root): + project = scenes_root / "broken" + project.mkdir() + + invalid = project / "invalid.json" + invalid.write_text("{not valid json") + _touch(str(invalid), 1000) + + non_dict = project / "list.json" + _write_json(str(non_dict), [1, 2, 3]) + _touch(str(non_dict), 2000) + + tree = list_scenes_tree() + + files = tree["projects"][0]["files"] + assert [entry["filename"] for entry in files] == ["list.json", "invalid.json"] + for entry in files: + assert entry["scene_name"] is None + assert entry["cover_image"] is None + + +def test_cover_probe_fallback_without_library(scenes_root): + project = _make_project( + scenes_root, + "no-library", + { + "save.json": ( + 1000, + {"name": "No Library", "assets": {"cover_image": COVER_ID}}, + ) + }, + cover_in_library=False, + ) + + # cover asset exists on disk but not in a library.json + asset_path = project / "assets" / f"{COVER_ID}.png" + os.makedirs(str(project / "assets"), exist_ok=True) + asset_path.write_bytes(b"png-bytes") + + tree = list_scenes_tree() + + assert tree["projects"][0]["cover_image"] == { + "id": COVER_ID, + "file_type": "png", + "media_type": "image/png", + "path": str(asset_path), + } + + +def test_cover_path_anchored_at_project_for_nested_saves(scenes_root): + # a project whose only save lives in a subdirectory still resolves the + # cover against the project-level assets dir + project = scenes_root / "nested" + _write_json( + str(project / "sub" / "save.json"), + {"name": "Nested", "assets": {"cover_image": COVER_ID}}, + ) + _write_json( + str(project / "assets" / "library.json"), + { + "assets": { + COVER_ID: { + "id": COVER_ID, + "file_type": "png", + "media_type": "image/png", + } + } + }, + ) + + tree = list_scenes_tree() + + cover = tree["projects"][0]["cover_image"] + assert cover["path"] == str(project / "assets" / f"{COVER_ID}.png") + + +def test_cover_unresolvable(scenes_root): + _make_project( + scenes_root, + "no-cover", + { + "save.json": ( + 1000, + {"name": "No Cover", "assets": {"cover_image": "missing-id"}}, + ) + }, + cover_in_library=False, + ) + + tree = list_scenes_tree() + assert tree["projects"][0]["cover_image"] is None + + +def test_character_cards_listing(scenes_root): + characters = scenes_root / "characters" + characters.mkdir() + + png = characters / "alice.png" + png.write_bytes(b"png") + _touch(str(png), 1000) + + webp = characters / "bob.webp" + webp.write_bytes(b"webp") + _touch(str(webp), 3000) + + card_json = characters / "carol.json" + _write_json(str(card_json), {"spec": "chara_card_v2"}) + _touch(str(card_json), 2000) + + v0_card = characters / "dave.json" + _write_json(str(v0_card), {"first_mes": "Hello"}) + _touch(str(v0_card), 1500) + + # scene uploads and other non-card json files land here too - not cards + scene_json = characters / "uploaded-scene.json" + _write_json(str(scene_json), {"name": "A Scene", "environment": "scene"}) + + broken_json = characters / "broken.json" + broken_json.write_text("{not json") + + ignored = characters / "readme.txt" + ignored.write_text("not a card") + + # upload side artifacts in subdirectories are never cards + _write_json(str(characters / "assets" / "library.json"), {"assets": {}}) + nested_png = characters / "assets" / "cover.png" + nested_png.write_bytes(b"png") + + tree = list_scenes_tree() + + # characters dir is not listed as a project + assert tree["projects"] == [] + + cards = tree["characters"] + assert [card["filename"] for card in cards] == [ + "bob.webp", + "carol.json", + "dave.json", + "alice.png", + ] + assert cards[0]["media_type"] == "image/webp" + assert cards[1]["media_type"] == "application/json" + assert cards[3]["media_type"] == "image/png" + + +def test_scene_meta_cache_invalidates_on_mtime(scenes_root): + project = scenes_root / "cached" + project.mkdir() + save = project / "save.json" + + _write_json(str(save), {"name": "First"}) + _touch(str(save), 1000) + tree = list_scenes_tree() + assert tree["projects"][0]["files"][0]["scene_name"] == "First" + + # same mtime -> cached name survives a content change + _write_json(str(save), {"name": "Second"}) + _touch(str(save), 1000) + tree = list_scenes_tree() + assert tree["projects"][0]["files"][0]["scene_name"] == "First" + + # new mtime -> cache invalidated + _touch(str(save), 2000) + tree = list_scenes_tree() + assert tree["projects"][0]["files"][0]["scene_name"] == "Second"