mirror of
https://github.com/vegu-ai/talemate.git
synced 2026-08-29 10:08:58 +02:00
0.33.0 (#229)
* linting * Add cleanup function for recent scenes in config to remove non-existent paths * remove leghacy world state manager buttons * move world state scene tools into sub component * linting * move module properties to navigation drawer * Update icons in NodeEditorLibrary and NodeEditorModuleProperties for improved UI clarity * prompt tweaks * director chat prompt simplifications * more prompt fixes * Enhance type hints for duration conversion functions in time.py * narrate time action now has access to response length instructions * Add IsoDateDuration node for ISO 8601 interval string construction * Update advance_time method to include return type annotation and return message * Add AdvanceTime node to world state for time advancement with duration and narration instructions * linting * Add agent state exclusions to changelog with a TODO for module migration * Add message emission for actor, narrator, and scene analysis guidance in respective components. Enhance AgentMessages and SceneTools for better message handling and visual feedback. * Remove agent messages from state when opening agent message view in SceneTools component. * linting * openroute fetch models on key set * Add input history functionality to message input in TalemateApp component. Implement keyboard shortcuts for navigating history (Ctrl+Up/Down) and limit history to the last 10 messages. Update message sending logic to store messages in history. * Update message input hint in TalemateApp component to include keyboard shortcuts for navigating input history (Ctrl+Up/Down). * node updates * unified data extraction function * prompt tweaks * Add gamestate context support in BuildPrompt and corresponding template. Introduced new property for gamestate context and updated rendering logic to include gamestate information in prompts. * Refactor Prompt class by removing LoopedPrompt and cleaning up related methods. Update data response parsing to streamline functionality and improve clarity. Adjust imports accordingly. * Add 'data_multiple' property to GenerateResponse class to allow multiple data structures in responses. Update output socket type for 'data_obj' to support both dict and list formats. * Add DictUpdate node * Add UnpackGameState node to retrieve and unpack game state variables * gamestate nodes * linting * Enhance scene view toggle functionality to support shift-click behavior for closing all drawers when hiding the scene view. * immutable scenes should reset context db on load * linting * node updates * prompt tweaks * Add context type output and filtering for creative context ID meta entries in PathToContextID and ContextIDMetaEntries nodes * Add string replacement functionality and Jinja2 formatting support in nodes. Introduced 'old' and 'new' properties for substring replacement in the Replace node, and added a new Jinja2Format node for template rendering using jinja2. * Add additional outputs for context validation in ValidateContextIDItem node, including context type, context value, and name. * prompt tweaks * node adjustments * linting * Add data_expected attribute to Focal and Prompt classes for enhanced response handling * node updates * node updates * node updates * prompt tweaks * director summary return appropriately on no action taken * Enhance action handling in DirectorChatMixin by skipping actions when a question is present in the parsed response, ensuring better response accuracy. * Enhance ConfirmActionPrompt component by adding anchorTop prop for dynamic alignment and adjusting icon size and color for improved UI consistency. * anchor clear chat confirm to top * responsive layout fixes in template editors * linting * relock * Add scene progression guidance to chat-common-tasks template * Refactor push_history method to be asynchronous across multiple agents and scenes, ensuring consistent handling of message history updates. * Update chat instructions to clarify user intent considerations and enhance decisiveness in responses. Added guidance on distinguishing between scene progression and background changes, and refined analysis requirements for user interactions. * Enhance DirectorConsoleChatsToolbar by adding a usage cheat sheet tooltip for user guidance and refining the Clear Chat button's UI for better accessibility. * store character data at unified point * fix button * fix world editor auto sync * Shared context 2 (#19) Shared context * Refactor NodeEditorLibrary to improve search functionality and debounce input handling. Updated v-text-field model and added a watcher for search input to enhance performance. * Refactor NodeEditor and TalemateApp components to enhance UI interactions. Removed the exit creative mode button from NodeEditor and updated tooltips for clarity. Adjusted app bar navigation icons for better accessibility and added functionality to switch between node editor and creative mode. * comment * Character.update deserialize voice value correctly * Enhance SharedContext.update_to_scene method to properly add or update character data in the scene based on existence checks. This improves the synchronization of character states between shared context and scene. * shared context static history support fix context memory db imports to always import * Update WorldStateManagerSceneSharedContext.vue to clarify sharing of character, world entries, and history across connected scenes. * linting * Enhance chat modes by adding 'nospoilers' option to DirectorChat and related payloads. Update chat instructions to reflect new mode behavior and improve UI to support mode-specific icons and colors in the DirectorConsoleChatsToolbar. * Comment out 'repetition_penalty_range' in TabbyAPIClient to prevent unexpected "<unk><unk> .." responses. Further investigation needed. * linting * Add active_characters and intro_instructions to Inheritance model; implement intro generation in load_scene_from_data. Update WorldStateManagerSceneSharedContext.vue to enhance new scene creation dialog with character selection and premise instructions. * rename inheritance to scene initialization * linting * Update WorldStateManagerSceneSharedContext.vue to conditionally display alert based on scene saving status and new scene creation state. * Refine messages for shared context checkboxes in WorldStateManagerCharacter and WorldStateManagerWorldEntries components for clarity. * Add scene title generation to load process and update contextual generation template. Introduced a new method in AssistantMixin for generating scene titles, ensuring titles are concise and free of special characters. Updated load_scene_from_data to assign generated titles to scenes. * linting * Refactor GameState component to integrate Codemirror for JSON editing, replacing the previous treeview structure. Implement validation for JSON input and enhance error handling. Remove unused methods and streamline state management. * Add lastLoadedJSON property to GameState component for change detection. Update validation logic to prevent unnecessary updates when game state has not changed. * Remove status emission for gameplay switch in CmdSetEnvironmentToScene class. * allow individual sharing of attributes and details * linting * Remove redundant question handling logic in DirectorChatMixin to streamline action selection process. * Update EXTERNAL_DESCRIPTION in TabbyAPI client to include notes on EXL3 model sensitivity to inference parameters. Adjust handling of 'repetition_penalty_range' in parameter list for clarity. * director chat support remove message and regenerate message * Refactor ConfirmActionInline component to improve button rendering logic. Introduced 'size' prop for button customization and added 'comfortable' density option. Simplified icon handling with computed property for better clarity. * linting * node updates * Add appBusy prop to DirectorConsoleChats and DirectorConsoleChatsToolbar components to manage button states during busy operations. * Refactor DirectorChatMixin to utilize standalone utility functions for parsing response sections and extracting action blocks. This improves code clarity and maintainability. Added tests for new utility functions in test_utils_prompt.py to ensure correct functionality. * Update clear chat button logic to consider appBusy state in DirectorConsoleChatsToolbar component, enhancing user experience during busy operations. * linting * Remove plan.md * Add chat template identifier support and error handling in ModelPrompt class - Implemented logic to check for 'chat_template.jinja2' in Hugging Face repository. - Added new template identifiers: GraniteIdentifier and GLMIdentifier. - Enhanced error handling to avoid logging 404 errors for missing templates. - Introduced Granite.jinja2 template file for prompt structure. * node fixes * remove debug msg * Enhance error handling in DynamicInstruction class by enforcing header requirement and ensuring content defaults to an empty string if not provided. * recet scene message visibility on scene load * prompt tweaks * Enhance data extraction in Focal class by adding a fallback mechanism. Implemented additional error handling to attempt data extraction from a fenced block if the initial extraction fails, improving robustness in handling responses. * linting * node fixes * Add relative_to_root function for path resolution and update node export logic - Introduced a new function `relative_to_root` in path.py to resolve paths relative to the TALEMATE_ROOT. - Updated the `export_node_definitions` function in registry.py to use `relative_to_root` for module path resolution. - Added a check to skip non-selectable node definitions in litegraphUtils.js during registration. * show icons * Improve error handling in export_node_definitions by adding a try-except block for module path resolution. Log a warning if the relative path conversion fails. * typo * Refactor base_attributes type in Character model to a more generic dict type for improved flexibility * relock * ensure character gets added to character_data * prompt tweaks * linting * properly activate characters * activate needs to happen explicitly now and deactivated is the default * missing arg * avoid changed size error * Refactor character removal logic in shared context to prevent deletion; characters are now only marked as non-shared. * Add update_from_scene method calls in SharedContextMixin for scene synchronization * Add ensure_changelogs_for_all_scenes function to manage changelog files for all scenes; integrate it into the server run process. * Enhance backup restore functionality by adding base and latest snapshot options; improve UI with clearer labels and alerts for restore actions. * Update _apply_delta function to enhance delta application handling by adding parameters for error logging and force application of changes on non-existent paths. * Skip processing of changelog files in _list_files_and_directories function to prevent unnecessary inclusion in file listings. * Update IntroRecentScenes.vue to use optional chaining for selectedScene properties and enhance backup timestamp display with revision info. * linting * Refactor source entry attribute access in collect_source_entries function to use getattr for optional attributes, improving robustness. * Implement logic to always show scene view in scene mode within TalemateApp.vue, enhancing user experience during scene interactions. * prompt tweaks * prompt tweaks * Update TalemateApp.vue to set the active tab to 'main' when switching to the node editor, improving navigation consistency. * Add active frontend websocket handler management in websocket_endpoint * agent websocket handler node support * Refactor init_nodes method in DirectorAgent to call superclass method and rename chat initialization method in DirectorChatMixin for clarity. * Add characters output to ContextHistory node to track active participants in the scene * Add Agent Websocket Handler option to Node Editor Library with corresponding icons and labels * Add check for node selectability in NodeEditorNodeSearch component to filter search results accordingly. * Add SummarizeWebsocketHandler to handle summarize actions and integrate it into SummarizeAgent * nodes * Add data property to QueueResponse class for websocket communication and update run method to include action and data in output values. * Update manual context handling in WorldStateManager to include shared property from existing context * Enhance GetWorldEntry node to include 'shared' property in output values from world entry context * Update scene loading to allow setting scene ID from data and include ID in scene serialization * Update icon for AgentWebsocketHandler in NodeEditorLibrary component to mdi-web-box * Refactor WorldStateManager components to enhance history management and sharing capabilities. Added summarized history titles, improved UI for sharing static history, and integrated scene summarization functionality. Removed deprecated methods related to shared context settings. * linting * Change log level from warning to debug for migrate_narrator_source_to_meta error handling in NarratorMessage class. * Update GLM-no-reasoning template to include <think></think> tag before coercion message for improved prompt structure. * allow prompt templates to specify reasoning pattern * Add Seed.jinja2 template for LLM prompts with reasoning patterns and user interaction handling * Enhance NarratorAgent to support dynamic response length configuration. Updated max generation length from 192 to 256 tokens and introduced a new method to calculate response length. Modified narration methods to accept and utilize response length parameter. Added response length property in GenerateNarrationBase class and updated templates to include response length handling. * Update response length calculation in RevisionMixin to include token count for improved text processing. * Refactor response identifier in RevisionMixin to dynamically use calculated response length for improved prompt handling. * linting * allow contextual generation of static history entries * Add is_static property to HistoryEntry for static history entry identification * Add "static history" option to ContextualGenerate node for enhanced contextual generation capabilities. * Add CreateStaticArchiveEntry and RemoveStaticArchiveEntry nodes for managing static history entries. Implement input/output properties and error handling for entry creation and deletion. * nodes updated * linting * Add assets field to SceneInitialization model and update load_scene_from_data function to handle scene assets. Update WorldStateManagerSceneSharedContext.vue to include assets in scene initialization parameters. * Refactor CoverImage component to enhance drag-and-drop functionality and improve styling for empty portrait state. * Add intent_state to SceneInitialization model and update load_scene_from_data function to handle intent state. Introduce story_intent property in Scene class and reset method in SceneIntent class. Update WorldStateManagerSceneSharedContext.vue to include intent state in scene initialization parameters. * Refactor WorldStateManagerSceneSharedContext.vue to improve cancel functionality by introducing a dedicated cancelCreate method and removing the direct dialog toggle from the Cancel button. This enhances code clarity and maintainability. * Update SharedContext to use await for set_shared method, ensuring proper asynchronous handling when modifying character sharing status. * Add MAX_CONTENT_WIDTH constant and update components to use it for consistent max width styling * fix issue with data structure parsing * linting * fix tests * nodes * fix update_introduction * Add building blocks template for story configuration and scene management * Refactor toggleNavigation method to accept an 'open' parameter for direct control over drawer visibility in TalemateApp.vue * Update usageCheatSheet text in DirectorConsoleChatsToolbar.vue for clarity and add pre-wrap styling to tooltip * Add cover image and writing style sections to story and character templates; update chat common tasks with new scene restrictions and user guide reference. * linting * relock * Add EmitWorldEditorSync node to handle world editor synchronization; update WorldStateManager to refresh active tab on sync action. * Update Anthropic client with new models and adjust default settings; introduce limited parameter models for specific configurations. * director action module updates * direct context update fn * director action updates * Update usageCheatSheet in DirectorConsoleChatsToolbar.vue to include recommendation for 100B+ models. * Remove debug diagnostics from DirectorConsoleChats.vue to clean up console output. * Update card styles in IntroRecentScenes.vue for improved visual consistency; change card color to grey-darken-3 and adjust text classes for titles and subtitles. * Update EmitWorldEditorSync node to include websocket passthrough in sync action for improved event handling. * Increase maximum changelog file size limit from 500KB to 1MB to accommodate larger change logs. * linting * director action module updates * 0.33 added * Add Nexus agent persona to talemate template and initialize phrases array * Add support for project-specific grouping in NodeEditorLibrary for templates/modules, enhancing organization of node groups. * docs * Enhance NodeEditorLibrary by adding primary color to tree component for improved visibility and user experience. * docs * Enhance NewSceneSetupModal to include subtitles for writing styles and director personas, improving context and usability. * Update agent persona description in WorldStateManagerTemplates to specify current support for director only, enhancing clarity for users. * Refine agent persona description in WorldStateManagerTemplates to clarify assignment per agent in Scene Settings, maintaining focus on current director-only support. * fix crash when attempting to delete some clients * Add TODO comments in finalize_llama3 and finalize_YI methods to indicate removable cruft * Add lock_template feature to Client configuration and update related components for template management * linting * persist client template lock through model changes * There is no longer a point to enforcing creative mode when there are no characters * fix direct_narrator character argument * Update CharacterContextItem to allow 'value' to accept dict type in addition to existing types * docs * Update lock_template field in Client model to allow None type in addition to bool * Remove unused template_file field from Defaults model in Client configuration * Refactor lock_template field in Client model and ClientModal component to ensure consistent boolean handling * Add field validator for lock_template in Client model to ensure boolean value is returned * fix issue where valid data processed in extract_data_with_ai_fallback was not returned * Update default_player_character assignment in ConfigPlugin to use GamePlayerCharacter schema for improved data validation * linting * add heiku 4.5 model and make default * opse 4.5 isnt a thing * fix issue where fork / restore would restore duplicate messages * improve autocomplete handling when prefill isn't available * prompt tweaks * linting * gracefully handle removed attributes * Refactor scene reference handling in delete_changelog_files to prevent incorrect deletions. Added a test to verify proper scene reference construction and ensure changelog files are deleted correctly. * forked scenes reset memory id and are not immutable * emit_status export rev * Update RequestInput.vue to handle extra_params more robustly, ensuring defaults are set correctly for input. * only allow forking on saved messages * linting * tweak defaults * summarizer fire off of push_history.after * docs * : in world entry titles will now load correctly * linting * docs * removing base attrib ute or detail also clears it from shared list * fix issue where cancelling some generations would cause errors * increase font size * formatting fixes * unhandled errors at the loop level should not crash the entire scene * separate message processing from main loop * linting * remove debug cruft * enhance error logging in background processing to include traceback information * linting * nothing to detemrine of no model is sent * fix some errors during kcpp client deletion * improve configuration issue alert visibility * restore input focus after autocomplete * linting
This commit is contained in:
veguAI
32
tests/data/graphs/results/test-harness-collectors.json
Normal file
32
tests/data/graphs/results/test-harness-collectors.json
Normal file
@@ -0,0 +1,32 @@
|
||||
{
|
||||
"TEST_1": {
|
||||
"test": "test",
|
||||
"test2": "test2",
|
||||
"works": true,
|
||||
"test3": "test3"
|
||||
},
|
||||
"TEST_3": [
|
||||
"test",
|
||||
"test2",
|
||||
true,
|
||||
"test3"
|
||||
],
|
||||
"TEST_2": {
|
||||
"test": "test",
|
||||
"test2": "test2",
|
||||
"works": true,
|
||||
"test3": "test3"
|
||||
},
|
||||
"TEST_4": [
|
||||
"test",
|
||||
"test2",
|
||||
true,
|
||||
"test3"
|
||||
],
|
||||
"TEST_5": {
|
||||
"name": "Joe",
|
||||
"is_player": false,
|
||||
"description": "This is Joe",
|
||||
"color": "red"
|
||||
}
|
||||
}
|
||||
20
tests/data/graphs/results/test-harness-context-ids.json
Normal file
20
tests/data/graphs/results/test-harness-context-ids.json
Normal file
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"test_001_001": "- `character.attribute:Sebastian.013f54400c82`\n- `character.detail:Sebastian.3fc9b689459d`\n- `character.description:Sebastian.description`",
|
||||
"test_002_001": "Information about Sebastian - 'age'",
|
||||
"test_001_002": 40,
|
||||
"test_001_003": 60,
|
||||
"test_001_004": 60,
|
||||
"test_002_002": "World entry - 'the cake incident'",
|
||||
"test_001_005": "there was cake",
|
||||
"test_003_002": "",
|
||||
"test_003_003": "",
|
||||
"test_003_001": "Sebastian",
|
||||
"test_004_001": "[{\"id\": \"roleplay\", \"name\": \"Roleplay\", \"description\": \"Freeform dialogue between one or more characters with occasional narration.\", \"instructions\": null}]",
|
||||
"test_004_002": "Freeform dialogue between one or more characters with occasional narration.",
|
||||
"test_005_001": "Updog",
|
||||
"test_006_001": true,
|
||||
"test_007_001": "",
|
||||
"test_008_001": true,
|
||||
"test_009_001": "Speak a certain way.",
|
||||
"test_010_001": "Speak a certain way."
|
||||
}
|
||||
@@ -4,5 +4,5 @@
|
||||
"test_001_003": {
|
||||
"result": 6
|
||||
},
|
||||
"test_001_004": "What is 1 + 5 ?\n\nProvide your response inside a \"result\" value in a data object\n\n<|BOT|>"
|
||||
"test_001_004": "What is 1 + 5 ?\n\nProvide your response inside a \"result\" value in a data object\n\n<|BOT|>```json"
|
||||
}
|
||||
534
tests/data/graphs/test-harness-collectors.json
Normal file
534
tests/data/graphs/test-harness-collectors.json
Normal file
@@ -0,0 +1,534 @@
|
||||
{
|
||||
"title": "Test Collectors",
|
||||
"id": "9e549b3e-b074-4001-a6a0-8c7220667607",
|
||||
"properties": {},
|
||||
"x": 0,
|
||||
"y": 0,
|
||||
"width": 200,
|
||||
"height": 100,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "test/testCollectors",
|
||||
"nodes": {
|
||||
"9f7b78fb-73e4-49d7-ac1c-798ccdf04ba4": {
|
||||
"title": "true",
|
||||
"id": "9f7b78fb-73e4-49d7-ac1c-798ccdf04ba4",
|
||||
"properties": {
|
||||
"value": true
|
||||
},
|
||||
"x": 36,
|
||||
"y": 517,
|
||||
"width": 210,
|
||||
"height": 58,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "core/MakeBool",
|
||||
"base_type": "core/Node"
|
||||
},
|
||||
"d15ae34b-23f1-498f-b239-5444dd5e1c4a": {
|
||||
"title": "Dict Collector",
|
||||
"id": "d15ae34b-23f1-498f-b239-5444dd5e1c4a",
|
||||
"properties": {},
|
||||
"x": 586,
|
||||
"y": 257,
|
||||
"width": 140,
|
||||
"height": 116,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "data/DictCollector",
|
||||
"dynamic_inputs": [
|
||||
{
|
||||
"name": "item0",
|
||||
"type": "key/value"
|
||||
},
|
||||
{
|
||||
"name": "item1",
|
||||
"type": "key/value"
|
||||
},
|
||||
{
|
||||
"name": "item2",
|
||||
"type": "key/value"
|
||||
}
|
||||
],
|
||||
"base_type": "core/DynamicSocketNodeBase"
|
||||
},
|
||||
"1eb9b4fe-9a58-496b-996b-4337f0e53bb9": {
|
||||
"title": "Dict Collector",
|
||||
"id": "1eb9b4fe-9a58-496b-996b-4337f0e53bb9",
|
||||
"properties": {},
|
||||
"x": 836,
|
||||
"y": 477,
|
||||
"width": 140,
|
||||
"height": 76,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "data/DictCollector",
|
||||
"dynamic_inputs": [
|
||||
{
|
||||
"name": "item0",
|
||||
"type": "key/value"
|
||||
}
|
||||
],
|
||||
"base_type": "core/DynamicSocketNodeBase"
|
||||
},
|
||||
"1d5e57a3-945a-4f62-9ccf-40a21c52df23": {
|
||||
"title": "GET local.works",
|
||||
"id": "1d5e57a3-945a-4f62-9ccf-40a21c52df23",
|
||||
"properties": {
|
||||
"name": "works",
|
||||
"scope": "local"
|
||||
},
|
||||
"x": 306,
|
||||
"y": 507,
|
||||
"width": 210,
|
||||
"height": 122,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "state/GetState",
|
||||
"base_type": "core/Node"
|
||||
},
|
||||
"bfd2d639-8e89-4924-a70d-43116db824e0": {
|
||||
"title": "List Collector",
|
||||
"id": "bfd2d639-8e89-4924-a70d-43116db824e0",
|
||||
"properties": {},
|
||||
"x": 806,
|
||||
"y": 737,
|
||||
"width": 140,
|
||||
"height": 116,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "data/ListCollector",
|
||||
"dynamic_inputs": [
|
||||
{
|
||||
"name": "item0",
|
||||
"type": "any"
|
||||
},
|
||||
{
|
||||
"name": "item1",
|
||||
"type": "any"
|
||||
},
|
||||
{
|
||||
"name": "item2",
|
||||
"type": "any"
|
||||
}
|
||||
],
|
||||
"base_type": "core/DynamicSocketNodeBase"
|
||||
},
|
||||
"59764ff7-c735-459b-9beb-2d8f9648ebdb": {
|
||||
"title": "List Collector",
|
||||
"id": "59764ff7-c735-459b-9beb-2d8f9648ebdb",
|
||||
"properties": {},
|
||||
"x": 1056,
|
||||
"y": 967,
|
||||
"width": 140,
|
||||
"height": 76,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "data/ListCollector",
|
||||
"dynamic_inputs": [
|
||||
{
|
||||
"name": "item0",
|
||||
"type": "any"
|
||||
}
|
||||
],
|
||||
"base_type": "core/DynamicSocketNodeBase"
|
||||
},
|
||||
"651bb9a1-ee87-499b-ac72-b12b618405a0": {
|
||||
"title": "Watch",
|
||||
"id": "651bb9a1-ee87-499b-ac72-b12b618405a0",
|
||||
"properties": {},
|
||||
"x": 786,
|
||||
"y": 237,
|
||||
"width": 140,
|
||||
"height": 26,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "core/Watch",
|
||||
"base_type": "core/Node"
|
||||
},
|
||||
"e0fd102c-c0d1-42e3-babc-5bed3be50d9f": {
|
||||
"title": "Second Pass",
|
||||
"id": "e0fd102c-c0d1-42e3-babc-5bed3be50d9f",
|
||||
"properties": {},
|
||||
"x": 1046,
|
||||
"y": 487,
|
||||
"width": 140,
|
||||
"height": 26,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "core/Watch",
|
||||
"base_type": "core/Node"
|
||||
},
|
||||
"1d6207a6-57a9-4a47-a310-0d0eefdf1a8e": {
|
||||
"title": "List 1",
|
||||
"id": "1d6207a6-57a9-4a47-a310-0d0eefdf1a8e",
|
||||
"properties": {},
|
||||
"x": 1036,
|
||||
"y": 787,
|
||||
"width": 140,
|
||||
"height": 26,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "core/Watch",
|
||||
"base_type": "core/Node"
|
||||
},
|
||||
"aae7ee59-c370-43a6-a1d5-aae4c91cd58c": {
|
||||
"title": "List 2",
|
||||
"id": "aae7ee59-c370-43a6-a1d5-aae4c91cd58c",
|
||||
"properties": {},
|
||||
"x": 1226,
|
||||
"y": 967,
|
||||
"width": 140,
|
||||
"height": 26,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "core/Watch",
|
||||
"base_type": "core/Node"
|
||||
},
|
||||
"220fb15d-eaa2-4d77-805f-2cfacb32de0e": {
|
||||
"title": "SET local.TEST_3",
|
||||
"id": "220fb15d-eaa2-4d77-805f-2cfacb32de0e",
|
||||
"properties": {
|
||||
"name": "TEST_3",
|
||||
"scope": "shared"
|
||||
},
|
||||
"x": 1216,
|
||||
"y": 767,
|
||||
"width": 210,
|
||||
"height": 122,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "state/SetState",
|
||||
"base_type": "core/Node"
|
||||
},
|
||||
"d0205baf-30d5-4b45-b053-da7894f222be": {
|
||||
"title": "SET local.TEST_2",
|
||||
"id": "d0205baf-30d5-4b45-b053-da7894f222be",
|
||||
"properties": {
|
||||
"name": "TEST_2",
|
||||
"scope": "shared"
|
||||
},
|
||||
"x": 1216,
|
||||
"y": 467,
|
||||
"width": 210,
|
||||
"height": 122,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "state/SetState",
|
||||
"base_type": "core/Node"
|
||||
},
|
||||
"55cae32f-a6bf-4781-a643-d6e01930df19": {
|
||||
"title": "SET shared.TEST_1",
|
||||
"id": "55cae32f-a6bf-4781-a643-d6e01930df19",
|
||||
"properties": {
|
||||
"name": "TEST_1",
|
||||
"scope": "shared"
|
||||
},
|
||||
"x": 966,
|
||||
"y": 217,
|
||||
"width": 210,
|
||||
"height": 122,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "state/SetState",
|
||||
"base_type": "core/Node"
|
||||
},
|
||||
"4c01ea63-7f0f-47af-9951-dbf6ed931d90": {
|
||||
"title": "KV test2",
|
||||
"id": "4c01ea63-7f0f-47af-9951-dbf6ed931d90",
|
||||
"properties": {
|
||||
"key": "test2",
|
||||
"value": "test2"
|
||||
},
|
||||
"x": 310,
|
||||
"y": 284,
|
||||
"width": 217,
|
||||
"height": 122,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "data/MakeKeyValuePair",
|
||||
"base_type": "core/Node"
|
||||
},
|
||||
"ea192757-38d4-415b-b498-9f49db8ef90e": {
|
||||
"title": "KV test",
|
||||
"id": "ea192757-38d4-415b-b498-9f49db8ef90e",
|
||||
"properties": {
|
||||
"key": "test",
|
||||
"value": "test"
|
||||
},
|
||||
"x": 310,
|
||||
"y": 84,
|
||||
"width": 211,
|
||||
"height": 122,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "data/MakeKeyValuePair",
|
||||
"base_type": "core/Node"
|
||||
},
|
||||
"60946315-d456-41dd-8c02-390a012973f6": {
|
||||
"title": "KV test3",
|
||||
"id": "60946315-d456-41dd-8c02-390a012973f6",
|
||||
"properties": {
|
||||
"key": "test3",
|
||||
"value": "test3"
|
||||
},
|
||||
"x": 306,
|
||||
"y": 727,
|
||||
"width": 217,
|
||||
"height": 122,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "data/MakeKeyValuePair",
|
||||
"base_type": "core/Node"
|
||||
},
|
||||
"097235ed-045b-4021-8dc5-264f528422e6": {
|
||||
"title": "SET local.TEST_4",
|
||||
"id": "097235ed-045b-4021-8dc5-264f528422e6",
|
||||
"properties": {
|
||||
"name": "TEST_4",
|
||||
"scope": "shared"
|
||||
},
|
||||
"x": 1396,
|
||||
"y": 947,
|
||||
"width": 210,
|
||||
"height": 122,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "state/SetState",
|
||||
"base_type": "core/Node"
|
||||
},
|
||||
"72fede9c-9035-4c81-b3eb-2ba7d4c70bcd": {
|
||||
"title": "Stage 0",
|
||||
"id": "72fede9c-9035-4c81-b3eb-2ba7d4c70bcd",
|
||||
"properties": {
|
||||
"stage": 0
|
||||
},
|
||||
"x": 1670,
|
||||
"y": 940,
|
||||
"width": 210,
|
||||
"height": 118,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "core/Stage",
|
||||
"base_type": "core/Node"
|
||||
},
|
||||
"c6910c79-54d0-48cc-ad62-d821a34a0ecf": {
|
||||
"title": "Unpack Character",
|
||||
"id": "c6910c79-54d0-48cc-ad62-d821a34a0ecf",
|
||||
"properties": {},
|
||||
"x": 638,
|
||||
"y": 1210,
|
||||
"width": 212,
|
||||
"height": 146,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "scene/UnpackCharacter",
|
||||
"base_type": "core/Node"
|
||||
},
|
||||
"2ce019c4-0524-4e7e-b968-68f3ffa24203": {
|
||||
"title": "Dict Collector",
|
||||
"id": "2ce019c4-0524-4e7e-b968-68f3ffa24203",
|
||||
"properties": {},
|
||||
"x": 948,
|
||||
"y": 1220,
|
||||
"width": 140,
|
||||
"height": 136,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "data/DictCollector",
|
||||
"dynamic_inputs": [
|
||||
{
|
||||
"name": "item0",
|
||||
"type": "*"
|
||||
},
|
||||
{
|
||||
"name": "item1",
|
||||
"type": "*"
|
||||
},
|
||||
{
|
||||
"name": "item2",
|
||||
"type": "*"
|
||||
},
|
||||
{
|
||||
"name": "item3",
|
||||
"type": "*"
|
||||
}
|
||||
],
|
||||
"base_type": "core/DynamicSocketNodeBase"
|
||||
},
|
||||
"01bbb65b-0f61-46bb-9229-06f266942c94": {
|
||||
"title": "SET shared.TEST_5",
|
||||
"id": "01bbb65b-0f61-46bb-9229-06f266942c94",
|
||||
"properties": {
|
||||
"name": "TEST_5",
|
||||
"scope": "shared"
|
||||
},
|
||||
"x": 1158,
|
||||
"y": 1220,
|
||||
"width": 210,
|
||||
"height": 122,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "state/SetState",
|
||||
"base_type": "core/Node"
|
||||
},
|
||||
"3a38ddc1-dc86-402e-ba6a-457dcf3b9d7d": {
|
||||
"title": "Make Character",
|
||||
"id": "3a38ddc1-dc86-402e-ba6a-457dcf3b9d7d",
|
||||
"properties": {
|
||||
"name": "Joe",
|
||||
"description": "This is Joe",
|
||||
"color": "red",
|
||||
"base_attributes": null,
|
||||
"is_player": false,
|
||||
"add_to_scene": true,
|
||||
"is_active": true
|
||||
},
|
||||
"x": 358,
|
||||
"y": 1180,
|
||||
"width": 212,
|
||||
"height": 322,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "scene/MakeCharacter",
|
||||
"base_type": "core/Node"
|
||||
},
|
||||
"36f8ba6f-b7e3-4f32-b5fc-366fd87358a1": {
|
||||
"title": "Make Dict",
|
||||
"id": "36f8ba6f-b7e3-4f32-b5fc-366fd87358a1",
|
||||
"properties": {
|
||||
"data": {}
|
||||
},
|
||||
"x": 38,
|
||||
"y": 1270,
|
||||
"width": 210,
|
||||
"height": 58,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "data/MakeDict",
|
||||
"base_type": "core/Node"
|
||||
},
|
||||
"aa0ca4a7-d162-4bad-8525-47396ac43314": {
|
||||
"title": "Stage 1",
|
||||
"id": "aa0ca4a7-d162-4bad-8525-47396ac43314",
|
||||
"properties": {
|
||||
"stage": 1
|
||||
},
|
||||
"x": 1471,
|
||||
"y": 1220,
|
||||
"width": 210,
|
||||
"height": 118,
|
||||
"collapsed": false,
|
||||
"inherited": false,
|
||||
"registry": "core/Stage",
|
||||
"base_type": "core/Node"
|
||||
}
|
||||
},
|
||||
"edges": {
|
||||
"9f7b78fb-73e4-49d7-ac1c-798ccdf04ba4.value": [
|
||||
"1d5e57a3-945a-4f62-9ccf-40a21c52df23.default"
|
||||
],
|
||||
"d15ae34b-23f1-498f-b239-5444dd5e1c4a.dict": [
|
||||
"1eb9b4fe-9a58-496b-996b-4337f0e53bb9.dict",
|
||||
"651bb9a1-ee87-499b-ac72-b12b618405a0.value"
|
||||
],
|
||||
"1eb9b4fe-9a58-496b-996b-4337f0e53bb9.dict": [
|
||||
"e0fd102c-c0d1-42e3-babc-5bed3be50d9f.value"
|
||||
],
|
||||
"1d5e57a3-945a-4f62-9ccf-40a21c52df23.value": [
|
||||
"d15ae34b-23f1-498f-b239-5444dd5e1c4a.item2",
|
||||
"bfd2d639-8e89-4924-a70d-43116db824e0.item2"
|
||||
],
|
||||
"bfd2d639-8e89-4924-a70d-43116db824e0.list": [
|
||||
"59764ff7-c735-459b-9beb-2d8f9648ebdb.list",
|
||||
"1d6207a6-57a9-4a47-a310-0d0eefdf1a8e.value"
|
||||
],
|
||||
"59764ff7-c735-459b-9beb-2d8f9648ebdb.list": [
|
||||
"aae7ee59-c370-43a6-a1d5-aae4c91cd58c.value"
|
||||
],
|
||||
"651bb9a1-ee87-499b-ac72-b12b618405a0.value": [
|
||||
"55cae32f-a6bf-4781-a643-d6e01930df19.value"
|
||||
],
|
||||
"e0fd102c-c0d1-42e3-babc-5bed3be50d9f.value": [
|
||||
"d0205baf-30d5-4b45-b053-da7894f222be.value"
|
||||
],
|
||||
"1d6207a6-57a9-4a47-a310-0d0eefdf1a8e.value": [
|
||||
"220fb15d-eaa2-4d77-805f-2cfacb32de0e.value"
|
||||
],
|
||||
"aae7ee59-c370-43a6-a1d5-aae4c91cd58c.value": [
|
||||
"097235ed-045b-4021-8dc5-264f528422e6.value"
|
||||
],
|
||||
"4c01ea63-7f0f-47af-9951-dbf6ed931d90.value": [
|
||||
"d15ae34b-23f1-498f-b239-5444dd5e1c4a.item1",
|
||||
"bfd2d639-8e89-4924-a70d-43116db824e0.item1"
|
||||
],
|
||||
"ea192757-38d4-415b-b498-9f49db8ef90e.kv": [
|
||||
"d15ae34b-23f1-498f-b239-5444dd5e1c4a.item0"
|
||||
],
|
||||
"ea192757-38d4-415b-b498-9f49db8ef90e.value": [
|
||||
"bfd2d639-8e89-4924-a70d-43116db824e0.item0"
|
||||
],
|
||||
"60946315-d456-41dd-8c02-390a012973f6.kv": [
|
||||
"1eb9b4fe-9a58-496b-996b-4337f0e53bb9.item0"
|
||||
],
|
||||
"60946315-d456-41dd-8c02-390a012973f6.value": [
|
||||
"59764ff7-c735-459b-9beb-2d8f9648ebdb.item0"
|
||||
],
|
||||
"097235ed-045b-4021-8dc5-264f528422e6.name": [
|
||||
"72fede9c-9035-4c81-b3eb-2ba7d4c70bcd.state"
|
||||
],
|
||||
"c6910c79-54d0-48cc-ad62-d821a34a0ecf.name": [
|
||||
"2ce019c4-0524-4e7e-b968-68f3ffa24203.item0"
|
||||
],
|
||||
"c6910c79-54d0-48cc-ad62-d821a34a0ecf.is_player": [
|
||||
"2ce019c4-0524-4e7e-b968-68f3ffa24203.item1"
|
||||
],
|
||||
"c6910c79-54d0-48cc-ad62-d821a34a0ecf.description": [
|
||||
"2ce019c4-0524-4e7e-b968-68f3ffa24203.item2"
|
||||
],
|
||||
"c6910c79-54d0-48cc-ad62-d821a34a0ecf.color": [
|
||||
"2ce019c4-0524-4e7e-b968-68f3ffa24203.item3"
|
||||
],
|
||||
"2ce019c4-0524-4e7e-b968-68f3ffa24203.dict": [
|
||||
"01bbb65b-0f61-46bb-9229-06f266942c94.value"
|
||||
],
|
||||
"01bbb65b-0f61-46bb-9229-06f266942c94.value": [
|
||||
"aa0ca4a7-d162-4bad-8525-47396ac43314.state"
|
||||
],
|
||||
"3a38ddc1-dc86-402e-ba6a-457dcf3b9d7d.character": [
|
||||
"c6910c79-54d0-48cc-ad62-d821a34a0ecf.character"
|
||||
],
|
||||
"36f8ba6f-b7e3-4f32-b5fc-366fd87358a1.dict": [
|
||||
"3a38ddc1-dc86-402e-ba6a-457dcf3b9d7d.base_attributes"
|
||||
]
|
||||
},
|
||||
"groups": [
|
||||
{
|
||||
"title": "Group",
|
||||
"x": 11,
|
||||
"y": 9,
|
||||
"width": 1884,
|
||||
"height": 1083,
|
||||
"color": "#3f789e",
|
||||
"font_size": 24,
|
||||
"inherited": false
|
||||
},
|
||||
{
|
||||
"title": "Group",
|
||||
"x": 13,
|
||||
"y": 1105,
|
||||
"width": 1693,
|
||||
"height": 422,
|
||||
"color": "#3f789e",
|
||||
"font_size": 24,
|
||||
"inherited": false
|
||||
}
|
||||
],
|
||||
"comments": [],
|
||||
"extends": null,
|
||||
"base_type": "core/Graph",
|
||||
"inputs": [],
|
||||
"outputs": [],
|
||||
"module_properties": {},
|
||||
"style": null
|
||||
}
|
||||
1807
tests/data/graphs/test-harness-context-ids.json
Normal file
1807
tests/data/graphs/test-harness-context-ids.json
Normal file
File diff suppressed because it is too large
Load Diff
22
tests/data/util/data/empty_codeblocks.txt
Normal file
22
tests/data/util/data/empty_codeblocks.txt
Normal file
@@ -0,0 +1,22 @@
|
||||
Some text:
|
||||
|
||||
```json
|
||||
```
|
||||
|
||||
Valid data:
|
||||
|
||||
```json
|
||||
{"name": "Valid", "id": 1}
|
||||
```
|
||||
|
||||
Another empty:
|
||||
|
||||
```yaml
|
||||
```
|
||||
|
||||
More valid data:
|
||||
|
||||
```yaml
|
||||
name: Valid YAML
|
||||
id: 2
|
||||
```
|
||||
11
tests/data/util/data/faulty_json_repairable.txt
Normal file
11
tests/data/util/data/faulty_json_repairable.txt
Normal file
@@ -0,0 +1,11 @@
|
||||
Faulty JSON with missing comma:
|
||||
|
||||
```json
|
||||
{"name": "Test"}{"id": 123}
|
||||
```
|
||||
|
||||
Faulty JSON with trailing comma:
|
||||
|
||||
```json
|
||||
{"name": "Another", "value": 42,}
|
||||
```
|
||||
22
tests/data/util/data/mixed_formats.txt
Normal file
22
tests/data/util/data/mixed_formats.txt
Normal file
@@ -0,0 +1,22 @@
|
||||
Here are some data structures:
|
||||
|
||||
```json
|
||||
{"name": "JSON Object", "id": 1, "type": "json"}
|
||||
```
|
||||
|
||||
And here's some YAML:
|
||||
|
||||
```yaml
|
||||
name: YAML Object
|
||||
id: 2
|
||||
type: yaml
|
||||
tags:
|
||||
- test
|
||||
- yaml
|
||||
```
|
||||
|
||||
Another JSON:
|
||||
|
||||
```json
|
||||
{"name": "Second JSON", "id": 3, "active": true}
|
||||
```
|
||||
11
tests/data/util/data/untyped_codeblocks_json.txt
Normal file
11
tests/data/util/data/untyped_codeblocks_json.txt
Normal file
@@ -0,0 +1,11 @@
|
||||
Some data:
|
||||
|
||||
```
|
||||
{"name": "Untyped JSON", "value": 42}
|
||||
```
|
||||
|
||||
More data:
|
||||
|
||||
```
|
||||
{"name": "Another JSON", "items": [1, 2, 3]}
|
||||
```
|
||||
16
tests/data/util/data/untyped_codeblocks_yaml.txt
Normal file
16
tests/data/util/data/untyped_codeblocks_yaml.txt
Normal file
@@ -0,0 +1,16 @@
|
||||
Some data:
|
||||
|
||||
```
|
||||
name: Untyped YAML
|
||||
value: 42
|
||||
```
|
||||
|
||||
More data:
|
||||
|
||||
```
|
||||
name: Another YAML
|
||||
items:
|
||||
- 1
|
||||
- 2
|
||||
- 3
|
||||
```
|
||||
1368
tests/test_changelog.py
Normal file
1368
tests/test_changelog.py
Normal file
File diff suppressed because it is too large
Load Diff
1028
tests/test_context_id.py
Normal file
1028
tests/test_context_id.py
Normal file
File diff suppressed because it is too large
Load Diff
466
tests/test_context_id_scan.py
Normal file
466
tests/test_context_id_scan.py
Normal file
@@ -0,0 +1,466 @@
|
||||
import pytest
|
||||
from unittest.mock import Mock, patch
|
||||
|
||||
from talemate.game.engine.context_id.scanner import (
|
||||
ContextIDScanResult,
|
||||
scan_text_for_context_ids,
|
||||
OpenContextIDScanCollector,
|
||||
context_id_scan_state,
|
||||
)
|
||||
from talemate.game.engine.context_id.base import ContextIDItem
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_scene():
|
||||
"""Create a mock scene for testing."""
|
||||
return Mock()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_context_id_item():
|
||||
"""Create a mock ContextIDItem."""
|
||||
item = Mock(spec=ContextIDItem)
|
||||
item.context_type = "world_entry"
|
||||
item.name = "test_item"
|
||||
item.value = "test_value"
|
||||
return item
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_scan_text_no_context_ids(mock_scene):
|
||||
"""Test scanning text with no context IDs."""
|
||||
text = "This is regular text with no context IDs."
|
||||
result = await scan_text_for_context_ids(text, mock_scene)
|
||||
|
||||
assert isinstance(result, ContextIDScanResult)
|
||||
assert len(result.resolved) == 0
|
||||
assert len(result.unresolved) == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_scan_text_single_context_id(mock_scene, mock_context_id_item):
|
||||
"""Test scanning text with a single valid context ID."""
|
||||
text = "Update `world_entry.manual:8f54a01ce9e8` with the following changes."
|
||||
|
||||
with patch(
|
||||
"talemate.game.engine.context_id.scanner.context_id_item_from_string"
|
||||
) as mock_get:
|
||||
mock_get.return_value = mock_context_id_item
|
||||
result = await scan_text_for_context_ids(text, mock_scene)
|
||||
|
||||
assert len(result.resolved) == 1
|
||||
assert len(result.unresolved) == 0
|
||||
assert result.resolved[0] == mock_context_id_item
|
||||
mock_get.assert_called_once_with("world_entry.manual:8f54a01ce9e8", mock_scene)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_scan_text_multiple_context_ids(mock_scene, mock_context_id_item):
|
||||
"""Test scanning text with multiple context IDs."""
|
||||
text = "Update `world_entry.manual:8f54a01ce9e8` and `character.description:alice123` items."
|
||||
|
||||
with patch(
|
||||
"talemate.game.engine.context_id.scanner.context_id_item_from_string"
|
||||
) as mock_get:
|
||||
mock_get.return_value = mock_context_id_item
|
||||
result = await scan_text_for_context_ids(text, mock_scene)
|
||||
|
||||
assert len(result.resolved) == 2
|
||||
assert len(result.unresolved) == 0
|
||||
assert mock_get.call_count == 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_scan_text_unresolved_context_id(mock_scene):
|
||||
"""Test scanning text with an unresolvable context ID."""
|
||||
text = "Update `invalid.context:nonexistent` item."
|
||||
|
||||
with patch(
|
||||
"talemate.game.engine.context_id.scanner.context_id_item_from_string"
|
||||
) as mock_get:
|
||||
mock_get.return_value = None # Cannot resolve
|
||||
result = await scan_text_for_context_ids(text, mock_scene)
|
||||
|
||||
assert len(result.resolved) == 0
|
||||
assert len(result.unresolved) == 1
|
||||
assert result.unresolved[0] == "invalid.context:nonexistent"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_scan_text_exception_handling(mock_scene):
|
||||
"""Test scanning text with context ID that throws an exception."""
|
||||
text = "Update `error.context:exception` item."
|
||||
|
||||
with patch(
|
||||
"talemate.game.engine.context_id.scanner.context_id_item_from_string"
|
||||
) as mock_get:
|
||||
mock_get.side_effect = Exception("Test exception")
|
||||
result = await scan_text_for_context_ids(text, mock_scene)
|
||||
|
||||
assert len(result.resolved) == 0
|
||||
assert len(result.unresolved) == 1
|
||||
assert result.unresolved[0] == "error.context:exception"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_scan_text_mixed_results(mock_scene, mock_context_id_item):
|
||||
"""Test scanning text with both resolvable and unresolvable context IDs."""
|
||||
text = "Update `world_entry.manual:valid123` and `invalid.context:bad456` items."
|
||||
|
||||
def mock_context_id_resolver(context_id_str, scene):
|
||||
if "valid123" in context_id_str:
|
||||
return mock_context_id_item
|
||||
return None
|
||||
|
||||
with patch(
|
||||
"talemate.game.engine.context_id.scanner.context_id_item_from_string",
|
||||
side_effect=mock_context_id_resolver,
|
||||
):
|
||||
result = await scan_text_for_context_ids(text, mock_scene)
|
||||
|
||||
assert len(result.resolved) == 1
|
||||
assert len(result.unresolved) == 1
|
||||
assert result.resolved[0] == mock_context_id_item
|
||||
assert result.unresolved[0] == "invalid.context:bad456"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_scan_text_various_patterns(mock_scene, mock_context_id_item):
|
||||
"""Test scanning text with various context ID patterns."""
|
||||
text = """
|
||||
Various patterns:
|
||||
- `simple:abc123`
|
||||
- `dotted.type:path.to.item`
|
||||
- `underscore_type:item_name`
|
||||
- `complex.nested.type:deep.nested.path.item`
|
||||
"""
|
||||
|
||||
with patch(
|
||||
"talemate.game.engine.context_id.scanner.context_id_item_from_string"
|
||||
) as mock_get:
|
||||
mock_get.return_value = mock_context_id_item
|
||||
result = await scan_text_for_context_ids(text, mock_scene)
|
||||
|
||||
assert len(result.resolved) == 4
|
||||
assert len(result.unresolved) == 0
|
||||
assert mock_get.call_count == 4
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_scan_text_ignore_non_backtick_patterns(mock_scene):
|
||||
"""Test that non-backtick patterns are ignored."""
|
||||
text = """
|
||||
Should ignore these:
|
||||
- world_entry.manual:8f54a01ce9e8 (no backticks)
|
||||
- 'world_entry.manual:8f54a01ce9e8' (single quotes)
|
||||
- "world_entry.manual:8f54a01ce9e8" (double quotes)
|
||||
But match this: `world_entry.manual:8f54a01ce9e8`
|
||||
"""
|
||||
|
||||
with patch(
|
||||
"talemate.game.engine.context_id.scanner.context_id_item_from_string"
|
||||
) as mock_get:
|
||||
mock_get.return_value = None
|
||||
await scan_text_for_context_ids(text, mock_scene)
|
||||
|
||||
assert mock_get.call_count == 1
|
||||
mock_get.assert_called_once_with("world_entry.manual:8f54a01ce9e8", mock_scene)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_scan_text_ignore_invalid_patterns(mock_scene):
|
||||
"""Test that invalid patterns are ignored by regex."""
|
||||
text = """
|
||||
Invalid patterns that should be ignored:
|
||||
- `:invalid` (no prefix)
|
||||
- `invalid:` (no suffix)
|
||||
- `123invalid:abc` (starts with number)
|
||||
- `valid-name:abc` (contains hyphen)
|
||||
Valid pattern: `valid_name:abc123`
|
||||
"""
|
||||
|
||||
with patch(
|
||||
"talemate.game.engine.context_id.scanner.context_id_item_from_string"
|
||||
) as mock_get:
|
||||
mock_get.return_value = None
|
||||
await scan_text_for_context_ids(text, mock_scene)
|
||||
|
||||
assert mock_get.call_count == 1
|
||||
mock_get.assert_called_once_with("valid_name:abc123", mock_scene)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_context_id_scan_result_model():
|
||||
"""Test the ContextIDScanResult Pydantic model."""
|
||||
# Test empty initialization
|
||||
result = ContextIDScanResult()
|
||||
assert result.resolved == []
|
||||
assert result.unresolved == []
|
||||
|
||||
# Test with data
|
||||
mock_item = Mock(spec=ContextIDItem)
|
||||
result = ContextIDScanResult(resolved=[mock_item], unresolved=["unresolved_id"])
|
||||
assert len(result.resolved) == 1
|
||||
assert len(result.unresolved) == 1
|
||||
assert result.resolved[0] == mock_item
|
||||
assert result.unresolved[0] == "unresolved_id"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_scan_text_duplicate_context_ids(mock_scene, mock_context_id_item):
|
||||
"""Test scanning text with duplicate context IDs."""
|
||||
text = "Update `world_entry:item1` and then update `world_entry:item1` again."
|
||||
|
||||
with patch(
|
||||
"talemate.game.engine.context_id.scanner.context_id_item_from_string"
|
||||
) as mock_get:
|
||||
mock_get.return_value = mock_context_id_item
|
||||
result = await scan_text_for_context_ids(text, mock_scene)
|
||||
|
||||
# Should find both instances
|
||||
assert len(result.resolved) == 2
|
||||
assert len(result.unresolved) == 0
|
||||
assert mock_get.call_count == 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_scan_text_context_ids_with_spaces(mock_scene, mock_context_id_item):
|
||||
"""Test scanning text with context IDs containing spaces after colon."""
|
||||
text = "Update `world_entry:hello world` and `character:john doe smith` items."
|
||||
|
||||
with patch(
|
||||
"talemate.game.engine.context_id.scanner.context_id_item_from_string"
|
||||
) as mock_get:
|
||||
mock_get.return_value = mock_context_id_item
|
||||
result = await scan_text_for_context_ids(text, mock_scene)
|
||||
|
||||
assert len(result.resolved) == 2
|
||||
assert len(result.unresolved) == 0
|
||||
assert mock_get.call_count == 2
|
||||
# Verify the exact strings passed to the resolver
|
||||
mock_get.assert_any_call("world_entry:hello world", mock_scene)
|
||||
mock_get.assert_any_call("character:john doe smith", mock_scene)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_scan_text_context_ids_mixed_spaces_and_no_spaces(
|
||||
mock_scene, mock_context_id_item
|
||||
):
|
||||
"""Test scanning text with both spaced and non-spaced context IDs."""
|
||||
text = "Update `world_entry:no_spaces` and `character:with spaces` items."
|
||||
|
||||
with patch(
|
||||
"talemate.game.engine.context_id.scanner.context_id_item_from_string"
|
||||
) as mock_get:
|
||||
mock_get.return_value = mock_context_id_item
|
||||
result = await scan_text_for_context_ids(text, mock_scene)
|
||||
|
||||
assert len(result.resolved) == 2
|
||||
assert len(result.unresolved) == 0
|
||||
assert mock_get.call_count == 2
|
||||
mock_get.assert_any_call("world_entry:no_spaces", mock_scene)
|
||||
mock_get.assert_any_call("character:with spaces", mock_scene)
|
||||
|
||||
|
||||
def test_open_context_id_scan_collector_init():
|
||||
"""Test OpenContextIDScanCollector initialization."""
|
||||
collector = OpenContextIDScanCollector()
|
||||
assert isinstance(collector.context_ids, set)
|
||||
assert len(collector.context_ids) == 0
|
||||
|
||||
|
||||
def test_open_context_id_scan_collector_context_manager():
|
||||
"""Test OpenContextIDScanCollector as context manager."""
|
||||
collector = OpenContextIDScanCollector()
|
||||
|
||||
# Initially no context variable should be set
|
||||
assert context_id_scan_state.get(None) is None
|
||||
|
||||
# Test entering context
|
||||
with collector:
|
||||
assert context_id_scan_state.get() is collector
|
||||
assert isinstance(collector.context_ids, set)
|
||||
|
||||
# Context should be reset after exiting
|
||||
assert context_id_scan_state.get(None) is None
|
||||
|
||||
|
||||
def test_open_context_id_scan_collector_manual_enter_exit():
|
||||
"""Test OpenContextIDScanCollector manual enter/exit."""
|
||||
collector = OpenContextIDScanCollector()
|
||||
|
||||
# Initially no context variable should be set
|
||||
assert context_id_scan_state.get(None) is None
|
||||
|
||||
# Manual enter
|
||||
result = collector.__enter__()
|
||||
assert result is collector
|
||||
assert context_id_scan_state.get() is collector
|
||||
|
||||
# Manual exit
|
||||
collector.__exit__(None, None, None)
|
||||
assert context_id_scan_state.get(None) is None
|
||||
|
||||
|
||||
def test_open_context_id_scan_collector_nested_contexts():
|
||||
"""Test nested OpenContextIDScanCollector contexts."""
|
||||
collector1 = OpenContextIDScanCollector()
|
||||
collector2 = OpenContextIDScanCollector()
|
||||
|
||||
assert context_id_scan_state.get(None) is None
|
||||
|
||||
with collector1:
|
||||
assert context_id_scan_state.get() is collector1
|
||||
collector1.context_ids.add("test1")
|
||||
|
||||
with collector2:
|
||||
assert context_id_scan_state.get() is collector2
|
||||
collector2.context_ids.add("test2")
|
||||
|
||||
# Should restore to collector1 after collector2 exits
|
||||
assert context_id_scan_state.get() is collector1
|
||||
assert "test1" in collector1.context_ids
|
||||
assert "test2" in collector2.context_ids
|
||||
|
||||
# Should be None after both exit
|
||||
assert context_id_scan_state.get(None) is None
|
||||
|
||||
|
||||
def test_open_context_id_scan_collector_exception_handling():
|
||||
"""Test OpenContextIDScanCollector handles exceptions properly."""
|
||||
collector = OpenContextIDScanCollector()
|
||||
|
||||
assert context_id_scan_state.get(None) is None
|
||||
|
||||
try:
|
||||
with collector:
|
||||
assert context_id_scan_state.get() is collector
|
||||
raise ValueError("Test exception")
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
# Context should still be reset even after exception
|
||||
assert context_id_scan_state.get(None) is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_scan_text_with_collector_context(mock_scene, mock_context_id_item):
|
||||
"""Test scanning text while in OpenContextIDScanCollector context."""
|
||||
text = "Update `world_entry.manual:8f54a01ce9e8` with changes."
|
||||
collector = OpenContextIDScanCollector()
|
||||
|
||||
# Mock the context ID item to have a context_id attribute
|
||||
mock_context_id_item.context_id = "world_entry.manual:8f54a01ce9e8"
|
||||
|
||||
with patch(
|
||||
"talemate.game.engine.context_id.scanner.context_id_item_from_string"
|
||||
) as mock_get:
|
||||
mock_get.return_value = mock_context_id_item
|
||||
|
||||
with collector:
|
||||
result = await scan_text_for_context_ids(text, mock_scene)
|
||||
|
||||
# Verify scan result
|
||||
assert len(result.resolved) == 1
|
||||
assert len(result.unresolved) == 0
|
||||
|
||||
# Verify collector captured the context ID
|
||||
assert "world_entry.manual:8f54a01ce9e8" in collector.context_ids
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_scan_text_multiple_items_with_collector(mock_scene):
|
||||
"""Test scanning text with multiple context IDs while using collector."""
|
||||
text = "Update `world_entry:item1` and `character:alice` items."
|
||||
collector = OpenContextIDScanCollector()
|
||||
|
||||
# Create mock context items with different IDs
|
||||
mock_item1 = Mock(spec=ContextIDItem)
|
||||
mock_item1.context_id = "world_entry:item1"
|
||||
mock_item2 = Mock(spec=ContextIDItem)
|
||||
mock_item2.context_id = "character:alice"
|
||||
|
||||
def mock_resolver(context_id_str, scene):
|
||||
if "item1" in context_id_str:
|
||||
return mock_item1
|
||||
elif "alice" in context_id_str:
|
||||
return mock_item2
|
||||
return None
|
||||
|
||||
with patch(
|
||||
"talemate.game.engine.context_id.scanner.context_id_item_from_string",
|
||||
side_effect=mock_resolver,
|
||||
):
|
||||
with collector:
|
||||
result = await scan_text_for_context_ids(text, mock_scene)
|
||||
|
||||
# Verify scan result
|
||||
assert len(result.resolved) == 2
|
||||
assert len(result.unresolved) == 0
|
||||
|
||||
# Verify collector captured both context IDs
|
||||
assert "world_entry:item1" in collector.context_ids
|
||||
assert "character:alice" in collector.context_ids
|
||||
assert len(collector.context_ids) == 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_scan_text_without_collector_context(mock_scene, mock_context_id_item):
|
||||
"""Test scanning text without OpenContextIDScanCollector context."""
|
||||
text = "Update `world_entry.manual:8f54a01ce9e8` with changes."
|
||||
|
||||
with patch(
|
||||
"talemate.game.engine.context_id.scanner.context_id_item_from_string"
|
||||
) as mock_get:
|
||||
mock_get.return_value = mock_context_id_item
|
||||
|
||||
# Scan without collector context
|
||||
result = await scan_text_for_context_ids(text, mock_scene)
|
||||
|
||||
# Should still work normally
|
||||
assert len(result.resolved) == 1
|
||||
assert len(result.unresolved) == 0
|
||||
|
||||
# But no collector should be active
|
||||
assert context_id_scan_state.get(None) is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_collector_context_ids_accumulate(mock_scene):
|
||||
"""Test that collector accumulates context IDs across multiple scans."""
|
||||
collector = OpenContextIDScanCollector()
|
||||
|
||||
# Mock context items
|
||||
mock_item1 = Mock(spec=ContextIDItem)
|
||||
mock_item1.context_id = "world_entry:item1"
|
||||
mock_item2 = Mock(spec=ContextIDItem)
|
||||
mock_item2.context_id = "character:alice"
|
||||
|
||||
def mock_resolver(context_id_str, scene):
|
||||
if "item1" in context_id_str:
|
||||
return mock_item1
|
||||
elif "alice" in context_id_str:
|
||||
return mock_item2
|
||||
return None
|
||||
|
||||
with patch(
|
||||
"talemate.game.engine.context_id.scanner.context_id_item_from_string",
|
||||
side_effect=mock_resolver,
|
||||
):
|
||||
with collector:
|
||||
# First scan
|
||||
await scan_text_for_context_ids("Update `world_entry:item1`", mock_scene)
|
||||
assert len(collector.context_ids) == 1
|
||||
assert "world_entry:item1" in collector.context_ids
|
||||
|
||||
# Second scan with different item
|
||||
await scan_text_for_context_ids("Update `character:alice`", mock_scene)
|
||||
assert len(collector.context_ids) == 2
|
||||
assert "world_entry:item1" in collector.context_ids
|
||||
assert "character:alice" in collector.context_ids
|
||||
|
||||
# Third scan with duplicate - should not increase count
|
||||
await scan_text_for_context_ids(
|
||||
"Update `world_entry:item1` again", mock_scene
|
||||
)
|
||||
assert len(collector.context_ids) == 2 # Still 2, not 3
|
||||
@@ -218,3 +218,15 @@ async def test_graph_prompt(mock_scene):
|
||||
client_reponses.append("The sum of 1 and 5 is 6.")
|
||||
client_reponses.append('```json\n{\n "result": 6\n}\n```')
|
||||
await fn(mock_scene)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_graph_collectors(mock_scene):
|
||||
fn = make_graph_test("test-harness-collectors", False)
|
||||
await fn(mock_scene)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_graph_context_ids(mock_scene):
|
||||
fn = make_graph_test("test-harness-context-ids", False)
|
||||
await fn(mock_scene)
|
||||
|
||||
@@ -2,11 +2,16 @@ import os
|
||||
import pytest
|
||||
import json
|
||||
import yaml
|
||||
from unittest.mock import MagicMock
|
||||
import talemate.util.data
|
||||
from talemate.util.data import (
|
||||
fix_faulty_json,
|
||||
extract_json,
|
||||
extract_json_v2,
|
||||
extract_yaml_v2,
|
||||
extract_data_auto,
|
||||
extract_data,
|
||||
extract_data_with_ai_fallback,
|
||||
JSONEncoder,
|
||||
DataParsingError,
|
||||
fix_yaml_colon_in_strings,
|
||||
@@ -20,6 +25,28 @@ def get_test_data_path(filename):
|
||||
return os.path.join(base_dir, "data", "util", "data", filename)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_client_and_prompt():
|
||||
"""Create mock client and prompt for extract_data_auto tests."""
|
||||
client = MagicMock()
|
||||
prompt_cls = MagicMock()
|
||||
|
||||
# Mock the extract_data_with_ai_fallback to just use extract_data
|
||||
async def mock_extract_with_ai(client, text, prompt_cls, schema_format):
|
||||
# Wrap in codeblock format and use existing extract_data
|
||||
wrapped = f"```{schema_format}\n{text}\n```"
|
||||
return extract_data(wrapped, schema_format)
|
||||
|
||||
# Patch the function during tests
|
||||
original_func = talemate.util.data.extract_data_with_ai_fallback
|
||||
talemate.util.data.extract_data_with_ai_fallback = mock_extract_with_ai
|
||||
|
||||
yield client, prompt_cls
|
||||
|
||||
# Restore original function
|
||||
talemate.util.data.extract_data_with_ai_fallback = original_func
|
||||
|
||||
|
||||
def test_json_encoder():
|
||||
"""Test JSONEncoder handles unknown types by converting to string."""
|
||||
|
||||
@@ -426,3 +453,391 @@ def test_extract_yaml_v2_with_colons():
|
||||
assert "Wait for me!" in instructions_obj["instructions_list"][0]
|
||||
assert "Look around" in instructions_obj["instructions_list"][1]
|
||||
assert "Is there another way out?" in instructions_obj["instructions_list"][1]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_extract_data_auto_mixed_formats(mock_client_and_prompt):
|
||||
"""Test extract_data_auto with mixed JSON and YAML codeblocks."""
|
||||
client, prompt_cls = mock_client_and_prompt
|
||||
|
||||
# Load test data
|
||||
with open(get_test_data_path("mixed_formats.txt"), "r") as f:
|
||||
mixed_text = f.read()
|
||||
|
||||
result = await extract_data_auto(mixed_text, client, prompt_cls)
|
||||
|
||||
# Should extract all three objects
|
||||
assert len(result) == 3
|
||||
|
||||
# Verify objects by ID
|
||||
objects_by_id = {obj["id"]: obj for obj in result}
|
||||
|
||||
assert objects_by_id[1]["name"] == "JSON Object"
|
||||
assert objects_by_id[1]["type"] == "json"
|
||||
|
||||
assert objects_by_id[2]["name"] == "YAML Object"
|
||||
assert objects_by_id[2]["type"] == "yaml"
|
||||
assert "test" in objects_by_id[2]["tags"]
|
||||
|
||||
assert objects_by_id[3]["name"] == "Second JSON"
|
||||
assert objects_by_id[3]["active"] is True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_extract_data_auto_untyped_codeblocks(mock_client_and_prompt):
|
||||
"""Test extract_data_auto with untyped codeblocks using default format."""
|
||||
# Test with JSON default
|
||||
with open(get_test_data_path("untyped_codeblocks_json.txt"), "r") as f:
|
||||
json_text = f.read()
|
||||
|
||||
client, prompt_cls = mock_client_and_prompt
|
||||
result = await extract_data_auto(
|
||||
json_text, client, prompt_cls, schema_format="json"
|
||||
)
|
||||
assert len(result) == 2
|
||||
|
||||
names = {obj["name"] for obj in result}
|
||||
assert "Untyped JSON" in names
|
||||
assert "Another JSON" in names
|
||||
|
||||
# Test with YAML default
|
||||
with open(get_test_data_path("untyped_codeblocks_yaml.txt"), "r") as f:
|
||||
yaml_text = f.read()
|
||||
|
||||
result = await extract_data_auto(
|
||||
yaml_text, client, prompt_cls, schema_format="yaml"
|
||||
)
|
||||
assert len(result) == 2
|
||||
|
||||
names = {obj["name"] for obj in result}
|
||||
assert "Untyped YAML" in names
|
||||
assert "Another YAML" in names
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_extract_data_auto_bare_codeblock(mock_client_and_prompt):
|
||||
"""Test extract_data_auto with entire text being just a codeblock."""
|
||||
# JSON codeblock
|
||||
json_codeblock = """```json
|
||||
{"name": "Bare JSON", "id": 123, "active": true}
|
||||
```"""
|
||||
|
||||
client, prompt_cls = mock_client_and_prompt
|
||||
result = await extract_data_auto(json_codeblock, client, prompt_cls)
|
||||
assert len(result) == 1
|
||||
assert result[0]["name"] == "Bare JSON"
|
||||
assert result[0]["id"] == 123
|
||||
|
||||
# YAML codeblock
|
||||
yaml_codeblock = """```yaml
|
||||
name: Bare YAML
|
||||
id: 456
|
||||
active: false
|
||||
tags:
|
||||
- bare
|
||||
- yaml
|
||||
```"""
|
||||
|
||||
result = await extract_data_auto(yaml_codeblock, client, prompt_cls)
|
||||
assert len(result) == 1
|
||||
assert result[0]["name"] == "Bare YAML"
|
||||
assert result[0]["id"] == 456
|
||||
assert "bare" in result[0]["tags"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_extract_data_auto_raw_data(mock_client_and_prompt):
|
||||
"""Test extract_data_auto with raw data structures (no codeblocks)."""
|
||||
# Raw JSON
|
||||
raw_json = '{"name": "Raw JSON", "value": 100}'
|
||||
client, prompt_cls = mock_client_and_prompt
|
||||
result = await extract_data_auto(raw_json, client, prompt_cls, schema_format="json")
|
||||
assert len(result) == 1
|
||||
assert result[0]["name"] == "Raw JSON"
|
||||
assert result[0]["value"] == 100
|
||||
|
||||
# Raw YAML
|
||||
raw_yaml = """name: Raw YAML
|
||||
value: 200
|
||||
metadata:
|
||||
created: 2023-01-01
|
||||
version: 1.0"""
|
||||
|
||||
result = await extract_data_auto(raw_yaml, client, prompt_cls, schema_format="yaml")
|
||||
assert len(result) == 1
|
||||
assert result[0]["name"] == "Raw YAML"
|
||||
assert result[0]["value"] == 200
|
||||
# YAML parser converts date strings to date objects
|
||||
assert str(result[0]["metadata"]["created"]) == "2023-01-01"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_extract_data_auto_empty_codeblocks(mock_client_and_prompt):
|
||||
"""Test extract_data_auto skips empty codeblocks."""
|
||||
# Load test data
|
||||
with open(get_test_data_path("empty_codeblocks.txt"), "r") as f:
|
||||
text_with_empty = f.read()
|
||||
|
||||
client, prompt_cls = mock_client_and_prompt
|
||||
result = await extract_data_auto(text_with_empty, client, prompt_cls)
|
||||
assert len(result) == 2
|
||||
|
||||
objects_by_id = {obj["id"]: obj for obj in result}
|
||||
assert objects_by_id[1]["name"] == "Valid"
|
||||
assert objects_by_id[2]["name"] == "Valid YAML"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_extract_data_auto_malformed_blocks(mock_client_and_prompt):
|
||||
"""Test extract_data_auto handles malformed blocks gracefully."""
|
||||
text_with_malformed = """
|
||||
Valid JSON:
|
||||
|
||||
```json
|
||||
{"name": "Valid", "id": 1}
|
||||
```
|
||||
|
||||
Malformed JSON:
|
||||
|
||||
```json
|
||||
{"name": "Broken", "id":
|
||||
```
|
||||
|
||||
Another valid JSON:
|
||||
|
||||
```json
|
||||
{"name": "Also Valid", "id": 2}
|
||||
```
|
||||
"""
|
||||
|
||||
client, prompt_cls = mock_client_and_prompt
|
||||
result = await extract_data_auto(text_with_malformed, client, prompt_cls)
|
||||
# Should extract the 2 valid objects and skip the malformed one
|
||||
assert len(result) == 2
|
||||
|
||||
names = {obj["name"] for obj in result}
|
||||
assert "Valid" in names
|
||||
assert "Also Valid" in names
|
||||
assert "Broken" not in names # Should be skipped
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_extract_data_auto_repairs_faulty_json(mock_client_and_prompt):
|
||||
"""Test extract_data_auto can repair faulty JSON blocks."""
|
||||
# Load test data
|
||||
with open(get_test_data_path("faulty_json_repairable.txt"), "r") as f:
|
||||
text_with_faulty = f.read()
|
||||
|
||||
client, prompt_cls = mock_client_and_prompt
|
||||
result = await extract_data_auto(text_with_faulty, client, prompt_cls)
|
||||
# Should successfully repair and extract both objects
|
||||
assert len(result) == 3 # Two from first block (after repair), one from second
|
||||
|
||||
# Check that repair worked
|
||||
names = {obj["name"] for obj in result if "name" in obj}
|
||||
assert "Test" in names
|
||||
assert "Another" in names
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_extract_data_auto_yml_identifier(mock_client_and_prompt):
|
||||
"""Test extract_data_auto recognizes 'yml' as YAML identifier."""
|
||||
yml_text = """
|
||||
Data with yml extension:
|
||||
|
||||
```yml
|
||||
name: YML Test
|
||||
id: 123
|
||||
config:
|
||||
enabled: true
|
||||
timeout: 30
|
||||
```
|
||||
"""
|
||||
|
||||
client, prompt_cls = mock_client_and_prompt
|
||||
result = await extract_data_auto(yml_text, client, prompt_cls)
|
||||
assert len(result) == 1
|
||||
assert result[0]["name"] == "YML Test"
|
||||
assert result[0]["id"] == 123
|
||||
assert result[0]["config"]["enabled"] is True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_extract_data_auto_invalid_raw_data(mock_client_and_prompt):
|
||||
"""Test extract_data_auto raises DataParsingError for invalid raw data."""
|
||||
# Invalid raw JSON
|
||||
invalid_json = '{"name": "Broken JSON", "id":'
|
||||
|
||||
with pytest.raises(DataParsingError) as exc_info:
|
||||
client, prompt_cls = mock_client_and_prompt
|
||||
await extract_data_auto(invalid_json, client, prompt_cls, schema_format="json")
|
||||
|
||||
assert "Failed to parse raw JSON data" in str(exc_info.value)
|
||||
|
||||
# Invalid raw YAML
|
||||
invalid_yaml = """name: Broken YAML
|
||||
- invalid: structure
|
||||
without: proper indentation"""
|
||||
|
||||
with pytest.raises(DataParsingError) as exc_info:
|
||||
await extract_data_auto(invalid_yaml, client, prompt_cls, schema_format="yaml")
|
||||
|
||||
assert "Failed to parse raw YAML data" in str(exc_info.value)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_extract_data_auto_unsupported_format(mock_client_and_prompt):
|
||||
"""Test extract_data_auto raises DataParsingError for unsupported formats."""
|
||||
text = '{"name": "test"}'
|
||||
|
||||
with pytest.raises(DataParsingError) as exc_info:
|
||||
client, prompt_cls = mock_client_and_prompt
|
||||
await extract_data_auto(text, client, prompt_cls, schema_format="xml")
|
||||
|
||||
assert "Failed to parse raw XML data" in str(exc_info.value)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_extract_data_auto_multiple_objects_in_single_block(
|
||||
mock_client_and_prompt,
|
||||
):
|
||||
"""Test extract_data_auto handles multiple objects within a single codeblock."""
|
||||
multiple_json = """
|
||||
```json
|
||||
{"id": 1, "name": "First"}
|
||||
{"id": 2, "name": "Second"}
|
||||
{"id": 3, "name": "Third"}
|
||||
```
|
||||
"""
|
||||
|
||||
client, prompt_cls = mock_client_and_prompt
|
||||
result = await extract_data_auto(multiple_json, client, prompt_cls)
|
||||
assert len(result) == 3
|
||||
|
||||
objects_by_id = {obj["id"]: obj for obj in result}
|
||||
assert objects_by_id[1]["name"] == "First"
|
||||
assert objects_by_id[2]["name"] == "Second"
|
||||
assert objects_by_id[3]["name"] == "Third"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_extract_data_with_ai_fallback_json_without_codeblock():
|
||||
"""Test extract_data_with_ai_fallback when AI returns JSON without code block."""
|
||||
# Mock client and prompt
|
||||
client = MagicMock()
|
||||
client.data_format = "json"
|
||||
prompt_cls = MagicMock()
|
||||
|
||||
# Simulate AI returning corrected JSON without code block
|
||||
async def mock_request(*args, **kwargs):
|
||||
return '{"name": "Fixed JSON", "id": 123, "active": true}'
|
||||
|
||||
prompt_cls.request = mock_request
|
||||
|
||||
# Malformed JSON that cannot be auto-fixed (invalid structure)
|
||||
malformed_json = '{"name": "Broken" this is broken, "id": 123}'
|
||||
|
||||
result = await extract_data_with_ai_fallback(
|
||||
client, malformed_json, prompt_cls, "json"
|
||||
)
|
||||
|
||||
# Should successfully extract the JSON even without code block
|
||||
assert len(result) == 1
|
||||
assert result[0]["name"] == "Fixed JSON"
|
||||
assert result[0]["id"] == 123
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_extract_data_with_ai_fallback_json_with_codeblock():
|
||||
"""Test extract_data_with_ai_fallback when AI returns JSON with code block."""
|
||||
# Mock client and prompt
|
||||
client = MagicMock()
|
||||
client.data_format = "json"
|
||||
prompt_cls = MagicMock()
|
||||
|
||||
# Simulate AI returning corrected JSON with code block
|
||||
async def mock_request(*args, **kwargs):
|
||||
return '```json\n{"name": "Fixed JSON", "id": 456, "active": false}\n```'
|
||||
|
||||
prompt_cls.request = mock_request
|
||||
|
||||
# Malformed JSON that will trigger AI fallback
|
||||
malformed_json = '{"name": "Broken", "id": 456,'
|
||||
|
||||
result = await extract_data_with_ai_fallback(
|
||||
client, malformed_json, prompt_cls, "json"
|
||||
)
|
||||
|
||||
# Should successfully extract the JSON
|
||||
assert len(result) == 1
|
||||
assert result[0]["name"] == "Fixed JSON"
|
||||
assert result[0]["id"] == 456
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_extract_data_with_ai_fallback_yaml_without_codeblock():
|
||||
"""Test extract_data_with_ai_fallback when AI returns YAML without code block."""
|
||||
# Mock client and prompt
|
||||
client = MagicMock()
|
||||
client.data_format = "yaml"
|
||||
prompt_cls = MagicMock()
|
||||
|
||||
# Simulate AI returning corrected YAML without code block
|
||||
async def mock_request(*args, **kwargs):
|
||||
return """name: Fixed YAML
|
||||
id: 789
|
||||
active: true
|
||||
tags:
|
||||
- test
|
||||
- fixed"""
|
||||
|
||||
prompt_cls.request = mock_request
|
||||
|
||||
# Malformed YAML that will trigger AI fallback
|
||||
malformed_yaml = """name: Broken
|
||||
id: 789
|
||||
active: true"""
|
||||
|
||||
result = await extract_data_with_ai_fallback(
|
||||
client, malformed_yaml, prompt_cls, "yaml"
|
||||
)
|
||||
|
||||
# Should successfully extract the YAML even without code block
|
||||
assert len(result) == 1
|
||||
assert result[0]["name"] == "Fixed YAML"
|
||||
assert result[0]["id"] == 789
|
||||
assert result[0]["active"] is True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_extract_data_with_ai_fallback_yaml_with_codeblock():
|
||||
"""Test extract_data_with_ai_fallback when AI returns YAML with code block."""
|
||||
# Mock client and prompt
|
||||
client = MagicMock()
|
||||
client.data_format = "yaml"
|
||||
prompt_cls = MagicMock()
|
||||
|
||||
# Simulate AI returning corrected YAML with code block
|
||||
async def mock_request(*args, **kwargs):
|
||||
return """```yaml
|
||||
name: Fixed YAML
|
||||
id: 999
|
||||
active: false
|
||||
```"""
|
||||
|
||||
prompt_cls.request = mock_request
|
||||
|
||||
# Malformed YAML that will trigger AI fallback
|
||||
malformed_yaml = """name: Broken
|
||||
id: 999
|
||||
active: false"""
|
||||
|
||||
result = await extract_data_with_ai_fallback(
|
||||
client, malformed_yaml, prompt_cls, "yaml"
|
||||
)
|
||||
|
||||
# Should successfully extract the YAML
|
||||
assert len(result) == 1
|
||||
assert result[0]["name"] == "Fixed YAML"
|
||||
assert result[0]["id"] == 999
|
||||
assert result[0]["active"] is False
|
||||
|
||||
393
tests/test_utils_prompt.py
Normal file
393
tests/test_utils_prompt.py
Normal file
@@ -0,0 +1,393 @@
|
||||
import json
|
||||
from talemate.util.prompt import (
|
||||
parse_response_section,
|
||||
extract_actions_block,
|
||||
clean_visible_response,
|
||||
)
|
||||
|
||||
|
||||
# Helper to parse extracted content (since extract_actions_block now returns raw string)
|
||||
def parse_actions_content(content: str | None) -> list[dict] | None:
|
||||
"""Parse the raw actions content string into a list of dicts."""
|
||||
if not content:
|
||||
return None
|
||||
try:
|
||||
data = json.loads(content)
|
||||
if isinstance(data, dict):
|
||||
data = [data]
|
||||
if not isinstance(data, list):
|
||||
return None
|
||||
|
||||
normalized = []
|
||||
for item in data:
|
||||
if isinstance(item, list):
|
||||
for sub in item:
|
||||
if isinstance(sub, dict):
|
||||
name = sub.get("name") or sub.get("function")
|
||||
instructions = sub.get("instructions") or ""
|
||||
if name:
|
||||
normalized.append(
|
||||
{"name": str(name), "instructions": str(instructions)}
|
||||
)
|
||||
continue
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
name = item.get("name") or item.get("function")
|
||||
instructions = item.get("instructions") or ""
|
||||
if name:
|
||||
normalized.append(
|
||||
{"name": str(name), "instructions": str(instructions)}
|
||||
)
|
||||
return normalized or None
|
||||
except json.JSONDecodeError:
|
||||
return None
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Tests for parse_response_section
|
||||
# ============================================================================
|
||||
|
||||
|
||||
class TestParseResponseSection:
|
||||
"""Tests for the parse_response_section function."""
|
||||
|
||||
def test_basic_message_with_analysis(self):
|
||||
"""Test extracting a MESSAGE after ANALYSIS block."""
|
||||
response = """
|
||||
<ANALYSIS>
|
||||
This is some analysis text.
|
||||
</ANALYSIS>
|
||||
<MESSAGE>
|
||||
This is the response message.
|
||||
</MESSAGE>
|
||||
"""
|
||||
result = parse_response_section(response)
|
||||
assert result == "This is the response message."
|
||||
|
||||
def test_message_without_analysis(self):
|
||||
"""Test extracting MESSAGE when no ANALYSIS block present."""
|
||||
response = """
|
||||
<MESSAGE>
|
||||
Simple message without analysis.
|
||||
</MESSAGE>
|
||||
"""
|
||||
result = parse_response_section(response)
|
||||
assert result == "Simple message without analysis."
|
||||
|
||||
def test_actions_in_analysis_ignored(self):
|
||||
"""Test that <ACTION> tags within ANALYSIS don't interfere."""
|
||||
response = """
|
||||
<ANALYSIS>
|
||||
The best action would be <ACTION>test</ACTION> but we need to consider:
|
||||
- Multiple <ACTION> tags here
|
||||
- Even nested or malformed ones
|
||||
</ANALYSIS>
|
||||
<MESSAGE>
|
||||
This is the actual message.
|
||||
</MESSAGE>
|
||||
"""
|
||||
result = parse_response_section(response)
|
||||
assert result == "This is the actual message."
|
||||
|
||||
def test_nested_message_like_text_in_analysis(self):
|
||||
"""Test MESSAGE-like text in ANALYSIS doesn't confuse parser."""
|
||||
response = """
|
||||
<ANALYSIS>
|
||||
The user might want to see <MESSAGE>this</MESSAGE> but that's just analysis.
|
||||
We could also say "<MESSAGE>something else</MESSAGE>" in quotes.
|
||||
</ANALYSIS>
|
||||
<MESSAGE>
|
||||
Real message here.
|
||||
</MESSAGE>
|
||||
"""
|
||||
result = parse_response_section(response)
|
||||
assert result == "Real message here."
|
||||
|
||||
def test_decision_tag_in_analysis_ignored(self):
|
||||
"""Test that <DECISION> tags within ANALYSIS don't interfere with MESSAGE parsing."""
|
||||
response = """
|
||||
<ANALYSIS>
|
||||
The best decision would be <DECISION>option_a</DECISION> based on:
|
||||
- Multiple <DECISION> tags here
|
||||
- Even nested ones like <DECISION>option_b</DECISION>
|
||||
</ANALYSIS>
|
||||
<MESSAGE>
|
||||
This is the actual message.
|
||||
</MESSAGE>
|
||||
"""
|
||||
result = parse_response_section(response)
|
||||
assert result == "This is the actual message."
|
||||
|
||||
def test_decision_block_in_message_extracted(self):
|
||||
"""Test that DECISION blocks within MESSAGE are extracted (but will be stripped later)."""
|
||||
response = """
|
||||
<ANALYSIS>
|
||||
Analysis text.
|
||||
</ANALYSIS>
|
||||
<MESSAGE>
|
||||
Here is my response with a decision:
|
||||
<DECISION>
|
||||
The character should proceed cautiously.
|
||||
</DECISION>
|
||||
More message text after decision.
|
||||
</MESSAGE>
|
||||
"""
|
||||
result = parse_response_section(response)
|
||||
assert "Here is my response" in result
|
||||
assert "<DECISION>" in result
|
||||
# Note: The actual stripping happens in chat_clean_visible_response
|
||||
# This test just verifies the MESSAGE is extracted with DECISION intact
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Tests for extract_actions_block
|
||||
# ============================================================================
|
||||
|
||||
|
||||
class TestExtractActionsBlock:
|
||||
"""Tests for the extract_actions_block function."""
|
||||
|
||||
def test_basic_actions_json(self):
|
||||
"""Test extracting basic ACTIONS block with JSON."""
|
||||
response = """
|
||||
<ACTIONS>
|
||||
```json
|
||||
[
|
||||
{"name": "test_action", "instructions": "Do something"}
|
||||
]
|
||||
```
|
||||
</ACTIONS>
|
||||
"""
|
||||
content = extract_actions_block(response)
|
||||
assert content is not None
|
||||
result = parse_actions_content(content)
|
||||
assert result is not None
|
||||
assert len(result) == 1
|
||||
assert result[0]["name"] == "test_action"
|
||||
assert result[0]["instructions"] == "Do something"
|
||||
|
||||
def test_actions_in_analysis_ignored(self):
|
||||
"""Test that ACTIONS blocks within ANALYSIS are not extracted."""
|
||||
response = """
|
||||
<ANALYSIS>
|
||||
We could perform <ACTIONS>
|
||||
```json
|
||||
[{"name": "fake", "instructions": "This is just analysis"}]
|
||||
```
|
||||
</ACTIONS>
|
||||
but that's just analysis.
|
||||
</ANALYSIS>
|
||||
<MESSAGE>
|
||||
Here's the real response.
|
||||
</MESSAGE>
|
||||
<ACTIONS>
|
||||
```json
|
||||
[{"name": "real_action", "instructions": "This is the real action"}]
|
||||
```
|
||||
</ACTIONS>
|
||||
"""
|
||||
content = extract_actions_block(response)
|
||||
result = parse_actions_content(content)
|
||||
assert result is not None
|
||||
assert len(result) == 1
|
||||
assert result[0]["name"] == "real_action"
|
||||
assert "real action" in result[0]["instructions"]
|
||||
|
||||
def test_only_actions_in_analysis_returns_none(self):
|
||||
"""
|
||||
Test that if ACTIONS only appears within ANALYSIS (and not after),
|
||||
we return None since those are not real actions.
|
||||
"""
|
||||
response = """
|
||||
<ANALYSIS>
|
||||
We could use <ACTIONS>
|
||||
```json
|
||||
[{"name": "fake_action", "instructions": "This is just theoretical"}]
|
||||
```
|
||||
</ACTIONS>
|
||||
but that's just analysis.
|
||||
</ANALYSIS>
|
||||
<MESSAGE>
|
||||
Let me think about this more.
|
||||
</MESSAGE>
|
||||
"""
|
||||
result = extract_actions_block(response)
|
||||
assert result is None
|
||||
|
||||
def test_actions_with_action_tag_in_analysis(self):
|
||||
"""Test the edge case where <ACTION> tags appear in <ANALYSIS> before <ACTIONS>."""
|
||||
response = """
|
||||
<ANALYSIS>
|
||||
Looking at the scene, I notice several things:
|
||||
- The character could <ACTION>move</ACTION> to the door
|
||||
- Or they could <ACTION>speak</ACTION> to the other character
|
||||
- Maybe even <ACTION>hide</ACTION> behind something
|
||||
Based on this analysis, I recommend we proceed.
|
||||
</ANALYSIS>
|
||||
<MESSAGE>
|
||||
I think the best course of action is to have them move cautiously.
|
||||
</MESSAGE>
|
||||
<ACTIONS>
|
||||
```json
|
||||
[
|
||||
{"name": "move", "instructions": "Move slowly toward the door"}
|
||||
]
|
||||
```
|
||||
</ACTIONS>
|
||||
"""
|
||||
content = extract_actions_block(response)
|
||||
result = parse_actions_content(content)
|
||||
assert result is not None
|
||||
assert len(result) == 1
|
||||
assert result[0]["name"] == "move"
|
||||
assert "door" in result[0]["instructions"]
|
||||
|
||||
def test_decision_in_analysis_ignored(self):
|
||||
"""Test that DECISION blocks within ANALYSIS are ignored when extracting ACTIONS."""
|
||||
response = """
|
||||
<ANALYSIS>
|
||||
I need to decide on the approach <DECISION>cautious_approach</DECISION>.
|
||||
Also considering <DECISION>aggressive_approach</DECISION> as alternative.
|
||||
</ANALYSIS>
|
||||
<MESSAGE>
|
||||
Based on my analysis, here's the plan.
|
||||
</MESSAGE>
|
||||
<ACTIONS>
|
||||
```json
|
||||
[{"name": "proceed", "instructions": "Move forward"}]
|
||||
```
|
||||
</ACTIONS>
|
||||
"""
|
||||
content = extract_actions_block(response)
|
||||
result = parse_actions_content(content)
|
||||
assert result is not None
|
||||
assert len(result) == 1
|
||||
assert result[0]["name"] == "proceed"
|
||||
|
||||
def test_actions_and_decision_in_message(self):
|
||||
"""Test that both ACTIONS and DECISION can appear in MESSAGE after ANALYSIS."""
|
||||
response = """
|
||||
<ANALYSIS>
|
||||
Analyzing the scene with potential <DECISION>test</DECISION> and <ACTIONS>```json
|
||||
[{"name": "fake", "instructions": "fake"}]
|
||||
```</ACTIONS> blocks.
|
||||
</ANALYSIS>
|
||||
<MESSAGE>
|
||||
My decision: <DECISION>proceed_with_caution</DECISION>
|
||||
And here are the actions to take:
|
||||
<ACTIONS>
|
||||
```json
|
||||
[{"name": "real_action", "instructions": "Do this"}]
|
||||
```
|
||||
</ACTIONS>
|
||||
</MESSAGE>
|
||||
"""
|
||||
content = extract_actions_block(response)
|
||||
result = parse_actions_content(content)
|
||||
assert result is not None
|
||||
assert len(result) == 1
|
||||
assert result[0]["name"] == "real_action"
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Tests for clean_visible_response
|
||||
# ============================================================================
|
||||
|
||||
|
||||
class TestCleanVisibleResponse:
|
||||
"""Tests for the clean_visible_response function."""
|
||||
|
||||
def test_removes_actions_block(self):
|
||||
"""Test that ACTIONS blocks are removed."""
|
||||
text = """Here is my response.
|
||||
<ACTIONS>
|
||||
```json
|
||||
[{"name": "test"}]
|
||||
```
|
||||
</ACTIONS>
|
||||
More text after."""
|
||||
result = clean_visible_response(text)
|
||||
assert result == "Here is my response.\n\nMore text after."
|
||||
|
||||
def test_removes_decision_and_everything_after(self):
|
||||
"""Test that everything from DECISION tag onwards is removed."""
|
||||
text = """Here is my response.
|
||||
<DECISION>
|
||||
Choose option A
|
||||
</DECISION>
|
||||
This text should be removed too."""
|
||||
result = clean_visible_response(text)
|
||||
assert result == "Here is my response."
|
||||
|
||||
def test_removes_legacy_actions_block(self):
|
||||
"""Test that legacy ```actions``` blocks are removed."""
|
||||
text = """Here is my response.
|
||||
```actions
|
||||
some action data
|
||||
```
|
||||
More text after."""
|
||||
result = clean_visible_response(text)
|
||||
assert result == "Here is my response.\n\nMore text after."
|
||||
|
||||
def test_removes_actions_then_decision(self):
|
||||
"""Test removing both ACTIONS and DECISION blocks."""
|
||||
text = """Here is my response.
|
||||
<ACTIONS>
|
||||
```json
|
||||
[{"name": "test"}]
|
||||
```
|
||||
</ACTIONS>
|
||||
<DECISION>Everything from here onwards is removed</DECISION>"""
|
||||
result = clean_visible_response(text)
|
||||
assert result == "Here is my response."
|
||||
|
||||
def test_no_special_tags(self):
|
||||
"""Test text without special tags is unchanged."""
|
||||
text = "Just a message with no decision or actions."
|
||||
result = clean_visible_response(text)
|
||||
assert result == "Just a message with no decision or actions."
|
||||
|
||||
def test_case_insensitive(self):
|
||||
"""Test that tag matching is case insensitive."""
|
||||
text = """My response.
|
||||
<actions>
|
||||
```json
|
||||
[{"name": "test"}]
|
||||
```
|
||||
</actions>
|
||||
<decision>removed</decision>"""
|
||||
result = clean_visible_response(text)
|
||||
assert result == "My response."
|
||||
|
||||
def test_decision_without_closing_tag(self):
|
||||
"""Test that DECISION without closing tag removes everything after."""
|
||||
text = """Here is my response.
|
||||
<DECISION>
|
||||
This is my decision and all this text
|
||||
continues for many lines
|
||||
and should all be removed."""
|
||||
result = clean_visible_response(text)
|
||||
assert result == "Here is my response."
|
||||
|
||||
def test_multiple_actions_blocks(self):
|
||||
"""Test that multiple ACTIONS blocks are all removed."""
|
||||
text = """Start.
|
||||
<ACTIONS>
|
||||
```json
|
||||
[{"name": "first"}]
|
||||
```
|
||||
</ACTIONS>
|
||||
Middle.
|
||||
<ACTIONS>
|
||||
```json
|
||||
[{"name": "second"}]
|
||||
```
|
||||
</ACTIONS>
|
||||
End."""
|
||||
result = clean_visible_response(text)
|
||||
assert "Start." in result
|
||||
assert "Middle." in result
|
||||
assert "End." in result
|
||||
assert "<ACTIONS>" not in result
|
||||
assert "first" not in result
|
||||
assert "second" not in result
|
||||
Reference in New Issue
Block a user