c12a82930e
- **Per-Scene Agent Overrides** — Override agent configuration for a single scene without touching your global config. - **Message Revision History** — Browse and continue from previous regenerations / revisions. - **Scene Perspective Overrides** — Set distinct narrative perspectives for the player, NPCs, and narrator. - **Durable World State Snapshot** — The world state snapshot now persists and updates entities across refreshes. - **World State Highlights** — Characters, objects and places marked by the current world state snapshot are now clickable in the recent message(s) for additional detail. Plus 30 improvements and 17 bug fixes
2.5 KiB
Concurrent Requests (Experimental)
Concurrent requests is an experimental feature that allows certain LLM clients to process multiple requests simultaneously rather than one at a time.
What It Does
When enabled, operations that require multiple LLM queries (such as generating image prompts) will execute those queries in parallel instead of sequentially. This can significantly reduce the total time needed for these batch operations.
Currently, this feature is only used for visual prompt generation (creating prompts for image generation). It is not applied to regular conversation or narration tasks.
Supported Clients
Concurrent requests are available for the following hosted API clients:
It is also available for the local llama.cpp client. For llama.cpp, your llama-server must be started with enough parallel slots to handle more than one request at a time; otherwise the extra requests are simply queued and you will not see a speed-up.
Other local clients (KoboldCpp, etc.) do not support this feature.
How to Enable
For clients that support concurrent requests, you will see a toggle button in the client list. The button uses a parallel lines icon.
Click the button to enable concurrent requests. When enabled, the button will appear highlighted.
You can also enable this feature through the client's settings dialog under the Concurrency tab.
Important Considerations
!!! warning "Experimental Feature" This feature is experimental and may behave unpredictably in certain situations, particularly when rate limiting is in effect.
Rate Limiting: If you have rate limiting configured for the client, concurrent requests may interact with the rate limiter in unexpected ways. If you experience issues, consider disabling concurrent requests or adjusting your rate limit settings.
When to Use This Feature
Consider enabling concurrent requests if:
- You frequently use the visual/image generation features
- You want to reduce wait times during image prompt generation
- You are not experiencing rate limit issues with the API
You can safely leave this disabled if:
- You rarely use image generation features
- You have strict rate limits configured
- You experience any instability with the feature enabled