Chat search built each result row with ChatTitleIdResponse(**chat.model_dump(), ...), which recursively copies the entire chat blob per row only for the constructor to ignore everything except id, title and timestamps: a 60-row search page deep-copied up to 60 full conversations. The folder listing, archived and export endpoints and every single-chat response did the same dump-and-revalidate dance via ChatResponse(**chat.model_dump()).
Search rows are now built from the five fields the response actually has (the snippet helper receives the blob by reference as before), and all 18 ChatResponse constructions use ChatResponse.model_validate(chat, from_attributes=True), which reads the fields off the already-validated ChatModel without copying the blob.
Benchmark (~500 KB chat blob):
| metric | before | after |
| --- | --- | --- |
| search result row | 0.05 ms | 0.003 ms |
| ChatResponse construction | 0.05 ms | 0.003 ms |
| per search page (60 rows) | 3 ms | 0.2 ms |
Beyond CPU, each converted row also stops materializing a second full copy of the conversation in memory while the page is being built.
Functionally verified: both construction styles produce identical model_dump() output for ChatResponse (including defaulted fields absent on ChatModel) and for search rows including the snippet.
DELETE /api/v1/chats/{id} called stop_item_tasks(id) before checking the
caller's chat.delete permission or ownership of the target chat. An
authenticated user who knew another user's chat id could therefore cancel that
chat's in-flight generation (streaming response, title or tag generation) even
though the deletion was then rejected. The chat id is discoverable through
legitimate read-only access to a shared chat or folder.
Reorder the handler to authorize first (admin, or owner holding chat.delete) and
only then cancel tasks and delete, matching the dedicated task-stop endpoint.
Legitimate deletions are unchanged; an unauthorized caller now returns 404 or 401
before any cancellation. The duplicated tag-cleanup and event-publish blocks are
merged.
Co-authored-by: GabrielGomesAL <193945687+GabrielGomesAL@users.noreply.github.com>
`_normalize_token_expiry()` derived the session expiry from the access token alone, and that value is what `oauth_session.expires_at` stores and what both `get_oauth_token()` implementations check to decide whether to refresh five minutes ahead. Providers that issue a shorter-lived id_token than access token (Microsoft Entra ID: roughly 60 minutes against 75) therefore left a window where the session still looked valid while the id_token had already expired, so pipes and tools reading `__oauth_token__["id_token"]` forwarded a dead JWT and downstream services rejected it with 401.
The stored expiry is now capped at the id_token's `exp` claim whenever that JWT expires first, which moves the refresh ahead of the earliest expiring token in the set. This is applied in the single function every session write already passes through, so it covers both the SSO manager and the MCP client manager on their callback and refresh paths alike. Sessions without an id_token, with an opaque one, or with no `exp` claim are unaffected.
Fixes#27066
When an audio file is split into multiple chunks for transcription, transcribe() collected the per-chunk results with asyncio.as_completed(), which yields results in completion order rather than submission order. Whenever a later chunk finished transcribing before an earlier one, the assembled transcript was scrambled, for example the second half of a recording appearing before the first, and the stored file content plus everything downstream (file preview, full-context retrieval) read out of chronological order. This change awaits the chunk tasks with asyncio.gather() instead, which runs them just as concurrently but returns the results in the order the tasks were created, i.e. chunk_paths order. The existing error handling and chunk cleanup are unchanged: an HTTPException from a chunk is re-raised as is and any other error is wrapped in a 500. Fixes#27143
When a filter's outlet() modified the structured assistant output in place, the change was shown immediately but lost after reload. outlet_filter_handler built its outlet payload with a shallow reference to the message's output list from messages_map, so the filter mutated the stored baseline itself and the subsequent output comparison compared the object against itself, never detecting a change and never persisting it. The same aliasing corrupted originalContent for messages whose text lives only in output. Deepcopy the output when building the outlet payload so messages_map stays a pristine pre-filter baseline and the existing change detection persists outlet-modified output through the existing upsert path. Fixes#27017.
Add self-hosted OpenSERP as a web search engine option. OpenSERP
provides browser-rendered search across Google, Bing, Yandex, Baidu,
DuckDuckGo, and Ecosia with no API keys required.
- New module: retrieval/web/openserp.py (async, uses aiohttp session pool)
- Config: OPENSERP_BASE_URL env var (defaults to http://localhost:7070)
- Routing: search_web() dispatch for 'openserp' engine
- Follows existing patterns (searxng, brave)
Co-authored-by: crustopher-lgtm <crustopher-lgtm@users.noreply.github.com>
When the same model is selected multiple times in a side-by-side chat,
each response is created with a distinct modelIdx (0,1,2,3) that
identifies its column. The backend now owns message persistence, but it
built the assistant placeholders without modelIdx, so the field was
never saved. On reload MultiResponseMessages groups responses by
modelIdx and falls back to grouping by model id when modelIdx is
missing; with duplicate models that fallback lumps every response into
each column, so all columns render the first response (and show a bogus
'1/N' pager).
Send modelIdx with each message_ids entry from the frontend and persist
it on the assistant placeholders in both the new-chat and existing-chat
paths. The message_ids list is now forwarded for every send (not just
multi-model ones) so single-column regenerations in a duplicate-model
chat also keep their column identity across reloads.
The OAuth / OIDC section in Admin Settings > Authentication had no
enable/disable switch, unlike the LDAP section above it. Add one that
persists via the existing Save flow and actually gates OAuth sign-in,
mirroring how the LDAP toggle works.
- config: new ENABLE_OAUTH persistent config ('oauth.enable'), defaulting
to True so existing deployments with a provider configured keep working.
- oauth: expose ENABLE_OAUTH via the OAuth runtime config and reject the
login and callback handlers with 404 when it is disabled.
- /api/config: report no OAuth providers when disabled so the login page
hides the OAuth buttons (and cannot auto-redirect), without clearing the
admin's provider configuration.
- auths: expose ENABLE_OAUTH through the admin OAuth config get/update
endpoints (OAuthConfigForm + OAUTH_CONFIG_KEYS).
- Authentication.svelte: bind the OAuth / OIDC header Switch to the
persisted oauthConfig.ENABLE_OAUTH and collapse the section when off,
matching the LDAP header (size, weight, alignment).
With ENABLE_PERSISTENT_CONFIG=False the admin's model order is reset on every restart because ui.model_order_list falls back to its DEFAULT_CONFIG default, and unlike every other Models setting (DEFAULT_MODELS, DEFAULT_PINNED_MODELS, DEFAULT_MODEL_METADATA and DEFAULT_MODEL_PARAMS) that default was hardcoded to an empty list with no environment variable to source it from. This adds a MODEL_ORDER_LIST environment variable parsed as a JSON array using the same guarded pattern as the neighbouring DEFAULT_MODEL_METADATA and DEFAULT_MODEL_PARAMS defaults, falling back to an empty list on parse errors. Behaviour when the variable is unset is unchanged.
Fixes#27206
`_detect_text_encoding()` hands the complete file to `chardet.detect()`. chardet is pure Python and costs roughly 1.3 seconds per megabyte, so uploading a large non-UTF-8 text file stalls for seconds inside encoding detection alone. A 4 MiB Shift-JIS file spends 6.4 seconds there. The UTF-8 fast path above it means only non-UTF-8 files reach this, which in practice are exactly the CJK documents the surrounding code was written to handle, so the slow case and the case that matters are the same case.
Detection does not need the whole file. It needs the bytes that are actually not UTF-8, and `UnicodeDecodeError.start` from the fast-path decode already says where those begin, so this samples a 256 KiB window around that offset.
Two things make that safe rather than merely fast.
Centring the window on the first non-UTF-8 byte instead of the file head is what keeps the common case correct. A plain head sample makes chardet report ascii for a file that is ASCII for its first few hundred KiB and only turns CJK later, and the method then falls through to latin-1 instead of the right codec.
The window still cannot help when a stray byte, a pasted Windows-1252 artifact for example, sits hundreds of KiB ahead of the real payload: the sample is then almost pure ASCII and carries no signal. So when the sample holds almost no non-ASCII bytes and is a strict subset of the file, detection falls back to the whole buffer. That case pays the old cost, which is the right trade, because it is precisely the case where sampling would otherwise be wrong. Without this guard a Cyrillic document with a stray leading byte was detected as ISO-8859-1 rather than windows-1251, which is silent mojibake.
Measured, with the encoding returned identical in every case:
| file | before | after |
|---|---|---|
| shift_jis 4 MiB | 6402ms | 755ms |
| gb18030 4 MiB | 3199ms | 449ms |
| big5 4 MiB | 2926ms | 413ms |
| euc-jp 4 MiB | 2456ms | 413ms |
| euc-kr 4 MiB | 2382ms | 468ms |
| latin-1 4 MiB | 1902ms | 394ms |
| gb18030 1 MiB | 807ms | 376ms |
| ascii head then gb18030 tail | 533ms | 294ms |
| stray byte then cp1251 payload | 496ms | 1051ms |
| any UTF-8 file | 8ms | 0ms |
29 cases, all returning an identical encoding before and after: six encodings at 100 KiB, 1 MiB and 4 MiB, three layouts where the non-UTF-8 bytes only begin beyond the window, four where a stray byte is separated from the payload, plus plain UTF-8, UTF-8 CJK and an empty file. The stray-byte rows are slower than before because they scan twice, once over the window and once over the whole buffer. They are the pathological shape, and correctness wins there.
The residual time is now the decode-and-validate loop below, which walks the file once per candidate codec, and `_has_cjk_characters`, which is a per-character Python loop over the decoded text. Both are the same "full scan for a detection decision" pattern and could take a bounded prefix too. That is left alone here.
`alazy_load()` builds every BeautifulSoup tree inline in an async function, so a web search that pulls in ten pages stops the entire worker for the whole time it spends parsing. Nothing else on that worker runs during it: not other users' token streams, not health checks, not socket.io traffic. Parsing is CPU work and it belongs in a thread.
Measured over 37 real pages, 13.5 MiB total, with a 5ms ticker sampling event-loop lag:
| | wall | worst loop stall | ticker fired |
|---|---|---|---|
| inline, html.parser (today) | 1793.8ms | 1788.8ms | 1 time |
| offloaded, html.parser | 1872.9ms | 82.9ms | 88 times |
| inline, lxml | 1346.7ms | 1341.8ms | 1 time |
| offloaded, lxml | 1445.4ms | 37.0ms | 118 times |
Today the loop is not merely slow during a batch, it is gone: a 5ms timer fired exactly once across 1.8 seconds. After the change it fires normally and the worst single stall drops by a factor of 20 to 36. The cost is 4 to 7 percent more wall time for the batch itself, from the thread handoffs, which is the right trade for a server handling more than one user.
Three details behind the shape of the change:
`get_text()` is only 2 percent of the cost (34ms against 1706ms of parsing over the corpus), so the whole per-page unit moves into the thread rather than the parse alone. Splitting them measured worse on both axes.
The offload is per page, not per batch. Handing the whole batch to one thread measured worse than either (2081ms wall, 235ms worst stall), so the loop is yielded to between pages.
The metadata block in `alazy_load()` was a duplicate of the module-level `extract_metadata()`, field for field, and `lazy_load()` was already using the shared helper. The new helper calls it too, which is why the diff removes more lines than it adds. The `ascrape_all()` override goes with it: it was a verbatim copy of the inherited implementation and `alazy_load()` was its only caller, so anything still calling it now gets the identical parent method, which resolves `self._unpack_fetch_results` to the override this class keeps.
Verified by feeding the real loader a 37 page corpus and comparing every resulting Document against the implementation this replaces:
```
PASS one Document per url (37)
PASS every Document identical to the pre-change implementation (0 differ)
PASS parsing ran off the main thread
PASS event loop kept running during parsing (90 ticks)
```
Both `page_content` and `metadata` are byte-identical on all 37 pages. This is independent of the parser in use and composes with switching the default parser to lxml: that change makes the stalls shorter, this one takes them off the loop.
OpenAPI specs with circular schema references, such as Mealie's where Recipe and RecipeCategory reference each other through properties and array items, crashed convert_openapi_to_tool_payload with a RecursionError, so the tool server produced no specs and the integration never appeared in the model or tool selection. resolve_schema already had a visited-set guard against circular references, but the recursive calls for properties and items dropped the set, so cycles running through those edges were never detected. This threads the visited set through those calls and passes a per-path copy when following a $ref, so only true ancestor cycles are pruned to an empty schema while sibling references to the same schema still resolve fully. Fixes#27239.
Every page pulled in by web search and web RAG is parsed with BeautifulSoup's `html.parser`, a pure-Python parser. It is the slowest option bs4 offers, and it is being handed 300 KiB to 1.5 MiB documents, several per query. `SafeWebBaseLoader` inherits `default_parser = "html.parser"` from langchain's `WebBaseLoader` and never overrides it, so this is an upstream default carried by accident, not a decision anyone made for Open WebUI.
`default_parser` is the single chokepoint for both the sync `_scrape()` path and the async `ascrape_all()` path, so one `setdefault` covers everything and an explicit caller override still wins.
lxml is already in the tree as a transitive hard dependency of ddgs, python-pptx and unstructured, so nothing new enters the image and `uv.lock` already resolves it at 6.1.1. The pin makes it explicit and closes a latent failure: bs4's `"xml"` feature, already used for `.xml` URLs in `_unpack_fetch_results()`, requires lxml and would raise `FeatureNotFound` the day that transitive dependency moves.
## Benchmarks
37 real pages, 13.8 MiB of HTML, median of 5 runs each. The timed operation is `BeautifulSoup(html, parser)` plus `get_text()` plus `extract_metadata()`, which is exactly what the loader does per page. bs4 4.14.3, lxml 6.1.1, CPython 3.12.
| | html.parser | lxml | |
|---|---|---|---|
| 37 pages, 13.8 MiB total | 1611.0ms | 1151.3ms | 1.4x faster, 460ms saved |
Largest pages:
| page | size | html.parser | lxml | speedup |
|---|---|---|---|---|
| pypi.org/project/aiohttp/ | 1259 KiB | 243.75ms | 180.51ms | 1.4x |
| gnu.org/software/bash/manual/bash.html | 1017 KiB | 257.97ms | 178.99ms | 1.4x |
| rfc-editor.org/rfc/rfc9110.html | 1157 KiB | 205.94ms | 154.87ms | 1.3x |
| docs.aiohttp.org/en/stable/client_reference.html | 403 KiB | 108.62ms | 84.93ms | 1.3x |
| ollama.com/library | 779 KiB | 117.55ms | 73.64ms | 1.6x |
| theregister.com | 1052 KiB | 88.32ms | 60.12ms | 1.5x |
| kubernetes.io/docs/concepts/services-networking/service/ | 563 KiB | 72.18ms | 43.43ms | 1.7x |
| docs.python.org/3/library/socket.html | 301 KiB | 71.88ms | 49.04ms | 1.5x |
Ranges from 1.1x to 1.7x, and the win grows with page size. A ten result web search sheds roughly 125ms of parsing. Because the async path builds its soups inline in `_unpack_fetch_results()`, that is 125ms the event loop spends parsing HTML instead of serving other users' streams. Pages under about 10 KiB are marginally slower under lxml due to fixed setup cost, which is worth nothing either way.
## Output verification
The risk in changing parser is silently different extracted text, so that was measured rather than assumed. Across all 37 real pages:
- **Zero characters of text were lost.** Every diff opcode against html.parser output was an insertion. Not one page dropped content under lxml.
- 659 characters were added, all on one page (docs.docker.com), where an inline Alpine.js `@click` handler containing a regex confuses libxml2's attribute handling and leaks a 73-character JS fragment into the text nine times. That is 659 characters of script noise in 27,206 characters of extracted text, with no content affected.
- Metadata (`title`, `description`, `language`) was identical on 35 of 37 pages. The two exceptions are 141-byte Wikipedia bot-block stubs with no `<html>` element, where lxml's fragment auto-wrapping adds `language: "No language found."`. Both parsers extract the same text from them.
Large documents were checked separately because libxml2 carries internal size caps. A 12 MiB single text node, 12 MiB spread across 400k nodes, a 3 MiB attribute value and 50k sibling elements with a trailing marker all produced byte-identical text under both parsers, with no truncation.
Malformed markup was checked too. lxml and html.parser diverge on unterminated comments, bare CDATA and duplicated `<html>` elements, all cases where both parsers are guessing and neither is correct. None of those shapes appeared in the 37 page corpus.
`backend/open_webui/env.py:184` also uses `html.parser`, on the local CHANGELOG at import time. That is trivial input on a startup path and is deliberately left alone.
The migration to joserfc completed the job but left the old dependency pinned. `python-jose` now has zero imports anywhere in the backend: the only `jose` references left are `joserfc` in `utils/oauth.py`, and a repo-wide search for `from jose`, `import jose` or `python_jose` returns nothing outside the three pin files.
Removing it also removes `ecdsa` and `rsa` from the image, which were pulled in only by python-jose. `uv lock` confirms that: it drops exactly those three packages and nothing else, because google-auth 2.55 depends on cryptography and pyasn1-modules rather than rsa. That is worth having beyond the size saving, since `ecdsa` ships a documented Minerva-style timing side-channel in its P-256 signing path that upstream has declined to fix, so keeping it in the image means shipping a flagged crypto library that nothing calls.
Verified by blocking the `jose` module at import time and importing the backend anyway:
```
PASS import open_webui.utils.auth
PASS import open_webui.utils.oauth
PASS import open_webui.main
jose in sys.modules: False
PASS create_token/decode_token round trip
```
One user-visible consequence worth stating: Tools and Functions run in the same interpreter, so a third-party plugin that imports `jose` directly stops working after this. Nothing in Open WebUI itself does, and PyJWT remains a dependency, but a plugin relying on a library the application never declared for that purpose is the only thing this can break.
`uv.lock` was edited surgically rather than regenerated, to avoid the unrelated whole-file churn a newer uv version introduces. The result was diffed against real `uv lock` output and matches it exactly apart from that version's cosmetic fields.
The events:chat socket handler called the ownership-checked update for last_read_at, discarded the boolean it returns, and then cancelled the chat's pending timers regardless of the answer. cancel_timers_for_chat selected on the internal marker, the type, the parent chat id and the status, and never on the owner, so it matched rows belonging to any user. An authenticated user who knew another user's chat id could mark that chat read over their own socket session and silently cancel the owner's pending timers, and the owner got no notification: the scheduled action simply never fired.
The missing owner predicate also cut the other way in ordinary use. Because the query matched every timer sharing a parent chat id, one user reading a chat cancelled the timers of anyone else holding one on the same chat, so this was collateral damage as much as an attack.
cancel_timers_for_chat now requires a user_id and filters on it, which is the durable fix, and the socket handler returns early unless the ownership-checked update reports that the caller owns the chat. The parameter is required rather than defaulted so a later caller cannot reintroduce the unscoped query by omission. Both existing call sites already know the acting user. Timer rows are created with the same owner as the parent chat and the execution path already refuses to run one whose owner does not match, so scoping the cancellation the same way cannot strand a timer that would otherwise have fired.
One behaviour change worth noting: an administrator posting into another user's chat no longer cancels that user's chat.user_message timers, because the acting user is the administrator. The timer fires instead of being cancelled, which is the safe direction.
* refac: use MilvusClient instead of deprecated ORM-style PyMilvus APIs
PyMilvus 2.6 emits a PyMilvusDeprecationWarning for every ORM-style call (`connections.connect`, `utility.*`, `Collection` and its methods) and will remove those APIs in PyMilvus 3.1. Both Milvus backends still used them, so a running instance floods its logs with deprecation warnings during indexing and retrieval, and would break outright once PyMilvus 3.1 lands.
Both vector clients now go through `MilvusClient`:
- `milvus_multitenancy.py`: collection creation, index creation, has_collection, insert, search, query iteration, delete and reset.
- `milvus.py`: the remaining ORM calls in `query()` (`connections.connect`, `Collection(...).load()`, `Collection.query_iterator`), plus the now-unused `FieldSchema` import.
Behaviour is unchanged: same schema, same index parameters and the same two-step scalar-index fallback, same filter expressions, same result shapes. Verified against embedded Milvus (milvus-lite, pymilvus 2.6.14) with a functional harness over both clients: insert, get, query by string/int/bool metadata filters, vector search, tenant isolation, oversized-text truncation, delete by id and by filter, delete_collection and reset all return identical results before and after, while the deprecation warnings drop from 57 to 0 for the multi-tenancy client and from 16 to 0 for the standard one.
One Milvus Lite nuance worth recording: `MilvusClient` sends index build parameters (`M`, `efConstruction`, `nlist`) as flat keys rather than as a nested `params` blob. A Milvus server accepts both forms, Milvus Lite only reads the nested one, so those tuning values are ignored on Lite. `MilvusClient` offers no way to send the nested form, and `milvus.py` already built its index parameters this way, so both backends are now consistent.
Fixes#26978
* refac: correct the Milvus scalar-index comment
The comment claimed that embedded Milvus Lite requires an explicit scalar index type. It does not: Milvus Lite rejects `create_index` on a VARCHAR field outright ("create_index only supports vector fields"), for every index type and with or without a metric type, so neither the parameterless call nor the explicit INVERTED fallback can succeed there. Filtered queries on `resource_id` still work on Lite, just unindexed.
Only the accurate half is kept, which is the reason the parameterless call is deliberate rather than an omission.
The `create_automation` and `update_automation` builtin tools wrote straight to `Automations.insert` / `Automations.update_by_id`, skipping the limit checks that `/api/v1/automations/create` and `/api/v1/automations/{id}/update` run through `check_automation_limits`. A non-admin user could therefore ask the model to create automations indefinitely, ignoring `AUTOMATION_MAX_COUNT`, and could schedule them below `AUTOMATION_MIN_INTERVAL`, on both create and update.
Both tools now call the same `check_automation_limits` helper the routers use, so the limits and the admin bypass cannot drift between the chat path and the HTTP path. A rejection is returned to the model as a plain error message instead of raising. `update_automation` also gained the missing user lookup guard, since the helper needs the user's role.
The `automations.enable` toggle and the `features.automations` user permission were already enforced when the tool set is assembled, so they are unaffected.
Fixes#27121
Uvicorn's `--ws auto` selected its `websockets_impl` protocol on 0.41.0, which is built on `websockets.legacy`. That module raises `AssertionError` in `_drain_helper` during keepalive pings and kills the websocket connection. Each crash runs the Socket.IO `disconnect` handler and drops the session from `SESSION_POOL`, so every subsequent server-to-browser call fails. The most visible symptom is the Pyodide code execution tool, which reaches the browser through `sio.call('events', ...)` and returns `{"stderr": "Client session disconnected."}` on every run.
Uvicorn 0.50.0 changed `--ws auto` to select the sans-io implementation whenever websockets is installed, and deprecated the legacy one. Bumping the pin therefore fixes this on every launch path at once, without adding a `--ws` flag to the startup scripts. Doing nothing is not stable either: websockets is unpinned apart from uvicorn's own `>=13.0` floor, and `websockets.legacy` is removed outright in websockets 17, which turns the current AssertionError into an ImportError on a fresh install.
Bumping to 0.51.0 rather than the minimum 0.50.0 also picks up the sans-io keepalive pings added in 0.44.0, so raw websocket endpoints keep the idle-timeout behaviour they have today behind a reverse proxy. Uvicorn 0.51.0 drops colorama from its `standard` extra and raises the httptools floor to 0.8.0, which the lockfile already satisfies.
Verified on the bumped pin: the backend boots, `/health` returns 200, `--ws auto` resolves to `WebSocketsSansIOProtocol`, a Socket.IO client completes a websocket handshake against the running app, and a bidirectional `sio.call` round trip succeeds. The unit test suite reports an identical 2273 passed / 7 failed on 0.41.0 and 0.51.0, with the 7 failures unrelated to uvicorn.
Fixes#27550
`from datetime import datetime, time, timedelta` shadows the `time` module, so `RateLimitMixin._sync_wait_for_rate_limit` calls `datetime.time.sleep` and raises `AttributeError: type object 'datetime.time' has no attribute 'sleep'` whenever it actually has to wait.
Every synchronous loader path that paces requests hits this. `SafeFireCrawlLoader.lazy_load` calls the limiter directly, and Tavily, Microsoft Web IQ and Playwright reach it through `_safe_process_url_sync`. The exception is raised inside their per-URL `try`, so with `continue_on_failure=True` (the default) the URL is logged as a per-URL failure and dropped instead of being scraped. This is live by default: `WEB_LOADER_CONCURRENT_REQUESTS` is passed as `requests_per_second` and defaults to 10, so any URL whose predecessor finished within 100ms takes the sleep branch and is lost. Tavily and Microsoft Web IQ report it as "SSL verification failed", which points at the wrong cause.
`_wait_for_rate_limit` uses `asyncio.sleep` and is unaffected, but `SafeMicrosoftWebIQLoader.alazy_load` runs `lazy_load` in a threadpool, so its async entry point is affected too.
`datetime.time` is not used anywhere in the file, so importing the `time` module instead is enough.
The per-URL `continue` half of #26079 landed in 6f8221df5, which also added the `_sync_wait_for_rate_limit()` call to the Firecrawl loop. This makes that call work rather than throw.
Fixes#26079
Both routes read `chat_id` from the request body and passed it into `get_event_emitter` without checking the caller owns that chat. The emitter persists through `upsert_message_to_chat_by_id_and_message_id`, which resolves by primary key and takes no owner argument, so an invoked filter or action wrote into whichever chat the caller named. `/api/chat/completions` already performs this check; these two routes did not.
Adds `verify_chat_ownership`, called at the top of both handlers. It runs before the existing try block because the `except Exception` there catches HTTPException and would rewrite the 404 into a 400. Admins are exempt, matching the completions path, so deliberate cross-user operations keep working.
`local:` chat ids are allowed through: they are per-socket, the emitter suppresses database writes for them, and the socket emit targets the caller's own room. `channel:` chat ids are rejected instead. They reach the channel emitter, whose write only checks that the message belongs to the channel and never that the caller may write it, and the membership and write-access gate for channels exists solely on `/api/chat/completions`. No caller sends a `channel:` id to these two routes: the only frontend callers are in the regular chat UI, and the backend channel path dispatches through the completions handler.
Co-authored-by: manus-use <213290975+manus-use@users.noreply.github.com>
`SafePlaywrightURLLoader` opened a new Playwright page for every URL and never closed it, and it only closed the browser after the URL loop finished normally. Pages therefore piled up for the whole batch, and any early exit (a raised error with `continue_on_failure=False`, or the caller abandoning/cancelling the generator mid-search) skipped `browser.close()` entirely.
With `PLAYWRIGHT_WS_URL` pointing at a remote Playwright server this leaks sessions on that server: navigation and route timeouts on slow or bot-protected pages leave pages and browser connections open until the server is restarted, which degrades every later web search.
Both `lazy_load()` and `alazy_load()` now scope the page to the per-URL loop body and the browser to the whole loop using their context managers, so each page is closed as soon as its URL is done and the browser is closed on success, on failure, and on cancellation. Closing a page also disposes the context implicitly created by `new_page()`. Exception handling is unchanged: a close error raised while `continue_on_failure` is set is still caught, logged, and the loop continues.
Fixes#25880
When `RAG_DOCUMENT_LOADER_ENGINE` is set to `paddleocr_vl`, the dispatch branch in `Loader._get_loader` checked only the engine name and a non-empty token, so every uploaded file was handed to the PaddleOCR-VL loader regardless of its type. Text based uploads such as `.md`, `.txt` and `.csv` were base64 encoded and posted to the `/layout-parsing` endpoint tagged as PDFs, and the API rejected them with `422 Unprocessable Entity` ("PDFium: Data format error"), so those files never indexed at all.
The loader already knows which extensions it can handle: it tags images with `fileType: 1` and treats everything else as a PDF. That list is now a module level constant, and the dispatch branch gates on `['pdf'] + images`, the same way `mistral_ocr`, `datalab_marker`, `document_intelligence` and `mineru` already limit themselves. Deriving the gate from the loader's own list keeps the two in sync, so a file can never be admitted by the gate and then mislabelled as a PDF on the wire. Everything outside that set falls through to the default loader chain, so `.md` and `.txt` load as text, `.csv` through `CSVLoader`, `.docx` through `Docx2txtLoader`, and so on.
The branch also never checked `PADDLEOCR_VL_BASE_URL`. With the URL cleared, `PaddleOCRVLLoader` raised `ValueError` from its constructor and the upload failed outright instead of falling back. Both settings are now required for the branch to be taken, matching how the other engines guard their own configuration.
Fixes#24988Fixes#26759
The Socket.IO handshake and the terminal WebSocket route each reimplement JWT authentication instead of going through the HTTP dependency chain. Both verified that the token decoded, that it had not been revoked, and that the user row existed, but neither applied the role check that `get_verified_user` enforces on every HTTP route, so any role outside `user` and `admin` was accepted.
That splits authorization across two planes. Deactivating an account by setting its role to `pending` takes effect immediately over HTTP, which returns 401, while the same JWT still opens a WebSocket. Changing a role disconnects the account's live sockets but does not revoke its token, so the client simply reconnects and gets a fresh session. Until the token expires, four weeks by default, a deactivated account keeps its channel rooms and can still read and write any note it holds an access grant on through the collaborative document handlers.
Resolve the user once, in `get_verified_user_by_token`, and route both WebSocket entry points through it. The role set moves into `VERIFIED_USER_ROLES` so the HTTP and WebSocket gates cannot drift apart, which is the underlying cause rather than either call site on its own. This also replaces five copies of the decode, revocation check and user lookup sequence.
`user-join` now resolves the user instead of reusing the identity cached in `SESSION_POOL`, which costs one extra query per handshake. Gating on the cached role would make the authorization decision depend on every future role-mutation path remembering to tear down the session pool, and that is precisely the invariant that failed here.
