Commit Graph

1661 Commits

Author SHA1 Message Date
Classic298
7cf6051a74 perf: resolve group membership once per folder listing instead of once per entry (#28810)
Listing a user's folders re-checks which entries they may still see, and it resolved their group membership again for every folder, then again inside the collection and note branches for every entry. A comment in that helper claims one membership fetch for the whole listing, but the caller invokes it once per folder, so the claim never held.

The listing now resolves membership once, and only when some folder actually carries entries, then threads it through the file, collection and note checks. Callers that do not supply it are unchanged and still resolve for themselves.

Measured with twenty folders holding six files, two knowledge bases and two notes each: 245 queries and ~145 ms before, 186 and ~117 ms after. The folders returned, and the entries the integrity pass writes back, are unchanged. That was checked against entries the caller owns, entries shared through a group, entries shared with nobody, another user's files, and an unrecognised entry type.
2026-08-19 11:16:29 -07:00
Classic298
2e7df54673 fix: surface attached chat references in <attached_files> (#28788)
A chat attached via the "+" menu or dropped from the sidebar references an
existing chat by id and carries no url. add_file_context() filtered on
`file.get('url')`, so the reference was dropped from <attached_files>
entirely and the model was never told it existed.

When the RAG file-context path is enabled the chat content still reaches
the model as <source> context, which masked this. With file_context
disabled that path is skipped, and get_attached_knowledge() only promotes
collection/note items into <attached_knowledge> - so an attached chat was
visible in the UI but invisible to the model, which then reported having
no chat attachments despite having a view_chat tool available.

Keep chat references and emit their id so the model can resolve them with
view_chat. The url attribute is now conditional, since a chat has none;
the id guard it replaces was dead once the filter guarantees a url or a
chat id.

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-19 11:15:22 -07:00
Classic298
4f98a5184f perf: resolve model-attached file access with a targeted query (#28802)
Checking whether a user may reach a file loaded and validated every workspace model that user can access, then scanned each model's knowledge list in Python for one file id. Folder listings run that check once per file, so opening a folder of twenty files rebuilt the whole accessible-model set twenty times, and the same check sits on every retrieval and download path.

The lookup now runs the other way round: the database returns the models that attach the file, and only those are access-checked. The text match on the metadata column is a prefilter and the knowledge entries still decide, so a file id that merely appears in a description grants nothing; file ids are server-generated uuids, so the match can only be too wide, never too narrow.

Measured with 500 accessible workspace models: a single check drops from 9 queries and ~20 ms to 6 and ~2.6 ms, and a twenty-file folder listing from 180 queries and ~680 ms to 120 and ~56 ms. A 72-case matrix over owner, public, direct-user and group grants, for both read and write, returns exactly what it returned before, and write still requires the model owner to own the file. The check also no longer writes to the database while answering a read-only question.
2026-08-19 11:07:43 -07:00
Classic298
dbf715cb63 perf: stop scanning every skill on each listing and chat turn (#28798)
Listing skills ran one database query per skill in the instance. A non-admin opening the list on a workspace with 500 skills issued over 500 queries, the paginated list re-resolved the caller's group membership once per row, and every chat message carrying a skill loaded every skill the user can read, full body and owner included, to use the two or three it actually referenced.

Skills now arrive already filtered: the owner-or-grant check runs in the query as an EXISTS subquery, the same way prompts and the search endpoints already do it, the per-item write flag uses the existing batch grant lookup, and the chat path asks only for the skill ids the request names.

Measured with 500 skills of which 3 are visible to the caller: 504 queries and ~300 ms before, 4 queries and ~2.6 ms after. The resulting set is unchanged for owner, public, direct-user, group and multi-grant entries, for both read and write.
2026-08-19 11:07:33 -07:00
Classic298
284da2ae49 perf: reuse the already loaded chat when assembling builtin tools (#28809)
Assembling the builtin tools for a chat message fetched the chat row a second time to answer one question: whether this is a note chat. The caller had loaded that same row a few lines earlier, from the same id in the same metadata dict, and had already evaluated the same predicate for its own note handling. So every message with builtin tools enabled read the whole conversation blob twice.

The caller now works the flag out once and passes it down. Tool assembly no longer touches a chat model at all, so the two files cannot drift apart when the shape of that metadata changes.

Measured with a stub request across five chat shapes, a note chat, a plain chat, an internal chat that is not a note, a chat id with no row behind it, and an unsaved chat id: the returned tool set is identical in every case and the query count drops from six to five. The note tools are still enabled for a note chat with the notes feature switched off, which is the only thing that predicate decides.
2026-08-19 11:06:49 -07:00
Classic298
21e390561d fix: revoke existing sessions when a password changes (#28725)
Changing a password left every other logged-in device working until the JWT expired on its own, up to four weeks with the default settings. The hardening docs already promise the opposite: with Redis configured a password change is supposed to put the user's tokens on the revocation list, but only sign-out and OIDC back-channel logout ever wrote to it.

Both password-change paths, self-service and an admin resetting someone's password, now stamp the per-user revocation marker that token validation already checks, so every session issued before the change stops working. The acting device is signed out as well and asked to sign in again, which is the safer default when the password is being changed precisely because the old one may be compromised. Without Redis nothing can be revoked, as before, and the backend now logs a warning saying so.

The marker is written through one shared helper, so its lifetime follows the configured JWT lifetime instead of a fixed 30 days and never expires at all when JWT_EXPIRES_IN disables expiry. Back-channel logout picks that up too, where a long or disabled JWT lifetime previously let the marker expire while the tokens it revoked were still valid. API keys keep working, they are separate credentials with their own lifecycle.

Discussed in #28647.
2026-08-17 13:56:29 -07:00
G30
88c55b86b1 feat: emit auth.login on SSO logins and attribute SSO logouts (#27619)
* feat: emit the auth.login event on SSO logins

* feat: attribute SSO logouts in the auth.logout event payload
2026-08-17 02:22:25 -06:00
Classic298
646a568ae6 fix: enforce global web search and image generation switches on the legacy function-calling path (#27669)
The legacy function-calling path acted on the client-supplied `features` dict after checking only the per-user permission, so a user who still held `features.web_search` or `features.image_generation` could keep triggering web searches and image generation after an administrator had switched those off instance-wide. The native function-calling path already gates the equivalent builtin tools on `web.search.enable` and `image_generation.enable` in `get_builtin_tools`, so the two paths disagreed and the admin-level switch did not actually stop the outbound provider calls it was turned off to stop.

Gate the legacy web search handler on `web.search.enable` at its call site, and gate `chat_image_generation_handler` on the two image switches internally. The image handler needs the check inside it because `image_generation.enable` and `images.edit.enable` are independent: editing stays available when generation is disabled, matching the `/images/generations` and `/images/edit` routes and the native `generate_image`/`edit_image` tools. The handler calls `image_generations`/`image_edits` directly and so bypasses the route guards, which is why the check has to live at the caller.

The "Creating image" status event moves below the new guard so a disabled configuration returns without leaving an unresolved progress indicator in the chat.
2026-08-17 02:15:05 -06:00
Timothy Jaeryang Baek
0b27fa5e87 refac 2026-08-17 00:57:57 -07:00
Timothy Jaeryang Baek
e968445812 refac 2026-08-17 00:43:47 -07:00
Timothy Jaeryang Baek
d799e81edb refac 2026-08-17 00:42:16 -07:00
Timothy Jaeryang Baek
f1a64ccfc2 refac 2026-08-17 00:35:12 -07:00
Classic298
189c14fc4d fix: match both JSON text spellings when searching serialised JSON columns (#28399)
Three searches LIKE against cast(json_col AS text), which means they have to match
bytes a JSON encoder wrote. Encoders disagree on non-ASCII: stdlib escapes it to
\uXXXX, orjson writes it raw. Which one produced a row depends on the codec in force
when it was written, so any single pattern finds only half the table.

models.py hard-codes the stdlib spelling, with a comment asserting SQLite stores
JSON via json.dumps(ensure_ascii=True). Model.meta is a JSONField, which has
serialised through JSONCodec since ENABLE_ORJSON was introduced, so on that setting
it stores raw UTF-8 and the escaped pattern matches nothing: non-ASCII workspace
model tag search is broken today. prompts.py and automations.py hard-code the
opposite spelling and miss rows written the other way.

json_text_variants returns both spellings a string can take inside serialised JSON,
collapsing to one for ASCII, and the three call sites OR over them. Rows written
under either setting are now found under either setting, which also covers a
database holding a mix of the two.

Case handling is unchanged. models.py keeps matching non-ASCII tags case-sensitively
on SQLite, whose LOWER() is ASCII-only and would not fold the stored text the way
str.lower() folds the tag. ASCII tags collapse to a single variant and take exactly
the query they took before.

Verified on SQLite across every combination of codec-that-wrote-the-row and
codec-the-app-is-running, for an ASCII and a CJK tag, over all three call sites: 24
of 24 match, against 12 of 24 before. Quoting still bounds whole-tag matches, so
searching "weather" does not match a row tagged "weathervane".

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-17 01:24:05 -06:00
G30
f5a5a434b9 fix: record an error state when a timer's chat completion raises (#27785) 2026-08-17 01:22:47 -06:00
Timothy Jaeryang Baek
b6dc70c93b refac 2026-08-17 00:16:07 -07:00
Timothy Jaeryang Baek
927ce0eae6 refac
Co-Authored-By: Classic298 <27028174+Classic298@users.noreply.github.com>
2026-08-16 23:58:56 -07:00
Classic298
805bfca5af feat: emit group events on OAuth group sync (#27657)
With ENABLE_OAUTH_GROUP_MANAGEMENT enabled, every SSO login reconciles the user's group membership against the IdP claims, adding and removing them from groups and, with ENABLE_OAUTH_GROUP_CREATION, creating groups that do not exist yet. None of it emitted an event, so the same membership change was observable when an admin made it through the UI or when it arrived over SCIM, but invisible when the IdP drove it. That is the path that changes membership most often.

Emits group.member_added and group.member_removed per membership transition and group.created for each auto-created group, using the same payload keys as the groups router. The member events are published only when the write returned a group, so a failed or no-op write emits nothing, and both loops already run only on an actual transition. update_user_groups takes the request so the events can be published; it has a single caller.
2026-08-17 00:50:47 -06:00
Timothy Jaeryang Baek
3e186abdd9 refac
Co-Authored-By: Classic298 <27028174+Classic298@users.noreply.github.com>
2026-08-16 23:41:58 -07:00
Classic298
cd9db21c52 refac: bind tool server cookies per connection (#28630)
The tool callable now takes its connection's cookie jar as a parameter, matching how its headers are already passed and how the terminal tool factory in the same module builds its callables.
2026-08-17 00:25:07 -06:00
Timothy Jaeryang Baek
3258330729 refac
Co-Authored-By: Classic298 <27028174+Classic298@users.noreply.github.com>
2026-08-16 22:56:19 -07:00
Timothy Jaeryang Baek
62fc436999 refac 2026-08-16 22:52:24 -07:00
Timothy Jaeryang Baek
d02b6a21fc refac 2026-08-14 00:55:21 -06:00
Timothy Jaeryang Baek
c755ef60c6 refac 2026-08-14 00:54:33 -06:00
Timothy Jaeryang Baek
a1579a01ff refac 2026-08-14 00:22:17 -06:00
Timothy Jaeryang Baek
fa94a5ab24 refac 2026-08-13 21:36:41 -06:00
Timothy Jaeryang Baek
653562d660 refac 2026-08-13 21:15:26 -06:00
Timothy Jaeryang Baek
a32a17965c refac 2026-08-13 21:06:51 -06:00
Timothy Jaeryang Baek
083e351441 refac 2026-08-13 21:02:17 -06:00
Timothy Jaeryang Baek
7d99b2716a refac 2026-08-13 19:59:11 -06:00
Timothy Jaeryang Baek
4465f52a3e refac 2026-08-13 18:13:43 -06:00
Timothy Jaeryang Baek
2649e3305c refac 2026-08-13 16:42:10 -06:00
Timothy Jaeryang Baek
85c3d0ae2f refac 2026-08-13 00:07:31 -06:00
G30
2ab0311b99 fix: reject COUNT rules without DTSTART so limited automations cannot run forever (#27781) 2026-08-12 01:08:55 -06:00
G30
e17dfae72e fix: enforce the image generation flag on the legacy chat feature path (#27759)
* fix: enforce the image generation flag on the legacy chat feature path

* fix: refresh the config store when image generation is disabled in admin settings

* fix: hide active feature pills when the feature is no longer available
2026-08-11 17:38:10 -06:00
Timothy Jaeryang Baek
865c80c160 refac 2026-08-11 01:17:19 -06:00
Timothy Jaeryang Baek
f0bfcd4097 refac 2026-08-11 01:15:05 -06:00
Classic298
934802e186 fix: enforce features.memories permission on the legacy memory context path (#27668)
Revoking a user's `features.memories` permission removed their access to the memories API and to the native function-calling memory tools, but their stored memories were still injected into the system context on the legacy function-calling path.

The branch in `process_chat_payload` only checked the client-supplied `features['memory']` flag plus the global `memories.system_context.enable` switch, with no user-permission check. `add_memory_context` did not compensate: it only checks `model_allows_memory`, which is a model capability rather than a permission, and the one call inside it that does check the permission (`query_memory`) has its 403 swallowed by a `try/except`, so `Memories.get_memories_by_user_id` and the neighbourhood scan still fed the system prompt.

Gate the branch with the same permission check the native path already performs in `get_builtin_tools`, matching the neighbouring `web_search` and `image_generation` branches.

Only the caller's own memories were injected into the caller's own context, so there was no cross-user exposure. The practical effect was that the permission toggle did not do what its name implies: an admin who revoked it still got memory content injected for that user.
2026-08-10 23:36:27 -06:00
Classic298
5c79ccc9e5 refactor: walk chat message history by map key (#28034)
`get_message_list` moves through `messages_map` by key but tracked each message's own `id` field, which the message body does not have to carry. Track the key instead.
2026-08-10 23:24:41 -06:00
Classic298
1f22cccd22 perf: stop formatting every exported log record twice under OTEL log export (#27840)
With ENABLE_OTEL and ENABLE_OTEL_LOGS set, InterceptHandler builds the message once for loguru and then hands the same LogRecord to the OpenTelemetry handler, whose _translate calls record.getMessage() a second time. That used to be free, because the message was already a finished f-string with nothing to substitute. Now that log calls pass lazy %-args, the second call re-runs the whole interpolation, so every exported record is formatted twice.

The two getMessage() calls on a 78 kB retrieval record:

    before  373.0 us
    after     0.1 us

Stamping the built message back onto the record makes the second call a plain string return. msg and args are both in OpenTelemetry's _RESERVED_ATTRS, so neither ever reaches the exported attributes. The isinstance guard matters: _translate exports a non-str msg such as the dicts routers/audio.py logs as a typed body rather than a string, so those records are left untouched, and they have no %-args to format twice anyway. Body, attributes and severity were compared against LoggingHandler._translate for str, dict, list, int, None, exception and exc_info records.
2026-08-10 23:13:52 -06:00
Timothy Jaeryang Baek
ce3c175e26 refac 2026-08-10 23:13:10 -06:00
Timothy Jaeryang Baek
c2107e5bb3 refac 2026-08-10 22:36:42 -06:00
Timothy Jaeryang Baek
5cecb7dbfa refac 2026-08-10 20:30:56 -06:00
Timothy Jaeryang Baek
5ec16e76e6 refac 2026-08-10 20:13:03 -06:00
Classic298
5462c02af0 fix: OIDC login fails when the provider adds a private JOSE header (#28065)
Logging in through CyberArk Identity dies at the callback with "Unsupported {'app_id'} in header" and the user sees "The email or password provided is incorrect". Any provider that puts a vendor-specific parameter in the ID token header hits this; CAS was already patched by name, CyberArk is the next one.

Authlib 1.7 verifies ID tokens with joserfc, which rejects header parameters it does not recognise. The old fix registered `client_id` so CAS would work, which only ever fixes one provider at a time. This turns off the unknown-header rejection instead, so any private header parameter is ignored rather than fatal. Signature verification, the algorithm allowlist, `crit` handling and value validation of registered headers all still run, so nothing that actually protects the token is relaxed.

Fixes #28062
2026-08-10 20:06:31 -06:00
Timothy Jaeryang Baek
b606e13da3 refac 2026-08-10 19:44:56 -06:00
Timothy Jaeryang Baek
11739a2de8 refac 2026-08-10 19:41:05 -06:00
Timothy Jaeryang Baek
d22bb6703f refac 2026-08-10 19:28:21 -06:00
Timothy Jaeryang Baek
ff74bfa6a1 refac 2026-08-10 19:25:26 -06:00
Classic298
d9e23b90c1 refac: share one folder write-access check across chat folder_id paths (#28366)
Chat creation and chat moves each carried their own copy of the same folder_id validation, resolving the folder and checking ownership and shared write access in slightly different ways. Both now call a single has_folder_write_access helper, which the chat-completions creation path uses as well, so ownership, inherited write grants and nonexistent or malformed ids behave identically everywhere a chat folder_id is set. The owner case also costs one query fewer than before.
2026-08-10 19:17:18 -06:00
Timothy Jaeryang Baek
a33fa05adc refac 2026-08-10 00:19:52 -06:00