Ylber Gashi
6378c55a16
Initial implementation for notion import
2025-05-01 20:42:45 +02:00
Ylber Gashi
81b8e8030b
implementation of deep search assistant
2025-05-01 11:17:59 +02:00
Hakan Shehu
399f20b1e4
Merge pull request #14 from colanode/ai-assistant-rag
...
Vector embeddings and initial implementation of AI assistant RAG pipeline
2025-03-10 22:18:04 +01:00
Ylber Gashi
0505e1fbe7
Remove automatic assistant response job scheduling on node creation
2025-03-05 20:53:01 +01:00
Hakan Shehu
d095dbbd16
Add recurring jobs to check node and document embeddings
2025-03-03 23:07:50 +01:00
Ylber Gashi
e83b60b513
Simpler chunk summaries and metadata handling at response time
...
- Extract common AI-related utility functions into a new ai-utils.ts module
- Create a metadata handling library with fetchMetadataForContextItems
- Update chunking and embedding services to use new utility functions
- Improve context retrieval and scoring with recency and hybrid search weights
- Enhance chunk summarization with a more flexible prompt template
- Reorganize types for better type safety and modularity
2025-03-02 17:54:30 +01:00
Hakan Shehu
c4b9aa320a
Minor refactoring
2025-02-27 22:30:00 +01:00
Ylber Gashi
c33fc4a024
Add summary column and enhance search vector generation for embeddings
2025-02-22 18:43:56 +01:00
Ylber Gashi
29c96f7dd6
Add support for specifying context nodes in AI assistant response
2025-02-21 22:49:00 +01:00
Ylber Gashi
93c9e26f4f
Improve document metadata retrieval and prompt formatting
2025-02-21 16:39:32 +01:00
Ylber Gashi
415922d73f
Enhance document metadata with database context and improve enrichment prompts
2025-02-21 16:00:52 +01:00
Ylber Gashi
45d43b1e95
minor refactoring on llm-prompts
2025-02-21 15:47:00 +01:00
Ylber Gashi
6f53f0bc1c
Refactor assistant library with improved context and document formatting
2025-02-21 15:43:49 +01:00
Ylber Gashi
0d7a902836
Enhance context retrieval with author information and metadata
...
- Add author name and ID to document and node retrieval services
- Implement bulk author fetching for more efficient metadata retrieval
- Update search result types to include created by information
- Improve context formatting in assistant response generation
- Modify metadata handling to include more detailed author information
2025-02-21 15:30:29 +01:00
Hakan Shehu
abeda565f0
Reinstall diff package in server because of some problems in production build
2025-02-21 14:45:22 +01:00
Ylber Gashi
320a9a0dcd
Merge branch 'main' of https://github.com/colanode/colanode into ai-assistant-rag
2025-02-21 11:54:49 +01:00
Ylber Gashi
00b0c116b5
Add LangGraph, Google GenAI, and Langfuse dependencies
2025-02-21 11:54:03 +01:00
Ylber Gashi
00162ee06e
Refactor assistant response job and extract core assistant logic
...
- Move assistant response chain and related functions to a dedicated `assistant.ts` library
- Simplify `assistant-response.ts` by delegating complex logic to the new library
- Create type definitions for assistant chain state and related types
- Improve modularity and separation of concerns in AI assistant implementation
- Remove deprecated `ChunkingService` and replace with functional `chunkText` approach
2025-02-21 11:52:59 +01:00
Hakan Shehu
c1bbf80909
Update some pending packages
2025-02-21 10:28:21 +01:00
Ylber Gashi
913c2f05cf
Merge branch 'main' of https://github.com/colanode/colanode into ai-assistant-rag
2025-02-20 22:21:22 +01:00
Ylber Gashi
16b78a0e02
Refactor assistant response generation and context retrieval
...
- Simplify response chain graph by removing unnecessary database context steps
- Improve context formatting and citation generation
- Enhance node text extraction for records with field-level details
- Optimize chunking service to handle node metadata more flexibly
- Update LLM service prompts for more precise context generation
2025-02-20 22:16:25 +01:00
Hakan Shehu
2b74c96e57
Remove unused packages in server
2025-02-20 19:08:46 +01:00
Hakan Shehu
8aa1f5a05f
Create welcome page document for new workspaces
2025-02-20 15:48:51 +01:00
Hakan Shehu
bab5df2000
Add indexes in node_paths and users tables
2025-02-20 15:38:39 +01:00
Hakan Shehu
81dd328d4b
Store all uploads in a special table in server
2025-02-20 08:55:25 +01:00
Hakan Shehu
88f6f2d2f2
Disable unimplemented folder layouts
2025-02-20 08:14:54 +01:00
Hakan Shehu
21b0b0cd11
Move 'ms' package into desktop
2025-02-20 00:35:54 +01:00
Hakan Shehu
e2466ba1e5
Fix document updates list query handler
2025-02-19 22:15:27 +01:00
Hakan Shehu
3a276d79d4
Update packages
2025-02-19 22:13:02 +01:00
Hakan Shehu
f7a1388536
Rename some pending entries to nodes
2025-02-19 19:50:43 +01:00
Ylber Gashi
6469255587
Refactored text extraction to return structured data (name/attributes), allowing for more context. Separated node/document embedding delay configurations. Updated services to use new extraction method and improved optional text handling.
2025-02-19 19:42:36 +01:00
Ylber Gashi
cce2d41f8d
Merge branch 'restructure' of https://github.com/colanode/colanode into ai-assistant-rag
2025-02-19 19:38:33 +01:00
Ylber Gashi
ee111a845f
Enhance assistant response handling and node type support
...
- Update assistant response job to handle optional citations
- Modify node creation to trigger assistant response for question messages
- Add new message subtypes: 'question' and 'answer'
- Refactor node embedding scheduling to skip embedding for message nodes
- Integrate Langfuse callback for LLM service tracing
- Update SQL vector similarity queries for embedding retrieval
2025-02-19 19:16:10 +01:00
Hakan Shehu
f9a72f49fd
Implement message container
2025-02-19 19:11:46 +01:00
Hakan Shehu
e162258ca7
Fix file container tab thumbnail
2025-02-19 16:09:54 +01:00
Hakan Shehu
1e5c88844c
Fix file thumbnail classes
2025-02-19 16:07:47 +01:00
Hakan Shehu
35e77ea906
Clean all data on desktop start for the new version
2025-02-19 16:06:49 +01:00
Hakan Shehu
a0eaf40735
Fix an issue with node sync
2025-02-19 15:17:48 +01:00
Hakan Shehu
0c2fb121c9
Refactor node updates
2025-02-19 13:14:57 +01:00
Hakan Shehu
25fb7ac46a
Refactor documents and text indexing
2025-02-19 11:23:03 +01:00
Ylber Gashi
a7eda1f746
Update embedding configuration and scheduling for documents and nodes
2025-02-18 19:54:52 +01:00
Ylber Gashi
9ea780813f
Refactor node embedding scheduling and clean-up with configuration updates
2025-02-18 19:51:14 +01:00
Ylber Gashi
2ba0d9e6be
Refactor chunking and embedding services with improved metadata handling and code clarity
2025-02-18 19:33:38 +01:00
Hakan Shehu
18b15e09c3
Add document update validation and revert
2025-02-18 11:23:36 +01:00
Hakan Shehu
99b3189fc3
Refactor node reaction permission check
2025-02-18 10:00:39 +01:00
Ylber Gashi
a0a160cc8f
initial version of using records/database filtering as context for assistant
2025-02-18 00:03:18 +01:00
Ylber Gashi
1217ee487b
Refactor assistant response handler with LangGraph-based response generation workflow
2025-02-17 23:43:03 +01:00
Ylber Gashi
40d094869f
Add user-based access control to document and node retrieval services
2025-02-17 16:33:39 +01:00
Ylber Gashi
2509ad5d75
Refactor LLM service with improved context generation for chunking
2025-02-17 15:13:51 +01:00
Ylber Gashi
4750c241a4
Enhance chunking service with comprehensive metadata retrieval and context enrichment
2025-02-17 14:13:20 +01:00
