Commit Graph

261 Commits

Author SHA1 Message Date
Ylber Gashi
147c4230d9 Enhance GitHub Actions workflow for server app deployment (#35)
- Added a step to set up QEMU for multi-platform builds
- Specified target platforms for Docker build: linux/amd64 and linux/arm64.
- Corrected the COPY command in Dockerfile to use relative paths correctly.
2025-05-07 18:44:44 +02:00
Hakan Shehu
f22e6d43bf Use zod for config parsing and some folder restructure (#29) 2025-05-04 11:31:41 +02:00
Ylber Gashi
14bcb55ea0 Updated docker-compose configuration: (#28)
- changed PostgreSQL and Redis passwords
- replaced Redis service with Valkey
- added Mailpit for local SMTP testing, and update MinIO image version

Updated email service to remove SMTP user and password checks.
2025-05-04 10:45:55 +02:00
Hakan Shehu
4998a9d4b2 Refactor and improve node cleanups on delete (#24) 2025-05-02 10:32:38 +02:00
Hakan Shehu
b6c21ac3db Update packages (#23) 2025-05-01 23:19:31 +02:00
Hakan Shehu
00b91c3e3c Implement workspace delete (#21) 2025-05-01 19:48:47 +02:00
Hakan Shehu
b86ae4e5e4 Implement auth ip rate-limits as fastify plugin (#20) 2025-05-01 11:30:12 +02:00
Hakan Shehu
90c7a22685 Send client metadata as headers on each request (#19) 2025-05-01 10:57:24 +02:00
Hakan Shehu
7e5c31d7f8 Include immutable version and commit sha as part of server docker build (#18) 2025-05-01 10:41:10 +02:00
Hakan Shehu
a32058cdab Change server host 2025-04-30 14:24:42 +02:00
Hakan Shehu
412a396b28 Switch from express to fastify (#16)
* Switch from express to fastify

* Use zod validation for api routes

* Fix type in password reset routes

* Use string as data type for bigint because of serialization issues

* Fix use of user id instead of account id for some file operations
2025-04-30 11:07:58 +02:00
Hakan Shehu
915f688115 Make SMTP service optional 2025-04-28 19:40:18 +02:00
Hakan Shehu
1d739879c3 Implement mentions 2025-04-28 19:28:58 +02:00
Hakan Shehu
a0df904981 Update packages 2025-04-23 18:04:40 +02:00
Hakan Shehu
63437894bf Implement password reset 2025-04-23 14:33:23 +02:00
Hakan Shehu
0b347db97c Check ai enabled before scheduling embeddings 2025-04-23 10:34:03 +02:00
Hakan Shehu
4ca3785db4 Update server packages 2025-04-08 16:51:53 +02:00
Hakan Shehu
130524708d Update packages 2025-03-17 23:33:48 +01:00
Hakan Shehu
399f20b1e4 Merge pull request #14 from colanode/ai-assistant-rag
Vector embeddings and initial implementation of AI assistant RAG pipeline
2025-03-10 22:18:04 +01:00
Ylber Gashi
0505e1fbe7 Remove automatic assistant response job scheduling on node creation 2025-03-05 20:53:01 +01:00
Hakan Shehu
d095dbbd16 Add recurring jobs to check node and document embeddings 2025-03-03 23:07:50 +01:00
Ylber Gashi
e83b60b513 Simpler chunk summaries and metadata handling at response time
- Extract common AI-related utility functions into a new ai-utils.ts module
- Create a metadata handling library with fetchMetadataForContextItems
- Update chunking and embedding services to use new utility functions
- Improve context retrieval and scoring with recency and hybrid search weights
- Enhance chunk summarization with a more flexible prompt template
- Reorganize types for better type safety and modularity
2025-03-02 17:54:30 +01:00
Hakan Shehu
c4b9aa320a Minor refactoring 2025-02-27 22:30:00 +01:00
Ylber Gashi
c33fc4a024 Add summary column and enhance search vector generation for embeddings 2025-02-22 18:43:56 +01:00
Ylber Gashi
29c96f7dd6 Add support for specifying context nodes in AI assistant response 2025-02-21 22:49:00 +01:00
Ylber Gashi
93c9e26f4f Improve document metadata retrieval and prompt formatting 2025-02-21 16:39:32 +01:00
Ylber Gashi
415922d73f Enhance document metadata with database context and improve enrichment prompts 2025-02-21 16:00:52 +01:00
Ylber Gashi
45d43b1e95 minor refactoring on llm-prompts 2025-02-21 15:47:00 +01:00
Ylber Gashi
6f53f0bc1c Refactor assistant library with improved context and document formatting 2025-02-21 15:43:49 +01:00
Ylber Gashi
0d7a902836 Enhance context retrieval with author information and metadata
- Add author name and ID to document and node retrieval services
- Implement bulk author fetching for more efficient metadata retrieval
- Update search result types to include created by information
- Improve context formatting in assistant response generation
- Modify metadata handling to include more detailed author information
2025-02-21 15:30:29 +01:00
Hakan Shehu
abeda565f0 Reinstall diff package in server because of some problems in production build 2025-02-21 14:45:22 +01:00
Ylber Gashi
320a9a0dcd Merge branch 'main' of https://github.com/colanode/colanode into ai-assistant-rag 2025-02-21 11:54:49 +01:00
Ylber Gashi
00b0c116b5 Add LangGraph, Google GenAI, and Langfuse dependencies 2025-02-21 11:54:03 +01:00
Ylber Gashi
00162ee06e Refactor assistant response job and extract core assistant logic
- Move assistant response chain and related functions to a dedicated `assistant.ts` library
- Simplify `assistant-response.ts` by delegating complex logic to the new library
- Create type definitions for assistant chain state and related types
- Improve modularity and separation of concerns in AI assistant implementation
- Remove deprecated `ChunkingService` and replace with functional `chunkText` approach
2025-02-21 11:52:59 +01:00
Hakan Shehu
c1bbf80909 Update some pending packages 2025-02-21 10:28:21 +01:00
Ylber Gashi
913c2f05cf Merge branch 'main' of https://github.com/colanode/colanode into ai-assistant-rag 2025-02-20 22:21:22 +01:00
Ylber Gashi
16b78a0e02 Refactor assistant response generation and context retrieval
- Simplify response chain graph by removing unnecessary database context steps
- Improve context formatting and citation generation
- Enhance node text extraction for records with field-level details
- Optimize chunking service to handle node metadata more flexibly
- Update LLM service prompts for more precise context generation
2025-02-20 22:16:25 +01:00
Hakan Shehu
2b74c96e57 Remove unused packages in server 2025-02-20 19:08:46 +01:00
Hakan Shehu
8aa1f5a05f Create welcome page document for new workspaces 2025-02-20 15:48:51 +01:00
Hakan Shehu
bab5df2000 Add indexes in node_paths and users tables 2025-02-20 15:38:39 +01:00
Hakan Shehu
81dd328d4b Store all uploads in a special table in server 2025-02-20 08:55:25 +01:00
Hakan Shehu
3a276d79d4 Update packages 2025-02-19 22:13:02 +01:00
Ylber Gashi
6469255587 Refactored text extraction to return structured data (name/attributes), allowing for more context. Separated node/document embedding delay configurations. Updated services to use new extraction method and improved optional text handling. 2025-02-19 19:42:36 +01:00
Ylber Gashi
cce2d41f8d Merge branch 'restructure' of https://github.com/colanode/colanode into ai-assistant-rag 2025-02-19 19:38:33 +01:00
Ylber Gashi
ee111a845f Enhance assistant response handling and node type support
- Update assistant response job to handle optional citations
- Modify node creation to trigger assistant response for question messages
- Add new message subtypes: 'question' and 'answer'
- Refactor node embedding scheduling to skip embedding for message nodes
- Integrate Langfuse callback for LLM service tracing
- Update SQL vector similarity queries for embedding retrieval
2025-02-19 19:16:10 +01:00
Hakan Shehu
a0eaf40735 Fix an issue with node sync 2025-02-19 15:17:48 +01:00
Hakan Shehu
0c2fb121c9 Refactor node updates 2025-02-19 13:14:57 +01:00
Hakan Shehu
25fb7ac46a Refactor documents and text indexing 2025-02-19 11:23:03 +01:00
Ylber Gashi
a7eda1f746 Update embedding configuration and scheduling for documents and nodes 2025-02-18 19:54:52 +01:00
Ylber Gashi
9ea780813f Refactor node embedding scheduling and clean-up with configuration updates 2025-02-18 19:51:14 +01:00