Commit Graph

10 Commits

Author SHA1 Message Date
Surya Prashanth
b4ee9fb2d5 [SILO-149] feat: re-design issue<->pr automation (#2992)
- extract all plane issues from PR title and description
- for PR automation, only consider issues wrapped in '[]'
2025-05-19 15:33:39 +05:30
Saurabh Kumar
6587284555 [SILO-134] Implement new OAuth flow for GitHub integration (#3135)
* [SILO-134] RUNWAY Implement new OAuth flow for Github integration (#3089)

* oauth helper service + db helpers + modify plane sdk to accomodate oauth token

* fix indentation + create util for remove undefined in object

* add fields to workspace credential for oauth adapt

* get oauth token helper method

* auth callback flow changes to add plane oauth flow for github integration

* integrate github user and auth callback with oauth

* add user callback support with oauth for github integration

* remove token from cache on app disconnection

* Get Connection Details refactor change

* remove unwanted logs

* add missing translations

* replace plane app client id and secret fetching from env to db

* fix app get from db

* refactoring and comments resolved

* rename installation_id to plane_installation_id

* use db query methods and complete the app secret fetch flow

* remove unwanted logs and send missing values in update creds method

* move plane app db call just before oauth token call + models export in init

* Fix ssl reject error for pg

* add NODE_ENV option for ssl db config + management command to reset app_secrets + iteration fix

* fix oauth rate limit cache key breaking

* Fix redirect uri issue

* Fix pull request behaviour

* remove NODE_ENV check in DB connection
2025-05-09 18:44:35 +05:30
Saurabh Kumar
6f718e2859 [SILO-183] Fix state mapping for PR Automation Github & Gitlab (#3084)
* change pr behaviour method to use entityconnections for right state mapping

* renames and sanitize + modify test for PR automation state update

* remove console logs

---------

Co-authored-by: Surya Prashanth <prashantsurya002@gmail.com>
2025-04-30 14:57:56 +05:30
Henit Chobisa
0c6d4610bf [SILO-137] fix: refactored pull-request handler into common behaviour (#2802)
* fix: refactored pull-request handler into common behaviour

* fix: refactored gitlab integration to use common pr behaviour

* chore: added test to support the pull request behaviour

* chore: transformed to composition from inheritance

* fix: PR comments

* fix: extended type annotations and behaviour
2025-04-11 18:58:52 +05:30
Surya Prashanth
13ccb63847 chore: use winston logger with all logs for tracing (#2877) 2025-04-04 20:10:22 +05:30
Saurabh Kumar
207e85e3a1 [SILO] : Importer and integrations db migration changes (#2363)
* `silo` refactoring changes  (#2304)

* add sdk and services for workspace connection and import jobs

* rename service

* move worker folder to root + remove controller folder

* remove controller folder

* add external api service in silo

* fix params in service

* fix params in service

* fix services + remove projectid in job endpoints

* chore: added generic config types for job store (#2158)

* chore: made import job as generic

* chore: added generic parameter inside the job service

* chore: added generic parameter inside the job service

* feat: remove projectId from jobs endpoint

* chore: modified job store for new services

* chore: revert go.mod.sum

* chore: spaces

* gitlab auth callback

* github controllers update with plane api

* create api client fix

* base integration importer store

* remove junk store

* silo external api token middleware + plane api to generate user tokens

* add ahmac permission method in apiserver + ahmac sign flow on silo

* remove junk

* getapi client remove unwanted options

* modify api route for silo access + github gitlab and slack controller clean up

* types modify + jira importer cleanup

* fix: nodemon setup

* make getAPIClient as a singleton

* chore: added new definitions in base-import-worker

* chore: modified jira migrator to adhere to the modified types

* feat: modified base migrator to adhere new types

* chore: modified jira-server-migrator to adhere new changes

* chore: modified asana importer to adhere to new types

* fix: migrated importer controllers for new schema

* fix: build

* chore: asana and linear controller

* feat: updated job controller for new schema changes

* chore: added permissions

* integrations api migrations changes (#2193)

* chore: added workspace_slug inside job data

* replaced ahmac with hmac implementation

* add new service base api view for microservices

* fix: apiserver changes and base view

* feat: added changes to services

* feat: modified engine controller for using api services

* feat: modified UI stores for modified jira importer

* remove-log

* resolve hmac issue + silo endpoint url fixes

* add silo target access token rotation in get

* integration ui migration changes

* revert changes in il8n

* fix crud for api

* fix crud for api

* fix: batch lock config

* fix silo bundler issue (#2233)

* chore: combined common dashboard components for silo (#2166)

* chore: added new types to job store

* feat: updated job store dependencies for new params

* feat: created common components for importers dashboard

* feat: modified jira importer to comply with new common components

* feat: modified jira server dashboard to comply with the new component changes

* feat: modified asana to comply with the common dashboard changes

* fix: linear to comply with the new changes

* reset job store

* feat: migrated importers to new schema

* fix: removed unused code

* fix: base job importer model and serializer

* fix-lint-errors

---------

Co-authored-by: Saurabh Kumar <70131915+Saurabhkmr98@users.noreply.github.com>

* fix api serialisers

* fix: stabilized all integrations and jira importer

* fix: linear, asana and jira-server importers

* fix: reinstantiated hmac permission

* chore: removed the db folder for silo

* feat: added types for integration config

* feat: added check for supported integration in silo and web

* feat: added self hosted and cloud message differences

* add en translations for importers and jira importer (#2264)

* feat: Added language support for integrations (#2277)

* feat: added translations for github, slack and gitlab integrations

* fix: slack integration project modal

* fix: gitlab organization disconnect

* fix: silo build fixes

* feat: added more language to integration language support

* fix: review

* feat: modified caddy and compose for silo selfhosted changes (#2276)

* silo - fixes for keys and integrations form reset (#2283)

* fix hardcoded values for integration importer keys

* reset state after integration form submit

* fix missing key integration

* fix gitlab connection update

* fix: added slack images being updated while creating issues (#2288)

Co-authored-by: Saurabh Kumar <70131915+Saurabhkmr98@users.noreply.github.com>

* silo - fix integration lang support (#2291)

* fix hardcoded values for integration importer keys

* reset state after integration form submit

* fix missing translation key integration

* fix gitlab connection update

* fix integration lang support

* fix integration lang support

---------

Co-authored-by: Saurabhkmr98 <saurabhkapur73@gmail.com>
Co-authored-by: Saurabh Kumar <70131915+Saurabhkmr98@users.noreply.github.com>

* [SILO-71] flatfile importer v1 (#2156)

* feat: initialized CSV importer

* feat: added flatfile components

* feat: setup workbook and listener for csv importer

* feat: added store structure for flatfile importer

* chore: translated csv importer to flatfile importer

* feat: setup listener

* fix: event listener for creating agent

* feat: added flatfile importer skeleton

* feat: added flatfile etl logic for pulling and transforming data

* feat: added flatfile controller and migrator

* feat: curated flatfile root store and importer UI

* feat: initialized flatfile migrator

* fix: flatfile

* feat: added transformers for cycles and issues

* feat: added mapping and transformation for flatfile

* feat: added assignee field for fixing migrator bugs

* fix:build

* feat: added metrics inside the data import task'

* fix:workbook

* feat: added validation of email and date for flatfile

* add sdk and services for workspace connection and import jobs

* rename service

* move worker folder to root + remove controller folder

* remove controller folder

* feat: added modules and issue types inside flatfile

* fix: build

* fix:response-handler

* fix:errors

* fix:deleted-file

* fix: readded jobs route

* add external api service in silo

* fix params in service

* fix params in service

* fix services + remove projectid in job endpoints

* chore: added generic config types for job store (#2158)

* chore: made import job as generic

* chore: added generic parameter inside the job service

* chore: added generic parameter inside the job service

* feat: remove projectId from jobs endpoint

* chore: modified job store for new services

* chore: revert go.mod.sum

* chore: spaces

* gitlab auth callback

* github controllers update with plane api

* create api client fix

* base integration importer store

* remove junk store

* silo external api token middleware + plane api to generate user tokens

* add ahmac permission method in apiserver + ahmac sign flow on silo

* remove junk

* getapi client remove unwanted options

* modify api route for silo access + github gitlab and slack controller clean up

* types modify + jira importer cleanup

* fix: nodemon setup

* make getAPIClient as a singleton

* chore: added new definitions in base-import-worker

* chore: modified jira migrator to adhere to the modified types

* feat: modified base migrator to adhere new types

* chore: modified jira-server-migrator to adhere new changes

* chore: modified asana importer to adhere to new types

* fix: migrated importer controllers for new schema

* fix: build

* chore: asana and linear controller

* feat: updated job controller for new schema changes

* chore: added permissions

* integrations api migrations changes (#2193)

* chore: added workspace_slug inside job data

* replaced ahmac with hmac implementation

* add new service base api view for microservices

* fix: apiserver changes and base view

* feat: added changes to services

* feat: modified engine controller for using api services

* feat: modified UI stores for modified jira importer

* remove-log

* resolve hmac issue + silo endpoint url fixes

* add silo target access token rotation in get

* integration ui migration changes

* revert changes in il8n

* fix crud for api

* fix crud for api

* fix: batch lock config

* fix silo bundler issue (#2233)

* chore: combined common dashboard components for silo (#2166)

* chore: added new types to job store

* feat: updated job store dependencies for new params

* feat: created common components for importers dashboard

* feat: modified jira importer to comply with new common components

* feat: modified jira server dashboard to comply with the new component changes

* feat: modified asana to comply with the common dashboard changes

* fix: linear to comply with the new changes

* reset job store

* feat: migrated importers to new schema

* fix: removed unused code

* fix: base job importer model and serializer

* fix-lint-errors

---------

Co-authored-by: Saurabh Kumar <70131915+Saurabhkmr98@users.noreply.github.com>

* fix api serialisers

* fix: stabilized all integrations and jira importer

* fix: linear, asana and jira-server importers

* fix: reinstantiated hmac permission

* chore: removed the db folder for silo

* feat: added types for integration config

* feat: added check for supported integration in silo and web

* feat: added self hosted and cloud message differences

* add en translations for importers and jira importer (#2264)

* feat: Added language support for integrations (#2277)

* feat: added translations for github, slack and gitlab integrations

* fix: slack integration project modal

* fix: gitlab organization disconnect

* fix: silo build fixes

* feat: added more language to integration language support

* fix: review

* feat: modified caddy and compose for silo selfhosted changes (#2276)

* silo - fixes for keys and integrations form reset (#2283)

* fix hardcoded values for integration importer keys

* reset state after integration form submit

* fix missing key integration

* fix gitlab connection update

* fix: added slack images being updated while creating issues (#2288)

Co-authored-by: Saurabh Kumar <70131915+Saurabhkmr98@users.noreply.github.com>

* silo - fix integration lang support (#2291)

* fix hardcoded values for integration importer keys

* reset state after integration form submit

* fix missing translation key integration

* fix gitlab connection update

* fix integration lang support

* fix integration lang support

* feat: added controllers as default exports

* feat: modified flatfile worker and controller to adhere new changes

* feat: modified flatfile importer to ahere silo api migration

* fix: source project and workspace hide props

---------

Co-authored-by: Saurabhkmr98 <saurabhkapur73@gmail.com>
Co-authored-by: Saurabh Kumar <70131915+Saurabhkmr98@users.noreply.github.com>

* add fuzzy find in state and priority mapping (#2310)

Co-authored-by: Saurabhkmr98 <saurabhkapur73@gmail.com>
Co-authored-by: sriram veeraghanta <veeraghanta.sriram@gmail.com>

* fix: build

* chore: moved flatfile agent to external

* chore:build-fixes

* fix: credential breaking for slack

* fix: credential route for api

* SILO - chore: Importers translation (#2352)

* add en translations for importers and jira importer

* add en translations for all the importers

* add importers language translation changes

* chore silo importer translation

* chore silo importer translation

* fix: env removing trailing slashes

* fix: trailing slashed env

* fix duplicate entity connection issue (#2362)

---------

Co-authored-by: Henit Chobisa <chobisa.henit@gmail.com>
Co-authored-by: sriram veeraghanta <veeraghanta.sriram@gmail.com>
2025-01-31 20:18:17 +05:30
Saurabh Kumar
fab70fb63f [SILO-75] chore: Silo error handling and sentry configuration (#2105)
* chore: global error handler and code cleanup

* chore: global error handler and code cleanup

* sentry config setup

* fix sentry source map issue + workers error handling

* serve cjs on start

* remove --source-map serve on silo start

* sentry release version via docker config + remove next and add response handler
2025-01-07 20:37:23 +05:30
sriram veeraghanta
9de132e2ed chore: Lint fixes (#1885)
* fix: lint related changes and packaging fixes

* adding color validations

* fix: linting fixes

* fix: build errors
2024-12-06 17:28:32 +05:30
guru_sainath
b8f78807d2 chore: updated the dynamic base path in silo (#1835)
* fix: silo base path improvements

* chore: added health check controller

---------

Co-authored-by: sriram veeraghanta <veeraghanta.sriram@gmail.com>
2024-11-29 13:02:38 +05:30
guru_sainath
1064e4fcdc Feat: Jira, Linear integration and GitHub importer (#1400)
* fix: silo service initial setup

* feat: moved controllers inside runner

* feat: created core definations for apps and context

* feat: added interfaces required for apps

* feat: moved shared definations

* feat: added logger as an injection for apps

* feat: changes to controllers

* feat: added vm based app execution inside manager

* feat: added dummy implementations for core constructs

* feat: moved controllers to routes

* feat: created worker moduler for processing message queue requests

* feat: added task manager defination

* feat: added context and manager separations

* feat: created defaults for the defination implementation

* feat: added async task management inside apps

* feat: removed unnecessary arguments in function calls

* feat: added context builder for app instances

* feat: added runtime type safety with zod

* feat: added packaged definition structure for silo apps

* feat: added package based configuration for engine and definitions

* feat: added custom eslint rule for throws decorator

* feat: added custom eslint rule for try catch

* feat: removed custom eslint config

* feat: added authentication controller inside engine

* feat: added auth in silo engine

* feat: migrated jira ui module to plane

* chore: removed jira app

* chore: moved engine components into src_engine directory

* feat: added base template inside new src directory

* feat: migrated worker interface with mq and redis store inside new src

* feat: created jira migrator and jira importer types

* chore: migrated worker's helper inside the base directory

* feat: added logic for booting up root worker

* feat: added jira and jira auth service inside jira app

* feat: added all transformers inside jira package

* feat: added authorization types for jira

* feat: added jira authentication oauth class

* feat: added jira transformer and pull mechanisms inside jira app

* feat: addded batching logic inside jira migrator

* feat: embedded silo sdk inside jira migrator

* feat: added plane migrator inside engine

* feat: added plane migrator for push inside migration controller

* feat: added controller methods for migration

* feat: added credentials and job routes in controller

* feat: added linear importer

* feat: added linear pull function as importer

* feat: added transformation for linear data

* feat: added pull mechanism for linear

* feat: attached linear data importer with migration controller

* fix: removed hardcoded jira from cycle and module migrator

* feat: fixed build errors

* chore: addeed example env

* feat: added authentication routes for jira

* feat: added linear route controller

* fix: restructuring

* fix: sdk configs setup

* fix: merge conflicts

* fix: sdk setup

* chore: added jira and linear importers and separate packages

* feat: moved transformation parts to linear package

* feat: decoupled jira logical parts with worker

* fix: linear silo app to use linear package

* fix: build errors and dependency resolution with packages in silo

* fix: module build errors in silo

* fix: linear authorization flow

* feat: added logic for segregated workers

* feat: attached task manager with the base starter

* feat: added migrations, query and schema into db directory

* feat: added linear importer and jira importer app structure

* feat: added silo core package

* chore: migrated worker and main engine controller inside apps

* fix: made linear integration working

* silo: added cors

* feat: added base64 state changes with jira

* chore: updated silo env

* fix: jira token cookie

* feat: added issue attachments in linear job

* feat: added credentials controller

* feat: added github package inside silo packages

* feat: added resource fetching in jira api service

* feat: added credentials locking

* feat: created resources endpoint for jira

* feat: added endpoints for getting jira data

* fix: credentials not working

* chore: exported jira types

* chore: added jira states

* fix: jira project pagination issue

* chore: initiated silo folder in web

* feat: added github routes and services in silo app

* fix: build updates

* fix: updated plane sdk and updated jira importer

* chore: updated the importer layout

* chore: removed as any from table component

* chore: integrated importer dashboard

* fix: tsup fixes

* fix: removed unnecessary files

* fix: removing tsup for building silo packages

* fix: build related issues

* fix: build issues

* fix: eslint fixes

* fix: silo build errors

* fix: silo app build errors types

* fix: reverting the cloud branch

* fix: updated package json in silo service

* fix: branch build cloud updated

* fix: build errors in apps while using sdk due to ts-alias paths

* fix: branch build cloud workflow fixes

* fix: docker compose setup updates

* fix: docker compose build fixes

* fix: docker build fixes envs updated in example file

* chore: updates folder structure and handled job services

* chore: resolved build errors in silo chore

* fix: docker compose cloud added

* fix: build process docker compose

* chore: updated jira workflow

* chore: handled the job start and jira dashboard

* chore: updated constants, file naming convention

* chore: resolved merge conflicts

* chore: integrated linear and updated jobs query

* feat: added hostname changes

* conflict: updated jira config and added issue transformation count in dashboard

* conflict: updated job

* chore: updated Jira status

* chore: updated Jira status

* fix: batch processing

* fix: batch key release

* chore: updated workflow for building images

* fix: detached silo build

* chore: updates linear and resolved build errors

* fix: batch key missing

* fix: linear workflow

* chore: updated linear queries

* fix: docker compose fixed for running silo services

---------

Co-authored-by: sriram veeraghanta <veeraghanta.sriram@gmail.com>
Co-authored-by: Henit Chobisa <chobisa.henit@gmail.com>
2024-10-08 23:16:28 +05:30