Done
Details
Assignee
Andrii NovytskyiAndrii NovytskyiReporter
Kruthi VuppalaKruthi VuppalaPriority
P3Story Points
3Sprint
NoneDevelopment Team
ConcordeFix versions
TestRail: Cases
Open TestRail: CasesTestRail: Runs
Open TestRail: Runs
Details
Details
Assignee
Andrii Novytskyi
Andrii NovytskyiReporter
Kruthi Vuppala
Kruthi VuppalaPriority
Story Points
3
Sprint
None
Development Team
Concorde
Fix versions
TestRail: Cases
Open TestRail: Cases
TestRail: Runs
Open TestRail: Runs
Created March 25, 2020 at 2:50 PM
Updated October 14, 2020 at 12:48 PM
Resolved September 18, 2020 at 7:59 PM
Overview
Currently as we are storing files locally before export job is triggered.If multiple instances of the module are running, It may happen that both the APIs are run by different instances, there by not finding the file. We need to be able to scale horizontally
Approach
The Ideal approach is to have a way to store the temporary files somewhere accessible to both running instances of module, which probably is needed at a FOLIO level.
APPROACH 1
One way to handle could be to store the uploaded file in S3. When export is triggered, file is looked in the local storage, if not found will be downloaded from S3 and export can continue as usual.
Disadvantages:
1. This method could introduce minor delays because the file needs to be uploaded to S3 and also downloaded before export can begin
2. File cleanup is another issue, because we run a periodic job to cleanup files, that would somehow need to know all the files to be cleaned up
APPROACH 2
This may be a major change ,we could be to load all the requested UUIDs to a database table and read from there, instead of maintaining a file in the local and clearing it up later.
This would need changes to be made to create a new table and also to upload endpoint and then reading would also change to read from the DB rather than from the file
Acceptance Criteria
Able to scale horizontally