[FOLIO-2030] Set up data-loading hosted environment Created: 20/May/19 Updated: 03/Jun/20 Resolved: 24/May/19 |
|
| Status: | Closed |
| Project: | FOLIO |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | Story | Priority: | P3 |
| Reporter: | Ann-Marie Breaux (Inactive) | Assignee: | Ian Hardy |
| Resolution: | Done | Votes: | 0 |
| Labels: | ci, data-import, devops, performance, platform-backlog | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Attachments: |
|
||||||||
| Issue links: |
|
||||||||
| Sprint: | CP: sprint 64 | ||||||||
| Story Points: | 2 | ||||||||
| Development Team: | Core: Platform | ||||||||
| Description |
|
Purpose: We need an environment to test loading of files and their impact on various apps, OKAPI, and other parts of FOLIO, without impeding the function of folio-testing and folio-snapshot As a staff person A small MARC file attached, and here's a link to a large MARC file (143 MB), in case they would be useful: https://drive.google.com/file/d/169Tnsa1KJoVwsHaSZfkML5jj9Zke9ESS/view?usp=sharing Requirements
Wayne Schneider Per Oleksii Kuzminov, Folijet can set a limiter on the size of MARC file being loaded until we have event-driven handling in place, which will then take the load off of OKAPI. Please advise what we should set as the maximum file size for now. Per Oleksii Kuzminov, once Kateryna Senchenko's data import performance testing is complete, we'll have a better understanding of current performance. |
| Comments |
| Comment by Ian Hardy [ 20/May/19 ] |
|
Hi Ann-Marie Breaux, just to clarify, are you looking for a one-off instance that will persist for as long as you need to test this feature, or something that gets rebuilt? |
| Comment by Ann-Marie Breaux (Inactive) [ 20/May/19 ] |
|
Hi Ian Hardy Something that gets rebuilt, and basically stays in sync with folio-snapshot |
| Comment by Oleksii Kuzminov [ 21/May/19 ] |
|
Ian Hardy we need to rebuild it like a snapshot env |
| Comment by Ian Hardy [ 23/May/19 ] |
|
https://folio-snapshot-load.aws.indexdata.com/ is available for loading data or other activities that might be a burden on the other shared environments. Its an equivalent configuration to folio-snapshot. The backend is at https://folio-snapshot-load-okapi.aws.indexdata.com/. |