[MODMARCMIG-11] Draft: Marc migrations dev testing Created: 30/Jan/24 Updated: 31/Jan/24 |
|
| Status: | Draft |
| Project: | mod-marc-migrations |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | Task | Priority: | TBD |
| Reporter: | Viacheslav Kolesnyk | Assignee: | Unassigned |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | back-end, epam-spitfire | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Issue links: |
|
||||||||
| Sprint: | Spitfire Sprint 186 | ||||||||
| Development Team: | Spitfire | ||||||||
| RCA Group: | TBD | ||||||||
| Description |
|
Overview Story for testing migration parts and some alternatives by developers Approach 1) Now on "preparation chunks" step we retrieve 100 ids from database and use 1'st+last for chunks. Then on chunks processing we retrieve ids inclusive first and last from chunk record. 1.1. Chunks preparation. Idea is to retrieve 101 ids from database only on first request (last will be endId in chunks). Then use seek method from last id inclusive + offset 100, limit 1 to retrieve end id for next chunk (which will have start id = end id from previous chunk). 1.2 Chunks processing. For this approach we need to use startId inclusive, end id exclusive This solution may be faster as it transfers less data from database to application on "Chunks preparation phase". |