Ramsons Performance Expectations
- Magda Zacharska
- Viachaslau Khandramai
Ramsons data volume
Instances: 21 000 000 records
Holdings: 23 000 000 records
Items: 29 000 000 records
Users: 500 000 records
Expected Response Time
Items, holdings, instances, users based on PTF reports for Item bulk edit in Morning Glory.
No. of records | Matched records (Identifiers) | Matched records (Query) | Are you sure? form | Confirmation form |
|---|---|---|---|---|
100 | 6 sec | 6 sec | 10 sec | 10 sec |
1000 | 40 sec | 40 sec | 1 min | 1 min |
10 000 | 8 min | 8 min | 13 min | 13 min |
100 000 | lack of prior data | lack of prior data | lack of prior data | Lack of prior data |
Observed Response Time
No. of records | Matched records (Identifiers) | Matched records (Query) | Are you sure? form | Confirmation form |
|---|---|---|---|---|
Items (Identifiers: status Missing → Unknown, Query: status Unknown → Missing, member tenant) | ||||
100 | 9 sec
(expected: 6 sec) | 9 sec
(expected: 6 sec) | 2 sec
(expected: 10 sec) | I: 6.4 sec Q: 6.4 sec (expected: 10 sec) |
1000 | 95 sec
(expected: 40 sec) | 93 sec
(expected: 40 sec) | 3 sec
(expected: 1 min) | I: 78 sec (1.3 min) Q: 102 sec (1.7 min) (expected: 1 min) |
10 000 | 763 sec (12.7 min)
(expected: 8 min) | 580 sec (9.7 min)
(expected: 8 min) | 5 sec
(expected: 13 min) | I: 654 sec (10.9 min) Q: 741 sec (12.4 min) (expected: 13 min) |
100 000 | needs data | needs data | needs data | needs data |
Instances (Identifiers: discovery suppressing → true + set to items/holdings, Query: discovery suppressing + set to items/holdings → false, member tenant) | ||||
100 | 14 sec
(expected: 6 sec) | 13 sec
(expected: 6 sec) | 3 sec
(expected: 10 sec) | I: 27 sec Q: 53 sec (expected: 10 sec) |
1000 | 98 sec
(expected: 40 sec) | 89 sec
(expected: 40 sec) | 6 sec
(expected: 1 min) | I: 295 sec (4.9 min) Q: 252 sec (4.2 min) (expected: 1 min) |
10 000 | 872 sec (14.5 min)
(expected: 8 min) | 705 sec (11.8 min)
(expected: 8 min) | 10 sec
(expected: 13 min) | I: 3058 sec (51.0 min) Q: 2194 sec (36.6 min) (expected: 13 min) |
100 000 | needs data | needs data | needs data | needs data |
Holdings ( ) | ||||
100 |
(expected: 6 sec) |
(expected: 6 sec) |
(expected: 10 sec) |
(expected: 10 sec) |
1000 |
(expected: 40 sec) |
(expected: 40 sec) |
(expected: 1 min) |
(expected: 1 min) |
10 000 |
(expected: 8 min) |
(expected: 8 min) |
(expected: 13 min) |
(expected: 13 min) |
100 000 | needs data | needs data | needs data | needs data |
Users ( ) | ||||
100 |
(expected: 6 sec) |
(expected: 6 sec) |
(expected: 10 sec) |
(expected: 10 sec) |
1000 |
(expected: 40 sec) |
(expected: 40 sec) |
(expected: 1 min) |
(expected: 1 min) |
10 000 |
(expected: 8 min) |
(expected: 8 min) |
(expected: 13 min) |
(expected: 13 min) |
100 000 | needs data | needs data | needs data | needs data |
Observed Response Time (Precise measurement)
No. of records | Type | Matched records, sec | Are you sure? form, sec | Confirmation form, sec |
|---|---|---|---|---|
Instances Query: (instance.discovery_suppress is null/empty true) AND (instance.source == "FOLIO") Modifications:
| ||||
100 | Identifiers | 8.13±0.75 | 1.58±0.02 | 7.35±0.96 |
Query | 11.08±1.26 | 1.58±0.03 | 9.38±1.64 | |
1000 | Identifiers | 87.64±24.90 | 2.90±0.42 | 108.06±37.69 |
Query | 84.87±29.90 | 3.23±1.35 | 107.33±34.46 | |
10 000 | Identifiers | 473.57±7.95 | 5.06±0.47 | 736.64±39.73 |
Query | 481.35±33.12 | 4.71±0.04 | 725.14±63.64 | |
100 000 | Identifiers |
|
|
|
Query |
|
|
| |
Examples of queries:
Item records:
(items.status_name == "Available") - returns 22 records in Sprint testing environment. Query tool returns records immediately, bulk edit populates Matching records form in 9 sec.
(temporary_location.name is null/empty false) - returns 3082 records, first 100 in Query tool in 12 sec. 3082 records in bulk edit displays after 2 min 15 sec
Holdings records:
(permanent_location.code in ("UC/HP/ASR/LawSupr")) - returns 258 records in Sprint testing environment instantly. Bulk edit populates Matching records form in (I gave up after 8 minutes)
Instance:
(instance.discovery_suppress is null/empty true) - returns 44 records in the Sprint testing environment. Query tool returns records instantly , bulk edit populates Matched records in 12 sec
(instance.title contains "train") - returns one record on snapshot. Query tool displays the record instantly, bulk edit populates Matched record in 6 seconds.
Users
(users.active == "true") returns 90 314 records in Sprint testing environment. Query tools returns first 100 records in 11 sec, bulk edit populated the Matched record in 19 minutes. The same query run on snapshot returns the 44 records in Query tool and Bulk edit instantly.
FOLIO Instances: Identifiers + Query
Scenario
Five measurements were taken for each scenario, after which the average time and measurement error were calculated. Each scenario included three stages: matching records, applying changes, and committing changes. The duration of each scenario was measured and analyzed separately.
Query: (instance.discovery_suppress is null/empty true) AND (instance.source == "FOLIO") (for query matching) or identifiers of instance that satisfy this query (for identifiers matching).
Modifications:
Set Discovery to false
Add a General note “Regular data cleanup” and mark it as Staff only
Results
Records Matching
# | Records number | Type | Time, min | Error, min |
1 | 100 | Identifiers | 0,14 | 0,01 |
Query | 0,18 | 0,02 | ||
2 | 1000 | Identifiers | 1,46 | 0,42 |
Query | 1,41 | 0,50 | ||
3 | 10000 | Identifiers | 7,89 | 0,30 |
Query | 8,02 | 0,55 |
With a large number of entities, the time spent on the matching stage for identifiers and queries is the same within the margin of measurement error.
For 100 and 1,000 records, the matching time is longer than expected for both identifiers and queries. For 10,000 records, the expected matching time is within the error margin for both the query and identifier flow.
Changes Applying
# | Records number | Type | Time, s | Error, s |
1 | 100 | Identifiers | 1,58 | 0,02 |
Query | 1,58 | 0,03 | ||
2 | 1000 | Identifiers | 2,90 | 0,42 |
Query | 3,23 | 1,35 | ||
3 | 10000 | Identifiers | 5,06 | 0,47 |
Query | 4,71 | 0,41 |
The time for applying changes for identifiers and queries is expected to be the same within the measurement error margin and increases with the number of entities.
Changes Committing
# | Records number | Type | Time, min | Error, min |
1 | 100 | Identifiers | 0,12 | 0,02 |
Query | 0,16 | 0,03 | ||
2 | 1000 | Identifiers | 1,80 | 0,63 |
Query | 1,79 | 0,57 | ||
3 | 10000 | Identifiers | 12,28 | 0,66 |
Query | 12,09 | 1,10 |
Both the query and identifier flows are within expected time margins and align closely with expected values. For 10,000 entities, the actual time is even less than expected. The “apply changes” and “committing” stages should take the same amount of time for identifiers and queries, as confirmed by the matching values within the measurement error.
Summary
The completion times for the identifier and query flow stages are within the margin of error, even though the average values may differ.
While for a small number of records, the matching and committing times are longer than expected, for a large number of records, the corresponding times match or are even less than expected.
Results for 100K records are currently unavailable but are scheduled for measurement.