...
More details on a feature: [UXPROD-3874] and spike story: [MODELINKS-79]
Requirements
Functional Requirements
...
- The UI sends a request to the backend API.
- mod-quick-marc receives the request and converts the record into an SRS-like format.
- mod-quick-marc sends a request to the mod-entities-links API.
- mod-entities-links receives the request and fetches linking rules from the database using cache.
- From MARC bib fields that are applicable for linking according to linking rules, $0 subfield values are extracted.
- $0 values are used for search authorities in mod-search. The current mod-search endpoint also calculates a number of already existing links in the instance index, this should be omitted to speed up the process. TBD: authorities naturalId is exist in the internal database table 'authority_data'. Should we use this data before doing a search in mod-search?
- mod-entities-links receives a collection of authority records and prepares a request to the mod-source-record-storage.
- mod-entities-links sends a request to the mod-source-record-storage bulk endpoint.
- mod-entities-links receives a collection of authority source records.
- mod-entities-links analyze results, prepare data for links according to linking rules, and set constructed links into the record.
- mod-quick-marc receives the record with links.
- mod-quick-marc converts the record into the appropriate format.
- UI receives the record with suggested links.
...
| Error cause code | Description |
|---|---|
| 101 | applicable authority was not found |
| 102 | 2 or more applicable authorities were found |
| 103 | auto linking feature is disabled |
| TBD |
mod-entities-links
POST /links
...
-suggestions/marc
| Code Block | ||||
|---|---|---|---|---|
| ||||
{ "records": [ { "fields": [ { "001": "393893" }, { "100": { "ind1": "/", "ind2": "/", "subfields": [ { "a": "Mozart, Wolfgang Amadeus," }, { "d": "1756-1791." }, { "0": "12345" }, { "9": "b9a5f035-de63-4e2c-92c2-07240c88b817" } ], "linkStatus": "ACTUAL" } }, { "110": { "ind1": "/", "ind2": "/", "subfields": [ { "a": "Mozart" } ] } } ], "leader": "01706ccm a2200361 4500" } |
...
]
} |
The response will include suggested links with the status "NEW"; fixed data and status "ACTUAL" for links, that had the status "ERROR"; links with the status "ERROR" and cause type for fields where a link can't be assigned.
| Code Block | ||||
|---|---|---|---|---|
| ||||
{ "records": [ { "fields": [ { "001": "393893" }, { "100": { "ind1": "/", "ind2": "/", "subfields": [ { "a": "Mozart, Wolfgang Amadeus," }, { "d": "1756-1791." }, { "0": "12345" }, { "9": "b9a5f035-de63-4e2c-92c2-07240c88b817" } ], "linkStatus": "ACTUAL" } }, { "110": { "ind1": "/", "ind2": "/", "subfields": [ { "a": "Mozart" }, { "0": "12345" }, { "9": "b9a5f035-de63-4e2c-92c2-07240c88b817" } ], "linkStatus": "NEW" } }, { "130": { "ind1": "/", "ind2": "/", "subfields": [ { "a": "Mozart" } ], "linkStatus": "ERROR", "errorStatusCode": "101" } } ], "leader": "01706ccm a2200361 4500" } ] } |
mod-source-record-storage
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
{
"records": [
{
"id": "c56b70ce-4ef6-47ef-8bc3-c470bafa0b8c",
"externalIdsHolder": {
"authorityId": "b9a5f035-de63-4e2c-92c2-07240c89b817"
},
"recordType": "MARC_AUTHORITY",
"recordState": "ACTUAL",
"parsedRecord": {
"id": "c9db5d7a-e1d4-11e8-9f32-f2801f1b9fd1",
"content": {
"fields": [
{
"010": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "2001000234"
}
]
}
},
{
"100": {
"ind1": "/",
"ind2": "/",
"subfields": [
{
"a": "Mozart, Wolfgang Amadeus"
},
{
"d": "1756-1791"
}
]
}
},
{
"110": {
"ind1": "1",
"ind2": "0",
"subfields": [
{
"a": "Works"
}
]
}
}
],
"leader": "01706ccm a2200361 4500"
}
}
}
],
"totalRecords": 1
}
|
mod-search
GET /search/authorities
New query parameter to add:
| Parameter | Type | Note |
|---|---|---|
| includeNumberOfTitles | boolean (default = true) | If true do not perform a search for a number of linked instances |
Performance
Considerations
- Using mod-search for searching by naturalId instead of just doing a search in mod-source-record-storage has to decrease response time when the number of records in the system is more than 1M. (Using mod-search will be needed for possible future requirements to have automated linking not only by $0 but by some other data)
- Disabling the linked instances counting for the mod-search authority request have to decrease the time of response.
- Having only required fields in the mod-source-record-storage response will decrease the size of data that has to be transferred via HTTP. The necessity of this should be tested to define if such processing will decrease performance. 2 options there: get the record as jsonb from the marc_records table and retain only needed fields or construct a record field-by-field from the marc_indexers partitioned table.
- Using mod-search and mod-source-record-storage bulk endpoints will decrease response time.
Testing
Performance testing has to be done on the environment with:
- > 1M authority records
- > 1M MARC-based instance records
- Prepared MARC bib records that have >50 fields that are applicable for linking and all these fields should have $0 values matched to existing in the system authorities..
Tests are needed for:
- 1 request/sec
- 10 requests/sec
- 100 requests/sec
- 1000 requests/sec
| Info |
|---|
| Based on testing results some performance improvements could be suggested if it will be required. |