Simple MARC Record Editor (quickMARC)

UXPROD-2272 - Getting issue details... STATUS  - Connect SRS and SMRE for MARC Bibs (SMRE work)

Requirements

Simple MARC Records Editor (SMRE) must be able to access an existing SRS MARC Bib via a button in the Instance (e.g. Edit MARC) and edit the MARC Bib. Once the edits have been made via SMRE, then:

• The edits are saved to the SRS MARC Bib (including updates to the size of the MARC record in MARC Leader bytes 00-04): (a) the corresponding updates are made to the MARCcat MARC bib automatically (once MARCcat and SRS are integrated); (b) the corresponding updates are made to the Inventory Instance automatically.
• Current scope for SMRE would be editing existing SRS Bib records only. Not creating new MARC records from Instances or creating new MARC records from scratch.
• SMRE will use a simple UI, rather than a text editor.
• The only expected validation is the general structural validation that SRS currently does. Any more rigorous MARC validation would happen in MARCcat or external to FOLIO.
• If displayed in the SMRE, data in MARC 001 and 999 ff fields would be visible, but not editable
• Once SRS is expanded to include MARC holdings records, the SMRE may be expanded to allow for editing of MARC holdings also.
• Since record update is performed asynchronously, SMRE must provide information about current updating status and last successful update.

Approach & Design

quickMARC application should be implemented as separate micro service in its own module mod-quick-marc. It allows to extend number of use cases in future, for example, for Data Import could involve editing MARC as part of those workflows. Two endpoints must be implemented to meet the requirements for the MARC editor:

Module permissions: records-editor.all

The first endpoint is responsible for the delivery and transformation of the MARC record to the client in the required format. The second one is necessary to update the record after corresponding validation and transformation.

(A) Get MARC record

1. Client makes request to quickMARC GET /records-editor/records/instanceId={instanceId}
2. quickMARC proxies request to source-record-storage: GET /source-storage/records?query=externalIdsHolder.instanceId==={instanceId} / CM: GET change-manager/parsedRecords?instanceId={instanceId}
3. quickMARC transforms request to appropriate format (accepted by UI)
4. quickMARC returns response 200 OK

(B) Update MARC record

1. Client makes request to inventory PUT /records-editor/records/{id}
2. quickMARC transform record from UI-compatible format to marc-json and updates to the size of the MARC record in MARC Leader bytes 00-04
3. quickMARC updates corresponding instance in CM
4. quickMARC returns response 202 ACCEPTED

Integration with SRS/SRM

quickMARC should operate with two endpoints to retrieve and update existed MARC records:

1) GET request to retrieve existing record (errors should be processed on SRS/CM)

2) PUT to update existing (errors should be processed on CM)

Full QM-CM/SRS diagram is presented below. More details about SRM/SRS can be found on SRM/SRS Design Document.

Key Components

UI compatible schema for MARC record representation

updateInfo

Converter QuickMarcJson ↔ ParsedRecordDto

Converters should be implemented as separate components implementing Converter<ParsedRecordDto, QuickMarcJson> and Converter<QuickMarcJson, ParsedRecordDto>. Since it is necessary to be able to integrate with SRS and CM, both of services should be implemented. Each of these services uses its own object types for MARC records representation: record.json (SRS) and parsedRecordDto.json (SRM). The quickMARC editor uses its own type quick_marc_json.json to represent the corresponding MARC record. The connection between the required IDs of is shown in the following diagram:

It should be noted that IDs population is responsibility of service but not converter because service is type-specific unlike converter. Converter operates only with ParsedRecord that doesn't contain information about its holder: record or parsedRecordDto.

Performance Testing

There are two different approaches for system load simulation:

• Closed systems (control of the concurrent number of users);
• Open systems (where you control the arrival rate of users).

Closed system are system where the number of concurrent users is capped. At full capacity, a new user can effectively enter the system only once another exits. On the contrary, open systems have no control over the number of concurrent users: users keep on arriving even though applications have trouble serving them. As far as it concerns FOLIO in general and the quickMARC in particular we deal with Open System. Therefore the most optimal load profile is gradually increasing number of concurrent users during fixed time period. For scenario with 1200 threads and the ramp-up period of 120 seconds, JMeter scenario will take 120 seconds to get all 1200 threads up and running. Each thread will start 1/10 (120/1200) second after the previous thread was begun. We tested both of quickMARC operations: GET MARC record by instanceId and PUT MARC record as sequential operations for each thread (user). Results of load tests are presented in the table below.

Jmeter report generation: jmeter -n -t <TestPlan.jmx> -l <report.jtl> -e -o dashboard

Links

https://github.com/marc4j/marc4j

^*integration with SRS is planned by source-record-manager; API isn't ready yet  UXPROD-2273 - Getting issue details... STATUS