Page Comparison

Table of Contents

Verifying the connection to OAI-PMH service

...

When a harvest starts the system checks first the inventory for the updates (new or modified records) then retrieves underlying MARC Bib records from SRS.   If the harvest was triggered with the metadataPrefix set to marc21_withholdings, the holdings and items data is append  as described in MODOAIPMH-102.   

Max number of records per response

...

The "from" and "until" parameters can take include timestamp and the format is then: yyyy-mm-ddThh:mm:ssZ.  This is especially helpful during the troubleshooting when the harvest needs to be limited to some specific hours when the records were updated.  The specified time is in UTC (Coordinated Universal Time).   

• Consecutive requests (with resumption token):

...

Sending the request with request identifier, for example GET /oai/request-metadata/{requestId}/failed-to-save-instances, will return the list of UUIDs for the records that failed to save.  The list might be used for troubleshooting existing data issues in FOLIO inventory or SRS.  For more information see mod-oai-pmh README.md file or FOLIO API documentation.

Suppressed records

When the Suppressed records processing setting (Settings→ OAI-PMH→ Behavior) is set to "Transfer suppressed records with discovery flag value", the records marked as suppressed are included in the response with added subfield t.   

...

<record>
<header status="deleted">
<identifier>oai:edge-bugfest-iris.folio.ebsco.com:fs09000000/ce064ce6-3d9c-4765-a3cf-564289f59b58</identifier>
<datestamp>2021-10-22T18:50:22Z</datestamp>
<setSpec>all</setSpec>
</header>
</record>

If you use an API process calls to delete Instances , be sure to delete the associated SRS record to trigger deleted record support. If you delete an Instance without knowing the associated SRS id, the only way to find it is by retrieving all Instances and SRS and then finding which SRS record points to an Instance that doesn't exist.the following steps are required to assure that the discovery is updated as well:

• Set ldr05 to "d" in the underlying SRS record.  This can be done by editing the record in QuickMarc.
• Let the incremental harvest get the information about the deleted record.
• Delete instance record and corresponding SRS record via  through API call

Slow Performance 

The harvesting of  5 million records should not take more than 11 hours in Juniper and less than 10 in Kiwi.  It is highly recommended to run REINDEX, VACUUM and ANALYZE after major updates to the inventory tables in PostgresSQL database.  It is highly recommended to run ANALYZE on a regular basis.

REINDEX index <tenant>_mod_inventory_storage.audit_item_pmh_createddate_idx ;
REINDEX xxxx=> REINDEX index <tenant>_mod_inventory_storage.audit_holdings_record_pmh_createddate_idx;
REINDEX
xxxx=> REINDEX index <tenant>_mod_inventory_storage.audit_holdings_record_pmh_createddate_idx;
REINDEX
xxxx=> REINDEX index <tenant>_mod_inventory_storage.audit_holdings_record_pmh_createddate_idx;
REINDEX
xxxx=> REINDEX index <tenant>_mod_inventory_storage.holdings_record_pmh_metadata_updateddate_idx;
REINDEX
xxxx=> REINDEX index <tenant>_mod_inventory_storage.item_pmh_metadata_updateddate_idx;
REINDEX
xxxx=> REINDEX index <tenant>_mod_inventory_storage.instance_pmh_metadata_updateddate_idx;
REINDEX
xxxx=> analyze verbose <tenant>_mod_inventory_storage.instance;
ANALYZE
xxxx=> analyze verbose <tenant>_mod_inventory_storage.item;
ANALYZE
xxxx=> analyze verbose <tenant>_mod_inventory_storage.holdings_record;
ANALYZE

...