Versions Compared

Old Version 16

changes.mady.by.user Shans Kaluhin

Saved on

compared with

New Version Current

changes.mady.by.user Pavlo Smahin

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Jira Legacy
serverSystem JiraJIRA
serverId01505d01-b853-3c2e-90f1-ee9b165564fc
keyMODQM-311

...

  1. Determine complexity of allowing for configuration of 001 generation (prefix + starting value) 
  2. Adding the ability to select the desired authority source file at point of creation of new record (so the system would know how to generate the 001 per the configuration)

Approach

...

Approach 

  1. Create new endpoint to retrieve HRID by authority file id 
  2. Implement sequence number generation mechanism for authority local files
  3. On the UI user will select file name. And before creation UI should put to 001 field value from the HRID endpoint. 

Tickets

OverviewApproachModule
Ticket1.Ability to store authority local filesmod-inventory-storage2.Link authority local file to the authoritydata-import-processing-core3.Initialize authority local files for Mapping Parametersmod-source-record-manager4.Select local authority file on the UIquick-marc

...

Story PointsJira ticket
Validation for authority local files
  1. New parameter 'startNumber' for authority source file creation is presented 
  2. Implement AuthoritySourceFileHridManager with Validation:
      - (if source = 'local' then codes.size = 1) else throw exception
      - (if source = 'local' and startNumber = null) set startValue = 1
      - prevent user to update "start with" and "prefix" after file creation
mod-inventory-storage3 sp


Jira Legacy
serverSystem JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId01505d01-b853-3c2e-90f1-ee9b165564fc
keyMODINVSTOR-1049
Status
colourGreen
titleApproved

Implement sequence number generation mechanism for authority local files (Sequences Approach)





  1. New endpoint to catch-up authority current hrid information is presented
  2. Extend AuthoritySourceFileHridManager to work with sequences
      - Create sequence for authority local file after validation
      - Drop sequence for authority local file after deletion
  3. Return hrid on request 
      - Retrieve next value from localFile sequence 
      - Format next value to NaturalId(HRID) format
mod-inventory-storage5 sp

Jira Legacy
serverSystem JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId01505d01-b853-3c2e-90f1-ee9b165564fc
keyMODINVSTOR-1050

Status
colourRed
titleDeclined

Implement sequence number generation mechanism for authority local files (Table with counter Approach)


  1. New endpoint to catch-up authority current hrid information is presented
  2. New table to work with authority files HRID presented
  3. Extend AuthoritySourceFileHridManager to work with table
      - All transactions should be done with SERIALIZABLE isolation
      - On request return current hrid value and increment it. 
      - Format value to NaturalId(HRID) format
mod-inventory-storage5 sp
Authority file id validation
  1. mod-quick-marc in scope of authority creation should convert 001 field value to UUID and retrieve available hrid for source file id.
  2. Replace 001 field value from authorityFileId to "prefix + hrid"
  3. If 001 value doesn't comply with UUID then skip it. And leave the original value
mod-quick-marc2 sp

Jira Legacy
serverSystem JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId01505d01-b853-3c2e-90f1-ee9b165564fc
keyMODQM-322

Local authority file selection
  1. User should have the ability to look-up for local authority file
    (As permanent-location look-up for Holdings)

  2. UI should put authorityFileId to 001 field
ui-inventory

Jira Legacy
serverSystem JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId01505d01-b853-3c2e-90f1-ee9b165564fc
keyUXPROD-3910

Ability to store authority local files

Settings application UI preview:

localauthoritydefinition.PNG

New mod-inventory-storage API to store local authority filesEndpoints: 

POST: /

...

authority-

...

source-

...

files/{id}/hrid
Description:  Endpoint returns current hrid value and increment it. 

Code Block
themeDJango
borderStyledashed
titleLocal Authority Source File HRID Example
collapsetrue
{
    "id": "cb58492d-018e-442d-9ce3-35aabfc524aa",
    "name": "Test",
    "type": "Subjects",
    "prefix": "localloc",
    "startWithhrid": "000000001"
}

POST: /authority-source-files
New query parameter: startNumber (default = 1)

Code Block
themeDJango
borderStyledashed
titleLocal Authority File SchemastartNumber parameter schema
collapsetrue
{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "description": "An Authority local file",
  "type": "object",
  "properties": "startNumber": {
    "id": {
      "description": "uniqueThe IDnumber offrom thewhich Authorityto localstart file; a UUIDgenerating HRIDs",
      "$reftype": "uuid.json"
    }integer",
    "nameminimum": {
      "type": "string"1,
      "descriptionmaximum": "Authority local file name"
    },
    "type": {
      "type": "string",
      "description": "Type of authority records stored in local file"
    },
    "prefix": {
      "type": "string",
      "description": "Assign local file prefix"
    },
    "startWith": {
      "type": "string",
      "description": "Assign local file start with value"
    },
    "metadata": {
      "type": "object",
      "$ref": "raml-util/schemas/metadata.schema",
      "readonly": true
    }
  },
  "additionalProperties": false,
  "required": [
    "name",
    "type",
    "prefix",
    "startWith"
  ]
}99999999999
}


Sequence number generation mechanism for authority local files

Sequences Approach

There is a similar approach that works already in the mod-inventory-storage. It generates Hrid numbers for Instances, Holdings, and Item records, and can be configured from UI (Settings → Inventory → Hrid handling);
The problem here is that we will create new sequences in Runtime for each new authority file. (format: "hrid_authority_local_file_{prefix}_seq") 

Status
colourGreen
titlePros

Status
colourRed
titleCons

Auto increment Separate sequence should be created for each authority file
Atomic transaction. Safety works with asynchronous work



Table with counter Approach

New table should be presented

Code Block
themeDJango
borderStyledashed
titleLocal Authority Files Schemahrid_authority_source_file
collapsetrue
{CREATE TABLE IF "$schema": "http://json-schema.org/draft-04/schema#",
  "description": "A collection of Authority local files",
  "type": "object",
  "properties": {NOT EXISTS hrid_authority_source_file
(
  authorityFileId UUID NOT NULL,
  hridSeq         "authorityLocalFiles":INT {       "description": "List of Authority local files"NOT NULL,
  CONSTRAINT hrid_authority_source_file_pkey PRIMARY  "id": "authorityLocalFile"KEY (authorityFileId),
  CONSTRAINT hrid_authority_source_file_fkey FOREIGN  "type": "array",KEY (authorityFileId)
    REFERENCES authority_source_file  "items": {
        "type": "object",
        "$ref": "authoritylocalfile.json"
      }
    },
    "totalRecords": {
      "description": "Estimated or exact total number of records",
      "type": "integer"
    }
  },
  "required": [
    "authorityLocalFiles",
    "totalRecords"
  ]
}

Link authority local file to the authority (data-import-processing-core)

...

Initialize authority local files for Mapping Parameters (mod-source-record-manager)

...

(id)
);


To work with table it's better to use SERIALIZABLE Isolation:
@Transactional(isolation = Isolation.SERIALIZABLE)

Status
colourGreen
titlePros

Status
colourRed
titleCons

One table for all authority filesAll work with select and increment should be done manually.

Harder work with transactions. 
All transactions should be fulfilled in SERIALIZABLE Isolation level. So it can reduce performance



Select local authority file on the UI

  • User should select local file for 001 field
    Image RemovedImage Added
  • We also need to warn the user that field 001 may be overwritten by field 010.


Q/A section

QuestionStatusAnswerNotes
How many files can be per one tenant? Do we expect millions of files? Can we restrict amount of authority files per tenant?

Status
colourYellow
titleopen

We can likely expect less than 10 local authority files to be created. Typically, records are created for local files for use cases such as:

  • A record does not yet exist in an external/national database (for example, a university library may have items in their collections written by alumni for which the Library of Congress does not have a name authority record)
  • A record exists in an external/national database but the headings do not align with the mission/objectives of the institution (such as headings for indigenous peoples)



Do we expect authority records matching with local authority file during DI process?

Status
colourYellow
titleopen

Yes, it is possible for the library to want to update an existing local authority record via DI and would need to match on the existing file.