SourceRecordStorage tools for test data populating

Endpoint

To test the module, there is a special endpoint for loading test data into the module's storage. Ability to load MARC records supports the loading of both source data and already parsed records in JSON format.

This endpoint does not need any permissions. All records imported through this endpoint would be linked with the stubbed snapshot.

Method

Description

POST

/populate-test-marc-records

Non-production endpoint into the mod-source-record-storage that allow the addition of MARC records for testing purposes.

mod-source-record-storage have a dedicated stub snapshot to store such records.

Schema

Endpoint works with a simple data structure for loading a bunch of source records.

MarcRecordCollection

Field	Description
sourceRecords	Array of SourceRecord objects that contains source data of MARC record and UUID

SourceRecord

Field	Description
id	UUID for particular record. Endpoint don't generate a UUID for records. Required field
source	String representation of single MARC record. The source can be either raw data or already parsed data in JSON format

Example

MarcRecordCollection in this example contains two records inside the array. First source record is on a raw format and the second one is represented as JSON.

{
  "sourceRecords": [
    {
      "id": "88dfac11-1caf-4470-9ad1-d533f6360bdd",
      "source": "01240cas a2200397   4500001000700000005001700007008004100024010001700065022001400082035002600096035002200122035001100144035001900155040004400174050001500218082001100233222004200244245004300286260004700329265003800376300001500414310002200429321002500451362002300476570002900499650003300528650004500561655004200606700004500648853001800693863002300711902001600734905002100750948003700771950003400808\u001E366832\u001E20141106221425.0\u001E750907c19509999enkqr p       0   a0eng d\u001E  \u001Fa   58020553 \u001E  \u001Fa0022-0469\u001E  \u001Fa(CStRLIN)NYCX1604275S\u001E  \u001Fa(NIC)notisABP6388\u001E  \u001Fa366832\u001E  \u001Fa(OCoLC)1604275\u001E  \u001FdCtY\u001FdMBTI\u001FdCtY\u001FdMBTI\u001FdNIC\u001FdCStRLIN\u001FdNIC\u001E0 \u001FaBR140\u001Fb.J6\u001E  \u001Fa270.05\u001E04\u001FaThe Journal of ecclesiastical history\u001E04\u001FaThe Journal of ecclesiastical history.\u001E  \u001FaLondon,\u001FbCambridge University Press [etc.]\u001E  \u001Fa32 East 57th St., New York, 10022\u001E  \u001Fav.\u001Fb25 cm.\u001E  \u001FaQuarterly,\u001Fb1970-\u001E  \u001FaSemiannual,\u001Fb1950-69\u001E0 \u001Fav. 1-   Apr. 1950-\u001E  \u001FaEditor:   C. W. Dugmore.\u001E 0\u001FaChurch history\u001FxPeriodicals.\u001E 7\u001FaChurch history\u001F2fast\u001F0(OCoLC)fst00860740\u001E 7\u001FaPeriodicals\u001F2fast\u001F0(OCoLC)fst01411641\u001E1 \u001FaDugmore, C. W.\u001Fq(Clifford William),\u001Feed.\u001E03\u001F81\u001Fav.\u001Fi(year)\u001E40\u001F81\u001Fa1-49\u001Fi1950-1998\u001E  \u001Fapfnd\u001FbLintz\u001E  \u001Fa19890510120000.0\u001E2 \u001Fa20141106\u001Fbm\u001Fdbatch\u001Felts\u001Fxaddfast\u001E  \u001FlOLIN\u001FaBR140\u001Fb.J86\u001Fh01/01/01 N\u001E\u001D"
    },
    {
      "id": "77dfac11-1caf-4470-9ad1-d533f6360bdd",
      "source": "{ "leader": "00508cjm a22001813 4500", "fields": [ { "001": "10062588" }, { "005": "20171013073237.0" }, { "007": "sd fsngnnmmneu" }, { "008": "170825s2017 xx nn n zxx d" }, { "024": { "subfields": [ { "a": "00190295755553" }, { "2": "gtin-14" } ], "ind1": "7", "ind2": " " } }, { "024": { "subfields": [ { "a": "190295755553" } ], "ind1": "1", "ind2": " " } }, { "035": { "subfields": [ { "a": "(OCoLC)1002130878" } ], "ind1": " ", "ind2": " " } }, { "035": { "subfields": [ { "a": "10062588" } ], "ind1": " ", "ind2": " " } }, { "040": { "subfields": [ { "a": "BTCTA" }, { "b": "eng" }, { "c": "BTCTA" } ], "ind1": " ", "ind2": " " } }, { "100": { "subfields": [ { "a": "Rossi, Daniele" } ], "ind1": "1", "ind2": " " } }, { "245": { "subfields": [ { "a": "Saint-Saens: Organ Symphony and Carnival of The Animals" } ], "ind1": "0", "ind2": "0" } }, { "260": { "subfields": [ { "b": "Wea Corp" }, { "c": "2017." } ], "ind1": " ", "ind2": " " } }, { "948": { "subfields": [ { "a": "20171013" }, { "b": "m" }, { "d": "batch" }, { "e": "lts" }, { "x": "deloclcprefix" } ], "ind1": "2", "ind2": " " } } ] }"
    }
  ]
}

How to load test data to the server

cURL

To load large amounts of test data, you can use the cURL CLI tool. The first step is preparing a file with request body in JSON format.

request.json

{
  "sourceRecords": [
    {
      "id": "88dfac11-1caf-4470-9ad1-d533f6360bdd",
      "source": "01240cas a2200397   4500001000700000005001700007008004100024010001700065022001400082035002600096035002200122035001100144035001900155040004400174050001500218082001100233222004200244245004300286260004700329265003800376300001500414310002200429321002500451362002300476570002900499650003300528650004500561655004200606700004500648853001800693863002300711902001600734905002100750948003700771950003400808\u001E366832\u001E20141106221425.0\u001E750907c19509999enkqr p       0   a0eng d\u001E  \u001Fa   58020553 \u001E  \u001Fa0022-0469\u001E  \u001Fa(CStRLIN)NYCX1604275S\u001E  \u001Fa(NIC)notisABP6388\u001E  \u001Fa366832\u001E  \u001Fa(OCoLC)1604275\u001E  \u001FdCtY\u001FdMBTI\u001FdCtY\u001FdMBTI\u001FdNIC\u001FdCStRLIN\u001FdNIC\u001E0 \u001FaBR140\u001Fb.J6\u001E  \u001Fa270.05\u001E04\u001FaThe Journal of ecclesiastical history\u001E04\u001FaThe Journal of ecclesiastical history.\u001E  \u001FaLondon,\u001FbCambridge University Press [etc.]\u001E  \u001Fa32 East 57th St., New York, 10022\u001E  \u001Fav.\u001Fb25 cm.\u001E  \u001FaQuarterly,\u001Fb1970-\u001E  \u001FaSemiannual,\u001Fb1950-69\u001E0 \u001Fav. 1-   Apr. 1950-\u001E  \u001FaEditor:   C. W. Dugmore.\u001E 0\u001FaChurch history\u001FxPeriodicals.\u001E 7\u001FaChurch history\u001F2fast\u001F0(OCoLC)fst00860740\u001E 7\u001FaPeriodicals\u001F2fast\u001F0(OCoLC)fst01411641\u001E1 \u001FaDugmore, C. W.\u001Fq(Clifford William),\u001Feed.\u001E03\u001F81\u001Fav.\u001Fi(year)\u001E40\u001F81\u001Fa1-49\u001Fi1950-1998\u001E  \u001Fapfnd\u001FbLintz\u001E  \u001Fa19890510120000.0\u001E2 \u001Fa20141106\u001Fbm\u001Fdbatch\u001Felts\u001Fxaddfast\u001E  \u001FlOLIN\u001FaBR140\u001Fb.J86\u001Fh01/01/01 N\u001E\u001D"
    },
    {
      "id": "77dfac11-1caf-4470-9ad1-d533f6360bdd",
      "source": "{ "leader": "00508cjm a22001813 4500", "fields": [ { "001": "10062588" }, { "005": "20171013073237.0" }, { "007": "sd fsngnnmmneu" }, { "008": "170825s2017 xx nn n zxx d" }, { "024": { "subfields": [ { "a": "00190295755553" }, { "2": "gtin-14" } ], "ind1": "7", "ind2": " " } }, { "024": { "subfields": [ { "a": "190295755553" } ], "ind1": "1", "ind2": " " } }, { "035": { "subfields": [ { "a": "(OCoLC)1002130878" } ], "ind1": " ", "ind2": " " } }, { "035": { "subfields": [ { "a": "10062588" } ], "ind1": " ", "ind2": " " } }, { "040": { "subfields": [ { "a": "BTCTA" }, { "b": "eng" }, { "c": "BTCTA" } ], "ind1": " ", "ind2": " " } }, { "100": { "subfields": [ { "a": "Rossi, Daniele" } ], "ind1": "1", "ind2": " " } }, { "245": { "subfields": [ { "a": "Saint-Saens: Organ Symphony and Carnival of The Animals" } ], "ind1": "0", "ind2": "0" } }, { "260": { "subfields": [ { "b": "Wea Corp" }, { "c": "2017." } ], "ind1": " ", "ind2": " " } }, { "948": { "subfields": [ { "a": "20171013" }, { "b": "m" }, { "d": "batch" }, { "e": "lts" }, { "x": "deloclcprefix" } ], "ind1": "2", "ind2": " " } } ] }"
    }
  ]
}

After file is ready you can run curl command.

curl -w '\n' -X POST -D -   \
    -H "Content-type: application/json"   \
    -H "x-okapi-tenant: diku"   \
    -d @request.json \
    http://localhost:9130/populate-test-marc-records

If all records are successfully saved on the server, the response will come with the status code 204.

The server will determine in what format the data came. If it is raw data, it will parse the records and save both the original and the parsed record in JSON format.

If the source record is already in JSON format, then the server will save this JSON as the raw record as well as the parsed one.

Java client generated by RMB

For using module's endpoints it provides generated by RMB client. This client is packaged into the lightweight jar. You can add mod-source-record-storage-client as a maven dependency.

<dependency>
  <groupId>org.folio</groupId>
  <artifactId>mod-source-record-storage-client</artifactId>
  <version>x.y.z</version>
  <type>jar</type>
</dependency>

Where x.y.z - version of mod-source-record-storage.

SourceStorageClient is generated by RMB and provide methods for all modules endpoints described into the RAML file. So in this case we can use /populate-test-marc-records endpoint throughs this client.

Clients methods works with generated by RMB data classes based on json schemas.
mod-source-record-storage-client jar contains only generated by RMB DTOs and clients.

Example with sending request to the mod-source-record-storage for creating test records

// create client object with okapi url, tenant id and token
SourceStorageClient client = new SourceStorageClient("localhost", "diku", "token");


// create new records entity
TestMarcRecordsCollection testData = new TestMarcRecordsCollection()
  .withSourceRecords(Arrays.asList(new SourceRecord()
    .withId("88dfac11-1caf-4470-9ad1-d533f6360bdd")
    .withSource("01240cas a2200397   4500001000700000005001700007008004100024010001700065022001400082035002600096035002200122035001100144035001900155040004400174050001500218082001100233222004200244245004300286260004700329265003800376300001500414310002200429321002500451362002300476570002900499650003300528650004500561655004200606700004500648853001800693863002300711902001600734905002100750948003700771950003400808\\u001E366832\\u001E20141106221425.0\\u001E750907c19509999enkqr p       0   a0eng d\\u001E  \\u001Fa   58020553 \\u001E  \\u001Fa0022-0469\\u001E  \\u001Fa(CStRLIN)NYCX1604275S\\u001E  \\u001Fa(NIC)notisABP6388\\u001E  \\u001Fa366832\\u001E  \\u001Fa(OCoLC)1604275\\u001E  \\u001FdCtY\\u001FdMBTI\\u001FdCtY\\u001FdMBTI\\u001FdNIC\\u001FdCStRLIN\\u001FdNIC\\u001E0 \\u001FaBR140\\u001Fb.J6\\u001E  \\u001Fa270.05\\u001E04\\u001FaThe Journal of ecclesiastical history\\u001E04\\u001FaThe Journal of ecclesiastical history.\\u001E  \\u001FaLondon,\\u001FbCambridge University Press [etc.]\\u001E  \\u001Fa32 East 57th St., New York, 10022\\u001E  \\u001Fav.\\u001Fb25 cm.\\u001E  \\u001FaQuarterly,\\u001Fb1970-\\u001E  \\u001FaSemiannual,\\u001Fb1950-69\\u001E0 \\u001Fav. 1-   Apr. 1950-\\u001E  \\u001FaEditor:   C. W. Dugmore.\\u001E 0\\u001FaChurch history\\u001FxPeriodicals.\\u001E 7\\u001FaChurch history\\u001F2fast\\u001F0(OCoLC)fst00860740\\u001E 7\\u001FaPeriodicals\\u001F2fast\\u001F0(OCoLC)fst01411641\\u001E1 \\u001FaDugmore, C. W.\\u001Fq(Clifford William),\\u001Feed.\\u001E03\\u001F81\\u001Fav.\\u001Fi(year)\\u001E40\\u001F81\\u001Fa1-49\\u001Fi1950-1998\\u001E  \\u001Fapfnd\\u001FbLintz\\u001E  \\u001Fa19890510120000.0\\u001E2 \\u001Fa20141106\\u001Fbm\\u001Fdbatch\\u001Felts\\u001Fxaddfast\\u001E  \\u001FlOLIN\\u001FaBR140\\u001Fb.J86\\u001Fh01/01/01 N\\u001E\\u001D"),
    new SourceRecord()
      .withId("77dfac11-1caf-4470-9ad1-d533f6360bdd")
      .withSource("{ \"leader\": \"00508cjm a22001813 4500\", \"fields\": [ { \"001\": \"10062588\" }, { \"005\": \"20171013073237.0\" }, { \"007\": \"sd fsngnnmmneu\" }, { \"008\": \"170825s2017 xx nn n zxx d\" }, { \"024\": { \"subfields\": [ { \"a\": \"00190295755553\" }, { \"2\": \"gtin-14\" } ], \"ind1\": \"7\", \"ind2\": \" \" } }, { \"024\": { \"subfields\": [ { \"a\": \"190295755553\" } ], \"ind1\": \"1\", \"ind2\": \" \" } }, { \"035\": { \"subfields\": [ { \"a\": \"(OCoLC)1002130878\" } ], \"ind1\": \" \", \"ind2\": \" \" } }, { \"035\": { \"subfields\": [ { \"a\": \"10062588\" } ], \"ind1\": \" \", \"ind2\": \" \" } }, { \"040\": { \"subfields\": [ { \"a\": \"BTCTA\" }, { \"b\": \"eng\" }, { \"c\": \"BTCTA\" } ], \"ind1\": \" \", \"ind2\": \" \" } }, { \"100\": { \"subfields\": [ { \"a\": \"Rossi, Daniele\" } ], \"ind1\": \"1\", \"ind2\": \" \" } }, { \"245\": { \"subfields\": [ { \"a\": \"Saint-Saens: Organ Symphony and Carnival of The Animals\" } ], \"ind1\": \"0\", \"ind2\": \"0\" } }, { \"260\": { \"subfields\": [ { \"b\": \"Wea Corp\" }, { \"c\": \"2017.\" } ], \"ind1\": \" \", \"ind2\": \" \" } }, { \"948\": { \"subfields\": [ { \"a\": \"20171013\" }, { \"b\": \"m\" }, { \"d\": \"batch\" }, { \"e\": \"lts\" }, { \"x\": \"deloclcprefix\" } ], \"ind1\": \"2\", \"ind2\": \" \" } } ] }")));


// send request to mod-source-record-storage
client.postSourceStoragePopulateTestMarcRecords(testData, handler->{
  if (handler.statusCode()==204){
    // processing response
    System.out.println("Records are successfully created.");
  }
});