[FOLIO-2102] Timestamp metadata missing from bootstrap data Created: 14/Jun/19 Updated: 03/Jun/20 Resolved: 06/Sep/19 |
|
| Status: | Closed |
| Project: | FOLIO |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | Bug | Priority: | P2 |
| Reporter: | Cate Boerema (Inactive) | Assignee: | Julian Ladisch |
| Resolution: | Done | Votes: | 0 |
| Labels: | platform-backlog | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Issue links: |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
| Sprint: | CP: sprint 70, CP: sprint 69, CP: sprint 71 | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| Story Points: | 2 | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| Development Team: | Core: Platform | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description |
|
Description The automatically loaded (bootsrapped) data should include timestamps (created at / updated at). Context We have suddenly found that records metadata seems to be missing from our bootstrap data (Users, Instances, Holdings, Items - maybe others not yet discovered). This causes the record metadata component to disappear or to display "invalid" where data should be.
After RMB has been updated as explained in
How to test: When this issue is done, please test
|
| Comments |
| Comment by Jakub Skoczen [ 14/Jun/19 ] |
|
Marc Johnson Cate Boerema is it all metadata including timestamps (createdDate..) or only related metadata? |
| Comment by Marc Johnson [ 14/Jun/19 ] |
|
Jakub Skoczen I think Zak Burke investigated briefly and mentioned that all of the change metadata was missing. Theodor Tolstoy (One-Group.se) confirmed that reason for this in the bug fest environment is due to the data being loaded directly. We think the same has been observed in the regular environments. |
| Comment by Cate Boerema (Inactive) [ 01/Jul/19 ] |
|
Jakub Skoczen this seems to have resolved itself. I am now seeing record metadata in folio-snapshot. I'll close this and all the various linked bugs. It would be interesting to know what happened here, though. It caused a ton of bug writing and discussion that would have better been avoided. Do you have thoughts on that? Maybe John Malconian knows what happened? |
| Comment by Cate Boerema (Inactive) [ 01/Jul/19 ] |
|
Actually, this is still a problem in folio-testing but not in folio-snapshot. Can you guys please fix it so we can close all these bugs? |
| Comment by Cate Boerema (Inactive) [ 01/Jul/19 ] |
|
Sorry guys - it looks like this is STILL a problem for both folio-testing and folio-snapshot. I mistakenly thought it was fixed for snapshot because the records I checked had been updated by a manual tester (that will set the last updated data). But if you find a record that hasn't been touched, the data is still missing in both environments. |
| Comment by Jakub Skoczen [ 05/Jul/19 ] |
|
Zak Burke Marc Johnson Adam Dickmeiss Guys, I don't get this issue. I understand the problem with the bugfest env was due to the fact the data was loaded directly into the DB and hence had to metadata. But for reference envs data loading is performed by the modules themselves so we should see "timestamps" but no users in the metadata. |
| Comment by Marc Johnson [ 05/Jul/19 ] |
Agreed, this is a red herring.
It seems the modules behave differently (see example records below). For users is no metadata property at all. For instances, sample records loaded at activation have a metadata object with explicitly null values for each property. For records loaded via the MODS ingest, the metadata is as expected. I don't think we know why this occurs. Could it be that some versions of the RAML Module Builder abort setting any of the properties if no user header is present? (I vaguely recall some conversations about this a few months ago). And that this behaviour varies by different versions of the RAML Module Builder? Users
{
"username" : "theodora",
"id" : "e59bc21a-884e-42a2-8792-163efc3662e7",
"barcode" : "144826156017341",
"active" : true,
"type" : "patron",
"patronGroup" : "3684a786-6671-4268-8ed0-9db82ebca60b",
"proxyFor" : [ ],
"personal" : {
"lastName" : "Dickinson",
"firstName" : "Francisca",
"middleName" : "Dock",
"email" : "rollin@jacobson-emmerich-and-schaefer.kg",
"phone" : "(071)378-8133 x66872",
"mobilePhone" : "373-226-0411",
"dateOfBirth" : "1945-12-20T00:00:00.000+0000",
"addresses" : [ {
"countryId" : "US",
"addressLine1" : "76540 Jaden Meadows #999",
"city" : "Toledo",
"region" : "MN",
"postalCode" : "49708",
"addressTypeId" : "93d3d88d-499b-45d0-9bc7-ac73c3a19880",
"primaryAddress" : true
} ],
"preferredContactTypeId" : "003"
},
"enrollmentDate" : "2016-08-13T00:00:00.000+0000",
"expirationDate" : "2020-03-18T00:00:00.000+0000",
"createdDate" : "2019-07-05T03:32:54.297+0000",
"updatedDate" : "2019-07-05T03:32:54.297+0000"
}
InstancesSample Records
{
"@context" : "http://folio-snapshot-okapi.aws.indexdata.com/inventory/instances/context",
"id" : "69640328-788e-43fc-9c3c-af39e243f3b7",
"hrid" : "in00000020",
"source" : "MARC",
"title" : "ABA Journal",
"parentInstances" : [ ],
"childInstances" : [ ],
"alternativeTitles" : [ ],
"editions" : [ ],
"series" : [ ],
"identifiers" : [ {
"identifierTypeId" : "913300b2-03ed-469a-8179-c1092c991227",
"value" : "0747-0088"
}, {
"identifierTypeId" : "c858e4f2-2b6b-4385-842b-60732ee14abb",
"value" : "84641839"
} ],
"contributors" : [ ],
"subjects" : [ ],
"classifications" : [ ],
"publication" : [ {
"publisher" : "American Bar Association",
"place" : "Chicago, Ill.",
"dateOfPublication" : "1915-1983",
"role" : null
} ],
"publicationFrequency" : [ ],
"publicationRange" : [ ],
"electronicAccess" : [ ],
"instanceTypeId" : "6312d172-f0cf-40f6-b27d-9fa8feaf332f",
"instanceFormatIds" : [ ],
"physicalDescriptions" : [ ],
"languages" : [ ],
"notes" : [ ],
"statisticalCodeIds" : [ ],
"metadata" : {
"createdDate" : null,
"createdByUserId" : null,
"updatedDate" : null,
"updatedByUserId" : null
},
"links" : {
"self" : "http://folio-snapshot-okapi.aws.indexdata.com/inventory/instances/69640328-788e-43fc-9c3c-af39e243f3b7"
}
}
MODS Ingest
{
"@context" : "http://folio-snapshot-okapi.aws.indexdata.com/inventory/instances/context",
"id" : "8d56cae7-3b5f-44c5-9a62-53b7d21fb06c",
"hrid" : "in00000120",
"source" : "Local: MODS",
"title" : "1963 Birmingham church bombing",
"parentInstances" : [ ],
"childInstances" : [ ],
"alternativeTitles" : [ ],
"editions" : [ ],
"series" : [ ],
"identifiers" : [ {
"identifierTypeId" : "8261054f-be78-422d-bd51-4ed9f33c3422",
"value" : "15440252"
}, {
"identifierTypeId" : "8261054f-be78-422d-bd51-4ed9f33c3422",
"value" : "9780756540920 (library binding)"
}, {
"identifierTypeId" : "8261054f-be78-422d-bd51-4ed9f33c3422",
"value" : "2008038921"
} ],
"contributors" : [ {
"contributorNameTypeId" : "2b94c631-fca9-4892-a730-03ee529ffe2a",
"name" : "Klobuchar, Lisa.",
"contributorTypeId" : "",
"contributorTypeText" : "",
"primary" : null
} ],
"subjects" : [ ],
"classifications" : [ ],
"publication" : [ ],
"publicationFrequency" : [ ],
"publicationRange" : [ ],
"electronicAccess" : [ ],
"instanceTypeId" : "6312d172-f0cf-40f6-b27d-9fa8feaf332f",
"instanceFormatIds" : [ ],
"physicalDescriptions" : [ ],
"languages" : [ ],
"notes" : [ ],
"statisticalCodeIds" : [ ],
"metadata" : {
"createdDate" : "2019-07-05T03:35:57.709+0000",
"createdByUserId" : "a5df7bf3-c1c0-5937-aa2a-1b2f301eba49",
"updatedDate" : "2019-07-05T03:35:57.709+0000",
"updatedByUserId" : "a5df7bf3-c1c0-5937-aa2a-1b2f301eba49"
},
"links" : {
"self" : "http://folio-snapshot-okapi.aws.indexdata.com/inventory/instances/8d56cae7-3b5f-44c5-9a62-53b7d21fb06c"
}
}
|
| Comment by Zak Burke [ 05/Jul/19 ] |
|
I think
Would completing that ticket be another path to closing these related tickets? I pushed the branch to GitHub if somebody else wants to pick it up. I think all that's left is unit tests. |
| Comment by Marc Johnson [ 05/Jul/19 ] |
I think completing
I believe the expectation is that it should be present, and so these issues won't be resolved until the correct aspects of change metadata are present. I shall leave it to Cate Boerema to specify what properties have to be present for this to be acceptable. Above, Jakub Skoczen suggested that only the date time properties were needed to resolve this. Is that the case? |
| Comment by Zak Burke [ 05/Jul/19 ] |
|
|
| Comment by Cate Boerema (Inactive) [ 05/Jul/19 ] |
This is what Jakub and I agreed to. Since the data is auto-generated in our test environments, we didn't really know what to use as the source. This is why we suggested modifying
UIIN issue filed for this:
Thanks Marc Johnson and Zak Burke |
| Comment by Oleksii Popov [ 10/Jul/19 ] |
|
As a result of investigation, we should come with a solution and an estimation. |
| Comment by Cate Boerema (Inactive) [ 29/Jul/19 ] |
|
Jakub Skoczen is this planned for sprint 69? |
| Comment by Julian Ladisch [ 13/Aug/19 ] |
|
See
|
| Comment by Julian Ladisch [ 27/Aug/19 ] |
|
Fixed in https://folio-testing.aws.indexdata.com/ and https://folio-snapshot.aws.indexdata.com/ |
| Comment by Ann-Marie Breaux (Inactive) [ 05/Sep/19 ] |
|
Julian Ladisch and Cate Boerema this showed up in the manual testing queue, but I don't think there's anything for us to test. Everything here seems to have been closed. Do you want to close this ticket? Or let us know if there's something we should test manually. Thank you! |
| Comment by Cate Boerema (Inactive) [ 06/Sep/19 ] |
|
Oh - yay! This is finally fixed. Thank you Julian Ladisch. So great to see all the dependent bugs closed, as well. |