This page is meant to track progress on issues such as bugs, new features, or topics to be discussed for Data Import. Topics or questions posted in slack will be added here as well.
Topic Status legend:
Status | ||||
---|---|---|---|---|
|
Status | ||
---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Priority Status legend:
Status | ||||
---|---|---|---|---|
|
Status | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
. Closed topics are found on our Archived Data Import Topics page.
How to contribute to other people's discussion topics:
Do not add detail to closed or discussed topics as your comments may be overlooked. In this situation, it might be best to Add your details as a new topic and reference the previous topic.
To contribute to an existing topic. Add a new paragraph to the description column.
@mention yourself at the beginning of the paragraph
How to indicate you are also interested in a topic:
@mention yourself in the "Interested parties" column and add your institution name
How are topics archived:
When a topic status is set to closed by it's "Owner". The topic must also be moved to the " Data Import Topic Tracker Archive" page.
Copy the topic and paste it at the top of the Archived topics page that is nested under this page
Delete the topic from this page
Data Import Issues by Status
Jira Charts | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Data Import Issues by Type
Jira Charts | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
List of Data Import Jira Issues
Jira Legacy | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Topics
Status can be sorted to see Open, In Progress, Closed or Blocked
Status | Topic | Description/use case | Date Added | Provided By (Name/Institution) | Interested Parties | Has Been Discussed (Link to agenda/minutes) | Jira Link | Action Required | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 |
| 2023-06-13 | All | 2024-1-17 Data Import Subgroup meeting 2024-1-10 Data Import Subgroup meeting
| MARC-MARC Matching Enhancements |
Expand |
---|
We want to ensure that MARC-MARC matching works properly for repeatable and non-repeatable fields, especially 0XX/9XX fields, and that they can pair well with Inventory submatches. In scope:
Out of scope:
Use case(s):
|
2020-05-13
All
All
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
- More Use Cases and scenarios
Status | ||||
---|---|---|---|---|
|
Data Import removes duplicate 856s in SRS
Overview: When updating an SRS record via Data Import, some MARC fields are duplicated while others are de-duped without notification or guidelines.
**
Expand |
---|
Steps to Reproduce:
Expected Results: The SRS record contains duplicate 856 fields. Additional Information: We know that Data Import does not de-dupe the 903 field, for example, during an update but it does the 856 field. Data Import jobs which create new SRS records includes the duplicate 856 fields. This raises several questions:
From testing, there appears to be no difference between de-duping of the 856 when field protections are applied or not. |
|
Jobs run immediately after canceled jobs take excess time
Overview: Jobs started immediately after canceling a job get stuck and don't progress
Expand |
---|
Steps to Reproduce:
Expected Results: The job cancels and stops processing. Data import jobs started after the cancellation act normally. Actual Results: Single record imports started after canceling a job like this are slow. One single record import after a cancelation took 17 minutes. Additional Information: In Nolana, canceled jobs created a large number of error messages that seemed to affect performance. Have the logs been checked to be sure this isn't still happening? Is something else causing this behavior? Attached file and profile "cornell ebz" can be used to replicate. BE notes (possible solution):
|
2023-03-29
All
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
- Need to define what deduplication means
- Clarify expectations
- Deduping in the UI vs SRS
- Deduping the incoming record
Status | ||||
---|---|---|---|---|
|
Jobs run immediately after canceled jobs take excess time
Overview: Jobs started immediately after canceling a job get stuck and don't progress
expand
|
Converted from bug to new feature.
Ryan T. to get information on how this affects slicing
Status | ||
---|---|---|
|
The number of created invoices is displayed when all invoices have errors with invoice lines
Overview: **
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Expand |
---|
The file has 18 invoices and 1104 invoice lines. Steps to Reproduce:
Expected Results: The job cancels and stops processing. Data import jobs started after the cancellation act normally. Actual Results: Single record imports started after canceling a job like this are slow. One single record import after a cancelation took 17 minutes. Additional Information: In Nolana, canceled jobs created a large number of error messages that seemed to affect performance. Have the logs been checked to be sure this isn't still happening? Is something else causing this behavior? Attached file and profile "cornell ebz" can be used to replicate. BE notes (possible solution):
|
2023-03-29
All
Steps to Reproduce:
3. Create a new "Action Profile" with a unique valid name and with the next properties in it:
4. Create a new "Job profile" with a unique valid name in it and with the next properties in it:
5. Upload a valid EDIFACT file using Job profile from the previous step. 6. Wait till the file is uploaded. 7. Check log UI and summary to see the record with the upload result. 8. Pay attention on the 'Invoice' column in the 'Created' row. Expected Results: The '0' number of created invoices is displayed in cells in the row with the 'Created' row in the 'Summary' table in logs. NOTE: Recreated on Poppy Bugfest:
|
2023-11-23
Tetiana Paranich
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Converted from bug to new feature.
Ryan T. to get information on how this affects slicing
|
2023-09-14
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Data Import log displays ambiguous information for successful matches on anything but 999i/instance uuid
f your match profile matches incoming MARC 020 on instance ISBN, and the incoming MARC record matches an instance, the reported status in the log of the SRS record will be Updated if the MARC record also contains a 999-ff-i matching the instance's UUID, but Created if the incoming MARC record does not contain a 999-ff-i. In both cases the status reported for the instance in the log is Updated.
Expand |
---|
To a librarian wishing to overlay existing FOLIO records with matching incoming MARC record, it is a source of great confusion that, when there is a match and an underlying SRS record, the status of the SRS record is sometimes reported as Updated and sometimes as Created – even if you are matching on the same field, overlaying the same record. This may cause enough uncertainty about whether Data Import works as expected, overlaying the right records when it should, to make the librarian reluctant about using Data Import. It seems that either
It would be great, if as a first step, someone with insight into how Data Import works could review the current behaviour to assess whether librarians can "safely" use Data Import with match profiles even though the log shows this ambiguous result. Steps to Reproduce: To test, you need:
Use the following job profile: Try the following three:
Expected Results:
|
2022-05-30
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Field is shown after being removed via data import when field mapping profile has rule allowing updates for this field
Field is shown after being removed via data import (when field mapping profile has rule allowing updates for this field)
Expand |
---|
Preconditions:
Steps to Reproduce:
Expected Results: Deleted "830" field (see step 6) is not shown. Actual Results: Deleted "830" field (see step 6) is shown and has divided boxes (see attached screencast). |
2023-03-09
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||
---|---|---|
|
match on 035$a with qualifier fails
When updating an SRS record using a match on the 035$a with a qualifier on the incoming MARC record, the match fails.
Expand |
---|
Steps to Reproduce:
Expected Results: The job matches the incoming records to the SRS records associated with in10783235 and in10783236 and updates these records with the new 856 in the incoming MARC record. Actual Results: The incoming records are not matched, the log for SRS MARC says 'No action', and the SRS record is not updated. Additional Information: I tested the Field Mapping Profile without the 856 protection and it still failed (Job 10641). When reviewing logs on an internal system, the error messages given note that a match is not found. Additional testing was done on changing the 'Match criterion' of the existing record and no value was found to make the Match profile successful. Original testing done in an Orchid environment. |
2023-11-15
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Ryan to review Jira with Folijet leads to understand current design and identify requirement gaps
Partial matching, e.g. begins with, ends with, is required but it does not function as it should. Only exact matching seems to work.
Status | ||||
---|---|---|---|---|
|
data import sorts protected fields out of order after update
Field protections moves the protected field to the first Nxx field. For instance, if the protected field is a 541, the protected 541 becomes the first of all 5xx tags. (see screen shot attached.)
Expand |
---|
Steps to Reproduce:
Expected Results: Both 541 fields will be next to each other filed according to their previous position. I think it can be assumed that protected 541 will be the first 541 in the new record where both 541 fields are present. Actual Results: The protected 541 is filed at the beginning of all 5xx fields in the record and the new 541 is in place as it was in the incoming file. |
2023-05-08
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Subfield can't be removed when updating Marc bib upon import
Subfield cannot be removed when updating "MARC Bib" upon import when field mapping profile has rules allowing update of several subfields in all fields (including the subfield which is being added)
Expand |
---|
Preconditions: • The job profile should be created for "Data import" app: "Update MARC Bib records by matching 999 ff $s subfield value (subfields 1, 2)" • "Match profile" should have following specified: Field: 999
• "Field mapping profile" should have following rules specified: Update specific fields only Field: * Field: * No Field protection overrides • Attached "MARC Bib" record should be imported to the system via "Default - Create instance and SRS MARC Bib" job profile (it has subfields "$1", "$2" in two fields) Steps to Reproduce:
Expected Results: "$2" removed from both edited fields. Following "$1"subfield values are shown in fields:
Actual Results: "$2" subfields are not removed from edited fields. "$1" subfields updated/added as expected |
2023-06-01
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Fields duplicated when adding one subfield when updating Marc bib upon import
Fields duplicated when adding one subfield when updating "MARC Bib" upon import when field mapping profile has rules allowing update of several subfields in all fields (including the subfield which is being added)
Expand |
---|
Preconditions: • The job profile should be created for "Data import" app: "Update MARC Bib records by matching 999 ff $s subfield value (subfields 1, 2)" • "Match profile" should have following specified: Field: 999
• "Field mapping profile" should have following rules specified: Update specific fields: Field: * Field: * No overrides to MARC field protection • Attached "MARC Bib" record should be imported to the system via "Default - Create instance and SRS MARC Bib" job profile Steps to Reproduce:
Expected Results: "$1" added to edited "035", "264" fields. Fields were not duplicated Actual Results: "035", "264" fields were duplicated. First copy of each remains unchanged (no added subfield), while second copies contain added "$1" subfield |
2023-06-01
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Fields duplicated when adding several subfields when updating marc bib upon import
Fields duplicated when adding several subfields when updating "MARC Bib" upon import when field mapping profile has rules allowing update of corresponding subfields in corresponding fields
Expand |
---|
Preconditions: • The job profile should be created for "Data import" app: "Update MARC Bib records by matching 999 ff $s subfield value (240, 600)" • "Match profile" should have following specified: Field: 999
• "Field mapping profile" should have following rules specified: Update specific MARC fields Field: 240 Field: 240 Field: 600 Field: 600 No overrides to the existing field protections • Attached "MARC Bib" record should be imported to the system via "Default - Create instance and SRS MARC Bib" job profile Steps to Reproduce:
Expected Results: "$1", "$2" added to edited "240", "600" fields. Fields were not duplicated Actual Results: "240", "600" fields were duplicated. First copy of each remains unchanged (no added subfields), while second copies contain added "$1", "$2" subfields |
2023-06-01
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Job summary: error column does not display errors
When there is any error related to an instance/authority/orders/invoice, the error column does not display it.
2024-01-25
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Field mapping profiles: state of the final form fields is not set
When switching between Folio record types fields with the same name do not reset the state (value, dirty, etc.), although the field values are equal to the initial.
Current workaround: start over/refresh page
2021-06-30
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Incorrect quantity is displayed in the cell of no action and error rows at the individual import job's log
The '1' number of Instance is displayed in cell in the row with the 'No action' and 'Error' rows header in the 'Summary table' at the individual import job's log.
2024-01-19
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
The status of srs marc is created after match+modify action
Expected Results: The status of SRS MARC is 'Updated' in the Import log after uploading MARC file for update.
2023-03-07
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
PMSystem displayed as source in quickmarc view when record was created by non matches action of job profile
"PMSystem" displayed as source (instead of User's last and first name) in "Edit MARC authority record" view when record was created by "Non-matches" action of job profile.
2023-03-07
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Duplicate field is added when updating $0 in linked marc bib field upon data import if field mapping profile allows $0 update
Duplicate field is added when updating "$0" in linked "MARC bib" field upon data import if field mapping profile specifically allows "$0" update
2023-02-15
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Single record overlay creates duplicate oclc #/035
When "Overlay source bibliographic record' is employed for the first time, duplicate 035 fields are created. **
2023-04-12
Status | ||||
---|---|---|---|---|
|
Adding MARC modifications to imports with update actions creates broken records
Overview:
Expand |
---|
Steps to Reproduce:
Expected Results: Overlaying the record works. The MARC is modified as described in the profile and the instance and SRS are updated. Actual Results: The instance is not updated. A modified SRS record is created but still has the original OCLC 001 and 003. QuickMARC will not work on the record. Additional Information: An example of an instance with this issue: https://bugfest-mg.int.aws.folio.org/inventory/view/53e28701-dccc-49be-a01d-9adaa15f4cb6?query=neuromancer&sort=title&xidtype=0dd718cf-a09a-4f1c-be6a-0cf0de58b424 Job profiles
BE Notes:
Note for QAs: |
2022-08-04
All
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Adding MARC modifications to single record overlay doesn't respect field protections
Overview: When a MARC modification action is added to the end of the single record overlay job, protected fields from the existing MARC SRS are removed rather than from the incoming file.
Expand |
---|
Steps to Reproduce:
Expected Results: The existing and protected fields should remain in the record. Actual Results: The protected fields have been removed. Additional Information: |
2023-08-23
All
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Functionality audit being prepared in DI lab
Status | ||||
---|---|---|---|---|
|
Single record overlay creates duplicate OCLC # / 035
Overview: When "Overlay source bibliographic record' is employed for the first time, duplicate 035 fields are created.
Expand |
---|
Steps to Reproduce:
Expected Results: The bibliographic record and instance are updated with the latest OCLC version. Records tested in bugfest-orchid: in523951 and in2486915 (screenshots of before overlay and after are attached) Duplicate data causes issues with integrations and other functions that rely on the OCLC# as a match point. |
2023-04-12
All
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||
---|---|---|
|
The number of created invoices is displayed when all invoices have errors with invoice lines
Overview: **
The file has 18 invoices and 1104 invoice lines.
Steps to Reproduce:
Admin user is logged in.
Steps to Reproduce:
Duplicate the "Default – GOBI monograph invoice" profile.
Update next fields into the copied profile:
Name: any unique name
Incoming record type: EDIFACT Invoice
FOLIO record type: Invoice
Description: clean-up the field
Details block:
1) Batch group: any option from the dropdown list
2) Vendor name: use Organization lookup to find and select GOBI Library Solutions (which will also automatically fill in the Accounting code)
3)Payment method: any option from the dropdown list.
3. Create a new "Action Profile" with a unique valid name and with the next properties in it:
Action: Create
FOLIO record type: Invoice
Link the field mapping profile from step#2.
4. Create a new "Job profile" with a unique valid name in it and with the next properties in it:
Accepted data type: EDIFACT
Link an action profile from step#3.
5. Upload a valid EDIFACT file using Job profile from the previous step.
6. Wait till the file is uploaded.
7. Check log UI and summary to see the record with the upload result.
8. Pay attention on the 'Invoice' column in the 'Created' row.
Expected Results: The '0' number of created invoices is displayed in cells in the row with the 'Created' row in the 'Summary' table in logs.
Actual Results: The '18' number of created invoices is displayed in cells in the row with the 'Created' row in the 'Summary' table in logs.
NOTE: Recreated on Poppy Bugfest:
Job profile: https://bugfest-poppy.int.aws.folio.org/settings/data-import/job-profiles/view/22c61a1b-3d03-4ddd-bdc8-f885ceab3c61?query=gobi&sort=name (did not include an EDIFACT mapping in the Invoice line description field, so every invoice line errored)
Log: https://bugfest-poppy.int.aws.folio.org/data-import/job-summary/c541334a-15c7-40e7-89a4-e135d87e11f6
2023-11-23
Tetiana Paranich
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Investigate deleting old versions of records from SRS, SPIKE
When SRS records are updated, the previous version is marked as old (and the newest version is marked as actual), but the older versions are not deleted. Over time, many, many previous versions of records will build up in SRS and potentially affect performance.
Expand |
---|
If we wanted to remove the old records, how complicated would that be, and what might we need to take into consideration? KS: there are also lots of "trash" data saved to SRS as a result of failed or stopped imports (records linked to a snapshot/jobExecution that is Cancelled or records that don't have 999 ff i UUIDs) - consider ways to clean up that data as well.
Results of this spike
|
2022-08-16
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Not able to use the system generated match profiles
I am attempting to create a new job profile for single record import to get rid of the junk fields. I was able to do this a while back on our test tenant and today I was finally able to get around to creating it on our production tenant.
Expand |
---|
However, when I went to add the Match profile (Inventory Single Record - Default match for existing SRS record) which is a system generated profile, it did not appear as one of the options. When I look at the Action profiles, I am able to choose system generated ones, so the problem is only with the Match profiles. We are on Orchid-SP-5. I was also able to recreate the issue on Snapshot. (I don't have Bugfest access so I did not try that.) Steps to Reproduce:
Expected Results: A list of all Match Profiles (both system provided and locally created) is shown Actual Results: Only locally created Match Profiles are shown (if there are none, you get "no results found") Additional Information: |
|
Investigate deleting old versions of records from SRS, SPIKE
When SRS records are updated, the previous version is marked as old (and the newest version is marked as actual), but the older versions are not deleted. Over time, many, many previous versions of records will build up in SRS and potentially affect performance.
Expand |
---|
If we wanted to remove the old records, how complicated would that be, and what might we need to take into consideration? KS: there are also lots of "trash" data saved to SRS as a result of failed or stopped imports (records linked to a snapshot/jobExecution that is Cancelled or records that don't have 999 ff i UUIDs) - consider ways to clean up that data as well.
Results of this spike
|
2022-08-16
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Status descending sort on Data Import view all page not working
In Honeysuckle Bugfest, on the Data Import View all, the status sort ascending works, but not descending
2020-12-09
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
MARC holdings update log has additional empty row
MARC Holdings update log has additional empty row. If two "MARC Holdings" records are updated by one job, then 2 additional empty rows will be displayed.
2023-05-01
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Data Import field mapping profile is saved with data deleted from the system
The user can save a mapping profile with data that has been deleted from the system.
2022-08-19
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Alert modal with error message is displayed on page after entering '##*' characters and clicking on search button
see steps in JIRA
2022-06-10
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
DI Log: Title missing but status reads updated
see JIRA
2023-10-24
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Asynchronous migration is not completed
The asynchronous migration script was run but migration has not been completed, the migration job is still IN_PROGRESS.
2023-06-04
Status | ||||
---|---|---|---|---|
|
Invoice level adjustments do not work
When loading an EDIFACT invoice using a field mapping profile with invoice-level adjustments, the adjustments error
2021-03-29
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Invoice line level adjustments don't work
When loading an EDIFACT invoice using a field mapping profile with invoice-level adjustments, the adjustments error
2021-03-29
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Data import incorrectly maps Resource type for no display constant generated
See steps in JIRA
2023-04-19
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Incorrect behavior of "Delete Files" button
Note: Does not always reproduce
"Delete files" request deletes the file, but does not always show in the UI
Delete request returns "Cannot delete uploadDefinition 09ef7415-34e4-44cd-9af3-31953df9f200 - linked files are already being processed".
2022-06-02
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
|
Field is shown after being removed via data import when field mapping profile has rule allowing updates for this field
Field is shown after being removed via data import (when field mapping profile has rule allowing updates for this field)
Expand |
---|
Preconditions:
Steps to Reproduce:
Expected Results: Deleted "830" field (see step 6) is not shown. Actual Results: Deleted "830" field (see step 6) is shown and has divided boxes (see attached screencast). |
2023-03-09
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
2023-02-20
Status | ||||
---|---|---|---|---|
|
RRT - Invoices don't display fund codes
Institution specific - MI State Univ./ Library of Michigan
2024-01-04
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Review and fix Marc updates for individual fields
Currently (as of Orchid), the Data Import MARC Updates for specific fields do not handle repeatable fields properly. The logic needs updating, and UI may need updating to indicate how incoming repeatable MARC fields should be handled vis-a-vis the same repeatable field(s) in the existing SRS MARC Bib. This is similar to how the field protection logic needed updating to handle repeatable vs non-repeatable fields properly.
|
match on 035$a with qualifier fails
When updating an SRS record using a match on the 035$a with a qualifier on the incoming MARC record, the match fails.
Expand |
---|
Steps to Reproduce:
Expected Results: The job matches the incoming records to the SRS records associated with in10783235 and in10783236 and updates these records with the new 856 in the incoming MARC record. Actual Results: The incoming records are not matched, the log for SRS MARC says 'No action', and the SRS record is not updated. Additional Information: I tested the Field Mapping Profile without the 856 protection and it still failed (Job 10641). When reviewing logs on an internal system, the error messages given note that a match is not found. Additional testing was done on changing the 'Match criterion' of the existing record and no value was found to make the Match profile successful. Original testing done in an Orchid environment. |
2023-11-15
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
|
Ryan to review Jira with Folijet leads to understand current design and identify requirement gaps
Partial matching, e.g. begins with, ends with, is required but it does not function as it should. Only exact matching seems to work.
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||
---|---|---|
|
Partial matching doesn't work
Partial matching, e.g. begins with, ends with, is required but it does not function as it should. Only exact matching seems to work.
2021-01-25
Yael Hod (Stanford)
Corrie Hutchinson (Unlicensed) Chicago
2024-1-31 Data Import Subgroup meeting
Status | ||||
---|---|---|---|---|
|
Job profile with POL/VRN match cascade does not finish properly
Some Inventory records (Instances/Holdings/Items) get created when orders are opened, depending on the Inventory setting in the POL. During testing of POL/VRN matching, I noticed that some holdings being updated by importing MARC Bibs were having their source changed from FOLIO to MARC. We need to ensure this DOES NOT happen.
2022-05-25
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Poppy data import log does not include links to records for SRS updates.
For SRS updates with no instance update, the changes to the import logs have resulted in no record links in the log. Before the log indicated that the instance was updated and included a link to the instance. Now, the log indicates that the SRS record is updated, but there is no link. I am guessing it is because there is no SRS record per se in FOLIO. Can we revisit the decision to not display the instance update status with a link? Or add a link to the instance to the SRS updated status? PPT from Data Import subgroup work:
Widget Connector | ||
---|---|---|
|
From this spreadsheet it appears that the instance should also be updated and provide the record link:
Lref gdrive file | ||
---|---|---|
|
|
Subfield can't be removed when updating Marc bib upon import
Subfield cannot be removed when updating "MARC Bib" upon import when field mapping profile has rules allowing update of several subfields in all fields (including the subfield which is being added)
Expand |
---|
Preconditions: • The job profile should be created for "Data import" app: "Update MARC Bib records by matching 999 ff $s subfield value (subfields 1, 2)" • "Match profile" should have following specified: Field: 999
• "Field mapping profile" should have following rules specified: Update specific fields only Field: * Field: * No Field protection overrides • Attached "MARC Bib" record should be imported to the system via "Default - Create instance and SRS MARC Bib" job profile (it has subfields "$1", "$2" in two fields) Steps to Reproduce:
Expected Results: "$2" removed from both edited fields. Following "$1"subfield values are shown in fields:
Actual Results: "$2" subfields are not removed from edited fields. "$1" subfields updated/added as expected |
2023-06-01
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
- Review Jira with Folijet leads to understand current design and identify requirement gaps.
|
2024-7-10 Data Import Subgroup meeting
Status | ||||
---|---|---|---|---|
|
RRT, 5C match bug
Problem of match that didn't match
2023-09-28
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Duplicate records in incoming file causes problems after overlay process with no error reported
When overlaying instance records, if the incoming file has duplicate records and therefore multiple incoming matches for one match in FOLIO, the record that was overlaid in FOLIO cannot be opened using quickMARC. Note: In this scenario the incoming file has the duplicate records and therefore duplicate match points. FOLIO does not have duplicates.
2021-12-15
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Add new subfields to Electronic access (856)
New subfields in the MARC 856 field need to be represented in Inventory data. The same elements should appear in the electronic access block in Instance, Holdings, and Item records. https://www.loc.gov/marc/bibliographic/bd856.html
2023-09-14
|
Fields duplicated when adding one subfield when updating Marc bib upon import
Fields duplicated when adding one subfield when updating "MARC Bib" upon import when field mapping profile has rules allowing update of several subfields in all fields (including the subfield which is being added)
Expand |
---|
Preconditions: • The job profile should be created for "Data import" app: "Update MARC Bib records by matching 999 ff $s subfield value (subfields 1, 2)" • "Match profile" should have following specified: Field: 999
• "Field mapping profile" should have following rules specified: Update specific fields: Field: * Field: * No overrides to MARC field protection • Attached "MARC Bib" record should be imported to the system via "Default - Create instance and SRS MARC Bib" job profile Steps to Reproduce:
Expected Results: "$1" added to edited "035", "264" fields. Fields were not duplicated Actual Results: "035", "264" fields were duplicated. First copy of each remains unchanged (no added subfield), while second copies contain added "$1" subfield |
2023-06-01
2024-01-31
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
- Developers will look into this
Status | ||||
---|---|---|---|---|
|
There needs to be a warning or error to stop the job when a job contains no action profiles.
If a job doesn't have any actions, nothing happens and there is a risk that the records are corrupted.
2021-02-08
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
- Ryan will ask what the behavior is for when actions are missing and you try to run a job or if you can edit a job with no actions.
Behavior now in FOLIO snapshot
Status | ||||
---|---|---|---|---|
|
When importing EDIFACT files, invoices lines aren't in order
When importing an Edifact file, the invoices lines aren't in order.
2024-02-06
Corrie Hutchinson, Jennifer Eustis
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Enforce an order to deletion for Data Import profiles
There needs to be an enforced order to deletion for Data import profiles to prevent this or there should be a confirmation window that lets you delete associated match/action profiles with the job profile if they aren't in use in another Job profile
Expand |
---|
From 2024-01-31 chat: Technically, not true Lynne. I accidentally deleted a job profile without unlinking stuff first (which I was not stopped from doing). I was then not allowed to delete the associated profiles because they needed to be unlinked first, which I could not do because I could no longer get to the job profile to unlink them. |
|
Status | ||||
---|---|---|---|---|
|
Fields duplicated when adding several subfields when updating marc bib upon import
Fields duplicated when adding several subfields when updating "MARC Bib" upon import when field mapping profile has rules allowing update of corresponding subfields in corresponding fields
Expand |
---|
Preconditions: • The job profile should be created for "Data import" app: "Update MARC Bib records by matching 999 ff $s subfield value (240, 600)" • "Match profile" should have following specified: Field: 999
• "Field mapping profile" should have following rules specified: Update specific MARC fields Field: 240 Field: 240 Field: 600 Field: 600 No overrides to the existing field protections • Attached "MARC Bib" record should be imported to the system via "Default - Create instance and SRS MARC Bib" job profile Steps to Reproduce:
Expected Results: "$1", "$2" added to edited "240", "600" fields. Fields were not duplicated Actual Results: "240", "600" fields were duplicated. First copy of each remains unchanged (no added subfields), while second copies contain added "$1", "$2" subfields |
2023-06-01
Jira Legacy | ||||||||
---|---|---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Duplicate field is added when updating $0 in linked marc bib field upon data import if field mapping profile allows $0 update
Duplicate field is added when updating "$0" in linked "MARC bib" field upon data import if field mapping profile specifically allows "$0" update
2023-02-15
Jira Legacy | ||
---|---|---|
|
|
|
Overview: The Poppy release introduced functionality to create multiple holdings and items from a single MARC bib, using data from 9xx fields in the MARC record (see UXPROD-2741: Import of MARC Bibs to create/update multiple holdings and items: BE workCLOSED )
However, if the holdings or item field mapping profile contains a conditional mapping (e.g. Permanent holdings location = 945$a; else “LOCCODE”), only the first specified 9xx field will be used to create a single holdings/item.
Expand |
---|
Steps to Reproduce:
Case 1:
Expected Results: Multiple holdings and items are created. Actual Results: Only a single holdings/item representing the first 945 field is created. Case 2:
Expected Results: Job finished with status “Completed with errors“, error log for holdings says that permanent location should be not null Actual Results: Job finished with status “Completed with errors“, error log for holdings show StackTraceThrowable exception Additional Information: Removing the ; else “LOCCODE” produced expected results. See my tests by looking at the logs for 12177 (successful creation of multiple holdings/items) and 12180 (only created single holdings/item). The MARC records used for these examples and the job profiles are listed within the log. |
Status | ||||
---|---|---|---|---|
|
Data import job profile will not create multiple holdings/items when conditional mapping is used in field mapping profiles
|
Incorrect behavior of "Delete Files" button
Note: Does not always reproduce
"Delete files" request deletes the file, but does not always show in the UI
Delete request returns "Cannot delete uploadDefinition 09ef7415-34e4-44cd-9af3-31953df9f200 - linked files are already being processed".
2022-06-02
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Asynchronous migration is not completed
The asynchronous migration script was run but migration has not been completed, the migration job is still IN_PROGRESS.
2023-06-04
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Review and fix Marc updates for individual fields
Currently (as of Orchid), the Data Import MARC Updates for specific fields do not handle repeatable fields properly. The logic needs updating, and UI may need updating to indicate how incoming repeatable MARC fields should be handled vis-a-vis the same repeatable field(s) in the existing SRS MARC Bib. This is similar to how the field protection logic needed updating to handle repeatable vs non-repeatable fields properly.
2023-02-20
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
- Request to be added to Poppy CSP 2
|
2024-02-27
Status | ||||
---|---|---|---|---|
|
Item creation using Data Import is missing data
Overview : When importing a MARC bibliographic record with 9xx fields designating order, holdings, and item record data, the enumeration and copy number fields of the item record fail to populate as instructed.
|
Partial matching doesn't work
Partial matching, e.g. begins with, ends with, is required but it does not function as it should. Only exact matching seems to work.
2021-01-25
Yael Hod (Stanford)
Corrie Hutchinson (Unlicensed), Christie Thomas Chicago
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
|
- Review Jira with Folijet leads to understand current design and identify requirement gaps.
2024-2-28 Data Import Subgroup meeting
Status | ||||
---|---|---|---|---|
|
Ability to change the link to a profile rather than just remove it
Current situation: We are only able to link or unlink profiles (field mapping to action, action to a job, match to a job, etc).
New Feature: We want to be able to change the link rather than just unlink
Expected behavior: There is another option that allows the user to change the link to a different profile.
Use case: The wrong profile was used and the new one needs to be added. Rather than unlinking everything, it'd be easier to just update the link to the correct one.
2024-02-27
|
Add new subfields to Electronic access (856)
New subfields in the MARC 856 field need to be represented in Inventory data. The same elements should appear in the electronic access block in Instance, Holdings, and Item records. https://www.loc.gov/marc/bibliographic/bd856.html
2023-09-14
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
- Ryan will create a ticket. This might involve rethinking the profiles page setup.
|
Overview : When creating orders with an order format of electronic, an instance, and a holdings records from a MARC bibliographic record using Data Import, the quantity in both the ‘Cost details' and ‘Location’ section of the POL is ‘0' despite the field mapping profile instructing it be '1’.
A nearly identical profile for order format equal to print does not display this behavior
Status | ||||
---|---|---|---|---|
|
Quantity = 0 in POL for order format equal to electronic
|
Ability to change the link to a profile rather than just remove it
Current situation: We are only able to link or unlink profiles (field mapping to action, action to a job, match to a job, etc).
New Feature: We want to be able to change the link rather than just unlink
Expected behavior: There is another option that allows the user to change the link to a different profile.
Use case: The wrong profile was used and the new one needs to be added. Rather than unlinking everything, it'd be easier to just update the link to the correct one.
2024-02-27
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
DI Jobs stall when matching on a holdings and/item nested under an instance
2024-03-01
2024-2-28 Jennifer Eustis
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
- CSP 3 Poppy
|
- Ryan will create a ticket. This might involve rethinking the profiles page setup.
Status | ||||
---|---|---|---|---|
|
Unable to pull vendor account number from POL when importing EDIFACT invoices
In our previous system the vendor account number lived at the PO/Invoice level. Now it is on the POL/invoice line. I have not found a way when loading EDIFACT invoice files to draw this directly from the POL or to retrieve it from the vendor file. This means for each invoice we must put in all of the vendor account numbers manually, which adds up and is prone to error. If there is a way that data import could pull this value from the POL it would save so much time in our processing.
2024-04-12
- Need more information from Kimberly Pamplin
Status | ||||
---|---|---|---|---|
|
Import profile with Instance match to POL and Vendor Reference Number not working
Overview: Vendor records were received containing the POL and Vendor Reference Number (VRN) are not matching to the source = FOLIO Instance records that the GOBI API created through the Orders app.
Expand |
---|
Current workaround: No workaround Steps to Reproduce:
Expected Results: Match on the PO Line and then the Vendor Order Reference number before updating the Instance record with full cataloging record and the holdings item with the correct holdings type and permanent location Actual Results: Records are all discarded |
2023-08-03
All
- information from Kimberly Pamplin
Status | ||||
---|---|---|---|---|
|
Additional values needed for Electronic access fields or 856 subfields
Issue: Right now, only a few subfields from the 856 are mapped. We would like to expand that ability to include the non public note (856$x), access status (856$7) and terms governing access (856$n).
All
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Fix is in Quesnelia
Formerly UIDATIMP-1506
|
- Ryan will also look into mapping indicators.
- Need to account for all Inventory record types
- Need to account for bulk edit, data import/export, ???
Status | ||||
---|---|---|---|---|
|
Additional values needed for Electronic access fields or 856 subfields
Issue: Right now, only a few subfields from the 856 are mapped. We would like to expand that ability to include the non public note (856$x), access status (856$7) and terms governing access (856$n).
Ensure consistency of UI for blank indicators between Bulk Edit, Data Export, quickMarc, and Data Import
Issue: Data Import displays blanks with a space. Quickmarc is a slash and so on. To avoid confusion, it'd be great to make sure that blank indicators and how marc fields and subfields are mapped in bulk edit, data import, and data export are done in a consistent way.
2024-02
All
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
- Ryan will also look into mapping indicators.
- Need to account for all Inventory record types
- Need to account for bulk edit, data import/export, ???
- Ryan is bringing this topic to Magda and Christine to discuss.
Status | ||||
---|---|---|---|---|
|
Reporting: Have the ability to download a list of instance, holdings, or item record identifiers that were successfully imported
Issue: There isn't a way to retrieve a list of identifiers through the Data Import log.
2024-07-11
All
- Ryan will look into making row 54-56 as one epic that has smaller stories.
Status | ||||
---|---|---|---|---|
|
Reporting: Have the ability to save a list of successfully imported records to a list in the Lists App
New Functionality. In addition to downloading a list, it would be great to be able to save the imported identifiers to a list in the Lists App
2024-07-11
All
Status | ||||
---|---|---|---|---|
|
Ensure consistency of UI for blank indicators between Bulk Edit, Data Export, quickMarc, and Data Import
Issue: Data Import displays blanks with a space. Quickmarc is a slash and so on. To avoid confusion, it'd be great to make sure that blank indicators and how marc fields and subfields are mapped in bulk edit, data import, and data export are done in a consistent way.
2024-02
All
- Ryan is bringing this topic to Magda and Christine to discuss.
Status | ||||
---|---|---|---|---|
|
Reporting: Have the ability to download a list of instance, holdings, or item record identifiers that were successfully imported
Issue: There isn't a way to retrieve a list of identifiers through the Data Import log.
2024-07-11
All
- Ryan will look into making row 54-56 as one epic that has smaller stories.
Status | ||||
---|---|---|---|---|
|
Reporting: Have the ability to save a list of successfully imported records to a list in the Lists App
New Functionality. In addition to downloading a list, it would be great to be able to save the imported identifiers to a list in the Lists App
2024-07-11
All
Status | ||||
---|---|---|---|---|
|
Reporting: Have the ability to download a list of errors from an import
Issue: The only way to see errors is to navigate in the log and to click on the title to see the jason. Having an export like in Bulk Edit would be helpful.
2024-07-11
All
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
|
Reporting: Have the ability to download a list of errors from an import
Issue: The only way to see errors is to navigate in the log and to click on the title to see the jason. Having an export like in Bulk Edit would be helpful.
2024-07-11
All
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Status | ||||
---|---|---|---|---|
|
Ability to view application log
When DI was in the planning phase with, there was a request to be able to view the application log. Examples were provided from other systems. This is still needed. This was shown as "server logs" in the original wireframes. See
Lref gdrive file | ||
---|---|---|
|
2024-08-15
Lab Session
ALL
Status | ||||
---|---|---|---|---|
|
Ability to update instance and marc srs in same job
Users need to be able to update the administrative data and also override protected fields to update the srs bib record. Tested in lab session 10-17-2024 this didn’t work in Poppy (Chicago test environment).
2024-10-17
Lab session
all
Status | ||||
---|---|---|---|---|
|
Add date and start/stop running date and times to the summary log page
Right now to see these times, you have to click out of the summary log view and back to the brief log view. Having this information displayed also on the summary page is helpful and needed.
2024-10-17
Lab session
ALL
Status | ||||
---|---|---|---|---|
|
Ability to view application log
When DI was in the planning phase with, there was a request to be able to view the application log. Examples were provided from other systems. This is still needed. This was shown as "server logs" in the original wireframes. See
Lref gdrive file | ||
---|---|---|
|
2024-08-15
Lab Session
ALL
Status | ||||
---|---|---|---|---|
|
Ability to update instance and marc srs in same job
Users need to be able to update the administrative data and also override protected fields to update the srs bib record. Tested in lab session 10-17-2024 this didn’t work in Poppy (Chicago test environment).
2024-10-17
Lab session
all
Status | ||||
---|---|---|---|---|
|
Add date and start/stop running date and times to the summary log page
Right now to see these times, you have to click out of the summary log view and back to the brief log view. Having this information displayed also on the summary page is helpful and needed.
2024-10-17
Lab session
ALL
Data Import log does not provide reason for No Action status.
The reason was previously provided as an error even though No action is means updates or creates were not taken because of the profile logic. (Multiple matches, single match with no create or update action provided, or no matches, etc.) The reason for No action should be provided in the log because it could be any of a number of scenarios.
2024-10-23
Christie Thomas University of Chicago
Update instance, holdings, and item in reverse order.
Right now the instance, holdings, and item must be updated in that order. It is also not possible to update an item independently and then, in the same job, match and update the instance and holdings. When updating all three records as a part of a shelf ready workflow, integrations (FOLIO app and external) require that the barcode be added to the item before the holdings record is updated. We need to be able to match and instance, holdings, and item (in that order) and then update the item, holdings, and instance or the item, instance, holdings in either of those orders.Data Import log does not provide reason for No Action status.
The reason was previously provided as an error even though No action is means updates or creates were not taken because of the profile logic. (Multiple matches, single match with no create or update action provided, or no matches, etc.) The reason for No action should be provided in the log because it could be any of a number of scenarios.
2024-10-23
Christie Thomas University of Chicago
Status | ||||
---|---|---|---|---|
|
Update instance, holdings, and item in reverse order.
Right now the instance, holdings, and item must be updated in that order. It is also not possible to update an item independently and then, in the same job, match and update the instance and holdings. When updating all three records as a part of a shelf ready workflow, integrations (FOLIO app and external) require that the barcode be added to the item before the holdings record is updated. We need to be able to match and instance, holdings, and item (in that order) and then update the item, holdings, and instance or the item, instance, holdings in either of those orders. Error message from import in Poppy: io.vertx.core.json.DecodeException: Failed to decode:Cannot deserialize value of type `java.util.LinkedHashMap<java.lang.Object,java.lang.Object>` from Array value (token `JsonToken.START_ARRAY`) at [Source: (String)"[{"id":"babefda2-17c3-4ff2-a677-f469c1b7bb59","_version":3,"hrid":"13642832","holdingsTypeId":"0c422f92-0f4d-4d32-8cbe-390ebc33a3e5","formerIds":[],"instanceId":"1980ec39-2d53-42d9-839b-d4d080850c76","permanentLocationId":"fad8517a-aae4-5b69-855e-01843e6e4d88","effectiveLocationId":"fad8517a-aae4-5b69-855e-01843e6e4d88","electronicAccess":[],"callNumberTypeId":"95467209-6d7b-468b-94df-0f5d7ad2747d","callNumber":"PL2260.52.B536A5 2019","notes":[],"holdingsStatements":[],"holdingsStatementsForInde"[truncated 371 chars]; line: 1, column: 1]""
2024-10-31
Christie Thomas University of Chicago
Status | ||||
---|---|---|---|---|
|
Update the SRS with override field protections enabled and update the instance status and cataloged date in a single job.
It is not possible to pair and SRS update with an instance update in the same job. We have the need to update the srs marc record and the instance record in a single job with a single match. (Match and instance or an srs marc record and specify an srs marc bibliographic update profile and an instance update profile. Or create an action that is linked to multiple field mapping profiles.
2024-10-31
University of Chicago
Delete holdings and items in batch via data import
When marking an instance for deletion we should be able to also delete all holdings and items attached to the instance or delete holdings and items targeted by identifier.
2024-11-21
Christie Thomas University of Chicago