2024-09-03 Better Sample Data Meeting notes

1. Split the project in three:

   1. Golden copy for Bugfest environment.

   2. Work on a small data set for FOLIO Snapshot.

   3. write robust data anonymization scripts

Follow up on activities since last meeting:

Plan - who will take on what task:

a. Golden Copy / Yogesh Kumar

uChicago can provide data for inventory, but not sensitive data as circulation, users, acquisition etc.

Michigan State: some hesitation at management level. New data for Sunflower. Timing January/February 2025.

A conversation also came up in the ARLEF group where a couple of large institutions said they could maybe provide data.

The QA team is working on fixing the current Bugfest Data set (September):

Link to wiki - https://folio-org.atlassian.net/wiki/spaces/DQA/pages/203685917

Has gotten some feed back from POs, but not enough.

Jira tickets has been written up for KitFox team - Sprint 199. Jira ticket: BF-752 (Epic).

Will then be what we will use as copy for Bugfest Ramsons. Expect to have this done by end of September.

b. Snapshot environment / Charlotte Whitt

Charlotte has reached out to ID Devops re. adding data to monthly build environment: https://folio-quesnelia.dev.folio.org/ (the link is not up and running yet)

If this is to take too long, Charlotte will look into other options. The important thing is:

• The environment will be in Quesneliar

• Data will not be overwritten.

• Charlotte will provide access to Autumn, Kristin and SMEs helping out.

Charlotte to fix the 36 instances with Instance source = FOLIO - aim for next week.

Recruit members from MM-SIG - extra eyes, review the records updated in FOLIO Instance = FOLIO. Charlotte will aim for doing the updates for next week.

Write up Data Import Job profiles

Data Import Job Profile which can import ~ 100 bibs in MARC 21 and create instance, holdings, item (corresponding to the locations we have set up). MM-SIG eyes on the 100 bibs. That these records has the right mix of misc. types, to cover the basic; incl. bound-with.

https://folio-org.atlassian.net/wiki/spaces/DQA/pages/203685917/Testing+Data+Feedback

Autumn has a music cataloger back ground. Will find records in OCLC. Will aim for next week.

Also include Serial records, with multiple holdings each with multiple items. Add holdings statements.

Will ask if MM-SIG member has experience with MFHD.

Load MARC Authority data - this task we will come back to after WOLFcon.

Then later we can move on to Order/Order lines data. We will ask the Acquisition SIG (Kritin Martin will take the lead on this), the ERM-SIG to review the entered data.

c. Status on writing robust data anonymization scripts / Lee Braginsky. Did get a university to provide data, which will be anonymized/scrambled

• Replace PII with randomly generated data

• Scramble loan history

• Scramble orders, invoice amounts, fund codes

• Replace vendor names with randomized names

• Strip out staff notes with initials, etc.

• One set of data for the general environment, and perhaps a second sample set for the ECS environment

  • Get this from a consortium!

2. Review timeline document

• Draft - timeline document -

We will split it up the the three tracks as listed above.

Yogesh will update the Golden copy (Bugfest) section.

Charlotte will update the FOLIO Snapshot section

Lee will update the time line doc for his work on anonymization scripts.