[FOLIO-715] User import script update Created: 13/Jul/17  Updated: 12/Nov/18  Resolved: 24/Jul/17

Status: Closed
Project: FOLIO
Components: None
Affects versions: None
Fix versions: None

Type: Task Priority: P2
Reporter: Katalin Lovagné Szűcs Assignee: Katalin Lovagné Szűcs
Resolution: Done Votes: 0
Labels: demo19, sprint18
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Sprint:

 Description   

On yesterday's UM SIG meeting we talked about the bulk user import script.
Some key points came up :

  • The externalSystemId should be used for checking existing users and as match point for updating users
  • Addresses:
    • Test what happens with addresses on user update if created an additional address in FOLIO - addresses created within FOLIO shouldn't be deleted unless something is explicitly sent via the update (a blank value? delete flag?)
    • Test how an address created in FOLIO can be deleted and document
  • Other fields
    • Similar to addresses, it would be nice if data added within FOLIO was ignored/left in tact unless something is explicitly sent to remove/update it. Please test whether that already works today and document.
  • There was a misunderstanding about username, maybe we should use userid in the script input and map it to FOLIO's username field CB: Actually, I think the desire was that FOLIOs internal field names match the names specified by the SIG in the User Metadata Document. Or, if the internal names can't be made to match, there should be some way to obscure any inconsistencies from those interacting with the system (script developers, people reading error messages). Let's get Jakub's take on this.
  • The code should be optimized for a large amount of records.
  • An equivalent version could be written in another language, e.g. Python or Perl. This is not a hard requirement, though, as most institutions are probably going to want to write their own scripts. The purpose of our writing a script is just as a proof-of-concept and the current script is probably fine for that purpose.
  • The script shouldn't have to supply a FOLIO ID/system identifier for the user and it seems like it does. Let's discuss with Jakub and file a bug, if needed.


 Comments   
Comment by Cate Boerema (Inactive) [ 13/Jul/17 ]

Jakub Skoczen, can you please take a look at this issue and weigh in with your thoughts?

Comment by Katalin Lovagné Szűcs [ 21/Jul/17 ]

The script has been updated with the following:

  • The externalSystemId is the matching point when creating/updating users.
  • The code is optimized for a larger amount of records (tested with a batch of 1000 records).
  • When creating a new user, a UUID is generated as FOLIO id for that specific user.
  • Error management which handles errors differently according to the error type. E.g. if a user can not be added because a required field is missing, it will be skipped but the import will handle the other users. But if the reason of the failure was that the user is not authenticated properly, there was an internal server error or the system is currently offline then the script will most certainly fail to import more users thus it is exiting in this case.

The field overwrites has been tested with the following result:

  • Addresses coming from the import always overwrite the ones in FOLIO.
  • The same is true for any other field.
Comment by Cate Boerema (Inactive) [ 21/Jul/17 ]

Great work Katalin Lovagné Szűcs! Have you heard from Chris Manly about the user set from Cornell? It would be great to get testing with a real data set.

The field overwrites has been tested with the following result:
Addresses coming from the import always overwrite the ones in FOLIO.
The same is true for any other field.

Just to confirm, if a home address has been added manually to User A in FOLIO and the import updates User A with only a campus address, the home address is cleared out? In other words, there is no way for the import to leave FOLIO data in tact upon update.

That is what I was expecting based on our last discussion with Jakub, but wanted to double check.

Comment by Katalin Lovagné Szűcs [ 21/Jul/17 ]

No, I have not heard from Chris about the data set yet.
Yes, currently the import works that way.

Generated at Thu Feb 08 23:07:49 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.