Page Comparison

Welcome and request for someone to take notes (Ingolf volunteers).

Jon Miller will give us a demonstration of a tool that he wrote to load json files into Folio, both through the APIs, and directly to the Folio data stores.

Meeting Notes

Confusion because Zoom link is not working (Getting the "host has another meeting in progress" message). We continue with WebEx, using Patty's account.

Jon Miller

demonstrates the migration tool he wrote. It's on github, https://github.com/jemiller0/Folio . It requires .NET core. This can be installed from here: https://dotnet.microsoft.com/download (the link is also in Jon's README on the github site). One downloads .NET Core SDK, then compiles the source code.

The project is being built like this

git clone https://github.com/jemiller0/Folio.git
cd Folio
dotnet build

Jon's application has two configuration files. They are being built automatically with default values if they are not present. The config files are called

ConnectionStrings.config
AppSettings.config

In ConnectionStrings.config replace host=localhost by your host. Other config variables are username, password and database.

Jon's application works in two modes: SQL mode and API mode. For SQL mode one needs a local instance to run. The config file AppSettings.config is only needed when one works in API mode. To load and save data using the FOLIO web API, specify the URL, tenant, username, and password in AppSettings.config

Sample calls of the application:

dotnet ../FolioConsoleApplication.dll -save -all

This saves (downloads) all data which are in the system to local JSON files. Jon demonstrates that 100,000 user records take 19 seconds to save. The app is fast because it uses postgres copy binary format.

dotnet ../FolioConsoleApplication.dll -delete -

load -all

This command first deletes all data sets which are already in the system and the uploads all data (in local JSON files) again.

When one deletes users one has to take care not to delete the admin user, otherwise one will block oneself.

dotnet ../FolioConsoleApplication.dll -save -all -api

This command does the same as "-save -all" execept that is uses the API instead of SQL statements. Jon finds that this takes 90 seconds for 100,000 user records.

Other options to the application are:

-All
-Api
-Delete
-Load
-Save
-Validate
-Verbose

which are self-explanatory. "validate" does nothing but validates the data for correct syntax.

In API mode, the query language is CQL, whereas in SQL mode it is SQL (guess). There are slight differences between CQL and SQL syntax, e.g. NOT EQUAL is '!=' in SQL but '<>' in CQL.

Jon explains some warnings and limitations. For example, the app uses the latest SQL and JSON schemas which may not be the ones of the FOLIO version one is using.

The recording of Jon Miller's presentation is here: https://ebscomeetings.webex.com/ebscomeetings/lsr.php?RCID=f1dacfa3bc9445ef9c41132af4862ec1