[FOLIO-1519] Automatic loading of sample and reference data Created: 19/Sep/18 Updated: 18/Aug/20 Resolved: 05/Apr/19 |
|
| Status: | Closed |
| Project: | FOLIO |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | Umbrella | Priority: | P2 |
| Reporter: | Jakub Skoczen | Assignee: | Adam Dickmeiss |
| Resolution: | Done | Votes: | 0 |
| Labels: | ci, core, core-spike, platform-backlog, q1-2019, sprint47, sprint48, sprint49, sprint50, sprint51, sprint52, sprint53, sprint54 | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Issue links: |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Sprint: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Development Team: | Core: Platform | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description |
|
We have discussed the following extensions to the tenant and Okapi install calls:
In the first phase we will address the implementation of this new functionality in mod-inventory-storage. Open questions:
Reference data The name of the parameter that controls reference data loading is loadReference with true|false (both in the tenantParams string query param and in the parameters kv array in the moduleInfo payload) By default the value of loadReference is false Sample data The name of the parameter that controls sample data loading is loadSample with true|false (both in the tenantParams string query param and in the parameters kv array in the moduleInfo payload) By default the value of loadSample is false |
| Comments |
| Comment by Jakub Skoczen [ 25/Sep/18 ] |
|
Guys, Adam has begun work on
|
| Comment by Jakub Skoczen [ 11/Oct/18 ] |
|
Adam Dickmeiss Wayne Schneider Niels Erik Nielsen Guys, do we need to get together on a call to try to push this forward? We need to come up with an approach suitable (especially) for loading inventory data. The way I see this we can expect the sample/reference data to be already processed to fit the modules schema (e.g I don't think we need to be able to load raw marc records for Inventory). Comments and thoughts? |
| Comment by Jakub Skoczen [ 30/Oct/18 ] |
|
Adam Dickmeiss can you see if you can flesh out a propsal for this? Besides
|
| Comment by Adam Dickmeiss [ 06/Nov/18 ] |
|
After "local" discussions this morning, I think we are about to drop the scripting idea as outlined in
1. modules deployed outside Okapi are problematic. We only specify a remote URL and that in no way can run a script. We could of course, haave something remote URI "script".. eg ag url to run a script remotely .. and then alternatively an Okapi local deployment way.. But that gives two ways and should module implements . 2. if a module already offers HTTP, why not use that anyway for loading data? Using the existing tenant web service is better.. Why? 1. It works with local and remote deployed service. 2. A simple standard way of preparing a tenant to use a module. It also informs about versions (upgrade, downgrade etc..) as well as providing all necessary X-Okapi stuff to do the work.. Such as token etc.. 3. Less "new" development .. which is always desirable. If we decide on usingt the Tenant init service... we should consider if we need to extend it. SHould we allow it to be controlled if/when reference data should be included? Note that it is also possible to create a NEW module that depends on the BARE module with the only purpose of loading reference data (eg use the module ID to determine what to do) |
| Comment by Marc Johnson [ 06/Nov/18 ] |
|
Adam Dickmeiss I like the idea of using the Tenant API for this purpose (given it has been decided that we are bundling the records with the module itself) and I think this is a good place to start. I think it is valuable to be able to decide whether reference, and especially sample, records are loaded during this initialisation. The pre-loading of sample records in mod-configuration caused signifiant confusion for me when trying to determine why the tests produced certain results. I imagine that many organisations (but maybe not all) may want the reference records in all of their environments. Do organisations want sample records for production? I think that many would not, or would need to delete them prior to loading their actual records. I'm aware that initially, all modules / tenants may need to use the same policy, as the Tenant API request is made during module enabling/activation (Adam, I believe you raised this in the meeting last week). I assume that these records will be loaded on the first initialisation of a module (and hence need to always be in line with that versions schema). What are we envisaging will happen on an upgrade (or is this part of the more general upgrade topic, as I don't believe we talked about how to handle record migration)? (It might be a orthogonal topic, I think we may also need to consider the impact of the Tenant API doing more activities (and hence get slower) on the general activation process, and especially upgrades) |
| Comment by Wayne Schneider [ 04/Jan/19 ] |
|
With the PR on mod-inventory-storage for
|
| Comment by Wayne Schneider [ 17/Jan/19 ] |
|
For posterity, when working with the
This is all documented in the Okapi guide. I also added documentation to mod-inventory-storage (since the support for the loadReference parameter is implemented there, not in RMB or elsewhere). |
| Comment by Adam Dickmeiss [ 18/Jan/19 ] |
|
Thx. I think eventually we'll make some of the "private" methods in mod-inventory-storage part of an RMB util. But let's see what other modules have reference data (I see
|
| Comment by Jakub Skoczen [ 22/Jan/19 ] |
|
Adam Dickmeiss We had a call today about loading "sample" data through this mechanism in the mod-source-record-storage and mod-order-storage. Wayne Schneider and I assume that the mechanism is the same, we just need new property called "loadSample" set in the CI and the rest belongs to the module itself. Correct? |
| Comment by Wayne Schneider [ 22/Jan/19 ] |
|
Adam Dickmeiss would it make sense to extend the TenantModuleDescriptor schema to allow passing tenant parameters on that level? Right now, you can pass the parameters in query parameters in a POST to the /_/proxy/tenant/<id>/install endpoint. This would pass the parameters to all modules – what if a user only wanted to load reference/sample data for only some modules? |
| Comment by Craig McNally [ 22/Jan/19 ] |
|
Wayne Schneider I guess I wasn't paying close enough attention. TBH that's the way I thought this was supposed to work, but now that I take another look I see that's not the case. I personally prefer this over the query argument approach... e.g. [ {
"id" : "mod-users-14.2.1-SNAPSHOT.299",
"action" : "enable"
}, {
"id" : "permissions-module-4.0.4",
"action" : "enable"
}, {
"id" : "mod-login-3.1.1-SNAPSHOT.42",
"action" : "enable"
}, {
"id" : "mod-orders-3.0.0-SNAPSHOT.37",
"action" : "enable",
"loadSample": true,
}, {
"id" : "mod-inventory-15.0.0",
"action" : "enable",
"loadReference": true
} ]
|
| Comment by Jakub Skoczen [ 05/Apr/19 ] |
|
Closing as all work has been completed. |