[FOLIO-1532] Add mod-data-import and mod-source-record-storage to CI testing builds Created: 26/Sep/18  Updated: 12/Nov/18  Resolved: 14/Oct/18

Status: Closed
Project: FOLIO
Components: None
Affects versions: None
Fix versions: None

Type: Task Priority: P2
Reporter: Dmytro Tur Assignee: Wayne Schneider
Resolution: Done Votes: 0
Labels: ci, integration, sprint48
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Blocks
blocks MODDATAIMP-14 Create mod-data-import module Closed
blocks MODSOURCE-1 Create mod-source-record-storage module Closed
blocks MODSOURMAN-1 Create Source records Manager module Closed
is blocked by FOLIO-968 Container RAM usage is getting out of... Closed
is blocked by FOLIO-1548 SPIKE: a lighter-weight folio/testing... Closed
is blocked by FOLIO-1571 Enable CI for mod-source-record-manager Closed
Relates
relates to FOLIO-1576 folio-ansible: allow for URL deployment Closed
Sprint:
Development Team: Prokopovych

 Description   

mod-data-import, mod-source-record-storage and mod-source-record-manager are ready to be included in the "testing" builds.



 Comments   
Comment by Dmytro Tur [ 26/Sep/18 ]

Wayne Schneider, this is task to add mod-data-import and mod-data-import-storage to CI testing builds

Comment by Kateryna Senchenko [ 01/Oct/18 ]

mod-data-import-storage has been renamed to mod-source-record-storage

Comment by Kateryna Senchenko [ 02/Oct/18 ]

mod-source-record-manager should also be added to CI testing builds

Comment by Wayne Schneider [ 10/Oct/18 ]

Hi, all. A couple of questions/issues:

  1. I notice that none of these modules have a requires section in their module descriptors. Is this really correct – there is no dependency on the backend storage interface from either mod-data-import or mod-source-record-manager?
  2. The interface provided by mod-source-record-storage is set to ${artifactId}. Shouldn't it be something like source-record-storage, to allow for other implementations?
  3. The only interfaces provided by mod-data-import and mod-source-record-manager are "ping" interfaces. Are they really ready to be included? Similarly, both modules register their name as "Ping", surely that is not intended? The name is only for human consumption, typically it is more descriptive, e.g. "Data Import Module".
  4. I assume mod-source-record-storage requires a PostgreSQL backend. Do either of the other modules require this?
  5. Can all these modules run with -Xmx256m?
Comment by Kateryna Senchenko [ 11/Oct/18 ]

Hello Wayne Schneider, thanks for the review, hope this would answer your questions:

1. Point noted. We added the dependencies between the modules (mod-data-import --> mod-source-record-manager --> mod-source-record-storage).
2. You're right, the interface has been renamed.
3. Thanks for catching that, we have changed the names of modules and interfaces in the ModuleDescriptor files. The actual endpoints for mod-data-import and mod-source-record-manager will be added in the subsequent stories. We'd like to have the modules included in the builds at this stage though, in order to prevent any blockage in the future.
4. That's right, only mod-source-record-storage will interact with the db.
5. 256m should be enough for the time being.

Please let us know if you have any questions/concerns. We do appreciate your help.

Comment by Wayne Schneider [ 11/Oct/18 ]

Thank you, this all looks good.

Unfortunately, adding 3 more modules pretty much maxes out RAM on an 8 GB VM, so I can (and will) integrate these modules into the folio-testing.aws.indexdata.com AWS instance, but not into the folio/testing-backend Vagrant box. I will block this issue on FOLIO-1548 Closed .

Comment by Wayne Schneider [ 11/Oct/18 ]

By the way, it appears that your commits to master failed to build in Jenkins for mod-data-import and mod-source-record-manager, because the interface dependency chain in the module descriptors could not be satisfied until mod-source-record-storage was built and the module descriptor posted to the FOLIO module descriptor registry. I rebuilt the projects in the correct order, and the CI artifacts are now available.

Comment by Kateryna Senchenko [ 12/Oct/18 ]

Thank you very much Wayne. Just a small correction to the previous questions - you asked if only mod-source-record-storage requires a PostgreSQL backend, also mod-source-record-manager will have to interact with the db.

Comment by Wayne Schneider [ 14/Oct/18 ]

The data import modules have been added to the build process for folio-testing, and should appear in the next overnight build.

Comment by Wayne Schneider [ 14/Oct/18 ]

All three modules are now deployed on folio-testing and enabled for the default tenant.

Generated at Thu Feb 08 23:14:02 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.