Batch Importer (Bib/Acq) (UXPROD-47)

[UXPROD-2697] NFR: Create integration tests for key Data Import functionality using Karate (Kiwi) Created: 29/Sep/20  Updated: 29/Oct/21  Resolved: 29/Oct/21

Status: Closed
Project: UX Product
Components: None
Affects versions: None
Fix versions: Kiwi (R3 2021)
Parent: Batch Importer (Bib/Acq)

Type: New Feature Priority: P2
Reporter: Ann-Marie Breaux (Inactive) Assignee: Ann-Marie Breaux (Inactive)
Resolution: Done Votes: 0
Labels: NFR, data-import, epam-folijet, quality_control, tech-debt, testing
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Continues
is continued by UXPROD-3121 NFR: Create integration tests for key... Closed
Defines
defines UXPROD-47 Batch Importer (Bib/Acq) Analysis Complete
is defined by FAT-139 Cover mod-data-import with API tests Closed
is defined by FAT-914 Spike: Prepare for API Karate tests Closed
is defined by FAT-936 mod-data-import-converter-storage: Cr... Closed
is defined by FAT-937 mod-data-import: Implement 'Create In... Closed
is defined by FAT-939 mod-data-import: Implement 'Modify MA... Closed
is defined by FAT-947 mod-data-import: Set up default data ... Closed
is defined by FAT-956 Cover file-extensions.feature with AP... Closed
is defined by FAT-968 Create API test for EDI invoice impor... Closed
is defined by MODDATAIMP-299 Test Update action for Instances Closed
is defined by FAT-27 SPIKE: Investigate if API Karate test... Closed
is defined by FAT-178 mod-source-record-storage: Setup API ... Closed
is defined by FAT-179 mod-source-record-manager: Setup API ... Closed
is defined by FAT-180 mod-data-import: Setup API Karate tes... Closed
is defined by FAT-181 mod-data-import-converter-storage: Se... Closed
is defined by FAT-183 mod-data-import: Create test plan to ... Closed
Relates
relates to FAT-2 Karate Setup in the Modules - Tech Debt Open
relates to FAT-152 API Karate tests plan creation and es... Open
relates to FAT-892 API Karate tests implementation - Tec... Open
relates to FAT-915 Spike: Decide on an approach for crea... Closed
Epic Link: Batch Importer (Bib/Acq)
Analysis Estimate: Medium < 5 days
Analysis Estimator: Oleksii Kuzminov
Front End Estimate: Very Small (VS) < 1day
Front End Estimator: Ann-Marie Breaux (Inactive)
Front-End Confidence factor: Low
Back End Estimate: Jumbo: > 45 days
Back End Estimator: Oleksii Kuzminov
Development Team: Folijet
PO Rank: 114

 Description   

Feature to create integration tests for main workflows in data import. Allows for testing integration between modules on live environment; test all endpoints; move API tests from modules to these integration tests. Can't really do end to end from uploading file to checking results after import; not supported by Karate

Preliminary estimate 45 days for completing this feature based on team estimation.
Will be continued in Lotus by UXPROD-3121 Closed

Kateryna Senchenko to review the remaining stories and update if necessary; some will need to be rewritten as workflows; some will move to the Lotus feature

Preparation: Team will allocate time for the following

  • Setup Karate in modules (also link these issues to FAT-2 Open ) - DONE
  • Define test plans (also link these issues to FAT-152 Open )
  • Implement Karate tests (also link these issues to FAT-892 Open )

https://folio-org.atlassian.net/wiki/display/FOLIJET/API+Integration+Testing

Scope for Kiwi

  • Team forecast for Karate test coverage up to end of September.
    • Create all inventory types - end of Sept
    • Update for all inventory types - including matches, MARC modify and updates, field protections: end of Sept:
    • Stretch: Match on instance + instance update.
    • Added scenario for creating EDIFACT invoices
    • Added scenario for file extensions, plus beginning of covering creation of profiles for Data Import

Within the tests:

  • Will need to configure the environment
  • Will need appropriate MARC/EDIFACT files (1 record in the file)
  • Will need the appropriate profiles
  • Test covers the workflow from profile creation through finishing the UI log
  • Reset environment after tests
  • Also need some ramp-up time to learn Karate

Acceptance criteria:



 Comments   
Comment by Ann-Marie Breaux (Inactive) [ 12/Feb/21 ]

Hi Oleksii Kuzminov Do you think that this is work that we could plan during sprints 110-111, or should we put it off until R2?

Comment by Oleksii Kuzminov [ 15/Feb/21 ]

Hi Ann-Marie Breaux! I think we can move it to the R2

Comment by Ann-Marie Breaux (Inactive) [ 09/Jul/21 ]

Hi Taisiya Trunova Are UXPROD-2697 Closed and UXPROD-3121 Closed duplicates of each other? If yes, let's move description details from UXPROD-3121 Closed to here and delete UXPROD-3121 Closed . If no, then let's make the difference more clear in the titles of these 2 features. Thank you!

Comment by Taisiya Trunova [ 09/Jul/21 ]

Ann-Marie Breaux https://folio-org.atlassian.net/browse/UXPROD-3121 is a pseudo feature to link the tasks needed for. 

  • Setup Karate in modules
  • Define test plans
  • Implement Karate tests

Such features are added to estimate the effort on that within R3 so that to understand how much is left for development. As I see, https://folio-org.atlassian.net/browse/UXPROD-2697 was created long ago without specified the exact release. 

It is better to have a pseudo feature for such staff each release, estimate the requited effort for the release, and link the corresponding stories.

Thus I am not sure is the two features are duplicates or not. It looks lite they have the same context, however I'd recommend having a feature for Karate API Integration Testing (NFR) per each release. With this we will be able to track how much effort the team requires on per release and plan future releasing having historical data. 

Comment by Ann-Marie Breaux (Inactive) [ 09/Jul/21 ]

I understand - I'm going to make them have the same description, and then mark one as Kiwi and one as Lotus. Thank you!

Comment by Ann-Marie Breaux (Inactive) [ 26/Jul/21 ]

From testing meeting - very large feature, work is all BE. Per Kateryna Senchenko needs additional BE grooming; will tell Ann-Marie Breaux if additional info is needed; will also need to determine if this feature needs to be split.

Comment by Taisiya Trunova [ 28/Jul/21 ]

Team forecast for Karate test coverage up to end of September. 

  • Create all inventory types - end of Sept
  • Update for all inventory types - including matches, MARC modify and updates, field protections:
  1. end of Sept:  match on instance + instance update. 
  2. the rest will be included into R3 Scope
  3. field protections - out of R3 scope. 

The above listed staff is recommended to exclude from R3 scope:

  • Update only holdings and/or item based on MARC bib import
  • Create invoice
  • Create file extensions (lower-priority)

Ann-Marie Breaux Kateryna Senchenko let's discuss splitting the current feature and include the out of scope staff into R1 2022. 

Re-estimate this feature at grooming. 

 

Comment by Ann-Marie Breaux (Inactive) [ 05/Aug/21 ]

Hi Taisiya Trunova I've tried to clean them up so that it's more clear what is covered on UXPROD-2697 Closed versus UXPROD-3121 Closed . I did not move any of the stories to UXPROD-3121 Closed . I think they will need to be rewritten to describe a workflow, rather than a particular module. Since several modules are invoked for each import, I'm not sure it makes sense to write the tests for each module separately. Let's discuss more if needed.

Comment by Ann-Marie Breaux (Inactive) [ 25/Aug/21 ]

Grooming: will probably add some more scenarios, and probably will need to split this into some Lotus work

Comment by Ann-Marie Breaux (Inactive) [ 10/Sep/21 ]

Hi Kateryna Senchenko I moved FAT-968 Closed (EDIFACT invoice import tests) back to this Kiwi feature. Please update the description when you have a chance, and let me know if you need a sample file. If we need to push out one or two MARC issues from this feature to Lotus, then please reassign them to UXPROD-3121 Closed . Thank you!

Generated at Fri Feb 09 00:26:06 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.