Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Test status: COMPLETED WITH ERRORS

Table of Contents
Overview IN PROGRESS

  • Regression testing of Fiscal Year Rollover (FYR) on Eureka based environment in quesnelia release. 
    • The testing triggered from UI for the ledger with generated amount of defined data set (Data quantity**: 50000 orders with Open status and 50000 orders with Pending status).
  • Testing include two different phases of the fiscal year transition process:
    • #1 phase: Test Fiscal Year Rollover - is a risk free simulation pre-step to #2 phase which includes only Finance part of the data (to preview how the rollover settings will affect funds, budgets, and encumbrances).
    • #2 phase: Fiscal Year Rollover - perform a permanent changes. It is the actual process which move financial data, budgets, and encumbrances into the new fiscal year. It includes not only Finance but Orders parts of the data too.
  • The purpose of testing is to define duration time of those 2 phases, to do comparison between current and previous results of each phase, to find any trends for resource utilization. To recommend improvements how to make it work better if any confirmed.
    • Previous successful testing for #2 phase took 29 hours
  • Jiras/ links:

Summary

  • Running FYR on eureka completed with errors. Tests failed multiple times on overall order rollover with 504 Gateway Time-out issue.
  • The problem confirmed on okapi based environment. In scope of investigation of the problem the Time-out was solved changing parameter max_locks_per_transaction in DB configuration from 64 to 1024.
  • Eureka based environment (after changes were applied): 
    • #1 Phase Test FYR took 3 hours 35 minutes, completed successfully.
    • #2 Phase actual FYR took 8 hours and completed with errors* (Order rollover Error).
  • Okapi based environment (after changes were applied):
    • #1 Phase Test FYR took 3 hours 20 minutes. Completed successfully.
    • #2 Phase actual FYR took 9 hours 7 minutes. Completed successfully.
  • Service CPU utilization for pre-strop to Fiscal Year Rollover (FYR) - Test FYR was less than 5% for all the modules, except a spike at the beginning of test rollover (mod-invoice reached 10%, mod-invoice-storage - 8%). Some modules were steady - mod-inventory - 11%, mod-pubsub - 8%.
  • DB CPU usage was on level of 9% during the test rollover and there was growing trend from 9% to 15% during the rollover itself.
  • No memory leaks were found.

Recommendations

  • Set max_locks_per_transaction parameter in DB cluster and instance configurations from default 64 to 1024 to avoid problems with ordersRollover.
  • Investigation of issues on Eureka based environment
    Jira Legacy
    serverSystem Jira
    serverId01505d01-b853-3c2e-90f1-ee9b165564fc
    keyMODFISTO-495
    - Time out issue 
    Jira Legacy
    serverSystem Jira
    serverId01505d01-b853-3c2e-90f1-ee9b165564fc
    keyEUREKA-518
    - UI pending issue

Errors

  • 127.0.0.1:43264 POST /finance/ledger-rollovers null HTTP_1_1 500 265 400027 tid=[tenant] Internal Server Error. This error happens when we try FYR with more than 10k open orders. The last attempt to run FYR with 50k open orders after changed recommended parameter max_locks_per_transaction failed. A new observation here is that budgets became unavailable with this amount of data. Probably it may cause fail of FYR. 
  • Error message: io.vertx.core.impl.NoStackTraceThrowable: Response status code 401 is not between 200 and 299,qelc2/mod-finance-storage/9044d47f773b48a99cc6c8420864b724 which indicate possible RTR issue.
  • WARN [org.fol.sid.ser.rou.RequestMatchingService] (vert.x-eventloop-thread-4) Egress routing entry was not found for the request's path. Forwarding request to the Gateway:moduleId = sc-mod-finance-storage, path = /orders/rollover, destination = https://kong-qelc2.int.aws.folio.org, x-okapi-module-id = null",qelc2/sidecar-mod-finance-storage/9044d47f773b48a99cc6c8420864b724,054267740449:qelc2-folio-eis
  • ERROR edgerRolloverService handleOrderRolloverError:: Orders rollover failed for ledger f38a29f5-feaa-4363-98a1-1eba79557c9c

Test Runs and Results

Data size: 100K orders (50K orders in open status, 50K - pending), 200K order lines, 400K transactions. 

...

Test #

...

Scenario

...

Fiscal year rollover duration Orchid*

...

Fiscal year rollover duration

Poppy*

...

FYR duration

Quesnelia (Eureka)

...

FYR duration

Quesnelia (non-Eureka)

...

-

...

3.5 hours

...

6 days (failed with 504 error)

...

29 hours

8 hours, failed due to 500(server error)

...

On eureka Rollover competed with error.

On non-eureka Rollover completed successfully.

*Results are taken from the previous test report. Details can be found at the link: [Orchid] Fiscal year rollover testing

*Results are taken from the previous test report. Details can be found at the link: [Poppy] Fiscal year rollover testing

**Data structure can be found at the link: Fiscal year rollover testing

Instance CPU Utilization

Test FY rollover

There's no affect on 

FY rollover

no graph

Service CPU Utilization

Test FY rollover

Image Removed

Maximum CPU utilization by modules:

mod-finance storage - 16% (spike at the beginning)

mod-inventory - 10%

mod-orders, mod-orders-storage - 1,5%, 3%

mod-finance - under 1%

FY rollover

Image Removed

Maximum CPU utilization by modules:

mod-inventory - 11%

mod-orders - 1.5%

mod-orders-storage - 3%

mod-finance - 0.15%

mod-finance storage - 1.2%

Memory Utilization

Test FY rollover

Image Removed

Memory usage by modules:

mod-finance-storage - 85%

mod-finance - 41%

mod-orders - 57%

mod-orders-storage - 46%

FY rollover

Image Removed

Memory usage by modules:

mod-finance-storage - 85%

mod-finance - 42%

mod-orders - 58%

mod-orders-storage - 46%

DB CPU Utilization

Test FY rollover (qelc2)

Image Removed

Maximum DB CPU usage - 9 - 20%

Test FY rollover (qcon)

Image Removed

FY rollover (qelc2)

Image Removed

Maximum DB CPU usage - 99%

FY rollover (qcon)

Spikes are connected to fqm manager queries (as example - REFRESH MATERIALIZED VIEW CONCURRENTLY cs00000int_0001_mod_fqm_manager.drv_langu...). Do not affect duration of FYR.

Image Removed

DB Connections

Test FY rollover (qelc2)

Image Removed

Test FY rollover (qcon)

Image Removed

FY rollover (qelc2)

Image Removed

FY rollover (qcon)

mod-serials was not paused for the test (because of that we see such connection number increase)

Image Removed

DB Load

Test FY rollover (qcon)

Image Removed

FY rollover (qcon)

Image Removed

Top SQL

Test FY rollover (qcon)

Image Removed

FY rollover (qcon)

Image Removed

Appendix

Infrastructure

PTF -environment qelc2

...

1 database  instance, writer

...

db.r6g.4xlarge

...

  • 4 m5.2xlarge brokers in 2 zones
  • Apache Kafka version 3.7.x

  • EBS storage volume per broker 300 GiB

  • auto.create.topics.enable=true
  • log.retention.minutes=480
  • default.replication.factor=3

PTF -environment qcon

...

1 database  instance, writer

...

db.r6g.xlarge

...

titleModule QELC2

Cluster Resources - qelc2-pvt (Wed Nov 06 11:34:51 UTC 2024)

...

Test status: COMPLETED WITH ERRORS

Table of Contents
Overview

  • Regression testing of Fiscal Year Rollover (FYR) on Eureka based environment in quesnelia release. 
    • The testing triggered from UI for the ledger with generated amount of defined data set (Data quantity**: 50000 orders with Open status and 50000 orders with Pending status).
  • Testing include two different phases of the fiscal year transition process:
    • #1 phase: Test Fiscal Year Rollover - is a risk free simulation pre-step to #2 phase which includes only Finance part of the data (to preview how the rollover settings will affect funds, budgets, and encumbrances).
    • #2 phase: Fiscal Year Rollover - perform a permanent changes. It is the actual process which move financial data, budgets, and encumbrances into the new fiscal year. It includes not only Finance but Orders parts of the data too.
  • The purpose of testing is to define duration time of those 2 phases, to do comparison between current and previous results of each phase, to find any trends for resource utilization. To recommend improvements how to make it work better if any confirmed.
    • Previous successful testing for #2 phase took 29 hours
  • Jiras/ links:

Summary

  • Running FYR on eureka completed with errors. Tests failed multiple times on overall order rollover with 504 Gateway Time-out issue.
  • The problem confirmed on okapi based environment. In scope of investigation of the problem the Time-out was solved changing parameter max_locks_per_transaction in DB configuration from 64 to 1024.
  • Eureka based environment (after changes were applied): 
    • #1 Phase Test FYR took 3 hours 35 minutes, completed successfully.
    • #2 Phase actual FYR took 8 hours and completed with errors* (Order rollover Error).
  • Okapi based environment (after changes were applied, testing is carried out in scope of investigation to check functionality of FYR):
    • #1 Phase Test FYR took 3 hours 20 minutes. Completed successfully.
    • #2 Phase actual FYR took 9 hours 7 minutes. Completed successfully.
  • Service CPU utilization for both phases was under 5% on eureka environment except a spike at the beginning of test rollover (mod-finance-storage - 16%, mod-invoice - 10%). Module mod-inventory - 11% during whole test. 
  • DB CPU usage was on level of 9% during #1 Phase and there was growing trend from 9% to 15% during the rollover itself on Eureka environment with 4xlarge database. On xlarge database - 39% without growth.
  • No memory leaks were found.

Recommendations & Jiras

  • Set max_locks_per_transaction parameter in DB cluster and instance configurations from default 64 to 1024 to avoid Time-out issue during actual rollover (#2 Phase).
  • Investigation of issues on Eureka based environment
    Jira Legacy
    serverSystem Jira
    serverId01505d01-b853-3c2e-90f1-ee9b165564fc
    keyMODFISTO-495
    - Time out issue.

Errors

  • 127.0.0.1:43264 POST /finance/ledger-rollovers null HTTP_1_1 500 265 400027 tid=[tenant] Internal Server Error. This error happens when we try FYR with more than 10k open orders. The last attempt to run FYR with 50k open orders after changed recommended parameter max_locks_per_transaction failed. A new observation here is that budgets became unavailable with open orders >= 42000 after max_locks_per_transaction parameter was changed from 64 to 1024. Probably it may cause fail of FYR. 
  • Error message: io.vertx.core.impl.NoStackTraceThrowable: Response status code 401 is not between 200 and 299,qelc2/mod-finance-storage/9044d47f773b48a99cc6c8420864b724 which indicate possible RTR issue.
  • WARN [org.fol.sid.ser.rou.RequestMatchingService] (vert.x-eventloop-thread-4) Egress routing entry was not found for the request's path. Forwarding request to the Gateway:moduleId = sc-mod-finance-storage, path = /orders/rollover, destination = https://kong-qelc2.int.aws.folio.org, x-okapi-module-id = null",qelc2/sidecar-mod-finance-storage/9044d47f773b48a99cc6c8420864b724,054267740449:qelc2-folio-eis
  • ERROR edgerRolloverService handleOrderRolloverError:: Orders rollover failed for ledger f38a29f5-feaa-4363-98a1-1eba79557c9c

Test Runs and Results

Data size: 100K orders (50K orders in open status, 50K - pending), 200K order lines, 400K transactions. 

Test #

Scenario

Load level

Fiscal year rollover duration Orchid*

Fiscal year rollover duration

Poppy*

FYR duration

Quesnelia (Eureka)

FYR duration

Quesnelia (non-Eureka)

Comments
1Test Fiscal year rollover1 rollover at a time

-

3.5 hours

3.5 hours3 hours 20 minutesTest rollover finished successfully.
2Fiscal year rollover1 rollover at a time

6 days (failed with 504 error)

29 hours

8 hours, failed due to 500(server error)


9 hours 7 minutes

On eureka Rollover competed with error.

On non-eureka Rollover completed successfully.

*Results are taken from the previous test report. Details can be found at the link: [Orchid] Fiscal year rollover testing

*Results are taken from the previous test report. Details can be found at the link: [Poppy] Fiscal year rollover testing

**Data structure can be found at the link: Fiscal year rollover testing

Service CPU Utilization

Note: Instance level CPU screenshots is not included in this report. Taking into account that Fiscal Year Rollover process is mostly happening on DB side - it's barely visible on service CPU, so it's invisible on instance CPU. 

Test FYR (Eureka)

Low CPU usage during FYR reflects that it can handle large fiscal year transitions efficiently. As the process runs mostly on database level. 

Image Added

Image Added

Maximum CPU utilization by modules:

mod-finance storage - 16% (spike at the beginning)

mod-inventory - 10%

mod-orders, mod-orders-storage - 1,5%, 3%

mod-finance - under 1%

FYR (Eureka)

Image Added

Image Added

Maximum CPU utilization by modules on eureka:

mod-inventory - 11%

mod-orders - 1.5%

mod-orders-storage - 3%

mod-finance - 0.15%

mod-finance storage - 1.2%

Memory Utilization

Test FYR (Eureka)

This graph shows memory usage during #1 phase of FYR. No signs of memory leaks trends.

Image Added

Image Added

Memory usage by modules:

mod-finance-storage - 85%

mod-finance - 41%

mod-orders - 57%

mod-orders-storage - 46%

FYR (Eureka)

This graph shows memory usage during #2 phase of FYR. No signs of memory leaks trends too. Consumption is on the same level as #1 phase.

Image Added

Image Added

Memory usage by modules:

mod-finance-storage - 85%

mod-finance - 42%

mod-orders - 58%

mod-orders-storage - 46%

DB CPU Utilization

Test FYR (Eureka)

DB CPU usage range: 9% - 20%. Utilization was stable during #1 phase except of spikes to 20%. It competed successfully so it didn't affect the test.

Image Added

Test FYR (non-Eureka)

The #1 phase of FYR was carried out on xlarge DB instance. That's the reason why we see 34% of DB CPU utilization. The utilization was stable - 34%. This phase completed successfully.

Image Added

FYR (Eureka)

Image Added

Maximum DB CPU usage - 99%

FYR (non-Eureka)

Spikes are connected to fqm manager queries (as example - REFRESH MATERIALIZED VIEW CONCURRENTLY [tenant]_0001_mod_fqm_manager.drv_langu...). Do not affect duration of FYR.

The #2 phase completed successfully in non-eureka. It utilized 35% which is almost the same as in #1 phase.

Image Added

DB Connections

DB connections stay on the same level during all phases of FYR. Connections in Eureka - 690 and non-Eureka 1380.. The difference can be explained by enabled mod-serials-management module on qcon environment. 

Test FYR (Eureka)

Image Added

Test FYR (non-Eureka)

Image Added

FYR (Eureka)

Image Added

FYR (non-Eureka)

On qcon module mod-serials spiking affect DB connections growth from 700 to 1400.

Image Added

DB Load

Test FY rollover (non-Eureka)

On this graphs sql statement responsible for budget encumbrances rollover consume most of CPU.

Image Added

FY rollover

Image Added


Top SQL

Test FY rollover (non-Eureka)

On this graphs sql statement responsible for budget encumbrances rollover consume most of CPU.

Image Added

FY rollover

Image Added

Appendix

Infrastructure

PTF -environment qelc2

  • 10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
  • 1 database  instance, writer.

    NameMemory GIBvCPUsmax_connections

    db.r6g.4xlarge

    128 GiB16 vCPUs4000
    db.r6g.xlarge32 GB4vCPUs2000

    database cluster and instance parameters max_locks_per_transaction - 64, changed to 1024

  • MSK fse-tenant
    • 4 kafka.m7g.xlarge brokers in 2 zones (2 brokers per zone)
    • Apache Kafka version 3.7.x, metadata mode - KRaft

    • EBS storage volume per broker 300 GiB

    • auto.create.topics.enable=true
    • log.retention.minutes=480
    • default.replication.factor=2

PTF -environment qcon

  • 10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
  • 1 database  instance, writer

    NameMemory GIBvCPUsmax_connections

    db.r6g.xlarge

    32 GiB4 vCPUs2000

    database cluster and instance parameters max_locks_per_transaction - 64, changed to 1024

Expand
titleModule QELC2

Cluster Resources - qelc2-pvt (Wed Nov 06 11:34:51 UTC 2024)

ModuleTask Definition RevisionModule VersionTask CountMem Hard LimitMem Soft LimitCPU UnitsXmxMetaspace SizeMax Metaspace Size
mod-remote-storage22mod-remote-storage:3.2.004920447210243960512512
mod-remote-storage - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-datafinance-importstorage2124mod-datafinance-importstorage:38.16.102048102418448962561024129270038488512128
mod-datafinance-import storage - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
edgemod-dematicncip923edgemod-dematicncip:21.214.3501024896076888128
mod-consortia-keycloak20mod-consortia-keycloak:1.4.205136477610244416384512mod-consortia-keycloak - -ncip - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-agreements22mod-agreements:7.0.60159214880000
mod-agreements - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-tagsebsconet21mod-tagsebsconet:2.2.0012481024896128070076812888128256
mod-tags ebsconet - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
edge-coursessip296edge-coursessip2:13.42.2501024896076888128
mod-inventory-updateorganizations2223mod-inventory-updateorganizations:31.39.1201024896076888128
mod-inventory-update organizations - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-notifysettings21mod-notifysettings:31.20.0301024896020076888128
mod-notify settings - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-serials-configurationmanagement2322mod-serials-configurationmanagement:51.100.0301024248089623120768179288384128512
mod-configuration serials-management - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-orders-storagesearch3621mod-orders-storagesearch:133.72.46010242592896248051220487001440885121281024
mod-orders-storage search - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
foliomod-data-keycloakimport1621foliomod-data-keycloakimport:253.01.1.1500168420481536184402562048129203840edge-caiasoft512
mod-data-import - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
edge-dematic9edge-caiasoftdematic:2.2.4301024896076888128
mod-passwordconsortia-validatorkeycloak2120mod-passwordconsortia-validatorkeycloak:31.4.2.101440513612984776010247684416384512
mod-passwordconsortia-validator keycloak - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-gobitags2321mod-gobitags:2.82.1001024896076888128
mod-gobi tags - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
modedge-licensescourses219modedge-licensescourses:61.04.32024801024231289651201792768384512mod-licenses 88128
mod-inventory-update22mod-inventory-update:3.3.101024896076888128
mod-inventory-update - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-bulk-operationsnotify1721mod-bulk-operationsnotify:3.2.0.203072102426008960153676838488512128
mod-bulknotify - operations - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-fqm-managerconfiguration2423mod-fqm-manager:2configuration:5.10.0.501024896076888128
mod-fqm-manager configuration - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-orders-graphqlstorage2136mod-orders-graphqlstorage:113.127.1401024896051276870088128
mod-orders-graphql storage - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
modfolio-financekeycloak2616modfolio-financekeycloak:425.0.91.01500102416848961536076820488801280
mod-finance - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-roles-keycloak29mod-roles-keycloak:1.4.401024896051288256
mod-roles-keycloak edge-caiasoft9edge-caiasoft:2.2.401024896076888128
mod-password-validator21mod-password-validator:3.2.10144012980768384512
mod-password-validator - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-batch-printgobi1823mod-batch-print:1gobi:2.8.1.001024896076888128
mod-batch-print gobi - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-copycatlicenses21mod-copycatlicenses:1.6.0.301024248089623120512768179288384128512
mod-copycat licenses - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-listsbulk-operations2317mod-bulk-listsoperations:2.0.6201024307289626000768153688384128512
mod-lists bulk-operations - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-permissionsfqm-manager3824mod-fqm-permissionsmanager:62.0.5.001684102415448965120102476838488512128
mod-permissions fqm-manager - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-entities-linksgraphql21mod-entities-linksgraphql:31.012.210259210242480896400014407680881024128
mod-entities-links graphql - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
pubmod-edgefinance226pubmod-edgefinance:20234.069.1400102489607680880128
mod-orders26mod-orders:12.8.802048174010241024384512mod-orders - finance - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800kong14kong:3.7.1.83061446144204876888128edge-patron9edge-patron:5.1.201024896256
76888128mod-usersroles-keycloak3229mod-usersroles-keycloak:1.54.3401024896051288256
mod-usersroles-keycloak - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
edgemod-ncipbatch-print918edgemod-batch-ncipprint:1.101.1001024896076888128mgr-applications10
mgr-applicationsmod-batch-print - Sidecar 1N/Afolio-module-sidecar:1.30.30.3121024N/A89666551205121288802560
mod-users-blcopycat1821mod-users-blcopycat:71.86.00144010241152896512092276888128
mod-userscopycat - bl - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mgrmod-tenantslists1023mgrmod-tenantslists:12.30.3601024896512051276888256128
mod-invoice29mod-invoice:5.8.201440115251292288128mod-invoice - Sidecar 1N/Afolio-module-sidecar:lists - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-inventory-storagepermissions2438mod-inventory-storagepermissions:276.15.4004096168436901544204851230761024384512
mod-inventory-storage permissions - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-userentities-importlinks21mod-userentities-importlinks:3.80.020102425928962480040076814408801281024
mod-userentities-import links - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
modpub-senderedge222modpub-senderedge:12023.1206.0140102489607688801280
mod-sender orders26mod-orders:12.8.802048174010241024384512
mod-orders - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
edge-oai-pmh9edge-oai-pmh:2.9.2kong14kong:3.7.1.830151261441360614410242048144076838488512128
mgredge-tenant-entitlementspatron109mgredge-tenant-entitlementspatron:5.1.2.40102489651225651276888256128
mod-loginusers-keycloak3132mod-loginusers-keycloak:1.45.301024896051288256
mod-loginusers-keycloak - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-data-export-worker21mod-data-export-worker:3.2.403072280010242048384512
mod-data-export-worker - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-rtac21mod-rtac:3.6.0010248960768edge-ncip9edge-ncip:1.10.101024896076888128
mgr-applications10mgr-applications:1.3.30102489651251288256
mod-users-bl18mod-users-bl:7.8.001440115251292288128
mod-users-rtac bl - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
modmgr-circulation-storagetenants3110modmgr-circulation-storagetenants:171.23.302880102425928961536512181451238488512256
mod-circulation-storage invoice29mod-invoice:5.8.201440115251292288128
mod-invoice - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-inventory-schedulerstorage2224mod-schedulerinventory-storage:27.1.2.34010244096896369002048512307688384256512
mod-inventory-scheduler storage - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-user-calendarimport21mod-calendaruser-import:3.18.001024896076888128
mod-calendar user-import - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-source-record-storage26sender22mod-source-record-storage:5.8.6sender:1.12.0056001024500089620480350076838488512128
mod-source-recordsender - storage - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
modedge-eventoai-configpmh219modedge-eventoai-configpmh:2.79.12010241512896136001024768144088384128512
modmgr-event-config - Sidecar 1N/Afolio-module-sidecartenant-entitlements10mgr-tenant-entitlements:1.02.40.312N/A102466589651205121288802560
mod-login-courseskeycloak2031mod-courseslogin-keycloak:1.4.10301024896076851288128256
mod-courses login-keycloak - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-data-export-inventoryworker2421mod-inventory:20data-export-worker:3.2.7402880307225922800102418142048384512
mod-data-inventory export- Sidecar worker - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-emailrtac21mod-emailrtac:13.176.002800102425508965120180076838488512128
mod-email rtac - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-di-convertercirculation-storage2331mod-di-convertercirculation-storage:217.2.23010242880896259201536768181488384128512
mod-di-convertercirculation-storage - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-circulationscheduler2822mod-circulationscheduler:241.2.53028801024259289615360181451238488512256
mod-circulation scheduler - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-pubsubcalendar2321mod-pubsubcalendar:23.141.001536102414408961024092276838488512128
mod-pubsub calendar - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
edge-orders11edge-orders:3.0.301024896076888128
edge-rtac9edge-rtac:2.7.301024896076888128
mod-users23mod-users:19.3mod-source-record-storage26mod-source-record-storage:5.8.605600500020483500384512
mod-source-record-storage - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-event-config21mod-event-config:2.7.101024896076888128
mod-event-users config - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-template-enginecourses2120mod-template-enginecourses:1.204.01001024896076888128
mod-templatecourses - engine - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-patron-blocksinventory1824mod-patron-blocksinventory:120.102.0701024288089625921024768181488384128512
mod-patroninventory - blocks - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-auditemail2221mod-auditemail:21.917.001024280089625500512768180088384128512
mod-audit email - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
edge-fqm9edge-fqmmod-di-converter-storage23mod-di-converter-storage:2.02.201024896076888128
mod-sourcedi-record-manager22mod-source-record-manager:3.8.605600500020483500384512mod-source-record-manager - converter-storage - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
nginxmod-edgecirculation228nginxmod-edgecirculation:202324.06.14010248960000mod-quick-marc21mod-quick-marc:5.1.12.5022882880217625920153616641814384512
mod-quick-marc circulation - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-feesfinespubsub2123mod-feesfinespubsub:192.114.001024153689614400102476892288384128512
mod-feesfines pubsub - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
modedge-invoice-storageorders2511modedge-invoice-storageorders:53.80.13018721024153689610240102476838488512128
mod-invoice-storage - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-service-interaction22mod-service-interaction:4.0.20204818442561290384512
mod-service-interaction edge-rtac9edge-rtac:2.7.301024896076888128
mod-users23mod-users:19.3.101024896076888128
mod-users - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-datatemplate-exportengine2021mod-datatemplate-exportengine:51.20.0.40259210242480896102401440768881024128
mod-datatemplate-export engine - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-patron-blocks2518mod-patron-blocks:61.110.0010248960102476888128
mod-patron-blocks - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-oai-pmhaudit22mod-oai-pmh:3.13.2audit:2.9.0040961024369089620480307676838488512128
mod-oai-pmh audit - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
edge-connexionfqm69edge-connexionfqm:12.30.1201024896076888128
mod-kbsource-ebscorecord-javamanager2122mod-kbsource-ebscorecord-javamanager:43.08.06010245600896500002048768350088384128512
mod-kbsource-ebscorecord-java manager - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
modnginx-notesedge222modnginx-notesedge:52023.206.014010248960952038405120
mod-notes quick- Sidecar 1N/Afolio-module-sidecar:1.0marc21mod-quick-marc:5.1.102288217601664384512
mod-quick-marc - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-data-export-spring20feesfines21mod-data-export-spring:3.3feesfines:19.1.002048102418448962560153676838488512128
mod-data-exportfeesfines - spring - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-organizationsinvoice-storage2325mod-organizationsinvoice-storage:45.78.01010241872896153601024768102488384128512
mod-organizationsinvoice-storage - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
edgemod-service-erminteraction622edgemod-service-erminteraction:14.0.2.101024204889618440256768129088384128
Expand
titleModule QCON

Cluster Resources - qcon-pvt (Wed Nov 06 11:28:29 UTC 2024)

ModuleTask Definition RevisionModule VersionTask CountMem Hard LimitMem Soft LimitCPU UnitsXmxMetaspace SizeMax Metaspace Size
mod-remote-storage1mod-remote-storage:3.2.024920447210243960512512
mod-finance-storage1mod-finance-storage:8.6.021024896102470088128
mod-ncip1mod-ncip:1.14.421024896128512
mod-service-interaction - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-data-export20mod-data-export:5.0.402592248010241440881024
mod-data-export - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-patron25mod-patron:6.1.001024896076888128
mod-agreements1mod-agreements:7patron - Sidecar 1N/Afolio-module-sidecar:1.0.0215921488.312N/A6655120128000
mod-oai-ebsconetpmh122mod-oai-ebsconetpmh:23.13.2.02409612483690102420481283076700384128512256
mod-consortia1mod-consortiaoai-pmh - Sidecar 1N/Afolio-module-sidecar:1.10.0230722048.312N/A66551201282048051201024
modedge-organizationsconnexion16modedge-organizationsconnexion:1.93.1021024896128070076888128
mod-kb-serialsebsco-managementjava121mod-serialskb-ebsco-managementjava:14.0.0202480102423128961280179276838488512128
mod-settings2mod-settings-kb-ebsco-java - Sidecar 1N/Afolio-module-sidecar:1.0.321024896200768881280.312N/A665512012800
mod-searchnotes922mod-searchnotes:35.32.00-SNAPSHOT.22421024259289624800204895214403845121024edge-dematic1edge-dematic:2.2.01102489612876888
128mod-notes - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-data-importexport-spring120mod-data-importexport-spring:3.13.0102048184425612921536384512
mod-tags1mod-tags:2.2.02102489612876888128
mod-authtoken6mod-authtoken:2.15.121440115251292288128
edge-courses1edge-courses:1.4.02102489612876888128
mod-inventory-update1mod-inventory-update:3.3.021024896128-data-export-spring - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-organizations-storage23mod-organizations-storage:4.7.001024896076888128
mod-organizations-storage - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
edge-erm6edge-erm:1.2.101024896076888128mod-notify



mod-configuration:5.10
Expand
1mod-notify:3.2.02102489612876888128mod-configuration1
titleModule QCON

Cluster Resources - qcon-pvt (Wed Nov 06 11:28:29 UTC 2024)

ModuleTask Definition RevisionModule VersionTask CountMem Hard LimitMem Soft LimitCPU UnitsXmxMetaspace SizeMax Metaspace Size
mod-remote-storage1mod-remote-storage:3.2.021024492089644721281024768396088512128512
mod-ordersfinance-storage31mod-ordersfinance-storage:138.76.4021024896512102470088128
edgemod-caiasoftncip1edgemod-caiasoftncip:21.214.042102489612876888128
mod-login-samlagreements1mod-login-samlagreements:27.80.0210241592896148812876808801280
mod-erm-usage-harvesterebsconet1mod-erm-usage-harvester:4.5.02102489612876888128mod-password-validator1mod-password-validator:3ebsconet:2.2.021440124812981024128768700384128512256
mod-licensesconsortia1mod-licensesconsortia:61.01.022480307223122048128179220483845125121024
mod-gobiorganizations1mod-gobiorganizations:21.89.02102489612870088128
mod-bulkserials-operationsmanagement1mod-bulkserials-operationsmanagement:21.0.023072248026002312102412815361792384512
mod-fqm-manager1settings2mod-fqm-managersettings:21.0.132300010242600896128200204876838488512128
edgemod-dcbsearch19edgemod-dcbsearch:13.13.0-SNAPSHOT.224210242592896248012820487681440885121281024
modedge-graphqldematic1modedge-graphqldematic:12.122.012102489612876888128
mod-financedata-import1mod-data-financeimport:43.91.021102420488961844128256700129288384128512
mod-erm-usagetags1mod-erm-usagetags:42.72.02102489612876888128
mod-batch-printauthtoken16mod-batch-print:1authtoken:2.15.1.0210241440896115212851276892288128
modedge-copycatcourses1modedge-copycatcourses:1.64.02102451289612876888128
mod-inventory-listsupdate1mod-inventory-listsupdate:23.03.02300010242600896128204876838488512128
mod-entities-linksnotify1mod-entities-linksnotify:3.02.0225921024248089640012814407680881024128
mod-permissionsconfiguration41mod-permissionsconfiguration:65.610.0-SNAPSHOT.3792168410241544896512128102476838488512128
pubmod-orders-edgestorage13pubmod-orders-edgestorage:202313.067.144210248961285127687000880128
modedge-orderscaiasoft31modedge-orderscaiasoft:122.82.8022048102417408961024128102476838488512128
edgemod-login-patronsaml1edgemod-login-patronsaml:52.18.02102489625612876888128edge
-ncipmod-erm-usage-harvester1edge-ncip:1.9.2mod-erm-usage-harvester:4.5.02102489612876888128
mod-userspassword-blvalidator1mod-userspassword-blvalidator:73.72.0214401152129851212892276888384128512
mod-invoicelicenses51mod-invoicelicenses:56.80.2022880230422048248023121281792384512
mod-gobi1mod-gobi:2.8.02102489612870088128
mod-inventorybulk-storageoperations21mod-inventorybulk-storageoperations:27.2.0-SNAPSHOT.7380240963072369026002048102430761536384512
mod-userfqm-importmanager1mod-userfqm-importmanager:32.80.012102430008962600128768204888384128512
modedge-senderdcb1modedge-senderdcb:1.121.02102489612876888128
edgemod-oai-pmhgraphql1edgemod-oai-pmhgraphql:21.912.0121512102413608961024128144076838488512128
mod-data-export-workerfinance1mod-data-export-worker:3.2.1finance:4.9.023072102420488961024128204870038488512128
mod-erm-rtacusage1mod-erm-rtacusage:34.67.02102489612876888128
mod-circulationbatch-storageprint1mod-circulationbatch-storageprint:171.21.022880102425928961536128181476838488512128
mod-source-record-storagecopycat1mod-source-record-storage:5.8copycat:1.6.025600102450005122048128350076838488512128
mod-calendarlists1mod-calendarlists:32.10.02102430008962600128768204888384128512
mod-evententities-configlinks1mod-evententities-configlinks:23.70.0210242592896248012840076814408801281024
mod-coursespermissions14mod-coursespermissions:1.4.106.6.0-SNAPSHOT.3792102416848961544128512768102488384128512
modpub-circulation-itemedge1modpub-circulation-itemedge:12023.006.01421024896128076800
mod-inventoryorders3mod-inventoryorders:2012.28.0822880204825921740102418141024384512
modedge-emailpatron1modedge-emailpatron:5.1.17.02102489612825676888128
modedge-circulationncip21modedge-circulationncip:241.9.2.122880102425928961536128181476838488512128
mod-pubsubusers-bl1mod-pubsubusers-bl:27.137.021536144014401152102451292238488512128
mod-di-converter-storage2invoice5mod-di-converter-storage:2.2invoice:5.8.221024288089623041282768204888128
edgemod-inventory-ordersstorage12edgemod-inventory-ordersstorage:327.2.0-SNAPSHOT.073821024409689636901282048768307688384128512
edgemod-user-rtacimport1edgemod-user-rtacimport:23.78.102102489612876888128
mod-userssender31mod-userssender:191.312.102204810241792896128102476888256128
modedge-templateoai-enginepmh1modedge-templateoai-enginepmh:12.209.021024151289613601281024768144088384128512
mod-patrondata-export-blocksworker1mod-data-patronexport-blocksworker:13.102.0121024307289620481024768204888384128512
mod-auditrtac1mod-auditrtac:23.96.02102489612876888128
edgemod-circulation-fqmstorage1edgemod-circulation-fqmstorage:17.2.0.021024288089625921281536768181488384128512
mod-source-record-managerstorage41mod-source-record-managerstorage:35.8.025600500020483500384512
nginxmod-edgecalendar1nginxmod-edgecalendar:20233.061.1402102489612807680880128
mod-quickevent-marcconfig21mod-quickevent-marcconfig:52.17.02102489612876888128
mod-courses1mod-courses:122882176.4.1021024896128166476838488512128
nginxmod-circulation-okapiitem1nginxmod-circulation-okapiitem:20231.060.14021024896128000
okapi-b1okapi:5.3.03168414401024922384512
mod-feesfines1mod-feesfines:19.1.02102489612876888128
mod-invoice-storage3mod-invoice-storage:5.8.121872153621024384512
mod-reading-room7mod-reading-room:1.0.0-SNAPSHOT.112102489612876888128
mod-service-interaction1mod-service-interaction:4.0.12204818442561290384512
mod-dcb1mod-dcb:1.1.02102489612876888128
mod-patron1mod-patron:6.1.02102489612876888128
mod-data-export1mod-data-export:5.0.01204815241024000
mod-oai-pmh1mod-oai-pmh:3.13.024096369020483076384512
edge-connexion1edge-connexion:1.2.02102489612876888128
mod-notes1mod-notes:5.2.021024896128952384512
mod-kb-ebsco-java1mod-kb-ebsco-java:4.0.02102489612876888128
mod-login2mod-login:7.11.02204817921024102488256
mod-organizations-storage1mod-organizations-storage:4.7.02102489612870088128
mod-data-export-spring1mod-data-export-spring:3.2.01204818442561536384512
pub-okapi1pub-okapi:2023.06.142102489612876800
mod-eusage-reports1mod-eusage-reports:2.1.12102489612876888128

Methodology/Approach

Files

  • View file
    namePERF_865_FYR_results.xlsx
    height250

...

mod-inventory3mod-inventory:20.2.022880259210241814384512
mod-email1mod-email:1.17.02102489612876888128
mod-circulation2mod-circulation:24.2.122880259215361814384512
mod-pubsub1mod-pubsub:2.13.02153614401024922384512
mod-di-converter-storage2mod-di-converter-storage:2.2.22102489612876888128
edge-orders1edge-orders:3.0.02102489612876888128
edge-rtac1edge-rtac:2.7.12102489612876888128
mod-users3mod-users:19.3.1220481792128102488256
mod-template-engine1mod-template-engine:1.20.02102489612876888128
mod-patron-blocks1mod-patron-blocks:1.10.021024896102476888128
mod-audit1mod-audit:2.9.02102489612876888128
edge-fqm1edge-fqm:2.0.02102489612876888128
mod-source-record-manager4mod-source-record-manager:3.8.025600500020483500384512
nginx-edge1nginx-edge:2023.06.1421024896128000
mod-quick-marc2mod-quick-marc:5.1.01228821761281664384512
nginx-okapi1nginx-okapi:2023.06.1421024896128000
okapi-b1okapi:5.3.03168414401024922384512
mod-feesfines1mod-feesfines:19.1.02102489612876888128
mod-invoice-storage3mod-invoice-storage:5.8.121872153621024384512
mod-reading-room7mod-reading-room:1.0.0-SNAPSHOT.112102489612876888128
mod-service-interaction1mod-service-interaction:4.0.12204818442561290384512
mod-dcb1mod-dcb:1.1.02102489612876888128
mod-patron1mod-patron:6.1.02102489612876888128
mod-data-export1mod-data-export:5.0.01204815241024000
mod-oai-pmh1mod-oai-pmh:3.13.024096369020483076384512
edge-connexion1edge-connexion:1.2.02102489612876888128
mod-notes1mod-notes:5.2.021024896128952384512
mod-kb-ebsco-java1mod-kb-ebsco-java:4.0.02102489612876888128
mod-login2mod-login:7.11.02204817921024102488256
mod-organizations-storage1mod-organizations-storage:4.7.02102489612870088128
mod-data-export-spring1mod-data-export-spring:3.2.01204818442561536384512
pub-okapi1pub-okapi:2023.06.142102489612876800
mod-eusage-reports1mod-eusage-reports:2.1.12102489612876888128


Methodology/Approach

In order to run FYR:

truncate tables with query

Code Block
languagesql
titleTruncate
truncate table [tenant]_mod_organizations_storage.organizations cascade;
truncate table [tenant]_mod_orders_storage.purchase_order cascade;
truncate table [tenant]_mod_invoice_storage.invoices cascade;

Create organization (Vendor) to reuse its id as access provider in the next query which generates needed data set.

Replace FYR id, access provider, fund id, fund name in accordance with FYR ledger info. Also check all instance related fields which should be replaced with actual data.  The script prepared to generate needed amount of organizations, orders, po lines, invoices.

Expand
titleQueries to generate data


Code Block
languagesql
titleGenerate
CREATE OR REPLACE FUNCTION public.generate_0062_data_for_edifact_export(
scenario_prefix TEXT,
organizations_amount INTEGER,
orders_per_vendor INTEGER,
polines_per_order INTEGER,
pieces_per_order INTEGER
) RETURNS VOID AS
$$
DECLARE
orgName TEXT DEFAULT 'perf_test_vendor';
orgCodePrefix TEXT DEFAULT 'PERF_TEST_ORG';
vendor_id TEXT;
BEGIN
FOR org_counter IN 1..organizations_amount
LOOP
INSERT INTO [tenant]_mod_organizations_storage.organizations (id, jsonb)
VALUES (
public.uuid_generate_v4(),
jsonb_build_object(
'code', concat(orgCodePrefix, '_', scenario_prefix, '_', org_counter),
'erpCode', '12345',
'isVendor', true,
'name', concat(orgCodePrefix, '_', scenario_prefix, '_', org_counter),
'status', 'Active',
'metadata', jsonb_build_object(
'createdDate', '2023-02-08T00:00:00.000+0000',
'createdByUserId', '9f9d1c46-52e1-4bb7-9c6c-56e6bb945c42',
'updatedDate', '2023-02-08T00:00:00.000+0000',
'updatedByUserId', '9f9d1c46-52e1-4bb7-9c6c-56e6bb945c42'
)
)
)
RETURNING id INTO vendor_id;

PERFORM public.generate_0062_orders(orders_per_vendor, polines_per_order, pieces_per_order, vendor_id,1,2);
END LOOP;
END
$$ LANGUAGE plpgsql;

CREATE OR REPLACE FUNCTION public.generate_0062_orders(
orders_per_vendor INTEGER,
polines_per_order INTEGER,
pieces_per_order INTEGER,
vendor_id TEXT,
invoices_per_organization INTEGER,
invoicelines_per_invoice INTEGER
) RETURNS VOID AS
$$
DECLARE
order_id TEXT;
poline_id TEXT;
newPoNumber INTEGER;
BEGIN
FOR order_counter IN 1..orders_per_vendor
LOOP
SELECT nextval('[tenant]_mod_orders_storage.po_number') INTO newPoNumber;

INSERT INTO [tenant]_mod_orders_storage.purchase_order (id, jsonb)
VALUES (
public.uuid_generate_v4(),
jsonb_build_object(
'id', public.uuid_generate_v4(),
'notes', jsonb_build_array(),
'acqUnitIds', jsonb_build_array(),
'approved', true,
'reEncumber', true,
'nextPolNumber', 2,
'workflowStatus', 'Pending',
'poNumber', newPoNumber,
'vendor', vendor_id,
'orderType', 'One-Time',
'metadata', jsonb_build_object(
'createdDate', '2024-10-06T00:00:00.000+0000',
'createdByUserId', '9f9d1c46-52e1-4bb7-9c6c-56e6bb945c42',
'updatedDate', '2024-10-06T00:00:00.000+0000',
'updatedByUserId', '9f9d1c46-52e1-4bb7-9c6c-56e6bb945c42'
)
)
)
RETURNING id INTO order_id;

PERFORM public.generate_0062_polines(order_id, polines_per_order, pieces_per_order, newPoNumber, vendor_id);
PERFORM public.generate_0062_invoices(invoices_per_organization, invoicelines_per_invoice, vendor_id, poline_id);
END LOOP;
END
$$ LANGUAGE plpgsql;

CREATE OR REPLACE FUNCTION public.generate_0062_polines(
order_id TEXT,
polines_per_order INTEGER,
pieces_per_order INTEGER,
ponumber INTEGER,
vendor_id TEXT
) RETURNS VOID AS
$$
DECLARE
polineNumber TEXT;
holding_id UUID;
currPoNumber INTEGER;
iteration_counter INT := 1;

BEGIN
SELECT id INTO holding_id
FROM [tenant]_mod_inventory_storage.holdings_record 
WHERE permanentlocationid = '9ffb23a9-0e83-4d8e-a70d-ddf44ab67c84' 
AND instanceid = 'b94d2138-bdfc-5dea-bab9-ac99e21b7970'
LIMIT 1;

FOR line_counter IN 1..polines_per_order
LOOP
SELECT currval('[tenant]_mod_orders_storage.po_number') INTO currPoNumber;

INSERT INTO [tenant]_mod_orders_storage.po_line (id, jsonb)
VALUES (
public.uuid_generate_v4(),
jsonb_build_object(
'id', public.uuid_generate_v4(),
'cost', jsonb_build_object(
'currency', 'USD',
'discountType', 'percentage',
'quantityElectronic', 2,
'poLineEstimatedPrice', 4,
'listUnitPriceElectronic', 2
),
'rush', false,
'alerts', jsonb_build_array(),
'claims', jsonb_build_array(),
'source', 'User',
'details', jsonb_build_object(
'productIds', jsonb_build_array(
jsonb_build_object(
'productId', '73121721',
'productIdType', 'c858e4f2-2b6b-4385-842b-60732ee14abb'
)
),
'isAcknowledged', false
),
'edition', '',
'metadata', jsonb_build_object(
'createdDate', '2024-10-06T11:09:59.181Z',
'updatedDate', '2024-10-06T11:10:48.652Z',
'createdByUserId', '9f9d1c46-52e1-4bb7-9c6c-56e6bb945c42',
'updatedByUserId', '9f9d1c46-52e1-4bb7-9c6c-56e6bb945c42'
),
'eresource', jsonb_build_object(
'trial', false,
'activated', false,
'accessProvider', 'c88742c3-dc2f-4589-a7cf-a90694c2a296',
'createInventory', 'Instance, Holding'
),
'isPackage', false,
'locations', jsonb_build_array(
jsonb_build_object(
'quantity', 2,
'locationId', '9ffb23a9-0e83-4d8e-a70d-ddf44ab67c84',
'quantityElectronic', 2
)
),
'publisher', 'Praeger Publishers',
'collection', false,
'instanceId', 'b94d2138-bdfc-5dea-bab9-ac99e21b7970',
'orderFormat', 'Electronic Resource',
'checkinItems', false,
'contributors', jsonb_build_array(
jsonb_build_object(
'contributor', 'Yurchenco, Henrietta',
'contributorNameTypeId', '2b94c631-fca9-4892-a730-03ee529ffe2a'
)
),
'poLineNumber', currPoNumber || '-' || iteration_counter,
'vendorDetail', jsonb_build_object(
'instructions', '',
'vendorAccount', '',
'referenceNumbers', jsonb_build_array()
),
'paymentStatus', 'Pending',
'receiptStatus', 'Pending',
'claimingActive', false,
'reportingCodes', jsonb_build_array(),
'titleOrPackage', '!Hablamos! Puerto Ricans speak / Photos. by Julia Singer.',
'automaticExport', false,
'publicationDate', '[1971]',
'purchaseOrderId', order_id,
'fundDistribution', jsonb_build_array(
jsonb_build_object(
'code', 'FYND',
'value', 100,
'fundId', '8003d427-34a3-4ad4-97cd-c1aae19f2bfc',
'distributionType', 'percentage'
)
),
'acquisitionMethod', '796596c4-62b5-4b64-a2ce-524c747afaa2',
'searchLocationIds', jsonb_build_array(
'9ffb23a9-0e83-4d8e-a70d-ddf44ab67c84'
),
'donorOrganizationIds', jsonb_build_array(),
'checkinItems', true
)
)
RETURNING id INTO polineNumber;
iteration_counter := iteration_counter + 1;

-- Call the function to generate data for titles table
PERFORM public.generate_0062_titles_data(polineNumber, concat(ponumber, '-', line_counter), pieces_per_order);
END LOOP;
END;
$$ LANGUAGE plpgsql;



CREATE OR REPLACE FUNCTION public.generate_0062_titles_data(
po_line_id TEXT,
po_line_number TEXT,
pieces_per_order INTEGER
) RETURNS VOID AS
$$
DECLARE
title_id UUID;
BEGIN
-- Insert data into titles table
INSERT INTO [tenant]_mod_orders_storage.titles (id, jsonb, creation_date, created_by, polineid)
VALUES (
public.uuid_generate_v4(),
jsonb_build_object(
'id', '4945c26f-6cf7-4050-93c1-0a8a25a2f4d7',
'title', '!Hablamos! Puerto Ricans speak / Photos. by Julia Singer.',
'edition', '',
'metadata', jsonb_build_object(
'createdDate', '2024-10-06T11:09:59.247Z',
'updatedDate', '2024-10-06T11:09:59.247Z',
'createdByUserId', '9f9d1c46-52e1-4bb7-9c6c-56e6bb945c42',
'updatedByUserId', '9f9d1c46-52e1-4bb7-9c6c-56e6bb945c42'
),
'poLineId', po_line_id,
'publisher', 'Praeger Publishers',
'acqUnitIds', jsonb_build_array(),
'instanceId', 'b94d2138-bdfc-5dea-bab9-ac99e21b7970',
'productIds', jsonb_build_array(
jsonb_build_object(
'productId', '73121721',
'productIdType', 'c858e4f2-2b6b-4385-842b-60732ee14abb'
)
),
'contributors', jsonb_build_array(
jsonb_build_object(
'contributor', 'Yurchenco, Henrietta',
'contributorNameTypeId', '2b94c631-fca9-4892-a730-03ee529ffe2a'
)
),
'poLineNumber', po_line_number,
'publishedDate', '[1971]',
'claimingActive', false,
'isAcknowledged', false
),
current_timestamp, -- Assuming creation date is the current timestamp
'created_by_user_id', -- Replace with the actual user ID
po_line_id::uuid
)
RETURNING id INTO title_id;

-- Call the function to generate data for pieces table
PERFORM public.generate_0062_pieces_data(title_id, po_line_id, pieces_per_order);
END
$$ LANGUAGE plpgsql;


CREATE OR REPLACE FUNCTION public.generate_0062_pieces_data(
title_id UUID,
polineNumber TEXT,
pieces_per_order INTEGER
) RETURNS VOID AS
$$
DECLARE
piece_counter INTEGER;
holding_id UUID;
BEGIN
SELECT id INTO holding_id
FROM [tenant]_mod_inventory_storage.holdings_record 
WHERE permanentlocationid = '9ffb23a9-0e83-4d8e-a70d-ddf44ab67c84' 
AND instanceid = 'b94d2138-bdfc-5dea-bab9-ac99e21b7970'
LIMIT 1;

FOR piece_counter IN 1..pieces_per_order
LOOP
-- Replace the following line with your actual poLineId
INSERT INTO [tenant]_mod_orders_storage.pieces VALUES (
public.uuid_generate_v4(),
jsonb_build_object(
'id', public.uuid_generate_v4(),
'format', 'Electronic',
'titleId', title_id,
'metadata', jsonb_build_object(
'createdDate', '2024-10-06T11:43:04.384Z',
'updatedDate', '2024-10-06T11:43:04.384Z',
'createdByUserId', '9f9d1c46-52e1-4bb7-9c6c-56e6bb945c42',
'updatedByUserId', '9f9d1c46-52e1-4bb7-9c6c-56e6bb945c42'
),
'poLineId', polineNumber,
'holdingId', 'aefd322b-9a32-4cd9-80e0-a5a3f882f2b1',
'copyNumber', piece_counter,
'supplement', true,
'displaySummary', piece_counter,
'receivingStatus', 'Expected',
'displayOnHolding', false,
'discoverySuppress', false,
'statusUpdatedDate', '2024-06-05T11:43:04.385+00:00'
),
NULL,
NULL,
'eb79cb72-0fb5-4773-a580-b1428fa724e9'::UUID, -- Replace with the actual poLineId
title_id
);
END LOOP;
END
$$ LANGUAGE plpgsql;






CREATE OR REPLACE FUNCTION public.generate_0062_invoices(
invoices_per_organization INTEGER,
invoicelines_per_invoice INTEGER,
vendor_id TEXT,
order_id TEXT
) RETURNS VOID AS
$$
DECLARE
invoice_id TEXT;
newInvoiceNumber INTEGER;
BEGIN
FOR invoice_counter IN 1..invoices_per_organization
LOOP
SELECT nextval('[tenant]_mod_invoice_storage.invoice_number') INTO newInvoiceNumber;

INSERT INTO [tenant]_mod_invoice_storage.invoices (id, jsonb)
VALUES (
public.uuid_generate_v4(),
jsonb_build_object(
'id', public.uuid_generate_v4(),
'total', 0.0,
'source', 'User',
'status', 'Approved',
'currency', 'USD',
'metadata', jsonb_build_object(
'createdDate', '2024-10-06T09:07:45.366Z',
'updatedDate', '2024-10-06T09:08:47.954Z',
'createdByUserId', '9f9d1c46-52e1-4bb7-9c6c-56e6bb945c42',
'updatedByUserId', '9f9d1c46-52e1-4bb7-9c6c-56e6bb945c42'
),
'subTotal', 0.0,
'vendorId', vendor_id,
'poNumbers', ARRAY[newInvoiceNumber]::TEXT[],
'acqUnitIds', ARRAY[]::UUID[],
'adjustments', ARRAY[]::JSONB[],
'invoiceDate', '2024-02-27T00:00:00.000+00:00',
'batchGroupId', '2a2cb998-1437-41d1-88ad-01930aaeadd5',
'fiscalYearId', '562bf586-11e9-4c36-98d7-3341042c856c',
'paymentMethod', 'Cash',
'accountingCode', '12345',
'folioInvoiceNo', newInvoiceNumber::TEXT,
'enclosureNeeded', false,
'vendorInvoiceNo', invoice_counter::TEXT,
'adjustmentsTotal', 0.0,
'exportToAccounting', false,
'nextInvoiceLineNumber', 2,
'chkSubscriptionOverlap', true
)
) RETURNING id INTO invoice_id;

PERFORM public.generate_0062_invoicelines(
invoicelines_per_invoice,
invoice_id,
newInvoiceNumber,
vendor_id,
order_id
);
END LOOP;
END
$$ LANGUAGE plpgsql;

CREATE OR REPLACE FUNCTION public.generate_0062_invoicelines(
invoicelines_per_invoice INTEGER,
invoice_id TEXT,
newInvoiceNumber INTEGER,
vendor_id TEXT,
order_id TEXT
) RETURNS VOID AS
$$
DECLARE
invoiceline_id TEXT;
BEGIN
FOR invoiceline_counter IN 1..invoicelines_per_invoice
LOOP
INSERT INTO [tenant]_mod_invoice_storage.invoice_lines (id, jsonb)
VALUES (
public.uuid_generate_v4(),
jsonb_build_object(
'id', public.uuid_generate_v4(),
'total', 0.0,
'metadata', jsonb_build_object(
'createdDate', '2024-10-06T09:08:47.393Z',
'updatedDate', '2024-10-06T09:08:47.393Z',
'createdByUserId', '9f9d1c46-52e1-4bb7-9c6c-56e6bb945c42',
'updatedByUserId', '9f9d1c46-52e1-4bb7-9c6c-56e6bb945c42'
),
'poLineId', order_id,
'quantity', 2,
'subTotal', 0.0,
'invoiceId', invoice_id,
'adjustments', ARRAY[]::JSONB[],
'description', 'Its awesome if its working',
'accountNumber', 'libraryorders@library.tam',
'accountingCode', '12345',
'adjustmentsTotal', 0.0,
'referenceNumbers', ARRAY[]::JSONB[],
'fundDistributions', jsonb_build_array(
jsonb_build_object(
'code', 'FYND',
'value', 100.0,
'fundId', '8003d427-34a3-4ad4-97cd-c1aae19f2bfc',
'distributionType', 'percentage'
)
),
'invoiceLineNumber', invoiceline_counter,
'invoiceLineStatus', 'Approved',
'releaseEncumbrance', false
)
) RETURNING id INTO invoiceline_id;

-- You can perform additional actions if needed
END LOOP;
END
$$ LANGUAGE plpgsql;


-- CREATE sample data
-- 1 - amount of organizations to be created
-- 2 - amount of orders per organization
-- 3 - amount of polines per order
-- 4 - amount of pieces per order

SELECT public.generate_0062_data_for_edifact_export('FYNG',1,1,1,1);



-- CLEANUP



To create encumbrances linked to the current budget orders should be with open status. To change the status from pending to open select needed data for the jmeter script. Save to use in the PUT request orders/composite-orders/${po_id}. Run needed number of requests to prepare open orders.

Code Block
titleData preparation to Open Orders
SELECT id as po_id, jsonb->>'vendor' as vendor_id, jsonb->>'poNumber' as po_number
select count(id)
FROM cs00000int_mod_orders_storage.purchase_order
where jsonb->>'workflowStatus'='Pending'


Code Block
titleBODY DATA
{"id":"${po_id}","approved":true,"notes":[],"poNumber":"${po_number}","orderType":"One-Time","reEncumber":true,"vendor":"${vendor_id}","workflowStatus":"Open","acqUnitIds":[],"needReEncumber":false}

Legacy approach about data preparation instructions can be found at the link: Steps for testing process#Fiscalyearrollover Use this link to get exact number of organizations, orders, POLined, invoices

  • Set max_locks_per_transaction parameter in DB cluster and instance configurations from default 64 to 1024 to avoid problems with ordersRollover.
  • Consider using prepared scripts and queries Additional info for testing (scripts, queries) saved in S3 bucket - Buckets/fse-ptf/FYR/

      ...

        • envs_data_gen_scripts - txt files with sql which populate DB with data
        • qelc2 and qcon jmeter scripts to change status from pending to open