Atlassian uses cookies to improve your browsing experience, perform analytics and research, and conduct advertising. Accept all cookies to indicate that you agree to our use of cookies on your device.
Atlassian uses cookies to improve your browsing experience, perform analytics and research, and conduct advertising. Accept all cookies to indicate that you agree to our use of cookies on your device. Atlassian cookies and tracking notice, (opens new window)
For each new release check the test script, and change it to be up to date, upload a new version to the Carrier.io circulation bucket. (For all tests test name and test plan name should be equal)
Start test from Jenkins with proper Carrier.io bucket and test name parameters. Typical tests parameters:
5, 8, 20, and 25 concurrent users for 30 minutes each. (repeat twice)
Also, run a longevity test of 24 hours to detect any trends in memory.
Analise results in Grafana and CloudWatch:
Analyze the response time of CI and CO from Grafana and compare it to the previous test
Analyze the slowest API - Grafana
Analyze memory utilization of the services in AWS (CloudWatch)
Analyze CPU and memory utilization of instances and modules involved
Analyze RDS parameters CPU utilization, Connection count, queries
Look through the AWS logs if there were any ERRORS during the test
Data Import is the FOLIO app where users can import one or more records in batch into various FOLIO apps. Data Import can import MARC Bibliographic records to create and/or update records in the Inventory App and Source Record Storage.
With data import, we can create new 1- to 500k records (commonly 200k new items and instance or holdings records created), and as a result, we have more and more records in the database after each test.
To clear the database after the DI tests use SQL:
Delete from [tenant]_mod_source_record_storage.records_lb where created_date > '[test date format:YYYY-MM-DD hh:mm:ss.00+00]';
Delete from [tenant]_mod_inventory_storage.item where creation_date > '[test date format:YYYY-MM-DD hh:mm:ss.00+00]';
Delete from [tenant]_mod_inventory_storage.holdings_record where creation_date > '[test date format:YYYY-MM-DD hh:mm:ss.00+00]';
Delete from [tenant]_mod_inventory_storage.instance where creation_date > '[test date format:YYYY-MM-DD hh:mm:ss.00+00]';
To analyze results:
Analyze the duration of DI jobs (the best way in DB table [tenant]_mod_source_record_manager.job_execution )and compare it to the previous test (or analyze from UI)
Analyze if all records created or updated successfully (can be performed from UI)
Analyze memory utilization of the DI services
Analyze CPU and memory utilization of instances and modules involved
Analyze RDS parameters CPU utilization, Connection count, queries
Look through the AWS logs if there were any ERRORS during the test
To run MARC BIB Update Select from the database the proper number of IDs(not updated records IDs). You can use records from the MARC BIB Create process.
To pick the exact execution time of DI Create (it is from started_date to completed_date) you can use:
select completed_date-started_date as job_time,hrid,file_name,started_date,completed_date, status, job_profile_name FROM [tenant]_mod_source_record_manager.job_execution order by started_date desc
The same SELECT can be used to analyze job duration.
2. To select IDs from the database for a future update:
select id from [tenant]_mod_inventory_storage.instance where creation_date >= '[started_date]' and creation_date<= '[completed_date]'
Save IDs to .csv file (IDs should start from the first line of the file)
Go to the Data EXPORT, choose created .csv file, choose profile "Export for Data Import updates" -> "instances" in drop-down -> "Run".
Click "View all", sort by ID and click on the generated .mrc file to download.
Go to the DATA IMPORT app and upload saved .mrc file with a proper number of exported marc records(1k, 2k, 5k, 10k, 25k, 50k, and 100k).
(Due to file corruption DI job can be finished with errors)
(Edited file, job could be completed without errors)
Jobs should be performed one at a time and 2-5 min pauses after.
MARC Holdings
1. Conduct data import for 1 record file with job profile Default - Create Holdings and SRS MARC Holdings.
2. After it completed - copy field 001 from newly created record (in UI). For this you should choose file name of the last DI job, than any title and copy 001 field (it should looks like “in00007579903”).
3. Open file that should be imported as marc holding using Marc edit. Replace all 004 fields with 001 that you have from previous step.
4. In UI go to ”APPS” -> Settings-> Tenant-> locations, choose any institution, campus, library and location. Copy code of this location and replace all 852 fields in file with this code. Note: not the whole field should be replaced, only part after "$b" and till the next "$".
5. Conduct data import for the files with job profile Default - Create Holdings and SRS MARC Holdings.
Main KPI here is duration of data MARC Authorities. To define exact duration of DI job use:
select completed_date-started_date as job_time,hrid,file_name,started_date,completed_date, status, job_profile_name FROM [tenant]_mod_source_record_manager.job_execution order by started_date desc
Additional info:
To debug failed jobs (or jobs that stuck) check the logs of next modules
mod-data-import
mod-data-import-cs
mod-source-record-storage
mod-source-record-manager
mod-inventory
mod-inventory-storage
Additionally : kafka broker logs,DB logs
Data export
MARC BIB
To run Data Export: create or take existing .csv files with instance IDs of 1k, 100k, 200k, and 500k records. (IDs should start from the first line of the file)
Go to the Data Export app and download the file with the proper number of marc records. Choose "srs - holdings and items" job profile → "Instance" → "Run".
Wait for the job to be finished. If you can not see your job on the preview page, click on "View all" and then click on column name "ID" twice to order jobs IDs desc.
Download exported file .mrc
Files should be exported one by one with 2-5 min pauses.
To create a Job Profile:
Go to "Settings" → "Data Export" → "Field Mapping Profiles" if the Mapping profile does not exist and click "New"
Fill in all parameters according to the screenshot. For "srs - holdings and items" job profile
Name:
srs - holdings and items locations
Output format:
MARC
FOLIO record type*(check):
Source record storage (entire record)
Holdings
Item
Description: Mapping profile for perf testing - it will get the records from SRS, inventory instance (if no underlying SRS record present) holdings and items.
Add Transformations:
Field name
Transformation
Field name
Transformation
Holdings - Permanent location
900 $a
Holdings - Temporary location
901 $a
Item - Effective call number
902 $a
Item - Effective location
903 $a
Save & close
Go to Job Profile and click "New"
Add parameters
Name: srs - holdings and items Mapping profile: srs - holdings and items locations Description: Make data export work more hard
Save & close
EDIFACT
The purpose of edifact export is to export orders (with or without PO lines) from organizations.
Unlike the others exports -edifact is using FTP server to store exported files.
We have a list of 10 predefined organizations to work with (below path to organizations integrations):
$$
DECLARE
-- !!! SET DEFAULT TENANT NAME !!!
orgName text DEFAULT 'perf_test_vendor';
orgCode TEXT default 'PERF_TEST_ORG';
vendor_id TEXT;
BEGIN
for org_counter in 1..organizations_amount
loop
/* INSERT INTO fs09000000_mod_organizations_storage.organizations (id, jsonb)
VALUES (public.uuid_generate_v4(),
jsonb_build_object('code', concat(orgCode, org_counter),
'erpCode', '12345',
'isVendor', true,
'name', concat(orgName, org_counter),
'status', 'Active',
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
))
RETURNING id INTO vendor_id;*/
PERFORM public.generate_orders(orders_per_vendor, polines_per_order, '2e6d8468-0620-475b-a092-045e659a0aaa');-------------------------------
end loop;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.generate_orders(orders_per_vendor integer, polines_per_order integer, vendor_id text) RETURNS VOID as
$$
DECLARE
order_id text;
newPoNumber integer;
BEGIN
for order_counter in 1..orders_per_vendor
loop
SELECT nextval('fs09000000_mod_orders_storage.po_number') INTO newPoNumber;
--
INSERT INTO fs09000000_mod_orders_storage.purchase_order (id, jsonb)
VALUES (public.uuid_generate_v4(),
jsonb_build_object('id', public.uuid_generate_v4(),
'reEncumber', true,
'workflowStatus', 'Open',
'poNumber', newPoNumber,
'vendor', vendor_id,
'orderType', 'One-Time',
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
))
RETURNING id INTO order_id;
PERFORM public.generate_polines(order_id, polines_per_order, newPoNumber);
end loop;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.generate_polines(order_id text, polines_per_order integer, ponumber integer) RETURNS VOID as
$$
DECLARE
polineNumber text;
BEGIN
for line_counter in 1..polines_per_order
loop
INSERT INTO fs09000000_mod_orders_storage.po_line (id, jsonb)
VALUES (public.uuid_generate_v4(),
-- add other fields to increase processing complexity
jsonb_build_object('id', public.uuid_generate_v4(),
'acquisitionMethod', 'df26d81b-9d63-4ff8-bf41-49bf75cfa70e',
'rush', false,
'cost', json_build_object(
'currency', 'USD',
'discountType', 'percentage',
'listUnitPrice', 1,
'quantityPhysical', 1,
'poLineEstimatedPrice', 1
),
'alerts', json_build_array(),
'source', 'User',
'physical', jsonb_build_object('createInventory', 'None'),
'isPackage', false,
'orderFormat', 'Physical Resource',
'vendorDetail', jsonb_build_object('vendorAccount', 'libraryorders@library.tam'),---------------------------------
'titleOrPackage', 'ABA Journal',
'automaticExport', true,
'publicationDate', '1915-1983',
'purchaseOrderId', order_id,
'poLineNumber', concat(ponumber, '-', line_counter),
'claims', json_build_array(),
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
));
end loop;
END
$$ LANGUAGE plpgsql;
-- CREATE sample data
-- 1 - amount of organizations to be created
-- 2 - amount of orders per organization
-- 3 - amount of polines per order
select public.generate_data_for_edifact_export(1, 1000, 1);
--check
--SELECT count() FROM fs09000000_mod_organizations_storage.organizations WHERE jsonb ->> 'code' LIKE 'PERF_TEST_ORG%';
--SELECT count(*) FROM fs09000000_mod_orders_storage.purchase_order;
CREATE OR REPLACE FUNCTION public.generate_data_for_edifact_export(organizations_amount integer,
orders_per_vendor integer,
polines_per_order integer) RETURNS VOID as
$$
DECLARE
-- !!! SET DEFAULT TENANT NAME !!!
orgName text DEFAULT 'perf_test_vendor';
orgCode TEXT default 'PERF_TEST_ORG';
vendor_id TEXT;
BEGIN
for org_counter in 1..organizations_amount
loop
/* INSERT INTO fs09000000_mod_organizations_storage.organizations (id, jsonb)
VALUES (public.uuid_generate_v4(),
jsonb_build_object('code', concat(orgCode, org_counter),
'erpCode', '12345',
'isVendor', true,
'name', concat(orgName, org_counter),
'status', 'Active',
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
))
RETURNING id INTO vendor_id;*/
PERFORM public.generate_orders(orders_per_vendor, polines_per_order, 'e02e4507-3c3a-40e5-b2f6-dbb9a15ac950');-------------------------------
end loop;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.generate_orders(orders_per_vendor integer, polines_per_order integer, vendor_id text) RETURNS VOID as
$$
DECLARE
order_id text;
newPoNumber integer;
BEGIN
for order_counter in 1..orders_per_vendor
loop
SELECT nextval('fs09000000_mod_orders_storage.po_number') INTO newPoNumber;
--
INSERT INTO fs09000000_mod_orders_storage.purchase_order (id, jsonb)
VALUES (public.uuid_generate_v4(),
jsonb_build_object('id', public.uuid_generate_v4(),
'reEncumber', true,
'workflowStatus', 'Open',
'poNumber', newPoNumber,
'vendor', vendor_id,
'orderType', 'One-Time',
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
))
RETURNING id INTO order_id;
PERFORM public.generate_polines(order_id, polines_per_order, newPoNumber);
end loop;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.generate_polines(order_id text, polines_per_order integer, ponumber integer) RETURNS VOID as
$$
DECLARE
polineNumber text;
BEGIN
for line_counter in 1..polines_per_order
loop
INSERT INTO fs09000000_mod_orders_storage.po_line (id, jsonb)
VALUES (public.uuid_generate_v4(),
-- add other fields to increase processing complexity
jsonb_build_object('id', public.uuid_generate_v4(),
'acquisitionMethod', 'df26d81b-9d63-4ff8-bf41-49bf75cfa70e',
'rush', false,
'cost', json_build_object(
'currency', 'USD',
'discountType', 'percentage',
'listUnitPrice', 1,
'quantityPhysical', 1,
'poLineEstimatedPrice', 1
),
'alerts', json_build_array(),
'source', 'User',
'physical', jsonb_build_object('createInventory', 'None'),
'isPackage', false,
'orderFormat', 'Physical Resource',
'vendorDetail', jsonb_build_object('vendorAccount', '852'),---------------------------------
'titleOrPackage', 'ABA Journal',
'automaticExport', true,
'publicationDate', '1915-1983',
'purchaseOrderId', order_id,
'poLineNumber', concat(ponumber, '-', line_counter),
'claims', json_build_array(),
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
));
end loop;
END
$$ LANGUAGE plpgsql;
-- CREATE sample data
-- 1 - amount of organizations to be created
-- 2 - amount of orders per organization
-- 3 - amount of polines per order
select public.generate_data_for_edifact_export(1, 1000, 1);
--check
--SELECT count() FROM fs09000000_mod_organizations_storage.organizations WHERE jsonb ->> 'code' LIKE 'PERF_TEST_ORG%';
--SELECT count(*) FROM fs09000000_mod_orders_storage.purchase_order;
CREATE OR REPLACE FUNCTION public.generate_data_for_edifact_export(organizations_amount integer,
orders_per_vendor integer,
polines_per_order integer) RETURNS VOID as
$$
DECLARE
-- !!! SET DEFAULT TENANT NAME !!!
orgName text DEFAULT 'perf_test_vendor';
orgCode TEXT default 'PERF_TEST_ORG';
vendor_id TEXT;
BEGIN
for org_counter in 1..organizations_amount
loop
/* INSERT INTO fs09000000_mod_organizations_storage.organizations (id, jsonb)
VALUES (public.uuid_generate_v4(),
jsonb_build_object('code', concat(orgCode, org_counter),
'erpCode', '12345',
'isVendor', true,
'name', concat(orgName, org_counter),
'status', 'Active',
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
))
RETURNING id INTO vendor_id;*/
PERFORM public.generate_orders(orders_per_vendor, polines_per_order, 'ede1513a-ea9b-46e5-8f2c-7d93836f9742');-------------------------------
end loop;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.generate_orders(orders_per_vendor integer, polines_per_order integer, vendor_id text) RETURNS VOID as
$$
DECLARE
order_id text;
newPoNumber integer;
BEGIN
for order_counter in 1..orders_per_vendor
loop
SELECT nextval('fs09000000_mod_orders_storage.po_number') INTO newPoNumber;
--
INSERT INTO fs09000000_mod_orders_storage.purchase_order (id, jsonb)
VALUES (public.uuid_generate_v4(),
jsonb_build_object('id', public.uuid_generate_v4(),
'reEncumber', true,
'workflowStatus', 'Open',
'poNumber', newPoNumber,
'vendor', vendor_id,
'orderType', 'One-Time',
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
))
RETURNING id INTO order_id;
PERFORM public.generate_polines(order_id, polines_per_order, newPoNumber);
end loop;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.generate_polines(order_id text, polines_per_order integer, ponumber integer) RETURNS VOID as
$$
DECLARE
polineNumber text;
BEGIN
for line_counter in 1..polines_per_order
loop
INSERT INTO fs09000000_mod_orders_storage.po_line (id, jsonb)
VALUES (public.uuid_generate_v4(),
-- add other fields to increase processing complexity
jsonb_build_object('id', public.uuid_generate_v4(),
'acquisitionMethod', 'df26d81b-9d63-4ff8-bf41-49bf75cfa70e',
'rush', false,
'cost', json_build_object(
'currency', 'USD',
'discountType', 'percentage',
'listUnitPrice', 1,
'quantityPhysical', 1,
'poLineEstimatedPrice', 1
),
'alerts', json_build_array(),
'source', 'User',
'physical', jsonb_build_object('createInventory', 'None'),
'isPackage', false,
'orderFormat', 'Physical Resource',
'vendorDetail', jsonb_build_object('vendorAccount', '548'),---------------------------------
'titleOrPackage', 'ABA Journal',
'automaticExport', true,
'publicationDate', '1915-1983',
'purchaseOrderId', order_id,
'poLineNumber', concat(ponumber, '-', line_counter),
'claims', json_build_array(),
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
));
end loop;
END
$$ LANGUAGE plpgsql;
-- CREATE sample data
-- 1 - amount of organizations to be created
-- 2 - amount of orders per organization
-- 3 - amount of polines per order
select public.generate_data_for_edifact_export(1, 1000, 1);
--check
--SELECT count() FROM fs09000000_mod_organizations_storage.organizations WHERE jsonb ->> 'code' LIKE 'PERF_TEST_ORG%';
--SELECT count(*) FROM fs09000000_mod_orders_storage.purchase_order;
CREATE OR REPLACE FUNCTION public.generate_data_for_edifact_export(organizations_amount integer,
orders_per_vendor integer,
polines_per_order integer) RETURNS VOID as
$$
DECLARE
-- !!! SET DEFAULT TENANT NAME !!!
orgName text DEFAULT 'perf_test_vendor';
orgCode TEXT default 'PERF_TEST_ORG';
vendor_id TEXT;
BEGIN
for org_counter in 1..organizations_amount
loop
/* INSERT INTO fs09000000_mod_organizations_storage.organizations (id, jsonb)
VALUES (public.uuid_generate_v4(),
jsonb_build_object('code', concat(orgCode, org_counter),
'erpCode', '12345',
'isVendor', true,
'name', concat(orgName, org_counter),
'status', 'Active',
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
))
RETURNING id INTO vendor_id;*/
PERFORM public.generate_orders(orders_per_vendor, polines_per_order, '1a36d83e-526f-48ad-9956-4e341a40fbbb');-------------------------------
end loop;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.generate_orders(orders_per_vendor integer, polines_per_order integer, vendor_id text) RETURNS VOID as
$$
DECLARE
order_id text;
newPoNumber integer;
BEGIN
for order_counter in 1..orders_per_vendor
loop
SELECT nextval('fs09000000_mod_orders_storage.po_number') INTO newPoNumber;
--
INSERT INTO fs09000000_mod_orders_storage.purchase_order (id, jsonb)
VALUES (public.uuid_generate_v4(),
jsonb_build_object('id', public.uuid_generate_v4(),
'reEncumber', true,
'workflowStatus', 'Open',
'poNumber', newPoNumber,
'vendor', vendor_id,
'orderType', 'One-Time',
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
))
RETURNING id INTO order_id;
PERFORM public.generate_polines(order_id, polines_per_order, newPoNumber);
end loop;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.generate_polines(order_id text, polines_per_order integer, ponumber integer) RETURNS VOID as
$$
DECLARE
polineNumber text;
BEGIN
for line_counter in 1..polines_per_order
loop
INSERT INTO fs09000000_mod_orders_storage.po_line (id, jsonb)
VALUES (public.uuid_generate_v4(),
-- add other fields to increase processing complexity
jsonb_build_object('id', public.uuid_generate_v4(),
'acquisitionMethod', 'df26d81b-9d63-4ff8-bf41-49bf75cfa70e',
'rush', false,
'cost', json_build_object(
'currency', 'USD',
'discountType', 'percentage',
'listUnitPrice', 1,
'quantityPhysical', 1,
'poLineEstimatedPrice', 1
),
'alerts', json_build_array(),
'source', 'User',
'physical', jsonb_build_object('createInventory', 'None'),
'isPackage', false,
'orderFormat', 'Physical Resource',
'vendorDetail', jsonb_build_object('vendorAccount', '144295-1'),---------------------------------
'titleOrPackage', 'ABA Journal',
'automaticExport', true,
'publicationDate', '1915-1983',
'purchaseOrderId', order_id,
'poLineNumber', concat(ponumber, '-', line_counter),
'claims', json_build_array(),
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
));
end loop;
END
$$ LANGUAGE plpgsql;
-- CREATE sample data
-- 1 - amount of organizations to be created
-- 2 - amount of orders per organization
-- 3 - amount of polines per order
select public.generate_data_for_edifact_export(1, 1000, 1);
--check
--SELECT count() FROM fs09000000_mod_organizations_storage.organizations WHERE jsonb ->> 'code' LIKE 'PERF_TEST_ORG%';
--SELECT count(*) FROM fs09000000_mod_orders_storage.purchase_order;
CREATE OR REPLACE FUNCTION public.generate_data_for_edifact_export(organizations_amount integer,
orders_per_vendor integer,
polines_per_order integer) RETURNS VOID as
$$
DECLARE
-- !!! SET DEFAULT TENANT NAME !!!
orgName text DEFAULT 'perf_test_vendor';
orgCode TEXT default 'PERF_TEST_ORG';
vendor_id TEXT;
BEGIN
for org_counter in 1..organizations_amount
loop
/* INSERT INTO fs09000000_mod_organizations_storage.organizations (id, jsonb)
VALUES (public.uuid_generate_v4(),
jsonb_build_object('code', concat(orgCode, org_counter),
'erpCode', '12345',
'isVendor', true,
'name', concat(orgName, org_counter),
'status', 'Active',
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
))
RETURNING id INTO vendor_id;*/
PERFORM public.generate_orders(orders_per_vendor, polines_per_order, '6d363fea-c0e3-4c32-8074-4124b0a2307e');-------------------------------
end loop;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.generate_orders(orders_per_vendor integer, polines_per_order integer, vendor_id text) RETURNS VOID as
$$
DECLARE
order_id text;
newPoNumber integer;
BEGIN
for order_counter in 1..orders_per_vendor
loop
SELECT nextval('fs09000000_mod_orders_storage.po_number') INTO newPoNumber;
--
INSERT INTO fs09000000_mod_orders_storage.purchase_order (id, jsonb)
VALUES (public.uuid_generate_v4(),
jsonb_build_object('id', public.uuid_generate_v4(),
'reEncumber', true,
'workflowStatus', 'Open',
'poNumber', newPoNumber,
'vendor', vendor_id,
'orderType', 'One-Time',
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
))
RETURNING id INTO order_id;
PERFORM public.generate_polines(order_id, polines_per_order, newPoNumber);
end loop;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.generate_polines(order_id text, polines_per_order integer, ponumber integer) RETURNS VOID as
$$
DECLARE
polineNumber text;
BEGIN
for line_counter in 1..polines_per_order
loop
INSERT INTO fs09000000_mod_orders_storage.po_line (id, jsonb)
VALUES (public.uuid_generate_v4(),
-- add other fields to increase processing complexity
jsonb_build_object('id', public.uuid_generate_v4(),
'acquisitionMethod', 'df26d81b-9d63-4ff8-bf41-49bf75cfa70e',
'rush', false,
'cost', json_build_object(
'currency', 'USD',
'discountType', 'percentage',
'listUnitPrice', 1,
'quantityPhysical', 1,
'poLineEstimatedPrice', 1
),
'alerts', json_build_array(),
'source', 'User',
'physical', jsonb_build_object('createInventory', 'None'),
'isPackage', false,
'orderFormat', 'Physical Resource',
'vendorDetail', jsonb_build_object('vendorAccount', '144295-2'),---------------------------------
'titleOrPackage', 'ABA Journal',
'automaticExport', true,
'publicationDate', '1915-1983',
'purchaseOrderId', order_id,
'poLineNumber', concat(ponumber, '-', line_counter),
'claims', json_build_array(),
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
));
end loop;
END
$$ LANGUAGE plpgsql;
-- CREATE sample data
-- 1 - amount of organizations to be created
-- 2 - amount of orders per organization
-- 3 - amount of polines per order
select public.generate_data_for_edifact_export(1, 1000, 1);
--check
--SELECT count() FROM fs09000000_mod_organizations_storage.organizations WHERE jsonb ->> 'code' LIKE 'PERF_TEST_ORG%';
--SELECT count(*) FROM fs09000000_mod_orders_storage.purchase_order;
CREATE OR REPLACE FUNCTION public.generate_data_for_edifact_export(organizations_amount integer,
orders_per_vendor integer,
polines_per_order integer) RETURNS VOID as
$$
DECLARE
-- !!! SET DEFAULT TENANT NAME !!!
orgName text DEFAULT 'perf_test_vendor';
orgCode TEXT default 'PERF_TEST_ORG';
vendor_id TEXT;
BEGIN
for org_counter in 1..organizations_amount
loop
/* INSERT INTO fs09000000_mod_organizations_storage.organizations (id, jsonb)
VALUES (public.uuid_generate_v4(),
jsonb_build_object('code', concat(orgCode, org_counter),
'erpCode', '12345',
'isVendor', true,
'name', concat(orgName, org_counter),
'status', 'Active',
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
))
RETURNING id INTO vendor_id;*/
PERFORM public.generate_orders(orders_per_vendor, polines_per_order, '382831f9-c680-4b7e-a3ab-daa3022fa4cb');-------------------------------
end loop;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.generate_orders(orders_per_vendor integer, polines_per_order integer, vendor_id text) RETURNS VOID as
$$
DECLARE
order_id text;
newPoNumber integer;
BEGIN
for order_counter in 1..orders_per_vendor
loop
SELECT nextval('fs09000000_mod_orders_storage.po_number') INTO newPoNumber;
--
INSERT INTO fs09000000_mod_orders_storage.purchase_order (id, jsonb)
VALUES (public.uuid_generate_v4(),
jsonb_build_object('id', public.uuid_generate_v4(),
'reEncumber', true,
'workflowStatus', 'Open',
'poNumber', newPoNumber,
'vendor', vendor_id,
'orderType', 'One-Time',
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
))
RETURNING id INTO order_id;
PERFORM public.generate_polines(order_id, polines_per_order, newPoNumber);
end loop;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.generate_polines(order_id text, polines_per_order integer, ponumber integer) RETURNS VOID as
$$
DECLARE
polineNumber text;
BEGIN
for line_counter in 1..polines_per_order
loop
INSERT INTO fs09000000_mod_orders_storage.po_line (id, jsonb)
VALUES (public.uuid_generate_v4(),
-- add other fields to increase processing complexity
jsonb_build_object('id', public.uuid_generate_v4(),
'acquisitionMethod', 'df26d81b-9d63-4ff8-bf41-49bf75cfa70e',
'rush', false,
'cost', json_build_object(
'currency', 'USD',
'discountType', 'percentage',
'listUnitPrice', 1,
'quantityPhysical', 1,
'poLineEstimatedPrice', 1
),
'alerts', json_build_array(),
'source', 'User',
'physical', jsonb_build_object('createInventory', 'None'),
'isPackage', false,
'orderFormat', 'Physical Resource',
'vendorDetail', jsonb_build_object('vendorAccount', '300693'),---------------------------------
'titleOrPackage', 'ABA Journal',
'automaticExport', true,
'publicationDate', '1915-1983',
'purchaseOrderId', order_id,
'poLineNumber', concat(ponumber, '-', line_counter),
'claims', json_build_array(),
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
));
end loop;
END
$$ LANGUAGE plpgsql;
-- CREATE sample data
-- 1 - amount of organizations to be created
-- 2 - amount of orders per organization
-- 3 - amount of polines per order
select public.generate_data_for_edifact_export(1, 1000, 1);
--check
--SELECT count() FROM fs09000000_mod_organizations_storage.organizations WHERE jsonb ->> 'code' LIKE 'PERF_TEST_ORG%';
--SELECT count(*) FROM fs09000000_mod_orders_storage.purchase_order;
CREATE OR REPLACE FUNCTION public.generate_data_for_edifact_export(organizations_amount integer,
orders_per_vendor integer,
polines_per_order integer) RETURNS VOID as
$$
DECLARE
-- !!! SET DEFAULT TENANT NAME !!!
orgName text DEFAULT 'perf_test_vendor';
orgCode TEXT default 'PERF_TEST_ORG';
vendor_id TEXT;
BEGIN
for org_counter in 1..organizations_amount
loop
/* INSERT INTO fs09000000_mod_organizations_storage.organizations (id, jsonb)
VALUES (public.uuid_generate_v4(),
jsonb_build_object('code', concat(orgCode, org_counter),
'erpCode', '12345',
'isVendor', true,
'name', concat(orgName, org_counter),
'status', 'Active',
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
))
RETURNING id INTO vendor_id;*/
PERFORM public.generate_orders(orders_per_vendor, polines_per_order, 'da29ee41-727a-4372-9472-7818c948f8c7');-------------------------------
end loop;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.generate_orders(orders_per_vendor integer, polines_per_order integer, vendor_id text) RETURNS VOID as
$$
DECLARE
order_id text;
newPoNumber integer;
BEGIN
for order_counter in 1..orders_per_vendor
loop
SELECT nextval('fs09000000_mod_orders_storage.po_number') INTO newPoNumber;
--
INSERT INTO fs09000000_mod_orders_storage.purchase_order (id, jsonb)
VALUES (public.uuid_generate_v4(),
jsonb_build_object('id', public.uuid_generate_v4(),
'reEncumber', true,
'workflowStatus', 'Open',
'poNumber', newPoNumber,
'vendor', vendor_id,
'orderType', 'One-Time',
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
))
RETURNING id INTO order_id;
PERFORM public.generate_polines(order_id, polines_per_order, newPoNumber);
end loop;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.generate_polines(order_id text, polines_per_order integer, ponumber integer) RETURNS VOID as
$$
DECLARE
polineNumber text;
BEGIN
for line_counter in 1..polines_per_order
loop
INSERT INTO fs09000000_mod_orders_storage.po_line (id, jsonb)
VALUES (public.uuid_generate_v4(),
-- add other fields to increase processing complexity
jsonb_build_object('id', public.uuid_generate_v4(),
'acquisitionMethod', 'df26d81b-9d63-4ff8-bf41-49bf75cfa70e',
'rush', false,
'cost', json_build_object(
'currency', 'USD',
'discountType', 'percentage',
'listUnitPrice', 1,
'quantityPhysical', 1,
'poLineEstimatedPrice', 1
),
'alerts', json_build_array(),
'source', 'User',
'physical', jsonb_build_object('createInventory', 'None'),
'isPackage', false,
'orderFormat', 'Physical Resource',
'vendorDetail', jsonb_build_object('vendorAccount', 'libraryorders@library.tam'),---------------------------------
'titleOrPackage', 'ABA Journal',
'automaticExport', true,
'publicationDate', '1915-1983',
'purchaseOrderId', order_id,
'poLineNumber', concat(ponumber, '-', line_counter),
'claims', json_build_array(),
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
));
end loop;
END
$$ LANGUAGE plpgsql;
-- CREATE sample data
-- 1 - amount of organizations to be created
-- 2 - amount of orders per organization
-- 3 - amount of polines per order
select public.generate_data_for_edifact_export(1, 1000, 1);
--check
--SELECT count() FROM fs09000000_mod_organizations_storage.organizations WHERE jsonb ->> 'code' LIKE 'PERF_TEST_ORG%';
--SELECT count(*) FROM fs09000000_mod_orders_storage.purchase_order;
CREATE OR REPLACE FUNCTION public.generate_data_for_edifact_export(organizations_amount integer,
orders_per_vendor integer,
polines_per_order integer) RETURNS VOID as
$$
DECLARE
-- !!! SET DEFAULT TENANT NAME !!!
orgName text DEFAULT 'perf_test_vendor';
orgCode TEXT default 'PERF_TEST_ORG';
vendor_id TEXT;
BEGIN
for org_counter in 1..organizations_amount
loop
/* INSERT INTO fs09000000_mod_organizations_storage.organizations (id, jsonb)
VALUES (public.uuid_generate_v4(),
jsonb_build_object('code', concat(orgCode, org_counter),
'erpCode', '12345',
'isVendor', true,
'name', concat(orgName, org_counter),
'status', 'Active',
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
))
RETURNING id INTO vendor_id;*/
PERFORM public.generate_orders(orders_per_vendor, polines_per_order, '02c0820c-108c-41c4-ab9c-d289877e8dfa');-------------------------------
end loop;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.generate_orders(orders_per_vendor integer, polines_per_order integer, vendor_id text) RETURNS VOID as
$$
DECLARE
order_id text;
newPoNumber integer;
BEGIN
for order_counter in 1..orders_per_vendor
loop
SELECT nextval('fs09000000_mod_orders_storage.po_number') INTO newPoNumber;
--
INSERT INTO fs09000000_mod_orders_storage.purchase_order (id, jsonb)
VALUES (public.uuid_generate_v4(),
jsonb_build_object('id', public.uuid_generate_v4(),
'reEncumber', true,
'workflowStatus', 'Open',
'poNumber', newPoNumber,
'vendor', vendor_id,
'orderType', 'One-Time',
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
))
RETURNING id INTO order_id;
PERFORM public.generate_polines(order_id, polines_per_order, newPoNumber);
end loop;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.generate_polines(order_id text, polines_per_order integer, ponumber integer) RETURNS VOID as
$$
DECLARE
polineNumber text;
BEGIN
for line_counter in 1..polines_per_order
loop
INSERT INTO fs09000000_mod_orders_storage.po_line (id, jsonb)
VALUES (public.uuid_generate_v4(),
-- add other fields to increase processing complexity
jsonb_build_object('id', public.uuid_generate_v4(),
'acquisitionMethod', 'df26d81b-9d63-4ff8-bf41-49bf75cfa70e',
'rush', false,
'cost', json_build_object(
'currency', 'USD',
'discountType', 'percentage',
'listUnitPrice', 1,
'quantityPhysical', 1,
'poLineEstimatedPrice', 1
),
'alerts', json_build_array(),
'source', 'User',
'physical', jsonb_build_object('createInventory', 'None'),
'isPackage', false,
'orderFormat', 'Physical Resource',
'vendorDetail', jsonb_build_object('vendorAccount', '123'),---------------------------------
'titleOrPackage', 'ABA Journal',
'automaticExport', true,
'publicationDate', '1915-1983',
'purchaseOrderId', order_id,
'poLineNumber', concat(ponumber, '-', line_counter),
'claims', json_build_array(),
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
));
end loop;
END
$$ LANGUAGE plpgsql;
-- CREATE sample data
-- 1 - amount of organizations to be created
-- 2 - amount of orders per organization
-- 3 - amount of polines per order
select public.generate_data_for_edifact_export(1, 1000, 1);
--check
--SELECT count() FROM fs09000000_mod_organizations_storage.organizations WHERE jsonb ->> 'code' LIKE 'PERF_TEST_ORG%';
--SELECT count(*) FROM fs09000000_mod_orders_storage.purchase_order;
CREATE OR REPLACE FUNCTION public.generate_data_for_edifact_export(organizations_amount integer,
orders_per_vendor integer,
polines_per_order integer) RETURNS VOID as
$$
DECLARE
-- !!! SET DEFAULT TENANT NAME !!!
orgName text DEFAULT 'perf_test_vendor';
orgCode TEXT default 'PERF_TEST_ORG';
vendor_id TEXT;
BEGIN
for org_counter in 1..organizations_amount
loop
/* INSERT INTO fs09000000_mod_organizations_storage.organizations (id, jsonb)
VALUES (public.uuid_generate_v4(),
jsonb_build_object('code', concat(orgCode, org_counter),
'erpCode', '12345',
'isVendor', true,
'name', concat(orgName, org_counter),
'status', 'Active',
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
))
RETURNING id INTO vendor_id;*/
PERFORM public.generate_orders(orders_per_vendor, polines_per_order, '2d6f2d47-9664-4794-8cd5-4e168bd57384');-------------------------------
end loop;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.generate_orders(orders_per_vendor integer, polines_per_order integer, vendor_id text) RETURNS VOID as
$$
DECLARE
order_id text;
newPoNumber integer;
BEGIN
for order_counter in 1..orders_per_vendor
loop
SELECT nextval('fs09000000_mod_orders_storage.po_number') INTO newPoNumber;
--
INSERT INTO fs09000000_mod_orders_storage.purchase_order (id, jsonb)
VALUES (public.uuid_generate_v4(),
jsonb_build_object('id', public.uuid_generate_v4(),
'reEncumber', true,
'workflowStatus', 'Open',
'poNumber', newPoNumber,
'vendor', vendor_id,
'orderType', 'One-Time',
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
))
RETURNING id INTO order_id;
PERFORM public.generate_polines(order_id, polines_per_order, newPoNumber);
end loop;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.generate_polines(order_id text, polines_per_order integer, ponumber integer) RETURNS VOID as
$$
DECLARE
polineNumber text;
BEGIN
for line_counter in 1..polines_per_order
loop
INSERT INTO fs09000000_mod_orders_storage.po_line (id, jsonb)
VALUES (public.uuid_generate_v4(),
-- add other fields to increase processing complexity
jsonb_build_object('id', public.uuid_generate_v4(),
'acquisitionMethod', 'df26d81b-9d63-4ff8-bf41-49bf75cfa70e',
'rush', false,
'cost', json_build_object(
'currency', 'USD',
'discountType', 'percentage',
'listUnitPrice', 1,
'quantityPhysical', 1,
'poLineEstimatedPrice', 1
),
'alerts', json_build_array(),
'source', 'User',
'physical', jsonb_build_object('createInventory', 'None'),
'isPackage', false,
'orderFormat', 'Physical Resource',
'vendorDetail', jsonb_build_object('vendorAccount', '111111'),---------------------------------
'titleOrPackage', 'ABA Journal',
'automaticExport', true,
'publicationDate', '1915-1983',
'purchaseOrderId', order_id,
'poLineNumber', concat(ponumber, '-', line_counter),
'claims', json_build_array(),
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
));
end loop;
END
$$ LANGUAGE plpgsql;
-- CREATE sample data
-- 1 - amount of organizations to be created
-- 2 - amount of orders per organization
-- 3 - amount of polines per order
select public.generate_data_for_edifact_export(1, 1000, 1);
--check
--SELECT count() FROM fs09000000_mod_organizations_storage.organizations WHERE jsonb ->> 'code' LIKE 'PERF_TEST_ORG%';
--SELECT count(*) FROM fs09000000_mod_orders_storage.purchase_order;
CREATE OR REPLACE FUNCTION public.generate_data_for_edifact_export(organizations_amount integer,
orders_per_vendor integer,
polines_per_order integer) RETURNS VOID as
$$
DECLARE
-- !!! SET DEFAULT TENANT NAME !!!
orgName text DEFAULT 'perf_test_vendor';
orgCode TEXT default 'PERF_TEST_ORG';
vendor_id TEXT;
BEGIN
for org_counter in 1..organizations_amount
loop
/* INSERT INTO fs09000000_mod_organizations_storage.organizations (id, jsonb)
VALUES (public.uuid_generate_v4(),
jsonb_build_object('code', concat(orgCode, org_counter),
'erpCode', '12345',
'isVendor', true,
'name', concat(orgName, org_counter),
'status', 'Active',
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
))
RETURNING id INTO vendor_id;*/
PERFORM public.generate_orders(orders_per_vendor, polines_per_order, 'a58a5dcc-7e60-492f-b795-7a791c0970fe');-------------------------------
end loop;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.generate_orders(orders_per_vendor integer, polines_per_order integer, vendor_id text) RETURNS VOID as
$$
DECLARE
order_id text;
newPoNumber integer;
BEGIN
for order_counter in 1..orders_per_vendor
loop
SELECT nextval('fs09000000_mod_orders_storage.po_number') INTO newPoNumber;
--
INSERT INTO fs09000000_mod_orders_storage.purchase_order (id, jsonb)
VALUES (public.uuid_generate_v4(),
jsonb_build_object('id', public.uuid_generate_v4(),
'reEncumber', true,
'workflowStatus', 'Open',
'poNumber', newPoNumber,
'vendor', vendor_id,
'orderType', 'One-Time',
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
))
RETURNING id INTO order_id;
PERFORM public.generate_polines(order_id, polines_per_order, newPoNumber);
end loop;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.generate_polines(order_id text, polines_per_order integer, ponumber integer) RETURNS VOID as
$$
DECLARE
polineNumber text;
BEGIN
for line_counter in 1..polines_per_order
loop
INSERT INTO fs09000000_mod_orders_storage.po_line (id, jsonb)
VALUES (public.uuid_generate_v4(),
-- add other fields to increase processing complexity
jsonb_build_object('id', public.uuid_generate_v4(),
'acquisitionMethod', 'df26d81b-9d63-4ff8-bf41-49bf75cfa70e',
'rush', false,
'cost', json_build_object(
'currency', 'USD',
'discountType', 'percentage',
'listUnitPrice', 1,
'quantityPhysical', 1,
'poLineEstimatedPrice', 1
),
'alerts', json_build_array(),
'source', 'User',
'physical', jsonb_build_object('createInventory', 'None'),
'isPackage', false,
'orderFormat', 'Physical Resource',
'vendorDetail', jsonb_build_object('vendorAccount', '22222'),---------------------------------
'titleOrPackage', 'ABA Journal',
'automaticExport', true,
'publicationDate', '1915-1983',
'purchaseOrderId', order_id,
'poLineNumber', concat(ponumber, '-', line_counter),
'claims', json_build_array(),
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
));
end loop;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.generate_data_for_edifact_export(organizations_amount integer,
orders_per_vendor integer,
polines_per_order integer) RETURNS VOID as
$$
DECLARE
-- !!! SET DEFAULT TENANT NAME !!!
orgName text DEFAULT 'perf_test_vendor';
orgCode TEXT default 'PERF_TEST_ORG';
vendor_id TEXT;
BEGIN
for org_counter in 1..organizations_amount
loop
/* INSERT INTO fs09000000_mod_organizations_storage.organizations (id, jsonb)
VALUES (public.uuid_generate_v4(),
jsonb_build_object('code', concat(orgCode, org_counter),
'erpCode', '12345',
'isVendor', true,
'name', concat(orgName, org_counter),
'status', 'Active',
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
))
RETURNING id INTO vendor_id;*/
PERFORM public.generate_orders(orders_per_vendor, polines_per_order, 'a58a5dcc-7e60-492f-b795-7a791c0970fe');-------------------------------
end loop;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.generate_orders(orders_per_vendor integer, polines_per_order integer, vendor_id text) RETURNS VOID as
$$
DECLARE
order_id text;
newPoNumber integer;
BEGIN
for order_counter in 1..orders_per_vendor
loop
SELECT nextval('fs09000000_mod_orders_storage.po_number') INTO newPoNumber;
--
INSERT INTO fs09000000_mod_orders_storage.purchase_order (id, jsonb)
VALUES (public.uuid_generate_v4(),
jsonb_build_object('id', public.uuid_generate_v4(),
'reEncumber', true,
'workflowStatus', 'Open',
'poNumber', newPoNumber,
'vendor', vendor_id,
'orderType', 'One-Time',
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
))
RETURNING id INTO order_id;
PERFORM public.generate_polines(order_id, polines_per_order, newPoNumber);
end loop;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.generate_polines(order_id text, polines_per_order integer, ponumber integer) RETURNS VOID as
$$
DECLARE
polineNumber text;
BEGIN
for line_counter in 1..polines_per_order
loop
INSERT INTO fs09000000_mod_orders_storage.po_line (id, jsonb)
VALUES (public.uuid_generate_v4(),
-- add other fields to increase processing complexity
jsonb_build_object('id', public.uuid_generate_v4(),
'acquisitionMethod', 'df26d81b-9d63-4ff8-bf41-49bf75cfa70e',
'rush', false,
'cost', json_build_object(
'currency', 'USD',
'discountType', 'percentage',
'listUnitPrice', 1,
'quantityPhysical', 1,
'poLineEstimatedPrice', 1
),
'alerts', json_build_array(),
'source', 'User',
'physical', jsonb_build_object('createInventory', 'None'),
'isPackage', false,
'orderFormat', 'Physical Resource',
'vendorDetail', jsonb_build_object('vendorAccount', '22222'),---------------------------------
'titleOrPackage', 'ABA Journal',
'automaticExport', true,
'publicationDate', '1915-1983',
'purchaseOrderId', order_id,
'poLineNumber', concat(ponumber, '-', line_counter),
'claims', json_build_array(),
'metadata', jsonb_build_object(
'createdDate', '2018-07-19T00:00:00.000+0000',
'createdByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1',
'updatedDate', '2018-07-19T00:00:00.000+0000',
'updatedByUserId', '28d1057c-d137-11e8-a8d5-f2801f1b9fd1'
)
));
end loop;
END
$$ LANGUAGE plpgsql;
-- CREATE sample data
-- 1 - amount of organizations to be created
-- 2 - amount of orders per organization
-- 3 - amount of polines per order
select public.generate_data_for_edifact_export(1, 1000, 1);
--check
--SELECT count() FROM fs09000000_mod_organizations_storage.organizations WHERE jsonb ->> 'code' LIKE 'PERF_TEST_ORG%';
--SELECT count(*) FROM fs09000000_mod_orders_storage.purchase_order;
SELECT jsonb->>'vendor', count(jsonb->>'vendor') FROM fs09000000_mod_orders_storage.purchase_order group by jsonb->>'vendor' limit 10
Script above consist of 10 parts (each part corresponding to each organization).
by changing next values we can manipulate with numbers of orders and PO lines will be created to each organization -- 1 - amount of organizations to be created
-- 2 - amount of orders per organization
-- 3 - amount of polines per order
select public.generate_data_for_edifact_export(1, 1000, 1); (in this particular case script will create 1000 orders with 1 PO line each)
to check if data were created successfully use query:
SELECT jsonb->>'vendor', count(jsonb->>'vendor') FROM fs09000000_mod_orders_storage.purchase_order where creation_date>'2022-08-30' group by jsonb->>'vendor' limit 10
Set scheduler
Using UI on each organization integration set up proper scheduler Actions→Edit
Set up scheduler on the same time to have job concurrency.
Testing scenario
Test #
Concurrent jobs
Orders per job
1
10
500
2
10
1000
3
10
2000
4
10
3000
After scheduler trigger the export job (or jobs) they will appear on export manager app on UI.
Bursar
Bursar export is the export of particular user fees/fines.
Data preparation
Check out several available items for particular user. According to policy wait until loan will have fees/fines.
Or create fee/fine for user using user app
test run
go on apps→settings→users
in fee/fine section go on "transfer criteria" tab
choose appropriate fee/fine owner (in PTF case it's test2)
You can either set a scheduler or run bursar export manually
After test is tarted it'll appear on data export manager page
Circulation log export is part of data-export manager workflow to export all circulation activities for specified time
data preparation
There is no special data preparation is needed. However in order to manipulate with the numbers should be exported we're using "date" filter in combination with other filters to get needed number to be exported .
On the screenshot you can see UI. By clicking "Export results (CSV)" export job going to appear on "data export manager" UI
Approximate number of logs to be exported is 60K.
EHoldings
EHoldings is part of data-export manager workflow to export titles of particular provider.
Each title can be exported numerous of times concurrently.
To start EHoldings - go on eholdings app, search for provider → go on provider. Find appropriate package and click on it.
Each package contains some number of titles.
To start export click "actions"→"export package CSV"
after export start - export job will appear on export-manager page.
Example packages to use
Note: each package should have at least 15 long notes.
TRUNCATE TABLE [tenant]_mod_patron_blocks.user_summary;
TRUNCATE TABLE [tenant]_mod_circulation_storage.loan;
TRUNCATE TABLE [tenant]_mod_circulation_storage.audit_loan;
TRUNCATE TABLE [tenant]_mod_circulation_storage.request;
TRUNCATE TABLE [tenant]_mod_circulation_storage.patron_action_session;
TRUNCATE TABLE [tenant]_mod_circulation_storage.scheduled_notice;
TRUNCATE TABLE [tenant]_mod_notify.notify_data;
UPDATE [tenant]_mod_inventory_storage.item SET jsonb = jsonb_set(jsonb, '{status, name}', '"Available"') WHERE jsonb->'status'->>'name' != 'Available';
UPDATE [tenant]_mod_inventory_storage.item SET jsonb = jsonb_set(jsonb, '{permanentLocationId}', '"08863ef2-508d-491e-9603-b6e1272f6855"') where jsonb->'permanentLocationId' != '"08863ef2-508d-491e-9603-b6e1272f6855"' and jsonb->'barcode' is not null and jsonb->'barcode' >= '"000"' and jsonb->'barcode'<= '"090155243"';
UPDATE [tenant]_mod_inventory_storage.item SET jsonb = jsonb_set(jsonb, '{temporaryLocationId}', '"6216269b-9c9e-4129-adc5-ca9397137edc"') where jsonb->'temporaryLocationId' is null and jsonb->'barcode' is not null and jsonb->'barcode' >= '"000"' and jsonb->'barcode'<= '"090155243"';
UPDATE [tenant]_mod_inventory_storage.item SET jsonb = jsonb_set(jsonb, '{permanentLoanTypeId}', '"ac19815e-1d8e-473f-bd5a-3193cb301b8b"') where jsonb->'permanentLoanTypeId' != '"ac19815e-1d8e-473f-bd5a-3193cb301b8b"' and jsonb->'barcode' is not null and jsonb->'barcode' >= '"000"' and jsonb->'barcode'<= '"090155243"';
UPDATE [tenant]_mod_inventory_storage.item SET jsonb = jsonb_set(jsonb, '{temporaryLoanTypeId}', '"23e4f1ec-cf31-4098-959e-de64ce4781ce"') where jsonb->'temporaryLoanTypeId' is null and jsonb->'barcode' is not null and jsonb->'barcode' >= '"000"' and jsonb->'barcode'<= '"090155243"';
Test run
For each new release check the test script, and change it to be up to date. To check if all is up to date you can perform Bulk edit of 1 record from UI and record it with the BlazeMeter Chrome plugin or Google Chrome Developer Tools, etc. and compare API calls.
Upload a new version to the Carrier.io export bucket. (For all tests test name and test plan name should be equal)
Enable thread groups with the proper number of records(100, 1k, 5k, 10k, 25k, 50k, and 100k) in the script, and ensure that all threads will be run one at a time.
select (end_time - start_time)as job_duration,matched_num_of_records,entity_type,* from [tenant]_mod_bulk_operations.bulk_operation order by start_time desc limit 1000
It is only 1 job ID for Items Bulk Editing for both parts: finding matched records and updating (job type BULK_EDIT_IDENTIFIERS).
Users (Orchid)
Preparation for the test
Populate database
UPDATE [tenant]_mod_users.users SET jsonb = jsonb_set(jsonb, '{personal,email}', '"[email].com"') where jsonb->'barcode' is not null and jsonb->'personal'->>'email' = '[email].org';
UPDATE [tenant]_mod_users.users SET jsonb = jsonb - 'expirationDate' where jsonb->'barcode' is not null and jsonb->'expirationDate' is not null;
UPDATE [tenant]_mod_users.users SET jsonb = jsonb_set(jsonb, '{patronGroup}', '"5fc96cbd-a860-42a7-8d2b-72af30206712"') where jsonb->'barcode' is not null and jsonb->'patronGroup' = '"294db32c-0675-4dd5-8c5f-e3974c4ab6f2"';
Test run
For each new release check the test script, and change it to be up to date, upload a new version to the Carrier.io export bucket. (For all tests test name and test plan name should be equal)
Enable thread groups with the proper number of records(100, 1k, 2.5k, 5k, 10k) in the script, and ensure that all threads will be run one at a time.
select (end_time - start_time)as job_duration,matched_num_of_records,entity_type,* from [tenant]_mod_bulk_operations.bulk_operation order by start_time desc limit 1000
It is only 1 job ID for Users Bulk Editing for both parts: finding matched records and updating (job type BULK_EDIT_IDENTIFIERS).
Holdings (Orchid)
Preparation for the test
Populate database
UPDATE [tenant]_mod_inventory_storage.holdings_record SET jsonb = jsonb_set(jsonb, '{permanentLocationId}', '"fac5de34-26ee-456d-86b1-f04fdf680d65"') WHERE jsonb->'hrid'>='"ho87055"' and jsonb->'hrid'<='"ho9999999"' and jsonb->'permanentLocationId'!='"fac5de34-26ee-456d-86b1-f04fdf680d65"';
UPDATE [tenant]_mod_inventory_storage.holdings_record SET jsonb = jsonb_set(jsonb, '{temporaryLocationId}', '"2b8f7d63-706a-4b56-8a5e-50ad24e33e4c"') WHERE jsonb->'hrid'>='"ho87055"' and jsonb->'hrid'<='"ho9999999"' and jsonb->'temporaryLocationId' is null;
Test run
For each new release check the test script, and change it to be up to date, upload a new version to the Carrier.io export bucket. (For all tests test name and test plan name should be equal)
Enable thread groups with the proper number of records(100, 1k, 5k, 10k, 100k) in the script, and ensure that all threads will be run one at a time.
select (end_time - start_time)as job_duration,matched_num_of_records,entity_type,* from [tenant]_mod_bulk_operations.bulk_operation order by start_time desc limit 1000
It is only 1 job ID for Holdings Bulk Editing for both parts: finding matched records and updating (job type BULK_EDIT_IDENTIFIERS).
OAI-PMH
To run oai-pmh test use script oai-pmh-test git repo.
Preparation for testing
In most of a times oai-pmh test doesn’t require additional preparations. However, usually after (or before) test database cleaning required.
to delete previous oai-pmh request run truncate table
These calls were performed repeatedly, harvesting 100 records each time until there is no more data in [tenant]_mod_oai_pmh.instances table to harvest.
[resumptionToken] was set to 100, returning in initial call response and in each harvesting call until there is no more records to harvest. When all data has being harvested - resumptionToken will not return with the response.
Additional information
Test will run until there is instances in DB to harvest (test designed in a way to run infinitely until first error. first error, which is expected, will happen when there will be no resumption token in response. Whisch will mean end of a test)
This test (and oai-pmh itself) aimed on one process at the time so we'll use next test parameters.
param
value
VUsers
1
RampUp
0
Duration
Until completion
Reindex
To get interim or final results of index size during reindexing go to Open Search service and copy Domain Endpoint (VPC).
Go to carrio-io and paste it there as a command. So as a result we get such data like this table
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size green open mtes_instance_fs GFpif_XjTh-1PJTZac1_Hw 408196944595151.4gb 51.4gb green open mtes_contributor_fs yFhS12Y4SviL170UFQZZlw 404739266112659010.3gb 10.3gb green open mtes_instance_subject_fs pdD9z85SQZOW8uHMgMXvBw 403824725136815612.2gb 2.2gb green open .kibana_1 47dox0rZTD-8gUtV-xwM1w 116146.4kb 18.3kb
Copy/paste table by SHIFT+INSERT into excel sheet and edit it to use it in future. Go to Data, then Text to Column and press next to finish.
Fiscal year rollover
Fiscal year brief data structure
Data preparation
Before each fiscal year rollover data should be prepared.
To restore the data (100K orders: 50% open, 50% closed) backups of DB schemas were created.
Go to the /home/ec2-user/Finance directory on the host and download backup files for the database ("backOrders.sql" and "backFinance.sql").
Restore data in PgAdmin. Choose "[tenant]_mod_orders_storage" → "Restore" → choose file "backOrders.sql" → "Restore". Then do the same for "[tenant]_mod_finance_storage" (use "backFinance.sql" file here).
Restore process can fail. In this case you can use the same approach but restore each table sequentially. Select tables it in the same seqence as they are listed in point 1 above.
For other orders quantity and data structure JMeter script should be used (perf-testing/workflows-scripts/fiscal_year/FY_rollover_preparation on the GitHub). But in this case data generation will take more time.
FY rollover start
Go to "Finance" → "Ledger" → Check status "Active" → "ROLLOVERTEST" ledger → "Actions" → "Rollover"
Fill in the form like this:
3. Click "Test Rollover". Test rollover is a feature that should be also tested and duration is measured.
