In-Progress

Overview

This document contains the results of testing Data Import in Kiwi and Juniper Hot Fix 3, and compare their results against each other to detect performance trends.

Software versions

Juniper

mod-data-import v2.1.4
mod-source-record-manager v3.1.3
mod-source-record-storage v5.1.6, started out with v5.1.6 but switched to v5.1.7 to match Juniper Bugfest
mod-inventory v17.0.6
mod-inventory-storage v21.0.9

(This set of Juniper modules requires the unique item barcode to be applied to the mod_inventory_storage.item barcode column because of MODINV-508 - Getting issue details... STATUS . Without it any Data Import job would not be able to proceed at all.)

MODSOURCE-422 Upgrade Vertx from 4.0.0 to 4.1.2 (Juniper HF5)

MODSOURCE-421 Add possibility to adjust parameter DB_MAXPOOLSIZE for configuring connection pool (Juniper HF5)

Results

Import durations are recorded in the following table. Note that we did a couple of imports and their durations are recorded here for completeness sake, separated by a comma.

Import Durations

Profiles used for PTF testing

	Profile	Juniper HF3	HF5 Vertex 4.1.2	HF5 Vertex 4.1.2 +DB_MAXPOOLSIZE (25) conf
5K MARC Create	PTF - Create 2	15, 17 min	15 min	11 min
5K MARC Update	PTF - Updates Success - 1	12 mins, FAILED	32 min	23 min
10K MARC Create	PTF - Create 2	28 min	26 min	24 min
10K MARC Update	PTF - Updates Success - 1	FAILED	1 hr	1 hr 14 min
20K MARC Create	PTF - Create 2	56 min		46 min
20K MARC Update	PTF - Updates Success - 1	DNR - slow		FAILED
25K MARC Create	PTF - Create 2		1 hr 7 min	1 hr 36 min
25K MARC Update	PTF - Updates Success - 1		2 hr 33 min	2 hr 24 min
50K MARC Create	PTF - Create 2		2 hr 7 min Failed	2 hr 3 min *
50K Update	PTF - Updates Success - 1			DNR

Observations Juniper HF3

mod-srs v5.1.6 tasks crashed every time doing an UPDATE import, which prompted us to use v5.1.7 to see if this problem was resolved.
UDPATEs on Juniper HF3 most of the time does not work. Getting various errors.
When

* Observations Juniper HF5 ( HF5 changes list)

50K Create is unstable on HF5 and may fail with 25 connections DB_MAXPOOLSIZE due to Error Connection is not active now, current status: CLOSED.

HF 5 DB_MAXPOOLSIZE experiments

notable observations:

More connections from SRS to DB means higher service memory usage;
Looks like DB_MAXPOOLSIZE parameter is the maximum connections per service (not task).

Test #	DB_MAXPOOLSIZE	duration
1	5	38
2	50	53
3	20	38
4	15	41

Overview

Software versions

Results

HF 5 DB_MAXPOOLSIZE experiments

Juniper resources usage

Browser not supported