Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

The following resources are used:

...

2 items for all items (except mod-data-import)

2 partition for all DI kafka topics:

Examples:

delete old topic
# ./kafka-topics.sh --bootstrap-server=<kafka-ip>:9092 --delete --topic perf-eks-folijet.Default.fs09000000.DI_ERROR

recreate topic with "--partitions 2 --replication-factor 1"
# ./kafka-topics.sh --bootstrap-server=<kafka-ip>:9092 --create --topic perf-eks-folijet.Default.fs09000000.DI_ERROR --partitions 2 --replication-factor 1
WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.
Created topic perf-eks-folijet.Default.fs09000000.DI_ERROR.

get topic info
# ./kafka-topics.sh --bootstrap-server=<kafka-ip>:9092 --describe --topic perf-eks-folijet.Default.fs09000000.DI_ERROR
Topic: perf-eks-folijet.Default.fs09000000.DI_ERROR PartitionCount: 2   ReplicationFactor: 1    Configs: min.insync.replicas=1,message.format.version=2.6-IV0,unclean.leader.election.enable=true
    Topic: perf-eks-folijet.Default.fs09000000.DI_ERROR Partition: 0    Leader: 1   Replicas: 1 Isr: 1
    Topic: perf-eks-folijet.Default.fs09000000.DI_ERROR Partition: 1    Leader: 2   Replicas: 2 Isr: 2 (edited) 


mod-data-import: -XX:MaxRAMPercentage=85.0 -XX:+UseG1GC / cpu: 128m/192m | memory: 1Gi/1Gi

...

Performance testing of update item scenario after removal of index (MODDATAIMP-697):

In the scope of FOLIO-3388 "item_status_name_idx_gin" index for the Item status field was removed from mod-inventory-storage. However, the this index potentially can be used for matching by Item status during Item update.

jobprofile structure

Test results:


records numbertimedescription
Testing with index50006 |40 |6 |6
Testing without index50006|
40
11|6|6


Example of the CQL query built during while processing the sub-match profile for matching by status: 

...

Code Block
languagesql
titlesdf
collapsetrue
SELECT id, jsonb, creation_date, created_by, holdingsrecordid, permanentloantypeid, temporaryloantypeid, materialtypeid, permanentlocationid, temporarylocationid, effectivelocationid
FROM fs09000000_mod_inventory_storage.item
WHERE (
	CASE WHEN length(lower(f_unaccent('Missing'))) <= 600 
		 THEN left(lower(f_unaccent(item.jsonb->'status'->>'name')),600) LIKE lower(f_unaccent('Missing')) 
		 ELSE left(lower(f_unaccent(item.jsonb->'status'->>'name')),600) LIKE left(lower(f_unaccent('Missing')),600) AND lower(f_unaccent(item.jsonb->'status'->>'name')) LIKE lower(f_unaccent('Missing')) 
	END
)
LIMIT 2 OFFSET 0

...