Overview

This document contains the results of testing Data Import MARC Authorities in Orchid release to detect performance trends.

Ticket:

Previous test report: Data Import MARC Authorities (Nolana)

Summary

• There is no speciefic trend for DI performance and resources utilization for Orchid release compared to Nonala. There is degradation for smaller file jobs, while for bigger file jobs processing time improved (up to 19%).
• Max CPU utilization was observed for mod-inventory, it reached 43% for 50K DI import.
• Max DB CPU reached 86% during 50K DI import.

...

PTF -environment ncp5 

• 11 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1  

• 2 database  instances, one reader, and one writer

  Name	API Name	Memory GIB	vCPUs	max_connections
  R6G Extra Large	db.r6g.xlarge	32 GiB	4 vCPUs	2731

  
  

• number of connections for mod-source-record-manager and mod-source-record-storage - 30 connections.
• MSK ptf-kakfa-3
  
  • 4 m5.2xlarge brokers in 2 zones

  • Apache Kafka version 2.8.0

  • EBS storage volume per broker 300 GiB

  • auto.create.topics.enable=true
  • log.retention.minutes=480
  • default.replication.factor=3
• Kafka topics partitioning: - 2 partitions for DI topics

...

512

...