/
Data Import test report QCON (Quesnelia)[ECS]

Data Import test report QCON (Quesnelia)[ECS]

Overview

This document contains the results of testing Data Import for MARC Bibliographic records at Quesnelia release [ECS].
Ticket: https://folio-org.atlassian.net/browse/PERF-858 on QCON environment.

Summary

  • All Data-imports jobs finished successfully without errors.

  • The PTF - Updates Success - 2 profile(based on qcp1: PTF - Updates Success - 6 ) was created for the QCON Quesnelai release on tenant: cs00000int_0001.

  • DI duration growth correlates to the number of records imported. 

  • No memory leak is suspected for DI modules.

  • Approximate DB CPU usage is close to 95% and this number goes for all jobs with files of more than 10k records. 

Comparison with previous testing results Data Import test report (Quesnelia)[non-ECS]

  • Duration for Data-import create better for for files with smaller size and the same for file with 500k records.

  • Duration for Data-import update better for for files with smaller size and slower on 20% for files with 100k and 500k records.

  • Services CPU utilization, Service memory utilization, and DB CPU utilization have the same utilization trend and values as in the Poppy release.

 

Results

Test #

Data-import test

Duration

Poppy 

Duration

Quesnelia (qcp1)

Duration

Quesnelia (qcon)

Difference, %

Results

Test #

Data-import test

Duration

Poppy 

Duration

Quesnelia (qcp1)

Duration

Quesnelia (qcon)

Difference, %

Results

1.

1k MARC BIB Create

PTF - Create 2

39 sec

54 sec

31 sec

-42%

Completed

2.

5k MARC BIB Create

PTF - Create 2

2 min 22 sec

3 min 20 sec

 

 

Not tested

3.

10k MARC BIB Create

PTF - Create 2

4 min 29 sec

6 minutes

4 min 14 sec

-29%

Completed

4.

25k MARC BIB Create

PTF - Create 2

10 min 38 sec

13 min 41 sec

9 min 41 sec

-29%

Completed 

5.

50k MARC BIB Create

PTF - Create 2

20 min 26 sec

21 min 59 sec

18 min 18 sec

-16%

Completed 

6.

100k MARC BIB Create

PTF - Create 2

2 hours 46 min

Cancelled

40 min 16 sec

38 min 36 sec

-4%

Completed

7.

500k MARC BIB Create

PTF - Create 2

Not Tested

3 hours 27 min

3 hours 30 min

+1.84%

Completed

8.

1k MARC BIB Update

PTF - Updates Success - 6

34 sec

(PTF - Updates Success - 1)

1 min 59 sec

44 sec

-63%

Completed

9

2k MARC BIB Update

PTF - Updates Success - 6

1 min 09 sec (PTF - Updates Success - 1)

2 min 43 sec

 

 

Not tested

10

5k MARC BIB Update

PTF - Updates Success - 6

2 min 31 sec (PTF - Updates Success - 1)

7 min 10 sec

 

 

Not tested

11

10k MARC BIB Update

PTF - Updates Success - 6

5 min 13 sec (PTF - Updates Success - 1)

10 min 27 sec

5 min 59 sec

-42%

Completed

12

25k MARC BIB Update

PTF - Updates Success - 6

12 min 27 sec (PTF - Updates Success - 1)

23 min 16 sec

19 min 52 sec

-14%

Completed

13

50k MARC BIB Update

PTF - Updates Success - 6

Not tested

40 min 52 sec

37 min 53 sec

-7%

Completed

14

100k MARC BIB Update

PTF - Updates Success - 6

Not tested

1 hrs 2 min

1 hrs 14 min

+19%

Completed

15

500k MARC BIB Update

PTF - Updates Success - 6

Not tested

5 hrs 31 min

6 hrs 39 min

+21%

Completed

Service CPU Utilization 

MARC BIB CREATE

CPU

 

RAM

 

CPU

 

RAM

 

mod-inventory-b

88%

mod-inventory-b

79%

mod-data-import-b

59%

mod-search-b

58%

mod-di-converter-storage-b

51%

mod-source-record-manager-b

57%

okapi-b

30%

mod-source-record-storage-b

48%

mod-inventory-storage-b

27%

mod-data-import-b

38%

mod-source-record-storage-b

15%

okapi-b

35%

mod-source-record-manager-b

14%

mod-di-converter-storage-b

34%

mod-search-b

11%

mod-permissions-b

29%

mod-configuration-b

2%

mod-configuration-b

28%

mod-permissions-b

1%

mod-inventory-storage-b

18%

Tests #1-7

1k, 10k, 25k, 50k, 100k, 500k records

CPU utilization for all modules returned by default numbers after all tests. Average for mod-inventory-b - 90%, mod-inventory-storage-b - 27%, mod-source-record-storage-b - 15%, mod-source-record-manager-b - 14%, mod-di-converter-storage-b - 51%, , mod-data-import - 350% spike for 500k job(same behavior on Poppy version).

 

image-20240528-101829.png

 

MARC BIB UPDATE

CPU

 

RAM

 

CPU

 

RAM

 

mod-inventory-b

91%

mod-inventory-b

70%

mod-di-converter-storage-b

54%

mod-search-b

62%

okapi-b

39%

mod-source-record-storage-b

51%

mod-source-record-storage-b

25%

mod-source-record-manager-b

51%

mod-inventory-storage-b

23%

mod-data-import-b

36%

mod-source-record-manager-b

11%

mod-di-converter-storage-b

35%

mod-search-b

8%

okapi-b

34%

mod-data-import-b

7%

mod-permissions-b

30%

mod-configuration-b

2%

mod-configuration-b

27%

mod-permissions-b

1%

mod-inventory-storage-b

21%

Tests #8-15

1k, 10k, 25k, 50k, 100k, 500k records

 

image-20240529-114453.png

 

Memory Utilization

No memory leak is suspected for DI modules.

MARC BIB CREATE

Tests #1-7

1k, 10k, 25k, 50k, 100k, 500k records

 

image-20240528-102909.png

 

MARC BIB UPDATE

Tests #8-15

1k, 10k, 25k, 50k, 100k, 500k records

image-20240529-115203.png

 

RDS CPU Utilization 

MARC BIB CREATE

Average 90% for DI jobs with more than 10k records for Create and Update profiles

 

image-20240528-102043.png

 

MARC BIB UPDATE

image-20240529-114729.png

 

RDS Database Connections

MARC BIB CREATE
DB connections was 1400 in average

image-20240528-102207.png

 

MARC BIB Update

DB connections was 1400 in average

image-20240529-114930.png

 

Average active sessions (AAS)

MARC BIB CREATE

image-20240528-103737.png

Top SQL

image-20240528-103927.png

 

MARC BIB UPDATE

 

image-20240529-115801.png

Top SQL

image-20240529-115947.png

 

OpenSearch Service

Cluster status was green during the tests
Master nodes
1. CPU utilization MasterCPUUtilization
https://us-east-1.console.aws.amazon.com/cloudwatch/home?region=us-east-1#metricsV2?graph=~(metrics~(~(~'AWS2fES~'MasterCPUUtilization~'DomainName~'fse~'ClientId~'054267740449))~view~'timeSeries~stacked~false~region~'us-east-1~title~'CPU20utilization2028Percent*29~period~60~stat~'Maximum~yAxis~(left~(showUnits~false)))

MARC BIB Create
CPU utilization was 20% in average

image-20240531-105522.png

MARC BIB Update

CPU utilization was 20% in average

image-20240531-105644.png

 


Data nodes
1. CPU utilization. CPUUtilization
https://us-east-1.console.aws.amazon.com/cloudwatch/home?region=us-east-1#metricsV2?graph=~(metrics~(~(~'AWS2fES~'CPUUtilization~'DomainName~'fse~'ClientId~'054267740449))~view~'timeSeries~stacked~false~region~'us-east-1~title~'CPU20utilization2028Percent*29~period~60~stat~'Maximum~yAxis~(left~(showUnits~false)))

2. Maximum memory utilization (SysMemoryUtilization)
https://us-east-1.console.aws.amazon.com/cloudwatch/home?region=us-east-1#metricsV2?graph=~(metrics~(~(~'AWS2fES~'SysMemoryUtilization~'DomainName~'fse~'ClientId~'054267740449))~view~'timeSeries~stacked~false~region~'us-east-1~title~'Maximum20memory20utilization2028Percent29~period~60~stat~'Maximum~yAxis~(left~(showUnits~false)))


MARC BIB Create
CPU utilization was 99% in average

image-20240528-124808.png

Maximum memory utilization was 92% in average

image-20240531-104655.png

 

MARC BIB Update

CPU utilization was 99% in average

image-20240529-121247.png

Maximum memory utilization was 94% in average

image-20240531-104919.png

 

Managed Streaming for Apache Kafka

CPU (User) usage by broker

MARC BIB Create

 

image-20240528-122500.png

MARC BIB Update

image-20240529-120826.png

 

Appendix

Infrastructure

  • 11 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1 [Number of ECS instances, instance type, location region]

  • 1 instance of db.r6.xlarge database instance: Writer instance

  • OpenSearch

    • domain: fse

    • Number of nodes: 9

    • Version: OpenSearch_2_7_R20240502

  • MSK - tenat

    • 4 kafka.m5.2xlarge brokers in 2 zones

    • Apache Kafka version 2.8.0

    • EBS storage volume per broker 300 GiB

    • auto.create.topics.enable=true

    • log.retention.minutes=480

    • default.replication.factor=3

    • Kafka consolidated topics enabled

     

Module

Task Def. Revision

Module Version

Task Count

Mem Hard Limit

Mem Soft limit

CPU units

Xmx

MetaspaceSize

MaxMetaspaceSize

qcon-pvt

Thu May 23 10:47:25 UTC 2024

mod-remote-storage

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-remote-storage:3.2.0

2

4920

4472

1024

3960

512

512

mod-finance-storage

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-finance-storage:8.6.0

2

1024

896

1024

700

88

128

mod-ncip

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-ncip:1.14.4

2

1024

896

128

768

88

128

mod-agreements

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-agreements:7.0.0

2

1592

1488

128

0

0

0

mod-ebsconet

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-ebsconet:2.2.0

2

1248

1024

128

700

128

256

mod-consortia

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-consortia:1.1.0

2

3072

2048

128

2048

512

1024

mod-organizations

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-organizations:1.9.0

2

1024

896

128

700

88

128

mod-serials-management

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-serials-management:1.0.0

2

2480

2312

128

1792

384

512

mod-settings

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-settings:1.0.3

2

1024

896

200

768

88

128

mod-search

9

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-search:3.3.0-SNAPSHOT.224

2

2592

2480

2048

1440

512

1024

edge-dematic

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-dematic:2.2.0

1

1024

896

128

768

88

128

mod-data-import

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-data-import:3.1.0

1

2048

1844

256

1292

384

512

mod-tags

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-tags:2.2.0

2

1024

896

128

768

88

128

mod-authtoken

3

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-authtoken:2.15.1

2

1440

1152

512

922

88

128

edge-courses

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-courses:1.4.0

2

1024

896

128

768

88

128

mod-inventory-update

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-inventory-update:3.3.0

2

1024

896

128

768

88

128

mod-notify

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-notify:3.2.0

2

1024

896

128

768

88

128

mod-configuration

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-configuration:5.10.0

2

1024

896

128

768

88

128

mod-orders-storage

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-orders-storage:13.7.0

2

1024

896

512

700

88

128

edge-caiasoft

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-caiasoft:2.2.0

2

1024

896

128

768

88

128

mod-login-saml

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-login-saml:2.8.0

2

1024

896

128

768

88

128

mod-erm-usage-harvester

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-erm-usage-harvester:4.5.0

2

1024

896

128

768

88

128

mod-password-validator

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-password-validator:3.2.0

2

1440

1298

128

768

384

512

mod-licenses

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-licenses:6.0.0

2

2480

2312

128

1792

384

512

mod-gobi

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-gobi:2.8.0

2

1024

896

128

700

88

128

mod-bulk-operations

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-bulk-operations:2.0.0

2

3072

2600

1024

1536

384

512

mod-fqm-manager

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-fqm-manager:2.0.1

2

3000

2600

128

2048

384

512

edge-dcb

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-dcb:1.1.0

2

1024

896

128

768

88

128

mod-graphql

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-graphql:1.12.1

2

1024

896

128

768

88

128

mod-finance

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-finance:4.9.0

2

1024

896

128

700

88

128

mod-erm-usage

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-erm-usage:4.7.0

2

1024

896

128

768

88

128

mod-batch-print

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-batch-print:1.1.0

2

1024

896

128

768

88

128

mod-copycat

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-copycat:1.6.0

2

1024

512

128

768

88

128

mod-lists

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-lists:2.0.0

2

3000

2600

128

2048

384

512

mod-entities-links

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-entities-links:3.0.0

2

2592

2480

400

1440

0

1024

mod-permissions

2

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-permissions:6.5.0

2

1684

1544

512

1024

384

512

pub-edge

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/pub-edge:2023.06.14

2

1024

896

128

768

0

0

mod-orders

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-orders:12.8.0

2

2048

1440

1024

1024

384

512

edge-patron

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-patron:5.1.0

2

1024

896

256

768

88

128

edge-ncip

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-ncip:1.9.2

2

1024

896

128

768

88

128

mod-users-bl

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-users-bl:7.7.0

2

1440

1152

512

922

88

128

mod-invoice

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-invoice:5.8.0

2

1440

1152

512

922

88

128

mod-inventory-storage

2

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-inventory-storage:27.2.0-SNAPSHOT.738

2

4096

3690

2048

3076

384

512

mod-user-import

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-user-import:3.8.0

2

1024

896

128

768

88

128

mod-sender

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-sender:1.12.0

2

1024

896

128

768

88

128

edge-oai-pmh

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-oai-pmh:2.9.0

2

1512

1360

1024

1440

384

512

mod-data-export-worker

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-data-export-worker:3.2.1

2

3072

2048

1024

2048

384

512

mod-rtac

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-rtac:3.6.0

2

1024

896

128

768

88

128

mod-circulation-storage

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-circulation-storage:17.2.0

2

2880

2592

1536

1814

384

512

mod-source-record-storage

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-source-record-storage:5.8.0

2

5600

5000

2048

3500

384

512

mod-calendar

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-calendar:3.1.0

2

1024

896

128

768

88

128

mod-event-config

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-event-config:2.7.0

2

1024

896

128

768

88

128

mod-courses

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-courses:1.4.10

2

1024

896

128

768

88

128

mod-circulation-item

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-circulation-item:1.0.0

2

1024

896

128

0

0

0

mod-inventory

3

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-inventory:20.2.0

2

2880

2592

1024

1814

384

512

mod-email

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-email:1.17.0

2

1024

896

128

768

88

128

mod-circulation

2

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-circulation:24.2.1

2

2880

2592

1536

1814

384

512

mod-pubsub

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-pubsub:2.13.0

2

1536

1440

1024

922

384

512

mod-di-converter-storage

2

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-di-converter-storage:2.2.2

2

1024

896

128

768

88

128

edge-orders

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-orders:3.0.0

2

1024

896

128

768

88

128

edge-rtac

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-rtac:2.7.1

2

1024

896

128

768

88

128

mod-users

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-users:19.3.1

2

1024

896

128

768

88

128

mod-template-engine

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-template-engine:1.20.0

2

1024

896

128

768

88

128

mod-patron-blocks

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-patron-blocks:1.10.0

2

1024

896

1024

768

88

128

mod-audit

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-audit:2.9.0

2

1024

896

128

768

88

128

edge-fqm

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-fqm:2.0.0

2

1024

896

128

768

88

128

mod-source-record-manager

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-source-record-manager:3.8.0

2

5600

5000

2048

3500

384

512

nginx-edge

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/nginx-edge:2023.06.14

2

1024

896

128

0

0

0

mod-quick-marc

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-quick-marc:5.1.0

1

2288

2176

128

1664

384

512

nginx-okapi

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/nginx-okapi:2023.06.14

2

1024

896

128

0

0

0

okapi-b

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/okapi:5.3.0

3

1684

1440

1024

922

384

512

mod-feesfines

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-feesfines:19.1.0

2

1024

896

128

768

88

128

mod-invoice-storage

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-invoice-storage:5.8.0

2

1872

1536

1024

1024

384

512

mod-service-interaction

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-service-interaction:4.0.1

2

2048

1844

256

1290

384

512

mod-dcb

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-dcb:1.1.0

2

1024

896

128

768

88

128

mod-patron

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-patron:6.1.0

2

1024

896

128

768

88

128

mod-data-export

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-data-export:5.0.0

1

2048

1524

1024

0

0

0

mod-oai-pmh

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-oai-pmh:3.13.0

2

4096

3690

2048

3076

384

512

edge-connexion

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-connexion:1.2.0

2

1024

896

128

768

88

128

mod-notes

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-notes:5.2.0

2

1024

896

128

952

384

512

mod-kb-ebsco-java

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-kb-ebsco-java:4.0.0

2

1024

896

128

768

88

128

mod-login

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-login:7.11.0

2

1440

1298

1024

768

384

512

mod-organizations-storage

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-organizations-storage:4.7.0

2

1024

896

128

700

88

128

mod-data-export-spring

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-data-export-spring:3.2.0

1

2048

1844

256

1536

384

512

pub-okapi

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/pub-okapi:2023.06.14

2

1024

896

128

768

0

0

mod-eusage-reports

1

579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-eusage-reports:2.1.1

2

1024

896

128

768

88

128

Methodology

  1. Pregenerated files were used for DI Create job profile

    • 1K, 10K, 25K, 50K, 100K and 500K files.

  2. Run DI Create on a single tenant(cs00000int_0001) one by one with the delay with files using PTF - Create 2 profile.

  3. Prepare files for DI Update with the Data export app, using previously imported items

  4. Run DI Update on a single tenant(cs00000int_0001) one by one with the delay with prepared files using PTF - Update Success 2 profile

  • 1K, 10K, 25K, 50K, 100K and 500K files.

  1. Data-import durations were obtained from DB using SQL query

select file_name,started_date,completed_date, completed_date - started_date as duration ,status from cs00000int_0001_mod_source_record_manager.job_execution order by started_date desc limit 1000;

Related content