Performance testing of mod-pubsub

Description

Motivation

To understand a real performance of one pub-sub module and investigate the possibility to scaling.

Steps

  • Create a stub module with pubsub client to process events

  • Create a stub module with pubsub client to send events

  • Create a batch function to send 1million? events

  • Count time for this operation

  • measure memory usage

  • write wiki page with results

Expected results

Wiki page with results of performance testing

Environment

None

Potential Workaround

None

Checklist

hide

TestRail: Results

Activity

Show:

Volodymyr RohachFebruary 17, 2020 at 10:23 AM

Hello ! I think that there will be 1 event for 1 record. So, there will be 2,000 events for 2,000 records in an incoming file.

Oleksii KuzminovFebruary 17, 2020 at 10:18 AM
Edited

Hello ! During the investigation in the scope of this task, we found a couple of bugs(already done) and a performance bottleneck. We want to fix it https://folio-org.atlassian.net/browse/MODPUBSUB-69#icft=MODPUBSUB-69 and than check performance with more events and how many HTTP requests effort we have MODPUBSUB-68

Ann-Marie BreauxFebruary 17, 2020 at 9:42 AM

Thanks for putting this together . One question from me: the webpage says that if the number of events is more than 500, they are not all received. If there were 2,000 records in an incoming import file, and each of them needs to match a number in the incoming record to a field in an Inventory Item record, would that be 1 event or 2,000 events? It will be pretty common to have large files being imported, so we'll definitely want to figure out what's causing events to get lost along the way.

Volodymyr RohachFebruary 11, 2020 at 4:13 PM

As a result of this ticket was created new page with mod-pubsub performance testing: https://folio-org.atlassian.net/wiki/display/FOLIJET/mod-pubsub+performance+testing

CC:

Done

Details

Assignee

Reporter

Priority

Story Points

Sprint

Development Team

Folijet

Fix versions

TestRail: Cases

Open TestRail: Cases

TestRail: Runs

Open TestRail: Runs

Created January 6, 2020 at 1:56 PM
Updated March 18, 2020 at 8:18 AM
Resolved February 24, 2020 at 12:02 PM
TestRail: Cases
TestRail: Runs

Flag notifications