, Ian and I are reviewing this ticket - would this be superceded by Modusers-3?

Still looking for a benchamrk of 70 records per second.

We performed new bulk import test in the following environment:

16GB DDR4 RAM, i7-7820HQ CPU, 240GB SSD

• base vagrant box : folio/testing version: 5.0.0-20180925.1100

• mod_users, version: 15.3.0

• mod_user_import, version: 3.1.1

• mod_permissions, version: 5.4.0

All modules are running in Vagrant box none of them hosted locally.

Experiences:

• Vagrant box and services are starting much slower than usual

• The performance of user import is much lower than it was during our last measure. 1000 user's import took more then 5 minutes!

• We experienced a lot of postgres process running at the same time in the box consuming huge CPU (see attached image)

We didn't have time to dig deeper and identify the root cause of poor performance but something may changed in a wrong way since our last benchmark. I cannot state that the cause of slow-down is mod-users or mod-permission modules. Perhaps there is something completely different reason.

What should we do now? What do you suggest considering we don't have too much time until our contract termination.

Test environment.

16GB DDR4 RAM, i7-7820HQ CPU, 240GB SSD

base vagrant box : folio/testing
Replaced/Redeployed modules, from locally source code with embedded postgres:
mod-users, mod-permissions, mod-user-import, mod-users-bl

Used ram for these 4 module (run from intelliJ IDEA) was 2.5GB(max value) when I imported 100.000 user at once. The json file with the users was 92MB.

Importing time test results:

5k insert - 1.0 min;0.78 minutes; 0.85 min, 0.72 min
5k update - 0.5 min; 0.47 min; 0.46 min, 0.47 min, 0.5 min
100k insert: 24.5 min, 25min

The 40% of the import time is spent in mod users to get users by external id-s.
The 20% is for the POST or PUT the users into mod-users.
The rest 40% is for POST permissions info into mod-permission module.

It is important to add temporarily more RAM for mod-user-import, mod-user, mod-permission if they want to import a lot of user (more than 10k).

The slowest parts of the user import are:
Querying mod-users by external-id (bulk GET with 10-30 user at once).
Insert permissions to mod-permissions.

Recommendation for mod-users developers:
The user import would be faster, if there will be an index on the users table on the externalSystemId field.

Recommendation for mod-permissions developers:
Review postPermsUsers implementation is necessary, because POST permessionUser it takes a little longer than we expected. (10 times longer(3-5ms vs 30-50ms)) than post new user into mod-users)

will do it in the current sprint. He will also give an estimate how many effort to integrate mod-user-import in mod-users. I hope we will be able to answer these questions at the end of this week.

Hi . Any test results yet?