Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Create a script to generate fake Data Import Jobs data

RCA Group

None

Description

Cornell has over 20,000 records in the mod_source_record_manager*.job_executions* table which makes the landing page loads very slowly or not even load at all. 

For testing purposes, create a script to generate fake data to populate in mod_source_record_manager*.job_executions* and 
mod_source_record_manager*.job_execution_progress* tables so that we can simulate a dataset that has tens of thousands of records which is used to test the efficiency of loading the DI landing page (or the queries that are involved on that page). 

 

AI: 1. Run attached script 

2. To create fake data use SELECT put_job_executions('diku',10); where 'diku' is a tenant and 10 is a number of new records.

3. Run needed tests.

4. To cleanup fake data, use SELECT delete_job_executions('diku'); where 'diku' is a tenant.

Environment

None

Potential Workaround

None

Attachments

1

Checklist

hide

TestRail: Results

Activity

Show:

Aivar IusupovDecember 2, 2021 at 12:43 PM

 I will close this task because all the work is done. Thank you.

Martin TranNovember 30, 2021 at 11:24 PM

JFYI: we put this code to use immediately and found that in Kiwi the DI landing page query {{protocol}}://{{url}}:{{okapiport}}/metadata-provider/jobExecutions?limit=100&query=((status any "COMMITTED ERROR")) sortby completedDate/sort.descending takes about 3 seconds more after adding 20K records. In Juniper HF4, it took about 2.6s more. 

Martin TranNovember 30, 2021 at 7:25 PM
Edited

Both functions work well and as intended. Thank you.

Aivar IusupovNovember 30, 2021 at 1:01 PM

Hey , added new version of script with cleanup function (will work only for new records created by put_job_executions function).

To create fake data use SELECT put_job_executions('diku',10);

To cleanup fake data, use SELECT delete_job_executions('diku');

Aivar IusupovNovember 30, 2021 at 9:23 AM

Hey  and ! I'll create a function for db cleanup. Also, there are some changes (In progress), that will affect UI/db structure for job_executions: . After will be done, we'll need to make changes in script due to changes in db structure.

Done

Details

Assignee

Reporter

Labels

Priority

Story Points

Development Team

Folijet Support

Parent

Release

Lotus R1 2022

TestRail: Cases

Open TestRail: Cases

TestRail: Runs

Open TestRail: Runs

Created November 22, 2021 at 10:03 PM
Updated December 29, 2021 at 2:48 PM
Resolved December 2, 2021 at 12:43 PM
TestRail: Cases
TestRail: Runs