Skip to end of banner
Go to start of banner

POC: Export records using CQL

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 4 Current »


Content:


PURPOSE:

MDEXP-281 - Getting issue details... STATUS

Explore approaches for export inventory instances by using a file with .cql extension with CQL query content.

OPTIONS:

There are two options, how we can use CQL query to export records:

  1.  Use the CQL to  fetch all the record`s that match a query from inventory;
  2. Use the CQL to fetch all the record`s UUIDs, that match a query from inventory;
  • Use the CQL to  fetch all the records that match a query from inventory:

Advantages:

  1. We don`t need to store a file with UUIDs while using this approach;
  2. Reduced calls to inventory;

Disadvantages:

  1. There is no way to show to the user the correct progress bar because for now, inventory returns not proper total records count.
  2. This approach requires a lot of changes in code and the whole architecture of the data-export module;
  • Use the CQL to fetch all records UUIDs, that match a query from inventory:

  Advantages:

  1. This approach requires minimal changes in code;
  2. While using this option, we will use the existing logic for calculating progress;

Disadvantages:

      1. Compared with existing logic, we will have to make 1 additional call to instance-bulk/ids endpoint to fetch record`s UUIDs;

CQL query sanitizing in mod-data-export

We are using the public API of mod-inventory-storage, in my opinion, we should not do any work at our side to sanitize the CQL query. However, if the user will send '*' as a query, the process of getting all instance`s UUIDs for the second option could be very long, it will hit the performance. In such a case, we can change the instance-bulk/ids endpoint in mod-inventory-storage, and add a query limit and offset, then collect the responses by using them.

CONCLUSION:

After analyzing both approaches, I came to the conclusion that it is better to use the second option.




  • No labels