/
[Spike] FQM - Bulk Operation integration (Query Builder support)

[Spike] FQM - Bulk Operation integration (Query Builder support)

Owned by Viachaslau Khandramai (Deactivated)
Last updated: Jan 22, 2024

FQM API https://github.com/folio-org/mod-fqm-manager/blob/master/README.md.

BulkOperation should utilize asynchronous calls.

Query Plugin RequirementsFQM APISchemaBulk Operations APIDescription
getAsyncContentDataGET /lists/${listID}/contents?offset=${offset}&size=${limit}FQM schema as in getAsyncContentData (not UnifiedTable)GET /bulk-ops/${bulkOpId}/preview/contents?offset=${offset}&size=${limit} (will be executed only after status = SUCCESS)This API is an example from Lists app, for bulk-edit we should have same API. It should return preview of records on the page where user want's to start bulk edit or download files
getAsyncEntityTypeGET /entity-types/${entityTypeId}

I think this is DUPLICATE of entityTypeDataSource
getParamsSourceGET /entity-types/${entityTypeId}/columns/${columnName}/values?search=${searchValue}

This API required for recieve values dynamicaly for some fiels inside query-builder.
cancelQueryDataSourceDELETE /query/${queryId}

This API called when user closing plugin but query isn't completed. We need to clenuap everything
entityTypeDataSourceGET /entity-types/${selectedType}

This API called for all coulumns in table + options in select menu (where we can select column)
queryDetailsDataSourceGET /query/${queryId}

This API called for preview inside query-plugin builder (calling every 5 seconds and checking the status). See flow bellow for details.
testQueryDataSourcePOST /query

This API called when "Test qury" button is clicked

  • Test query on BulkOps side?
runQueryDataSourcePOST /lists
POST /bulk-ops/query fqmQuery as Request body returns queryIdThis API called when "Run and save" button is clicked
queryDetailsDataSourceGET queryDetailsDataSource schema (content = null, the only case includeResults=false should be supported)GET /bulk-ops/query/${queryId}


Current flow:

First of all, the user must create queries in the form, otherwise all clickable elements will be disabled

Once query configured, then “Test query” button will be enabled

After the user clicks on "Test query”, a chain of calls is launched:

  • Plugin is calling testQueryDataSource with the next payload: 

    // payload
{entityTypeId:"0cb79a4c-f7eb-4941-a104-745224ae0292", fqlQuery:"{\"item_status\":{\"$eq\":\"Available\"}}"}

// response
{"queryId":"b97465ae-7dcc-4168-8a41-a913ec8a652e"}


    Once we get a response we do another 2 requests in parallel entityTypeDataSource and queryDetailsDataSource

  • entityTypeDataSource - will return us all columns for preview, we are calling it only once
  • queryDetailsDataSource - will return us content for the preview. Inside the plugin, we only show 100 records, but Query can return many more. The plugin will make requests every 5 seconds until the request is completed. The operating algorithm is given below: 
    1) get all columns metadata with entityTypeDataSource
    2) call queryDetailsDataSource with next search params includeResults=true&offset=0&limit=100. 
    3) check the response of queryDetailsDataSource, we are checking status field and content.length in respose, see possible cases:
    status: IN_PROGRESS and content.length < 100 - continue calling queryDetailsDataSource every 5 seconds
    status: IN_PROGRESS and content.length >= 100 - continue calling queryDetailsDataSource every 5 seconds BUT change search params to includeResults=false, because we showed user first 100 items and no need extra data
    status: SUCCESS and any content.length - stop calling queryDetailsDataSource and enable both buttons, Test query and Run query and save 
    status: FAILEDm just stop calling everything and inform user about fail with callout
    Example of response for queryDetailsDataSource:

At this moment query-builder plugin looks like this: 


After the user is shown the final preview and all buttons are undisabled, he has the option to Retest the query (return to step 1) or click Run query and save

Let's see the case with Run query. Once this button clicked, then runQueryDataSource will be executed, from the plugin it will have next payload { fqlQuery, queryId } and this logic can be different based on host app needs.