Data export and CSV file generation on Front-end side

Owned by Vasily Gancharov
Last updated: Dec 15, 2020

Data export in this case is performed on the Front-end side:

  1. UI module retrieves data from a back-end module by making queries to its APIs. Data is usually retrieved sequentially page by page until there is no data returned for the query that UI module makes.
    All data stored in the RAM. 
  2. Once the data is retrieved, it may be transformed by the UI module.
  3. Then the UI module leverages the stripes-util method to generate a CSV file from the data. Stripes-util internally uses the json2csv library to convert JSON into CSV.
  4. The file download is either started automatically by the browser (if such capability is supported) or a link to the file is added to a page (to the same page from which UI module methods were invoked).

Pic. 1.

Advantages of the approach:

  1. It is relatively fast and easy to implement it


Drawbacks of the approach:

  1. UI module downloads data from a back-end module sequentially - chunk by chunk. It makes download slower comparing to the multi-threaded approach.
  2. All data is stored in RAM until the last chunk is downloaded from the back-end module. If there are millions of records, it can take a good amount of RAM.
    The more RAM it takes the higher probability that it will worsen front-end performance and hence user experience.
  3. CSV creation is started only after the last data chunk is downloaded. It makes the overall data export process slower.
  4. The download process is synchronous, which means that if a user closes the page, then the process finishes. It makes the user wait until the CSV is created, which isn't a really good user experience.
  5. No data download / CSV generation progress is shown to the user, which can call user frustration in the case of big CSV file generation.
  6. If multiple customers will massively export data at the same time, it can create high pressure on the back-end module. It should be an edge case since it is expected that export operations are rare. 
    Collisions may occur if data is exported at the same time of the year regularly. For instance, if taxes should be paid by the end of the year, all customers may download data every year within the same time period.

This approach can work only in cases when small CSV files are generated. The bigger the file the less efficient the approach will be and the more user frustration it will make.