Spike [Bulk-Edit]: temporary files operations optimization

Spike [Bulk-Edit]: temporary files operations optimization

Role

Person

Comments

Role

Person

Comments

Solution Architect

@Taras Spashchenko 

 

Java Lead

@Viachaslau Khandramai (Deactivated)

.

UI Lead

@Uladislau Samets

 

Product Owner

@Magda Zacharska

 

 

https://folio-org.atlassian.net/browse/MODEXPW-234 - Spike [Bulk-Edit]:  temporary files operations optimization

Optimized operations

  • BulkEditErrorService should append errors directly to remote storage

  • BulkEdit jobs should be saved to use both local and remote storages as it is already implemented

  • Avoid files creation in services, write/append directly to remote storage (excl. spring batch operations)

  • Avoid writing content as byte arrays, streams should be used

  • Files should be saved in the separate directories with format bucket/{jobId}/{filename}

  • Writers hierarchy should be refactored as following:

FOLIO S3-storage client

  • S3-client should be moved as separate library

  • S3-client should support both AWS and Minio Server

  • InputStream should be used instead of byte[]

  • append() should work with using buffer to reduce number of write operations

  • Storages should aggregate corresponding S3Client