Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

MinioS3Client#write fails after 52428800000 bytes

Description

MinioS3Client#write fails when the stream exceeds 52428800000 bytes.

This is caused by setting the partSite to MIN_MULTIPART_SIZE = 5242880.
https://github.com/folio-org/folio-s3-client/blob/v1.0.0/src/main/java/org/folio/s3/client/MinioS3Client.java#L169

S3 allows a maximum of only 10000 parts:
https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html

When starting the 10001st part it fails.

The write method gets the data to upload from an InputStream that needs to be processed in sequence. Splitting it into parts for parallel upload cannot give significant performance advantages. Therefore the maximum part size, not the minimum part size, should be used.

CSP Request Details

None

CSP Rejection Details

None

Potential Workaround

None

Checklist

hide

TestRail: Results

Activity

Show:

Julian Ladisch March 13, 2023 at 8:52 AM

Done

Details

Assignee

Reporter

Priority

Development Team

Firebird

Fix versions

Release

Orchid (R1 2023) Bug Fix

RCA Group

Implementation coding issue

Affected releases

Orchid (R1 2023)

TestRail: Cases

Open TestRail: Cases

TestRail: Runs

Open TestRail: Runs

Created March 13, 2023 at 8:08 AM
Updated October 12, 2023 at 3:50 AM
Resolved March 14, 2023 at 12:01 PM
TestRail: Cases
TestRail: Runs