-
Notifications
You must be signed in to change notification settings - Fork 29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix multipart upload - move query parameters to HTTP request query object #47
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work @immavalls 🙇🏻 👍🏻
Could we add the following example in the examples
folder please (as s3-multipart.js
?)
import crypto from 'k6/crypto'
import exec from 'k6/execution'
import { AWSConfig, S3Client } from '../build/s3.js'
const awsConfig = new AWSConfig({
region: __ENV.AWS_REGION,
accessKeyId: __ENV.AWS_ACCESS_KEY_ID,
secretAccessKey: __ENV.AWS_SECRET_ACCESS_KEY,
sessionToken: __ENV.AWS_SESSION_TOKEN,
})
const s3 = new S3Client(awsConfig)
const testBucketName = 'test-jslib-aws'
const testFileKey = 'multipart.txt'
export default function () {
// List the buckets the AWS authentication configuration
// gives us access to.
const buckets = s3.listBuckets()
// If our test bucket does not exist, abort the execution.
if (buckets.filter((b) => b.name === testBucketName).length == 0) {
exec.test.abort()
}
// Produce random bytes to upload of size ~12MB, that
// we will upload in two 6MB parts. This is done as the
// minimum part size supported by S3 is 5MB.
const bigFile = crypto.randomBytes(12 * 1024 * 1024)
// Initialize a multipart upload
const multipartUpload = s3.createMultipartUpload(testBucketName, testFileKey)
// Upload the first part
const firstPartData = bigFile.slice(0, 6 * 1024 * 1024)
const firstPart = s3.uploadPart(
testBucketName,
testFileKey,
multipartUpload.uploadId,
1,
firstPartData
)
// Upload the second part
const secondPartData = bigFile.slice(6 * 1024 * 1024, 12 * 1024 * 1024)
const secondPart = s3.uploadPart(
testBucketName,
testFileKey,
multipartUpload.uploadId,
2,
secondPartData
)
// Complete the multipart upload
s3.completeMultipartUpload(testBucketName, testFileKey, multipartUpload.uploadId, [
firstPart,
secondPart,
])
// Let's redownload it verify it's correct, and delete it
const obj = s3.getObject(testBucketName, testFileKey)
s3.deleteObject(testBucketName, testFileKey)
}
Thanks for the review and the example @oleiade, I just added it to the examples folder, so we might be ready to merge? I will work on finishing the docs part now with the example. |
correct file
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's do it 👏🏻
Corrects #42, which was not working in AWS s3.
I also corrected some typos in the comments and fixed the
protocol
to bethis.scheme
, allowing for either http or https.