S3 Body.toString() not returning the object's content

[imedadel]

Describe the bug
Running GetObject and then Body.toString() is suddenly not returning a string anymore. Instead, I am getting an IncomingMessage object (stringified). I am using TypeScript 3.8.3.

SDK version number
1.0.0-beta.3

Is the issue in the browser/Node.js?
Browser/Node.js

Details of the browser/Node.js version
Tested on v12.6.1 and v13.12

To Reproduce (observed behavior)
Steps to reproduce the behavior (please share code or minimal repo)

import {
	S3Client,
	GetObjectCommand,
} from "@aws-sdk/client-s3"

async function getContent({ uid }) {
	try {
		console.log("> Getting content from S3")
		const s3Client = new S3Client({
			region: "us-east-1",
		})
		const command = new GetObjectCommand({
			Key: `content/${uid}.txt`,
			Bucket: "archive",
		})
		const s3Item = await s3Client.send(command)
		console.log("> Content received.")

		return s3Item.Body.toString()
	} catch {
		console.log(`> Error.`)
	}
}

Expected behavior
A string containing the content stored in under the object's key in S3.

[makeitraina]

+1 reproduced on NodeJs v12.6.1 with version 1.0.0-beta.3

[abierbaum]

Reproduced here too. I can't find a way to download an S3 object with the SDK. Does anyone have a full example of how to make this work with the latest APIs?

[imedadel]

I went back to using v2 :(

[abierbaum]

I think I figured it out. The new APIs are returning Node streams. So you need to read the streams using the stream interfaces. In my case I am just going to use the get-stream npm package to read into a Buffer.

[imedadel]

Please tell us if the this solution worked for you :)

[abierbaum]

@imedadel Yes, it is working. I haven't cleaned up error handling, but this is working for me right now.

const resp: s3.GetObjectCommandOutput = await this.s3Client.getObject({
   Bucket: this.opts.bucketName,
   Key: objkey,
});

const file_stream = resp.Body!;
let content_buffer: Buffer | null = null;

if (file_stream instanceof Readable) {
   content_buffer = await getStream.buffer(file_stream as any);
} else {
   throw new Error('Unknown object stream type.');
}

[AllanZhengYP]

Hi, @abierbaum @imedadel @makeitraina

This is an expected API change from V2 to V3. In v2 the SDK always buffer the downloaded data so you can call toString() on the data easily. But it will bring us problem when the data is very big, and SDK will buffer all of the data in the memory. Now the response Body is a ReadableStream. As @abierbaum has pointed out you can consume the stream like in the sample. You can download the data easily like:

import {
	S3Client,
	GetObjectCommand,
} from "@aws-sdk/client-s3";
import {Readable} from "stream";
import {createWriteStream} from "fs";

async function getContent({ uid }) {
	try {
		console.log("> Getting content from S3");
		const s3Client = new S3Client({
			region: "us-east-1",
		});
		const command = new GetObjectCommand({
			Key: `content/${uid}.txt`,
			Bucket: "archive",
		});
		const s3Item = await s3Client.send(command);
                 s3Item.Body.pipe(createWriteStream(fileName));
	} catch {
		console.log(`> Error.`);
	}
}

[imedadel]

Thanks, everyone! Since it's intentional and I suppose will be documented soon. We can close this issue :)

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.