Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(streams): add readme section about stream mixins #4036

Merged
merged 5 commits into from
Oct 13, 2022
Merged

docs(streams): add readme section about stream mixins #4036

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
df2fc78
docs(streams): add readme section about stream mixins
kuhe Oct 12, 2022
eff1b55
Update README.md
kuhe Oct 12, 2022
c4bd296
Update README.md
kuhe Oct 12, 2022
38e81dd
docs(streams): add newlines
kuhe Oct 12, 2022
fa5f81d
docs(streams): update streaming mixin docs
kuhe Oct 13, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 34 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
![Build Status](https://codebuild.us-west-2.amazonaws.com/badges?uuid=eyJlbmNyeXB0ZWREYXRhIjoiMmtFajZWQmNUbEhidnBKN1VncjRrNVI3d0JUcFpGWUd3STh4T3N3Rnljc1BMaEIrYm9HU2t4YTV1RlE1YmlnUG9XM3luY0Ftc2tBc0xTeVFJMkVOa24wPSIsIml2UGFyYW1ldGVyU3BlYyI6IlBDMDl6UEROK1dlU1h1OWciLCJtYXRlcmlhbFNldFNlcmlhbCI6MX0%3D&branch=main)
[![codecov](https://codecov.io/gh/aws/aws-sdk-js-v3/branch/main/graph/badge.svg)](https://codecov.io/gh/aws/aws-sdk-js-v3)
[![code style: prettier](https://img.shields.io/badge/code_style-prettier-ff69b4.svg)](https://github.com/prettier/prettier)
[![Dependabot Status](https://api.dependabot.com/badges/status?host=github&repo=aws/aws-sdk-js-v3)](https://dependabot.com)

The **AWS SDK for JavaScript v3** is a rewrite of v2 with some great new features.
As with version 2, it enables you to easily work with [Amazon Web Services](https://aws.amazon.com/),
Expand Down Expand Up @@ -37,6 +36,7 @@ visit our [code samples repo](https://github.com/aws-samples/aws-sdk-js-tests).
1. [How to upgrade](#other-changes)
1. [High Level Concepts in V3](#high-level-concepts)
1. [Generated Packages](#generated-code)
1. [Streams](#streams)
1. [Paginators](#paginators)
1. [Abort Controller](#abort-controller)
1. [Middleware Stack](#middleware-stack)
Expand Down Expand Up @@ -282,6 +282,39 @@ Lastly we have higher level libraries in `/lib`. These are javascript specific l
1. `/clients`. This sub directory is code generated and depends on code published from `/packages` . It is 1:1 with AWS services and operations. Manual edits should generally not occur here. These are published to NPM under `@aws-sdk/client-XXXX`.
1. `/lib`. This sub directory depends on generated code published from `/clients`. It wraps existing AWS services and operations to make them easier to work with in Javascript. These are published to NPM under `@aws-sdk/lib-XXXX`

### Streams

Certain command outputs include streams, which have different implementations in NodeJS and browsers. For convenience, a set of stream handling methods will be merged (`Object.assign`) to the output stream object, as defined in [SdkStreamMixin](https://github.com/aws/aws-sdk-js-v3/blob/ad99b66944b6fe1ce83082c1a33193dff033cfaf/packages/types/src/serde.ts#L77-L81).
kuhe marked this conversation as resolved.
Show resolved Hide resolved
kuhe marked this conversation as resolved.
Show resolved Hide resolved

Output types having this feature will be indicated by the `WithSdkStreamMixin<T, BaseStream>` [wrapper type](https://github.com/aws/aws-sdk-js-v3/blob/ad99b66944b6fe1ce83082c1a33193dff033cfaf/packages/types/src/serde.ts#L89-L96), where `T` is the original output type and `BaseStream` is
the stream type specific to the runtime environment.

Here is an example using `S3::GetObject`.

```js
import { S3 } from "@aws-sdk/client-s3";

const client = new S3({});

const getObjectResult = await client.getObject({
Bucket: '...',
kuhe marked this conversation as resolved.
Show resolved Hide resolved
Key: "...",
});

// env-specific stream with added mixin methods.
const bodyStream = getObjectResult.Body;

// one-time transform.
const bodyAsString = await bodyStream.transformToString();

// throws an error on 2nd call, stream cannot be rewound.
const __error__ = await bodyStream.transformToString();
```

Note that these methods will read the stream in order to collect it,
so **you must save the output**. The methods cannot be called more than once
on a stream.

### Paginators

Many AWS operations return paginated results when the response object is too large to return in a single response. In AWS SDK for JavaScript v2, the response contains a token you can use to retrieve the next page of results. You then need to write additional functions to process pages of results.
Expand Down
9 changes: 4 additions & 5 deletions packages/types/src/serde.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,12 +88,12 @@ export type SdkStream<BaseStream> = BaseStream & SdkStreamMixin;

/**
* Indicates that any members of type T
* that were of type BaseStream have been extended
* that were of type T[StreamKey] have been extended
* with the SdkStreamMixin helper methods.
*/
export type WithSdkStreamMixin<T, BaseStream> = {
[key in keyof T]: T[key] extends BaseStream ? SdkStream<BaseStream> : T[key]
}
export type WithSdkStreamMixin<T, StreamKey extends keyof T> = {
[key in keyof T]: T[key] extends T[StreamKey] ? SdkStream<T[StreamKey]> : T[key];
};

/**
* Interface for internal function to inject stream utility functions
Expand All @@ -111,4 +111,3 @@ export interface SdkStreamMixinInjector {
export interface SdkStreamSerdeContext {
sdkStreamMixin: SdkStreamMixinInjector;
}