Generating Narrower Types

[harrysolovay]

Is your feature request related to a problem? Please describe.

The smithy-generated TypeScript types can be safer and more inference-producing. For example:

Cognito User Pool's IntiateAuth command accounts for many auth flows (represented by the AuthFlowType enum). Based on that supplied enum, we can discriminate between AuthParameters to type-check flow-specific input fields. We can also return the correct flow-specific outputs (narrowed ChallengeName, ChallengeParameters, AuthenticationResult (its omission), etc.).

For the RespondToAuthChallenge command, here are some of the varying signatures we might get (currently untyped).

const smsMfa = await client.send(new RespondToAuthChallengeCommand({
  AuthFlow: ChallengeTypeName.SMS_MFA,
  ...otherFields,
}));

// {
//   ChallengeName: ChallengeTypeName.SMS_MFA;
//   SMS_MFA_CODE: string;
//   USERNAME: string;
// }


const passwordVerifier = await client.send(new RespondToAuthChallengeCommand({
  AuthFlow: ChallengeTypeName.PASSWORD_VERIFIER,
  ...otherFields,
}))

// {
//   ChallengeName: ChallengeTypeName.PASSWORD_VERIFIER;
//   PASSWORD_CLAIM_SIGNATURE: string;
//   PASSWORD_CLAIM_SECRET_BLOCK: string;
//   TIMESTAMP: string;
//   USERNAME: string;
// }


const newPasswordRequired = await client.send(new RespondToAuthChallengeCommand({
  AuthFlow: ChallengeTypeName.NEW_PASSWORD_REQUIRED,
  ...otherFields,
}))

// {
//   ChallengeName: ChallengeTypeName.NEW_PASSWORD_REQUIRED;
//   NEW_PASSWORD: string;
//   USERNAME: string;
//   attributes: {Name: string; Value: string}[];
// }

Describe the solution you'd like

Let's discuss the approach to smithy-model-to-ts-type codegen. Ideally, we could provide more information, not present in the Smithy models. This info could enable inference of the correct return types based on inputs.

[EDIT]:

If we could specify the input-specific output types like this (for InitiateAuth, in this example):

{
  InitiateAuth: {
    ifInput: [{
      extends: {
        AuthFlow: "AuthFlowType.USER_SRP_AUTH",
      },
      outputConstraints: {
        AuthenticationResult: "undefined",
        ChallengeName: "ChallengeNameType.PASSWORD_VERIFIER",
        ChallengeParameters: {
          SALT: "string",
          SECRET_BLOCK: "string",
          SRP_B: "string",
          USERNAME: "string",
          USER_ID_FOR_SRP: "string",
        },
        SESSION: "undefined",
      },
    },
  }],
}

We could generate a generic output type such as this:

type InitiateAuthCommandOutput<I extends InitiateAuthInput> = I extends {
  AuthFlow: AuthFlowType.USER_SRP_AUTH;
}
  ? {
      AuthenticationResult: undefined;
      ChallengeName: ChallengeNameType.PASSWORD_VERIFIER;
      ChallengeParameters: {
        SALT: string;
        SECRET_BLOCK: string;
        SRP_B: string;
        USERNAME: string;
        USER_ID_FOR_SRP: string;
      };
      SESSION: undefined;
    }
  : WideInitiateAuthOutput;

The InitiateAuth command can keep a generic reference to the constructor param, and pass the narrowed type into $Command.

class InitiateAuthCommand<
  I extends InitiateAuthCommandInput,
  O extends InitiateAuthCommandOutput<I>
> extends $Command<
  InitiateAuthCommandInput,
  O,
  CognitoIdentityProviderClientResolvedConfig
> {
  readonly input: InitiateAuthCommandInput;
  constructor(input: I);
  resolveMiddleware(
    clientStack: MiddlewareStack<ServiceInputTypes, ServiceOutputTypes>,
    configuration: CognitoIdentityProviderClientResolvedConfig,
    options?: __HttpHandlerOptions,
  ): Handler<InitiateAuthCommandInput, O>;
  private serialize;
  private deserialize;
}

The caveat is that the input params need to be narrowed (readonly), but it's still a huge DX improvement for TypeScript power users, and would enable us to safeguard––at the type-level––against all kinds of misuse.

Additional context

To avoid breaking backwards-compat, we could expose the narrowed types as a set of module augmentations, available in @aws-sdk/types/strict.ts.

[EDIT]: moved semi-related (but probably for-now unproductive snippet to comment below)

In Summary

I believe stricter types are essential to customer success, as they serve both in protecting against misuse and ease of experience.

Feedback would be greatly appreciated!

[harrysolovay]

Eventually, it'd be nice to focus on the story around genericization as well.

In certain cases, users may want to supply their own types. DynamoDB document clients, for instance, could accept an Item type param, as to type-check their requests.

import "@aws-sdk/types/strict"; // module augmentations
import {DocDBClient} from "@aws-sdk/client-docdb";

interface Todo {
  id: string;
  title: string;
  body: string;
}

// thanks to the augmentation, we'd be able to specify this generic type param
const client = new DocDBClient<Todo>({
  region: "us-west-2",
});

In the rough example above, passing Todo to DocDBClient narrows the client's allowed operations to the specified data type, ensuring correct IO.

[harrysolovay]

Also, can someone possibly explain the generated AuthFlow field type? Why is the AuthFlowType enum "union-ed" with string, as opposed to the narrowed string literals (AuthFlowType[keyof AuthFlowType])? And why is it "union-ed" with undefined, when AuthFlow is required by the service? And... if it could be undefined... why wouldn't AuthFlow be optional?

...
- SomeField: SomeValue | undefined;
+ SomeField?: SomeValue;
...

[trivikr]

Also, can someone possibly explain the generated AuthFlow field type? Why is the AuthFlowType enum "union-ed" with string, as opposed to the narrowed string literals (AuthFlowType[keyof AuthFlowType])? And why is it "union-ed" with undefined, when AuthFlow is required by the service? And... if it could be undefined... why wouldn't AuthFlow be optional?

...
- SomeField: SomeValue | undefined;
+ SomeField?: SomeValue;
...

This was done for future-proofing the code, for cases in which required parameters and changed to optional in service types.

Details in #1124 (comment)

[EDIT]
Me (@trivikr) personally don't like the solution of passing | undefined for the same reason of having less strict types. However, ensuring that customers don't have to update SDK version is important.

[harrysolovay]

I don't feel as though that's the right approach to such future-proofing, as it doesn't take into account other changes to service types (like string -> number, etc.).

Nonetheless, how's this for a solution: we could make stricter types available though module augmentation.

import "@aws-sdk/types/strict";

[trivikr]

I don't feel as though that's the right approach to such future-proofing, as it doesn't take into account other changes to service types (like string -> number, etc.).

Changing the type is not allowed under backward compatibility rules.
Making a required type optional is allowed under backward compatibility, and is followed by some service teams.

[trivikr]

Nonetheless, how's this for a solution: we could make stricter types available though module augmentation.

import "@aws-sdk/types/strict";

How are stricter types expected to work for specific operations?
For example, the RespondToAuthChallengeCommand operation is imported as follows:

import { RespondToAuthChallengeCommand } from "@aws-sdk/client-cognito-identity-provider";

[harrysolovay]

@aws-sdk/types/strict/client-cognito-identity-provider

declare module "@aws-sdk/client-cognito-identity-provider" {
  export type RespondToAuthChallengeCommand = NarrowedRespondToAuthChallengeCommand;
}

Just to reiterate...

The generated RespondToAuthChallengeCommandOutput type would be generic (accepting a type which extends RespondToAuthChallengeCommandInput). The RespondToAuthChallengeCommand would be typed such that we'd have the narrowed input type as a type variable, which we could then supply to the RespondToAuthChallengeCommandOutput, thereby narrowing the output type based on the input type. In the example above, this narrowed command is NarrowedRespondToAuthChallengeCommand.

The augmentation would be applied with the following import statement.

import "@aws-sdk/types/strict/client-cognito-identity-provider";

[harrysolovay]

Loosely (and without the extras):

class RespondToAuthChallengeCommand<
  Input extends RespondToAuthChallengeCommandInput,
  Output extends RespondToAuthChallengeCommandOutput<Input>
> extends $Command<Input, Output> {
  constructor(input: Input) {}
}

[github-actions]

Greetings! We’re closing this issue because it has been open a long time and hasn’t been updated in a while and may not be getting the attention it deserves. We encourage you to check if this is still an issue in the latest release and if you find that this is still a problem, please feel free to comment or open a new issue.

[harrysolovay]

Commenting so the actions bot doesn't close this out, as I still believe this is an important issue. Hope all is well!

[github-actions]

Greetings! We’re closing this issue because it has been open a long time and hasn’t been updated in a while and may not be getting the attention it deserves. We encourage you to check if this is still an issue in the latest release and if you find that this is still a problem, please feel free to comment or open a new issue.

[harrysolovay]

Over two years and still no narrowing? How is this possible? Why is there no emphasis on offering a more-narrowly-typed (and therefore safer) DX. Quite disappointing.

---

For those reading this issue, also disappointed: check out typesafe-dynamodb and franken-srp. typesafe-dynamodb is an absolute gem. The latter––franken-srp––doesn't seem to have caught on... but it's a well-written lib (and the only alternative to Amplify's auth client).

[acjay]

I definitely support at least the option of being able to use stricter types.

Upgrading from V2, I was surprised to find that APIs i was using before, like ParameterStore did not have source direct replacements, and instead, I have to use APIs with types that force me to use postfix ! to override the type checker. As someone using these AWS services directly for the first time, it's a very confusing experience. The lack of documentation around this decision and overall sparse documentation in this library make that even worse. I'm kind of surprised V2 was deprecated with V3 in this state.

I guess if AWS isn't going to fix this, it might just be on the community to write a wrapper with accurate types.

[wesbos]

Ran into this today with the union types. Something like MediaFormat or Subtitles.Formats have a clear list of options in the docs, but are just listed as a string in the types. Only once you run the code do you get errors like Member must satisfy enum value set: [amr, flac, wav, ogg, mp3, mp4, webm]

would it not make sense to have these values as a string union instead of just a string?

[kuhe]

partially addressed in #5356, which closed string unions to only the enumerated values.

[kuhe]

The service models from which we generate our types do not contain enough information to produce the inference types. We cannot implement that part of the feature request practically.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.