Proposal: HTTP status code response binding support in the HTTP client

[TharmiganK]

Summary

The HTTP services sends different status code responses depends on the business logic. The Ballerina HTTP services supports this scenario using the StatusCodeResponse records. Users can define their own status code records by type inclusion and modify the body and headers. But at the client side, users have to check the status code before processing the response payload and/or headers. This disallows them to use direct data-binding since data-binding works only for the successful responses. This proposal is to provide a way to directly data-bind the response to specific status code record types.

Goals

• Provide a way to directly data-bind status code records with defined payload and headers structures

Motivation

Consider the below example service which returns different status code responses:

import ballerina/http;

type Album record {|
    readonly string id;
    string name;
    string artist;
    string genre;
|};

table<Album> key(id) albums = table [
    {id: "1", name: "The Dark Side of the Moon", artist: "Pink Floyd", genre: "Progressive Rock"},
    {id: "2", name: "Back in Black", artist: "AC/DC", genre: "Hard Rock"},
    {id: "3", name: "The Wall", artist: "Pink Floyd", genre: "Progressive Rock"}
];

type ErrorMessage record {|
    string albumId;
    string message;
|};

type Headers record {|
    string user\-id;
    int req\-id;
|};

type AlbumNotFound record {|
    *http:NotFound;
    ErrorMessage body;
    Headers headers;
|};

type AlbumFound record {|
    *http:Ok;
    Album body;
    Headers headers;
|};

service /api on new http:Listener(9090) {

    resource function get albums/[string id]() returns AlbumFound|AlbumNotFound {
        if albums.hasKey(id) {
            return {
                body: albums.get(id),
                headers: {user\-id: "user-1", req\-id: 1}
            };
        }
        return {
            body: {albumId: id, message: "Album not found"},
            headers: {user\-id: "user-1", req\-id: 1}
        };
    }
}

Key points to consider:

• The service returns two status code responses - 200 - OK and 404 - Not Found
• Both status code responses has defined payload structure - 200 - Album and 404 - ErrorMessage
• Both status code responses has common header structure - Headers

Now, lets try to write an HTTP client to capture the above details:

import ballerina/http;

public function main() returns error? {
    http:Client albumClient = check new ("localhost:9090/api");
    http:Response res = check albumClient->/albums/'3;
    
    // The Payload
    json payload = check res.getJsonPayload();
    if res.statusCode == http:STATUS_OK {
        Album _ = check payload.fromJsonWithType();
    } else if res.statusCode == http:STATUS_NOT_FOUND {
        ErrorMessage _ = check payload.fromJsonWithType();
    } else {
        // Unexpected status code responses
    }

   // There is another way to do direct data-binding and obtain the `ApplicationResponseError`
   // for failure status codes. The error details has the body and headers.
   // But most of the users are not aware of this error type

    // The Headers
    do {
        Headers _ = {
            user\-id: check res.getHeader("user-id"),
            req\-id: check int:fromString(check res.getHeader("req-id"))
        };
    } on fail {
        // Unexpected response headers
    }
}

Even though this works, users has to a lot of work to obtain the necessary details. In addition they have to do unnecessary checks. This becomes more complex when we have more status code responses with different media-types, payload types and headers.

This also has an impact on the client generation from the OpenAPI. Consider the below OpenAPI specification for the above service:

paths:
  /albums/{id}:
    get:
      operationId: getAlbumsId
      parameters:
      - name: id
        in: path
        required: true
        schema:
          type: string
      responses:
        "200":
          description: Ok
          headers:
            req-id:
              required: true
              schema:
                type: integer
                format: int64
            user-id:
              required: true
              schema:
                type: string
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Album'
        "404":
          description: NotFound
          headers:
            req-id:
              required: true
              schema:
                type: integer
                format: int64
            user-id:
              required: true
              schema:
                type: string
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorMessage'

Lets generate a client from the above specification:

public isolated client class Client {
    ...

    resource isolated function get albums/[string id]() returns Album|error {
        string resourcePath = string `/albums/${getEncodedUri(id)}`;
        Album response = check self.clientEp->get(resourcePath);
        return response;
    }
}

Here,

• the return type is only for the successful response other responses will be notified as an error
• if the user want to access the payload for 404 - Not Found scenarios, he has to get the ApplicationResponseError and get the details to extract the payload and headers. But mostly, the users are not aware of such error type
• the header information is not exposed

Even though the OpenAPI specification gives a well defined status code responses, currently it is not possible to expose them properly in the Ballerina client/service generation.

Description

To address the above issue, the following should be supported in the HTTP module

// Client errors and other status code responses will be returned as error
AlbumFound|AlbumNotFound res = check albumClient->/albums/'3;

There can be other variations of this status code response support:

• Binding the success payload directly

   Album|AlbumNotFound res = check albumClient->/albums/'3;

• Binding only the success payload directly. So this will not break the existing behaviour

  Album res = check albumClient->/albums/'3;

• Binding only the 404 - Not Found. This is also possible if the user is only concern about the non-success response

  AlbumNotFound res = check albumClient->/albums/'3;

• Binding the other status code responses as http:Response

  AlbumFound|AlbumNotFound|http:Response res = check albumClient->/albums/'3;

Design points

1. There can be scenarios where the user defined response type can be a union of multiple status code responses which represents the same status code. In that case, the return type should be the first matching status code response (this is inline with cloneWithType specification)

    // Consider the scenario for `404 - Not Found`
    AlbumNotFound res = check albumClient->/albums/'3; // => matched to `AlbumNotFound`
    AlbumNotFound|http:NotFound res = check albumClient->/albums/'3; // => matched to `AlbumNotFound`
    http:NotFound|AlbumNotFound res = check albumClient->/albums/'3; // => matched to `http:NotFound`

2. The payload binding will return an anydata type for a response which does not have a 4XX or 5XX status code (Contradiction: The anydata returned from a resource will be converted to the default response with respect to the resource method). So if there are two successful responses with same/different payload structure, data binding will work for those payload structures.

   • For example consider the scenario when there is a two record types for the following successful responses: 201 Created - CreatedAlbum and 202 Accepted - AcceptedAlbum

      // This works
      CreatedAlbum|AcceptedAlbum albumResponse = check albumClient->/albums.post(album);

   • Lets consider the status code response as well: 201 Created - AlbumCreatedRes and 202 Accepted - AlbumAcceptedRes

      // This will work with this proposal
      AlbumCreatedRes|AlbumAcceptedRes res = check albumClient->/albums.post(album);

   • The conflict occur if they use the union mix of the record types and status code response types.

     CreatedAlbum|AlbumCreatedRes|AlbumAcceptedRes|AcceptedAlbum res = check albumClient->/albums.post(album);

     Possible solution is to destructure the target type into three type one with all the anydata types, status code response types and normal http:Response. First try the status code response/response binding and if it fails then try to bind the data to the provided anydata types and if it fails return as http:Response(if that type exist otherwise it is an error) . This way this status code response binding is not breaking the existing behaviour. We are checking against the status code responses first since the different status code responses can have the same body type. So the status code response has the most relevant representation.

3. The construction of the status code response should consider the following:

   • body - should be cloned with type defined in the status code response type
   • headers - should be a narrowed projection of the headers with parsing the header string value with the header type defined in the status code response type
   • media-type - should be extracted from the content-type header
   • construct the status code response with the above values and add the appropriate status code object field - this is only possible in the Java native side since we need to resolve the extract type to map if there is a union type provided as a target type(as discussed in the first two points)

Dependencies

This implementation is required for the following proposal in the OpenAPI tool:

• [Proposal] Improve the return type for client generation to handle error status code details and extra details #6099

[TharmiganK]

With this PR: ballerina-platform/module-ballerina-http#1954, the above status code binding feature is moved to a separate client: http:StatusCodeClient. This is to have a clear separation from normal data-binding and the special status code data-binding.