New Feature - Advanced Request Aggregation

[maartenmensink]

First of all thanks for the effort you guys put in the awesome project!

New Feature: Advanced Request Aggregation

A more advanced version of the Request Aggregation Feature to allow smarter composition of multiple request and or subsequent request.

Motivation for New Feautre

One of the problems developers face when developing applications that talks to microservices is that you will get a chatty client(lots of little request). By using a [api gateway(http://microservices.io/patterns/apigateway.html) we can work towards a optimized api for each client. To further support the effort for getting a optimized a api gateway should have a feature as API Composition which you guys made in to the awesome feature of Request Aggregation

Proposal

Currently there is support for creating a aggregate route by defining keys on routes and aggregating the keys into on route. It would be awesome if we could provide more intelligence to the processing of the route.

Perhaps a way of adding a config option to the aggregate route that will instantiate a instance of a IAggregateRouteComposer. Which would get the input params of the route

public interface IAggregateRouteComposer
{
    Task<object> Process(object input);
}

public class CustomAggregateRouteComposer : IAggregateRouteComposer
{
    public Task<object> Process(object input)
    {
        //compose response
     }
}

{
    "ReRoutes": [
        {
            "DownstreamPathTemplate": "/customer",
            "UpstreamPathTemplate": "/customer/{customerId}",
            "UpstreamHttpMethod": [
                "Get"
            ],
            "DownstreamScheme": "http",
            "DownstreamHostAndPorts": [
                {
                    "Host": "localhost",
                    "Port": 51881
                }
            ],
            "Key": "Customer"
        },
        {
            "DownstreamPathTemplate": "/orders",
            "UpstreamPathTemplate": "/orders",
            "UpstreamHttpMethod": [
                "Get"
            ],
            "DownstreamScheme": "http",
            "DownstreamHostAndPorts": [
                {
                    "Host": "localhost",
                    "Port": 51882
                }
            ],
            "Key": "Orders"
        }
    ],
    "Aggregates": [
        {
            "Composer": "CustomAggregateRouteComposer",
            "UpstreamPathTemplate": "/product-detail/{productId}"
        }
    ]
}

[TomPallister]

@maartenmensink thank you very much for your interest in the project.

I agree with you, we need a better / smarter way to do request aggregation.

I like your approach so let's leave this issue open and I will get to it at some point. If you would like to have a go yourself I am very happy to accept PRs and support / advise in the implementation.

Once again thank you very much! 😄

[TomPallister]

I think I will make it so you have to configure the aggregates and reroutes but you will be able to provide the implementation that maps the reroutes into the reponse. Interface might be something like..

public Task<HttpResponseMessage> Map(List<HttpResponseMessage> originalResponses)
{
        //do stuff with original responses
       return new HttpResponseMessage() { Content = asdf };
}

This means you can even aggregate the headers you want returning etc...

[maartenmensink]

I think if we have a way to configure all scenario's this would be a great way to quickly expand the gateway api. If we could support scenarios like the following scenario.

Customer is placing a order at webshop. Customer retrieves the basket and you want to return product suggestions for all the products in the current basket.

/basket

gives you the current basket

{
	"lines":[{
		"productId": 1
	},
	{
		"productId": 2
	}]
}

/suggestions?productId=1&productId2

gives you the suggestion for the your basket.

{
	"suggestions":[{
		"productId": 1
	},
	{
		"productId": 2
	}]
}

We would need a way of expressing output params and be able to provide them as input params

/basket => productIds = lines.Select(p => p.productId)
/suggestions <= productIds

[TomPallister]

Ive started hacking around in #310 will try make this nicer...it would allow you to do the scenario above but is clunky...for me there is tradeoff between clunky + what a user needs to hand roll themselves vs cleaner, less hand roll, less functionality.

[TomPallister]

@maartenmensink I have thought about the feature you need and can't think of a better way to implement it that GraphQL.

To clarify in your scenario you want to retrieve the customers basket and use the products in the basket offer them suggestions.

Ocelot doesn't really have a concept of proper types, it just pushes http (strings) around really. In order for Ocelot to be able to get the basket, parse it, then get the suggestions would require a major change in Philosophy. I just want Ocelot to be a reverse proxy with a few extra features. GraphQL is doing a much better job than I could do for that :)

Anyway I recently added a sample to show how you can easily integrate GraphQL with Ocelot and this might help you out, it is here.

In addition to this Microsoft are looking into this problem in their eShopOnContainers project, the branch is here. The approach is to use Ocelot as an API gateway to handle cross cutting concerns but pass the request on to an aggregating service that would handle the basket / suggestions logic above.

I hope this makes sense and I would like to close the issue now! Please let me know if you have an idea and we can reopen.

[adamlepkowski]

@maartenmensink How did you solve your problem with basket and suggestions? I have similar issue right now.

[maartenmensink]

@adamlepkowski At this moment we have dedicated WebApi Controllers that handle request that handle data composition. This way we can handle complex cases and later on create the right plumbing to handle it genericly.