Request aggregation

[peerpalo]

What should we add or change to make your life better?

Ocelot has a very useful feature called request aggregation, basically allows to send multiple GETs to different routes and aggregate them in a single object. The idea is to create a cluster with multiple destinations to same route and add an "aggregation policy" (load balancing would not be used in that case).

E.g.

{
  "ReverseProxy": {
    "Routes": {
      "route1": {
        "ClusterId": "cluster1",
        "Match": {
          "Path": "/products"
        }
      }
    },
    "Clusters": {
      "cluster1": {
        "Destinations": [
          {
            "cluster1/destination1": {
              "Address": "https://api1.com/products"
            }
          },
          {
            "cluster1/destination2": {
              "Address": "https://api2.com/products"
            }
          }

        ],
        "AggregatePolicy": "Merge"
      }
    }
  }
}

Should produce a result set like this

{
  "cluster1/destination1": [
    {
      "id": 1
    },
    {
      "id": 3
    }
  ],
  "cluster1/destination2": [
    {
      "id": 5
    }
  ]
}

Why is this important to you?

In our microservice structure we have the same service across various server around the world because of our business intelligence aggregator. We would like to have a unique api access point for specific data so we can merge and decorate our objects before give them to client.

Looking inside code, ForwarderMiddleware and HttpForwarder needs to accept multiple destinations, ProxiedDestination inside IReverseProxyFeature needs also to be renamed to ProxiedDestinations with all of the consequences of that.

[samsp-msft]

We are aware of most of the features that Ocelot has in this area - and made a conscious decision to not include direct support for them in YARP. A lot of the reasons is around performance - to enable this kind of functionality YARP would need to be intelligent about the contents of request/response bodies.

To me, what is unclear is how much value being able to do this kind of thing with configuration really is? The code for each of these individual operations isn't substantial, and API endpoints/routes can be mixed with YARP as part of the overall ASP.NET route table. I suspect if we try to solve these with config, that the scope of what people want to do is substantial - so where would we draw the line - the potential for scope creep is large?

What of the other capabilities of YARP would you want? is it the cluster/load balancing/health? If we had an API to select a destination from a cluster meeting the other rules - would that be sufficient?

[alrz]

to enable this kind of functionality YARP would need to be intelligent about the contents of request/response bodies.

would it make sense to add consecutive responses in the output stream regardless of the content, that is, the same way streaming works in grpc json transcoding?

[peerpalo]

Well, you're right, project name is "Yet Another Reverse Proxy", not "Yet Another API Gateway". To be honest, I totally agree about configuration or code question. In fact I tried to implement a custom middleware to handle aggregation and, well, it works.

https://gist.github.com/peerpalo/b83598e2c98413a6f5af92fe4800985b

But as you can see we have some problems

1. If we put LoadBalancingMiddleware before (because we need it for normal calls) AvailableDestinations will be always reduced to one destination
2. AggregationMiddleware is not using HttpForwarder, we got plenty of code duplication
3. It doesn't really "fit well" in the entire pipeline (can't access to same log formats, eventids, ForwarderErrorFeature, etc etc...)

What can be improved? In my opinion:

• Refactor HttpForwarder so it can be nicely inherited and only the effective "send" part can be enhanced (no more code duplication)
• Allow developers to improve (or change) logic behind ForwarderMiddleware

[samsp-msft]

I wasn't expecting the gateway scenarios to be written as a proxy middleware, more as an explicit endpoint using

app.MapGet("/path/to/map", MyHandler);

Task MyHandler(HttpContext context)
{
  // get addresses for outbound calls
  // make calls
  // aggregate results
}

The part from the proxy that would be interesting is how to get the addresses of the destinations for outbound calls. We have talked about having something in HttpClient that does something similar to the proxy in terms of managing a collection of destinations and balancing calls between them, handling heath checks etc. This is needed for services like gRPC where you probably have more than one destination server for a request, and want to be smart about spreading load, and handling downtime.

For your scenario - do you have other routes that the proxy will be handing that don't do the request aggregation and are just passthrough, or is this the main scenario?

I was thinking if we had something like:

async Task MyHandler(HttpContext context)
{
    var pf = context.GetReverseProxyFeature();
    var cluster = pf.Clusters["cluster1"];
    var dest = cluster.NextDestination();
    var client = cluster.GetHttpClient();

    await client.GetAsync($"https://{dest.Model.Address}/Service1");
    ...
}

So this would retrieve the next applicable destination, and an HttpClient (wrapper) based on the YARP config. Would that be the pieces that you need?

[maxiptah]

@samsp-msft As I understand what you propose here was never implemented, right? Through context I can get the current cluster, but not the list of available clusters.
I still do not understand how aggregation is supposed to work. In the YARP service I created an AggregationController, and I route in the YARP config to this controller for special routes that need info from multiple microservices. This works basically, but in the AggregationController I call the microservices myself and merge the result. Like this I effectively lose most YARP benefits, like load balancing etc.
Is there a recommendation on performing such a task? I understand it's not the purpose of YARP, but that a valid and common use case on top of YARP.

[Tratcher]

I understand it's not the purpose of YARP, but that a valid and common use case on top of YARP.

I don't understand how YARP applies here. At it's core, YARP forwards an incoming request to a destination, and then forwards back the response. It's a 1 to 1 relationship. Routes, clusters, load balancing, etc. are all helpers built on top of that used to pick the destination to forward to.

Taking a request, splitting it out to N microservice calls, and assembling the result is what an Application Gateway does, not a Reverse Proxy. There's not much overlap between the two models.

[peerpalo]

Yes, we have also other routes, in fact the idea is to have something like

https://gateway/products

that would return

{
  "destination1": [
    {
      "id": 1
    },
    {
      "id": 3
    }
  ],
  "destination2": [
    {
      "id": 5
    }
  ]
}

so client can be do calls like this

https://gateway/products/destination1/3
https://gateway/products/destination2/5

I wasn't expecting the gateway scenarios to be written as a proxy middleware

I was thinking if we had something like:

async Task MyHandler(HttpContext context)
{
    var pf = context.GetReverseProxyFeature();
    var cluster = pf.Clusters["cluster1"];
    var dest = cluster.NextDestination();
    var client = cluster.GetHttpClient();

    await client.GetAsync($"https://{dest.Model.Address}/Service1");
    ...
}

So this would retrieve the next applicable destination, and an HttpClient (wrapper) based on the YARP config. Would that be the pieces that you need?

Currently we wrote that as a middleware because it's the only way to have AvailableDestinations "cleaned" by health monitor (and other things) but yes, if we can implement a direct route handler having already all needed info (AvailableDestinations, HttpClient for current cluster, transforms, cache, etc, etc...) it would be a good solution!

[Kahbazi]

if we can implement a direct route handler having already all needed info (AvailableDestinations, HttpClient for current cluster, transforms, cache, etc, etc...) it would be a good solution!

Perhaps Similar to #1165?

[ardove]

I've stumbled upon this discussion several times now so I thought I'd leave a comment.

I'd vote in favor of getting request aggregation added as well. I currently work on a large application which doesn't use a reverse proxy (like Yarp) because the argument was always "it's not that difficult to write a WebApi controller which takes a dependency on one or more HttpClients that call downstream services". You're right, it's not.

The problem begins when engineers take a couple of properties from response 1 and a couple of other properties from response 2 and create a new, composite model which represents the merge operation of those two payloads. This is a problem because it increases coupling between the downstream services and the "request aggregator". Furthermore, it causes a code reuse problem which is very difficult to rationalize without reviewing tens of thousands of pull requests from engineers on multiple teams. People are more likely to create a new composite model with the properties they need because it's less work than figuring out if there's already a composite model out there which would support thier needs. Additionally, people become inclined to add other functionality that is not strictly related to request proxying -- perhaps a call to publish a message to a message broker. As time goes on, the boundaries become unclear about what the "request aggregator" is supposed to do or not do and what contexts it's safe to be called from.

I know, 20/20 in hindsight, but fast forward 10 years and I'm wishing we had a generalized component which did request aggregation naturally because it would've eliminated a collection of problems that are actually quite difficult to govern without a technology solution. Furthermore, we have an enterprise Kong license, but we're avoiding it because they don't support request aggregation (and also, because their extension hooks require Lua and we are a .NET shop). With the deprecation of Ocelot, we're in a bit of a pinch with respect to a real solution that is based on .NET. I know this is a reverse proxy, but my personal opinion is that a reverse proxy isn't quite complete without request aggregation and there's not really another .NET option out there.

In any case, I did manage to solve this with a specific endpoint route and I think the solution is fairly elegant (requires less than 100 lines of C# and configuration for the route aggregates that are interesting to you). I'm not sure I'm ready to give it the "looks good for production" stamp yet, but I only spent about an hour writing it and hopefully it helps the next reader.

appsettings.json

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  },
  "AllowedHosts": "*",
  "Proxy": {
    "Routes": {
      "api": {
        "ClusterId": "api",
        "Match": {
          "Path": "api/{**catch-all}"
        }
      }
    },
    "Clusters": {
      "api": {
        "Destinations": {
          "example.com": {
            "Address": "http://localhost:5000"
          }
        }
      }
    },
    "Aggregates": [
      {
        "route": "api/v1/routesAndSites",
        "destinations": [
          {
            "name": "routes",
            "address": "http://localhost:5000/api/v1/routes"
          },
          {
            "name": "sites",
            "address": "http://localhost:5000/api/v1/sites"
          }
        ]
      }
    ]
  }
}

Program.cs

using System.Text.Json.Nodes;

WebApplicationBuilder builder = WebApplication.CreateBuilder(args);

IConfigurationSection? proxyConfig = builder.Configuration.GetSection("Proxy");
builder.Services
    .AddReverseProxy()
    .LoadFromConfig(proxyConfig);

builder.Services.AddHttpClient();

WebApplication app = builder.Build();
app.MapReverseProxy();

RequestAggregateConfig[]? requestAggregationConfig = proxyConfig.GetSection("Aggregates").Get<RequestAggregateConfig[]>();
foreach (RequestAggregateConfig aggregate in requestAggregationConfig)
{
    app.MapGet(
        aggregate.Route,
        async context =>
        {
            var httpClientFactory = context.RequestServices.GetRequiredService<IHttpClientFactory>();

            // first, fire the requests
            var allResponses = new List<IntermediateResponse>();
            foreach (AggregateDestinationConfig dest in aggregate.Destinations)
            {
                HttpClient? httpClient = httpClientFactory.CreateClient(dest.Name);
                string address = dest.Address;

                // we only support GETs
                var req = new HttpRequestMessage(HttpMethod.Get, address);

                Task<HttpResponseMessage> responseTask = httpClient.SendAsync(req, CancellationToken.None);

                allResponses.Add(new IntermediateResponse
                {
                    Name = dest.Name,
                    Response = responseTask
                });
            }

            // next, build the payload
            var finalResult = new JsonObject();
            foreach (IntermediateResponse response in allResponses)
            {
                // todo: error handling
                HttpResponseMessage httpResponse = await response.Response;
                
                var content = await httpResponse.Content.ReadFromJsonAsync<JsonObject>();

                finalResult[response.Name] = content;
            }

            await context.Response.WriteAsJsonAsync(finalResult);
        });
}


app.Run();


public class IntermediateResponse
{
    public string Name { get; set; }
    public Task<HttpResponseMessage> Response { get; set; }
}

public class RequestAggregateConfig
{
    public string Route { get; set; }

    public AggregateDestinationConfig[] Destinations { get; set; }
}

public class AggregateDestinationConfig
{
    public string Name { get; set; }

    public string Address { get; set; }
}