Functions with "type": "rest" type produce (type: 'Action')/r/nDetails:/r/nSystem.Net.Http.HttpRequestException: Cannot assign requested address (localhost:5296)

[Kounavi]

Hello.
I am trying to run this workflow:

{
  "id": "invoke-services-async",
  "name": "Invoke Services Async",
  "version": "0.1.13",
  "specVersion": "0.8",
  "functions": [
    {
      "name": "Invoke Send Email Function",
      "operation": "http://localhost:5296/swagger/v1/swagger.json#customer",
      "type": "rest"
    }
  ],
  "states": [
    {
      "name": "Send Email",
      "type": "operation",
      "actions": [
        {
          "name": "Invoke Send Email Action",
          "functionRef": {
            "refName": "Invoke Send Email Function",
            "arguments": {
              "customer": "${ .customer }"
            }
          }
        }
      ],
      "end": true
    }
  ]
}

I have a ASP .NET Core REST API (OpenAPI compatible) that reads and exposes this JSON via a GET:

{
    "customer": {
        "name": "Iraklis",
        "surname": "D.",
        "from": "[email protected]",
        "to": ["[email protected]", "[email protected]"],
        "cc": ["[email protected]"],
        "subject": "Hello everyone :)",
        "body": ["This is a test email", "", "--", "Kind regards,", "Iraklis D."]
    }
}

Service runs on http://localhost:5296

However, every time I try and run the workflow the workflow is in a faulted state with this output:

{
  "code": "HttpRequest",
  "message": "Cannot assign requested address (localhost:5296)"
}

Logs show this:

Sending HTTP request GET http://localhost:5295/swagger/v1/swagger.json
[14:38:24] warning: Synapse.Worker.Services.Processors.OpenApiFunctionProcessor[0]
An error occured while executing the activity '5ec6dd91-7e37-41cc-8b6c-d8a04f502362' (type: 'Action')/r/nDetails:/r/nSystem.Net.Http.HttpRequestException: Cannot assign requested address (localhost:5296)
---> System.Net.Sockets.SocketException (99): Cannot assign requested address

What am I doing wrong?

I have made certain that the function URL contains the proper operation id based on my swagger.json but the error seems unrelated.
Any ideas? Is this a k8s thing and I need to expose a port or something? Something else?

My swagger.json is as follows:

{
  "openapi": "3.0.1",
  "info": {
    "title": "WorkflowEmail",
    "version": "1.0"
  },
  "paths": {
    "/api/Email/Customer": {
      "get": {
        "tags": [
          "Email"
        ],
        "operationId": "customer",
        "responses": {
          "200": {
            "description": "Success",
            "content": {
              "text/plain": {
                "schema": {
                  "$ref": "#/components/schemas/Email"
                }
              },
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/Email"
                }
              },
              "text/json": {
                "schema": {
                  "$ref": "#/components/schemas/Email"
                }
              }
            }
          },
          "404": {
            "description": "Not Found",
            "content": {
              "text/plain": {
                "schema": {
                  "$ref": "#/components/schemas/ProblemDetails"
                }
              },
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/ProblemDetails"
                }
              },
              "text/json": {
                "schema": {
                  "$ref": "#/components/schemas/ProblemDetails"
                }
              }
            }
          }
        }
      }
    }
  },
  "components": {
    "schemas": {
      "Customer": {
        "type": "object",
        "properties": {
          "name": {
            "type": "string",
            "nullable": true
          },
          "surname": {
            "type": "string",
            "nullable": true
          },
          "from": {
            "type": "string",
            "nullable": true
          },
          "to": {
            "type": "array",
            "items": {
              "type": "string"
            },
            "nullable": true
          },
          "cc": {
            "type": "array",
            "items": {
              "type": "string"
            },
            "nullable": true
          },
          "subject": {
            "type": "string",
            "nullable": true
          },
          "body": {
            "type": "array",
            "items": {
              "type": "string"
            },
            "nullable": true
          }
        },
        "additionalProperties": false
      },
      "Email": {
        "type": "object",
        "properties": {
          "customer": {
            "$ref": "#/components/schemas/Customer"
          }
        },
        "additionalProperties": false
      },
      "ProblemDetails": {
        "type": "object",
        "properties": {
          "type": {
            "type": "string",
            "nullable": true
          },
          "title": {
            "type": "string",
            "nullable": true
          },
          "status": {
            "type": "integer",
            "format": "int32",
            "nullable": true
          },
          "detail": {
            "type": "string",
            "nullable": true
          },
          "instance": {
            "type": "string",
            "nullable": true
          }
        },
        "additionalProperties": { }
      }
    }
  }
}

Any kind of assistance, suggestion and / or indication would be very much appreciated! 😄

[cdavernas]

Hey @Kounavi!
Your problem is indeed related to K8s and/or Docker, that reserve the localhost hostname.
You should replace localhost in your operation definition by the pod/service name of your API, allowing Synapse to contact your service using the k8s DNS from within the cluster.
Hope this helps!

...
 "functions": [
    {
      "name": "Invoke Send Email Function",
      "operation": "http://myapi/swagger/v1/swagger.json#customer",
      "type": "rest"
    }
  ]
...

[cdavernas]

I'm on PTO until tomorrow, I'll have a look as soon as I'm back and will share my deployments if that helps!

That being said, port 41387 is the HTTP/2 on which the GRPC runtime API is hosted, which is used by runner containers to update activities and workflows.

[Kounavi]

@cdavernas Apologies. Enjoy your PTO and please let me know when you can at your earliest convenience! :)

[Kounavi]

Hey @cdavernas. Do you have any deployments to share? 😄

[Kounavi]

@cdavernas So, it seems that for Kubernetes you need to provide the POD's exact IP address rather than localhost as localhost is tied to 127.0.0.1 which would effectively prevent traffic to the pod unless the correct combination of IP+port is used. I guess I could try binding PODs to 0.0.0.0 but I would have to research and figure how to do this. In the meantime I have solved my issue.
Now I am getting another error regarding my JSON file but it is irrelevant to this issue.

If you would like me to contribute in providing YAML files for development scenarios please let me know.
Happy to share these! :)

[cdavernas]

Yeah indeed! If you use Docker or K8s runtime, localhost will be the container/pod's loopback interface.
On another note, instead of IP addresses, you should imho prefer using the targeted service/container name.

[Kounavi]

@cdavernas Yeah, possibly I should follow your advice. I will have a further look into these but I was finally able to invoke a service and get back the response using the POD's IP and port:

{
  "id": "invoke-services-async",
  "name": "Invoke Services Async",
  "version": "0.1.13",
  "specVersion": "0.8",
  "functions": [
    {
      "name": "Invoke Send Email Function",
      "operation": "http://POD_IP:32296/swagger/v1/swagger.json#customer",
      "type": "rest"
    }
  ],
  "states": [
    {
      "name": "Send Email",
      "type": "operation",
      "actions": [
        {
          "name": "Invoke Send Email Action",
          "functionRef": {
            "refName": "Invoke Send Email Function",
            "arguments": {
              "customer": "${ .customer }"
            }
          }
        }
      ],
      "end": true
    }
  ]
}
```

The above one worked like a charm and got back the expected response. So awesome! 😄