Service mesh appmesh (signalfx#28)

* Get host ip from EC2 service

* ECommerce App Mesh demo

* Add lead-in for App Mesh

* Add App Mesh to main list

* Add link to App Mesh product page and example

* Add refreshdb resources

README.md

@@ -107,6 +107,8 @@ language/platform.
     with Istio tracing using the SignalFx adapter and Envoy tracing.
 - [Envoy](./envoy): the Envoy proxy can be configured to report trace spans to
   SignalFx.
+- [AWS App Mesh](./service-mesh/appmesh): An example E-Commerce application with
+  deployment and configuration files for App Mesh on ECS.
 
 ### AWS Lambda Functions
 

service-mesh/README.md

@@ -6,3 +6,5 @@ source is viewable at [app-src](./app-src).
 
 - [Istio](https://istio.io/):
   - [E-Commerce on Istio](./istio)
+- [AWS App Mesh](https://aws.amazon.com/app-mesh/)
+  - [E-Commerce on App Mesh](./appmesh)

service-mesh/app-src/apps/Dockerfile

@@ -7,6 +7,7 @@ RUN mkdir /app/${APP}
 RUN touch /app/${APP}/__init__.py
 COPY demo/${APP}.py /app/${APP}/
 COPY demo/utils.py /app/${APP}/
+COPY startup.sh /app/startup.sh
 WORKDIR /app/
 
-CMD python -m ${APP}.${APP}
+CMD ./startup.sh

service-mesh/app-src/apps/demo/payment.py

@@ -29,19 +29,6 @@ def process():
         processor = 'PayPal'
         s.span.set_tag('processor', processor)
         s.span.set_tag('transactionId', transactionId)
-        s.span.set_tag('error', 'true')
-
-        # Fake API request to a payment service
-        payment_api_tags={
-                'span.kind': 'CLIENT',
-                'http.method': 'POST',
-                'http.url': 'https://api.paypal.com/payment',
-                'http.status_code': 504,
-                'error': 'true',
-                'message': 'Request timeout'
-                }
-        with tracer.start_active_span('http.post https://api.paypal.com/payment', tags=payment_api_tags, finish_on_close=True) as child_scope:
-            time.sleep(int(os.getenv('DELAY', 0)))
 
     return jsonify({
         'transactionId': transactionId,

service-mesh/app-src/apps/demo/utils.py

@@ -8,7 +8,7 @@
 
 def get_ingest():
     endpoint = os.getenv('SIGNALFX_ENDPOINT_URL', 'https://ingest.signalfx.com')
-    host_ip = os.getenv('POD_HOST_IP')
+    host_ip = os.getenv('POD_HOST_IP', '0.0.0.0')
 
     endpoint = endpoint.replace('POD_HOST_IP', host_ip)
 

service-mesh/app-src/apps/startup.sh

@@ -0,0 +1,5 @@
+#!/bin/sh
+
+export POD_HOST_IP=${POD_HOST_IP:-$(curl http://169.254.169.254/latest/meta-data/local-ipv4)}
+
+python -m ${APP}.${APP}

service-mesh/app-src/base/Dockerfile

@@ -1,7 +1,7 @@
 FROM ubuntu:18.04
 
 RUN apt-get update -y && \
-    apt-get install -y python-pip python-dev
+    apt-get install -y python-pip python-dev curl
 
 COPY . /app/
 WORKDIR /app/

service-mesh/appmesh/README.md

@@ -0,0 +1,86 @@
+# E-Commerce on AWS App Mesh
+
+App Mesh is a new service mesh offering from AWS. They provide a managed control
+plane so that users are only required to deploy a simple data plane with Envoy.
+A brief introduction to the core ideas can be read in their [documentation](https://docs.aws.amazon.com/app-mesh/latest/userguide/what-is-app-mesh.html).
+
+The deployment scaffolding of this example has been adopted, in large part, from
+the [AWS App Mesh Examples](https://github.com/aws/aws-app-mesh-examples).
+
+Build the application images used for this example by following the instructions at [../app-src](../app-src).
+
+## Setup
+
+Set the following environment variables to configure the infrastructure deployment:
+
+```
+AWS_PROFILE
+AWS_REGION
+AWS_DEFAULT_REGION
+ENVIRONMENT_NAME
+MESH_NAME
+KEY_PAIR_NAME
+ENVOY_IMAGE
+CLUSTER_SIZE
+SERVICES_DOMAIN
+DOCKER_REPO
+```
+
+Deploy the example VPC and ECS cluster, and App Mesh:
+
+```bash
+infra/vpc.sh
+infra/ecs-cluster.sh
+infra/appmesh-mesh.sh
+```
+
+Set an environment variable, `SIGNALFX_ENDPOINT_URL` for the trace endpoint. If sending directly to a Gateway, it would look similar to this:
+
+```bash
+export SIGNALFX_ENDPOINT_URL=http://gateway.servicesdomain:8080
+```
+
+If sending to an Agent running on the task host instead, substitute the hostname
+with the literal, `POD_HOST_IP`.
+
+```bash
+export SIGNALFX_ENDPOINT_URL=http://POD_HOST_IP:9080
+```
+
+This will make Envoy and the apps send traces to the host at port `9080`.
+
+Deploy the App Mesh virtual nodes, routers, routes, and services with the provided
+Cloudformation template. This template has an included virtual node and virtual
+service to handle egress to the SignalFx Gateway, if sending directly to that.
+Set two environment variables, `GATEWAY_URL` and `GATEWAY_PORT`, `GatewayVirtualService`
+and `GatewayVirtualNode` to match the actual Gateway deployment.
+
+```bash
+export GATEWAY_URL=gateway.${SERVICES_DOMAIN}
+export GATEWAY_PORT=8080
+./ecommerce-mesh-resources.sh
+```
+
+Build the config image and push it to the docker repository. The startup script
+will read the value of `SIGNALFX_ENDPOINT_URL` and populate the Envoy
+configuration accordingly.
+
+```bash
+cd config-image
+./build.sh
+```
+
+## Running the example
+
+Deploy and start the app using the adapted deployment script:
+
+```bash
+./ecs/ecommerce-app.sh
+```
+
+By default, all routing is done through `catalog-v1`. To modify this and enable
+`catalog-v2`, edit the `CatalogRoute` in `ecommerce-mesh-resources.yaml` and
+uncommment the second weighted target. Various weights can be assigned to each
+node.
+
+Apply the modified route with `ecommerce-mesh-resources.sh`.

service-mesh/appmesh/config-image/Dockerfile

@@ -0,0 +1,18 @@
+FROM alpine
+
+ARG ENDPOINT_URL
+ENV ENDPOINT=$ENDPOINT_URL
+
+RUN apk add --update \
+    curl \
+    && rm -rf /var/cache/apk/* 
+
+ADD tracing_envoy.yaml /envoy_config/tracing.yaml
+ADD startup.sh /startup.sh
+
+RUN chmod +x /startup.sh
+
+VOLUME /envoy_config
+
+CMD /startup.sh $ENDPOINT
+# ENTRYPOINT [ "/bin/sh", "-c", "sed", "-i", "'s/@@HOST_IP@@/$(curl http://169.254.169.254/latest/meta-data/local-ipv4)/g'", "/tmp/envoy.yaml" ]

service-mesh/appmesh/config-image/build.sh

@@ -0,0 +1,5 @@
+#!/bin/bash
+
+docker build ./ -t ecommerce-envoy-config:latest --build-arg ENDPOINT_URL=${SIGNALFX_ENDPOINT_URL}
+docker tag ecommerce-envoy-config:latest ${DOCKER_REPO}:config
+docker push ${DOCKER_REPO}:config

service-mesh/appmesh/config-image/startup.sh

@@ -0,0 +1,15 @@
+#!/bin/sh
+
+set -ex
+
+export HOST_IP=$(curl http://169.254.169.254/latest/meta-data/local-ipv4)
+
+echo $HOST_IP
+echo $1
+
+endpoint_url=$(echo $1 | sed ' s/\/\///g; s/POD_HOST_IP/'"${HOST_IP}"'/g' | awk '{split($0,a,":"); print a[2]}')
+endpoint_port=$(echo $1 | awk '{split($0,a,":"); print a[3]}')
+
+sed -i 's/SIGNALFX_ENDPOINT_URL/'"${endpoint_url}"'/g; s/SIGNALFX_ENDPOINT_PORT/'"${endpoint_port}"'/g' /envoy_config/tracing.yaml
+
+cat /envoy_config/tracing.yaml

service-mesh/appmesh/config-image/tracing_envoy.yaml

@@ -0,0 +1,17 @@
+static_resources:
+  clusters:
+  - name: zipkin
+    connect_timeout: 2s
+    type: LOGICAL_DNS
+    lb_policy: ROUND_ROBIN
+    hosts:
+    - socket_address:
+        address: SIGNALFX_ENDPOINT_URL
+        port_value: SIGNALFX_ENDPOINT_PORT
+
+tracing:
+  http:
+    name: envoy.zipkin
+    config:
+      collector_cluster: zipkin
+      collector_endpoint: "/v1/trace"

service-mesh/appmesh/ecommerce-mesh-resources.sh

@@ -0,0 +1,17 @@
+#!/bin/bash
+
+set -ex 
+
+DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"
+
+aws --profile "${AWS_PROFILE}" --region "${AWS_DEFAULT_REGION}" \
+    cloudformation deploy \
+    --stack-name "${ENVIRONMENT_NAME}-mesh-resources" \
+    --capabilities CAPABILITY_IAM \
+    --template-file "${DIR}/ecommerce-mesh-resources.yaml"  \
+    --parameter-overrides \
+    EnvironmentName="${ENVIRONMENT_NAME}" \
+    ServicesDomain="${SERVICES_DOMAIN}" \
+    AppMeshName="${MESH_NAME}" \
+    GatewayURL="${GATEWAY_URL}" \
+    GatewayPort="${GATEWAY_PORT}" \