Skip to content

jroper/akka-grpc-ddata-shopping-cart

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Akka ddata shopping cart KNative experiment

The goal of this project is to get an Akka Cluster working in KNative.

This basically takes the Akka Distributed Data Highly Available Shopping Cart, and places an Akka HTTP interface in front of it.

It configures the cluster formation using instructions found at the Lightbend OpenShift deployment guide, but with additional changes as described in this issue, which includes:

  • Always bind to 127.0.0.1
  • Use the pod service name for communication between pods
  • Configure ServiceEntry's for intra-pod communication
  • Don't specify health checks as these will prevent communication prior to readiness
  • Modify docker entry point to set the pod ip address to be a dash separated address for the purposes of constructing pod service names in configuration

Downing

Downing hasn't been addressed yet. It's CRDTs, and it does attempt to do quorum reads, with fallback to local reads if that fails, so we could just use auto downing, though that's not ideal.

One major problem at the moment is I think when a pod is stopped, both the sidecar and the pod are stopped simultaneously, and because the sidecar stops, this means that communication via Akka Remoting will likely fail before the node has a chance to gracefully leave the cluster, so every scale down or rolling upgraded does non graceful stops and requires a downing strategy. I'm not sure if there is any way around this, I can't see anything in Kubernetes that allows controlling the order of container shutdown within a pod. This might be a good reason to have Akka remoting traffic bypass the Istio sidecar - if that were possible.

Running in Minikube

So far, I have managed to get the project running on Minikube with Istio (I haven't yet attempted to run in KNative).

To get to where I've got, follow the instructions for installing Knative on Minikube up to Istio:

https://www.knative.dev/docs/install/knative-with-minikube/#installing-istio

Now build and deploy the docker image:

eval $(minikube docker-env)
sbt docker:publishLocal

And deploy the app:

kubectl apply -f shopping-cart.yaml

You should see two pods start, and if you check their logs, you should see them form a cluster with each other. The instructions I used to get to this point come from here:

akka/akka-management#209

To actually test out the service, set up the gateway (at this point I've got no idea what I'm doing, could be all wrong):

kubectl apply -f istio-ingress.yaml

Now if you run

$ minikube service istio-ingressgateway -n istio-system --url
http://192.168.39.149:31380
http://192.168.39.149:31390
http://192.168.39.149:31400
http://192.168.39.149:30092
http://192.168.39.149:30053
http://192.168.39.149:30318
http://192.168.39.149:31580
http://192.168.39.149:32354

You can see a bunch of URLs. One of them will be the correct URL for accessing the shopping cart. To know which, run:

$ kubectl get svc istio-ingressgateway -n istio-system
NAME                   TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)                                                                                                                   AGE
istio-ingressgateway   NodePort   10.105.81.196   <none>        80:31380/TCP,443:31390/TCP,31400:31400/TCP,15011:30092/TCP,8060:30053/TCP,853:30318/TCP,15030:31580/TCP,15031:32354/TCP   173m

So, I want port 80, its mapped to port 31380, so I can access my service on http://192.168.39.149:31380. Let's have a go (each command has echo appended to insert a newline, otherwise, things get weird):

$ curl http://192.168.39.149:31380/cart/mycart; echo
{"items":[]}
$ curl http://192.168.39.149:31380/cart/mycart/product/somebike -d 2 -H "Content-Type: application/json" -X PUT; echo
Item added
$ curl http://192.168.39.149:31380/cart/mycart; echo
{"items":[{"productId":"somebike","quantity":2}]}

Now you can try deleting pods and see that the data sticks around (delete them all and it doesn't, of course).

Next steps

Try and run the above but in KNative.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published