1478 - ERROR_CLUSTER_BACKEND_UNAVAILABLE

[haseebnaseem]

I am running arangodb cluster on kubernetes on AWS. One error frequently occurs in the cluster that does not get solved unless all of the pods are restarted. The error message is 1478 - ERROR_CLUSTER_BACKEND_UNAVAILABLE . We have not been able to reproduce the error to get to the cause of the error.

[dothebart]

Hi,
did you already try https://github.com/arangodb/kube-arangodb ?
https://arangodb.com/2018/03/arangodb-operator-kubernetes-stateful-cluster-deployments/ has the details on how to work with it.

[haseebnaseem]

I did have a look at it but the doc itself said not ready for production use so that ruled it out.

…

On Fri, 27 Apr 2018, 17:16 Wilfried Goesgens, ***@***.***> wrote: Hi, did you already try https://github.com/arangodb/kube-arangodb ? https://arangodb.com/2018/03/arangodb-operator-kubernetes-stateful-cluster-deployments/ has the details on how to work with it. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#47 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJ1EQBNVjNcLavHJurCgUI_sGsaEtCejks5tswwZgaJpZM4TqM6M> .

[dothebart]

I guess then you should switch to DC/OS instead.

[haseebnaseem]

Is this a known bug ?

[dothebart]

Hi,
if you want to have a resilient cluster setup, you can achieve that in DC/OS. This is pretty well tested and confirmed to be working stable. If instances fail, they're restarted properly etc. Upscaling and all that works properly. DC/OS templates are well available for AWS. ArangoDB can be installed via the DC/OS Appstore.

If you don't want to use DC/OS the next supported way is using the ArangoDB Starter to orchestrate your docker containers to form and maintain a cluster.

Kubernetes support as you read it is implementation ready as far as running a cluster. It can also up/downscale the cluster. It should handle forming and running the cluster well. It handles all the pods that should be available, their storage and networking needs.

The error ERROR_CLUSTER_BACKEND_UNAVAILABLE will be thrown if the database components can't reach each other. However, it can also mean that one of the DB-Servers doesn't respond to queries from the Coordinator on time.

DB-Servers may suffer from exhausting their I/O bandwith, which may lead to running out of Burst-credits on AWS. This may be resolveable by adding local SSDs or adding more RAM to them.

[gkarthiks]

Hi, I've opened an issue for the OpenShift cluster deployment.
arangodb/kube-arangodb#128

Is there any other workaround for cluster as of now? Appreciate any helps.

[dothebart]

Hi,
Openshift tends to behave different in some situations, which are incompatible with docker.
They have closed bugs about these differences "because of nobody cared".

Thus I can't recommend using openshift.

[gkarthiks]

@dothebart Agree, that OpenShift is different from any other cluster. But in our company, we are using only OpenShift. 😞

Please let me know if there's any progress on OpenShift ArangoDB

[dothebart]

@ewoutp and @neunhoef have worked to get openshift working. You should now be able to run this docker container directly or via kube-arangodb as stated in arangodb/kube-arangodb#128 - closing this issue.