feat(exporter): export records to a remote Hazelcast instance

[saig0]

Set the environment variable ZEEBE_HAZELCAST_REMOTE_ADDRESS or the exporter argument remoteAddress to connect to a remote Hazelcast instance/cluster.

Example configuration from docker-compose:

version: "2"

networks:
  zeebe_network:
    driver: bridge

services:
  zeebe:
    container_name: zeebe_broker
    image: camunda/zeebe:0.23.0
    environment:
      - ZEEBE_LOG_LEVEL=debug
      - ZEEBE_HAZELCAST_REMOTE_ADDRESS=hazelcast:5701
    ports:
      - "26500:26500"
      - "9600:9600"
    volumes:
      - ../exporter/target/zeebe-hazelcast-exporter-${EXPORTER_VERSION}-jar-with-dependencies.jar:/usr/local/zeebe/exporters/zeebe-hazelcast-exporter.jar
      - ./application.yaml:/usr/local/zeebe/config/application.yaml
    networks:
      - zeebe_network
    depends_on:
      - hazelcast

  hazelcast:
    container_name: hazelcast
    image: hazelcast/hazelcast:3.12.6
    ports:
      - "5701:5701"
    environment:
      - JAVA_OPTS="-Dhazelcast.local.publicAddress=hazelcast:5701"
    networks:
      - zeebe_network

closes #42

[kratosmat]

Hi all,
I am using zeebe with simple monitor and hazelcast exporter in a kubernetes env.
I have noticed that each node of the cluster start a different hazelcast server, so I am trying to use an external hazelcast and send data to it using remote address.

I thing there is a bug, I am not able to set remote address variable.
Can I have some help?

[saig0]

@kratosmat, sure!

The environment variable should be called ZEEBE_HAZELCAST_REMOTE_ADDRESS.

Please make sure that you're using the version 0.9.0 of the Hazelcast exporter. Let's create a new issue and provide all information that helps to analyze the behavior.

[mcusmai]

@saig0 thanks, it seems work, but there is something of strange.
I have installed an hazelcast external and a hazelcast management GUI:

version: "2"

networks:
    zeebe_network:
        driver: bridge

services:
    zeebe-0:
        container_name: zeebe_broker_0
        image: zeebe:0.23.1-hazelcast
        environment:
            - ZEEBE_LOG_LEVEL=debug
            - ZEEBE_NODE_ID=0
            - ZEEBE_PARTITIONS_COUNT=2
            - ZEEBE_REPLICATION_FACTOR=3
            - ZEEBE_CLUSTER_SIZE=3
            - ZEEBE_CONTACT_POINTS=zeebe-0:26502
            - ZEEBE_HAZELCAST_REMOTE_ADDRESS=hazelcast-server:5701
        volumes:
            - ./application.yaml:/usr/local/zeebe/config/application.yaml
        networks:
            - zeebe_network
        depends_on:
            - hazelcast-server
    zeebe-1:
        container_name: zeebe_broker_1
        image: zeebe:0.23.1-hazelcast
        environment:
            - ZEEBE_LOG_LEVEL=debug
            - ZEEBE_NODE_ID=1
            - ZEEBE_PARTITIONS_COUNT=2
            - ZEEBE_REPLICATION_FACTOR=3
            - ZEEBE_CLUSTER_SIZE=3
            - ZEEBE_CONTACT_POINTS=zeebe-0:26502
            - ZEEBE_HAZELCAST_REMOTE_ADDRESS=hazelcast-server:5701
        volumes:
            - ./application.yaml:/usr/local/zeebe/config/application.yaml
        networks:
            - zeebe_network
        depends_on:
            - hazelcast-server
    zeebe-2:
        container_name: zeebe_broker_2
        image: zeebe:0.23.1-hazelcast
        environment:
            - ZEEBE_LOG_LEVEL=debug
            - ZEEBE_NODE_ID=2
            - ZEEBE_PARTITIONS_COUNT=2
            - ZEEBE_REPLICATION_FACTOR=3
            - ZEEBE_CLUSTER_SIZE=3
            - ZEEBE_CONTACT_POINTS=zeebe-0:26502
            - ZEEBE_HAZELCAST_REMOTE_ADDRESS=hazelcast-server:5701
        volumes:
            - ./application.yaml:/usr/local/zeebe/config/application.yaml
        networks:
            - zeebe_network
        depends_on:
            - hazelcast-server
    hazelcast-server:
        container_name: hazelcast-server
        image: hazelcast/hazelcast:4.0.1
        environment:
            - JAVA_OPTS="-Dhazelcast.local.publicAddress=hazelcast-server:5701"
        networks:
            - zeebe_network
    hazelcast-management:
        container_name: hazelcast-management
        image: hazelcast/management-center:4.0.1
        ports:
            - "8083:8080"
        networks:
            - zeebe_network
        depends_on:
            - hazelcast-server
    monitor:
        container_name: zeebe_monitor
        image: camunda/zeebe-simple-monitor:0.19.0
        volumes:
            - ./simple-monitor-application.yaml:/app/resources/application.yaml
        ports:
            - "8082:8082"
        networks:
            - zeebe_network
        depends_on:
            - hazelcast-server

---

But as you can see it seems that only 2 node are attaching the hazelcast server.
Is it normal?

[saig0]

Looking at the docker-compose, this is expected. You configured the Zeebe cluster to have 2 partitions (ZEEBE_PARTITIONS_COUNT=2).

So, only 2 partitions are processing the workflows and exporting data to Hazelcast. The exporters are only active on the leaders of the partitions. On the followers, the exporters are not active.

[mcusmai]

@saig0 thanks a lot. If you think this docker compose could be useful for others please I will contribute to the project.

I would like to ask about other issue regarding simple monitor.
I notice a strange behaviour as you can see on image below.

The number of active instance on the image is 3 while they are 5 on the list.
Is it a bug of simplemonitor UI or there is a misconfiguration?

[saig0]

Interesting. Please open an issue in the simple monitor repository. If possible, please add the steps to reproduce this issue.

I updated the docker-compose.yml on the README page based on yours 👍 Thanks for sharing.