Kafka event payload not passing to playbook

[darthVikes]

Trying to do a simple test using ansible EDA / Kafka but not getting the varibles sent through to the playbook.
Also, using a docker version of kafka setup like: https://developer.confluent.io/quickstart/kafka-docker/
then created eda-topic as in the quickstart guide url above.

ansible-rulebook -r kafkaServer.yaml -i inventory.yml --verbose

kafkaServer.yaml

• name: Read messages from a kafka topic and act on them
  hosts: all

  Define our source for events

  sources:

  • ansible.eda.kafka:
    host: localhost
    port: 9092
    topic: eda-topic
    group_id:

  Define the conditions we are looking for

  rules:

  • name: Say Hello
    condition: event.message == "Ansible is cool"

    Define the action we should take should the condition be met

    action:
    run_playbook:
    name: say-what.yml

say-what.yml

• name: say thanks
  hosts: localhost
  gather_facts: false
  tasks:

  • debug:
    msg: "{{ event | default('my friend') }}"

• name: say thanks
  hosts: localhost
  gather_facts: false
  tasks:

  • debug:
    msg: "Thank you, {{ event.sender | default('my friend') }}"

• name: say thanks
  hosts: localhost
  gather_facts: false
  tasks:

  • debug:
    msg: "Thank you, {{ event.test | default('my friend') }}"

---

when sending the mesage:

{"message":"Ansible is cool", "sender":"DV", "test":"testing", "first":"darth", "last":"vikes"}

getting:

2023-03-09 17:56:31,038 - ansible_rulebook.app - INFO - Starting sources
2023-03-09 17:56:31,039 - ansible_rulebook.app - INFO - Starting rules
2023-03-09 17:56:31,039 - ansible_rulebook.engine - INFO - run_ruleset
2023-03-09 17:56:31,663 - ansible_rulebook.engine - INFO - ruleset define: {"name": "Read messages from a kafka topic and act on them", "hosts": ["all"], "sources": [{"EventSource": {"name": "ansible.eda.kafka", "source_name": "ansible.eda.kafka", "source_args": {"host": "localhost", "port": 9092, "topic": "eda-topic", "group_id": null}, "source_filters": []}}], "rules": [{"Rule": {"name": "Say Hello", "condition": {"AllCondition": [{"EqualsExpression": {"lhs": {"Event": "message"}, "rhs": {"String": "Ansible is cool"}}}]}, "actions": [{"Action": {"action": "run_playbook", "action_args": {"name": "say-what.yml"}}}], "enabled": true}}]}
2023-03-09 17:56:31,677 - ansible_rulebook.engine - INFO - load source
2023-03-09 17:56:32,409 - ansible_rulebook.engine - INFO - load source filters
2023-03-09 17:56:32,410 - ansible_rulebook.engine - INFO - Calling main in ansible.eda.kafka
2023-03-09 17:56:32,410 - aiokafka.consumer.subscription_state - INFO - Updating subscribed topics to: frozenset({'eda-topic'})
2023-03-09 17:56:32,412 - ansible_rulebook.engine - INFO - Waiting for all ruleset tasks to end
2023-03-09 17:56:32,412 - ansible_rulebook.rule_set_runner - INFO - Waiting for actions on events from Read messages from a kafka topic and act on them
2023-03-09 17:56:32,412 - ansible_rulebook.rule_set_runner - INFO - Waiting for events, ruleset: Read messages from a kafka topic and act on them
2023-03-09 17:56:32 414 [drools-async-evaluator-thread] INFO org.drools.ansible.rulebook.integration.api.io.RuleExecutorChannel - Async channel connected
2023-03-09 17:56:32,421 - aiokafka.consumer.group_coordinator - INFO - Metadata for topic has changed from {} to {'eda-topic': 1}.
2023-03-09 17:56:36 380 [main] INFO org.drools.ansible.rulebook.integration.api.rulesengine.RegisterOnlyAgendaFilter - Activation of effective rule "Say Hello" with facts: [Event DROOLS_PROTOTYPE with values = {test=testing, last=vikes, sender=DV, message=Ansible is cool, first=darth}]
2023-03-09 17:56:36,391 - ansible_rulebook.rule_generator - INFO - calling Say Hello
2023-03-09 17:56:36,392 - ansible_rulebook.rule_set_runner - INFO - call_action run_playbook
2023-03-09 17:56:36,392 - ansible_rulebook.rule_set_runner - INFO - substitute_variables [{'name': 'say-what.yml'}] [{'event': {'test': 'testing', 'last': 'vikes', 'sender': 'DV', 'message': 'Ansible is cool', 'first': 'darth'}}]
2023-03-09 17:56:36,392 - ansible_rulebook.rule_set_runner - INFO - action args: {'name': 'say-what.yml'}
2023-03-09 17:56:36,392 - ansible_rulebook.builtin - INFO - running Ansible playbook: say-what.yml
2023-03-09 17:56:36,395 - ansible_rulebook.builtin - INFO - ruleset: Read messages from a kafka topic and act on them, rule: Say Hello
2023-03-09 17:56:36,395 - ansible_rulebook.builtin - INFO - Calling Ansible runner

PLAY [say thanks] **************************************************************

TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": "my friend"
}

PLAY [say thanks] **************************************************************

TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": "Thank you, my friend"
}

PLAY [say thanks] **************************************************************

TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": "Thank you, my friend"
}

PLAY RECAP *********************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
2023-03-09 17:56:38,060 - ansible_rulebook.builtin - INFO - Ansible Runner Queue task cancelled
2023-03-09 17:56:38,061 - ansible_rulebook.builtin - INFO - Playbook rc: 0, status: successful

ansible-rulebook --version
version = '0.11.0'
Executable location = /usr/local/bin/ansible-rulebook
Drools_jpy version = 0.2.5
Java home = /usr/lib/jvm/java-17-openjdk-17.0.6.0.10-3.el9_1.x86_64
Java version = 17.0.6
Python version = 3.9.14 (main, Nov 7 2022, 00:00:00) [GCC 11.3.1 20220421 (Red Hat 11.3.1-2)]

[mkanoor]

@darthVikes In your playbook you need to access variables using the ansible_eda prefix. Like shown here https://github.com/ansible/ansible-rulebook/blob/c7cccce5d8d011dec5c912c479f965e0299ea59d/tests/playbooks/hello_events.yml#L8

[darthVikes]

Ok that helped! Thanks!

I changed it to this and seemed to fix it.

It would be great if there was specific examples in the documentation for each plugin type to test it. I didn't realize the event based test ws what I was looking for.

Thanks again!

say-what.yml

• name: say thanks
  hosts: localhost
  gather_facts: false
  tasks:

  • debug:
    msg: "{{ ansible_eda.event | default('fail') }}"

• name: debug event
  hosts: localhost
  gather_facts: false
  tasks:

  • debug:
    msg: "{{ ansible_eda.event.message | default('fail') }}"

• name: debug event
  hosts: localhost
  gather_facts: false
  tasks:

  • debug:
    msg: "{{ ansible_eda.event.first | default('fail') }}"

• name: debug event
  hosts: localhost
  gather_facts: false
  tasks:

  • debug:
    msg: "{{ ansible_eda.event.last | default('fail') }}"

• name: debug event
  hosts: localhost
  gather_facts: false
  tasks:

  • debug:
    msg: "{{ ansible_eda.event.sender | default('fail') }}"

• name: debug event
  hosts: localhost
  gather_facts: false
  tasks:

  • debug:
    msg: "{{ ansible_eda.event.test | default('fail') }}"

[darthVikes]

Oops. I'll leave it open as maybe you want to do something further with this. aka do additional work on documentation/example etc.

[Alex-Izquierdo]

There are some places in the documentation where the use of ansible_eda prefix is registered but no one explicit.
We should create a new subsection in https://ansible-rulebook.readthedocs.io/en/stable/variables.html , accessing-variables-in-your-playbook for example.

[darthVikes]

That sounds good Alex. Think of someone that hasn't implmented it before, if tehre was a specific like use case example. aka For Ansible eda with Kafka example, here is what a rulebook would be, the playbook, how to do variables, and what the json message might be.