events/facts matching with non-literals strings

[alialkhalidi]

Please confirm the following

• I agree to follow this project's code of conduct.
• I have checked the current issues for duplicates.
• I understand that ansible-rulebook is open source software provided for free and that I might not receive a timely response.

Bug Summary

trying to figure how to parameterise a condition with the following rulebook, except that the commented line results in no match of the condition:

.
├── hosts.yml
├── local_copy.yml
├── playbook-create.yml
├── playbook-delete.yml
└── webhooks.yml

hosts.yml

all:
  children:
    local:
      hosts:
        localhost:
          ansible_connection: local

local_copy.yml

resource_list: []

playbook-create.yml

---
- hosts: local
  gather_facts: false
  vars:
    local_copy_file: "/tmp/local_copy.yml"
  tasks:
    - ansible.builtin.stat:
        path: "{{ local_copy_file }}"
      register: local_copy_stat

    - ansible.builtin.copy:
        dest: "{{ local_copy_file }}"
        content: "resource_list: []"
      when: local_copy_stat.stat.exists == false

    - ansible.builtin.include_vars:
        file: "{{ local_copy_file }}"
        name: observed

    - set_fact:
        cacheable: true
        active:
          resource_list: "{{ observed.resource_list + [ansible_eda.event.payload.resource_name] }}"

    - copy:
        dest: "{{ local_copy_file }}"
        content: "resource_list: {{ active.resource_list }}"

playbook-delete.yml

---
- hosts: local
  gather_facts: false
  vars:
    local_copy_file: "/tmp/local_copy.yml"
  tasks:
    - ansible.builtin.include_vars:
        file: "{{ local_copy_file }}"
        name: observed
    - set_fact:
        cacheable: true
        active:
          resource_list: "{{ observed.resource_list | difference([ansible_eda.events.m_0.payload.resource_name]) }}"
    - copy:
        dest: "{{ local_copy_file }}"
        content: "resource_list: {{ active.resource_list }}"

webhooks.yml

---
- name: Ruleset 1
  hosts: all
  sources:   
    - ansible.eda.webhook:
        host: 0.0.0.0
        port: 5000
  rules:
    - name: Inspect Create request
      condition: >
        event.payload.event_name == "Create"
      action:
        run_playbook:
          name: playbook-create.yml
          post_events: true
          set_facts: true
    - name: Post Create request
      condition:
        all:
          - events.scoped << event.active.resource_list is defined
      action:
        post_event:
          ruleset: Ruleset 2
          event:
            payload:
              resource_list: "{{ events.scoped.active.resource_list }}"

- name: Ruleset 2
  hosts: all
  sources:   
    - ansible.eda.webhook:
        host: 0.0.0.0
        port: 5001
  rules:
    - name: Process Delete
      condition:
        all:
          - event.payload.event_name == "Delete"
          - event.payload.resource_list is select("search", "r1")
#          - event.payload.resource_list is select("search", events.m_0.payload.resource_name)
      action:
        run_playbook:
          name: playbook-delete.yml

Environment

__version__ = '0.11.0'
  Executable location = /home/alialkhalidi/work/ansible-04062023/bin/ansible-rulebook
  Drools_jpy version = 0.2.8
  Java home = /usr/lib/jvm/java-17-openjdk-17.0.6.0.10-1.fc36.x86_64/
  Java version = 17.0.6
  Python version = 3.10.10 (main, Feb  8 2023, 00:00:00) [GCC 12.2.1 20221121 (Red Hat 12.2.1-4)]

Steps to reproduce

1- run the rulebook with: ansible-rulebook --rulebook webhooks.yml -i hosts.yml -v
2- run a call with: curl -d '{"event_name":"Create","resource_name":"r1"}' localhost:5000
3- run a call with: curl -d '{"event_name":"Delete","resource_name":"r1"}' localhost:5001
4- local_copy.yml will have active_list: []

Actual results

• un-comment the line, and repeat steps 1-4, the condition for Ruleset 2 does not match, and playbook-delete.yml does not get triggered.

Expected results

the condition for Ruleset 2 matches, and playbook-delete.yml gets triggered.

Additional information

ansible-rulebook is from current tip main.

[Alex-Izquierdo]

Hello @alialkhalidi
I think there are some misunderstandings. Let me understand better the use case, because I don't know what are you trying to do and I suspect your rulebook is wrong. With the rulebook and instructions provided, the behavior seems the expected.

First at all, the second ruleset listens on 5001 port, but in your instructions there is no any request to 5001, so the second ruleset will only receive the event from the rule Post Create request in the first ruleset.

Even in that case, the rule of the second ruleset will never be triggered because you are using the "all" clause, which matches multiple events. all is not a replacement for and if you want to check multiple field of the same condition you must use the and operator in the same condition.

Note that events are not shared across rulesets, and when an event meets the condition is discarded and can not meet any other rule of the ruleset.

Tip: with -vv you can see as well all the received events, this is can be useful for you to debug your rules.

[alialkhalidi]

@Alex-Izquierdo thanks for the reply. corrected the steps to re-produce, fixing the port number of the second ruleset to 5001.

In a nutshell, its a generic model, where I'm trying to maintain a list using events, but utilize the rulebook engine as a first stage to "filter" events, i.e. before the ansible engine. the list is needed as the events are not similar in their structure; some fields appearing in the create , and are necessary to execute the delete, are not available in the delete request.

let me just start by saying that without the commented line (in the rulebook), the expected logic works:

• Ruleset 1, inspect and processes the request, then posts to Ruleset 2 (facts compiled in the playbook)
• Ruleset 2, and upon reception of the Delete request/event, combines the event posted from Ruleset 1 and what the Delete event have (resouce_name) and does the match.

now the issue, is when I replace the literal value for the resource_name with one from the matched condition, which according to this, should work as well.

I hope its clearer now.

[alialkhalidi]

Managed to get it to work, and the events matching, though not sure how to explain for the operator behaviour difference.
webhooks.yml, lines 39-40
become:

#          - event.payload.resource_list is select("search", "r1")
          - event.payload.resource_list contains events.m_0.payload.resource_name

[mkanoor]

@alialkhalidi Is the arrival of the events guaranteed to be in a specific order?

          - event.payload.event_name == "Delete"
          - event.payload.resource_list is select("search", events.m_0.payload.resource_name)

This assumes that the Delete from the second web hook source would arrive before the second event from the first ruleset. The reason it works when you hardcode a value like r1 is because then the order of events doesn't matter.

[alialkhalidi]

@mkanoor the order of the events is one mentioned in the "steps-to-reproduce": Delete in Ruleset 2 comes "after" Create is processed in Ruleset 1, and a corresponding event is posted.

on another aspect, I think the order within the condition block is pretty much explicit with the use of the m_0 placeholder, which more or less stipulates the order of evaluation rather than the order of arrival, right?

[mkanoor]

@alialkhalidi the search in select is a bug we are fixing it.

Regarding the order of events there is a subtlety there, in your case the event.payload.resource_list comes in first and when it comes in the event.payload.event_name is not there, so what the rule engine does is it will store the events till the other event arrives. When the event arrives later the whole set is re-evaluated again. The events are evicted only if they match, so its possible if you have a huge number of events coming in they will sit in storage till the rules match, once all the conditions match the related events will be evicted. So it will work irrespective of the order of the arrival of events since the events are cached and re-evaluated the only penalty being the caching of events.