[source_nat] - Rules are not applied

Question

[source_nat] - Rules are not applied

mikehulsman opened this issue a year ago · 2 comments

mikehulsman commented a year ago

Tasks

Please make sure to go through these steps before opening an issue:

Read the documentation of the affected module: Docs
Read the troubleshooting info: Info

Basic info

Affected Module:
source_nat

Versions

Controller
- Collection version:
  1.2.6
  (ansible-galaxy collection list | grep opnsense)
- Ansible & Python version:
  ansible [core 2.15.3]
  python version = 3.11.5
  (ansible --version)
OPNSense
- System version:
  23.7.4
- Plugin version:
  os-firewall 1.4_1
  os-wireguard 2.1
  (if applicable)

Describe the bug

When Adding nat rules with the moduile source_nat the rules are added under Firewall -> Automation -> Source NAT
But the rules are not applied.
When I manually with the Webui apply the rules under Firewall -> Automation -> Source NAT -> apply than the rules are working.

Expected behavior

When I add source_nat rules I expect that the new rules are applied, not that I have to do this manually in the webui.

Debug output

I enabled debugging and these are the API calls when I try to add 2 rules

2023-09-18 10:48:36:104222 | POST => https://xx.xx.xx.xx/api/firewall/source_nat/addRule
2023-09-18 12:59:16:757571 | GET => https://xx.xx.xx.xx/api/firewall/source_nat/get
2023-09-18 12:59:16:982807 | POST => https://xx.xx.xx.xx/api/firewall/source_nat/addRule
2023-09-18 13:01:03:421363 | GET => https://xx.xx.xx.xx/api/firewall/source_nat/get
2023-09-18 13:01:03:646326 | POST => https://xx.xx.xx.xx/api/firewall/source_nat/addRule
2023-09-18 13:01:32:032643 | GET => https://xx.xx.xx.xx/api/firewall/source_nat/get
2023-09-18 13:01:32:265746 | POST => https://xx.xx.xx.xx/api/firewall/source_nat/addRule
2023-09-18 13:04:20:494033 | GET => https://xx.xx.xx.xx/api/firewall/source_nat/get
2023-09-18 13:04:20:722383 | POST => https://xx.xx.xx.xx/api/firewall/source_nat/setRule/3b63ecf0-ee2a-4662-ad1a-8ad911994098
2023-09-18 13:04:49:100628 | GET => https://xx.xx.xx.xx/api/firewall/source_nat/get
2023-09-18 13:04:49:328184 | POST => https://xx.xx.xx.xx/api/firewall/source_nat/setRule/3b63ecf0-ee2a-4662-ad1a-8ad911994098
2023-09-21 15:08:42:238500 | GET => https://xx.xx.xx.xx/api/firewall/source_nat/get
2023-09-21 15:08:42:874805 | GET => https://xx.xx.xx.xx/api/firewall/source_nat/get
~

If the issue is related to time-consumption, you may also add the content of the profiling logs.

2023-09-18 13:01:32:404385 |          6031 function calls (5917 primitive calls) in 0.364 seconds
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        2    0.297    0.149    0.297    0.149 {method 'read' of '_ssl._SSLSocket' objects}
        1    0.029    0.029    0.029    0.029 {method 'do_handshake' of '_ssl._SSLSocket' objects}
        1    0.026    0.026    0.026    0.026 {method 'connect' of '_socket.socket' objects}
        1    0.001    0.001    0.002    0.002 {built-in method _socket.getaddrinfo}
        4    0.000    0.000    0.000    0.000 {method '__exit__' of '_io._IOBase' objects}
  705/665    0.000    0.000    0.000    0.000 {built-in method builtins.isinstance}
        5    0.000    0.000    0.000    0.000 {built-in method syslog.syslog}
      6/2    0.000    0.000    0.000    0.000 /usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/re/_parser.py:509(_parse)
        3    0.000    0.000    0.000    0.000 {method 'write' of '_ssl._SSLSocket' objects}
        4    0.000    0.000    0.000    0.000 /usr/local/lib/python3.11/site-packages/h11/_headers.py:150(normalize_and_validate)
        4    0.000    0.000    0.001    0.000 /var/folders/6n/spmk0cv14s10d35669xt3kf00000gn/T/ansible_ansibleguy.opnsense.source_nat_payload_ft1evg_f/ansible_ansibleguy.opnsense.source_nat_payload.zip/ansible_collections/ansibleguy/opnsense/plugins/module_utils/helper/api.py:112(debug_api)
       12    0.000    0.000    0.001    0.000 /usr/local/lib/python3.11/site-packages/h11/_connection.py:260(_process_event)
        2    0.000    0.000    0.000    0.000 {built-in method io.open}
        2    0.000    0.000    0.000    0.000 /usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/decoder.py:343(raw_decode)
        2    0.000    0.000    0.000    0.000 {built-in method marshal.loads}
     10/2    0.000    0.000    0.000    0.000 /usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/re/_compiler.py:37(_compile)
        6    0.000    0.000    0.299    0.050 /usr/local/lib/python3.11/site-packages/httpcore/_sync/http11.py:192(_receive_event)
       53    0.000    0.000    0.000    0.000 {method 'fullmatch' of 're.Pattern' objects}
        3    0.000    0.000    0.000    0.000 {method 'poll' of 'select.poll' objects}
       29    0.000    0.000    0.000    0.000 /usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py:104(__init__)


2023-09-18 13:04:20:862065 |          6533 function calls (6419 primitive calls) in 0.360 seconds
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        2    0.298    0.149    0.298    0.149 {method 'read' of '_ssl._SSLSocket' objects}
        1    0.025    0.025    0.025    0.025 {method 'do_handshake' of '_ssl._SSLSocket' objects}
        1    0.024    0.024    0.024    0.024 {method 'connect' of '_socket.socket' objects}
        1    0.001    0.001    0.001    0.001 {built-in method _socket.getaddrinfo}
        8    0.000    0.000    0.000    0.000 {built-in method syslog.syslog}
        2    0.000    0.000    0.000    0.000 /usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/decoder.py:343(raw_decode)
        4    0.000    0.000    0.000    0.000 {method '__exit__' of '_io._IOBase' objects}
  862/822    0.000    0.000    0.000    0.000 {built-in method builtins.isinstance}
        4    0.000    0.000    0.001    0.000 /var/folders/6n/spmk0cv14s10d35669xt3kf00000gn/T/ansible_ansibleguy.opnsense.source_nat_payload_t621k_wm/ansible_ansibleguy.opnsense.source_nat_payload.zip/ansible_collections/ansibleguy/opnsense/plugins/module_utils/helper/api.py:112(debug_api)
        3    0.000    0.000    0.000    0.000 {method 'write' of '_ssl._SSLSocket' objects}
      6/2    0.000    0.000    0.000    0.000 /usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/re/_parser.py:509(_parse)
        4    0.000    0.000    0.000    0.000 /usr/local/lib/python3.11/site-packages/h11/_headers.py:150(normalize_and_validate)
        2    0.000    0.000    0.000    0.000 /var/folders/6n/spmk0cv14s10d35669xt3kf00000gn/T/ansible_ansibleguy.opnsense.source_nat_payload_t621k_wm/ansible_ansibleguy.opnsense.source_nat_payload.zip/ansible_collections/ansibleguy/opnsense/plugins/module_utils/base/base.py:337(build_diff)
        2    0.000    0.000    0.000    0.000 {built-in method io.open}
       12    0.000    0.000    0.001    0.000 /usr/local/lib/python3.11/site-packages/h11/_connection.py:260(_process_event)
        2    0.000    0.000    0.000    0.000 {built-in method marshal.loads}
       53    0.000    0.000    0.000    0.000 {method 'fullmatch' of 're.Pattern' objects}
        6    0.000    0.000    0.299    0.050 /usr/local/lib/python3.11/site-packages/httpcore/_sync/http11.py:192(_receive_event)
        3    0.000    0.000    0.000    0.000 {method 'poll' of 'select.poll' objects}
     10/2    0.000    0.000    0.000    0.000 /usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/re/_compiler.py:37(_compile)


2023-09-18 13:04:49:471698 |          6533 function calls (6419 primitive calls) in 0.364 seconds
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        2    0.308    0.154    0.308    0.154 {method 'read' of '_ssl._SSLSocket' objects}
        1    0.022    0.022    0.022    0.022 {method 'do_handshake' of '_ssl._SSLSocket' objects}
        1    0.021    0.021    0.021    0.021 {method 'connect' of '_socket.socket' objects}
        1    0.001    0.001    0.002    0.002 {built-in method _socket.getaddrinfo}
        4    0.000    0.000    0.001    0.000 /var/folders/6n/spmk0cv14s10d35669xt3kf00000gn/T/ansible_ansibleguy.opnsense.source_nat_payload_h0ysbhki/ansible_ansibleguy.opnsense.source_nat_payload.zip/ansible_collections/ansibleguy/opnsense/plugins/module_utils/helper/api.py:112(debug_api)
        8    0.000    0.000    0.000    0.000 {built-in method syslog.syslog}
        2    0.000    0.000    0.000    0.000 /usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/decoder.py:343(raw_decode)
  862/822    0.000    0.000    0.000    0.000 {built-in method builtins.isinstance}
        4    0.000    0.000    0.000    0.000 {method '__exit__' of '_io._IOBase' objects}
        3    0.000    0.000    0.000    0.000 {method 'write' of '_ssl._SSLSocket' objects}
      6/2    0.000    0.000    0.000    0.000 /usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/re/_parser.py:509(_parse)
        4    0.000    0.000    0.000    0.000 /usr/local/lib/python3.11/site-packages/h11/_headers.py:150(normalize_and_validate)
        2    0.000    0.000    0.000    0.000 /var/folders/6n/spmk0cv14s10d35669xt3kf00000gn/T/ansible_ansibleguy.opnsense.source_nat_payload_h0ysbhki/ansible_ansibleguy.opnsense.source_nat_payload.zip/ansible_collections/ansibleguy/opnsense/plugins/module_utils/base/base.py:337(build_diff)
        2    0.000    0.000    0.000    0.000 {built-in method io.open}
        2    0.000    0.000    0.000    0.000 {built-in method marshal.loads}
       12    0.000    0.000    0.001    0.000 /usr/local/lib/python3.11/site-packages/h11/_connection.py:260(_process_event)
        6    0.000    0.000    0.309    0.052 /usr/local/lib/python3.11/site-packages/httpcore/_sync/http11.py:192(_receive_event)
     10/2    0.000    0.000    0.000    0.000 /usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/re/_compiler.py:37(_compile)
        3    0.000    0.000    0.000    0.000 {method 'poll' of 'select.poll' objects}
        4    0.000    0.000    0.001    0.000 /usr/local/lib/python3.11/site-packages/httpx/_urlparse.py:149(urlparse)


2023-09-21 15:08:42:450430 |          3792 function calls (3702 primitive calls) in 0.206 seconds
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.151    0.151    0.151    0.151 {method 'read' of '_ssl._SSLSocket' objects}
        1    0.023    0.023    0.023    0.023 {method 'connect' of '_socket.socket' objects}
        1    0.022    0.022    0.022    0.022 {method 'do_handshake' of '_ssl._SSLSocket' objects}
        7    0.001    0.000    0.001    0.000 {built-in method posix.stat}
        1    0.001    0.001    0.001    0.001 {built-in method _socket.getaddrinfo}
        3    0.000    0.000    0.000    0.000 {method '__exit__' of '_io._IOBase' objects}
        1    0.000    0.000    0.000    0.000 /usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/decoder.py:343(raw_decode)
      6/2    0.000    0.000    0.000    0.000 /usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/re/_parser.py:509(_parse)
        2    0.000    0.000    0.002    0.001 /var/folders/6n/spmk0cv14s10d35669xt3kf00000gn/T/ansible_ansibleguy.opnsense.source_nat_payload_bqyurshm/ansible_ansibleguy.opnsense.source_nat_payload.zip/ansible_collections/ansibleguy/opnsense/plugins/module_utils/helper/api.py:112(debug_api)
        2    0.000    0.000    0.000    0.000 /var/folders/6n/spmk0cv14s10d35669xt3kf00000gn/T/ansible_ansibleguy.opnsense.source_nat_payload_bqyurshm/ansible_ansibleguy.opnsense.source_nat_payload.zip/ansible_collections/ansibleguy/opnsense/plugins/module_utils/base/base.py:337(build_diff)
      6/2    0.000    0.000    0.001    0.000 /usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/re/_parser.py:449(_parse_sub)
        2    0.000    0.000    0.000    0.000 {built-in method marshal.loads}
  519/499    0.000    0.000    0.000    0.000 {built-in method builtins.isinstance}
     10/2    0.000    0.000    0.000    0.000 /usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/re/_compiler.py:37(_compile)
        1    0.000    0.000    0.000    0.000 /usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/stringprep.py:1(<module>)
        1    0.000    0.000    0.000    0.000 {built-in method io.open}
        2    0.000    0.000    0.000    0.000 {built-in method syslog.syslog}
        1    0.000    0.000    0.000    0.000 {method 'write' of '_ssl._SSLSocket' objects}
        5    0.000    0.000    0.000    0.000 {built-in method builtins.__build_class__}
        2    0.000    0.000    0.000    0.000 /usr/local/lib/python3.11/site-packages/h11/_headers.py:150(normalize_and_validate)


2023-09-21 15:08:43:064619 |          3896 function calls (3806 primitive calls) in 0.183 seconds
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.128    0.128    0.128    0.128 {method 'read' of '_ssl._SSLSocket' objects}
        1    0.023    0.023    0.023    0.023 {method 'do_handshake' of '_ssl._SSLSocket' objects}
        1    0.023    0.023    0.023    0.023 {method 'connect' of '_socket.socket' objects}
        1    0.001    0.001    0.001    0.001 {built-in method _socket.getaddrinfo}
        1    0.000    0.000    0.000    0.000 /usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/decoder.py:343(raw_decode)
      6/2    0.000    0.000    0.000    0.000 /usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/re/_parser.py:509(_parse)
  564/544    0.000    0.000    0.000    0.000 {built-in method builtins.isinstance}
        2    0.000    0.000    0.001    0.000 /var/folders/6n/spmk0cv14s10d35669xt3kf00000gn/T/ansible_ansibleguy.opnsense.source_nat_payload_1qgcgwwz/ansible_ansibleguy.opnsense.source_nat_payload.zip/ansible_collections/ansibleguy/opnsense/plugins/module_utils/helper/api.py:112(debug_api)
        2    0.000    0.000    0.000    0.000 /var/folders/6n/spmk0cv14s10d35669xt3kf00000gn/T/ansible_ansibleguy.opnsense.source_nat_payload_1qgcgwwz/ansible_ansibleguy.opnsense.source_nat_payload.zip/ansible_collections/ansibleguy/opnsense/plugins/module_utils/base/base.py:337(build_diff)
        2    0.000    0.000    0.000    0.000 {built-in method marshal.loads}
     10/2    0.000    0.000    0.000    0.000 /usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/re/_compiler.py:37(_compile)
        3    0.000    0.000    0.000    0.000 {built-in method syslog.syslog}
        2    0.000    0.000    0.000    0.000 /var/folders/6n/spmk0cv14s10d35669xt3kf00000gn/T/ansible_ansibleguy.opnsense.source_nat_payload_1qgcgwwz/ansible_ansibleguy.opnsense.source_nat_payload.zip/ansible_collections/ansibleguy/opnsense/plugins/module_utils/helper/main.py:333(simplify_translate)
      6/2    0.000    0.000    0.001    0.000 /usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/re/_parser.py:449(_parse_sub)
        6    0.000    0.000    0.001    0.000 /usr/local/lib/python3.11/site-packages/h11/_connection.py:260(_process_event)
        2    0.000    0.000    0.000    0.000 /usr/local/lib/python3.11/site-packages/h11/_headers.py:150(normalize_and_validate)
        3    0.000    0.000    0.000    0.000 {method '__exit__' of '_io._IOBase' objects}
        1    0.000    0.000    0.000    0.000 {method 'write' of '_ssl._SSLSocket' objects}
        1    0.000    0.000    0.000    0.000 /usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/stringprep.py:1(<module>)
        5    0.000    0.000    0.000    0.000 {built-in method builtins.__build_class__}

Screenshots

If applicable, add screenshots to help explain your problem.

Additional context

Add any other context about the problem here.

Reproduce

Tasks

Task(s) that produce the error:

    - name: Add Outbound NAT for Wireguard to separate based on source address
      ansibleguy.opnsense.source_nat:
        debug: true
        sequence: "{{ item.sequence }}"
        description: "{{ item.description }}"
        interface: "{{ item.interface }}"
        source_net: "{{ item.source_net }}"
        destination: "{{ item.destination }}"
        match_fields: "{{ item.match_fields }}"
        target: "{{ item.target }}"
        log: "{{ item.log }}"
      loop: "{{ opnsense_outbound_nat }}"

Config

Config used for the task(s):

opnsense_outbound_nat:
  - interface: 'lan'
    sequence: '1'
    match_fields: '[''description'']'
    description: 'translate from  AMS'
    source_net: '192.168.255.2/32'
    destination: 'any'
    target: '10.0.0.10/32'
    log: 'true'
  - interface: 'lan'
    sequence: '2'
    match_fields: '[''description'']'
    description: 'translate from OTHER'
    source_net: '192.168.250.4/32'
    destination: 'any'
    target: '10.0.0.20/32'
    log: 'true'

OPNSense config

(If the issue only occurs when non ansible-managed config is modified)

Answer 1 · 2023-09-24T14:47:54.000Z

@mikehulsman The latest version should now have a fix for this.
Please test it: ansible-galaxy collection install git+https://github.com/ansibleguy/collection_opnsense.git

Answer 2 · 2023-09-25T06:12:35.000Z

@ansibleguy I have installed the latest version and the problem is fixed now.
Thanks for the fast response.