netenglabs/suzieq

[Bug]: Processing data failed for service vlan

FloLaco opened this issue · 10 comments

Suzieq version

0.19.1

Install Type

container

Python version

3.8

Impacted component

sq-poller

Steps to Reproduce

host$ export SUZIEQ_NET_PASS=xxxx
host$ docker run -it -e SUZIEQ_NET_PASS -v /home/florian/suzieq/dbdir:/home/suzieq/parquet --name sq-poller --rm  netenglabs/suzieq
container$ cat parquet/inventory.yml
sources:
- name: testbed-native
  type: native
  hosts:
  # - url: ssh://vagrant@10.0.0.1:22 keyfile=/path/to/private_key
  # - url: ssh://vagrant@10.0.0.2:22 devtype=eos keyfile=/path/to/private_key
  - url: ssh://florian_lacommare@x.x.x.x:22 devtype=ios


devices:
- name: devices-with-jump-hosts
  transport: ssh
  jump-host: xxxx@y.y.y.y
  jump-host-key-file: /home/suzieq/parquet/ssh_cred_conf/id_rsa
  ignore-known-hosts: true
  port: 22

auths:
- name: suzieq-user-01
  username: florian_lacommare
  password: plain:test

- name: suzieq-user-02
  username: florian_lacommare
  password: env:SUZIEQ_NET_PASS

# - name: suzieq-user-03
#   username: florian_lacommare
#   password: ask

namespaces:
- name: testing
  source: testbed-native
  device: devices-with-jump-hosts
  auth: suzieq-user-02

container$ sq-poller -I parquet/inventory.yml

Expected Behavior

Should get data

Observed Behavior

2022-10-10 13:21:31,320 - suzieq.poller.controller - WARNING - log level WARNING
[WORKER 0]: 2022-10-10 13:21:32,510 - suzieq.poller.worker - WARNING - log level WARNING
[WORKER 0]: 2022-10-10 13:21:32,919 - suzieq.poller.worker.nodes.node - WARNING - ios supplied for x.x.x.x:22
[WORKER 0]: 2022-10-10 13:21:36,632 - suzieq.poller.worker.services.service - ERROR - Processing data failed for service vlan on node RSWBB-1-001
[WORKER 0]: Traceback (most recent call last):
[WORKER 0]:   File "/usr/local/lib/python3.8/site-packages/suzieq/poller/worker/services/service.py", line 819, in run
[WORKER 0]:     result = self.process_data(output)
[WORKER 0]:   File "/usr/local/lib/python3.8/site-packages/suzieq/poller/worker/services/service.py", line 478, in process_data
[WORKER 0]:     return self.clean_data(result, data)
[WORKER 0]:   File "/usr/local/lib/python3.8/site-packages/suzieq/poller/worker/services/service.py", line 537, in clean_data
[WORKER 0]:     processed_data = dev_clean_fn(processed_data, raw_data)
[WORKER 0]:   File "/usr/local/lib/python3.8/site-packages/suzieq/poller/worker/services/vlan.py", line 161, in _clean_ios_data
[WORKER 0]:     vlan_dict[str(vlan)]['interfaces'].append(ifname)
[WORKER 0]: KeyError: '999'

Screenshots

Additional Context

I'm supposing that is related to a native vlan on Gi3/0/23 which is not declared ?
But suzeiq should not crash if device is permissive, right ?

RSWBB-1-001#sh vlan

VLAN Name                             Status    Ports
---- -------------------------------- --------- -------------------------------
1    default                          active    Fa1/0/3, Fa1/0/4, Fa1/0/9, Fa1/0/10, Fa1/0/11, Fa1/0/12, Fa1/0/13, Fa1/0/16, Fa1/0/17, Fa1/0/18, Fa1/0/19, Fa1/0/20, Fa1/0/21, Fa1/0/22, Fa1/0/23, Fa1/0/24, Fa2/0/1, Fa2/0/2, Fa2/0/3
                                                Fa2/0/4, Fa2/0/7, Fa2/0/13, Fa2/0/14, Fa2/0/15, Fa2/0/16, Fa2/0/18, Fa2/0/19, Fa2/0/20, Fa2/0/21, Gi3/0/6, Gi3/0/8, Gi3/0/10, Gi3/0/11, Gi3/0/12, Gi3/0/13, Gi3/0/14, Gi3/0/19, Gi3/0/24
10   BACK-OFFICE                      active    Fa1/0/6, Fa2/0/8, Fa2/0/10
20   FRONT-OFFICE                     active    Fa1/0/8
30   SALE-AREA                        active
40   SERVICES                         active
41   Service2_41                      active    Gi3/0/2, Gi3/0/3, Gi3/0/7, Gi3/0/15
50   LOCAL-WIRELESS-DEVICES-1         active
51   LOCAL-WIRELESS-DEVICES-2         active
52   LOCAL-WIRELESS-DEVICES-3         active
60   GLOBAL-WIRELESS-DEVICES          active    Gi2/0/1
70   WIRELESS-TELEPHONY               active
100  Multimedia                       active
110  IP-Telephony                     active    Gi3/0/21
120  Third-party                      active
200  VLAN0200                         active    Fa2/0/9, Fa2/0/12, Fa2/0/17
202  Guest-Partner                    active    Gi3/0/4, Gi3/0/18
313  VLAN0313                         active
500  VLAN500_SDWAN                    active    Gi3/0/17
600  VLAN600_SDWAN                    active    Gi3/0/16
666  management-aruba                 active    Gi3/0/9, Gi3/0/20
700  VLAN700_SDWAN                    active
1002 fddi-default                     act/unsup
1003 token-ring-default               act/unsup
1004 fddinet-default                  act/unsup
1005 trnet-default                    act/unsup

VLAN Type  SAID       MTU   Parent RingNo BridgeNo Stp  BrdgMode Trans1 Trans2
---- ----- ---------- ----- ------ ------ -------- ---- -------- ------ ------
1    enet  100001     1500  -      -      -        -    -        0      0
10   enet  100010     1500  -      -      -        -    -        0      0
20   enet  100020     1500  -      -      -        -    -        0      0
30   enet  100030     1500  -      -      -        -    -        0      0
40   enet  100040     1500  -      -      -        -    -        0      0
41   enet  100041     1500  -      -      -        -    -        0      0
50   enet  100050     1500  -      -      -        -    -        0      0
51   enet  100051     1500  -      -      -        -    -        0      0
52   enet  100052     1500  -      -      -        -    -        0      0
60   enet  100060     1500  -      -      -        -    -        0      0
70   enet  100070     1500  -      -      -        -    -        0      0
100  enet  100100     1500  -      -      -        -    -        0      0
110  enet  100110     1500  -      -      -        -    -        0      0
120  enet  100120     1500  -      -      -        -    -        0      0
200  enet  100200     1500  -      -      -        -    -        0      0
202  enet  100202     1500  -      -      -        -    -        0      0
313  enet  100313     1500  -      -      -        -    -        0      0
500  enet  100500     1500  -      -      -        -    -        0      0
600  enet  100600     1500  -      -      -        -    -        0      0
666  enet  100666     1500  -      -      -        -    -        0      0
700  enet  100700     1500  -      -      -        -    -        0      0
1002 fddi  101002     1500  -      -      -        -    -        0      0
1003 tr    101003     1500  -      -      -        -    -        0      0
1004 fdnet 101004     1500  -      -      -        ieee -        0      0
1005 trnet 101005     1500  -      -      -        ibm  -        0      0

Remote SPAN VLANs
------------------------------------------------------------------------------


Primary Secondary Type              Ports
------- --------- ----------------- ------------------------------------------

RSWBB-1-001#show interface trunk

Port        Mode             Encapsulation  Status        Native vlan
Fa1/0/5     on               802.1q         trunking      1
Fa1/0/7     on               802.1q         trunking      1
Gi1/0/1     on               802.1q         trunking      1
Fa2/0/5     on               802.1q         trunking      1
Gi3/0/23    on               802.1q         trunking      999
Po1         on               802.1q         trunking      1
Po2         on               802.1q         trunking      1

Port        Vlans allowed on trunk
Fa1/0/5     1-4094
Fa1/0/7     1-4094
Gi1/0/1     1-4094
Fa2/0/5     1-4094
Gi3/0/23    1-4094
Po1         1-4094
Po2         1,20

Port        Vlans allowed and active in management domain
Fa1/0/5     1,10,20,30,40-41,50-52,60,70,100,110,120,200,202,313,500,600,666,700
Fa1/0/7     1,10,20,30,40-41,50-52,60,70,100,110,120,200,202,313,500,600,666,700
Gi1/0/1     1,10,20,30,40-41,50-52,60,70,100,110,120,200,202,313,500,600,666,700
Fa2/0/5     1,10,20,30,40-41,50-52,60,70,100,110,120,200,202,313,500,600,666,700
Gi3/0/23    1,10,20,30,40-41,50-52,60,70,100,110,120,200,202,313,500,600,666,700
Po1         1,10,20,30,40-41,50-52,60,70,100,110,120,200,202,313,500,600,666,700
Po2         1,20

Port        Vlans in spanning tree forwarding state and not pruned
Fa1/0/5     1,10,20,30,40-41,50-52,60,70,100,110,120,200,202,313,500,600,666,700
Fa1/0/7     1,10,20,30,40-41,50-52,60,70,100,110,120,200,202,313,500,600,666,700
Gi1/0/1     1,10,20,30,40-41,50-52,60,70,100,110,120,200,202,313,500,600,666,700
Fa2/0/5     1,10,20,30,40-41,50-52,60,70,100,110,120,200,202,313,500,600,666,700
Gi3/0/23    1,10,20,30,40-41,50-52,60,70,100,110,120,200,202,313,500,600,666,700
Po1         1,10,20,30,40-41,50-52,60,70,100,110,120,200,202,313,500,600,666,700
Po2         1,20
RSWBB-1-001#
ddutt commented

Hi @FloLaco, thanks for reporting the bug. Yes, suzieq poller should not crash. Is the above the entire output of show vlan that causes the crash? I see your point above native VLAN. Let me look at the parser with this output

@ddutt Yes it's the full output. Seems that we can specify a native vlan on interface even if it's not declared

ddutt commented

OK, understood. I was about to release 0.19.2. Will see if I can merge this fix in

awesome 👌🏻

ddutt commented

Do you test using container or python lib?

I can test with container as soon you release it

ddutt commented

I'l put one out shortly. Would it be possible for you join the Slack? Its sometimes easier to communicate than via github issue?

Sure, can you provide me invitation ?

Fixed with PR #819