Very slow parsing of cisco_ios_show_version
dainok opened this issue · 0 comments
dainok commented
ISSUE TYPE
- Template Issue with error and raw data
TEMPLATE USING
Value SOFTWARE_IMAGE (\S+)
Value VERSION (.+?)
Value RELEASE (\S+)
Value ROMMON (\S+)
Value HOSTNAME (\S+)
Value UPTIME (.+)
Value UPTIME_YEARS (\d+)
Value UPTIME_WEEKS (\d+)
Value UPTIME_DAYS (\d+)
Value UPTIME_HOURS (\d+)
Value UPTIME_MINUTES (\d+)
Value RELOAD_REASON (.+?)
Value RUNNING_IMAGE (\S+)
Value List HARDWARE (\S+|\S+\d\S+)
Value List SERIAL (\S+)
Value CONFIG_REGISTER (\S+)
Value List MAC ([0-9a-fA-F]{2}(:[0-9a-fA-F]{2}){5})
Value RESTARTED (.+)
Start
^.*Software\s\(${SOFTWARE_IMAGE}+\),\sVersion\s${VERSION},*\s+RELEASE.*\(${RELEASE}\)
^ROM:\s+${ROMMON}
^\s*${HOSTNAME}\s+uptime\s+is\s+${UPTIME} -> Continue
^.*\s+uptime\s+is.*\s+${UPTIME_YEARS}\syear -> Continue
^.*\s+uptime\s+is.*\s+${UPTIME_WEEKS}\sweek -> Continue
^.*\s+uptime\s+is.*\s+${UPTIME_DAYS}\sday -> Continue
^.*\s+uptime\s+is.*\s+${UPTIME_HOURS}\shour -> Continue
^.*\s+uptime\s+is.*\s+${UPTIME_MINUTES}\sminute
^[sS]ystem\s+image\s+file\s+is\s+"(.*?):${RUNNING_IMAGE}"
^(?:[lL]ast\s+reload\s+reason:|System\s+returned\s+to\s+ROM\s+by)\s+${RELOAD_REASON}\s*$$
^[Pp]rocessor\s+board\s+ID\s+${SERIAL}
^[Cc]isco\s+${HARDWARE}\s+\(.+\).+
^[Cc]onfiguration\s+register\s+is\s+${CONFIG_REGISTER}
^Base\s+[Ee]thernet\s+MAC\s+[Aa]ddress\s+:\s+${MAC}
^System\s+restarted\s+at\s+${RESTARTED}$$
^Switch\s+Port -> Stack
# Capture time-stamp if vty line has command time-stamping turned on
^Switch\s\d+ -> Stack
^Load\s+for\s+
^Time\s+source\s+is
Stack
^[Ss]ystem\s+[Ss]erial\s+[Nn]umber\s+:\s+${SERIAL}
^[Mm]odel\s+[Nn]umber\s+:\s+${HARDWARE}\s*
^[Cc]onfiguration\s+register\s+is\s+${CONFIG_REGISTER}
^Base [Ee]thernet MAC [Aa]ddress\s+:\s+${MAC}
SAMPLE COMMAND OUTPUT
Cisco IOS Software, vios_l2 Software (vios_l2-ADVENTERPRISEK9-M), Version 15.2(4.0.55)E, TEST ENGINEERING ESTG_WEEKLY BUILD, synced to END_OF_FLO_ISP
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2015 by Cisco Systems, Inc.
Compiled Tue 28-Jul-15 18:52 by sasyamal
ROM: Bootstrap program is IOSv
Switch5 uptime is 3 weeks, 6 days, 32 minutes
System returned to ROM by reload
System image file is "flash0:/vios_l2-adventerprisek9-m"
Last reload reason: Unknown reason
This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.
A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html
If you require further assistance please contact us by sending email to
export@cisco.com.
Cisco IOSv () processor (revision 1.0) with 935161K/111616K bytes of memory.
Processor board ID 9RYX5605M6M
8 Gigabit Ethernet interfaces
DRAM configuration is 72 bits wide with parity disabled.
256K bytes of non-volatile configuration memory.
2097152K bytes of ATA System CompactFlash 0 (Read/Write)
0K bytes of ATA CompactFlash 1 (Read/Write)
0K bytes of ATA CompactFlash 2 (Read/Write)
0K bytes of ATA CompactFlash 3 (Read/Write)
Configuration register is 0x0
SUMMARY
Using the last template (attached above) the parsing takes about 1 minute. The old template took milliseconds.
STEPS TO REPRODUCE
Use the following files:
raw.out:
Cisco IOS Software, vios_l2 Software (vios_l2-ADVENTERPRISEK9-M), Version 15.2(4.0.55)E, TEST ENGINEERING ESTG_WEEKLY BUILD, synced to END_OF_FLO_ISP
cisco_ios_show_version_new.textfsm:
Value SOFTWARE_IMAGE (\S+)
Value VERSION (.+?)
Value RELEASE (\S+)
Value ROMMON (\S+)
Value HOSTNAME (\S+)
Value UPTIME (.+)
Value UPTIME_YEARS (\d+)
Value UPTIME_WEEKS (\d+)
Value UPTIME_DAYS (\d+)
Value UPTIME_HOURS (\d+)
Value UPTIME_MINUTES (\d+)
Value RELOAD_REASON (.+?)
Value RUNNING_IMAGE (\S+)
Value List HARDWARE (\S+|\S+\d\S+)
Value List SERIAL (\S+)
Value CONFIG_REGISTER (\S+)
Value List MAC ([0-9a-fA-F]{2}(:[0-9a-fA-F]{2}){5})
Value RESTARTED (.+)
Start
^.*Software\s\(${SOFTWARE_IMAGE}+\),\sVersion\s${VERSION},*\s+RELEASE.*\(${RELEASE}\)
^ROM:\s+${ROMMON}
^\s*${HOSTNAME}\s+uptime\s+is\s+${UPTIME} -> Continue
^.*\s+uptime\s+is.*\s+${UPTIME_YEARS}\syear -> Continue
^.*\s+uptime\s+is.*\s+${UPTIME_WEEKS}\sweek -> Continue
^.*\s+uptime\s+is.*\s+${UPTIME_DAYS}\sday -> Continue
^.*\s+uptime\s+is.*\s+${UPTIME_HOURS}\shour -> Continue
^.*\s+uptime\s+is.*\s+${UPTIME_MINUTES}\sminute
^[sS]ystem\s+image\s+file\s+is\s+"(.*?):${RUNNING_IMAGE}"
^(?:[lL]ast\s+reload\s+reason:|System\s+returned\s+to\s+ROM\s+by)\s+${RELOAD_REASON}\s*$$
^[Pp]rocessor\s+board\s+ID\s+${SERIAL}
^[Cc]isco\s+${HARDWARE}\s+\(.+\).+
^[Cc]onfiguration\s+register\s+is\s+${CONFIG_REGISTER}
^Base\s+[Ee]thernet\s+MAC\s+[Aa]ddress\s+:\s+${MAC}
^System\s+restarted\s+at\s+${RESTARTED}$$
^Switch\s+Port -> Stack
# Capture time-stamp if vty line has command time-stamping turned on
^Switch\s\d+ -> Stack
^Load\s+for\s+
^Time\s+source\s+is
Stack
^[Ss]ystem\s+[Ss]erial\s+[Nn]umber\s+:\s+${SERIAL}
^[Mm]odel\s+[Nn]umber\s+:\s+${HARDWARE}\s*
^[Cc]onfiguration\s+register\s+is\s+${CONFIG_REGISTER}
^Base [Ee]thernet MAC [Aa]ddress\s+:\s+${MAC}
cisco_ios_show_version_old.textfsm:
Value VERSION (.+?)
Value ROMMON (\S+)
Value HOSTNAME (\S+)
Value UPTIME (.+)
Value UPTIME_YEARS (\d+)
Value UPTIME_WEEKS (\d+)
Value UPTIME_DAYS (\d+)
Value UPTIME_HOURS (\d+)
Value UPTIME_MINUTES (\d+)
Value RELOAD_REASON (.+?)
Value RUNNING_IMAGE (\S+)
Value List HARDWARE (\S+|\S+\d\S+)
Value List SERIAL (\S+)
Value CONFIG_REGISTER (\S+)
Value List MAC ([0-9a-fA-F]{2}(:[0-9a-fA-F]{2}){5})
Value RESTARTED (.+)
Start
^.*Software\s.+\),\sVersion\s${VERSION},*\s+RELEASE.*
^ROM:\s+${ROMMON}
^\s*${HOSTNAME}\s+uptime\s+is\s+${UPTIME} -> Continue
^.*\s+uptime\s+is.*\s+${UPTIME_YEARS}\syear -> Continue
^.*\s+uptime\s+is.*\s+${UPTIME_WEEKS}\sweek -> Continue
^.*\s+uptime\s+is.*\s+${UPTIME_DAYS}\sday -> Continue
^.*\s+uptime\s+is.*\s+${UPTIME_HOURS}\shour -> Continue
^.*\s+uptime\s+is.*\s+${UPTIME_MINUTES}\sminute
^[sS]ystem\s+image\s+file\s+is\s+"(.*?):${RUNNING_IMAGE}"
^(?:[lL]ast\s+reload\s+reason:|System\s+returned\s+to\s+ROM\s+by)\s+${RELOAD_REASON}\s*$$
^[Pp]rocessor\s+board\s+ID\s+${SERIAL}
^[Cc]isco\s+${HARDWARE}\s+\(.+\).+
^[Cc]onfiguration\s+register\s+is\s+${CONFIG_REGISTER}
^Base\s+[Ee]thernet\s+MAC\s+[Aa]ddress\s+:\s+${MAC}
^System\s+restarted\s+at\s+${RESTARTED}$$
^Switch\s+Port -> Stack
# Capture time-stamp if vty line has command time-stamping turned on
^Load\s+for\s+
^Time\s+source\s+is
Stack
^[Ss]ystem\s+[Ss]erial\s+[Nn]umber\s+:\s+${SERIAL}
^[Mm]odel\s+[Nn]umber\s+:\s+${HARDWARE}\s*
^[Cc]onfiguration\s+register\s+is\s+${CONFIG_REGISTER}
^Base [Ee]thernet MAC [Aa]ddress\s+:\s+${MAC}
Run:
import textfsm
import pprint
template_file = 'cisco_ios_show_version_new.textfsm'
raw_output_file = 'raw.out'
fd_t1 = open('cisco_ios_show_version_old.textfsm')
fd_t2 = open('cisco_ios_show_version_new.textfsm')
fd_o = open(raw_output_file)
raw_output = fd_o.read()
re_table_1 = textfsm.TextFSM(fd_t1)
re_table_2 = textfsm.TextFSM(fd_t2)
parsed_output = re_table_1.ParseText(raw_output)
parsed_output = re_table_2.ParseText(raw_output)
You can see that the second ParseText takes a lot compared to the previous one.
According to my tests the issue is the following line:
^.*Software\s\(${SOFTWARE_IMAGE}+\),\sVersion\s${VERSION},*\s+RELEASE.*\(${RELEASE}\)
To be specific the issue is the "+" after SOFTWARE_IMAGE. I think that it is not required.
EXPECTED RESULTS
The parsing should complete in milliseconds.
ACTUAL RESULTS
The parsing completes in 1 minute.