peakwinter/python-nginx

Regexp issue with lot of spaces in the end of config

Shagrat opened this issue · 1 comments

Hello!

Noticed strange delays, or even and seemingly endless loop while parsing a bunch of config files. Was able to determine that

python-nginx/nginx.py

Lines 540 to 554 in 20a8484

double = r'\s*"[^"]*"'
single = r'\s*\'[^\']*\''
normal = r'\s*[^;\s]*'
s1 = r'{}|{}|{}'.format(double, single, normal)
s = r'^\s*({})\s*((?:{})+);'.format(s1, s1)
m = re.compile(s).search(data[index:])
if m:
logging.debug("Key {0} {1}".format(m.group(1), m.group(2)))
k = Key(m.group(1), m.group(2))
if lopen and isinstance(lopen[0], (Container, Server)):
lopen[0].add(k)
else:
f.add(k) if conf else f.append(k)
index += m.end()
continue

in "loads" function (lines 540-554)
takes an exponentially long time if the end of config file has lot's of spaces (starting from 18 it can take up to 5 seconds end each next space increases time more)

Sadly I'm not a regexp ninja so I can't fix it by myself and just started to clean end of a file with strip, but maybe this knowledge will help someone somehow.

python-nginx==1.5.6
Python 3.6.12 (default, Dec 1 2020, 13:45:56)
[GCC 10.2.0] on linux

Example config file attached
example.conf.txt

def remove_trailing_whitespace(text): lines = text.split('\n') cleaned_lines = [line.rstrip() for line in lines] cleaned_text = '\n'.join(cleaned_lines) return cleaned_text I've encountered the same issue, so before loading, I will remove all unnecessary invisible characters from the configuration text.

Hello!

Noticed strange delays, or even and seemingly endless loop while parsing a bunch of config files. Was able to determine that

python-nginx/nginx.py

Lines 540 to 554 in 20a8484

double = r'\s*"[^"]*"'
single = r'\s*\'[^\']*\''
normal = r'\s*[^;\s]*'
s1 = r'{}|{}|{}'.format(double, single, normal)
s = r'^\s*({})\s*((?:{})+);'.format(s1, s1)
m = re.compile(s).search(data[index:])
if m:
logging.debug("Key {0} {1}".format(m.group(1), m.group(2)))
k = Key(m.group(1), m.group(2))
if lopen and isinstance(lopen[0], (Container, Server)):
lopen[0].add(k)
else:
f.add(k) if conf else f.append(k)
index += m.end()
continue

in "loads" function (lines 540-554) takes an exponentially long time if the end of config file has lot's of spaces (starting from 18 it can take up to 5 seconds end each next space increases time more)

Sadly I'm not a regexp ninja so I can't fix it by myself and just started to clean end of a file with strip, but maybe this knowledge will help someone somehow.

python-nginx==1.5.6 Python 3.6.12 (default, Dec 1 2020, 13:45:56) [GCC 10.2.0] on linux

Example config file attached example.conf.txt

def remove_trailing_whitespace(text): lines = text.split('\n') cleaned_lines = [line.rstrip() for line in lines] cleaned_text = '\n'.join(cleaned_lines) return cleaned_text
I've encountered the same issue, so before loading, I will remove all unnecessary invisible characters from the configuration text.