d0c-s4vage/gramfuzz

UnicodeDecodeError: 'ascii' codec can't decode byte 0xcb in position 2542: ordinal not in range(128)

docfate111 opened this issue · 4 comments

docfate111 commented

After creating the issue, checkboxes will appear where [] label exist in the
markdown. You can check/uncheck them to fill out the environment section.

Checklist

  • [ x ] I have included the [relevant portions of the] grammar used that caused the bug
  • [ x ] I have filled out the environment section

Environment

Platform

  • Windows
  • Mac
  • [ x ] Linux
  • Other (please specify)

Python Version

  • Python 2.7
  • Python 3.4
  • Python 3.5
  • [ x ] Python 3.6
  • Python 3.7
  • Python 3.8
  • Other (please specify)

Describe the bug

A clear and concise description of what the bug is.
The file isn't opened with the encoding='utf-8' flag so I end with this error when I run main.py https://github.com/mgree/smoosh-fuzz/blob/master/src/posix/main.py in ubuntu docker python3

  File "main.py", line 4, in <module>
    fuzzer.load_grammar('words.py')
  File "/usr/local/lib/python3.6/dist-packages/gramfuzz/__init__.py", line 125, in load_grammar
    data = f.read()
  File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xcb in position 2542: ordinal not in range(128)

To Reproduce

The issue is in the function load_grammar in class GramFuzzer:
--->here: with open(path, "r", WE NEED: encoding='utf-8') as f:
data = f.read()
code = compile(data, path, "exec")
Steps to reproduce

Expected Behavior

I expected the behavior to be the same as in Python3.7 on Mac- a grammar is generated and strings from that grammar.
A clear and concise description of what you expected to happen.

d0c-s4vage commented

Oof, thanks for this! I'll take a look this week

docfate111 commented

May I have permission to git push please?
The solution is just 1 line of code that I wrote above.

docfate111 commented

the line is 124 in init.py
with open(path, "r")as f:
to
with open(path, "r", encoding='utf-8') as f:
otherwise this breaks on Linux

d0c-s4vage commented

@docfate111 Thanks for being patient on this! Things have been up in the air lately...

May I have permission to git push please?

PRs are definitely welcome! You can fork the repository, create a new branch in your forked repository, and then submit a new PR from your branch into this project.