Export error on Windows with nbconvert v6.0

Question

Export error on Windows with nbconvert v6.0

jayqi opened this issue 4 years ago · 0 comments

nbconvert v6.0 introduced some kind of file encoding change that breaks exporting in nbautoexport.

Here is a stack trace from our scheduled test build.

nbautoexport\export.py:110: in export_notebook
    converter.convert_notebooks()
c:\miniconda\envs\test\lib\site-packages\nbconvert\nbconvertapp.py:524: in convert_notebooks
    self.convert_single_notebook(notebook_filename)
c:\miniconda\envs\test\lib\site-packages\nbconvert\nbconvertapp.py:491: in convert_single_notebook
    self.postprocess_single_notebook(write_results)
c:\miniconda\envs\test\lib\site-packages\nbconvert\nbconvertapp.py:463: in postprocess_single_notebook
    self.postprocessor(write_results)
c:\miniconda\envs\test\lib\site-packages\nbconvert\postprocessors\base.py:28: in __call__
    self.postprocess(input)
nbautoexport\export.py:41: in postprocess
    text = f.read()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <encodings.cp1252.IncrementalDecoder object at 0x00000186A042E190>
input = b'<!DOCTYPE html>\r\n<html>\r\n<head><meta charset="utf-8" />\r\n<meta name="viewport" content="width=device-width, in...>\r\n\r\n     </div>\r\n</div>\r\n</div>\r\n</div>\r\n\r\n</div>\r\n</body>\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n</html>\r\n'
final = True

    def decode(self, input, final=False):
>       return codecs.charmap_decode(input,self.errors,decoding_table)[0]
E       UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 226813: character maps to <undefined>

c:\miniconda\envs\test\lib\encodings\cp1252.py:23: UnicodeDecodeError
---------------------------- Captured stderr call -----------------------------
[NbConvertApp] Converting notebook C:\Users\runneradmin\AppData\Local\Temp\pytest-of-runneradmin\pytest-0\test_notebook_exports_generato2\the_notebook_0.ipynb to html
[NbConvertApp] Writing 588470 bytes to C:\Users\runneradmin\AppData\Local\Temp\pytest-of-runneradmin\pytest-0\test_notebook_exports_generato2\the_notebook_0.html

https://github.com/drivendataorg/nbautoexport/runs/1131815930?check_suite_focus=true#step:6:137

It looks like the converted files are now using a Windows encoding, but when we read back in to remove cell numbers, we're expecting unicode?

When fixing this, we may want to consider doing it in a way that ensures backwards compatibility with nbconvert v5.6.