Not Support UTF8 ell

[ukhack]

always output UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe4 in position 0: unexpected end of data
like "什么是地球？"
It simply means "whats earth?"

[ParisNeo]

I'll look it up this evening.
Thanks for testing

[nietocjr]

Related to this error, I will share more details of the error,
running with the docker-compose.yml from the repository

when asking to translate any word from English to Japanese this is the error

GPT4All: これは[2023-04-08 01:42:09,370] {_internal.py:224} ERROR - Error on request:
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/werkzeug/serving.py", line 333, in run_wsgi
    execute(self.server.app)
  File "/usr/local/lib/python3.10/site-packages/werkzeug/serving.py", line 322, in execute
    for data in application_iter:
  File "/usr/local/lib/python3.10/site-packages/werkzeug/wsgi.py", line 500, in __next__
    return self._next()
  File "/usr/local/lib/python3.10/site-packages/werkzeug/wrappers/response.py", line 50, in _iter_encoded
    for item in iterable:
  File "/usr/local/lib/python3.10/site-packages/flask/helpers.py", line 149, in generator
    yield from gen
  File "/usr/local/lib/python3.10/site-packages/flask/helpers.py", line 149, in generator
    yield from gen
  File "/srv/app.py", line 327, in parse_to_prompt_stream
    self.chatbot_bindings.generate(
  File "/usr/local/lib/python3.10/site-packages/pyllamacpp/model.py", line 112, in generate
    pp.llama_generate(self._ctx, self.gpt_params, self._call_new_text_callback, verbose)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe7 in position 0: unexpected end of data

when asking to translate from English to Spanish this is the error

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/werkzeug/serving.py", line 333, in run_wsgi
    execute(self.server.app)
  File "/usr/local/lib/python3.10/site-packages/werkzeug/serving.py", line 322, in execute
    for data in application_iter:
  File "/usr/local/lib/python3.10/site-packages/werkzeug/wsgi.py", line 500, in __next__
    return self._next()
  File "/usr/local/lib/python3.10/site-packages/werkzeug/wrappers/response.py", line 50, in _iter_encoded
    for item in iterable:
  File "/usr/local/lib/python3.10/site-packages/flask/helpers.py", line 149, in generator
    yield from gen
  File "/usr/local/lib/python3.10/site-packages/flask/helpers.py", line 149, in generator
    yield from gen
  File "/srv/app.py", line 327, in parse_to_prompt_stream
    self.chatbot_bindings.generate(
  File "/usr/local/lib/python3.10/site-packages/pyllamacpp/model.py", line 112, in generate
    pp.llama_generate(self._ctx, self.gpt_params, self._call_new_text_callback, verbose)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf0 in position 0: unexpected end of data

I would like to share more or even give any solution, but just recently started learning python, so I don't know where to begin to look for the issue.

[ParisNeo]

Thanks alot. We are running in some issues with the backend as we depend heavily on another library and it is is being updated. I'll try my best to fix the problems this evening.