nomic-ai/pygpt4all

no way to suppress `gptj_model_load:` output lines

kim-em opened this issue · 5 comments

kim-em commented

How can I suppress these output lines?

abdeladim-s commented

@semorrison I think I answered that question in the discussions, check it here #98

semorrison commented

Thanks --- unfortunately the gptj_model_load output is also on stdout, so your answer doesn't help.

g0dEngineer commented

Using cpp_generate instead of generate seems to partially solve the issue:

from pygpt4all import GPT4All_J

AI_MODEL = GPT4All_J('data/models/ggml-gpt4all-j-v1.3-groovy.bin')

def TEXT_CALLBACK(text):
    print(text, end="")
    
AI_MODEL.cpp_generate(input_prompt, n_predict=128, new_text_callback=TEXT_CALLBACK)

Oddly enough, pygpt4all.models.gpt4all 's output are suppressed/non-existent via cpp_generate, but not pygpt4all's outputs for some reason from a similar cpp_generate function.

Tried to also modify the source code (%AppData% ...\Local\Programs\Python\Python39\Lib\site-packages\pygpt....\pygptj\model.py)

.... related to the unsuppressed output with no success to be similar to the other cpp_generate (%AppData% \Local\Programs\Python\Python39\Lib\site-packages\pyllamacpp...\pyllamacpp\model.py) that doesn't print output to cmd.

g0dEngineer commented

Looks like it boils down to "pp.gptj_generate" in "(%AppData% ...\Local\Programs\Python\Python39\Lib\site-packages\pygpt....\pygptj\model.py".

  • The above is derived from importing _pygptj as pp, and _pygptj seems to be related to "_pygptj.cp39-win_amd64.pyd", a DLL like object, which is reasonably closed off.

  • Since there's apparently no external "flag" that seems to suppress/desuppress cmd printouts of output tokens, it's left to modify the internal items which seem compressed into the .pyd object.