Basic CLI tool for using the Text Generation Web UI websocket API.
- Clone this Repository.
- Run
npm i
to install project dependencies. - Run
npm run build
to build the project. - Run the script locally like
node ./dist/index.js --prompt "whatever"
Flag | Default | Description |
---|---|---|
--prompt PROMPT |
Pass the desired prompt Required | |
--model MODEL |
Change to the specified model prior to passing the prompt | |
--baseUrl BASE_URL |
ws://127.0.0.1:7860/queue/join |
the websocket "join" URL |
--softPromptPath SOFT_PROMPT_PATH |
Loads a template to wrap the prompt, must contain __prompt__ which will be replaced with the prompt value |
|
--promptReplacePath PROMPT_REPLACE_PATH |
Loads and parses a newline-limited list of values to generate output for each value. The prompt must contain the string __replace__ which will be (logically) replaced with a value in the list for each generation |
|
--max_new_tokens MAX_NEW_TOKENS |
200 |
See: HuggingFace Generation Config |
--seed SEED |
-1 |
See: HuggingFace Generation Config |
--temperature TEMPERATURE |
0.7 |
See: HuggingFace Generation Config |
--top-p TOP_P |
40 |
See: HuggingFace Generation Config |
--top-k TOP_K |
0.1 |
See: HuggingFace Generation Config |
--typical-p TYPICAL_P |
1 |
See: HuggingFace Generation Config |
--repetition_penalty REPETITION_PENALTY |
1.18 |
See: HuggingFace Generation Config |
--encoder_repetitition_penalty ENCODER_REPETITION_PENALTY |
1 |
See: HuggingFace Generation Config |
--no_repeat_ngram_size NO_REPEAT_NGRAM_SIZE |
1 |
See: HuggingFace Generation Config |
--min_length MIN_LENGTH |
0 |
See: HuggingFace Generation Config |
`--do_sample true | false` | true |
--penalty_alpha PENALTY_ALPHA |
0 |
See: HuggingFace Generation Config |
--num_beams NUM_BEAMS |
1 |
See: HuggingFace Generation Config |
--length_penalty LENGTH_PENALTY |
1 |
See: HuggingFace Generation Config |
`--early_stopping true | false` | false |
`--add_bos_token true | false` | true |
`--ban_bos_token true | false` | false |
--truncation_length TRUNCATION_LENGTH |
2048 |
See: HuggingFace Generation Config |
--custom_stopping_strings CUSTOM_STOPPING_STRINGS |
"" |
See: HuggingFace Generation Config |
--gpu_memory_0 0...MAX_VRAM |
0 |
See: HuggingFace Generation Config |
--cpu_memory 0...MAX_CPU |
"" |
See: HuggingFace Generation Config |
`--auto_devices true | false` | "" |
`--disk true | false` | "" |
`--cpu true | false` | "" |
`--bf16 true | false` | "" |
`--load_in_8bit true | false` | "" |
`--wbits 1 | 2 | 3 |
`--groupsize 32 | 64 | 128` |
`--model_type None | llama | opt |
--pre_layer 0...100 |
"" |
See: HuggingFace Generation Config |