addyosmani/chatty

support dynamic model overrides

Opened this issue · 0 comments

As a user, I want to control the chat configuration dynamically.

Depending on the machine specs, the sliding_window_size and attention_sink_size may be needed to avoid a 'max window length exceeded' error for example.

Reference: https://github.com/mlc-ai/web-llm/blob/main/src/config.ts#L66