support dynamic model overrides
Opened this issue · 0 comments
mikestaub commented
As a user, I want to control the chat configuration dynamically.
Depending on the machine specs, the sliding_window_size
and attention_sink_size
may be needed to avoid a 'max window length exceeded' error for example.
Reference: https://github.com/mlc-ai/web-llm/blob/main/src/config.ts#L66