Add the variables from the config file as parameters with default values

Question

Add the variables from the config file as parameters with default values

Closed this issue a year ago · 2 comments

This will allow a user to run

hugie endpoint create --model-name tiiuae/falcon-40b --instance-type g5.12xlarge

and get and endpoint with an LLM relatively easily whereas with the config file, it is easy to mess something up.

Answer 1 · 2023-07-28T23:01:42.000Z

Nice idea. My only question is how this can interact with the config, as I don't think we want to replace that functionality entirely.

Maybe we could have a flag to update the current config with the variables that we pass on the command line.

Perhaps also a command to output a config file based on the configuration of an existing endpoint. Not sure if this is possible. But fully recognise that many people will not care about the config at all, unless they have a production use case.

Also note #62. When thinking about this.

Answer 2 · 2023-07-31T07:32:41.000Z

The user would not pass a config and if they do the params could be overriden by params, same way as spacy works with a config.