what are slots?
Opened this issue · 0 comments
siddhsql commented
hello, could you tell me what are slots?
https://github.com/kherud/java-llama.cpp/blob/master/src/main/cpp/server.hpp#L106
struct server_slot
{
int id;
int id_task = -1;
int id_multi = -1;
struct slot_params params;
slot_state state = SLOT_STATE_IDLE;
slot_command command = SLOT_COMMAND_NONE;
// used to determine the slot that has been used the longest
int64_t t_last_used = -1;
// generation props
int32_t n_ctx = 0; // context size per slot
int32_t n_past = 0;
int32_t n_decoded = 0;
int32_t n_remaining = -1;
int32_t i_batch = -1;
int32_t n_predict = -1; // T