MadcowD/ell

Issues

• No client found for model (Anthropic)

  #408 opened 5 days ago by britannio
  0

• Have no response when using the custom model client

  #407 opened 5 days ago by YuhaoCheng
  0

• Prompt not properly versioned when in a class

  #404 opened 8 days ago by brenorb
  2

• Out of date doc reference in ImportError raised

  #402 opened 11 days ago by apandy02
  1

• Weird error when executing import ell

  #405 opened 6 days ago by a2ajinkya
  1

• Evaluation - ValueError: Invalid input type: <class 'str'> - Correction Found

  #398 opened 16 days ago by brenorb
  2

• eval error

  #399 opened 14 days ago by MadcowD
  0

• Evaluation example is wrong.

  #397 opened 17 days ago by brenorb
  0

• about multimodal

  #367 opened 2 months ago by a136214808
  2

• Embedding view for distribution of outputs with a good looking TSNE for prompting to show if meaning ful distributional changes are occuring or something.

  #348 opened 2 months ago by MadcowD
  1

• Cache completions

  #389 opened a month ago by abrichr
  2

• Stream api

  #392 opened a month ago by alex-dixon
  2

• LMPs return a 3-tuple when `exempt_from_tracking=True`

  #394 opened 24 days ago by jwang-paradise
  1

• Export dataset functionality in ell-studio

  #393 opened 25 days ago by lingster
  0

• api design: parsed.model_dump()

  #391 opened a month ago by pearli-pearli
  1

• Tool call renders inconsistently in studio

  #387 opened a month ago by alex-dixon
  0

• Tool call missing params in studio

  #386 opened a month ago by alex-dixon
  0

• Dataset construction needs to be easy and there should be libraries around this, also matching parity with OpenAI evals

  #382 opened a month ago by MadcowD
  0

• Easily be able to isolate datapoints below. a certain score etx

  #381 opened a month ago by MadcowD
  0

• # Be able to see the outputs of the criterion function as it used in the eval.

  #380 opened a month ago by MadcowD
  0

• Working verbose mode for @function

  #379 opened a month ago by MadcowD
  0

• Easy comparison across many models

  #378 opened a month ago by MadcowD
  0

• Run summaries should work with annotations and no metrics or no metrics at all

  #377 opened a month ago by MadcowD
  0

• Build a UX which is compatible with no metrics & prompt engineering by vibes.

  #376 opened a month ago by MadcowD
  0

• Summaries for other type of data like annotations or criterion

  #374 opened a month ago by MadcowD
  0

• Implement a bunch of standard criteria

  #373 opened a month ago by MadcowD
  0

• UX for showing the model is different

  #372 opened a month ago by MadcowD
  0

• UX for api params

  #371 opened a month ago by MadcowD
  0

• clicking the graph or tooltip opens the run

  #370 opened a month ago by MadcowD
  0

• default_client is ignored if model is in default registry

  #366 opened 2 months ago by ppabis
  3

• how to host studio on the linux server?

  #340 opened 2 months ago by keyuchen21
  4

• Fix the store interface.

  #360 opened 2 months ago by MadcowD
  1

• need to sovle a really good UX for prpompt engineering with no criterion comparion outputs etc

  #359 opened 2 months ago by MadcowD
  0

• Clarity into why a currently running invocation is working or not

  #358 opened 2 months ago by MadcowD
  0

• Full UX for comparing different evals across any arbitrary axis

  #357 opened 2 months ago by MadcowD
  0

• Group runs more cleanly so that they are a part of an eval in the invocation view

  #356 opened 2 months ago by MadcowD
  0

• Implement parsers/structured outputs to make this cleaner

  #355 opened 2 months ago by MadcowD
  0

• Support failure modes in metric computation

  #354 opened 2 months ago by MadcowD
  0

• Fix ell.function in general

  #353 opened 2 months ago by MadcowD
  0

• Distributional view if you hover over one individual datapoint.

  #352 opened 2 months ago by MadcowD
  0

• Resumption of partilaly executed evals..

  #351 opened 2 months ago by MadcowD
  0

• Commit messages for evals.

  #350 opened 2 months ago by MadcowD
  0

• Show the actual scores for a given input on ell studio as opposed to just the mean

  #349 opened 2 months ago by MadcowD
  0

• Open ell studio automatically when the eval gets run...

  #347 opened 2 months ago by MadcowD
  0

• Arbitrary support for failure mode in lmp invocations

  #346 opened 2 months ago by MadcowD
  0

• ell studio can't work

  #345 opened 2 months ago by femto
  1

• changing model at runtime is not the cleanest

  #334 opened 2 months ago by MadcowD
  4

• Round trip Message serialization/deserialization

  #336 opened 2 months ago by alex-dixon
  3

• ToolCall uses sqlmodel Field instead of pydantic Field for tool_call_id

  #342 opened 2 months ago by alex-dixon
  0

• Ell not tracking invocation chain between nested calls

  #338 opened 2 months ago by JohnPeng47
  1