symflower/eval-dev-quality

Issues

• Automatic Markdown export

  #49 opened 8 months ago
  0

• Automatic Symflower installation with fixed version

  #47 opened 8 months ago
  0

• Infer if a model produced "too much" code

  #44 opened 8 months ago
  0

• Infer if a model actually returned source code

  #43 opened 5 months ago
  0

• Scoring and Ranking

  #41 opened 8 months ago
  1

• Roadmap for v0.4.0

  #35 opened 7 months ago
  10

• Refactor metrics to assessment

  #34 opened 8 months ago
  0

• Automatic response categorization

  #32 opened 8 months ago
  7

• Implement "chain of thought" tasks

  #31 opened 8 months ago
  0

• Retry with feedback and retry without feedback.

  #30 opened 8 months ago
  0

• Better prompting with templates: Setting a mandatory document start

  #29 opened 8 months ago
  0

• Better error handling - panic instead of "failing up"

  #28 opened 2 months ago
  2

• Do an up-to-date leaderboard/dashboard for current models current evaluation

  #26 opened 8 months ago
  0

• Upload binaries of the evaluation binary for all OSes and architectures for users that only want to benchmark

  #20 opened 8 months ago
  0

• Include linters in the development environment and CI

  #18 opened 2 months ago
  1

• Sandbox execution

  #17 opened 5 months ago
  1