stanford-futuredata/ARES

Issues

• Evaluation process only works with demo datasets, fails with any real dataset (that has only the columns described in the paper)

  #53 opened a month ago by elsatch
  3

• Azure OpenAI support

  #59 opened a month ago by Lilian-Silveira
  2

• Difference between Answer Relevance and Answer Faithfulness

  #72 opened 3 months ago by WJ44
  1

• What is meant by contrastive learning objective?

  #74 opened 3 months ago by majdsalman
  0

• No strong negative generation

  #73 opened 3 months ago by WJ44
  0

• Unclear about the code of calculate_ppi function

  #64 opened 5 months ago by kitkhai
  1

• Error while training LLM judge

  #66 opened 5 months ago by WJ44
  4

• Source of nq_labeled_output.tsv

  #70 opened 4 months ago by WJ44
  0

• Reproducing results

  #65 opened 4 months ago by WJ44
  3

• OutOfMemoryError when Running Example Code with ARES

  #61 opened 5 months ago by chienyutseng
  1

• Other checkpoints

  #60 opened 5 months ago by MarineHoche
  1

• Error while training classifier

  #68 opened 5 months ago by islambs231bi
  0

• Error while evaluating with PPI with API model LLM judge

  #67 opened 5 months ago by islambs231bi
  0

• The local model needs to pass in a token in order to request it, May I ask if there is such a parameter

  #62 opened 5 months ago by starplatinum3
  1

• Training classifiers error

  #63 opened 5 months ago by Islam231bi
  0

• LLM approach for synthetic query generation not using specified prompt

  #58 opened 5 months ago by WJ44
  1

• ARES as a Chunk Reranker in a RAG app?

  #55 opened 5 months ago by naskovai
  3

• Missing packages in linked Colab notebook

  #46 opened 5 months ago by elsatch
  3

• None of the tutorials work

  #54 opened 5 months ago by naskovai
  9

• TGI Support

  #56 opened 5 months ago by ss2342
  0

• Missing protobuf dependencies in ARES 0.6.1 PyPi package

  #49 opened 6 months ago by elsatch
  1

• [bug] error during import - vLLM not imported.

  #48 opened 6 months ago by mglowacki100
  1

• getting TypeError: 'type' object is not subscriptable when importing the package

  #47 opened 6 months ago by kabani7
  1

• Iteration over labels and datasets not working in PPI

  #44 opened 6 months ago by WJ44
  4

• Evaluating more than one dataset at a time returns incorrect results

  #50 opened 6 months ago by elsatch
  0

• Precision-Performance Iteration (PPI) in README

  #43 opened 7 months ago by marcjulianschwarz
  1

• [Feature Request] Multilingual support

  #20 opened 7 months ago by sociengineer
  1

• Unable to import without setting OpenAI key

  #30 opened 7 months ago by sumitsoman
  1

• [Feature Request] AWS Bedrock / Anthropic Claude

  #18 opened 7 months ago by austinmw
  1

• --labels <label columns>

  #19 opened 8 months ago by Pranshul200
  2

• Clarification Needed on the Specificity of test_dataset

  #22 opened 8 months ago by ViceSilva
  1

• Checkpoint folder is not created automatically after training classifiers

  #24 opened 8 months ago by elsatch
  1

• New README file instructions are incorrect

  #27 opened 8 months ago by elsatch
  1

• Documentation and code are so broken!

  #23 opened 8 months ago by elsatch
  5

• Switch openai embeddings to local multilingual embeddings

  #25 opened 8 months ago by elsatch
  2

• RAGAS score calculation from annotations is unclear

  #4 opened a year ago by pchunduri6
  1

• strong negative generation

  #1 opened a year ago by SimonYx42
  1