gkamradt/LLMTest_NeedleInAHaystack

Issues

• Can I use local LLM as the evaluator and provider?

  #47 opened 7 months ago by hijkzzz
  6

• Different prompts in providers - I just wonder why cohere don't have "Don't give information outside the document or repeat your findings" and does it make a bit difference?

  #50 opened 6 months ago by radarFudan
  0

• add base_url env in openai provider - to support OpenAI compatibility local inference like - ollama, tgi, etc

  #49 opened 6 months ago by backroom-coder
  0

• How can we cite the Needle-in-a-Haystack?

  #48 opened 7 months ago by rozyang
  1

• Possibility to specify custom API endpoint address?

  #45 opened 8 months ago by ElliottDyson
  4

• Question: Can the Haystack have variations?

  #44 opened 8 months ago by BradKML
  3

• does it run at all? Basic commands failed to run as per the README.

  #43 opened 8 months ago by iitrsamrat
  1

• [Feature Proposal] Multi-needle in a haystack

  #41 opened 8 months ago by jsharf
  2

• I was wondering about the evaluation method

  #39 opened 8 months ago by gauss5930
  2

• multi-needle-eval-pizza-3 dataset not found

  #34 opened 9 months ago by gkamradt
  1

• Convert the repository to a PyPi package

  #31 opened 9 months ago by kedarchandrayan
  1

• Remove passing of API keys as parameters and read them from environment variables

  #32 opened 9 months ago by kedarchandrayan
  1

• Update package Anthropic

  #13 opened 9 months ago by pavelkraleu
  2

• Model kwargs support

  #21 opened 9 months ago by LazaroHurtado
  1

• Standard Tokenizer

  #25 opened 9 months ago by prabha-git
  12

• Install pre-commit with end-of-file-fixer

  #11 opened 9 months ago by pavelkraleu
  0

• Replace os.path with Pathlib

  #12 opened 9 months ago by pavelkraleu
  0

• Implement Docker for testing

  #17 opened 9 months ago by pavelkraleu
  1

• Add Makefile target for resetting run results

  #23 opened 9 months ago by LazaroHurtado
  1

• Anthropic Naming Conflict Error

  #15 opened 9 months ago by prabha-git
  2

• Code optimizations

  #20 opened 9 months ago by LazaroHurtado
  0

• hard coding of 'gpt-4' for evaluation

  #10 opened 9 months ago by prabha-git
  0

• Azure OpenAI key

  #3 opened a year ago by cobraheleah
  2

• Add license file

  #4 opened a year ago by haesleinhuepf
  4

• Can you share the PowerPoint where you mapped the final result?

  #1 opened a year ago by jarheadjoe
  1