IINemo/lm-polygraph

Issues

• cannot import name 'normalize_ue' from 'lm_polygraph.utils.normalize'

  #276 opened 10 months ago by chivalry1314
  1

• lack package.json file

  #277 opened 10 months ago by chivalry1314
  1

• It seems that the estimater_uncertainty method can only handle one str at a time. How to process multiple str in parallel? Can you provide corresponding examples

  #275 opened a year ago by eLeventhw
  1

• Using LM-Polygraph with Custom OpenAI Endpoint / with Pre-Generated Responses

  #263 opened a year ago by muelphil
  3

• Perplexity Calculation

  #273 opened a year ago by vthost
  3

• I would like to ask about the output of GPT3.5, how can GPT4 recognize related words when encountering this situation

  #260 opened a year ago by wangzhonghai
  6

• ways to compute ROC-AUC and the label

  #262 opened a year ago by EdwardChang5467
  4

• How to support VLM?

  #267 opened a year ago by Zhitao-He
  1

• Load custom estimators and stat_calculators in the evaluation script

  #219 opened a year ago by IINemo
  1

• Isn't CCP a white box model? How to test GPT-3.5-turbo

  #255 opened a year ago by wangzhonghai
  6

• Is there a requirement for the Python version to run this repository on a Linux system？

  #261 opened a year ago by EdwardChang5467
  8

• Problems with tokenizer for LLMs without instruct-tuning

  #239 opened a year ago by IINemo
  0

• Get the uncertainty scores without rerun the models (for NumSets, Deg, Ecc)

  #155 opened 2 years ago by caiqizh
  4

• Example for normalizaiton

  #226 opened a year ago by YiJohnny
  1

• Semantic entropy is using probabilites greater than 1

  #223 opened a year ago by gyampols
  2

• generate_texts on wbmodel ignores generation parameters and stopping critera.

  #224 opened a year ago by rvashurin
  0

• About the generation metric for questions with multiple correct answers

  #207 opened a year ago by caiqizh
  2

• Dockerfile adjustments

  #206 opened a year ago by SuperCoolCucumber
  3

• using the openai API for a BlackBox model for non OPENAI hosted platforms

  #166 opened 2 years ago by psteinb
  4

• Error loading larger models - You shouldn't move a model when it is dispatched on multiple devices

  #147 opened a year ago by avi-jain
  5

• Not 100% sure if sampling parameters is correct

  #119 opened a year ago by kirill-fedyanin
  3

• Possible mismatching max_length and max_new_tokens in example eval script

  #118 opened 2 years ago by kirill-fedyanin
  4

• Entropy calculation maybe wrong?

  #195 opened 2 years ago by athrvkk
  6

• May I know how to run an ensemble-based uncertainty estimation method?

  #170 opened 2 years ago by he159ok
  2

• [Question] Pipeline integration (Langchain)

  #165 opened 2 years ago by Rebell-Leader
  1

• What is the different between pe_uncertainties and ep_uncertainties?

  #164 opened 2 years ago by HuihuiChyan
  2

• AutoModelForCausalLM max_length

  #163 opened 2 years ago by amayuelas
  1

• Get the uncertainty scores without rerun the models

  #144 opened 2 years ago by caiqizh
  4

• Demo doesn't work.

  #142 opened 2 years ago by caiqizh
  4