openai/human-eval

Issues

• I have the windows problem

  #51 opened 17 days ago by albertimff
  1

• issue

  #50 opened 25 days ago by grattitude
  1

• Problems with installation instructions

  #4 opened 3 years ago by jonpincus
  4

• I run the test program ，but why my “example_samples.jsonl_results.jsonl” file displayed “all passed” ？

  #49 opened 3 months ago by CHfaithwy
  0

• Evaluation doesn't work on Windows

  #45 opened 6 months ago by peter-ch
  4

• Error running evaluate_functional_correctness samples.json

  #48 opened 4 months ago by nextdoorUncleLiu
  0

• About how to use

  #47 opened 4 months ago by RIKI0913
  0

• why use ThreadPoolExecutor with GIL in background?

  #36 opened a year ago by johnmclain
  1

• Tests in task 67 are impossible to satisfy

  #44 opened 7 months ago by jack-jjm
  0

• Task 145 makes no sense

  #43 opened 7 months ago by jack-jjm
  0

• evaluate_functional_correctness can't run

  #18 opened 7 months ago by BoyuanJackChen
  9

• AttributeError: Can't pickle local object 'check_correctness.<locals>.unsafe_execute'

  #27 opened 2 years ago by tianzhaotju
  6

• Removed deprecated fields.

  #42 opened 9 months ago by Youniqueli
  0

• Error in tests for HumanEval/163

  #41 opened 10 months ago by mono-jiarui
  0

• Evaluations timing out

  #40 opened 10 months ago by antonkarlsson1
  0

• When running the code generated by the model, an error occurs: failed: No module named 'scipy'

  #39 opened a year ago by HangXue-lab
  0

• HE vAL

  #37 opened a year ago by Youniqueli
  1

• Why pass@k =1.0? use the "evaluate_functional_correctness data/example_samples.jsonl --problem_file=data/example_problem.jsonl"

  #16 opened 2 years ago by Smithol
  3

• bug in estimate_pass_at_k

  #35 opened a year ago by sidaw
  0

• Entry point error while Installing the package.

  #12 opened 3 years ago by Ali1858
  1

• Why do I use the phi model to output the same result for all samples at a temperature of 0.8?

  #34 opened a year ago by Mrzhang-dada
  0

• Where to find the leaderboard?

  #33 opened a year ago by zhimin-z
  0

• I do not understand how to run human eval

  #32 opened a year ago by teknium1
  1

• Error in canonical solution and tests for HumanEval/163

  #20 opened 2 years ago by bmosaicml
  1

• Evaluation.py failing on KeyError: 'test/0'

  #10 opened 3 years ago by briviere
  3

• Error in the prompt of HumanEval/47

  #6 opened 3 years ago by Kim-mins
  1

• file missing

  #24 opened 2 years ago by shuaiwang2022
  1

• Codex Training Data

  #19 opened 2 years ago by zachares
  1

• # Realizar ajustes en la arquitectura

  #26 opened 2 years ago by D0yi
  1

• Error in canonical solution 95 check_dict_case

  #22 opened 2 years ago by PootieT
  0

• Re-produce raw GPT-Neo with 125M and 1.3B on this human-eval dataset

  #8 opened 3 years ago by BitcoinNLPer
  1

• Will this be helpful for people reading the paper ?

  #1 opened 3 years ago by jalotra
  2

• Finetuning With HumanEval

  #17 opened 2 years ago by MT010104
  0

• Why not allow contribution?

  #15 opened 3 years ago by rafidka
  0

• pass@k on filtered samples

  #13 opened 3 years ago by henryhungle
  0

• Prompt used in APPS

  #11 opened 3 years ago by henryhungle
  2

• Question about the generate_one_completion

  #5 opened 3 years ago by xupeng1910
  1

• execution.py bug request

  #3 opened 3 years ago by rainmaker712
  1