NVIDIA/NeMo-Skills

Issues

• Add back vllm dockerfile to be able to make changes or have custom commit

  #335 opened 2 months ago by Kipok
  1

• Upgrade rm api to be compatible with latest vllm

  #311 opened 2 months ago by Kipok
  1

• Add a way to automatically split generation/eval into chunks

  #312 opened 3 months ago by Kipok
  0

• Raise an error in input files are empty

  #150 opened 3 months ago by Kipok
  1

• Have a way to use only part of a slurm node

  #165 opened 3 months ago by Kipok
  1

• It's not possible to run multiple tasks at the same time locally

  #155 opened 3 months ago by Kipok
  2

• Expose nemo_skills/inference/retrieve_similar.py as a pipeline script

  #156 opened 3 months ago by Kipok
  1

• Expose nemo_skills/evaluation/fill_majority_answer.py as a pipeline script

  #157 opened 3 months ago by Kipok
  1

• Expose nemo_skills/training/prepare_sft_data.py as a pipeline script

  #158 opened 3 months ago by Kipok
  1

• Replace llm-math-judge and decontamination inference scripts with pre/post-process commands

  #280 opened 4 months ago by Kipok
  0

• Skip conversion to a tmp model for trtllm if it's present. Add a way to pass parameters to the first stage

  #149 opened 4 months ago by Kipok
  0

• Add a check that benchmarks parameter in eval is properly formatted

  #153 opened 4 months ago by Kipok
  0

• Upgrade our stack to trtllm 0.13

  #173 opened 4 months ago by Kipok
  1

• Change metrics logic to always report all available aggregations

  #184 opened 4 months ago by Kipok
  1

• Issues of OpenMathInstruct-2 dataset decontamination

  #235 opened 5 months ago by henryj18
  1

• error while converting to TensorRT-LLM

  #234 opened 5 months ago by stephencge
  10

• Support packing?

  #214 opened 5 months ago by fzyzcjy
  15

• Why cmdline completion doesn't work?

  #152 opened 5 months ago by Kipok
  6

• Display progress in summarize_results download

  #147 opened 5 months ago by Kipok
  3

• Check for the required antlr package version

  #169 opened 5 months ago by Kipok
  0

• Change docs to be in github pages

  #172 opened 6 months ago by Kipok
  1

• error while reproducing results (llm_math_judge)

  #185 opened 6 months ago by yld3
  6

• how to cleanup an interrupted run ?

  #188 opened 6 months ago by yld3
  1

• Expose data preparation commands through ns interface

  #174 opened 6 months ago by Kipok
  0

• Add unittests for cluster config logic

  #171 opened 6 months ago by Kipok
  0

• Properly integrate OpenAI's batch api into all generation scripts

  #170 opened 6 months ago by Kipok
  0

• Set a good default validation batch size or find a way to disable it

  #167 opened 6 months ago by Kipok
  0

• Verify that we can use NIM containers on slurm and locally in a convenient way

  #166 opened 6 months ago by Kipok
  0

• Add some helper functions to create new pipeline scripts

  #159 opened 6 months ago by Kipok
  0

• Add some simple cli to upload/download data from slurm clusters by using mounts in our configs

  #162 opened 6 months ago by Kipok
  0

• Add checks for extra arguments correctness to some of the scripts

  #161 opened 6 months ago by Kipok
  0

• Add an option to prepare data in the DPO format (or a separate script)

  #160 opened 6 months ago by Kipok
  0

• Add another test(s) with pip installable library without -e flag

  #154 opened 6 months ago by Kipok
  0

• Make all pipeline scripts easily importable

  #151 opened 6 months ago by Kipok
  0

• Change all scripts in pipeline to explicitly accept any parameters that need to exist on filesystem and check that

  #148 opened 6 months ago by Kipok
  0

• Use SSHTunnel for slurm-based start_server requests

  #146 opened 6 months ago by Kipok
  0

• KeyError: 'reference_masked_solution'

  #26 opened 10 months ago by zdaiot
  27

• Some Issues Regarding Code Replication

  #21 opened a year ago by seamoke
  5

• Consider open sourcing eval generations

  #8 opened a year ago by dylan-slack
  2

• Could we use this to improve MMLU capability?

  #3 opened a year ago by YixinSong-e
  1

• Some links are broken

  #2 opened a year ago by maxjeblick
  2