Pinned Repositories
architecture-objective
bigscience
Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.
biomedical
Tools for curating biomedical training data for large-scale language modeling
data-preparation
Code used for sourcing and cleaning the BigScience ROOTS corpus
lm-evaluation-harness
A framework for few-shot evaluation of autoregressive language models.
Megatron-DeepSpeed
Ongoing research training transformer language models at scale, including: BERT & GPT-2
petals
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
promptsource
Toolkit for creating, sharing and using natural language prompts.
t-zero
Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)
xmtf
Crosslingual Generalization through Multitask Finetuning
BigScience Workshop's Repositories
bigscience-workshop/data_sourcing
This directory gathers the tools developed by the Data Sourcing Working Group
bigscience-workshop/bigscience-workshop.github.io
Alternative to https://github.com/Dynalon/mdwiki-seed
bigscience-workshop/amazon-sagemaker-mlflow-fargate
Managing your machine learning lifecycle with MLflow and Amazon SageMaker
bigscience-workshop/scaling-laws-tokenization
scaling-laws-tokenization
bigscience-workshop/codecarbon
Track emissions from Compute and recommend ways to reduce their impact on the environment.