shigashiyama's Stars
HumanSignal/label-studio
Label Studio is a multi-type data labeling and annotation tool with standardized output format
openai/gpt-3
GPT-3: Language Models are Few-Shot Learners
BlinkDL/RWKV-LM
RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
openai/tiktoken
tiktoken is a fast BPE tokeniser for use with OpenAI's models.
Mooler0410/LLMsPracticalGuide
A curated list of practical guide resources of LLMs (LLMs Tree, Examples, Papers)
microsoft/DeepSpeedExamples
Example models using DeepSpeed
iryna-kondr/scikit-llm
Seamlessly integrate LLMs into scikit-learn.
kha-white/manga-ocr
Optical character recognition for Japanese text, with the main focus being Japanese manga
explosion/spacy-llm
🦙 Integrating LLMs into structured NLP pipelines
allenai/mmc4
MultimodalC4 is a multimodal extension of c4 that interleaves millions of images with text.
mandarjoshi90/coref
BERT for Coreference Resolution
ghmagazine/llm-book
「大規模言語モデル入門」(2023)と「大規模言語モデル入門Ⅱ〜生成型LLMの実装と評価」(2024)のGitHubリポジトリ
clarkkev/deep-coref
webanno/webanno
🆕 Work continues on INCEpTION 👉 https://github.com/inception-project/inception 👈 -- ⚠️ The official WebAnno repository has reached the end of the line. -- 🚀 To migrate, export your annotation projects from WebAnno, then import them into INCEpTION and just work on.
google-research-datasets/KELM-corpus
Kei18/awesome_cs-ja_phd_life
collection of articles about PhD life written in 🇯🇵
C-J-Cundy/gpt4-tokenizer
Hosting the JSON for the GPT4 Tokenizer
jqk09a/japanese-daily-dialogue
ns-moosavi/coval
A coreference evaluation package for the CoNLL and ARRAU datasets
KoichiYasuoka/UniDic2UD
Tokenizer POS-tagger Lemmatizer and Dependency-parser for modern and contemporary Japanese
aiishii/JEMHopQA
flairNLP/zelda
A comprehensive benchmark for entity disambiguation
jonnyli1125/gector-ja
BERT-based GEC tagging for Japanese
CLARIN-PL/Inforex
Inforex is a web system for text corpora construction.
explosion/conll-2012
A slightly cleaned up version of the scripts & data for the CoNLL 2012 Coreference task.
t-sagara/Japanese-Address-testdata
解析が難しい日本の住所のテストデータセット
henryrosalesmendez/NIFify_v2
korguchi/text2phoneme
日本語文を音素列へ変換するスクリプト
AnhHoang0529/Small-LexNormViHSD
A Dataset for Vietnamese Lexical Normalization
henryrosalesmendez/categorized_EMNLP_datasets