pretrain

keyu-tian/SparK

[ICLR'23 Spotlight🔥] The first successful BERT/MAE-style pretraining on any convolutional network; Pytorch impl. of "Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling"

CLUEbenchmark/CLUECorpus2020

Large-scale Pre-training Corpus for Chinese 100G 中文预训练语料

yangjianxin1/Firefly-LLaMA2-Chinese

Firefly中文LLaMA-2大模型，支持增量预训练Baichuan2、Llama2、Llama、Falcon、Qwen、Baichuan、InternLM、Bloom等大模型

microsoft/UniVL

An official implementation for " UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation"

xcfcode/What-I-Have-Read

Paper Lists, Notes and Slides, Focus on NLP. For summarization, please refer to https://github.com/xcfcode/Summarization-Papers

THUNLP-AIPoet/BERT-CCPoem

BERT-CCPoem is an BERT-based pre-trained model particularly for Chinese classical poetry

thunlp/RE-Context-or-Names

Bert-based models(BERT, MTB, CP) for relation extraction.

huzongxiang/MatDGL

MatDGL is a neural network package that allows researchers to train custom models for crystal modeling tasks. It aims to accelerate the research and application of material science.

CoinCheung/MFM

code for paper "Masked Frequency Modeling for Self-Supervised Visual Pre-Training" (https://arxiv.org/pdf/2206.07706.pdf)

SalesforceAIResearch/pretrain-time-series-cloudops

Official code repository for the paper "Pushing the Limits of Pre-training for Time Series Forecasting in the CloudOps Domain"

nancheng58/SSL4SR

[CCIR 2023] Self-supervised learning for Sequential Recommender Systems

bayartsogt-ya/albert-mongolian

ALBERT trained on Mongolian text corpus

yongzhuo/MacroGPT-Pretrain

macrogpt大模型全量预训练(1b3,32层), 多卡deepspeed/单卡adafactor

mrzjy/hoyo_public_wiki_parser

Parsing Hoyoverse game text corpus from public wikipedia

pskliff/vtb-data-fusion

This repository provides code solution for Data Fusion Contest task 1

arrrrrmin/albert-guide

Understanding "A Lite BERT". An Transformer approach for learning self-supervised Language Models.

janelu9/flash-finetuning

Running Large Language Model easily.

tianhao-ai/Detecting-Machine-Generated-Text-COMP90051-2023S1-Project-1

This project is about to detecting the text generated by different LLM given prompt. The instance is labeled by Human and Machine, and this project utilised both traditional machine learning method and deep learning method to classify the instance.

afogarty85/applied_nlp_demos

stoneyang/cv-arxiv-daily

🎓Automatically Update CV Papers Daily using Github Actions (Update Every 24th hours)