NV-JM-DD

A collection of resources related to Project Name_TBD

Introduction, Theory

Must Watch

CS224n: Natural Language Processing with Deep Learning
http://onlinehub.stanford.edu/cs224

NLP, General

Repository to track the progress in Natural Language Processing
https://github.com/sebastianruder/NLP-progress

Modern Deep Learning Techniques Applied to Natural Language Processing
https://nlpoverview.com/

Word/Sentence Contextual representation, Language Models

Methods for computing sentence representations from pretrained word embeddings without any additional training
https://code.fb.com/ml-applications/random-encoders/

Contextual Word Representations
https://arxiv.org/abs/1902.06006

Sentence and Contextualized Word Representations
https://www.youtube.com/watch?v=Ioqrw4sCcwQ

Introduction and discussion on various Generalized Language Models
https://lilianweng.github.io/lil-log/2019/01/31/generalized-language-models.html

Cr5, model for crosslingual document embedding Input: text in any of 28 languages Output: language-independent vector representation, so you can compare text across langs Pre-trained model and API: https://github.com/epfl-dlab/Cr5 (Numpy/cython/Intel stuff) Paper: "Crosslingual Document Embeddingas Reduced-Rank Ridge Regression" https://dlab.epfl.ch/people/west/pub/Josifoski-Paskov-Paskov-Jaggi-West_WSDM-19.pdf

Knowledge and Representations

A Unified Theory of Inference for Text Understanding
https://www2.eecs.berkeley.edu/Pubs/TechRpts/1987/CSD-87-339.pdf

Knowledge Graph Intros
https://towardsdatascience.com/the-data-fabric-for-machine-learning-part-2-building-a-knowledge-graph-2fdd1370bb0a https://medium.com/@sderymail/challenges-of-knowledge-graph-part-1-d9ffe9e35214

A general framework for learning context-based action representation for Reinforcement Learning
https://arxiv.org/abs/1902.01119

Deep Learning with Knowledge Graphs
https://medium.com/octavian-ai/deep-learning-with-knowledge-graphs-3df0b469a61a

Geometry of Thought
http://nautil.us/blog/new-evidence-for-the-geometry-of-thought

Geometry of Meaning
https://www.youtube.com/watch?v=L0X9mEe9aY0&app=desktop

Interacting Conceptual Spaces
https://arxiv.org/abs/1608.01402

Decentralised Knowledge Graph
https://www.youtube.com/watch?v=hm9ibPZOUcw https://fosdem.org/2019/schedule/event/graph_weaviate_knowledge_graph/

Sign Language

Sign Language & Linguistics
https://benjamins.com/catalog/sll

Tool, Code, Application

NLP

StanfordNLP, a Python natural language analysis package
https://stanfordnlp.github.io/stanfordnlp/

OpenAI GPT2
https://openai.com/blog/better-language-models/

PyTorch Pretrained BERT: The Big & Extending Repository of pretrained Transformers
https://github.com/huggingface/pytorch-pretrained-BERT

Knowledge

How NASA experiments with knowledge discovery
https://linkurio.us/blog/how-nasa-experiments-with-knowledge-discovery/

Inspection tools

Giant Language model Test Room
http://gltr.io/dist/index.html

Generation

Plotto, "algebra" for stories
https://github.com/garykac/plotto/blob/gh-pages/how-to.md

Graph neural network in NLP, Geometric Deeplearning on graphs

Geometric deep learning
http://geometricdeeplearning.com/

Data

Corpus

A straightfoward library that allows you to crawl, clean up, and deduplicate webpages to create massive monolingual datasets
https://github.com/chiphuyen/lazynlp