/AdvancedNLP

Material for a course on Advanced NLP

Primary LanguageHTML

AdvancedNLP

Sessions

  1. Recap on Deep Learning & basic NLP (slides / lab session)
  2. Tokenization (slides / lab session)
  3. Language Modeling (slides / lab session)
  4. NLP without 2048 GPUs (slides / lab session)
  5. Handling the Risks of Language Models (slides / lab session)
  6. Advanced NLP tasks (slides / lab session)
  7. Domain-specific NLP (slides / [lab session])
  8. Multilingual NLP (slides / lab session)
  9. Multimodal NLP (slides / lab session)

Evaluation

⚠️ There is no oral presentation/evaluation for this course.

The evaluation consists in a team project (3-5 people). There are two options:

  • Demo : Use a well-known approach to produce a MVP for an original use-case and present it in a demo.
    • Example: An online platform that detects AI-generated text.
  • R&D : Based on a research article, conduct original experiments and produce a report. (see Potential articles)
    • Example: Do we need Next Sentence Prediction in BERT? (Answer: No)

It will consist of three steps:

  • Team announcement (before 15/12/23): send an email to nathan.godey@inria.fr with cc's matthieu.futeral@inria.fr and francis.kulumba@inria.fr explaining
    • The team members (also cc'ed)
    • Type of project and vague description (can change afterwards)
  • Project plan (30% of final grade, before 07/01/23): following this template, produce a project plan explaining first attempts (e.g. version alpha), how they failed/succeeded and what you want to do before the delivery.
  • Project delivery (70% of final grade, before mid-February): deliver a nb_team_members * 2 pages project report and a GitHub repo (more details coming soon)

Potential articles

Tokenization

Fast inference

LLM detection

SSMs (off-program)

Alignment & Safety