The objective of this project is to re-implement the Transformer architecture as described in the paper "Attention is All You Need" and use it to train a machine translation model for translating English to Indonesian on the Opus-100 dataset from Helsinki-NLP, which consists of 1 million sentence pairs.
The project utilizes the Opus-100 dataset from Helsinki-NLP, containing 1 million parallel sentence pairs in English and Indonesian. This extensive dataset enables the development of a robust machine translation model.
- Improved Translation Quality: Enhanced translation accuracy and fluency between English and Indonesian.
- Scalable Training: Distributed training setup allows efficient use of multiple GPUs for faster training times.
- Research Contribution: Provides a high-quality implementation of the Transformer architecture for the research community.
- Resource Optimization: Efficient utilization of computational resources through distributed training.
- Enhanced NLP Capabilities: Contributes to the development of NLP tools and applications for the Indonesian language.
The project focuses on:
- Re-implementing the Transformer architecture as described in the seminal paper.
- Training the model on the Opus-100 dataset for English-Indonesian translation.
- Setting up distributed training using PyTorch to leverage multiple GPUs for efficient training.
- Evaluating the model's performance and comparing it with existing benchmarks.