/atis_solution_mz

Primary LanguagePython

Text NER Model Training and Evaluation

This codebase includes Python code to train and evaluate a Named Entity Recognition (NER) model for identifying return date and time entities in text data.

Overview

The code employs Hugging Face's Transformers library to fine-tune a pre-trained model for NER on a dataset containing text excerpts. The model aims to identify return date and time entities within the provided conversations.

Code Structure

  • train_atis.py: Python script containing the pipeline for data preprocessing, model fine-tuning, and evaluation.
  • atis_train.csv: Placeholder for the actual dataset containing text data.

Getting Started

Requirements

  • Python (>=3.6)
  • Hugging Face Transformers library
  • pandas
  • sklearn

Usage

  1. Place your text dataset file (your_dataset.csv) in the repository.
  2. Update the code in main.py for data preprocessing and model fine-tuning based on your specific dataset.
  3. Run the main.py script to train the NER model and evaluate its performance.

Evaluation Metrics

After model training, the script computes and prints the following evaluation metrics:

  • Accuracy
  • Precision
  • Recall
  • F1-score

These metrics indicate the model's performance in identifying return date and time entities within the text data.

Feel free to modify the code according to your dataset and requirements for NER tasks on text data.