/FinTab-LLaVA

Financial Domain-Specific Table Understanding Multimodal LLM

Primary LanguagePythonApache License 2.0Apache-2.0

FinTab-LLaVA

  • Last updated 2024.10.15
  • The code has been updated! The tutorial will be updated soon.

Model

  • The FinTab-LLaVA model is built on the multimodal architectures of LLaVA 1.5 and Table-LLaVA.
  • We fine-tuned Table-LLaVA using the Finance Table Multimodal Dataset (FinTMD).
  • Additionally, we fine-tuned the model using domain knowledge training on math tasks and financial text data.
Finetuning Version Checkpoint
Only Finance Table FinTable-Llava-Task
Finance Table + Math FinTable-Llava-Math
Finance Table + Finance FinTable-Llava-Finance
Finance Table + Math + Finance FinTable-Llava-All

Dataset