Data-Centric FinGPT: Open-source for Open Finance.

Let us DO NOT expect Wall Street to open-source LLMs nor open APIs, due to FinTech institutes' internal regulations and policies.

We democratize Internet-scale data for financial large language models (FinLLMs) at FinNLP and FinNLP Website

Blueprint of FinGPT

Disclaimer: We are sharing codes for academic purposes under the MIT education license. Nothing herein is financial advice, and NOT a recommendation to trade real money. Please use common sense and always first consult a professional before trading or investing.

Why FinGPT?

1). Finance is highly dynamic. BloombergGPT retrains an LLM using a mixed dataset of finance and general data sources, which is too expensive (1.3M GPU hours, a cost of around $5M). It is costly to retrain an LLM model every month or every week, so lightweight adaptation is highly favorable in finance. Instead of undertaking a costly and time-consuming process of retraining a model from scratch with every significant change in the financial landscape, FinGPT can be fine-tuned swiftly to align with new data (the cost of adaptation falls significantly, estimated at less than $416 per training).

2). Democratizing Internet-scale financial data is critical, which should allow timely updates (monthly or weekly updates) using an automatic data curation pipeline. But, BloombergGPT has privileged data access and APIs. FinGPT presents a more accessible alternative. It prioritizes lightweight adaptation, leveraging the strengths of some of the best available open-source LLMs, which are then fed with financial data and fine-tuned for financial language modeling.

3). The key technology is "RLHF (Reinforcement learning from human feedback)", which is missing in BloombergGPT. RLHF enables an LLM model to learn individual preferences (risk-aversion level, investing habits, personalized robo-advisor, etc.), which is the "secret" ingredient of ChatGPT and GPT4.

FinGPT Demos

FinGPT V3 (Updated on 7/11/2023)

FinGPT v3 (FinGPT_ChatGLM2_Sentiment_Instruction_LoRA_FT) is a LLM finetuned with LoRA method on the News and Tweets sentiment analysis dataset which achieve best scores on most of the financial sentiment analysis datasets.

Benchmark Results:

Weighted F1	BloombergGPT	ChatGLM2	ChatGLM2 (8-bit)	FinGPT v3	FinGPT v3 (8-bit)
FPB	0.511	0.381	0.398	0.795	0.778
FiQA-SA	0.751	0.79	0.801	0.806	0.801
TFNS	-	0.189	0.19	0.74	0.721
NWGI	-	0.449	0.452	0.578	0.578

FinGPT V2
- Let's train our own FinGPT in American Financial Market with LLaMA and LoRA (Low-Rank Adaptation)
FinGPT V1
- Let's train our own FinGPT in Chinese Financial Market with ChatGLM and LoRA (Low-Rank Adaptation)

Understanding FinGPT: An Educational Blog Series

What is FinGPT and FinNLP?

The Goals of FinGPT

Real-time data curation pipeline to democratize data for FinGPT
Lightweight adaptation to democratize the FinGPT model for both individuals and institutes (frequent updates)
Support various financial applications

FinNLP provides a playground for all people interested in LLMs and NLP in Finance. Here we provide full pipelines for LLM training and finetuning in the field of finance. The full architecture is shown in the following picture. Detail codes and introductions can be found here. Or you may refer to the wiki

End-to-end framework: FinGPT embraces a full-stack framework for FinLLMs with four layers:

Data source layer: This layer assures comprehensive market coverage, addressing the temporal sensitivity of financial data through real-time information capture.
Data engineering layer: Primed for real-time NLP data processing, this layer tackles the inherent challenges of high temporal sensitivity and low signal-to-noise ratio in financial data.
LLMs layer: Focusing on a range of fine-tuning methodologies such as LoRA, this layer mitigates the highly dynamic nature of financial data, ensuring the model’s relevance and accuracy.
Application layer: Showcasing practical applications and demos, this layer highlights the potential capability of FinGPT in the financial sector.

News

Columbia Perspectives on ChatGPT
[MIT Technology Review] ChatGPT is about to revolutionize the economy. We need to decide what that looks like
[BloombergGPT] BloombergGPT: A Large Language Model for Finance
[Finextra] ChatGPT and Bing AI to sit as panellists at fintech conference

ChatGPT at AI4Finance

[YouTube video] I Built a Trading Bot with ChatGPT, combining ChatGPT and FinRL.
Hey, ChatGPT! Explain FinRL code to me!
ChatGPT Robo Advisor v2
ChatGPT Robo Advisor v1
- A demo of using ChatGPT to build a Robo-advisor
ChatGPT Trading Agent V2
- A FinRL agent that trades as smartly as ChatGPT by using the large language model behind ChatGPT
ChatGPT Trading Agent V1
- Trade with the suggestions given by ChatGPT
ChatGPT adds technical indicators into FinRL

Introductory

Sparks of artificial general intelligence: Early experiments with GPT-4
[GPT-4] GPT-4 Technical Report
[InstructGPT] Training language models to follow instructions with human feedback NeurIPS 2022.

The Journey of Open AI GPT models. GPT models explained. Open AI's GPT-1, GPT-2, GPT-3.

[GPT-3] Language models are few-shot learners NeurIPS 2020.
[GPT-2] Language Models are Unsupervised Multitask Learners
[GPT-1] Improving Language Understanding by Generative Pre-Training
[Transformer] Attention is All you Need NeurIPS 2017.

(Financial) Big Data

[BloombergGPT] BloombergGPT: A Large Language Model for Finance
WHAT’S IN MY AI? A Comprehensive Analysis of Datasets Used to Train GPT-1, GPT-2, GPT-3, GPT-NeoX-20B, Megatron-11B, MT-NLG, and Gopher
FinRL-Meta Repo and paper FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learning. Advances in Neural Information Processing Systems, 2022.
[AI4Finance] FinNLP Democratizing Internet-scale financial data.

Interesting Demos

GPT-3 Creative Fiction Creative writing by OpenAI’s GPT-3 model, demonstrating poetry, dialogue, puns, literary parodies, and storytelling. Plus advice on effective GPT-3 prompt programming & avoiding common errors.