🧪 Mol-Instructions

An open, large-scale biomolecular instruction dataset for large language models.

📃 Paper • ⏬ Dataset

🆕 News

[October 2023] We've updated the train/test splits on Mol-Instructions, along with the evaluation code.
[September 2023] We've released the molecule and biotext model weights on Hugging Face, built upon LLaMa-7b-chat.
[August 2023] We've added quantitative experiments on Mol-Instructions.
[June 2023] We've open-sourced KnowLM, a knowledgeable large language model framework! It comes with pre-training and instruction fine-tuning code (supports multi-machine multi-GPU setup).
[June 2023] We release the first version (v1) of datasets and model weights.
[March 2023] We propose MolGen, a robust pre-trained molecular generative model with self-feedback.

📌 Contents

Overview
Tasks
Demo
Notices
- Usage and License
- Limitations
About
- References
- Acknowledgements

1. Overview

📊 1.1 Data Stats

Mol-Instructions comprises three cardinal components:

🔬 Molecule-oriented instructions: This component delves into the world of small molecules, emphasizing their inherent properties and behaviors. It sheds light on the fundamental challenges of diverse chemical reactions and molecular design, with 148,4K instructions across six tasks.
🧬 Protein-oriented instructions: Rooted in the biosciences, this component presents 505K instructions across five distinct categories of tasks. These tasks aim to predict the structure, function, and activity of proteins, and facilitate protein design based on textual directives.
🥼 Biomolecular text instructions: Predominantly designed to cater to NLP tasks within the fields of bioinformatics and chemoinformatics, this part encapsulates six information extraction and Q&A tasks represented through 53K instructions.

🛠️ 1.2 Data Construction

🤖️ Human-AI Collaboration Task Description Creation: In real-world applications, task instructions must be able to accommodate the varied and dynamic nature of human needs and queries. We emulate this diversity by starting with a clear, human-crafted description for each task, which is then used as an input to GPT-3.5-turbo.
📖 Information Derivation from Existing Data: Biomolecular data often requires specialist laboratory experiments and expert analysis, making authoritative and recognized biochemistry databases an ideal source of our data. With suitable processing, these resources enable us to extract the required instruction data.
📜 Template-based Conversion of Biological Data into Textual Format: To facilitate the transformation of these structured annotations into a textual format, we design a diverse array of templates. Each resulting text-based annotation serves as a guideline for protein design.
✅ Quality Control: To expedite the model's ability to generate precise biomolecules, we implement stringent quality control measures for our biomolecular data.

🤗 1.3 Data Release

We release the dataset on Hugging Face at zjunlp/Mol-Instructions.

2. Tasks

🔬 2.1 Molecule-oriented

Molecule description generation

Please give me some details about this molecule: [C][C][C][C][C][C][C][C][C][C][C][C][C][C][C][C][C][C][=Branch1][C][=O][O][C@H1][Branch2][Ring1][=Branch1][C][O][C][=Branch1][C][=O][C][C][C][C][C][C][C][C][C][C][C][C][C][C][C][C][O][P][=Branch1][C][=O][Branch1][C][O][O][C][C@@H1][Branch1][=Branch1][C][=Branch1][C][=O][O][N]
```
The molecule is a 3-sn-phosphatidyl-L-serine in which the phosphatidyl acyl groups at positions 1 and 2 are specified as stearoyl and arachidonoyl respectively. 
It is functionally related to an arachidonic acid and an octadecanoic acid.
```

Description-guided molecule design

Create a molecule with the structure as the one described: The molecule is a primary arylamine in which an amino functional group is substituted for one of the benzene hydrogens. It is a primary arylamine and a member of anilines.
```
[N][C][=C][C][=C][C][=C][Ring1][=Branch1]
```

Forward reaction prediction

With the provided reactants and reagents, propose a potential product: [O][=N+1][Branch1][C][O-1][C][=C][N][=C][Branch1][C][Cl][C][Branch1][C][I][=C][Ring1][Branch2].[Fe]
```
[N][C][=C][N][=C][Branch1][C][Cl][C][Branch1][C][I][=C][Ring1][Branch2]
```

Retrosynthesis

Please suggest potential reactants used in the synthesis of the provided product: [C][=C][C][C][N][C][=Branch1][C][=O][O][C][Branch1][C][C][Branch1][C][C][C]
```
[C][=C][C][C][N].[C][C][Branch1][C][C][Branch1][C][C][O][C][=Branch1][C][=O][O][C][=Branch1][C][=O][O][C][Branch1][C][C][Branch1][C][C][C]
```

Reagent prediction

Please provide possible reagents based on the following chemical reaction: [C][C][=C][C][=C][Branch1][C][N][C][=N][Ring1][#Branch1].[O][=C][Branch1][C][Cl][C][Cl]>>[C][C][=C][C][=C][Branch1][Branch2][N][C][=Branch1][C][=O][C][Cl][C][=N][Ring1][O]
```
[C][C][C][O][C][Ring1][Branch1].[C][C][N][Branch1][Ring1][C][C][C][C].[O]
```

Property prediction

Please provide the HOMO energy value for this molecule: [C][C][O][C][C][Branch1][C][C][C][Branch1][C][C][C]
```
-0.2482
```

🧬 2.2 Protein-oriented

Protein design

Design a protein sequence that exhibits the desired activity and specificity.

The presence of Mg(2+) is necessary for the protein to function in the desired environment.
The AMP, (6S)-NADPHX binding site should be located in a region of the protein that is accessible to the ligand.
The designed protein should have ATP binding, NADPHX epimerase activity, metal ion binding, ADP-dependent NAD(P)H-hydrate dehydratase activity to facilitate nicotinamide nucleotide metabolic process.
For general function, the protein need meet that Catalyzes the epimerization of the S- and R-forms of NAD(P)HX, a damaged form of NAD(P)H that is a result of enzymatic or heat-dependent hydration

MSNELVLSREQVRRVDQRAIEAYGVPGIVLMENAGRGAAEIIRAACPSAQRVLIACGPGNNGGDGFVIARHLANAGWMVELLLACPADRITGDAQGNHEIIRRMNLPCAVMADARDLEAANDRFATADVIVDALLGTGASGPPREPIASLIRAINEAHRRVSAQPAPSVFAVDIPSGLDCDTGEAANPTVRADHTITFVARKIGFRNPAARDLLGRVHVVDIGAPRAAIQDALTGKSG

Catalytic activity prediction

Please evaluate the following protein sequence and provide an explanation of the enzyme's catalytic activity, including the chemical reaction it facilitates: MDKVAVAGFLPEELCASLSLSPSFRGNQIFQWIGKGVDSFDAMTNLSAELRASLAEKAILRSTRVSDVLKADDGTVKLQIQTEDDLAVETVLLTDKAARKTACVSCQAGCAMGCAFCKTGTLGLARNLSAAEIVEQFLYLEKHAGALDNIVFMGMGEPLLNLDALRKAIAVLTDKRGRNLSSRRITVSTVGIVSGIYDLANNGPDVRLAVSLTTADETLRRELMPASLTNPLSDLRQAISYYIEKTGKRVTLEAVLLSGKNTSEKNADSLIAFAKGLDVHVNLIPWNPVEGLSFVTPDPEETAQFVSRLEKGGLNVTLRMHRGKSISGACGQLGKTNPYA
```
Based on the provided protein sequence, the enzyme appears to facilitate the chemical reaction: adenosine(37) in tRNA + 2 reduced [2Fe-2S]-[ferredoxin] + 2 S- adenosyl-L-methionine = 2-methyladenosine(37) in tRNA + 5'-   deoxyadenosine + L-methionine + 2 oxidized [2Fe-2S]-[ferredoxin] + S- adenosyl-L-homocysteine.
```

Protein function prediction

Analyze the following amino acid sequence, and determine the function of the resulting protein, its subcellular localization, and any biological processes it may be part of: MNGTVNASAPSKMSEVAVERLSNDKALKVIFVLGGPGSGKGTQCAKIAKHFGFTHLSVGDLLRAEINSGSKNGTMIESMINEGKIVRSEVTIKLLQRAMHESGNDKFLIDGFPRNEENRAAFENLEKIEPEFVLFFDCPMEEMERRILNRNQGRDDDKMETIRKRFKVFIESTLPVIEFYNLKGKLYKIDACKPADEVFEDVKAIFSRFRAKEDSSQQTNICTAKRFELVMCLIKRLFREIKRMWSSFFCKAL
```
The protein characterized by the amino acid sequence demonstrates ATP binding, cytidylate kinase activity, uridylate kinase activity and is implicated in the 'de novo' pyrimidine nucleobase biosynthetic process, phosphorylation, pyrimidine nucleotide biosynthetic process. Its subcellular localization is primarily within the cytoplasm, nucleus.
```

Functional description generation

Examine the given protein sequence and share a brief overview of its attributes: MKIVLASNNQGKLAELKAMLAPLGVQLLRQAELGIPEAAEPFRTFVENALAKARHASALSGLPALADDAGLCVEAFGGLPGVDTAFYAVQFGYAKGDANNVKALLAQMAGITDRRAALVSTLVAVRSAEDPEPLIACGRVAGEVALEPMGSNGFGFDPVMFIPEFGQTFAQLPVEVKNANSHRGKATRQMMALMRERWIT

A concise description of the protein with the specified amino acid sequence includes: Pyrophosphatase that catalyzes the hydrolysis of nucleoside triphosphates to their monophosphate derivatives, with a high preference for the non-canonical purine nucleotides XTP (xanthosine triphosphate), dITP (deoxyinosine triphosphate) and ITP. Seems to function as a house-cleaning enzyme that removes non-canonical purine nucleotides from the nucleotide pool, thus preventing their incorporation into DNA/RNA and avoiding chromosomal lesions.

Domain/Motif prediction

Given this protein sequence, can you identify any common protein motifs or domains that it may contain? MANTKYIFITGGVVSSLGKGIAAASIGALLESRGLSVSLIKVDPYINVDPGTMSPFQHGEVFVTEDGTETDLDLGHYERFVRFKASKKNNFTAGKVYETVIRNERKGNYLGGTVQVIPHITNEIKKRIKKGGQNKDIAIVEVGGTVGDIESQPFVEALRQMALELPNSSWAFVHLTLVPFINASGELKTKPTQHSVKELRSLGISPDVLVCRSEQELPKDEKNKIALFCSVPAKSVISMHDVDTVYSIPILLNKQKVDDTILKKLNLKIKKPNLNDWKRVVKAKLLPEKEVNVSFVGKYTELKDSYKSINEALEHAGIQNKAKVNINFVEAEQITSQNVRKVLKKSDAILVPGGFGERGIEGMILACKYARENNVPYLGICLGMQIAIIEYARNVLKLKSANSTEFDSSTKFPVIGLITEWSDISGKKEKRTKNSDLGGTMRLGGQVCKLKKKSNSYKMYKKSEIIERHRHRYEVNPNYKDKMIEQGLDVVGTSIDGKLVEMIELPSHKWFLACQFHPEFTSNPRDGHPIFNSYIKSTITK
```
Our predictive analysis of the given protein sequence reveals possible domains or motifs. These include: Glutamine amidotransferase, CTP synthase N-terminal domains.
```

🥼 2.3 Biomolecule text

Chemical entity recognition

Find and list all the instances of the chemical entities in the following content: "Both the control and caramiphen groups with double cannulas had significantly shorter latencies to seizure onset than the corresponding groups with single cannula."
```
caramiphen
```

Chemical-disease interaction extraction

You are provided with a set of clinical trial summaries. Extract the chemical-disease relations from the summaries and present your findings in the format of (Subject, Object): "Eating disorders and the associated behavioural problems and drug abuse are uncommon in pregnancy. When they do occur they are often unrecognized because of denial but when significant may pose a risk to both the mother and her fetus. This case illustrates a number of problems that may be encountered in women with eating disorders in pregnancy, including prolonged and recurrent metabolic disturbances and diuretic abuse. In particular it illustrates the derangements of thyroid function seen in pregnant women with eating disorders and reminds us that when a cause for thyrotoxicosis remains obscure, thyroxine abuse should be considered and explored."
```
(thyroxine, thyrotoxicosis) 
```

Chemical-protein interaction extraction

Your task is to extract the subject and object entities from the provided text and establish the corresponding relation triples. Please format your answers as (Subject, Relation, Object): "Opioid receptors are the membrane proteins that mediate the pain-relieving effect of opioid drugs, such as morphine and fentanyl as well as endogenous opioid peptides enkephalins and endorphins. Using cDNAs for the mu and the kappa opioid receptors, we mapped the chromosomal locations of their genes in mouse. Multilocus cross analysis located the mu receptor gene Oprm on Chr 10 and the kappa receptor gene Oprk1 on Chr 1. Both genes are near centromere, with no markers more centromeric. These data indicate that the two opioid receptors are different gene products, ruling out the possibility that they may be differential splicing products from the same gene."
```
(morphine, regulator, Opioid receptors), (fentanyl, regulator, Opioid receptors)
```

Multiple-choice question

Which of the following conditions does not show multifactorial inheritance? What of the following is the right choice? (A) Pyloric stenosis (B) Schizophrenia (C) Spina bifida (neural tube defects) (D) Marfan syndrome
```
The final answer is (D).
```

True or False question

Does feeding tube insertion and its timing improve survival?

No, neither insertion of PEG tubes nor timing of insertion affect survival.

Open question

What virus causes chronic disease?
```
Hepatitis C virus (HCV).
```

3. Demo

🤗 3.1 Model Weight Release

We release the model weights on Hugging Face at:

🔬 Molecule-oriented: zjunlp/llama-molinst-molecule-7b, zjunlp/llama2-molinst-molecule-7b
🧬 Protein-oriented: zjunlp/llama-molinst-protein-7b, zjunlp/llama2-molinst-biotext-7b
🥼 Biomolecular text: zjunlp/llama-molinst-biotext-7b

📝 3.2 Model Usage Guide

We have provided a web version demo based on Gradio. To use it, you first need to download this repository:

>> git clone https://github.com/zjunlp/Mol-Instruction
>> cd demo

Step 1, install Gradio by running：pip install gradio.

Step 2, specify the parameters in the generate.sh file.

>> CUDA_VISIBLE_DEVICES=0 python generate.py \
    --CLI False\
    --protein False\
    --load_8bit \
    --base_model $BASE_MODEL_PATH \
    --share_gradio True\
    --lora_weights $FINETUNED_MODEL_PATH \

For models fine-tuned on molecule-oriented and biomolecular text instructions, please set $FINETUNED_MODEL_PATH to 'zjunlp/llama-molinst-molecule-7b' or 'zjunlp/llama-molinst-biotext-7b'.

For the model fine-tuned on protein-oriented instructions, you need to perform additional steps as described in this folder.

Step 3, run the generate.sh file in the repository：

>> sh generate.sh

We offer two methods: the first one is command-line interaction, and the second one is web-based interaction, which provides greater flexibility.

Use the following command to enter web-based interaction:

>> python generate.py

The program will run a web server and output an address. Open the output address in a browser to use it.

Use the following command to enter command-line interaction:

>> python generate.py --CLI True

The disadvantage is the inability to dynamically change decoding parameters.

💡 3.3 Quantitative Experiments

To investigate whether Mol-Instructions can enhance LLM’s understanding of biomolecules, we conduct the following quantitative experiments. For detailed experimental settings and analysis, please refer to our paper. Please refer to the evaluation code to conduct the same experiments.

🧪 Molecular generation tasks

🔍 Molecular property prediction task & Molecule and protein understanding tasks

🧫 Bioinformatic NLP tasks

💡 3.4 FAQ

Question: What action should be taken if the model encounters <unk> and subsequently repeats the input during decoding?

Answer: Consider reducing the value of the max tokens.
Question: What should I do if the model encounters � during decoding?

Answer: If this symbol emerges in the middle of the decoded sentence, we recommend changing the input. If it shows up at the end of the sentence, you can tackle this issue by extending the output length.
Question: Why do I receive varied results despite using identical decoding parameters?

Answer: This might occur if you have enabled do_sample=True. Another factor could be the order in which tasks are executed. A useful approach would be to use a for loop to generate multiple outputs with the same decoding parameters, enabling you to note the variance in each output.
Question: What could be the reason for subpar answer quality?

Answer: Modifying the decoding parameters could help in improving the quality of the extraction or the answer.

4. Notices

🚨 4.1. Usage and License

Please note that all data and model weights of Mol-Instructions is exclusively licensed for research purposes. The accompanying dataset is licensed under CC BY 4.0, which permits solely non-commercial usage.

We emphatically urge all users to adhere to the highest ethical standards when using our dataset, including maintaining fairness, transparency, and responsibility in their research. Any usage of the dataset that may lead to harm or pose a detriment to society is strictly forbidden.

In terms of dataset maintenance, we pledge our commitment to provide necessary upkeep. This will ensure the continued relevance and usability of the dataset in light of evolving research landscapes. This commitment encompasses regular updates, error checks, and amendments in accordance with field advancements and user feedback.

⚠️ 4.2. Limitations

The current state of the model, obtained via instruction tuning, is a preliminary demonstration. Its capacity to handle real-world, production-grade tasks remains limited. Moreover, there is a vast reservoir of rich instruction data that remains to be collected and exploited.

5. About

📚 5.1 References

If you use our repository, please cite the following related paper:

@article{fang2023mol,
  title={Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Models},
  author={Fang, Yin and Liang, Xiaozhuan and Zhang, Ningyu and Liu, Kangwei and Huang, Rui and Chen, Zhuo and Fan, Xiaohui and Chen, Huajun},
  journal={arXiv preprint arXiv:2306.08018},
  year={2023}
}

🫱🏻‍🫲 5.2 Acknowledgements

We appreciate LLaMA, Huggingface Transformers Llama, Alpaca, Alpaca-LoRA, Chatbot Service and many other related works for their open-source contributions. The logo of the model is automatically generated by Wenxin Yige.

tiejundong/Mol-Instructions