MOFTransformer

Do you train machine learning models for every application? This package provides universal transfer learing for metal-organic frameworks(MOFs) to construct structure-property relationships. MOFTransformer obtains state-of-the-art performance to predict accross various properties that include gas adsorption, diffusion, electronic properties regardless of gas types. Beyond its universal transfer learning capabilityies, it provides feature importance analysis from its attentions scores to capture chemical intution.

Install

Depedencies

python>=3.8

Installation using PIP

$ pip install moftransfomer

Installation from github repository

you can also download from the github repository.

$ git clone https://github.com/hspark1212/MOFTransformer.git
$ cd moftransformer
$ python setup.py install

Download the pretrained model (ckpt file)

you can download the pretrained model with 1 M hMOFs in figshare or you can download with a command line:

$ moftransformer download pretrain_model

(Optional) Download dataset for CoREMOF, QMOF

we've provide the dataset of MOFTransformer (i.e., atom-based graph embeddings and energy-grid embeddings) for CoREMOF, QMOF

$ moftransformer download coremof
$ moftransformer download qmof

Getting Started

At first, you download dataset of hMOFs (20,000 MOFs) as an example.

$ moftransformer download hmof

Fine-tune the pretrained MOFTransformer.

import moftransformer
from moftransformer.examples import example_path

# data root and downstream from example
data_root = example_path['data_root']
downstream = example_path['downstream']
log_dir = './logs/'

moftransformer.run(data_root, downstream, log_dir=log_dir, 
                   max_epochs=max_epochs, batch_size=batch_size,)

Visualize analysis of feature importance for the fine-tuned model.

%matplotlib widget
from visualize import PatchVisualizer

model_path = "examples/finetuned_bandgap.ckpt" # or 'examples/finetuned_h2_uptake.ckpt'
data_path = 'examples/visualize/dataset/'
cifname = 'MIBQAR01_FSR'

vis = PatchVisualizer.from_cifname(cifname, model_path, data_path)
vis.draw_graph() # or vis.draw_grid()

Architecture

MOFTransformeris a multi-modal Transformer pre-trained with 1 million hypothetical MOFs so that it efficiently capture both local and global feeatures of MOFs.

MOFformer takes two different representations as input
- Atom-based Graph Embedding : CGCNN w/o pooling layer -> local features
- Energy-grid Embedding : 1D flatten patches of 3D energy grid -> global features

Feature Importance Anaylsis

you can easily visualize feature importance analysis of atom-based graph embeddings and energy-grid embeddings.

%matplotlib widget
from visualize import PatchVisualizer

model_path = "examples/finetuned_bandgap.ckpt" # or 'examples/finetuned_h2_uptake.ckpt'
data_path = 'examples/visualize/dataset/'
cifname = 'MIBQAR01_FSR'

vis = PatchVisualizer.from_cifname(cifname, model_path, data_path)
vis.draw_graph()

vis = PatchVisualizer.from_cifname(cifname, model_path, data_path)
vis.draw_grid()

Universal Transfer Learning

Property	MOFTransformer	Original Paper	Number of Data	Remarks	Reference
N₂ uptake	R2: 0.78	R2: 0.71	5,286	CoRE MOF	1
O₂ uptake	R2: 0.83	R2: 0.74	5,286	CoRE MOF	1
N₂ diffusivity	R2: 0.77	R2: 0.76	5,286	CoRE MOF	1
O₂ diffusivity	R2: 0.78	R2: 0.74	5,286	CoRE MOF	1
CO₂ Henry coefficient	MAE : 0.30	MAE : 0.42	8,183	CoRE MOF	2
Solvent removal stability classification	ACC : 0.76	ACC : 0.76	2,148	Text-mining data	3
Thermal stability regression	R2 : 0.44	R2 : 0.46	3,098	Text-mining data	3

eren125/MOFTransformer