Library and Plots based on the paper by B. Bohn et al.
NOTE: it is recommended to use a seperate virtual environment
Make sure that you are inside the root directory of this repo
Step 1: install imagemagick
> apt install imagemagickStep 2: install python dependencies
> pip install -r requirements.txtLets perform Deep Kernel ridge regression on synthetic data similar to the paper.
from compositeKRR import DeepKernelRegression as dkr
from utils import createSyntheticData, train_loop
import gpytorch as gpy
import torch.nn as nn
import torch
# specify the data.
num_data_points = 10
_,r,data_x,data_y, data_y_h2 = createSyntheticData(num_data_points)
# create deep kernel model with 2 layers
degree = 2
K_1 = gpy.kernels.PolynomialKernel(degree) # inner kernel
K_2 = gpy.kernels.MaternKernel() # outer kernel
kernels = [K_1, K_2]
ranges = [2, 1] # output dim of each kernel layer.
model = dkr(ranges, data_x, kernels, device="cpu")
# training our model
num_epochs = 5000
learning_rate=0.0001
loss = nn.MSELoss()
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate, weight_decay=1e-5)
train_loop(data_x, data_y, model, loss, optimizer, num_epochs)after training it for 10K epochs, the inner layer representation looks like this ( which is awefully close to our original function) :-
we can also look at the computation graph of our model using torchviz:
.
umm... Play with it! add new kernels, increase the depth, using gpus to speed things up... sky is the limit! ;D
If you find some bugs or create a cool new feature then please feel free to file an issue or send a PR!.
- Now we can use more then 2 layers. 🚀
- Added the functionality of loading and saving model in e2eKRR 🤓
- Restructured Codebase and removed dead code ⚰️