A quick question about model design

[HelloWorldLTY]

Hi, I wonder why Anti DGN does not choose to compress the feature, like an encoder structure. If our input graph has feature dim like 10000, is it still reasonable to choose the output dimensions the same as the input dimensions? I think pyg uses in=out design.
https://pytorch-geometric.readthedocs.io/en/latest/generated/torch_geometric.nn.conv.AntiSymmetricConv.html#torch_geometric.nn.conv.AntiSymmetricConv

Thanks a lot!

[gravins]

Dear @HelloWorldLTY ,
Thanks for the question.

In PyG the model is implemented as a recurrent layer, thus you can still compress the input features but you need to do it with a different encoder.
A common and easy way to do it consists on creating a class that given an input it maps it into a different features space, applies A-DGN, and lastly maps the features to the output space. Which is something that you can do with a class implemented like this:

import torch
from torch.nn import Module, Linear
from torch_geometric.nn import AntiSymmetricConv
from torch_geometric.data import Data

class ADGN(Module):
    def __init__(self, 
                 input_dim, output_dim, hidden_dim, 
                 phi, num_iters, epsilon, gamma, 
                 act = 'tanh', act_kwargs = None, 
                 bias = True) -> None:
        super(ADGN, self).__init__()
        self.input_dim = input_dim
        self.output_dim = output_dim
        self.hidden_dim = hidden_dim
        self.phi = phi
        self.num_iters = num_iters
        self.epsilon = epsilon
        self.gamma = gamma
        self.act = act
        self.act_kwargs = act_kwargs
        self.bias = bias

        self.emb = Linear(self.input_dim, self.hidden_dim)
        self.conv = AntiSymmetricConv(
            self.hidden_dim,
            self.phi,
            self.num_iters,
            self.epsilon,
            self.gamma,
            self.act,
            self.act_kwargs,
            self.bias
        )
        self.readout = Linear(self.hidden_dim, self.output_dim)

    def forward(self, data: Data) -> torch.Tensor:
        x, edge_index, edge_weight = data.x, data.edge_index, data.edge_weight

        x = self.emb(x)
        x = self.conv(x, edge_index, edge_weight)
        x = self.readout(x)

        return x

[HelloWorldLTY]

Thanks a lot!

[HelloWorldLTY]

Hi, I think there is a bug for network init based on the previous code. The correct version might be:

import torch
from torch.nn import Module, Linear
from antisymmetric_conv import AntiSymmetricConv
from torch_geometric.data import Data


class ADGN(nn.Module):

    def __init__(self, 
                 input_dim, output_dim, hidden_dim, 
                 phi=None, num_iters: int = 1,
        epsilon: float = 0.1,
        gamma: float = 0.1, 
                 act = 'tanh', act_kwargs = None, 
                 bias = True) -> None:
        super(ADGN, self).__init__()
        self.input_dim = input_dim
        self.output_dim = output_dim
        self.hidden_dim = hidden_dim
        self.phi = phi
        self.num_iters = num_iters
        self.epsilon = epsilon
        self.gamma = gamma
        self.act = act
        self.act_kwargs = act_kwargs
        self.bias = bias
        self.emb = nn.Linear(self.input_dim, self.hidden_dim)
        self.conv = AntiSymmetricConv(
            self.hidden_dim,
            self.phi,
            self.num_iters,
            self.epsilon,
            self.gamma,
            self.act,
            self.act_kwargs,
            self.bias
        )
        self.readout = nn.Linear(self.hidden_dim, self.output_dim)

    def forward(self, data: Data) -> torch.Tensor:

        x, edge_index, edge_weight = data.x, data.edge_index
        x = self.emb(x)
        x = self.conv(x, edge_index)
        x = self.readout(x)

        return x

[gravins]

In the previous comment I missed the call for the super init. I updated the previous comment accordingly.
Regarding the default value for some arguments, it is not mandatory to specify that. In the version that I propose the user should specify the values at each call.
Lastly, in the forward function you wrote, it is missing the encoder. Thus, you are not able to map the input into a new hidden space, as you asked in your first comment. Be careful.