create new class

[linhanxiao]

i want to inherit from the class FConvModell,
the code is
captionmodel,parent=torch.class('captionmodel','FConvModel')
what else should i configure?

[jgehring]

Sorry for the late reply -- it depends on the exact changes you want to make to the model, or how you want to extend it. FConvModel inherits from AvgpoolModel and I'd recommend looking at both models to see what functions are used. For example, if you want to change the layout of the encoder, you might want to override makeEncoderStack; if you want to provide alternative input alongside the source sentence you would also override prepareSource (defined in avgpool_model.lua).

[linhanxiao]

in the dictionary of model in fairseq，there is a lua script named init,should i configure new class in the script,what else should i configure?

[jgehring]

Ah, you're asking whether you need to change anything in order to be able to use your new model? Sorry for the misunderstanding. Adding your model to models/init.lua should be sufficient.

[jgehring]

Also, and the file name format has to be <name>_model.lua. See make_model_fn in train.lua for the code that creates a new model based on the -model command-line argument.

[linhanxiao]

thank you very much

[linhanxiao]

@jgehring @michaelauli I want to create new class inherited from FConvModel,I want to override the makeEncoder method,

MineModel.makeEncoder = argcheck{
{name='self', type='MineModel'},
{name='config', type='table'},
call = function(self, config)
local sourceIn = nn.Identity()()
local tokens, positions = sourceIn:split(2)
local embed = self:makeEmbedding(config, config.srcdict,
config.dropout_src)({tokens, positions})

     local cnn
     if #config.nhids > 0 then
           cnn = nn.Sequential()
           cnn:add(nn.Bottle(
                self:makeLinear(config.nembed, config.nhids[1],
                config.dropout_src)
           ))
          cnn:add(nn.Contiguous())
          cnn:add(self:makeEncoderStack(config, config.nhids))
          cnn:add(nn.Bottle(self:makeLinear(config.nhids[#config.nhids],
          config.nembed)))
     else
           cnn = nn.Identity()
     end


     -- The encoder stack will receive gradients *twice* for each attention
     -- pass: dot product and weighted sum.
    local nattn = #attentionLayers(#config.nlmhids, config.attnlayers)
    cnn = nn.GradMultiply(cnn, 1 / (2 * nattn))

------------------------------------- here is my added code
local r=embed.output:size(2)
i will use r variable to create four new layer ,but the embed doesn't have the output field,
so what can i do to implement the code that has the similar function

  local outputA = cnn(embed)
  -- Source feeding for weighted sum in attention: add embeddings to CNN
  -- output
  local outputC = nn.CAddTableMulConstant(
  math.sqrt(0.5))({outputA, embed})
 return nn.gModule({sourceIn}, {outputA, outputC})

end
}

[linhanxiao]

@jgehring @michaelauli

[jgehring]

Do you need the embedding size? That's available in config.nembed.

[linhanxiao]

when network input is single sample,i want to know embed.output:size(1),when network input is batch sample,i want to know embed.output:size(2),i want to use this number to dynamically set the input dimension of nn.linear layer

[linhanxiao]

@jgehring

[jgehring]

@linhanxiao you can't do that while wiring up the nngraph structure with nn.Module objects; the exact input and output shapes are only available during forward and backward passes. I suggest reshaping single examples so that they have a batch dimension of 1, which is also done in generate-lines.lua#L119.