Question: Skip connections

Question

Question: Skip connections

ajoomye opened this issue 7 months ago · 2 comments

Hi, Thank you for the amazing repo. What does the skip connection do? If I want a simple TCN as shown in the examples with a kernel size of 2 and dilations [1, 2, 4, 8]. Do I have to set the skip connection as false? And if it is set to True, is there a separate connection between the input and output layer?

Answer 1 · 2024-02-02T01:02:50.000Z

@ajoomye hi sorry for late reply. Skip connections allow a better gradient flow. Each output layer is connected to each other. The outputs are sum-ed and pass to the last layer to calculate the prediction. I'd say, set it to True always unless you have a good reason not to.

Answer 2 · 2024-02-08T03:16:01.000Z

Dear @philipperemy , Thank you for the response.