Dilations Parameter Advice

Firstly, the documentation in this readme is really good, it's been a great help to me working out the TCN architecture.

I had a quick question on this line:

keras-tcn/README.md

Line 129 in 81706ea

    
           - `dilations`: List/Tuple. A dilation list. Example is: [1, 2, 4, 8, 16, 32, 64].

When you refer to a 'sequence' length, do you mean looking at the number of timesteps? Or the length of an 'event' in your data?

@evanharwin thank you very much! A TCN model is more or less a stack of layers containing "a trous" (dilated) convolutions. The dilations parameter controls how dilated each convolution is in the network.

If you specify d=[1,2,4,8],

you will have 4 layers because len(d)=4.
d[0]=1 is your first dilated layer with dilations=1. You can read the doc of dilated conv here: https://www.tensorflow.org/api_docs/python/tf/nn/conv1d.
d[1]=2 will point to your second dilated layer with dilations=2.
etc...

Dilated convolutions are used to keep the number of parameters "low" while increasing how far the network can see in the sequence.

The "trous" or dilated values are displayed in white on this visual, with the example d=[1,2,4,8]:

To find how much dilations you need, you can refer to Table 2 on the original paper: https://arxiv.org/pdf/1803.01271.pdf.

It's the column n.

To answer your question: ideally you want your receptive field to be bigger than the largest of your sequence. Imagine if you have sequences of 20 elements but your receptive field is only 5. That would mean that replacing the first 15 elements with 0 and appending the last 5 values would give you EXACTLY the same result because the TCN will be unable to see further than 5 values in the past. However, if your sequences are of length 20 but you choose a receptive field of 999: you would just waste a lot of resources for nothing and you could even "dilate" your network too much and lose in precision.

Wow thanks for that ♥

That makes a lot of sense. It's a little tricky transferring the understanding from image CNN's, but the exponential dilations are pretty smart.

Do you think it's worth me making a PR like this? or shall I leave it be?

- `dilations`: It controls how deep your TCN layer is. Usually, consider a list with multiple of 
- two. You can guess how many dilations you need by matching the receptive field (of the TCN) with 
- the sequences' lengths.
+ - `dilations`: It controls how deep your TCN layer is. Usually, consider a list with multiple of
+ two. You can guess how many dilations you need by matching the receptive field (of the TCN) with
+ the length of features in your sequence. For example, if your timeseries is periodic, you might 
+ want to have multiples of that period as dilations.

If you think it's good how it is, feel free to close this issue right away 🙂

Sure you can open a PR. That sounds good to me!