Confusion about regression or classification head

[samshipengs]

This is not a bug but more of a question due to my not-so-clear understanding of TCN or the repo.

Say, I'm trying to classify a time series (non-causal), so it's Many-to-One, for example, given an audio signal, classify what type of bird created the audio.

how should I initialize the TCN class or modify the network?

What I try to do is simply take the last time step of the output, but isn't all the calculation before the last time step all wasted? i.e. the upper left triangle, if my understanding is right. Can't we down-sample the temporal dimension in the encoding phase (like most CNN networks where it gets downsampled towards the end of the layers)?