KULeuven-MICAS/zigzag

Non-integer scaling factors in dimension_relations

Closed this issue · 2 comments

b0g2 commented

Perhaps this is a bit nitpicking, but when passing non integer scaling factors in 'dimension_relations': ['ix=0.1*ox+0.1*fx'] in the workload file, there is no error (while I suppose you only accept integers here?).

Anyway, in some cases it could be that ox=2*ix+1*fx, and then if you'd invert this equation with respect to ix, you would have ix=0.5*ox-0.5*fx; but I am not sure if you'd support the special case of rational numbers. However, in the future our code analysis tool may output such relationships (but likely we will make sure to compare both relationships to decide which one has integer scaling factors). We even know of applications for which e.g., ox=2*ix+1+2*fx.

asyms commented

I agree that the parsing should be made more error-proof. I'll leave this issue open.

Could you elaborate on your last point? Which applications would require such a relationship? The case where ox=2*ix+2*fx represents a strided convolution with stride 2, and in addition the scaling factor of fx represents the "dilation rate" on the filter, as is shown in this figure: https://www.researchgate.net/publication/336002670/figure/fig1/AS:806667134455815@1569335840531/An-illustration-of-the-receptive-field-for-one-dilated-convolution-with-different.png

The constant +1 you added in the equation, what does it represent?

b0g2 commented

It is a common filter operation in image demosaicking. You have then 4 images that are "interleaved" in a Bayer pattern, and you do convolutions on each of the 4 images separately (or even mixed in some form) in order to reconstruct a fully sampled RGB image step-by-step. Of course, with reordering the pixel intensities, it can always be converted to 4 images, avoiding these operations. However, when doing raw sensor processing, people tend to avoid this due to the extra costly memory reordering operation.

Strictly speaking, in 1D, this would actually be rather like this:

for ix:
	for fx:
		f[2*ix+2*fx] += g[2*ix] * h1[fx]
		f[2*ix+2*fx+1] += g[2*ix+1] * h2[fx]

or similarly (after a coordinate transform)

for ix:
	for fx:
		f[2*ox] += g[2*ox-2*fx] * h1[fx]
		f[2*ox+1] += g[2*ox-2*fx+1] * h2[fx]

A type of stride-2 convolution in input and output. I am not sure if you can handle such indexing scheme right now. It looks similar to an atrous/dilated convolution, but the odd and even coefficients have different filters:-)
For such cases, it is useful to have separate affine index transforms for inputs and outputs. This gives also some extra freedom, for example it may be: f[2*ox+1] += g[2*ox-2*fx] * h1[fx]