comp_p_draft - aggregation of variables from a different type and norm deriv?
Closed this issue · 58 comments
I assume that the summations of the vars in CogAlg in general maps to the old concept of "aggregation" for reduction of the resolution of the input, losing the lowest bits?
In comp_P_
You are adding match and difference for the variables of the current patterns, however after transforming the initial structure coming from the blobs:
s, [min_x, max_x, min_y, max_y, xD, abs_xD, Ly], [L, I, G, Dx, Dy, abs_Dx, abs_Dy], root_ = blob
Adding m,d to the new pattern generates variables of the style mX, dX... :
Pm, Pd, mx, dx, mL, dL, mI, dI, mD, dD, mDy, dDy, mM, dM, mMy, dMy, div_f, nvars = P_ders
Extended with two auxilliary variables, div_f and nvars (normalized vars).
Also, I see the "normalized" and a "norm" in general are supposed to be something about division or fitting within a "normal"/canonical range; also, in a broader view, that's related to the vars related to redundancy rdn and filter (ave) adjustments with multipliers, which also "normalize" - compensate for different lengths, number of vars etc.
That makes sense for "incrementallity", "projecting" the aggregated lower-resolution variables to lower level patterns, especially if same-type vars are compared.
You mention "match deviation" in the header, but currently I see some ratios (some in comments), the rest seems normal comparison operators between two variables of two patterns, but I don't see the base line to which a deviation is computed?
Also I assume a meaningful sum of deviation would count if all variables from all types are within the same range/scale, but are they supposed to be? Thus the question:
How the summation of variables from different types is justified and what it means?
Such as:
Pnm = xm + nmI + nmDx + nmDy # defines norm_mPP, no ndx: single, but nmx is summed
if Pm > Pnm: nmPP_rdn = 1; mPP_rdn = 0 # added to rdn, or diff alt, olp, div rdn?
else: mPP_rdn = 1; nmPP_rdn = 0
Pnd = xdd + ndI + ndDx + ndDy # normalized d defines norm_dPP or ndPP
They resemble a sum of some variation of the basic 4-tuples of patterns, but this is a scalar.
If xm, xdd were also "normalized" and say all were within the same range of (min,max) as the other in the expression, then summing them and comparing them to a "norm sum" would make sense (say 1 + 1 + 1 +1 is max, or some range+range...).
However the first is xmatch (of coordinates?...), the other mDx etc.
Also, I assume that the spawning and collection of all those variables is in order to compare them in another step, to allow each of them to be compared and used for another partitioning etc., until the magnitudes of the variables or the lengths of the containers run out of "valuable" magnitude.
If it keeps going like that and doubling the number, it is asking for at least some partial code generation anyway.
Or/and/eventually starting to compare some "normalized" matches of the different types of variables between each-other and eliminating or not comparing some with lower match in the following levels which fan-out more variables. That however would require additional "maps" and checks per variable.
I assume that the summations of the vars in CogAlg in general maps to the
old concept of "aggregation" for reduction of the resolution of the input,
losing the lowest bits?It's a reduction of positional resolution, but not per bit. Bit-wise
resolution will be controlled by bit filters, which we don't use yet.
Is it possible to give an example related to current code?
norm here refers to normalizing for length L only
i.e. var/L? (L, Ly, _L, _Ly) ...
You mention "match deviation" in the header, but currently I see some
ratios (some in comments), the rest seems normal comparison operators
between two variables of two patterns, but I don't see the base line to
which a deviation is computed?
The base is ave, line 61 in frame_blobs:
g__ = np.abs(dy__) + np.abs(dx__) - ave # deviation of gradient,
initially approximated as |dy| + |dx|
OK, I've seen this usage, but expected another concept.
How the summation of variables from different types is justified and what
it means?Such as:
Pnm = xm + nmI + nmDx + nmDy # defines norm_mPP, no ndx: single, but nmx is summed
if Pm > Pnm: nmPP_rdn = 1; mPP_rdn = 0 # added to rdn, or diff alt, olp, div rdn?
else: mPP_rdn = 1; nmPP_rdn = 0Pnd = xdd + ndI + ndDx + ndDy # normalized d defines norm_dPP or ndPP
They resemble a sum of some variation of the basic 4-tuples of patterns,
but this is a scalar.
(1)
All variables are derived from pixels with conserved >resolution, so their predictive value corresponds to magnitude,
to the same extent as in pixels
(2)
Match is a reduction of represented magnitude, and difference >is a complementary.
So, their value is also proportional to magnitude, regardless of type.
(3)
If xm, xdd were also "normalized" and say all were within the same range
of (min,max) as the other in the expression, then summing them and
comparing them to a "norm sum" would make sense (say 1 + 1 + 1 +1 is max,
or some range+range...).However the first is xmatch (of coordinates?...),
the other mDx etc.
Resolution of coordinates is supposed to be adjusted so that >their average magnitude and predictive value approximates that >of inputs
OK, I think these are good insights overall for CogAlg, clarified like that. Is that related to the section:
"As distinct from autoencoders (current mainstay in unsupervised learning), there is no need for decoding: comparison is done on each level, whose output is also fed back to filter lower levels."
i.e. that the process of unfolding of the higher level patterns eventually produces the pixel-representations.
Since the aggregated variables, big letters, cover wider span etc., when converted to smaller spans (divided) they can be "projected" as shorter span variables or eventually as a single-pixel one. The bigger-span variables correspond to a bigger predictive value both as magnitude and as they are produced by comparing a wider range, they have higher certainty than a single, current pixel.
Also, does "predictive value" correspond to the literal value - a specific magnitude (1.46, 2.955, 124.56...) of the variables of the respective lower level?
And/or as their part/fraction of the whole expanded/unfolded expression which eventually computes the magnitude of the respective lower level/higher level/feedback variable?
Initial xm is the extent of overlap (full x match), but I will probably
change that to inverse deviation normalized for L: ave_dx - overlap_L /
offset_L, or something like that.
Thus (just marking it):
deviation = var - ave
"inverse deviation" = ave - var
"normalized for " is the ratio
norm = var/normalized_to
extent of overlap (full x match) etc.
That code?
xm = (x0 + L-1) - _x0 # x olp, ave - xd -> vxP: low partial
if x0 > _x0: xm -= x0 - _x0 # vx only for distant-P comp? no if mx > ave, only PP termination by comp_P? distance, or relative: olp_L / min_L (dderived)?
min_L is min(L, _L), but what are overlap_L and offset_L? (since they are lenghts - 1D) and the above formula is x olp?
Is this correct:
Coordinate(overlap, offset) maps to Input(match, miss)?
At least related to, because overlap is one length, while offset could be one both sides.
If following the logic of match (the smaller comparand), offset is supposed to be the smaller one between ((x0,_x0), (xn,_xn))? Overlap is one value.
If it keeps going like that and doubling the number, it is asking for at
least some partial code generation anyway.The only code generation I can justify in principle is macro-recursion,
which will increment algorithm of higher levels,
and corresponding feedback, which will prune | extend algorithm of lower
levels.
Micro-recursion within a level is only reusing the same code.
In this post I meant mostly tools for the developers. For the other thread also that would be a start.
Here, like taking this:
Pm, Pd, mx, dx, mL, dL, mI, dI, mD, dD, mDy, dDy, mM, dM, mMy, dMy, div_f,
And generating:
mPm, dPm, mPd, dPd, mmx, dmx, mdx, ddx, ... = ...
also summing etc.
Also assisted checking whether all variables are compared etc.
It's simple, while may save some mistakes and tediousness.
Bit filters will adjust most and least significant bits per input
parameter.
All summed params have positional resolution of their structure (P, blob,
etc.), which lower than that of constituent pixels
What do you mean technically? A method for adjusting the effective and, more importantly, the interpreted range of the magnitudes of the variables?
What I understand is:
A) Bit-wise shifts (if integers): Shift Left multiplies by 2, Shift Right divides by 2. It would require additional range-adjusting parameter.
B) Floats can't be shifted, they can be divided and multiplied as for extending the range of E+... of the represented magnitudes by a range-adjusting parameter.
The highest resolution is of the raw input, so the maximum is having all bits legal. Reducing the lower bits is by Shift left/MUL
In special interface levels between the substantial ones - "Type converting levels"?
I understand that it is meaningful theoretically, that the ranges should be adjustable for the generality of the definitions and extension of the levels, and not to be limited by an initial format.
I guess in practice it may be needed when the 32-bit runs out for the integers. Then it has to adjust with Shift/Mul or/and to jump to 64-bit. It could also just start with 64-bit eventually, as with a Double.
The magnitudes of the derivatives will grow or shrink after many operations, but they lose precision after the span is expanded anyway. There are "long long long" types like __int128 in C++, but AFAIK the range and precision of double or 64-bit int is still considered big enough for scientific calculations, the scientific GPUs use 64-bit doubles.
OK. @khanh93vn @boris-kz
BTW, may I join a live discussion of yours some time? I don't mind being just a reader and listener at first, in order not to interfere your flow if I'm not prepared to join.
An error in the order in intra_comp_debug, form_P:
CogAlg/frame_2D_alg/intra_comp_debug.py
Line 81 in 888b7f7
i, ncomp, dy, dx, g = derts_[1][0] # first derts
x0, L, I, Dy, Dx, G = x_start, 1, i, g, dy // ncomp, dx // ncomp # P params
==>
x0, L, I, Dy, Dx, G = x_start, 1, i, dy // ncomp, dx // ncomp, g # P params
OK
Regarding the question about ncomp - that there are 4 default comparisons per dx,dy in comp_pixel, not two as in the past, in order to avoid unilateral artifacts. @khanh93vn, @boris-kz
May you give me an example with specific content of the pixels and what is expected as output values after comparison given that input?
Because when I do something like that:
a = np.ones(shape=(5, 5), dtype=int)
>>> a
array([[1, 1, 1, 1, 1],
[1, 1, 1, 1, 1],
[1, 1, 1, 1, 1],
[1, 1, 1, 1, 1],
[1, 1, 1, 1, 1]])
>>> c = a[2:, 1:-1] - a[:-2, 1:-1]
The output seems to me as if doing unilateral subtraction, current - previous from the raw input, as far as I expect what it should do:
>>> c
array([[0, 0, 0],
[0, 0, 0],
[0, 0, 0]])
or
c = a[1:-1, 2:] - a[1:-1, :-2]
>>> c
array([[0, 0, 0],
[0, 0, 0],
[0, 0, 0]])
I suspect that maybe the expected or desirable behavior were that the calculations scroll down or something, to compute the difference in place for the line, they continue with the next line and use the updated previous one as current etc., thus to have double subtraction?
However the results I see above suggest me that one comparison is done, which is not taking into account the result of the previous one stored in place - if that was the goal at all...
If it was scrolling and using values in-place, I assume it would be:
1-1 = 0 and storing it on that line, in place.
Then the following line/row would be: 1-0 = 1
Then the following again: 1-1 = 0
Etc. Which is not "pretty" so may be it shouldn't do.
If the direction is taken into account and for the next lines it's +1, then 1-1+1 = 1 for all.
EDIT: I got it in the next comment. It's a sum of the differences, thus -(1-1) + +(1-1) = (-0) + (+0) = 0 :)
What is the expected output? I'm still misunderstanding something...
EDIT: I think I found out.
Well, I see I was using a bad example with equal values (dx,dy should be 0)
Another example makes more sense so I guess it's "correct".
a[1] = [2,2,2,2,2]
>>> a[2] = [4,4,4,4,4]
>>> a
array([[1, 1, 1, 1, 1],
[2, 2, 2, 2, 2],
[4, 4, 4, 4, 4],
[1, 1, 1, 1, 1],
[1, 1, 1, 1, 1]])
>>> dx = a[1:-1, 2:] - a[1:-1, :-2]
>>> dx
array([[0, 0, 0],
[0, 0, 0],
[0, 0, 0]])
>>> dy = a[2:, 1:-1] - a[:-2, 1:-1]
>>> dy
array([[ 3, 3, 3],
[-1, -1, -1],
[-3, -3, -3]])
The purpose, I guess, is to build a little "model" at every pixel to predict (project) values of nearby pixels.
Consider p as the function of x and y: p = F(x, y). It's graph is 3D, 3 axes: x, y, p, this apparently isn't linear. Now, consider a region R of the graph. If R is small enough, it could approximately be considered linear (being a plane) inside R. Given that, p could be computed using a linear model:
p = f(x, y) = p0 + a * x + b * y with x, y ∈ R
This is what Boris called "fine grained search", well, at least that's what I think it is.
Now what we have are pixel values, with comparison, we could produce 2D partial derivatives, so projection of nearby pixel values:
p = f(x, y) = _p + y * dy + x * dy
with: p is the projected/predicted pixel
_p is the current pixel
x, y: difference in coordinate of p vs _p. Can be considered coordinate of p, with (0, 0) being at
where _p is
dx, dy: results from comparison
The notation here is a little different from normal notation, where derivatives are dp/dx, dp/dy. But that's where problem is: dx, dy we computed are not rates of change, so the above model is wrong. For it to be correct, dy, dx must be divided by ncomp, which is always 4 in the original comparison (this is a coincident, distances between compared pixels are always 1 there, so comparison with increasing range should be reconsidered too). We don't do that immediately because dividing with integer will cause increasing error over time, so ncomp is saved for future operations.
You could try this, using parameters generated from frame_blobs to try to predict further pixels (for example, the nearby diagonal pixels that haven't been compared yet) to see how well it performs. The further projected pixel is, the less accuracy it could produce.
Thanks for the clarificaton! I already had an idea about that (from a Khan explanations earlier) and before that (derivatives are about that in general), but that "4" had to be clarified.
p = ((_py + dy / 2) + (_px + dx / 2)) / 2
OK, and there the "4 goes", /2 (for y) and /2 for x.
For recovering (if direct) at longer distances it has to be adjusted by the number of pixels and the accumulated ncomps. When there is a discontinuity, though, it switches to next-level/higher span with a more complex expression/another representation.
Consider p as the function of x and y: p = F(x, y). It's graph is 3D, 3 axes: x, y, p, this apparently isn't linear. Now, consider a region R of the graph. If R is small enough, it could approximately be considered linear (being a plane) inside R. Given that, p could be computed using a linear model:
That reminds me of a discussion two years ago, about similarities between CogAlg to Clustering and the so called "Spectral Clustering" and pixel-grid-graph representations.
One good point there was about principles in clustering:
- similarity
- proximity
- good continuation
CogAlg was supposed to "define them at initial / maximal resolution, AKA accuracy, which matters".
In the hierarchical representation, there are "nodes" of different generality (depth), the tree of patterns that grow.
K: You could try this, using parameters generated from frame_blobs to try to predict further pixels (for example, the nearby diagonal pixels that haven't been compared yet) to see how well it performs. The further projected pixel is, the less accuracy it could produce.
B: Because it is more tentative, its comparison (verification) is conditional
on its magnitude.
These secondary comparisons of incremental distance and derivation create
deeper blob layers.
And the expression G, Ga ... *cost etc. are to adjust for the "distance", in that context?
However before a real run through a hierarchy with data, the cost of discontinuity/levels/"syntax" seem not really justified to me. I see multiplies by a number of derivatives, division by spans, but it is to be checked...
In intra_blob:
for ablob in blob.sub_blob_: # ablob is defined by ga sign
Ga_rdn = 0
G = ablob.Derts[-2][-1] # I, Dx, Dy, G; Derts: current + higher-layers params, no lower layers yet
Ga = ablob.Derts[-1][-1] # I converted per layer, not redundant to higher layers I
Is that mean that Derts is supposed to be loaded with altering type derts (last one -1 has gradient Ga, while the previous -2 is G)? Thus [-2] is the "higher layer"?
Is that ave_blob in: if Ga > ave_blob: the mentioned ave_n_sub_blob (a length) from the FB discussion, or it's a regular magnitude? (Because G and Ga are not counts, but accumulated magnitudes from the derivatives?)
understand it: for _derts, derts in zip(derts[_start:end], derts[start, end]): i = derts[-1][-rng][0] ncomp, dy, dx = derts[-1] _i = _derts[-rng][0] _ncomp, _dy, _dx = _derts[-1] and then the assignment: derts[-1] = ncomp, dy, dx # assign back into dert _derts[-1] = _ncomp, _dy, _dx # assign back into _dert
That's a mistake, should be i = dert[-rng][0].
@khanh93vn Thanks, the now complete source suggests also.
I noticed you've marked that possible usage of a look-up table that we mentioned earlier:
hyp = hypot(xd, yd)
y_coef = yd / hyp # to decompose d into dy. Whole computation could be replaced with a look-up table instead?
x_coef = xd / hyp # to decompose d into dx. Whole computation could be replaced with a look-up table instead?
Higher filters only apply to deeper layers of each pattern.
Higher filters - ranges with an upper bound as well or lists of ranges?
There are no upper bounds, only incrementally higher filters: ave *
incremented rdn.
So, bigger and bigger and bigger... The ultimate patterns will have filters of 99999999999999999999999999999E+9999, matching the Universe. :)
What makes sense to me is that it's assumed that if the comparison is at certain level, that implies that there is a minimum cumulative match between derivatives, so the current one should be bigger than that.
Since each level works within given limited coordinated borders (by the nature of the lower level limited input magnitudes and coordinate ranges), the following possible upper bounds could be also eventually estimated, based on the "magic number" and costs calculations, number of variables, spans of pixels with accumulated magnitudes etc.
These filters don't define higher-layer root patterns, only sub_patterns
nested within them.
Here I referred to "higher filters" as to expected provisional "higher-order filters" which could have an upper bound and also be expressed as lists of ranges, instead of just one filter-value and the default bigger-than comparison. :)
And in some point initiating a new level? After evaluating all collected
patterns, some buffer is filled-up?
Well, the "buffer" is either frame, or single top-layer pattern for
"frameless" input.
They are forwarded to | initiate higher level after intra_blob is finished
exploring nested sub_patterns
My topic was for the moments/criteria for the conceptual generation of a new level.
I realize that there is one process that either includes the input into current pattern or creates a new one - it is similar to parsing code, the parser either begins recognizing and building some more complex syntax structure, say a loop, a function, an expression, recognized syntactic pattern, or it is including the required elements into the recently opened structure that reflects the loop etc.
However there is also a "higher" "supervisor process" that generates the code (you, us) and at certain moments it finishes the search for patterns the way it does for the current level (or stage), packs it and sends it to something else.
The initial limits are the bounds of the input space, the coordinates. The following, traversing all patterns given the defined laws for defining a pattern and enumerating them within the whole previous coordinate space etc.
I see that here you don't refer to these current stages as levels, but "levels of encoding Le", it would be a
complete level after the yet non-existing inter-blobs which encompass the whole frame.
Also, in a future setting where there is time or a series of frames, I think naturally there is a recent "buffer" that limits the depth of the comparison.
One path is possibly comparing two-adjacent, producing a higher level (or stage) pattern, then comparing two such patterns, then comparing two of the higher etc....
In time there is also a flow which is like scanning patterns in a frame, just the coordinates will be in "t", thus an iteration like current would be like:
for t, frame in enumerate(frames_, start=window):
frame =frames_[-t] #searching within a window of frames back
Then "same sign g" would be searched over sequences of frames, which may produce "Ps", "Segs", "Blobs" etc.
An initial meaningful "buffer" would be one that is big enough to capture basic gestures and movement, like a reaching and grasping movement, closing or opening a door, turning a head, a smile from a neutral position or another expression, clapping hands, one, two or three steps of a walk (capturing repetition) etc. Sure it has to adjust the length and could start with two frames, then extending it.
Cross-compare
Indeed, I remember you used to use the expression "cross compare".
Is that meaning all-to-all comparison of all patterns within a range of coordinates, producing derivatives for all? (Which technically is a buffer of patterns or inputs at the low/lower level, producing new derts__, maps of derts__ etc.)
Currently comparison is of adjacent (or rng-distant-coordinates-corresponding) patterns - is the latter corresponding to the "cross comparison"?
...
@khanh93vn Thanks, the now complete source suggests also.
I noticed you've marked that possible usage of a look-up table that we mentioned earlier:
hyp = hypot(xd, yd) y_coef = yd / hyp # to decompose d into dy. Whole computation could be replaced with a look-up table instead? x_coef = xd / hyp # to decompose d into dx. Whole computation could be replaced with a look-up table instead?
Do you mean the fast-clean macros you mentioned before? I only though of using pre-generated value, but macro should make better choice.
I'm not used to using macros yet, just happened to know it is used in C. Is it possible in Python?
In intra_comp's form_blob: https://github.com/khanh93vn/CogAlg/blob/master/frame_2D_alg/intra_comp.py
For positive blobs, why only Ly and L are summed? (While other params are also summed for negatives as well) Something like "dimensions in pixel-space of same-sign positive sub_blobs"?
(Possibly some indentation issue, in github's view the [par1+...] part is at the level of if ..., not enclosed)
Also shouldn't the [par1 ... part be params[2:-1] (not [2:]):
root_blob.sub_Derts[2:-1] = [par1 + par2 for par1, par2 in zip(params[2:-1], root_blob.sub_Derts[2:-1])]
(Since [-1] is the blob etc.)
if s: # for positive sub_blobs only
root_blob.sub_Derts[0] += Ly
root_blob.sub_Derts[1] += L
root_blob.sub_Derts[2:] = [par1 + par2 for par1, par2 in zip(params[2:], root_blob.sub_Derts[2:])]
root_blob.sub_Derts[-1] \
.append(nt_blob( Derts=[(0, s, params, [])], # typ=0, sign: positive for sub_blob_ > 0?
# last term is sub_blob_ of nesting depth = Derts[index]
(...)
recursive intra_blob' eval_layer' branch() calls add new dert to derts, Dert to Derts, layer of sub_blobs to blob,
where Dert params are summed params of all positive blobs per structure, with optional layer_:
For positive blobs, why only Ly and L are summed
not sure but I think it's because otherwise they would be identical to Ly and L in super-blob params. Other params are freshly generated so they'll be summed regardless.
Also shouldn't the [par1 ... part be params[2:-1] (not [2:])
Ah, you're right. I'll update it next time.
I meant also only the pre-generated only, like traversing a range of xd, yd and storing hypot ... And then reading hyp from the table, for optimization. (I didnt think about including / hyp also, maybe there might be too much values if we want precision (I dont know what precision is needed). You're right that it could go into a macro for shortening the code but AFAIK Boris wouldnt like it. :) It seems there is a library called MacroPy https://pypi.org/project/MacroPy/ My idea was for custom tool, I dont know if the available macros would fit my ideas for general code synthesis (and it has its compatibility requirements), mine which would take into account CogAlg concepts and code patterns (try to recognize and generalize them) and may include GUI and visualisation without complete compilation. EDIT (fix): As of the code synthesis - e.g. for routines like the ones you have debugged lately, if the limits and the traversed containers are defined, an already debugged automated traverse-select-... synthesizable iterator.
I see. Thanks for those infos!
In def intra_comp, one code style proposal:
I don't like how the index "i" and y are declared too far from their first usage, while "i" is used only in LOOP2. When possible, the vars are best declared close to their usage, especially counters.
Instead of:
y0, yn, x0, xn = blob.box
y = y0 # current y, from seg y0 -> yn - 1
i = 0 # segment index
blob.seg_.sort(key=lambda seg: seg[0]) # sort by y0 coordinate
buff___ = deque(maxlen=rng)
seg_ = [] # buffer of segments containing line y
sseg_ = deque() # buffer of sub-segments
I'd do:
blob.seg_.sort(key=lambda seg: seg[0]) # sort by y0 coordinate
buff___ = deque(maxlen=rng)
seg_ = [] # buffer of segments containing line y
sseg_ = deque() # buffer of sub-segments
y0, yn, x0, xn = blob.box
y = y0 # current y, from seg y0 -> yn - 1
i = 0 # segment index
while y < yn and i < len(blob.seg_):
...
It also emphasizes the sort which is an important suggestion.
Or:
buff___ = deque(maxlen=rng)
seg_ = [] # buffer of segments containing line y
sseg_ = deque() # buffer of sub-segments
blob.seg_.sort(key=lambda seg: seg[0]) # sort by y0 coordinate
y0, yn, x0, xn = blob.box
y = y0 # current y, from seg y0 -> yn - 1
i = 0 # segment index
while y < yn and i < len(blob.seg_):
...
I see. Sorry for the mess, and thank you!
NP. It was not a mess, just that order is a bit more logical.
Khan, could you add comments with the structures of the derts (all kinds) in comp_range, for all functions? Are they completed?
The indices look confusing to me, suggesting different structures (or I'm missing something...)
For lateral_comp and comp_slice:
for x, derts in enumerate(derts_, start=x0):
i = derts[-rng+1][0] # +1 for future derts.append(new_dert)
ncomp, dy, dx = derts[-1][-4:-1]
I think that suggests derts is a list of [i, ncomp, dy, dx]
for _derts, derts in zip(_derts_[_start:_end], derts_[start:end]):
i = derts[comparand_index][0]
ncomp, dy, dx = derts[-1]
The first line implies that the 0-th element is i, like [i,ncomp,dy,dx], while the second line suggests that derts consists of a list of 3-tuples, now reading the last one: (ncomp, dy, dx), and i is somehow separate.
In the end as well:
# return:
derts[-1] = ncomp, dy, dx
_derts[-1] = _ncomp, _dy, _dx
(Supposing the containers are accessed by reference and changed in place.)
Is it something about the alternating derts format?
seg_ = # seg_s of lower Derts are packed in their sub_blobs
[ seg_params,
Py_ = # vertical buffer of Ps per segment
[ P_params,
derts = [ dert = p|a, ncomp, dx, dy, g]]], # one dert per current and higher layers
# alternating p|a type of dert in derts:
# derts [even][0] = p if top dert, else none or ncomp
# derts [odd] [0] = angle
Thanks. Whatever is the case though, IMO such format transformations and different formats in the same container and similar identifiers are fragile and they should be reminded thoroughly with expected structures' content near the usage (top comment is too far away).
The first line implies that the 0-th element is i, like [i,ncomp,dy,dx], while the second line suggests that derts consists of a list of 3-tuples, now reading the last one: (ncomp, dy, dx), and i is somehow separate.
My apologies for the lack of comments.
Boris has stated somewhere that dert structure is (i, ncomp, dy, dx, g) for one, (a, ncomp, dy, dx, g) for angle branches, otherwise it's just (ncomp, dy, dx, g), because params used as comparands already existed elsewhere. Given there's no g at that point, in general case we'll have ncomp, dy, dx as 3 last params, i will be branch-specific
i = derts[comparand_index][0]
ncomp, dy, dx = derts[-1]
that is for comp_range only, should be:
derts[i_dert][i_param]
ncomp, dy, dx = derts[-1][-3:]
i_params is 0 for comp_range and comp_angle, -1 for comp_deriv (see above structures).
Not sure about i_dert though. i_dert is -rng in comp_range(), as dert that contain i is -rng away from the list end, but Boris recently updated it into -rng * 2. Anyway, there'll be updates for that.
I just posted edited comp_range in my repo
Great. But new_dert__ in lateral_comp() is intentionally removed. In fact, I don't think there's functional bugs anywhere in comp_range.py. Beside variable names and comments, could you tell me if there's anything else inappropriate?
ncomp, dy, dx = derts[-1][-3:]
i_params is 0 for comp_range and comp_angle, -1 for comp_deriv (see above structures).
OK, I've missed the details of i_indices coming from intra_comp. Sometimes g is the input at different position.
If dert has 4-elements (no matter a or i is the first),
This:
_, ncomp, dy, dx = derts[-1] #skipped first element, read next 3
is the same as this:
ncomp, dy, dx = derts[-1][-3:] #read the last 3 elements
But right -3 is more flexible if the tuple could be 3,4,5 ... then such skipping won't work.
This:
_, ncomp, dy, dx = derts[-1] #skipped first element, read next 3
is the same as this:
ncomp, dy, dx = derts[-1][-3:] #read the last 3 elements
I see, so that technique could be used for skipping as well..
...but you did have it your last push, and it wasn't initialised.
Oh ok, seems like my mess. That was embarrassing. Sorry