Incompatible changes when updating from nd4j-api version 1.0.0-beta3 to 1.0.0-beta4/M2.1
wolfig opened this issue · 2 comments
Issue Description
Currently, I am trying to upgrade code of the project matrix-profile JAVA to the latest versions of nd4j (matrix-profile makes heavy use of nd4j). When running UnitTests using nd4j 1.0.0-beta3, the following code is working fine:
public INDArray getDistanceProfile(INDArray tsA, INDArray tsB, int index, int w) { INDArrayIndex indices = NDArrayIndex.interval(index, index + w); INDArray query = tsA.get(indices);
When using nd4j 1.0.0-beta4 (and newer) I get an error:
java.lang.IllegalStateException: Indices are out of range: Cannot get interval index Interval(b=1,e=5,s=1) on array with size(0)=1. Array shape: [1, 5], indices: [Interval(b=1,e=5,s=1), all()] at org.nd4j.linalg.api.ndarray.BaseNDArray.get(BaseNDArray.java:4239)
Entering method 'BaseNDArray:get(...)', the variable indexes looks like this:
After having passes through lines 4173-4184, the variable indexeslooks like this:
As far as I understand, the error occurs, because indexes[0] is of type IntervalIndex and has the wrong dimensions. indexes[1] is of type NDArrayIndexAll. As far as I see, the error could be avoided if the order of indexes[0] and indexes[1] were reversed - or am I wrong there?
I do not think that your code has errors, but rather that the new (beta4) library needs to be used differently. My issue would be fixed if you could help me to provide it with the right input.
Since the release of beta 3 in 2018 quite a lot has changed. Some of those changes include better sanity checks of inputs and different handling of singular dimensions.
I think in your case the singular dimension changes are the reason for the issues. Specify the indices for all dimensions (i.e. include the leading singular dimension) and it should work as expected again.
@wolfig I'm going to close this and ask that you migrate this over to the forums: https://community.konduit.ai/ - apologies after 1.0 is fully released there will be a lot less changes.
One major indexing change we did was migrate scalars and vectors from requiring minimum 2d to being more flexible.
This might cause some issues with the get(...) calls.
That's already fairly close. A lot of clean up and pruning has been done to get to that point. If you have anything beyond @treo 's suggestion please let us know and we'll help you work around the issue. Thanks!