DASH error when using dash::copy with 3D arrays

[Mietzsch]

When this code is run with 4 or more units, I get the following error: [ 0 ERROR ] [ 3008 ] GlobPtrBase.h :264 | GlobPtr.increment | offset goes beyond the global memory end 1. I want to copy the last plane of the z NArray to a local vector. This is actually successfull as seen by the output, but I still get this error. When I change the dash::copy to std::copy, the issue is resolved, and if I want to copy the second-to-last plane, everything works fine. Even if I change the manual calculation z.begin()+z_size*(i3+1) to z.end() this gives the DASH-error. Note that with a unitcount of 4 and a 6x6x6 NArray, the array is distributed among units 0-2. I am using dash-0.4.0.

#include <iostream>
#include <libdash.h>

#include <vector>
using namespace std;

int main(int argc, char* argv[])
{
   dash::init(&argc, &argv);
   int mm1 = 6;
   int mm2 = 6;
   int mm3 = 6;

   auto distspec2 = dash::DistributionSpec<3>( dash::BLOCKED, dash::NONE , dash::NONE);
   dash::NArray<double, 3> z(mm1,mm2,mm3, distspec2);

   if(0 == dash::myid()) {
     z(4,5,5) = 10;
     z(4,1,3) = 10;
     z(5,5,5) = 10;
     z(5,1,3) = 10;
   }

   dash::barrier();

   if(dash::myid() == 0) {
     std::vector<double> z_local(mm1*mm2);
     int z_size = mm1*mm2;
     int i3 = 5;
     dash::copy(z.begin()+z_size*(i3), z.end(), z_local.data());
     printf("z(%d,1,3)=%f\n", i3, z_local[1*mm1+3]);
     printf("z(%d,5,5)=%f\n", i3, z_local[5*mm1+5]);
   }

   dash::finalize();

   return EXIT_SUCCESS;
}

[Mietzsch]

Addition: I am using MPI version 3.3a2 and my build.sh can be found under #668.

[devreal]

Thanks for the report. I tried running your code on the latest development commit and that seems to work for me (only ran locally on my laptop but that should not have an impact on the issue). You said you used dash-0.4.0. Does that mean you're using the development branch, too?

[Mietzsch]

Yes, I am using the development branch. I will upgrade to the latest commit and then check back.

[devreal]

We have had some significant work on dash::copy here: #659

[rkowalewski]

dash::copy was always ad is still a nightmare. We had lots of fixes in the past which probably are still lying around in some PRs and always got rejected. Let's look into it face to face if you are here at university again.

[Mietzsch]

Well, at least this instance of the nightmare got fixed in last past months, it works with the most recent iteration of the development branch. Sorry for not checking beforehand and thanks to all.