Get to the bottom of pesky MueLu output

[bgranzow]

Using the MueLu method: CreateTpetraPreconditioner
with the optional argument inCoords, and specifying the appropriate number of PDE
equations in the input parameterlist inParamList with the parameter number of equations yields the following warning:

******* WARNING *******
Setting matrix block size to 2 (value of the parameter in the list) instead of 1 (provided matrix).
You may want to check "number of equations" (or "PDE equations" for factory style list) parameter.

Omitting the number of equations throws an assertion, presumably because the coordinate vector is assumed to have a 1-1 mapping with the DOF map.

Throw test that evaluated to true: (static_cast<size_t> (X_j_av.size ()) < lclNumRows)

#0  __cxa_throw () at ../../../../gcc-4.9.2/libstdc++-v3/libsupc++/eh_throw.cc:62
#1  0x0000000001c682ba in Tpetra::MultiVector<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>, false>::MultiVector(Teuchos::RCP<Tpetra::Map<int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> > const> const&, Teuchos::ArrayView<Teuchos::ArrayView<double const> const> const&, unsigned long) ()
#2  0x000000000120f025 in Xpetra::MultiVectorFactory<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >::Build(Teuchos::RCP<Xpetra::Map<int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> > const> const&, Teuchos::ArrayView<Teuchos::ArrayView<double const> const> const&, unsigned long) ()
#3  0x000000000122fe63 in MueLu::Hierarchy<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >::ReplaceCoordinateMap(MueLu::Level&) ()
#4  0x0000000001231821 in MueLu::Hierarchy<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >::Setup(int, Teuchos::RCP<MueLu::FactoryManagerBase const>, Teuchos::RCP<MueLu::FactoryManagerBase const>, Teuchos::RCP<MueLu::FactoryManagerBase const>) ()
#5  0x0000000000ee28ef in MueLu::HierarchyManager<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >::SetupHierarchy(MueLu::Hierarchy<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >&) const ()
#6  0x0000000000ec0895 in Teuchos::RCP<MueLu::Hierarchy<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> > > MueLu::CreateXpetraPreconditioner<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >(Teuchos::RCP<Xpetra::Matrix<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> > >, Teuchos::ParameterList const&, Teuchos::RCP<Xpetra::MultiVector<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> > >, Teuchos::RCP<Xpetra::MultiVector<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> > >) ()
#7  0x0000000000ec1a69 in Teuchos::RCP<MueLu::TpetraOperator<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> > > MueLu::CreateTpetraPreconditioner<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >(Teuchos::RCP<Tpetra::Operator<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> > > const&, Teuchos::ParameterList&, Teuchos::RCP<Tpetra::MultiVector<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>::classic> > const&, Teuchos::RCP<Tpetra::MultiVector<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>::classic> > const&) ()
#8  0x0000000000e0a889 in goal::build_multigrid_solver(Teuchos::RCP<Teuchos::ParameterList const>, Teuchos::RCP<goal::Indexer>, Teuchos::RCP<Tpetra::CrsMatrix<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>, false> >, Teuchos::RCP<Tpetra::Vector<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>, false> >, Teuchos::RCP<Tpetra::Vector<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>, false> >, int) () at /lore/granzb/goal/src/goal_linear_solvers.cpp:109
#9  0x0000000000e0b407 in goal::solve_multigrid_gmres(Teuchos::RCP<Teuchos::ParameterList const>, Teuchos::RCP<Tpetra::CrsMatrix<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>, false> >, Teuchos::RCP<Tpetra::Vector<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>, false> >, Teuchos::RCP<Tpetra::Vector<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>, false> >, Teuchos::RCP<goal::Indexer>) () at /lore/granzb/goal/src/goal_linear_solvers.cpp:191
#10 0x0000000000e0bba7 in goal::solve_linear_system(Teuchos::RCP<Teuchos::ParameterList const>, Teuchos::RCP<Tpetra::CrsMatrix<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>, false> >, Teuchos::RCP<Tpetra::Vector<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>, false> >, Teuchos::RCP<Tpetra::Vector<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>, false> >, Teuchos::RCP<goal::Indexer>) () at /lore/granzb/goal/src/goal_linear_solvers.cpp:213
#11 0x0000000000d9a85c in elast::Solver::solve_primal() () at /lore/granzb/goal/miniapps/elasticity/elast_solver.cpp:98
#12 0x0000000000d9b149 in elast::Solver::solve_primal_only() () at /lore/granzb/goal/miniapps/elasticity/elast_solver.cpp:135
#13 0x0000000000d9f2b1 in main () at /lore/granzb/goal/miniapps/elasticity/elast_solver.cpp:164

Why is this happening? I tried to follow very closely (read exactly) what was done in Albany's CTM, where I did not see this issue.

@ibaned, you mentioned you had seen this in the wild before? Any ideas?

[ibaned]

I've seen Albany output the matrix block size warning before. Don't know how to fix it, but I suspect our preconditioning won't be quite right until we do.

[bgranzow]

Could you point me to an example where this occurs in Albany if you can find one & when you get a chance?

[bgranzow]

Rebuilding CTM to try to do a one to one comparison between Albany and Goal of:

owned dof map
ghost dof map
owned node map
coordinate multi vector

:(

[bgranzow]

Apparently it does not actually affect the solve itself. I'm still curious why this warning doesn't get printed in CTM, but I'm going to close this for now.

https://trilinos.org/pipermail/trilinos-users/2017-April/005933.html