VLSV plugin broken on Linux 16.04 with recent Visit versions.

[ykempf]

I have in recent months tried to (re)compile the VLSV plugin with newer VisIt versions (2.12.1 at the moment). It generally works (although I sometimes have to link some system MPI libraries as it won't find them properly). However VisIt's mdserver dies when launching VisIt, which it doesn't do when I remove the Vlsv plugin files from the corresponding directory.

I have two theories, either the compatibility of the plugin with 16.04 linuxes is gone, or the VisIt plugin interface has been somewhat modified.

I have next to no clue about the VisIt plugin things, that's why I post it here. @ursg you have more experience so maybe you can look at some point. I understand that @sandroos you don't have a linux available for testing at the moment, do you? In any case, hints are welcome as well.

It is not a big hindrance as I mostly use the plugin on remote engines (taito.csc.fi) and there it works properly, somehow...

[sandroos]

@ykempf same issue as #23 ?

[ykempf]

Yes, right, except that in my case even recompiling the whole dependencies + VisIt from scratch didn't help...

[sandroos]

Can you run visit -debug 5 and post the output of the mdserver.5.vlog (or whatever it was), it should say something in it about vlsv plugin. If it actually manages to get past metadata server the error is in engine.5.vlog.

[ykempf]

I don't see any useful messages there, but here you go. Let's try to attach the files somehow.

A.gui.5.txt
A.mdserver.5.txt
A.viewer.5.txt

[sandroos]

I suspect this is the same issue as before, VisIt and plugin compilations are not compatible, i.e., different compilers somehow got used. I'll make a mental note to check this..

[ykempf]

From what I can see I use the system gcc/mpic++ (5.4.0) both for the visit compilation and for vlsv. However I am not 100% sure what visit will use down the line in the install script. This is only what I set with environment variables in my install script.

[ursg]

I've dug more deeply into this yesterday evening, and the problem seems to be in some way related to signedness conversion of int64_t vs. uint64_t, although I don't quite see what the problem is yet.

The vlsv library, when called from the visit plugin, fails to read the XML footer from a file (in vlsv_reader.cpp, line 228).

I instrumented that section with a bit of debugging code of the form

         filein.seekg(footerOffset);
         streampos pos = filein.tellg();                                                                               
         cerr << "Seek offset was " << footerOffset << ", actual position is " << pos << " => difference = " << (pos - footerOffset) << endl;                                                   
         std::string footerpeek;                                                                                       
         filein >> footerpeek;                                                                                         
         cerr << "Data at footer location: " << footerpeek;                    

And the output I get for an examplary vlsv file (from ABC) is Seek offset was 2459598420, actual position is -1 => difference = -2459598421. Looking with dd at the same byte offset, I do find the xml footer though. So something must be going wrong when seeking the file here. Note that the number is just a little bit above 2^31... is a signed 32bit integer being used somewhere inside the standard library here?

[ursg]

Actually, I was mistaken there: I tried to seek in the file after it's EOF had been reached, which seemingly doesn't work anymore? filein.good() was returning false at least, and that's also what caused the footer reader in muxml to fail.

While this patch fixes it, and with it, vlsv files are again readable, I still don't fully understand why it's necessary. Has the behaviour of ios::good() changed recently?

diff --git a/muxml.cpp b/muxml.cpp
index 4b8016d..32b7a8e 100644
--- a/muxml.cpp
+++ b/muxml.cpp
@@ -225,7 +225,10 @@ bool MuXML::read(std::istream& in,XMLNode* parent,const int& level,const char& c
         while ((c == ' ' || c == '\t' || c == '\n') && in.good() == true) in >> c;
       }
       
-      if (in.good() == false) return false;
+      if (in.good() == false && in.eof() == false) return false;
+      if (in.eof() == true) {
+         return true;
+      }
    } while (success == true);
    return true;
 }

[iljah]

Looks like ios::good() has been the same always: http://en.cppreference.com/w/cpp/io/basic_ios/good and is only true if nothing else, including eof, has happened. Don't know much about the standard wrt i/o but SO has dealt with this: https://stackoverflow.com/questions/24457557

[sandroos]

I still suspect there's an issue with linked library versions or something similar, muxml.cpp hasn't been touched in years. Then again, I haven't had time to work on this.

[sandroos]

I'll take a look at this once I get our move out of the way

[ykempf]

I can confirm that this also bites using vlsvdiff in Marconi.

[sandroos]

I suppose the conclusion of flowdock discussion was that there are no major issues in getting vlsv plugin to compile and work in Ubuntu 16.04 ? If that still stands, I'll close this issue.