Dymola 2019 model check fails on class methods

Question

Dymola 2019 model check fails on class methods

hyumo opened this issue 6 years ago · 18 comments

It is found that in Dymola 2019, modelcheck would fail on ExternData reader class methods, e.g. XMLFile.getReal, JSONFile.getReal etc. It seems that Dymola 2019 has changed their model check routine. It is complaining about not able to find the constructed object when doing a individual model check on the methods.

Check of ExternData.JSONFile.getReal:

Undeclared variable: ExternData.JSONFile.json
in the definition equation
json = ExternData.JSONFile.json

Failed in variable declarations of:
function ExternData.JSONFile.getReal 
input String varName "Key";
output Real y "Real value";
output Boolean exist "= true, if varName exits; = false, if it does not exist and y is set to 0.0";
input ExternData.Types.ExternJSONFile json := ExternData.JSONFile.json;
end ExternData.JSONFile.getReal;

Errors detected in functions.

Check aborted.

ERRORS have been issued.

ExternData version: 2.3.4
Dymola version: 2019
OS: Windows 10 Pro

Answer 1 · 2018-08-04T09:20:11.000Z

Did it work in older Dymola versions? If yes, can you contact the 3Ds support to ask if the change is by purpose or rather a side-effect? Thanks. I believe it is related to my question in #9 (comment) where I wondered if it is legal Modelica.

Answer 2 · 2018-08-04T12:36:50.000Z

Yes, it does work before Dymola 2019. I'll try to contact DS and let you know.

Answer 3 · 2018-08-05T17:29:28.000Z

Thanks. If it really is an intended error message it would be the show-stopper for ExternData functions. See also mo:#979 (comment).

Answer 4 · 2018-08-06T12:36:40.000Z

I've contacted our DS rep on this issue, let's wait and see what was going on.

I'm afraid that, if this is intended, ExternData has to be refactored a bit to pass the constructed object to those reader functions as inputs. It might also be good to encapsulate the c wrapper functions into the class as well. It will also fix the OpenModelica issue I believe.

Let me spend some time on it tonight and see if I can fix it.

Answer 5 · 2018-08-06T12:41:20.000Z

BTW, the library still works/simulates as it is before, phew~ It is just those reader functions won't pass the model check and it breaks the unit testing procedure we have internally.

Answer 6 · 2018-08-06T18:48:00.000Z

Yes, it does work before Dymola 2019.

Are you sure it is a regression? I can reproduce also in Dymola 2018 and 2018 FD01.

Answer 7 · 2018-08-06T18:54:35.000Z

I just checked on Dymola 2018FD01. Yes, I can reproduce this error. Sorry about this. I need to check which version of Dymola the our CI server is using.

Answer 8 · 2018-08-06T19:01:28.000Z

It also occurs on older versions, e.g., Dymola 2016.

Answer 9 · 2018-08-06T19:25:58.000Z

I think I figured what was going on.

I usually just do a model check on the whole package, Dymola usually skips those encapsulated functions. That's why they never showed up as errors. Using the testing library provided by Dymola, for some reason, gives different results between 2018FD01 and 2019.

Dymola 2018FD01 Testing lib behaves identical as doing a modelcheck on the whole package, which means all the encapsulated functions are excluded.
Dymola 2019 Testing lib behaves differently, it check all functions, no function is skipped.

Conclusion: this is a testing library inconsistent issue.

Thanks for the help, please close this issue if needed. I still think the syntax should be changed though, just to make it legal Modelica.

Answer 10 · 2018-08-06T19:29:12.000Z

Thanks for figuring it out.

Please tell me what to change to make it legal!

Answer 11 · 2018-08-06T19:35:02.000Z

Documentation of Testing.Runners.checkPackage prints that classes with the annotation __Dymola_interactive=true are skipped. Could this be a solution for the Testing issue?

Answer 12 · 2018-08-06T19:39:18.000Z

Yes, for sure.

Answer 13 · 2018-08-06T19:57:16.000Z

OK, check again then.

Answer 14 · 2018-08-06T20:01:30.000Z

Thanks for the fix. All passed.

> Checking all classes in ExternData
(i) started at 2018-8-6 16:00:21
(i) pedantic check is off
(i) found 160 classes
  #ok: ExternData.UsersGuide.References
  #ok: ExternData.UsersGuide.License
  #ok: ExternData.UsersGuide.Contact
  #ok: ExternData.Examples.CSVTest
  #ok: ExternData.Examples.INITest
  #ok: ExternData.Examples.JSONTest
  #ok: ExternData.Examples.MATTest
  #ok: ExternData.Examples.XLSTest
  #ok: ExternData.Examples.XLSTest.computeColSum
  #ok: ExternData.Examples.XLSXTest
  #ok: ExternData.Examples.XLSXTest.computeColSum
  #ok: ExternData.Examples.XMLTest
  #ok: ExternData.Examples.TIRTest
  #ok: ExternData.CSVFile
#skip:ExternData.CSVFile.getRealArray2D (__Dymola_interactive=true)
#skip:ExternData.CSVFile.getArraySize2D (__Dymola_interactive=true)
#skip:ExternData.CSVFile.getArrayRows2D (__Dymola_interactive=true)
#skip:ExternData.CSVFile.getArrayColumns2D (__Dymola_interactive=true)
  #ok: ExternData.INIFile
#skip:ExternData.INIFile.getReal (__Dymola_interactive=true)
#skip:ExternData.INIFile.getInteger (__Dymola_interactive=true)
#skip:ExternData.INIFile.getBoolean (__Dymola_interactive=true)
#skip:ExternData.INIFile.getString (__Dymola_interactive=true)
  #ok: ExternData.JSONFile
#skip:ExternData.JSONFile.getReal (__Dymola_interactive=true)
#skip:ExternData.JSONFile.getRealArray1D (__Dymola_interactive=true)
#skip:ExternData.JSONFile.getRealArray2D (__Dymola_interactive=true)
#skip:ExternData.JSONFile.getInteger (__Dymola_interactive=true)
#skip:ExternData.JSONFile.getIntegerArray1D (__Dymola_interactive=true)
#skip:ExternData.JSONFile.getIntegerArray2D (__Dymola_interactive=true)
#skip:ExternData.JSONFile.getBoolean (__Dymola_interactive=true)
#skip:ExternData.JSONFile.getBooleanArray1D (__Dymola_interactive=true)
#skip:ExternData.JSONFile.getBooleanArray2D (__Dymola_interactive=true)
#skip:ExternData.JSONFile.getString (__Dymola_interactive=true)
#skip:ExternData.JSONFile.getStringArray1D (__Dymola_interactive=true)
#skip:ExternData.JSONFile.getStringArray2D (__Dymola_interactive=true)
#skip:ExternData.JSONFile.getArraySize1D (__Dymola_interactive=true)
#skip:ExternData.JSONFile.getArraySize2D (__Dymola_interactive=true)
#skip:ExternData.JSONFile.getArrayRows2D (__Dymola_interactive=true)
#skip:ExternData.JSONFile.getArrayColumns2D (__Dymola_interactive=true)
  #ok: ExternData.MATFile
#skip:ExternData.MATFile.getRealArray2D (__Dymola_interactive=true)
#skip:ExternData.MATFile.getStringArray1D (__Dymola_interactive=true)
#skip:ExternData.MATFile.getArraySize2D (__Dymola_interactive=true)
#skip:ExternData.MATFile.getArrayRows2D (__Dymola_interactive=true)
#skip:ExternData.MATFile.getArrayColumns2D (__Dymola_interactive=true)
  #ok: ExternData.XLSFile
#skip:ExternData.XLSFile.getReal (__Dymola_interactive=true)
#skip:ExternData.XLSFile.getRealArray2D (__Dymola_interactive=true)
#skip:ExternData.XLSFile.getInteger (__Dymola_interactive=true)
#skip:ExternData.XLSFile.getBoolean (__Dymola_interactive=true)
#skip:ExternData.XLSFile.getString (__Dymola_interactive=true)
#skip:ExternData.XLSFile.getArraySize2D (__Dymola_interactive=true)
#skip:ExternData.XLSFile.getArrayRows2D (__Dymola_interactive=true)
#skip:ExternData.XLSFile.getArrayColumns2D (__Dymola_interactive=true)
  #ok: ExternData.XLSXFile
#skip:ExternData.XLSXFile.getReal (__Dymola_interactive=true)
#skip:ExternData.XLSXFile.getRealArray2D (__Dymola_interactive=true)
#skip:ExternData.XLSXFile.getInteger (__Dymola_interactive=true)
#skip:ExternData.XLSXFile.getBoolean (__Dymola_interactive=true)
#skip:ExternData.XLSXFile.getString (__Dymola_interactive=true)
#skip:ExternData.XLSXFile.getArraySize2D (__Dymola_interactive=true)
#skip:ExternData.XLSXFile.getArrayRows2D (__Dymola_interactive=true)
#skip:ExternData.XLSXFile.getArrayColumns2D (__Dymola_interactive=true)
  #ok: ExternData.XMLFile
#skip:ExternData.XMLFile.getReal (__Dymola_interactive=true)
#skip:ExternData.XMLFile.getRealArray1D (__Dymola_interactive=true)
#skip:ExternData.XMLFile.getRealArray2D (__Dymola_interactive=true)
#skip:ExternData.XMLFile.getInteger (__Dymola_interactive=true)
#skip:ExternData.XMLFile.getBoolean (__Dymola_interactive=true)
#skip:ExternData.XMLFile.getString (__Dymola_interactive=true)
#skip:ExternData.XMLFile.getArraySize1D (__Dymola_interactive=true)
#skip:ExternData.XMLFile.getArraySize2D (__Dymola_interactive=true)
#skip:ExternData.XMLFile.getArrayRows2D (__Dymola_interactive=true)
#skip:ExternData.XMLFile.getArrayColumns2D (__Dymola_interactive=true)
  #ok: ExternData.TIRFile
#skip:ExternData.TIRFile.getReal (__Dymola_interactive=true)
#skip:ExternData.TIRFile.getInteger (__Dymola_interactive=true)
#skip:ExternData.TIRFile.getBoolean (__Dymola_interactive=true)
#skip:ExternData.TIRFile.getString (__Dymola_interactive=true)
  #ok: ExternData.Functions.CSV.getRealArray2D
  #ok: ExternData.Functions.CSV.getArraySize2D
  #ok: ExternData.Functions.CSV.getArrayRows2D
  #ok: ExternData.Functions.CSV.getArrayColumns2D
  #ok: ExternData.Functions.INI.getReal
  #ok: ExternData.Functions.INI.getInteger
  #ok: ExternData.Functions.INI.getBoolean
  #ok: ExternData.Functions.INI.getString
  #ok: ExternData.Functions.JSON.getReal
  #ok: ExternData.Functions.JSON.getRealArray1D
  #ok: ExternData.Functions.JSON.getRealArray2D
  #ok: ExternData.Functions.JSON.getInteger
  #ok: ExternData.Functions.JSON.getIntegerArray1D
  #ok: ExternData.Functions.JSON.getIntegerArray2D
  #ok: ExternData.Functions.JSON.getBoolean
  #ok: ExternData.Functions.JSON.getBooleanArray1D
  #ok: ExternData.Functions.JSON.getBooleanArray2D
  #ok: ExternData.Functions.JSON.getString
  #ok: ExternData.Functions.JSON.getStringArray1D
  #ok: ExternData.Functions.JSON.getStringArray2D
  #ok: ExternData.Functions.JSON.getArraySize1D
  #ok: ExternData.Functions.JSON.getArraySize2D
  #ok: ExternData.Functions.JSON.getArrayRows2D
  #ok: ExternData.Functions.JSON.getArrayColumns2D
  #ok: ExternData.Functions.MAT.getRealArray2D
  #ok: ExternData.Functions.MAT.getStringArray1D
  #ok: ExternData.Functions.MAT.getArraySize2D
  #ok: ExternData.Functions.MAT.getArrayRows2D
  #ok: ExternData.Functions.MAT.getArrayColumns2D
  #ok: ExternData.Functions.XLS.getReal
  #ok: ExternData.Functions.XLS.getRealArray2D
  #ok: ExternData.Functions.XLS.getInteger
  #ok: ExternData.Functions.XLS.getBoolean
  #ok: ExternData.Functions.XLS.getString
  #ok: ExternData.Functions.XLS.getArraySize2D
  #ok: ExternData.Functions.XLS.getArrayRows2D
  #ok: ExternData.Functions.XLS.getArrayColumns2D
  #ok: ExternData.Functions.XLSX.getReal
  #ok: ExternData.Functions.XLSX.getRealArray2D
  #ok: ExternData.Functions.XLSX.getInteger
  #ok: ExternData.Functions.XLSX.getBoolean
  #ok: ExternData.Functions.XLSX.getString
  #ok: ExternData.Functions.XLSX.getArraySize2D
  #ok: ExternData.Functions.XLSX.getArrayRows2D
  #ok: ExternData.Functions.XLSX.getArrayColumns2D
  #ok: ExternData.Functions.XML.getReal
  #ok: ExternData.Functions.XML.getRealArray1D
  #ok: ExternData.Functions.XML.getRealArray2D
  #ok: ExternData.Functions.XML.getInteger
  #ok: ExternData.Functions.XML.getBoolean
  #ok: ExternData.Functions.XML.getString
  #ok: ExternData.Functions.XML.getArraySize1D
  #ok: ExternData.Functions.XML.getArraySize2D
  #ok: ExternData.Functions.XML.getArrayRows2D
  #ok: ExternData.Functions.XML.getArrayColumns2D
  #ok: ExternData.Interfaces.partialGetReal
  #ok: ExternData.Interfaces.partialGetInteger
  #ok: ExternData.Interfaces.partialGetBoolean
  #ok: ExternData.Interfaces.partialGetString
  #ok: ExternData.Types.ExternCSVFile
  #ok: ExternData.Types.ExternCSVFile.constructor
  #ok: ExternData.Types.ExternCSVFile.destructor
  #ok: ExternData.Types.ExternINIFile
  #ok: ExternData.Types.ExternINIFile.constructor
  #ok: ExternData.Types.ExternINIFile.destructor
  #ok: ExternData.Types.ExternJSONFile
  #ok: ExternData.Types.ExternJSONFile.constructor
  #ok: ExternData.Types.ExternJSONFile.destructor
  #ok: ExternData.Types.ExternMATFile
  #ok: ExternData.Types.ExternMATFile.constructor
  #ok: ExternData.Types.ExternMATFile.destructor
  #ok: ExternData.Types.ExternXLSFile
  #ok: ExternData.Types.ExternXLSFile.constructor
  #ok: ExternData.Types.ExternXLSFile.destructor
  #ok: ExternData.Types.ExternXLSXFile
  #ok: ExternData.Types.ExternXLSXFile.constructor
  #ok: ExternData.Types.ExternXLSXFile.destructor
  #ok: ExternData.Types.ExternXMLFile
  #ok: ExternData.Types.ExternXMLFile.constructor
  #ok: ExternData.Types.ExternXMLFile.destructor
(i) check finished at 2018-8-6 16:00:43
(i) required time: 0d 0h 0min 22s 322ms
(i) Summary
  160 classes
  101 ok
    0 fail
   59 skipped

Answer 15 · 2018-08-07T03:24:09.000Z

I've made an example of CSV reader

https://github.com/hyumo/CSVReader

Answer 16 · 2018-08-07T06:54:16.000Z

I've made an example of CSV reader

Nice demonstration. Thanks a lot! I will adapt ExternData to your proposed structure though OpenModelica still gives the same "cyclic dependency" error.

Answer 17 · 2018-08-07T12:56:51.000Z

I tried different things in OpenModelica too, but none of them works. It's giving me different errors, but only one is consistent: Not being able to find the constructed csv object etc.

I'll keep trying, I am not a fan of passing the filename and constructor settings to each function. Actually, I should say I don't know if it calls the constructor function for each any every getXXX methods during initialization, this will be a big performance hit. Does the current fix for openmodelica constructs a reader object in each get method?

Answer 18 · 2018-08-07T17:35:37.000Z

See also omc:#4448 regarding ExternData in OpenModelica.