edf-hpc/verrou

RDDMin's dd.sym folder name can go over the filesystem limit

HadrienG2 opened this issue · 1 comments

HadrienG2 commented

Modern programming languages use name mangling for the purpose of namespace disambiguation. In the presence of generic types, name mangling can generate very long symbol names because a type's methods are preceded by all the the type parameters. When this is combined with RDDMin's willingness to feature all faulty symbole names in a set as a comma-separated list, the filesystem name length limit can be reached:

dd: done
ddmin23 (
  _ZN4Acts12AtlasStepperINS_14ConstantBFieldEE5State6updateINS_26SingleBoundTrackParametersINS_13ChargedPolicyEEEEEvRKT_        /root/acts-core/spack-build/Tests/Integration/PropagationTests
  _ZNK4Acts15IntegrationTest29covariance_validation_fixtureINS_10PropagatorINS_12AtlasStepperINS_14ConstantBFieldEEENS_13VoidNavigatorEEEE19calculateCovarianceINS_26SingleBoundTrackParametersINS_13ChargedPolicyEEESC_NS7_7OptionsINS_10ActionListIJEEENS_9AbortListIJEEEEEEEN5Eigen6MatrixIdLi5ELi5ELi0ELi5ELi5EEERKT_RKSL_RKT0_RKT1_     /root/acts-core/spack-build/Tests/Integration/PropagationTests
  _ZSt18generate_canonicalIdLm53ESt26linear_congruential_engineImLm16807ELm0ELm2147483647EEET_RT1_      /root/acts-core/spack-build/Tests/Integration/PropagationTests
):
/root/acts-core/spack-build/Tests/Integration/dd.sym/84a02f517d8a210580d728fbb41b2286 --( run )-> PASS
/root/acts-core/spack-build/Tests/Integration/dd.sym/fb2eb4131a126169ec3705da3b4f7f1f --(cache)-> FAIL
/root/acts-core/spack-build/Tests/Integration/dd.sym/577d59aae63c7a37b3ec8411a8577ed7 --(cache)-> FAIL
/root/acts-core/spack-build/Tests/Integration/dd.sym/ebcc1ee1dcb242877e864e535f6ba54b --(cache)-> FAIL
/root/acts-core/spack-build/Tests/Integration/dd.sym/bdbbcdaf58738172d79424cb10a9ab5c --(cache)-> FAIL
/root/acts-core/spack-build/Tests/Integration/dd.sym/52c72e8cce653bc26e5ee8deb1ee256a --(cache)-> FAIL
/root/acts-core/spack-build/Tests/Integration/dd.sym/9746df00562dffd2feb25f3d4d5c0056 --(cache)-> FAIL
/root/acts-core/spack-build/Tests/Integration/dd.sym/6d933a0326f9294f0b020a172d958430 --(cache)-> FAIL
Traceback (most recent call last):
  File "/opt/spack/opt/spack/linux-opensuse_tumbleweed20180919-x86_64/gcc-8.2.1/verrou-valgrind-update-g3q4crfabmhq4qddliomqlfjpfrhxqw3/bin/verrou_dd", line 638, in <module>
    main(runScript, cmpScript, algoSearch=ddAlgo)
  File "/opt/spack/opt/spack/linux-opensuse_tumbleweed20180919-x86_64/gcc-8.2.1/verrou-valgrind-update-g3q4crfabmhq4qddliomqlfjpfrhxqw3/bin/verrou_dd", line 613, in main
    (refSym, confSymsTab) = ddSymRDDMin(run, compare)
  File "/opt/spack/opt/spack/linux-opensuse_tumbleweed20180919-x86_64/gcc-8.2.1/verrou-valgrind-update-g3q4crfabmhq4qddliomqlfjpfrhxqw3/bin/verrou_dd", line 457, in ddSymRDDMin
    dd.testSym(conf)
  File "/opt/spack/opt/spack/linux-opensuse_tumbleweed20180919-x86_64/gcc-8.2.1/verrou-valgrind-update-g3q4crfabmhq4qddliomqlfjpfrhxqw3/bin/verrou_dd", line 428, in testSym
    symlink(dirname, linkname)
  File "/opt/spack/opt/spack/linux-opensuse_tumbleweed20180919-x86_64/gcc-8.2.1/verrou-valgrind-update-g3q4crfabmhq4qddliomqlfjpfrhxqw3/bin/verrou_dd", line 288, in symlink
    os.symlink(src, dst)
OSError: [Errno 36] File name too long: '/root/acts-core/spack-build/Tests/Integration/dd.sym/6d933a0326f9294f0b020a172d958430' -> '/root/acts-core/spack-build/Tests/Integration/dd.sym._ZNK4Acts15IntegrationTest29covariance_validation_fixtureINS_17PropagatorWrapperISt10shared_ptrINS_16RungeKuttaEngineINS_14ConstantBFieldEEEEEEE19calculateCovarianceINS_32SingleCurvilinearTrackParametersINS_13ChargedPolicyEEESD_NS8_7OptionsINS_10ActionListIJEEENS_9AbortListIJEEEEEEEN5Eigen6MatrixIdLi5ELi5ELi0ELi5ELi5EEERKT_RKSM_RKT0_RKT1_,_ZNK4Acts15RungeKuttaUtils22transformLocalToGlobalEbPKNS_7SurfaceEPKdPd'

Since we can't tune up the filesystem name length limit easily (the Linux VFS name length limit is set at kernel build time, and on-disk filesystems have hard length limits in their specification), we need to get around it somehow. I can think of several possibilities:

  • Do not put symbol names in file names, instead only use hashes and provide the list of associated symbol names in a different way (e.g. by a text file index)
  • Carefully truncate symbol names so that we stay within our file name budget

These possibilities are not mutually exclusive. It could seem unfair to penalize the ergonomics of C and Fortran users just because languages with namespaces and generics generate ridiculously long symbol names. At the same time, if we truncate symbol names, we do need to provide the full list somewhere in case the truncated version is ambiguous (name mangling is used for a reason).

If we do end up truncating, I think it is best to truncate in the middle. That is because the beginning of the mangled name tells us which part of the program (namespace) we are talking about, whereas the end of the mangled name tells us which end-user method we are dealing with. The middle part of the mangled name is only important in cases where a generic type's parameters strongly affect its behaviour, which can happen but is not a very common issue.

lathuili commented

With a nice side effect of an other development, the non yet stabilized branch struct_rddmin (with last valgrind version) should solve your problem.
Indeed with the rddmin family algorithm (rddmin, srddmin and drddmin-2) there is a new output format.
Now the name of the symlinks are ddmin-i with i the rank of ddmin set. You can get the symbols corresponding to a ddmin set with
cat ddmin-i/dd.exclude

  • 👍1