unable to create log file
shubh0155 opened this issue · 4 comments
hey i have installed darshan via spack on our hpc system and while running an IO benchmark using darshan i am getting this error
darshan_library_warning: unable to create log file cosmo.darshan.
the directory has all read write permiissions.
kindly help
I think the darshan-runtime is unable to do an operation similar to mkdir -p, even if the top-level dir has the appropriate permissions. I believe the usual suggestion is to either run the darshan-mk-log-dirs.pl script, or (what I usually do) use the --with-log-path-by-env=DARSHAN_LOGPATH when configuring, and then specify the log path directly that way.
I believe the Argonne folks usually recommend using that perl script for system-level setups, because it organizes by year, month, day for directories to keep logs organized, but I'll let them chime in on that perhaps.
Ah, the spack build for darshan-runtime is a little special/unusual. It assumes that the use case is installing Darshan as a user, so when the darshan-runtime spack module is loaded it sets an environment variable to control the log path: $DARSHAN_LOG_DIR_PATH. This path doesn't need to be populated with a subdirectory hierarchy. It just puts logs files directly in that directory.
You can set this environment variable to anything you would like, but it defaults to $HOME. Is it possible that the home directory is not writable on your system? Or have you set the environment variable to point elsewhere?
The log file is created with MPI-IO and must be on a shared file system.
@carns @tylerjereddy thank you for your shelp the issue has been ressolved i installed darshan manually and generated the logs. however while running the darshan log file on vani iam facing a new challenge i.e its unable to find the columns in my log file i even compared it with test.darshan log file it has same number of columns. i have attached the screenshots kindly have a look please
.
I'm only generally familiar with the VaniDL tool (i.e., a trace analysis tool for deep learning apps using DXT trace data), but it's not clear to me from your screenshots what the issue you're describing is. The error backtrace shown doesn't appear to show an issue in any PyDarshan code, so you might have better luck asking on the VaniDL github page.
I'll close this for now as I think the original issue has been resolved. Feel free to open a new issue here with more details if you think you've found a problem specifically with Darshan or PyDarshan code.