Caliper overwriting output file?
Closed this issue · 7 comments
I have a (Fortran) code that has been instrumented with Caliper and is configured so that it can trigger Caliper to write a report at regular intervals through the 'flush' method on the config-manager.
If I configure Caliper at runtime to collect the runtime-report and report it at, say, every 5 cycles to stdout (i.e. the default output destination), I get a runtime-report written to stdout every 5 cycles as expected.
However, if I set Caliper to write the runtime-reports to a file instead of stdout (with the 'output=my_output.txt' option), the final file that is written only contains the last runtime-report, and not any of the previous ones that should have been emitted during the run. It looks to me that Caliper is overwriting the file every time it writes a report.
I would have expected the behavior to be the same, regardless of the destination of the output. Is there an 'append' option that I'm missing? Or is this a bug in Caliper?
Thanks for any help!
Hi @cielling, there is currently no append option unfortunately.
You can however set a Caliper attribute and use that to create a new filename for every flush. That may be a workaround. The example below shows how to set an attribute "cycle" for the flush. You can then set output=my_output_%cycle%.txt for the filename, and it should create my_output_5.txt, my_output_10.txt, etc.:
call cali_begin_int_byname('cycle', i)
call mgr%flush
call cali_end_byname('cycle')
Hope this helps!
Hi @daboehme, thanks for the response!
So what I'm hearing is that output to a file is at least somewhat intentionally different from output to stdout?
Would Caliper be able to provide us with a flag to append the outputs when writing to a file?
The concern that the code team has with creating a new file for every flush is that this could potentially create thousands of files which would be stressing the file system.
Hi @cielling, yes the output to file is different from the stdout/stderr output.
I think it would make sense to offer an append option though and even make that the default. I'll look into it.
Hi @daboehme, I wanted to check in to see when you/the Caliper team will have time to look into the append option? Thanks!