cihga39871/JobSchedulers.jl

Can I see how much memory a Job actually took?

Closed this issue · 2 comments

Not sure if this is a feature request or a question: I'm not sure how much memory my Jobs take relative to the inputs they receive. I'd like to update the estimate I provide to the mem keyword based on the amount of memory the last jobs took. Is the high-water mark of memory consumed by a given job recorded by the scheduler? How could I access it?

If it isn't recorded, would it be feasible to add that feature?

Thanks for the question. If you are using Linux, you can use /usr/bin/time -v your command. And Maximum resident set size in the output is the maximum memory. Example:

/usr/bin/time -v echo 123                    
123
	Command being timed: "echo 123"
	User time (seconds): 0.00
	System time (seconds): 0.00
	Percent of CPU this job got: 94%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.00
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 1792
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 131
	Voluntary context switches: 1
	Involuntary context switches: 0
	Swaps: 0
	File system inputs: 0
	File system outputs: 0
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

JobScheduler does not assess the actual CPU or memory a task uses.

Thank you!