Feature request: tmp/media files cleanup script
iankevinmcdonald opened this issue · 3 comments
When the script crashes (which is does fairly often for me) it leave a bunch of media files behind.
A cron script to neatly clean up files would be much appreciated. Removing audio, video, and tmp files manually seems to break audiogram completely (perhaps because it is expecting to use the files I have removed.)
Short of a cleanup script, just documenting when it's okay to delete which files would be really useful. I'm running it on an old netbook to produce audiograms for my community radio / podcast series, so running out of filespace is a real issue.
Deleting the contents of [AUDIOGRAM_ROOT]/tmp/ should always be safe as long as a rendering is not in progress. This folder will only be non-empty in the case of a crash.
Deleting the contents of [AUDIOGRAM_ROOT]/media/ should always be safe (though you'll lose the videos).
Removing those files shouldn't break Audiogram as long as it's not running/rendering while you delete them. Can you give me some more details on how deleting the files breaks it for you?
A cron entry like the following would probably work to clear both folders daily, but use caution when setting up any automatic file deletion!
0 0 * * * rm -r /where/you/installed/audiogram/media/* /where/you/installed/audiogram/tmp/*An early version of Audiogram included a cleanup command, but we ultimately left it out because it would be hard to guarantee someone couldn't delete something important accidentally with a weird configuration.
Thanks for the explanation.
It's an issue #63 concurrency issue. I thought it was because of the deleted files because the error would begin with the name of a non-existent file.
I found that I could get it to work again (well, at least once) by untarring the whole of the audiogram directory from a .tgz file. So I assumed that the remaining tmp files had something to do with the crash. Of course, that could just have been luck.
It's stopped crashing now because I've followed the #63 fix.
`ian@orwell ~/audiogram $ npm start
audiogram@0.9.5 start /home/ian/audiogram
npm run postinstall && bin/server
audiogram@0.9.5 postinstall /home/ian/audiogram
mkdir -p editor/js && browserify client/index.js > editor/js/bundle.js
info: Listening on 8888
undefined:1
{"jobs":[],"f372a7a0-c3a0-11e6-922e-d5a644c93225":{"status":"error","numFrames":868,"framesComplete":868,"error":"Error: Command failed: /bin/sh -c ffmpeg -r 20 -i "/home/ian/audiogram/tmp/f372a7a0-c3a0-11e6-922e-d5a644c93225/frames/%06d.png" -i "/home/ian/audiogram/tmp/f372a7a0-c3a0-11e6-922e-d5a644c93225/audio" -c:v libx264 -c:a aac -strict experimental -shortest -pix_fmt yuv420p "/home/ian/audiogram/tmp/f372a7a0-c3a0-11e6-922e-d5a644c93225/video.mp4"\nffmpeg version 0.8.17-4:0.8.17-0ubuntu0.12.04.2, Copyright (c) 2000-2014 the Libav developers\n built on Apr 1 2016 14:28:02 with gcc 4.6.3\nThe ffmpeg program is only provided for script compatibility and will be removed\nin a future release. It has been deprecated in the Libav project to allow for\nincompatible command line syntax improvements in its replacement called avconv\n(see Changelog for details). Please use avconv instead.\n[image2 @ 0x982b4a0] max_analyze_duration reached\nInput #0, image2, from '/home/ian/audiogram/tmp/f372a
SyntaxError: Unexpected end of input
at Object.parse (native)
at load (/home/ian/audiogram/lib/transports/redis/fake.js:14:15)
at update (/home/ian/audiogram/lib/transports/redis/fake.js:24:12)
at Object.addJob (/home/ian/audiogram/lib/transports/redis/fake.js:72:5)
at WriteStream. (/home/ian/audiogram/server/render.js:48:16)
at emitNone (events.js:72:20)
at WriteStream.emit (events.js:166:7)
at fs.js:1772:14
at FSReqWrap.oncomplete (fs.js:82:15)
npm ERR! Linux 3.2.0-23-generic
npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "start"
npm ERR! node v4.7.0
npm ERR! npm v2.15.11
npm ERR! code ELIFECYCLE
npm ERR! audiogram@0.9.5 start: npm run postinstall && bin/server
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the audiogram@0.9.5 start script 'npm run postinstall && bin/server'.
npm ERR! This is most likely a problem with the audiogram package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! npm run postinstall && bin/server
npm ERR! You can get information on how to open an issue for this project with:
npm ERR! npm bugs audiogram
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!
npm ERR! npm owner ls audiogram
npm ERR! There is likely additional logging output above.
npm ERR! Please include the following file with any support request:
npm ERR! /home/ian/audiogram/npm-debug.log`
Here's our daily cron, uses the find command and deletes any files over 24hrs old.
find /where/you/installed/audiogram/media/* /where/you/installed/audiogram/tmp -type f -mtime +1 -delete