Benchmark service don't correctly clean intermediate files

[elmattic]

Describe the bug

We currently don't manage to run forest-benchmark service for an extended period of time.

The service fails by indicating a No space left on device error during one of the benchmark steps.

To Reproduce
Steps to reproduce the behavior:

1. Configure correctly terraform to run the service on a test droplet
2. Launch the service and wait for a few days
3. See error

Log output

Log Output

I, [2023-09-17T16:29:55.715066 #2671492]  INFO -- : $ ./forest/target/release/forest-tool fetch-params --keys --config /chainsafe/tmp/.forest/forest.toml
I, [2023-09-17T16:29:56.734915 #2671492]  INFO -- : $ ./forest/target/release/forest --config /chainsafe/tmp/.forest/forest.toml --encrypt-keystore false --import-snapshot /chainsafe/tmp/3221520_2023_09_17T12_00_00Z.car --halt-after-import
2023-09-17T16:29:56.765060Z  INFO forest_filecoin::daemon: Starting Forest daemon, version 0.13.0+git.63598718e
2023-09-17T16:29:56.765188Z  INFO forest_filecoin::libp2p::keypair: Recovered libp2p keypair from /chainsafe/tmp/.forest/libp2p/keypair
2023-09-17T16:29:56.765201Z  INFO forest_filecoin::daemon: PeerId: 12D3KooWCtT7dcq4wjFvQdFg7puHFadnMXYhc8sHYRGN1M8YHwqw
2023-09-17T16:29:56.765218Z  WARN forest_filecoin::daemon: Forest has encryption disabled
2023-09-17T16:29:56.765295Z  INFO forest_filecoin::daemon: Admin token: eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJBbGxvdyI6WyJyZWFkIiwid3JpdGUiLCJzaWduIiwiYWRtaW4iXSwiZXhwIjoxNzAwMTUyMTk2fQ.dZ1Puv05pZnv_mW2MsoNQA-7w7QP0TWC9lV_n6llYXU
2023-09-17T16:29:56.765329Z  INFO forest_filecoin::db::migration::db_migration: No database migration required
2023-09-17T16:29:56.780171Z  INFO forest_filecoin::daemon: Prometheus server started at 0.0.0.0:6116
2023-09-17T16:29:56.786132Z  INFO forest_filecoin::genesis: Initialized genesis: BlockHeader: Cid(bafy2bzacecnamqgqmifpluoeldx7zzglxcljo6oja4vrmtj7432rphldpdmm2)
2023-09-17T16:29:56.786279Z  INFO forest_filecoin::db::rolling::gc: Running automatic database garbage collection task
2023-09-17T16:29:56.786330Z  INFO forest_filecoin::db::rolling::gc: Listening on database garbage collection events
2023-09-17T16:29:56.786556Z  INFO forest_filecoin::daemon: Using network :: mainnet
2023-09-17T16:29:57.153038Z  WARN forest_filecoin::libp2p::behaviour: libp2p Forest version: 0.13.0+git.63598718e
2023-09-17T16:29:57.153457Z  INFO forest_filecoin::chain_sync::chain_muxer: Evaluating network head...
2023-09-17T16:29:57.153475Z  INFO forest_filecoin::daemon::db_util: Importing chain from snapshot at: /chainsafe/tmp/3221520_2023_09_17T12_00_00Z.car
2023-09-17T16:29:57.153497Z  INFO forest_filecoin::daemon: JSON-RPC endpoint started at 127.0.0.1:2345
2023-09-17T16:29:57.153615Z  INFO forest_filecoin::rpc: Ready for RPC connections
23-09-17T16:50:17.656762Z  INFO forest_filecoin::daemon::main: Shutting down tokio...
Error: No space left on device (os error 28)
2023-09-17T16:50:18.111235Z  INFO forest_filecoin::daemon::main: Forest finish shutdown

Expected behaviour

We should be able to run the service continuously, but also keep track of disk usage.

Other information and links