When using default TMY weather URL receiving error-- OSError: [Errno 28] No space left on device
whiphi92 opened this issue · 3 comments
Describe the bug
Receiving immediate failure during the sampling step of buildstockbatch when pointing to a URL for weather files.
e.g., weather_files_url: https://data.nrel.gov/system/files/156/BuildStock_TMY3_FIPS.zip
To Reproduce
You can reproduce that by using buildstock-2023.06.0
environment and running national_baseline.yml on Eagle while pointing to the weather file URL.
Logs
sampling.out:
DEBUG:2023-08-29 08:40:27:buildstockbatch.base:Using OpenStudio version: 3.6.1 with SHA: bb9481519e
DEBUG:2023-08-29 08:40:27:__main__:Output directory = /scratch/pwhite/durham_buildstock_v2
DEBUG:2023-08-29 08:40:27:buildstockbatch.base:Downloading weather files
Traceback (most recent call last):
File "/shared-projects/buildstock/envs/buildstock-2023.06.0/lib/python3.10/site-packages/buildstockbatch/base.py", line 115, in _get_weather_files
f.write(chunk)
OSError: [Errno 28] No space left on device
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/shared-projects/buildstock/envs/buildstock-2023.06.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/shared-projects/buildstock/envs/buildstock-2023.06.0/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/shared-projects/buildstock/envs/buildstock-2023.06.0/lib/python3.10/site-packages/buildstockbatch/eagle.py", line 875, in <module>
main()
File "/shared-projects/buildstock/envs/buildstock-2023.06.0/lib/python3.10/site-packages/buildstockbatch/utils.py", line 107, in run_with_error_capture
return func(*args, **kwargs)
File "/shared-projects/buildstock/envs/buildstock-2023.06.0/lib/python3.10/site-packages/buildstockbatch/eagle.py", line 840, in main
batch = EagleBatch(args.project_filename)
File "/shared-projects/buildstock/envs/buildstock-2023.06.0/lib/python3.10/site-packages/buildstockbatch/eagle.py", line 80, in __init__
weather_dir = self.weather_dir # noqa E841
File "/shared-projects/buildstock/envs/buildstock-2023.06.0/lib/python3.10/site-packages/buildstockbatch/eagle.py", line 168, in weather_dir
self._get_weather_files()
File "/shared-projects/buildstock/envs/buildstock-2023.06.0/lib/python3.10/site-packages/buildstockbatch/base.py", line 112, in _get_weather_files
with tempfile.TemporaryFile() as f:
OSError: [Errno 28] No space left on device
real 0m13.196s
user 0m2.673s
sys 0m2.517s
Platform (please complete the following information):
- Simulation platform: Eagle
- BuildStockBatch version, branch, or sha: buildstock-2023.06.0
Seems like either you have run out of space in /scratch/pwhite/ or there is some problem in eagle. You can try to create a file manually in /scratch/pwhite/ see what happens.
I troubleshooted this with @nmerket and he requested I post this issue. We do not think it is a space issue.
This happens when downloading and extracting weather files to a temporary file. I think on Linux that lands in /tmp
. On Eagle compute nodes that's a RAM disk with very little capacity. I need to confirm that, though. If that's the case it's an easy fix, just point it to /tmp/scratch
, which is a 1 TB attached hard drive. The weird thing is that this hasn't happened before, which is kind of surprising.