Receiving error that database name for a new s3 bucket is not a regex match
whiphi92 opened this issue · 2 comments
Describe the bug
I received the following error message during YML validation:
yamale.yamale_error.YamaleError: Error validating data '/home/pwhite/projects/heat-pump-deployment/resstock/project_national/national_upgrades_HPs.yml' with schema '/kfs2/shared-projects/buildstock/envs/buildstock-2023.11.0/lib/python3.11/site-packages/buildstockbatch/schemas/v0.3.yaml' postprocessing.aws.athena.database_name: 'heat-pump-deployment' is not a regex match.
I commented out the database block in the YML and it validated successfully.
For reference, my s3 bucket is heat-pump-deployment
, which at the beginning of this ResStock run was completely empty and no database had been created yet.
To Reproduce
Steps to reproduce the behavior:
- Use buildstock_kestrel on a YML that has an empty s3 bucket and no database yet
- This error should proc
Expected behavior
I expect bsb to automatically make the database during the upload and pass this error. Maybe it works differently? I have never encountered this before.
Logs
(buildstock-2023.11.0) [pwhite@kl1 resstock]$ buildstock_kestrel project_national/national_upgrades_HPs.yml
_ __ _ __, _ __
( / ) o // /( _/_ / ( / ) _/_ /
/--< , ,, // __/ `. / __ _, /< /--< __, / _, /
/___/(_/_(_(/_(_/_(___)(__(_)(__/ |_/___/(_/(_(__(__/ /_
Executing BuildStock projects with grace since 2018
Traceback (most recent call last):
File "/kfs2/shared-projects/buildstock/envs/buildstock-2023.11.0/bin/buildstock_kestrel", line 8, in <module>
sys.exit(kestrel_cli())
^^^^^^^^^^^^^
File "/kfs2/shared-projects/buildstock/envs/buildstock-2023.11.0/lib/python3.11/site-packages/buildstockbatch/hpc.py", line 838, in kestrel_cli
user_cli(KestrelBatch, argv)
File "/kfs2/shared-projects/buildstock/envs/buildstock-2023.11.0/lib/python3.11/site-packages/buildstockbatch/hpc.py", line 893, in user_cli
Batch.validate_project(project_filename)
File "/kfs2/shared-projects/buildstock/envs/buildstock-2023.11.0/lib/python3.11/site-packages/buildstockbatch/hpc.py", line 794, in validate_project
super().validate_project(project_file)
File "/kfs2/shared-projects/buildstock/envs/buildstock-2023.11.0/lib/python3.11/site-packages/buildstockbatch/hpc.py", line 82, in validate_project
super().validate_project(project_file)
File "/kfs2/shared-projects/buildstock/envs/buildstock-2023.11.0/lib/python3.11/site-packages/buildstockbatch/base.py", line 271, in validate_project
assert cls.validate_project_schema(project_file)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/kfs2/shared-projects/buildstock/envs/buildstock-2023.11.0/lib/python3.11/site-packages/buildstockbatch/base.py", line 381, in validate_project_schema
return yamale.validate(schema, data, strict=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/kfs2/shared-projects/buildstock/envs/buildstock-2023.11.0/lib/python3.11/site-packages/yamale/yamale.py", line 43, in validate
raise YamaleError(results)
yamale.yamale_error.YamaleError: Error validating data '/home/pwhite/projects/heat-pump-deployment/resstock/project_national/national_upgrades_HPs.yml' with schema '/kfs2/shared-projects/buildstock/envs/buildstock-2023.11.0/lib/python3.11/site-packages/buildstockbatch/schemas/v0.3.yaml'
postprocessing.aws.athena.database_name: 'heat-pump-deployment' is not a regex match.
(buildstock-2023.11.0) [pwhite@kl1 resstock]$ buildstock_kestrel project_national/national_upgrades_HPs.yml
_ __ _ __, _ __
( / ) o // /( _/_ / ( / ) _/_ /
/--< , ,, // __/ `. / __ _, /< /--< __, / _, /
/___/(_/_(_(/_(_/_(___)(__(_)(__/ |_/___/(_/(_(__(__/ /_
Executing BuildStock projects with grace since 2018
INFO:2024-02-21 09:41:37:buildstockbatch.base:Base Validation Successful
INFO:2024-02-21 09:41:37:buildstockbatch.hpc:HPC Validation Successful
INFO:2024-02-21 09:41:37:buildstockbatch.hpc:Kestrel Validation Successful
INFO:2024-02-21 09:41:37:buildstockbatch.hpc:Creating output directory /projects/heatpump/test_runs/n_500_20240221_v1
INFO:2024-02-21 09:41:37:buildstockbatch.hpc:Submitting sampling job to task scheduler
Submitted batch job 2668859
INFO:2024-02-21 09:41:38:buildstockbatch.hpc:Run squeue -u $USER to monitor the progress of your jobs
(buildstock-2023.11.0) [pwhite@kl1 resstock]$ watch squeue -u pwhite
(buildstock-2023.11.0) [pwhite@kl1 resstock]$
The successful validation is after I removed the database block
Platform (please complete the following information):
- Simulation platform: Kestrel
- BuildStockBatch version, branch, or sha: buildstock-2023.11.0
- resstock or comstock repo version, branch, or sha: max-airflow-args2 branch of resstock
- Local Desktop OS: Windows
Additional context
None
Database name must be lowercase, start from letters and contain only letters, numbers and underscore character.
The database name cannot contain dash.
We want to re-open this issue to get a more intellagent error message in the validator.