Port AbacusSummit EmulatorGrid to ltu-cmass
Closed this issue · 4 comments
We want to use the AbacusSummit_base_c{130-181}_ph000 halos to validate the products of the ltu-cmass forward model on 2 Gpc/h scales. We need them in the same format as the outputs of ltu-cmass halo generation so that they can be adopted to the pipeline.
We want the z=0.5 snapshots, since CMASS is z~0.55.
Definition of done:
A script to adopt the raw Abacus outputs to ltu-cmass OR catalogs of Abacus halos uploaded to OSN
I will start working on it tomorrow.
Rk: useful example of loading CompaSO Halo catalogs with the Abacus API in there.
(Notes for abacusnbody dependency issues for a fresh installation)
First, create a new python virtual environment (mine was exactly python --version 3.10.12)
python 3.10 -m venv abacus_env
source abacus_env/bin/activate
pip install abacusutils
Now, if you try playing with your catalog as in the available example, you will have errors at the first import attempt of CompaSOHaloCatalog related to asdf. Unfortunately, since the Abacus catalog use a custom compression scheme called "blsc", we cannot get away with just using asdf and a few quick workarounds.
For a code we will use just a few times, rather than forking and fixing the abacusnbody.data dependency inconsistencies (eg asdf.compression is now asdf._compression, some asdf function does not exist or has been moved god knows where), the following works:
pip install asdf ==2.13
The only bad piece of news is that running a fresh pip install jupyterlab for easy notebook testing will break abacusnbody with the module jsonschema (maybe an earlier jupyter-lab version is fine, if you have time to play around). To test if you can at last play with Abacus CompaSO catalogs, run (only the first line should be really necessary)
python
>>> from abacusnbody.data.compaso_halo_catalog import CompaSOHaloCatalog
>>> import abacusnbody.data.asdf
>>> Nth =" whatever int"
>>> abacusnbody.data.asdf.set_nthreads(8)
>>> import asdf.compression
>>> asdf.compression.validate("blsc")
Initial tests look good!
Done