Reproducing first IA tutorial

Question

Reproducing first IA tutorial

Opened this issue 5 months ago · 11 comments

I'm running into issues reproducing hod_modeling_tutorial_with_alignments.ipynb.
I've been able to download and use the example halo catalog (simname='bolplanck', halo_finder='rockstar', redshift=0, version_name='halotools_v0p4'), but when running mask_bad_halocat(halocat) or model_instance.populate_mock(halocat, seed=seed), I get a few errors:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
File c:\Users\cmlam\anaconda3\Lib\site-packages\halotools\sim_manager\cached_halo_catalog.py:543, in CachedHaloCatalog.halo_table(self)
    [542](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/sim_manager/cached_halo_catalog.py:542) try:
--> [543](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/sim_manager/cached_halo_catalog.py:543)     return self._halo_table
    [544](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/sim_manager/cached_halo_catalog.py:544) except AttributeError:

AttributeError: 'CachedHaloCatalog' object has no attribute '_halo_table'

During handling of the above exception, another exception occurred:

AssertionError                            Traceback (most recent call last)
File c:\Users\cmlam\anaconda3\Lib\site-packages\halotools\utils\crossmatch.py:143, in crossmatch(x, y, skip_bounds_checking)
    [142](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/utils/crossmatch.py:142) assert len(set(y)) == len(y)
--> [143](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/utils/crossmatch.py:143) assert np.all(np.array(y).astype(int) == y)
    [144](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/utils/crossmatch.py:144) assert np.shape(y) == (len(y),)

AssertionError: 

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
Cell In[5], [line 19](vscode-notebook-cell:?execution_count=5&line=19)
     [17](vscode-notebook-cell:?execution_count=5&line=17) #seed=132358712
     [18](vscode-notebook-cell:?execution_count=5&line=18) seed=None
...
--> [147](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/utils/crossmatch.py:147)     raise ValueError(msg)
    [148](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/utils/crossmatch.py:148) try:
    [149](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/utils/crossmatch.py:149)     assert np.all(np.array(x).astype(int) == x)

ValueError: Input array y must be a 1d sequence of unique integers
Output is truncated. View as a [scrollable element](command:cellOutput.enableScrolling?ce174f84-da1c-4f2c-94d6-0aeeb8d14c4a) or open in a [text editor](command:workbench.action.openLargeOutput?ce174f84-da1c-4f2c-94d6-0aeeb8d14c4a). Adjust cell output [settings](command:workbench.action.openSettings?%5B%22%40tag%3AnotebookOutputLayout%22%5D)...

Answer 1 · 2024-11-25T18:50:29.000Z

@cmlamman Interesting. I have two initial idea for what might be causing this. Either the cached catalog hasn't been downloaded, or I've set up something in my environment that's important but have forgotten to document.

For the first point: after first installing halotools, you need to download a cached halo catalog. You only need to do this once (for any cached catalog you want to use). I'll outline that process here, but if this doesn't solve it, it might be that I've set something up in my environment that I haven't documented (in which case I'll very much want to figure out what that is so I can document it).

Have you been able to run any of the non-IA tutorial notebooks?

To download a cached halocat, run the following code:

from halotools.sim_manager import DownloadManager
dman = DownloadManager()
dman.download_processed_halo_table('bolplanck', 'rockstar', 0)

The full documentation is here if you need a deeper look.
https://halotools.readthedocs.io/en/latest/quickstart_and_tutorials/quickstart_guides/working_with_halotools_provided_catalogs.html
If this doesn't work, please let me know and I'll track down the issue.

Answer 2 · 2024-11-25T19:00:24.000Z

Yes, I downloaded the cached halo catalog (simname='bolplanck', halo_finder='rockstar', redshift=0, version_name='halotools_v0p4') that you used in the tutorial and was able to read it into the notebook. I was also able to run other tutorials.

Answer 3 · 2024-11-25T21:13:46.000Z

I made a fresh virtual environment, but still couldn't recreate this error. Can you tell me the following:

Your version number of halotools (astropy, scipy, and numpy too just in case).
It looks like the error message you sent was truncated. Other than the AttributeError: 'CachedHaloCatalog' object has no attribute '_halo_table' message, can you paste the full message, or the last couple blocks of the untruncated message?
And this this is happening on the third cell on the mask_bad_halocat(halocat) line?
Anything else you think could help me recreate this error.

Of the two errors I do see in the trace you posted:

I have seen AttributeError: 'CachedHaloCatalog' object has no attribute '_halo_table' before when the cached halocat didn't properly initialize.
The error ValueError: Input array y must be a 1d sequence of unique integers occurs when crossmatch is called with an array with duplicate values or non integers. The fact that we see this error on a notebook where you haven't made any changes (especially if it has been triggered by mask_bad_halocat) does feel odd to me. I hope the full trace can clear it up (if there's more to that error trace)

Answer 4 · 2024-11-27T20:46:59.000Z

Thanks Nick. If the bug is on my end, it seems most likely due to an issue with how I created the CashedHaloCatalog. I made it by running this command: python download_additional_halocat.py bolplanck rockstar halotools_v0p4 0 . It reads in when I use halocat = CachedHaloCatalog() and halocat.halo_table looks right to me, at least - it is a table with values. However it doesn't have a column for "halo_mvir_host_halo" (only "halo_mvir_firstacc") which seems important.

mask_bad_halocat(halocat) in the third cell produces the error, which I pasted in full below.
I found a weird way to solve this error: running halocat.halo_table in a separate cell twice before running mask_bad_halocat(halocat). However, then I get a new error when running the fifth cell in your notebook, which I also included below.

Here's the versions I'm using:

halotools: 0.9.1
astropy: 6.1.3
scipy: 1.14.1
1.26.4

Error Message from 3rd cell in tutorial:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
File c:\Users\cmlam\anaconda3\Lib\site-packages\halotools\sim_manager\cached_halo_catalog.py:543, in CachedHaloCatalog.halo_table(self)
    [542](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/sim_manager/cached_halo_catalog.py:542) try:
--> [543](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/sim_manager/cached_halo_catalog.py:543)     return self._halo_table
    [544](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/sim_manager/cached_halo_catalog.py:544) except AttributeError:

AttributeError: 'CachedHaloCatalog' object has no attribute '_halo_table'

During handling of the above exception, another exception occurred:

AssertionError                            Traceback (most recent call last)
File c:\Users\cmlam\anaconda3\Lib\site-packages\halotools\utils\crossmatch.py:143, in crossmatch(x, y, skip_bounds_checking)
    [142](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/utils/crossmatch.py:142) assert len(set(y)) == len(y)
--> [143](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/utils/crossmatch.py:143) assert np.all(np.array(y).astype(int) == y)
    [144](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/utils/crossmatch.py:144) assert np.shape(y) == (len(y),)

AssertionError: 

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
Cell In[3], [line 2](vscode-notebook-cell:?execution_count=3&line=2)
      [1](vscode-notebook-cell:?execution_count=3&line=1) halocat = CachedHaloCatalog(simname='bolplanck', halo_finder='rockstar', redshift=0, version_name='halotools_v0p4')
----> [2](vscode-notebook-cell:?execution_count=3&line=2) mask_bad_halocat(halocat)

Cell In[2], [line 3](vscode-notebook-cell:?execution_count=2&line=3)
      [2](vscode-notebook-cell:?execution_count=2&line=2) def mask_bad_halocat(halocat):
----> [3](vscode-notebook-cell:?execution_count=2&line=3)     halocat.halo_table
      [4](vscode-notebook-cell:?execution_count=2&line=4)     bad_mask = (halocat.halo_table["halo_axisA_x"] == 0) & (halocat.halo_table["halo_axisA_y"] == 0) & (halocat.halo_table["halo_axisA_z"] == 0)
      [5](vscode-notebook-cell:?execution_count=2&line=5)     halocat._halo_table = halocat.halo_table[ ~bad_mask ]

File c:\Users\cmlam\anaconda3\Lib\site-packages\halotools\sim_manager\cached_halo_catalog.py:547, in CachedHaloCatalog.halo_table(self)
    [545](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/sim_manager/cached_halo_catalog.py:545) if self.log_entry.safe_for_cache is True:
    [546](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/sim_manager/cached_halo_catalog.py:546)     self._halo_table = Table.read(_passively_decode_string(self.fname), path='data')
--> [547](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/sim_manager/cached_halo_catalog.py:547)     self._add_new_derived_columns(self._halo_table)
    [548](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/sim_manager/cached_halo_catalog.py:548)     return self._halo_table
    [549](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/sim_manager/cached_halo_catalog.py:549) else:

File c:\Users\cmlam\anaconda3\Lib\site-packages\halotools\sim_manager\cached_halo_catalog.py:557, in CachedHaloCatalog._add_new_derived_columns(self, t)
    [554](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/sim_manager/cached_halo_catalog.py:554)     add_halo_hostid(t)
    [556](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/sim_manager/cached_halo_catalog.py:556) if 'halo_mvir_host_halo' not in list(t.keys()):
--> [557](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/sim_manager/cached_halo_catalog.py:557)     broadcast_host_halo_property(t, 'halo_mvir')

File c:\Users\cmlam\anaconda3\Lib\site-packages\halotools\utils\value_added_halo_table_functions.py:83, in broadcast_host_halo_property(table, halo_property_key, delete_possibly_existing_column)
     [78](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/utils/value_added_halo_table_functions.py:78) elif (new_colname in list(table.keys())) & (
     [79](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/utils/value_added_halo_table_functions.py:79)     delete_possibly_existing_column is True
     [80](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/utils/value_added_halo_table_functions.py:80) ):
     [81](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/utils/value_added_halo_table_functions.py:81)     del table[new_colname]
---> [83](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/utils/value_added_halo_table_functions.py:83) idx_halos, idx_hosts = crossmatch(table["halo_hostid"].data, table["halo_id"].data)
     [84](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/utils/value_added_halo_table_functions.py:84) table[new_colname] = np.zeros(len(table), dtype=table[halo_property_key].dtype)
     [85](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/utils/value_added_halo_table_functions.py:85) table[new_colname][idx_halos] = table[halo_property_key][idx_hosts]

File c:\Users\cmlam\anaconda3\Lib\site-packages\halotools\utils\crossmatch.py:147, in crossmatch(x, y, skip_bounds_checking)
    [145](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/utils/crossmatch.py:145) except:
    [146](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/utils/crossmatch.py:146)     msg = "Input array y must be a 1d sequence of unique integers"
--> [147](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/utils/crossmatch.py:147)     raise ValueError(msg)
    [148](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/utils/crossmatch.py:148) try:
    [149](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/utils/crossmatch.py:149)     assert np.all(np.array(x).astype(int) == x)

ValueError: Input array y must be a 1d sequence of unique integers

Error message from 5th cell


---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[11], [line 19](vscode-notebook-cell:?execution_count=11&line=19)
     [17](vscode-notebook-cell:?execution_count=11&line=17) #seed=132358712
     [18](vscode-notebook-cell:?execution_count=11&line=18) seed=None
---> [19](vscode-notebook-cell:?execution_count=11&line=19) model_instance.populate_mock(halocat, seed=seed)

File c:\Users\cmlam\anaconda3\Lib\site-packages\halotools\empirical_models\factories\hod_model_factory.py:1209, in HodModelFactory.populate_mock(self, halocat, **kwargs)
   [1090](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/hod_model_factory.py:1090) def populate_mock(self, halocat, **kwargs):
   [1091](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/hod_model_factory.py:1091)     """
   [1092](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/hod_model_factory.py:1092)     Method used to populate a simulation
   [1093](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/hod_model_factory.py:1093)     with a Monte Carlo realization of a model.
   (...)
   [1207](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/hod_model_factory.py:1207) 
   [1208](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/hod_model_factory.py:1208)     """
-> [1209](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/hod_model_factory.py:1209)     ModelFactory.populate_mock(self, halocat, **kwargs)

File c:\Users\cmlam\anaconda3\Lib\site-packages\halotools\empirical_models\factories\model_factory_template.py:244, in ModelFactory.populate_mock(self, halocat, Num_ptcl_requirement, **kwargs)
    [242](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/model_factory_template.py:242) except KeyError:
    [243](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/model_factory_template.py:243)     pass
--> [244](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/model_factory_template.py:244) self.mock = self.mock_factory(**mock_factory_init_args)
    [246](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/model_factory_template.py:246) additional_potential_kwargs = (
    [247](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/model_factory_template.py:247)     "masking_function",
    [248](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/model_factory_template.py:248)     "_testing_mode",
    [249](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/model_factory_template.py:249)     "enforce_PBC",
    [250](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/model_factory_template.py:250)     "seed",
    [251](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/model_factory_template.py:251) )
    [252](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/model_factory_template.py:252) mockpop_keys = set(additional_potential_kwargs) & set(kwargs)

File c:\Users\cmlam\anaconda3\Lib\site-packages\halotools\empirical_models\factories\hod_mock_factory.py:101, in HodMockFactory.__init__(self, Num_ptcl_requirement, halo_mass_column_key, **kwargs)
     [98](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/hod_mock_factory.py:98) self.Num_ptcl_requirement = Num_ptcl_requirement
     [99](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/hod_mock_factory.py:99) self.halo_mass_column_key = halo_mass_column_key
--> [101](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/hod_mock_factory.py:101) self.preprocess_halo_catalog(halocat)

File c:\Users\cmlam\anaconda3\Lib\site-packages\halotools\empirical_models\factories\hod_mock_factory.py:172, in HodMockFactory.preprocess_halo_catalog(self, halocat)
    [170](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/hod_mock_factory.py:170) try:
    [171](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/hod_mock_factory.py:171)     f = getattr(component_model, "preprocess_subhalo_table")
--> [172](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/hod_mock_factory.py:172)     halo_table, self.subhalo_table = f(halo_table, subhalo_table)
    [173](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/hod_mock_factory.py:173) except AttributeError:
    [174](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/factories/hod_mock_factory.py:174)     pass

File c:\Users\cmlam\anaconda3\Lib\site-packages\halotools\empirical_models\phase_space_models\subhalo_based_models\subhalo_phase_space.py:274, in SubhaloPhaseSpace.preprocess_subhalo_table(self, host_halo_table, subhalo_table)
    [236](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/phase_space_models/subhalo_based_models/subhalo_phase_space.py:236) def preprocess_subhalo_table(self, host_halo_table, subhalo_table):
    [237](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/phase_space_models/subhalo_based_models/subhalo_phase_space.py:237)     """ Method makes cuts and organizes the memory layout of the input
    [238](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/phase_space_models/subhalo_based_models/subhalo_phase_space.py:238)     ``subhalo_table`` and ``host_halo_table``.
    [239](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/phase_space_models/subhalo_based_models/subhalo_phase_space.py:239) 
   (...)
    [272](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/phase_space_models/subhalo_based_models/subhalo_phase_space.py:272)     and are thrown out to make bookkeeping simpler.
    [273](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/phase_space_models/subhalo_based_models/subhalo_phase_space.py:273)     """
--> [274](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/phase_space_models/subhalo_based_models/subhalo_phase_space.py:274)     host_halo_table.sort(self.binning_key)
    [275](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/phase_space_models/subhalo_based_models/subhalo_phase_space.py:275)     host_halo_table['_subhalo_inheritance_id'] = np.arange(len(host_halo_table))
    [277](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/phase_space_models/subhalo_based_models/subhalo_phase_space.py:277)     idxA, idxB = crossmatch(subhalo_table['halo_hostid'].data,
    [278](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/halotools/empirical_models/phase_space_models/subhalo_based_models/subhalo_phase_space.py:278)         host_halo_table['halo_id'].data)

File c:\Users\cmlam\anaconda3\Lib\site-packages\astropy\table\table.py:3648, in Table.sort(self, keys, kind, reverse)
   [3645](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/astropy/table/table.py:3645) if isinstance(keys, str):
   [3646](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/astropy/table/table.py:3646)     keys = [keys]
-> [3648](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/astropy/table/table.py:3648) indexes = self.argsort(keys, kind=kind, reverse=reverse)
   [3650](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/astropy/table/table.py:3650) with self.index_mode("freeze"):
   [3651](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/astropy/table/table.py:3651)     for col in self.columns.values():
   [3652](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/astropy/table/table.py:3652)         # Make a new sorted column.  This requires that take() also copies
   [3653](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/astropy/table/table.py:3653)         # relevant info attributes for mixin columns.

File c:\Users\cmlam\anaconda3\Lib\site-packages\astropy\table\table.py:3560, in Table.argsort(self, keys, kind, reverse)
   [3558](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/astropy/table/table.py:3558) # use index sorted order if possible
   [3559](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/astropy/table/table.py:3559) if keys is not None:
-> [3560](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/astropy/table/table.py:3560)     index = get_index(self, names=keys)
   [3561](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/astropy/table/table.py:3561)     if index is not None:
   [3562](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/astropy/table/table.py:3562)         idx = np.asarray(index.sorted_data())

File c:\Users\cmlam\anaconda3\Lib\site-packages\astropy\table\index.py:647, in get_index(table, table_copy, names)
    [644](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/astropy/table/index.py:644)     names = set(table_copy.colnames)
    [646](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/astropy/table/index.py:646) if not names <= set(table.colnames):
--> [647](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/astropy/table/index.py:647)     raise ValueError(f"{names} is not a subset of table columns")
    [649](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/astropy/table/index.py:649) for name in names:
    [650](file:///C:/Users/cmlam/anaconda3/Lib/site-packages/astropy/table/index.py:650)     for index in table[name].info.indices:

ValueError: {'halo_mvir_host_halo'} is not a subset of table columns

Answer 5 · 2024-12-02T15:35:17.000Z

@cmlamman Ok, I'm still looking into this but I wanted to give you an (unfortunately not very helpful) update on what I've seen so far. The versions of the packages you're using are the exact same as what I'm running, so no issues there. It ran when I made a fresh virtual environment, but to make sure, I tried on a fresh computer.

On this fresh computer, there was no halotools installation (now or ever), so I installed halotools and tried to run the notebook (nothing beyond pip3 install halotools in this environment, as well as downgrading the astropy version to what's given above as there seem to be issues on later versions). I expected (and got) the error that I hadn't yet downloaded a cached halocat. So I ran the command you gave earlier, namely python3 download_additional_halocat.py bolplanck rockstar halotools_v0p4 0 to download the proper halocat. Then I restarted the notebook kernel and ran the whole thing, but it went smoothly and I wasn't able to recreate the errors you're getting.

You could try uninstalling halotools, then reinstalling (or even simply trying to freshly download the halocat), but let's check a few more things before that.
What's your OS?
What version of python are you running?
Do you have cython installed and what version?
Do you have gcc installed, and what version?

For reference, here's what I'm running (using the fresh computer specs):
Ubuntu 24.04
Python 3.12.3
Cython 3.0.11
gcc 13.2.0

cython and gcc may not be related to the problem at all (it doesn't sound like they should be), but I have had issues related to errors compiling various pyx files and the like. Even in that case, I believe anything higher than gcc 4 worked.

I'm sorry this has been such a struggle. I really want to figure out what's going on so I can document this problem better.

Answer 6 · 2024-12-18T16:44:14.000Z

FWIW, I was able to install things and run the IA tutorial notebooks without any issues.

However, I did notice another minor thing to fix, which is that installing the master branch failed when doing it into the suggested virtualenv installation setup, which installs a python version <3.10, while the master branch expects python>=3.10.

Answer 7 · 2024-12-18T16:54:56.000Z

@matroxel this is in reference to the listen Python dependency at https://halotools.readthedocs.io/en/latest/install.html correct? Thank you for pointing that out. That's an important point I had noticed but hadn't realized I didn't update documentation for. I will get on fixing that. Did you see it listed incorrectly anywhere else?

Answer 8 · 2024-12-18T16:57:17.000Z

Also here: https://github.com/nvanalfen/halotools/blob/master/docs/installing_halotools_with_virtualenv.rst

Answer 9 · 2024-12-18T17:05:13.000Z

@matroxel Thanks! I've fixed those and submitted a PR to the Astropy master branch. Once that's merged, I'll update my paper branch to reflect that just for consistency.

Answer 10 · 2024-12-18T18:36:49.000Z

@matroxel just an update, I changed the required version in the documentation to 3.11 or later as halotools is planning to drop support for 3.10 soon. 3.10 still works for the version you have now (and in this paper), but we're updating the installation docs to suggest >=3.11 now. Thanks for pointing this out!

Answer 11 · 2024-12-22T21:00:59.000Z

@nvanalfen I spent more time trying various instillation methods and the issue is with my OS (Windows). I had no problems with installation and the tutorials when I tried it on Linux.
It seems they are known issues with halotools (astropy#1055, astropy#1071), not your contribution specifically so I added relevant comments to those existing issues and will finish up the review as is on my linux machine.