Cannot set custom colors for categorical variables
Opened this issue · 9 comments
Hi!
I am testing napari-spatialdata on a public Nanostring Cosmx dataset (https://kero.hgc.jp/Breast_Cancer_Spatial.html), and I would like to set a custom color for each label of a specific categorical variable, i.e., the cell types obtained with an external tool. I tried to manually insert a dictionary in the uns slot of the anndata object used as 'table' in the spatialdata object, in the format {<label_name1>:<hex_code1>, <label_name2>:<hex_code2>, ..... }. However, after I load the spatialdata object on napari and I select the desired annotation in the 'observations' panel on the right, an error message pops up, and the annotation is not visualized at all. Here is the full error:
######################################################################################################
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
File ~/anaconda3/envs/SpatialData_prova/lib/python3.12/site-packages/napari_spatialdata/_widgets.py:65, in ListWidget.__init__.<locals>.<lambda>(item=<PyQt5.QtWidgets.QListWidgetItem object>)
62 self._unique = unique
63 self._viewer = viewer
---> 65 self.itemDoubleClicked.connect(lambda item: self._onAction((item.text(),)))
self = <napari_spatialdata._widgets.AListWidget object at 0x7f989cf30680>
item = <PyQt5.QtWidgets.QListWidgetItem object at 0x7f98dff132f0>
66 self.enterPressed.connect(self._onAction)
67 self.indexChanged.connect(self._onAction)
File ~/anaconda3/envs/SpatialData_prova/lib/python3.12/site-packages/napari_spatialdata/_widgets.py:142, in AListWidget._onAction(self=<napari_spatialdata._widgets.AListWidget object>, items=('InSituType_Simple',))
139 features["index"] = index
140 self.model.layer.features = features
--> 142 properties = self._get_points_properties(vec, key=item, layer=self.model.layer)
item = 'InSituType_Simple'
vec = cell_ID
1 BC cells
2 BC cells
3 Mix BC cells TAMs
4 Mix BC cells TAMs
5 BC cells
...
1850 Blood ECs
1851 Mast cells
1852 Myoepithelial cells
1853 Mix BC cells TAMs
1854 CAFs
Name: InSituType_Simple, Length: 1832, dtype: category
Categories (12, object): ['BC cells', 'Blood ECs', 'CAFs', 'DCs', ..., 'NK cells', 'Plasma cells',
'T cells', 'TAMs']
self = <napari_spatialdata._widgets.AListWidget object at 0x7f989cf30680>
143 self.model.color_by = "" if self.model.system_name is None else item
144 if isinstance(self.model.layer, (Points, Shapes)):
File ~/anaconda3/envs/SpatialData_prova/lib/python3.12/functools.py:946, in singledispatchmethod.__get__.<locals>._method(*args=(cell_ID
1 BC cells
2 ...ls',
'T cells', 'TAMs'],), **kwargs={'key': 'InSituType_Simple', 'layer': <Labels layer '11_labels'>})
944 def _method(*args, **kwargs):
945 method = self.dispatcher.dispatch(args[0].__class__)
--> 946 return method.__get__(obj, cls)(*args, **kwargs)
method = <function AListWidget._ at 0x7f98df9387c0>
obj = <napari_spatialdata._widgets.AListWidget object at 0x7f989cf30680>
cls = <class 'napari_spatialdata._widgets.AListWidget'>
args = (cell_ID
1 BC cells
2 BC cells
3 Mix BC cells TAMs
4 Mix BC cells TAMs
5 BC cells
...
1850 Blood ECs
1851 Mast cells
1852 Myoepithelial cells
1853 Mix BC cells TAMs
1854 CAFs
Name: InSituType_Simple, Length: 1832, dtype: category
Categories (12, object): ['BC cells', 'Blood ECs', 'CAFs', 'DCs', ..., 'NK cells', 'Plasma cells',
'T cells', 'TAMs'],)
kwargs = {'key': 'InSituType_Simple', 'layer': <Labels layer '11_labels' at 0x7f989d46d520>}
File ~/anaconda3/envs/SpatialData_prova/lib/python3.12/site-packages/napari_spatialdata/_widgets.py:248, in AListWidget._(self=<napari_spatialdata._widgets.AListWidget object>, vec=cell_ID
1 BC cells
2 ...ls',
'T cells', 'TAMs'], **kwargs={'key': 'InSituType_Simple'})
245 else:
246 merge_df = pd.merge(element_indices, vec, left_on="element_indices", right_index=True, how="left")
--> 248 merge_df["color"] = merge_df[[vec.name](http://vec.name/)].map(color_dict)
merge_df = element_indices InSituType_Simple
0 1 BC cells
1 2 BC cells
2 3 Mix BC cells TAMs
3 4 Mix BC cells TAMs
4 5 BC cells
... ... ...
1849 1850 Blood ECs
1850 1851 Mast cells
1851 1852 Myoepithelial cells
1852 1853 Mix BC cells TAMs
1853 1854 CAFs
[1854 rows x 2 columns]
color_dict = array([{'BC cells': '#ff4500', 'CAFs': '#f7e9c7', 'Mix BC cells TAMs': '#800080', 'Myoepithelial cells': '#daa520', 'Blood ECs': '#efd594', 'Mural cells': '#a77e18', 'T cells': '#001b00', 'Plasma cells': '#ff728a', 'TAMs': '#0000ff', 'DCs': '#000037', 'NK cells': '#003400', 'Mast cells': '#add8e6'}],
dtype=object)
vec = cell_ID
1 BC cells
2 BC cells
3 Mix BC cells TAMs
4 Mix BC cells TAMs
5 BC cells
...
1850 Blood ECs
1851 Mast cells
1852 Myoepithelial cells
1853 Mix BC cells TAMs
1854 CAFs
Name: InSituType_Simple, Length: 1832, dtype: category
Categories (12, object): ['BC cells', 'Blood ECs', 'CAFs', 'DCs', ..., 'NK cells', 'Plasma cells',
'T cells', 'TAMs']
249 if layer is not None and isinstance(layer, Labels):
250 index_color_mapping = dict(zip(merge_df["element_indices"], merge_df["color"]))
File ~/anaconda3/envs/SpatialData_prova/lib/python3.12/site-packages/pandas/core/series.py:4700, in Series.map(self=0 BC cells
1 B...ls',
'T cells', 'TAMs'], arg=array([{'BC cells': '#ff4500', 'CAFs': '#f7e9c7'...', 'Mast cells': '#add8e6'}],
dtype=object), na_action=None)
4620 def map(
4621 self,
4622 arg: Callable | Mapping | Series,
4623 na_action: Literal["ignore"] | None = None,
4624 ) -> Series:
4625 """
4626 Map values of Series according to an input mapping or function.
4627
(...)
4698 dtype: object
4699 """
-> 4700 new_values = self._map_values(arg, na_action=na_action)
arg = array([{'BC cells': '#ff4500', 'CAFs': '#f7e9c7', 'Mix BC cells TAMs': '#800080', 'Myoepithelial cells': '#daa520', 'Blood ECs': '#efd594', 'Mural cells': '#a77e18', 'T cells': '#001b00', 'Plasma cells': '#ff728a', 'TAMs': '#0000ff', 'DCs': '#000037', 'NK cells': '#003400', 'Mast cells': '#add8e6'}],
dtype=object)
self = 0 BC cells
1 BC cells
2 Mix BC cells TAMs
3 Mix BC cells TAMs
4 BC cells
...
1849 Blood ECs
1850 Mast cells
1851 Myoepithelial cells
1852 Mix BC cells TAMs
1853 CAFs
Name: InSituType_Simple, Length: 1854, dtype: category
Categories (12, object): ['BC cells', 'Blood ECs', 'CAFs', 'DCs', ..., 'NK cells', 'Plasma cells',
'T cells', 'TAMs']
na_action = None
4701 return self._constructor(new_values, index=self.index, copy=False).__finalize__(
4702 self, method="map"
4703 )
File ~/anaconda3/envs/SpatialData_prova/lib/python3.12/site-packages/pandas/core/base.py:919, in IndexOpsMixin._map_values(self=0 BC cells
1 B...ls',
'T cells', 'TAMs'], mapper=array([{'BC cells': '#ff4500', 'CAFs': '#f7e9c7'...', 'Mast cells': '#add8e6'}],
dtype=object), na_action=None, convert=True)
916 arr = self._values
918 if isinstance(arr, ExtensionArray):
--> 919 return arr.map(mapper, na_action=na_action)
arr = ['BC cells', 'BC cells', 'Mix BC cells TAMs', 'Mix BC cells TAMs', 'BC cells', ..., 'Blood ECs', 'Mast cells', 'Myoepithelial cells', 'Mix BC cells TAMs', 'CAFs']
Length: 1854
Categories (12, object): ['BC cells', 'Blood ECs', 'CAFs', 'DCs', ..., 'NK cells', 'Plasma cells',
'T cells', 'TAMs']
mapper = array([{'BC cells': '#ff4500', 'CAFs': '#f7e9c7', 'Mix BC cells TAMs': '#800080', 'Myoepithelial cells': '#daa520', 'Blood ECs': '#efd594', 'Mural cells': '#a77e18', 'T cells': '#001b00', 'Plasma cells': '#ff728a', 'TAMs': '#0000ff', 'DCs': '#000037', 'NK cells': '#003400', 'Mast cells': '#add8e6'}],
dtype=object)
na_action = None
921 return algorithms.map_array(arr, mapper, na_action=na_action, convert=convert)
File ~/anaconda3/envs/SpatialData_prova/lib/python3.12/site-packages/pandas/core/arrays/categorical.py:1555, in Categorical.map(self=['BC cells', 'BC cells', 'Mix BC cells TAMs', 'M...ls',
'T cells', 'TAMs'], mapper=array([{'BC cells': '#ff4500', 'CAFs': '#f7e9c7'...', 'Mast cells': '#add8e6'}],
dtype=object), na_action=None)
1545 warnings.warn(
1546 "The default value of 'ignore' for the `na_action` parameter in "
1547 "pandas.Categorical.map is deprecated and will be "
(...)
1551 stacklevel=find_stack_level(),
1552 )
1553 na_action = "ignore"
-> 1555 assert callable(mapper) or is_dict_like(mapper)
mapper = array([{'BC cells': '#ff4500', 'CAFs': '#f7e9c7', 'Mix BC cells TAMs': '#800080', 'Myoepithelial cells': '#daa520', 'Blood ECs': '#efd594', 'Mural cells': '#a77e18', 'T cells': '#001b00', 'Plasma cells': '#ff728a', 'TAMs': '#0000ff', 'DCs': '#000037', 'NK cells': '#003400', 'Mast cells': '#add8e6'}],
dtype=object)
1557 new_categories = self.categories.map(mapper)
1559 has_nans = np.any(self._codes == -1)
AssertionError:
######################################################################################################
I have no clue about this, especially considering that a student of mine previously succeeded in setting custom colors with an older version of napari-spatialdata. Currently I am using the 0.5.4.dev2+gf84b79b version of napari-spatialdata on napari 0.5.4 in python 3.12.3 on an ubuntu machine.
I am still a beginner with python, can anyone give me any advice on how to solve this?
Thank you in advance!
Hi, can you please try this code here: #242 (comment)?
There was a typo in the code I linked, please try this one on the latest napari-spatialdata main:
from spatialdata.datasets import blobs_annotating_element
from napari_spatialdata import Interactive
import spatialdata as sd
sdata = blobs_annotating_element("blobs_labels")
##
labels = sdata["blobs_labels"]
print(sd.get_element_instances(labels))
obs = sdata["table"].obs
obs["strings"] = ["A", "A", "B", "B", "C"]
##
sdata["table"].uns["strings_colors"] = {
"A": "#FF5733", # red
"B": "#3498DB", # blue
"C": "#2ECC71", # green
}
Interactive(sdata)It should give you this:
If you can't reproduce the bug with the code above please modify the code so that I can reproduce your bug with a small script. Thanks a lot!
Hi!
I lauched the code above, but it gave me different colors:
I did not incur in the bug though. I launched the code above on my windows laptop after I downloaded the latest main branch, so as soon as I get back to my ubuntu machine in my workplace I'll try to do the same and see if the bug occurs again. On the ubuntu machine I downloaded the development branch 0.5.4.dev2+gf84b79b because I needed a specific fix, but now I see that it has been merged into the main branch so I should have no problems.
Thank you for sending the details. It's a bit strange because yesterday another user tried the code and it seems that things worked: https://scverse.zulipchat.com/#narrow/stream/315824-spatial/topic/napari-spatialdata/near/476109235. I will look into this; meanwhile please send any other details that could help reproduce the bug.
It seems these bugs somewhat depend on the operating system. For example, I just tried on my windows laptop to visualize some custom annotations that were not included in the metadata csv file. I manually inserted them in the adata inside the spatialdata object as additional entries of the obs slot. This operation worked perfectly on linux, but on my windows it does not work, for some reason all cells are uniformly colored with a dull gray color, regardless of the annotation I select on the right. Even the annotations taken from the metadata csv are not colored. The same happened some time ago to my student on his windows machine, that's why I suspect these bugs might be different for each OS.
Please check the napari versions. We have introduced bugfixes in rendering colormaps. So using the old napari version may impact different visible colors
I confirm I am using the latest napari (0.5.4) on both windows and ubuntu.
Hi!
I have another update: I just downloaded the latest napari-spatialdata main branch on the linux machine in my workplace, and then I tried again to set customized colors through a dictionary in the uns slot of the anndata object. This time no error messages popped up, however napari ignored my instructions and colored the annotation with its default palette. Furthermore, I cannot visualize anymore the annotation label of each cell by hovering on it with the mouse cursor.
Hey @LucaCalderoni, @LucaMarconato
I just wanted to add weight to this. a legend of what colors mean in Napari interactive is really critical for smooth quality control.
Hover information in a little corner in my opinion is not enough.
I am not sure of the capabilities of napari in this sense, but from my naive point of view a little legend matching color to categorical in a small pop up or box would be good enough, setting our own { categoricals : color } dictionaries to set colors would be even better.