Problem deserializing IndexMap where keys are numeric
Closed this issue · 8 comments
I have the following type: IndexMap<u8, String>
Serializing it works fine:
{
"1": "76a04053bda0a88bda5177b86d7881f27eaeb5ce575901bb4",
"2": "6a15c3b29f559873cb481232fe5417f143d66fda9d32f9807",
"3": "299cd5743151ac4b2d63ae197f6cd879046f2856f44b9783c"
}
However, deserializing results in the following error:
Err
value: Error("invalid type: string "1", expected u8", line: 107, column: 4)'
It looks like there might need to be a bit more logic around deserializing numeric keys.
Does that work with a regular HashMap
? AFAICS we do the same thing, letting MapAccess::next_entry
deal with it.
This is a general serde issue, so for workarounds, I'd look around for the people who have already encountered this with other map types in serde.
We already provide serde_seq serialization for indexmap, which corresponds exactly to one of the common workarounds I find (serialize it as a sequence, not a map).
The following test passes for me - it deserializes the integer keys. Code for reproducing the reported issue is needed.
use serde_json;
#[test]
fn test_deser_map() {
use indexmap::IndexMap;
let m = serde_json::from_str::<IndexMap<u8, String>>(r#"
{
"1": "76a04053bda0a88bda5177b86d7881f27eaeb5ce575901bb4",
"2": "6a15c3b29f559873cb481232fe5417f143d66fda9d32f9807",
"3": "299cd5743151ac4b2d63ae197f6cd879046f2856f44b9783c"
}
"#);
println!("{:?}", m);
let m = m.unwrap();
assert!(m.contains_key(&1));
assert!(m.contains_key(&2));
assert!(m.contains_key(&3));
}
I have a reproducer, though it's not minimal in any way...
https://github.com/rust-malaysia/hyfetch/tree/3c355f54f21e8a7873583da34264f78a28965c65
Steps to reproduce:
- Run the Python version of
hyfetch
to generate the config. Make sure to choose a custom / "random" color alignment. Choose to save the config. cargo run -- --debug
You should get an error like this:
Error: Failed to read config
Caused by:
0: Failed to parse "/home/teohhanhui/.config/hyfetch.json"
1: color_align: invalid type: string "2", expected u8 at line 13 column 5
Here's a reproducer from that, even after switching to HashMap
:
use serde::Deserialize;
use std::collections::HashMap;
#[derive(Eq, PartialEq, Debug, Deserialize)]
#[serde(rename_all = "lowercase")]
enum ExternalEnum {
Custom {
colors: HashMap<u8, usize>,
},
}
#[derive(Eq, PartialEq, Debug, Deserialize)]
#[serde(rename_all = "lowercase", tag = "mode")]
enum InternalEnum {
Custom {
colors: HashMap<u8, usize>,
},
}
fn main() {
serde_json::from_str::<ExternalEnum>(
r#"{
"custom": {
"colors": {
"1": 123,
"2": 234
}
}
}"#,
)
.expect("ExternalEnum");
serde_json::from_str::<InternalEnum>(
r#"{
"mode": "custom",
"colors": {
"1": 123,
"2": 234
}
}"#,
)
.expect("InternalEnum");
}
ExternalEnum
passes, but InternalEnum
fails:
thread 'main' panicked at src/main.rs:42:6:
InternalEnum: Error("invalid type: string \"1\", expected u8", line: 0, column: 0)
In cargo expand
, they look pretty similar in how they call the HashMap
deserialization, so I'm not sure what's going on, but I think this is clearly a serde
issue.
It might be serde-rs/json#560 -> serde-rs/serde#1183
For anyone looking for a workaround:
pub(crate) mod index_map_serde {
use std::fmt;
use std::fmt::Display;
use std::hash::Hash;
use std::marker::PhantomData;
use std::str::FromStr;
use indexmap::IndexMap;
use serde::de::{self, DeserializeSeed, MapAccess, Visitor};
use serde::{Deserialize, Deserializer};
pub(crate) fn deserialize<'de, D, K, V>(deserializer: D) -> Result<IndexMap<K, V>, D::Error>
where
D: Deserializer<'de>,
K: Eq + Hash + FromStr,
K::Err: Display,
V: Deserialize<'de>,
{
struct KeySeed<K> {
k: PhantomData<K>,
}
impl<'de, K> DeserializeSeed<'de> for KeySeed<K>
where
K: FromStr,
K::Err: Display,
{
type Value = K;
fn deserialize<D>(self, deserializer: D) -> Result<Self::Value, D::Error>
where
D: Deserializer<'de>,
{
deserializer.deserialize_str(self)
}
}
impl<'de, K> Visitor<'de> for KeySeed<K>
where
K: FromStr,
K::Err: Display,
{
type Value = K;
fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
formatter.write_str("a string")
}
fn visit_str<E>(self, s: &str) -> Result<Self::Value, E>
where
E: de::Error,
{
K::from_str(s).map_err(de::Error::custom)
}
}
struct MapVisitor<K, V> {
k: PhantomData<K>,
v: PhantomData<V>,
}
impl<'de, K, V> Visitor<'de> for MapVisitor<K, V>
where
K: Eq + Hash + FromStr,
K::Err: Display,
V: Deserialize<'de>,
{
type Value = IndexMap<K, V>;
fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
formatter.write_str("a map")
}
fn visit_map<A>(self, mut input: A) -> Result<Self::Value, A::Error>
where
A: MapAccess<'de>,
{
let mut map = IndexMap::new();
while let Some((k, v)) =
input.next_entry_seed(KeySeed { k: PhantomData }, PhantomData)?
{
map.insert(k, v);
}
Ok(map)
}
}
deserializer.deserialize_map(MapVisitor {
k: PhantomData,
v: PhantomData,
})
}
}
(Lightly adapted from serde-rs/json#560 (comment))
serde_with
has support for indexmap
v1 and v2. You can then annotate the problematic fields with this:
#[serde_as(deserialize_as = "IndexMap<serde_with::DisplayFromStr, _>")]