bluss/maplit

Fail to compile when using large map

SunDoge opened this issue · 8 comments

Hi.

I try to convert a python dict to rust HashMap using hashmap! but it fails to compile this piece of code.

https://gist.github.com/SunDoge/5dca199bd2c962688258d951cb9423db

It works when reducing data. I've reviewed your macro source code but still don't know what cause the problem.

I used to use json as bridge but it didn't work when converting dict with tuple as key. That's why I changed to maplit. Or is there another way to solve this problem?

bluss commented

What error does the compiler output?

No error output, it is just stuck there, so does rls.

3631a831-9fb6-44e3-9d0f-938f6ab3aad1

At first I thought it was caused by @count, so I wrote a new macro without specifying capacity.

macro_rules! hashmap {
    ( $( $k:expr => $v:expr ),* ) => {{
	let mut m = HashMap::new();
        $(
		m.insert($k, $v);
        )*
        m
    }};
}

And it got stuck when compiling, too...

Finally, I use serde_pickle instead to read data from python and initialize small map using macro.

bluss commented

Thanks for filling in with more information!

This seems like a rustc bug -- it's related to type inference or assinging types to all those literals.

Compiles in 10 seconds (debug build) if one uses explicit type literals (f64 here). Code: https://gist.github.com/74908850aa4b3c38b95b9d16dca7c7df

I think this should be filed against rustc.

Oh I see. I should have realize that compiler may not be able to get enough information when using macro. I think it actually not a bug. Thank you very much.

bluss commented

The long compile time definitely is a bug.

bluss commented

maplit can mitigate this in a few ways

  • Documentation for how to handle big maps
  • Key/value remapping features would make it easy for the user to use explicit types with a one line change, like in #13 (comment)
bluss commented

When #15 merges, this issue is considered fixed by maplit. We have done our part to make it easy for users to fix the compile time.

To underline, the long compile time itself is a rustc bug.

With #15 the values can be given explicit types "easily" using a conversion function with an explicit type like this:

fn id<T>(t: T) -> T { t }
let map = convert_args!(keys=id::<char>, values=id::<f64>, hashmap!(
    '\u{4e00}'=> -3.6544978750449433,
    '\u{4e01}'=> -8.125041941842026,
    '\u{4e03}'=> -7.817392401429855,
    '\u{4e07}'=> -6.3096425804013165,
    ...
));
bluss commented

Fixed (maplit part) by #15.