dtolnay/paste

Parser error with `:lifetime` param in expanded code

danielhenrymantilla opened this issue · 2 comments

Minimal repro:

macro_rules! m {(
    $lt:lifetime
) => (
    ::paste::item! {
        impl<$lt> () {}
    }
)}

m! { 'lt }

leads to a parser error saying that:

  • 1.45.0-nightly (90931d9b3 2020-04-28)

    error: lifetime in trait object type must be followed by `+`
    
  • 1.43.0 (3532cf738 2020-03-17)

    error: expected type, found `'lt`
    
  • A run with cargo bisect seems to indicate this bug has always been present.


EDIT: After some testing, here is a minimal proc-macro repro (that is, if we replace ::paste::item! by foo!, the same parsing error is triggered):

#[proc_macro] pub
fn foo (it: TokenStream) -> TokenStream
{
    it  .into_iter()
        .map(|tt| tt) // without this line, there is no error (I imagine due to some specialization) 
        .collect::<TokenStream>()
      //.to_string().parse().unwrap() /* workaround! */
}

This minimal repro suggests that this is not really an issue with paste, but with some proc-macro :lifetime capture interaction... Since you know the internals of a TokenStream better, @dtolnay, I'll let you investigate it further and post an appropriate issue on rust-lang/rust.

For those needing a workaround, "flattening" / unwrapping the Grouped :lifetime into the two tokens that compose it seems to solve the issue. That is, wrapping the returned TokenStream within the following function avoids the compilation error:

fn fix_lifetime_capture_hack (ts: TokenStream)
  -> TokenStream
{
    use TokenTree as TT;
    ts.into_iter().map(|tt| match tt {
        | TT::Group(group) if true
            && group.delimiter() == Delimiter::None
            && matches!(
                group.stream().into_iter().next(),
                Some(TT::Punct(ref punct)) if punct.as_char() == '\''
            )
        => group.stream(),

        | TT::Group(ref group) => {
            let span = tt.span();
            let mut tt = TT::Group(Group::new(
                group.delimiter(),
                fix_lifetime_capture_hack(group.stream()),
            ));
            tt.set_span(span);
            iter::once(tt).collect()
        },

        | _ => iter::once(tt).collect(),
    }).collect()
}

This seems like a compiler bug, but I've released 0.1.12 with a workaround.