Defer calls to proc_macro::TokenStream::extend

[dtolnay]

Fixes #198 by batching invocations of the impl Extend<TokenTree> for TokenStream impl.

For the benchmark in https://github.com/illicitonion/num_enum_regression_13 this reduces compile time from 118 seconds to 2.0 seconds on my machine.

Once the Extend performance is fixed in libproc_macro (rust-lang/rust#65080), this commit should be reverted.

[dtolnay]

N=	before PR	with PR
2	0.1ms	0.04ms
10	0.3ms	0.1ms
100	8.7ms	0.6ms
1000	610ms	6.2ms
2000	2400ms	11ms
3000	5600ms	18ms
4000	9500ms	23ms
5000	15000ms	28ms
6000	23000ms	34ms
7000	32000ms	37ms
8000	47000ms	41ms

extern crate proc_macro;

use proc_macro::TokenStream;
use quote::quote;
use std::time::Instant;

const N: u32 = 7000;

#[proc_macro]
pub fn bench(_input: TokenStream) -> TokenStream {
    let start = Instant::now();
    let counter = 0..N;
    let _ = quote! {
        #(
            #counter
            core::option::Option::None,
        )*
    };
    eprintln!("ELAPSED: {} millis", start.elapsed().as_millis());

    TokenStream::new()
}

[LunaBorowska]

Cannot all of this be put into a single vector?

[dtolnay]

That has different performance characteristics when converting between proc_macro::TokenStream and proc_macro2::TokenStream.