Writing Procedural Macros with the syn Crate (Part 2: Code Generation)

This is a continuation of part.1.

Generating the code

Now that we can retrieve the necessary data, let's write the code generation part. First, here is the rough outline.

#[proc_macro]
pub fn shiika_method_ref(input: TokenStream) -> TokenStream {
    let _spec = parse_macro_input!(input as ShiikaMethodRef);
    let mangled_name = "a";
    let function_definition = "b";
    let parameters = "c";
    let return_type = "d";
    let wrapper_name = "e";
    let args = "f";
    let gen = quote! {
        extern "C" {
            #[allow(improper_ctypes)]
            fn #mangled_name(#parameters) -> #return_type;
        }
        pub fn #wrapper_name(#parameters) -> #return_type {
            unsafe { #mangled_name(#args) }
        }
    };
    gen.into()
}

The value parts are filled with temporary placeholders, so running this will result in an error like this:

   Compiling skc_rustlib v0.1.0 (/Users/yhara/Dropbox/proj/shiika/lib/skc_rustlib)
error: expected item after attributes
  --> lib/skc_rustlib/src/sk_methods.rs:15:1
   |
15 | / shiika_method_ref!(
16 | |     "Meta:Class#new",
17 | |     fn(receiver: *const u8) -> SkClass,
18 | |     "meta_class_new"
19 | | );
   | |_^
   |
   = note: this error originates in the macro `shiika_method_ref` (in Nightly builds, run with -Z macro-backtrace for more info)

It is a bit inconvenient that it doesn't tell you how it was expanded when it fails (-Z macro-backtrace only shows the call history), but since the expansion results of a successful case can be viewed with cargo-expand, I'll do my best using that.

String literal -> Identifier

First, we need a way to create a Rust identifier from a String like "Meta_Class_new". This is because trying to embed the String directly would result in invalid Rust code like:

  fn "Meta_Class_new"(...)

Since there appears to be a type called syn::Ident, I'll include proc_macro2 by referring to the sample code.

#[proc_macro]
pub fn shiika_method_ref(input: TokenStream) -> TokenStream {
    let spec = parse_macro_input!(input as ShiikaMethodRef);
    let mangled_name = Ident::new(
        &mangle_method(&spec.method_name.value()),
        Span::call_site());
    let parameters = &spec.parameters;
    let return_type = &spec.ret_ty;
    let wrapper_name = Ident::new(
        &spec.rust_func_name.value(),
        Span::call_site());
    let args = "todo";
    let gen = quote! {
        extern "C" {
            #[allow(improper_ctypes)]
            fn #mangled_name(#parameters) -> #return_type;
        }
        pub fn #wrapper_name(#parameters) -> #return_type {
            unsafe { #mangled_name(#args) }
        }
    };
    gen.into()
}

Running cargo expand with this definition:

/Users/yhara/proj/shiika/lib/skc_rustlib % cargo expand sk_methods
   Compiling shiika_ffi_macro v0.1.0 (/Users/yhara/Dropbox/proj/shiika/lib/shiika_ffi_macro)
    Checking skc_rustlib v0.1.0 (/Users/yhara/Dropbox/proj/shiika/lib/skc_rustlib)
    Finished dev [unoptimized + debuginfo] target(s) in 0.82s

mod sk_methods {
    (omitted)
    extern "C" {
        #[allow(improper_ctypes)]
        fn Meta_Class_new(receiver: *const u8) -> SkClass;
    }
    pub fn meta_class_new(receiver: *const u8) -> SkClass {
        unsafe { Meta_Class_new("todo") }
    }
}

It was output as above. It looks quite good.

Making it a method

This is fine as is, but if we make the part that creates the Ident into a method like this:

impl ShiikaMethodRef {
    /// Returns mangled llvm func name (eg. `Meta_Class_new`)
    pub fn mangled_name(&self) -> Ident {
        Ident::new(
            &mangle_method(&self.method_name.value()),
            Span::call_site())
    }

    /// Returns user-specified func name. (eg. `meta_class_new`)
    pub fn wrapper_name(&self) -> Ident {
        Ident::new(
            &self.rust_func_name.value(),
            Span::call_site())
    }
}

The caller side becomes much cleaner like this:

#[proc_macro]
pub fn shiika_method_ref(input: TokenStream) -> TokenStream {
    let spec = parse_macro_input!(input as ShiikaMethodRef);
    let mangled_name = spec.mangled_name();
    let parameters = &spec.parameters;
    let return_type = &spec.ret_ty;
    let wrapper_name = spec.wrapper_name();
    let args = "todo";
    let gen = quote! {
        extern "C" {
            #[allow(improper_ctypes)]
            fn #mangled_name(#parameters) -> #return_type;
        }
        pub fn #wrapper_name(#parameters) -> #return_type {
            unsafe { #mangled_name(#args) }
        }
    };
    gen.into()
}

Generating the arguments part

The last part remaining is generating the part that forwards the arguments. What we want to do here is to extract only the variable name part from:

a: A, b: B

and create a sequence like:

a, b

The former was of type Punctuated<Field, Token![,]>, and looking at the Punctuated reference shows that you can get an iterator using iter. So, it seems possible to create a Vec<Ident> for now by mapping and getting the ident from Field.

Embedding a Vec

However, a Vec<Ident> cannot be embedded directly into Rust source as it is. This is because it is unclear what the delimiter should be. In this case, we want to list them separated by commas, so... I have a feeling that Punctuated can be used. It worked well by adding a method like the following to ShiikaMethodRef.

    /// Returns list of parameters for forwarding function call (eg. `a, b, c`)
    pub fn forwaring_args(&self) -> Punctuated<Ident, Token![,]> {
        self.parameters.iter().map(|field| {
            field.ident.clone().expect("Field::ident is None. why?")
        }).collect()
    }

Finished!

So, here is the finished product. Good job.

https://github.com/shiika-lang/shiika/pull/403