Avoid the proc_macro crate being dynamically linked.

[mystor]

Currently the proc_macro crate is dynamically linked. This is so that it can be provided by the compiler when writing procedural macros and has access to unstable compiler internals.

Unfortunately, it's possible for programs which are not compiler plugins to extern crate proc_macro. This happens right now if you use any crate which wants to provide conversions for its types from proc_macro::TokenStream. This can potentially happen deep in your dependencies without it being obvious. This means that suddenly your program will no longer work when run on systems which either don't have the same version of the proc_macro dylib installed, or which don't have it installed at all.

Ideally, for the best experience, we'd make the proc_macro crate a small statically linked shim, which when loaded into the compiler, uses dlsym(3) (and the windows equivalent) to actually gain access to compiler internals. When not loaded into the compiler, it would panic when used like it does today.

This way crates which depend on proc_macro to provide things like From impls won't be accidental footguns which break end user applications. IIRC @dtolnay has run into this in the past with the proc_macro_hack crate, and the current master version of syn has this problem as well.

cc @alexcrichton who I talked to about this a bit at Rust Belt Rust.

[sfackler]

I've wished for a while that proc macros were binaries the compiler booted up and talked to rather than dynamic libraries. This might be a good reason to make that change - libproc_macro would just be a normal rlib that defines the IPC protocol.

[mystor]

Yes, that's another approach which I should've mentioned in the first comment. The reason why I suggested the dlsym approach was that, to my understanding, it would be a much larger re-architecture of the system to add IPC.

I don't know what tools we have access to from within rustc for IPC, and what the best approach would be to implement it, but if we decide that that is the approach we want to take I could look into it.

[retep998]

I've wished for a while that proc macros were binaries the compiler booted up and talked to rather than dynamic libraries.

I am strongly in favor of anything that moves us away from using the awful dylib crate type.

[Rantanen]

Just got hit by this today by having an indirect syn dependency in a cdylib library.

I'm partial to sfackler's suggestion of turning proc_macros into binaries with IPC instead of dynamic libraries. Has there been any decision on whether this should be the future goal - or any other decision for a solution to this current issue?

I might be interested in working on this; but without a concrete plan it's difficult to say what "working on this" would mean.

[alexcrichton]

FWIW this is being done in #49219 (cc @eddyb)

[steveklabnik]

Triage: that PR was merged, did it end up fixing this, or not?

[eddyb]

I believe this was fixed. Proc macro crates are a combination of dylib and cdylibs, but libproc_macro itself is a statically linked rlib.