Pre-RFC: Allow Custom Syntax Parsing for Attribute Macro Inputs

language design
1

​​Problem:​​ Currently, attribute macros require their input to be valid Rust syntax before macro expansion. This causes false errors in tooling (rustc, rust-analyzer) when the macro expects a custom DSL. For example:

#[proc_macro_attribute]
pub fn dummy_attr(_attr: TokenStream, input: TokenStream) -> TokenStream {
   //do something
}

fn main() {
    #[dummy_attr]
    {key:value} // now this give error
    #[dummy_attr]
    {key::value} //but this is allowed
}

Tools like rust-analyzer flag {key:value} as invalid Rust syntax during analysis, disrupting developer workflows even though the macro would accept it. Function-like macros don’t have this limitation because they participate in token-tree expansion.

​​Proposal:​​ Enable attribute macros to declare a custom grammar for their input body. This would instruct tooling to use:

  1. Rust syntax by default (backward-compatible behavior)
  2. A macro-specified grammar when explicitly declared

​​Suggested Implementation:​​ Add a new attribute to mark macros with custom syntax:

#[proc_macro_attribute]
#[custom_grammar = "grammar_identifier"] // New annotation
pub fn my_macro(_attr: TokenStream, input: TokenStream) -> TokenStream { /* ... */ }

Where grammar_identifier could resolve to:

  • A ​​built-in grammar​​ (e.g., json, toml, ron) supported natively by tooling
  • A ​​user-defined grammar​​ via:
    • ​​Syn-powered parser​​: Share logic between the macro and tooling
    • ​​TextMate grammar​​: Leverage existing editor syntax definitions
    • ​​WASM-compiled parser​​: Portable syntax engine

​​Workflow Integration:​​

  1. ​​Compiler​​: Ignore custom syntax validation in early passes (treat input as token stream)
  2. ​​rust-analyzer/LSP​​:
  • Use custom parsers for syntax highlighting/validation
  • Provide diagnostics based on macro-specific rules
  • Use the macro’s grammar for auto-completion in attribute bodies

​​Motivation:​​

  • Eliminate erroneous errors in IDEs for valid macro input
  • Enable rich tooling support for domain-specific syntax
  • Unify compiler/tool behavior without changing macro expansion rules
  • Backward-compatible via opt-in syntax declaration

​​Challenges:​​

  1. Standardizing Grammar Definitions: Need consensus on cross-tool grammar format
  2. Parser Sandboxing: Security implications for user-provided parsers
  3. Performance: On-demand grammar loading to avoid tooling slowdown

​​Alternatives Considered:​​

  1. Do nothing: Forces macros to accept Rust syntax or tolerate tooling noise
  2. IDE-only solutions: Fragmented support without compiler awareness
  3. Full syntax plugins: More complex and conflicts with stable Rust

​​Open Questions:​​

  • Should grammars be definable inline in proc macros?
  • How to handle versioning for user-defined grammars?
  • Preferred grammar format priority (Syn, TextMate, WASM, etc.)?

​​Why this approach?​​

  1. ​​Clear problem statement​​ with concrete code example
  2. ​​Backward compatible​​ via opt-in annotation
  3. ​​Tooling-focused​​ while preserving current compiler behavior
  4. ​​Practical deployment path​​ using existing ecosystem (Syn/TextMate)
  5. Exposes ​​tradeoffs and alternatives​​ transparently

This structure aligns with RFC conventions by separating motivation, design, and open questions. The grammar format flexibility increases adoption potential without mandating a single solution.

2

We have to parse source files before we are able to macro-expand them, or even know which proc macro a macro invocation refers to. Having parsing itself depend on macro information is incompatible with that. For custom DSLs you should use bang macros (my_macro!()) rather than attribute macros as for those there is a much more flexible (but still independent of the macro it will resolve to) grammar accepting everything that tokenizes as rust code and has balanced parens.

9 Likes
3

I've been thinking about a related problem, of giving correct syntax highlighting and IDE support to macros in general. If a library like leptos or dioxus could provide some custom grammar files and IDEs supports some of these, it might improve usability of macros and allow library-specific IDE plugins. But requiring this feature at the language level feels wrong, because new grammars shouldn't break public interfaces if rust ever chose to use a new grammar language for everything, else rustc would forever have to support the grammars we chose to stabilize today

4

I can't think of a single actual use case where you would want to have non-Rust syntax specifically in an attribute macro. IMHO, attributes are for Rust code.

5

I once ran into a situation where I wanted to write a proc-macro that would automatically call some functions in its impl based on the enum definition, or something like that.

I no longer remember what exactly I wanted to do, but it was perhaps something like this:

#[my_attribute]
enum Message {
    OnAction1 = handlers::on_action1,
    OnAction2 = handlers::on_action2,
}

I only remember the compiler telling me that the proc_macro needed to have valid Rust syntax as an input and me thinking it was a bummer. So there are use-cases.

6

Perhaps this will be "fixed" by Declarative `macro_rules!` attribute macros by joshtriplett · Pull Request #3697 · rust-lang/rfcs · GitHub

As far as I remember, declarative macros had no such syntax restrictions.

7

You can represent that today with

#[my_attribute]
enum Message {
    #[my_attr_action = handlers::on_action1]
    OnAction1,
    #[my_attr_action = handlers::on_action2]
    OnAction2,
}
8

No, that RFC is about something else, not about lifting syntax restrictions.