[naga] Expose lexer/parser in frontend gated by feature flag?

[DexterHill0]

Is your feature request related to a problem? Please describe.
There are some crates out there that, in my opinion, would benefit from being able to directly access the lexer and its tokens, and/or the parser and AST. One such crate that comes to mind is naga_oil.
I would personally consider the implementation of the extra features in naga_oil (like imports and preprocessor definitions) to be fairly hacky as they are parsed using a regex or very basic lexer, which comes with some downsides.

Describe the solution you'd like
Exposing the lexer+/parser would allow these crates to use the tokens to their advantage which would be far more flexible and maintainable.

Describe alternatives you've considered
An alternative is to use or make another wgsl (or other language) parser, like wgsl-parser. The issue with this is you would be relying on another user to have correctly followed the wgsl spec, correctly implemented the parser and also would be keen on maintining it in the future.
There is also then the need to potentially have to take one AST, stringify it, and then run it back through naga's parser. Having the official naga implementation exported would allow people to have flexible wgsl syntax while waiting on those features to possibly be implemented in naga itself, in the future, without needing hacks/forks/other parsers.

On one hand I can understand that perhaps this is not in scope for naga as it could cause people to create custom wgsl shaders that only work using another dep alongside naga, while having others believe that shader should function correctly with naga alone, being unaware that another dep is required. On the other hand, I think this is really unlikely and realistically could already happen with crates like naga_oil.

[teoxoy]

We've previously discussed this in #5713 and are open to making those items public (no feature needed). A PR would be welcome.

[DexterHill0]

That's great! I'll look at making a PR very soon.

[DexterHill0]

Actually, I've had a think and a look through the source and I think I will hold off on the PR for now. I opened this issue in order to guage whether it would be possible for naga_oil to make use of naga in the future. As that will be allowed, I think it makes more sense to experiment with updating naga_oil first, using the features of naga. If I attempt to publicize some of the modules now, I doubt I would expose all of what is required to make those changes. I think it's better to open an issue in naga_oil, which I will do, and collate all the features required there first.

Does that sound reasonable?

[teoxoy]

It does, it's what I would have expected as well so, I'm glad you thought about it!

[jimblandy]

The wgpu maintainers talked about this today, and we're generally favorable towards exposing these parts of the WGSL front end for others to use --- with the understanding that they're not going to be any more stable than naga::Module is.