Add a way to fully customize how schemas are fetched

[EricWittmann]

Description
Allow an optional function to be provided when using this component that will be called to fetch a schema when e.g. a reference needs to be resolved.

Reasons

In our project (Apicurio Registry) it would really helpful to be able to customize how the schema (and any references) is fetched. Currently the FetchingSchema interface is good if I just want to provide e.g. auth information (great) but not if I need to actually change how a schema is fetched. In our case the value of the $ref can be a "logical" value that is then mapped to coordinates in the registry (mapping is configured externally to the AsyncAPI document).

Additionally, we would load the content by making a REST call using our SDK/client - so being able to provide our own function to resolve and then load the ref content would be ideal.

Is there already a way to do this? If not, perhaps I could contribute...

[github-actions]

Welcome to AsyncAPI. Thanks a lot for reporting your first issue. Please check out our contributors guide and the instructions about a basic recommended setup useful for opening a pull request.
Keep in mind there are also other channels you can use to interact with AsyncAPI community. For more details check out this issue.

[derberg]

@EricWittmann hey, definitely makes sense. Would you be interested in contributing it?

[EricWittmann]

I would be interested. If you have some ideas about direction, that would help. Having recently implemented support for authentication when using the AsyncAPI react component, I discovered that auth information needed to be provided twice - once in the FetchingSchemaInterface (as request options) and again in the ConfigInterface in the resolve section.

If I could just provide my own resolver, that same resolver should be used for all external references, not just the main one. I didn't dig deeply enough to determine how feasible that is. It will depend on how much of the parser's external $ref resolving code is first-party vs. a library. I didn't really check.

[EricWittmann]

This has actually risen a bit in our priority list because alternate approaches to solve our problem are ... problematic. :) I implemented a feature in registry where we rewrite the values of the $refs to be valid, absolute registry URLs. That works great if you have our "Anonymous read access" feature enabled. This allows the react component to properly follow the registry URLs without requiring an auth token. However, if that feature is not enabled then we would need to inject the auth token into the react component. I've done that successfully in a POC but it's not really a viable solution because the document may have a mix of Registry URLs and non-Registry URLs as the $ref values. In this case we would be leaking the Registry user's auth token to all of the non-Registry reference URLs. That's super bad.

We could implement a proxy, or find some other approach. But ultimately we think the right approach is for the client to understand how to resolve and fetch the schemas using a custom resolver that we provide.

So yeah - I'd love to contribute that. If you have any pointers for getting started with it, great. If not I'll dig in as soon as I can.

[derberg]

@EricWittmann if I understand your request correctly, the solution is easier, but not really 😄

long story short: this component uses our JavaScript Parser to validate, dereference and parse AsyncAPI documents. So any customisations for $ref behaviour should first be enabled in JavaScript Parser. The parser v2 uses json-ref-resolver and v1 uses another $ref parser. This will not change very soon. React component is still using parser v1 and closest plan to migrate to v2 is few months, unless you want to try to help.

why am I writing the whole story? as to address your idea, any customizations should be done without workarounds, by using the customer resolvers functionality that the $ref parsers that we use expose.

open questions are:

• we need some proposal and proof that custom resolvers can be passed to parser at the moment and they do what is needed
• we need some proposal, like if the custom resolver is generic and can be part of the parser itself, or maybe it should be just located in the react component (best would be parser)
• there is no way to do above 2 bullet points in parser v1, which means that if they are done in v2, react component requires refactor first

related issues:

• Support resolution of refs which require authentication parser-js#771
• Add ability to configure ssl certificate authentication to avro parser parser-js#207

---

this is super complicated topic as you can see because we are in the middle of AsyncAPI 3.0 preps + new parser intro and some other stuff

and regarding migration of parser v1 to v2 in react, there is a pretty good migration guide that will help

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity 😴

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience ❤️

[dweber019]

Had an issue where I require to use my own fetch implementation as resolver of https schemas.
See backstage/backstage#22160 tested with this plugin version 1.2.6.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity 😴

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience ❤️