You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
The ingestion sources are currently mingled in with all the other ingestion code in the transforms package. This makes it hard to distinguish what all the sources are and makes it trickier for people to see how they might extend feast to support new sources.
Describe the solution you'd like
I'd like to create a source package in the ingestion module, and create common factory type class that can create FeatureIO.Read instances and have a function for returning their type string. New source implementations need only implement this interface and use an @AutoService annotation to make it available.
The import job can then load all available source factories from the class path and find the one with the matching type string against the import spec.
This follows the same pattern we use for storage types and allows us a path for including new source types as external dependencies, or moving existing ones into their own module. And then making them available simply by adding their jars on the class path.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
The ingestion sources are currently mingled in with all the other ingestion code in the
transforms
package. This makes it hard to distinguish what all the sources are and makes it trickier for people to see how they might extend feast to support new sources.Describe the solution you'd like
I'd like to create a
source
package in the ingestion module, and create common factory type class that can create FeatureIO.Read instances and have a function for returning their type string. New source implementations need only implement this interface and use an@AutoService
annotation to make it available.The import job can then load all available source factories from the class path and find the one with the matching type string against the import spec.
This follows the same pattern we use for storage types and allows us a path for including new source types as external dependencies, or moving existing ones into their own module. And then making them available simply by adding their jars on the class path.
The text was updated successfully, but these errors were encountered: