-
Notifications
You must be signed in to change notification settings - Fork 6
Plugins
RAIS has a rudimentary plugin system in place based on the way Go plugins work. Developers looking to create a plugin should read the Go plugin documentation.
A simple JSON tracer plugin is compiled via a standard make run. Other
plugins must be compiled manually as they're either full of system dependencies
(Image Magick) or examples that most users won't actually use (Datadog).
Compiling plugins is still fairly straightforward, e.g., make bin/plugins/imagick-decoder.so.
Plugins are enabled via a "Plugins" entry in /etc/rais.toml, an environment
variable named RAIS_PLUGINS, or a command-line flag, --plugins. However
they're configured, plugins must be explicitly loaded (unlike in v3.0.x where
they auto-loaded).
General behavior of plugins:
- Plugin patterns that don't start with a slash are loaded from within a
"plugins" directory which lives alongside the
rais-serverbinary - Plugin files must have the
.soextension - Plugin order matters!
- When a plugin pattern matches multiple files (e.g.,
Plugins="*.so"), plugins will be processed in alphabetic order. - If you need plugins to have clear priorities, but you still want to load
them all via a wildcard pattern, a naming scheme such as
00-name.somay make sense (replacing00with a two-digit load order value). - If you specify plugins individually (e.g.,
Plugins=s3-images.so), they're loaded in the order you specify.
- When a plugin pattern matches multiple files (e.g.,
- If you specify the same plugin multiple times, even on accident (e.g.,
Plugins=s3-images.so,*.so, RAIS won't start - Plugins must be compiled on the same architecture that is used to compile the RAIS server. This means the same server, the same openjpeg libraries, and the same version of Go.
Other than the JSON Tracer, all plugins must be built individually (e.g., make bin/plugins/datadog.so).
The JSON tracer plugin produces a JSON output file with very verbose
information about every request handled by RAIS, including how long the request
took to complete. This is built in a standard make invocation, and can be
enabled by first putting json-tracer.so in your plugins list in rais.toml,
and then configuring it:
- Set
TracerOutinrais.tomlor export the environment variableRAIS_TRACEROUTto point to a file for the traces to be written - Set
TracerFlushSecondsinrais.tomlor export the environment variableRAIS_TRACERFLUSHSECONDSto the duration in seconds between writes to the JSON file
The datadog plugin shows the use of WrapHandler (see below) by adding the DataDog tracing agent to all clients' requests. This allows high-level performance monitoring with minimal code.
This is mostly an example, so the documentation is primarily contained within the plugin's source. See main.go.
This plugin exposes a decoder for non-JP2 files. It currently only works for
file:// URLs, partially because the time to read and process a large non-JP2
file can be enormous even when a local file is in use. JP2s can be read in
small parts to decode a section of the file, but JPGs and TIFFs, for instance,
have to be completely read into memory even if a small operation is desired on
just a slice of the image.
This decoder can be very handy when you can't produce JP2 images for whatever
reason, but it should be used very carefully. Because of its inefficient
nature, and because it cannot be built without some extra system dependencies,
it is not built when running make. You must manually compile it via
make bin/plugins/imagick-decoder.so.
The plugin registers the decoder for the following file types: .tif, .tiff,
.png, .jpg, .jpeg, and .gif. No configuration is necessary beyond
enabling the plugin in rais.toml (e.g., Plugins="blah,imagick-decoder.so").
This section contains the complete list of functions a plugin may expose, and a detailed description of their function signature and behavior. This is mostly useful for plugin authors as opposed to end users.
func SetLogger(raisLogger *logger.Logger)
All plugins may define this function, and there are no side-effects to worry about when defining it (regarding ordering or multiple plugins "competing"). This function allows a plugin to make use of the central RAIS log subsystem so that logged messages can be consistent. Plugins don't have to expose this function if they aren't logging any messages.
logger.Logger is defined by package github.com/uoregon-libraries/gopkg/logger.
func Initialize()
All plugins may define this function. This can be used to handle things like custom configuration a plugin may need. See the s3 plugin's Initialize method for an example of that.
Initialize() is run after the logger is set up (unlike Go's internal init()
function), so you can safely use it.
var Disabled bool
Disabled lets a plugin state to the plugin manager that it is explicitly
disabled. If it's set to true, that typically means something happened in its
Initialize() function that prevented it from being usable. The JSON tracer
plugin uses this to prevent its exposed functions from being used when bad
configuration is detected, for instance.
func WrapHandler(pattern string, handler http.Handler) (http.Handler, error)
WrapHandler is called in the main IIIF handler, which occurs each time a user requests any IIIF image or info.json data from RAIS. It is meant only as a very high-level wrapper for the moment, and doesn't (easily) allow adding custom handlers to RAIS.
A plugin implementing WrapHandler can use the pattern to identify the path being wrapped -- though the IIIF path is variable, so this can be used for logging or other "identity" logic, but not easily for filtering. The handler passed in is the current state of the handler. A wrapper could add middleware, logging, etc. See the JSON tracer or datadog plugins for examples.
Any number of plugins can implement WrapHandler. Each plugin's returned handler is sent to the next plugin in the list.
If a plugin handles this function, but needs to skip a particular pattern, it
should return nil, plugins.ErrSkipped. This indicates to RAIS that the
plugin didn't fail, but simply chose to avoid trying to handle the given
pattern and handler.
TODO
TODO
func Teardown()
All plugins may define a Teardown function for handling any necessary cleanup. This is called when RAIS is about to exit, though it is not guaranteed to be called (for instance, if power goes out or the server is force-killed).
In addition to exposing the above functions, plugins may register Streamers and Decoders in their Initialize methods. Though registration could technically occur elsewhere, it is almost guaranteed not to work and could instead cause problems.
The low-level technical documenation, including interfaces you must implement,
should always be determined by running go doc from the RAIS project directory
so you know you're looking at the Decoder interface you need to support for the
version of RAIS your plugin is targeting. Additionally, the documentation
discovered by go doc will always be more up-to-date than this wiki.
The package name that's relevant for Streamers and Decoders is rais/src/img.
go doc is itself well-documented, but here are a few examples to help:
# Get a quick overview of what exists in the "img" package
go doc rais/src/img
# Get all documentation for everything in the "img" package (note that some of
# this will not be relevant to creating a plugin)
go doc --all rais/src/img
# Get a detailed view of the Streamer interface
go doc rais/src/img.Streamer
# Get a detailed view of the Decoder interface
go doc rais/src/img.DecoderAt a high level, a Streamer is a type that can read, seek, report some basic metadata, and be closed by RAIS. The default Streamers RAIS uses simply open files and cloud resources using built-in types and the gocloud.dev project, respectively.
Plugins which stream from custom sources have to register a StreamReader with RAIS. The StreamReader examines a URL and returns an OpenStreamFunc bound to the given URL if the plugin reads from the given URL. The OpenStreamFunc, once called, returns a Streamer ready for use. All this indirection means that RAIS doesn't have to actually open any files until it's ready to use them, while still being able to determine if something is available to open them in the first place.
For a more concrete example, consider the built-in FileStream:
- When the RAIS server starts, the
fileStreamReaderfunction is registered as a StreamReader (img.RegisterStreamReader(fileStreamReader)is called) - At some point, a user requests
blah.jp2- By default, when RAIS is sent a request for an asset that doesn't have a
full URL, "file://" and the TilePath are prepended to it, e.g.,
blah.jp2becomesfile:///var/local/images/blah.jp2
- By default, when RAIS is sent a request for an asset that doesn't have a
full URL, "file://" and the TilePath are prepended to it, e.g.,
- All StreamReaders are consulted - with no other plugins, the only ones will
be
fileStreamReaderandcloudStreamReader, which can be found insrc/cmd/rais-server/register.go - The requested URL has the
filescheme, sofileStreamReaderis used and returns an anonymousOpenStreamFuncbound to the URL so that RAIS can call that function when it's ready to actually open the stream - The result of calling an
OpenStreamFuncis always a Streamer, in this case an instantiatedFileStream(which is defined insrc/img/file_stream.go)
A Decoder handles all the actual image manipulation: reading, cropping, getting image dimensions, etc. A Decoder reads from a Streamer, and generally shouldn't resort to direct filesystem access even when the Streamer's data is locally available.
Decoders are handled very similarly to Streamers in how they're registered with
RAIS: a DecodeHandler via
RegisterDecodeHandler(). A DecodeHandler is given a Streamer and returns
a DecodeFunc which is bound to the given Streamer. When the
DecodeFunc is called, it returns a Decoder, ready to be used to
manipulate and return image data.
An example of a custom decoder can be seen in the ImageMagick decoder plugin.