Loading files

ml-javaclient-util includes support for loading files from disk into a MarkLogic database, with specific support for loading modules and schemas.

For information specifically on modules, please see Loading modules.

Loading any kind of file

Support for loading any kind of files is handled by GenericFileLoader, which provides a flexible and extensible API for loading files from multiple directories, where permissions and collections can be specified on a per-file basis.

GenericFileLoader provides the following features:

1. Files can be loaded from one or more paths.
2. Files are loaded via a BatchWriter, which means they can be loaded via the REST API, DMSDK, or XCC.
3. Zero or more FileFilter objects can be used to configure which files are loaded.
4. Zero or more DocumentFileProcessor objects can be used to process each file as it's loaded. A processor can decide not to load the file, or it could modify the target URI, or the collections or permissions (see more details below), etc.
5. A default set of permissions and default set of collections can be defined.
6. Tokens can be replaced in the text of a file before it's loaded.
7. GenericFileLoader is aware of a number of extensions that indicate a file should be loaded as a binary, and more extensions can be added.

Specifying collections and permissions

As of 2.13.0, you can now specify collections and permissions for schemas, and this will soon be supported for modules too (this is true as of 3.0.0, and works for any kind of file). This has been added specifically for making it easier to add ML9 redaction rulesets to specific collections. You can do this by defining either of the following files in any directory containing schemas:

• collections.properties
• permissions.properties

These are expected to have key/value pairs of filename=collection1,collection2 and filename=role,capability,role,capability.

For example, for a file named "my.ruleset", you could have the following in collections.properties:

my.ruleset=coll1,coll2

And in permissions.properties:

my.ruleset=rest-reader,read,rest-writer,update

And your directory would look like this:

collections.properties
my.ruleset
permissions.properties

Note that these special properties files will NOT be loaded into MarkLogic - they're just there to provide metadata for files that you do want to load.