A very minimal and simple library to interact with different storage systems
File systems and storage systems all expose very similar, but somewhat different, interfaces to interact with files and directories. Here we provide a very simple and basic interface that abstract various storage systems.
To use castore you can add the following maven dependency:
<dependency>
<groupId>uk.ac.standrews.cs</groupId>
<artifactId>castore</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
This project is not available via a maven repository yet, so you will have to install this yourself by running mvn install.
Alternatively you can produce a .jar for the project, mvn package -am -DskipTests.
This implementation is based on the local file system.
Settings: type, root path
In order to work with an AWS S3 storage, you need to provide an access_key_id and a secret_access_key. This can be done explicitly or by setting the environment variables:
export AWS_ACCESS_KEY_ID=<KEY>
export AWS_SECRET_KEY=<KEY>
Settings: type, bucket name, credentials (optional)
Redis is an in-memory data structure store, which is networked and very fast. A Redis-based storage can be used to store small amount of data with good performance.
Before using the Redis-based storage, you need Redis installed. Then run a server instance: redis-server
Settings: type, host, port (optional)
First you need to create a Dropbox app and get an App access token.
Settings: type, path to dropbox folder, access token
More on how to generate an access token here
Need to create a Google Drive App.
Settings: type, credentials json file
No need to set the path for the credentials file explicitly, if at the path "~/.drive/credentials.json"
Examples are available here.
- More implementations:
- In-memory
- GUID-based
- OneDrive
- WebDAV
- FTP
- Make a storage instance available via a REST API
- CLI
CircleCI automatically builds and runs the tests for the project.
Codecov, with JaCoCo, is used to measure the test coverage for the project. Info on how to read the Codecov reports: https://docs.codecov.io/v4.3.6/docs/viewing-source-code
The following are the settings used in CircleCI to run the tests successfully.
Environment Variables
- DRIVE_SERVICE_KEY = result of
base64 <your-service-account.json> - DROPBOX_TOKEN = secret token generated by Dropbox
- AWS_ACCESS_KEY_ID and AWS_SECRET_KEY = as generated by AWS
Pre-tests
mkdir -p ${HOME}/.drive/
touch ${HOME}/.drive/credentials.json
echo ${DRIVE_SERVICE_KEY} | base64 --decode > ${HOME}/.drive/credentials.json
This project is inspired by the asa/filesystem project developed by Alan Dearle, Graham Kirby, and Stuart Norcross at the University of St Andrews.
The castore project has been developed, and is currently maintained, by Simone I. Conte at the University of St Andrews.
Suggestions and Pull Requests are more than welcome.