Store

Store

The store in forerunner is responsible for keeping a record of all jobs that move through the system. The store is intended to be a database such as PostgreSQL, but other solutions such as MongoDB will work as well.

Writing your own store

The only requirement for the store is that it passes the tests provided in the store tests module. To pass these tests, the following methods need to be implemented.

create(jobType, rawData, callback(err, jobId, jobData))

This function will create a new record of a job of a given type and raw input. The callback takes the id that is assigned to the job and the jobData that is to be sent off to the workers. The id can be any unique string or number which can be used to identify the job at a later time. The jobData is the normalized input to be sent off to the worker pool (ie. if the database generates some default values on insert). This function can simply pass the rawData directly back as the jobData if it is not needed.

progress(jobId, progress, callback(err))

If a job on a worker emits back a progress event, the store will be notified of the new progress. Calling back with an error will not cause the job itself to error, but will cause the forerunner manager to log the error.

countFailed(jobId, message, callback(err, failcount, forceFail))

If a job has failed this method will be called with the jobId and the error message sent back by the worker. The method must callback with the number of times the job has failed so that the manager can decide if the job should be requeued or not. Failing to due so will cause jobs to never be marked as failed.

If the function calls back with forceFail as truthy then the job will be marked as failed no matter the failCount. This is useful if it is obvious a job should not be run again and should be taken out of queue right away.

failed(jobId, callback(err))

Marks a job as failed. Calling back with an error will do nothing more than causing the manager to log the error.

complete(jobId, jobResult, callback(err))

Marks a job as complete. Calling back with an error will do nothing more than causing the manager to log the error.