Worker

Worker

A worker is a separate process that is able to handle a variety of jobs. Workers can be configured to take on different jobs, allowing for a more efficient use of resourced. For example, jobs that require less processing power can be set up on cheaper less powerful machines, memory heavy jobs placed on high memory machines, and so on.

Setting up a worker

var worker = require('forerunner').worker;

var job = require('./my-custom-job');
worker.registerJobHandler('job_type', job);

worker.start('http://location.of.manager:1337');

Creating Jobs

Jobs are a series of tasks that will take place on one worker. Jobs are made up of individual tasks, each task doing one (and only one!) specific operation. A simple job can consist of one task, but more complicated jobs can be made up of many tasks and even other jobs. When a job is made up many tasks each task will be executed in order. If a task calls back with an error the job will exit immediately and tell the manager that the job has failed.

Basic example

Jobs are functions with the following structure:

function exampleJob(id, data, callback) {
  // id - the assigned job id
  // data - the job data to work will
  // callback(err, results) - callback to be called when the job is done
  //                        - if err is not falsely then the job will be marked as failed and NO results will be returned to the manager

  // ... Do some mind-bending javascript ...

  callback(null, {foo: bar});
}

// and registered to the worker like so:
worker.registerJobHandler('job_type', exampleJob);

More complex jobs

Jobs can be composed together, allowing for arbitrarily functionality to be created easily.

worker.registerJobHandler('some_registered_job', require('./a-task-somewhere'));

// This job consists of four tasks.
// It will execute 'some-job', 'another-job', emit a progress event to the manager
// and then execute the job handler 'some_registered_job'

var customJob = worker.compose([
  require('./some-job'),
  require('./another-job')
  'progress:hey a progress event!'
  'some_registered_job'
]);

// This job shows the power of composing jobs
// It will execute the 4 steps of customJob, followed by another progress event and 'one-last-thing'

var anotherCustomJob = worker.compose([
   customJob,
  'progress:another progress event',
   require('./one-last-thing')
]);

// Workers only tell the manager about job handlers that you have registered
// in this example the worker will be able to take on the jobTypes: 
//    some_registered_job, custom_job, and another_custom_job
//
// Notice that 'some_registered_job is both its only job handler AND a task inside of another job
// By doing this the worker is able to limit what jobs it will take on, but without having to waste code

worker.registerJobHandler('custom_job', customJob);
worker.registerJobHandler('another_custom_job', anotherCustomJob);

Methods

start(managerLocation)

Attempts to connect to the forerunner manager at the given url. The worker will not take on or process jobs until the connection is established.

compose(arrayOfJobs)

Takes an array of job functions or strings of jobTypes and returns a job that will execute all each of the given job in order. Progress events can be sent back by including a string with the format 'progress:message to send back'.

registerJobHandler(jobType, jobHandler)

Registers a job handler against for a specific job type. Note that 'progress' is a reserved word and as a result you cannot have a jobType with that name.