Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: classify #23

Merged
merged 7 commits into from
Oct 11, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .autod.conf.js
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ module.exports = {
'autod',
'eslint',
'eslint-config-egg',
'supertest',
],
exclude: [
'./test/fixtures',
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@ npm-debug.log
node_modules/
coverage/
run/
.vscode
1 change: 0 additions & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ sudo: false
language: node_js
node_js:
- '6'
- '7'
- '8'
install:
- npm i npminstall && npminstall
Expand Down
65 changes: 38 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ $ npm i egg-schedule --save

`egg-schedule` is a plugin that has been built-in for egg. It is enabled by default.

```javascript
```js
// {app_root}/config/plugin.js
exports.schedule = {
package: 'egg-schedule',
Expand All @@ -42,8 +42,9 @@ exports.schedule = {
/**
* @property {Object} schedule
* - {String} type - schedule type, `worker` or `all`
* - {String} [cron] - cron expression, [see below](#cron-style-scheduling)
* - {String | Number} [interval] - interval expression in millisecond or express explicitly like '1h'. [see below](#interval-style-scheduling)
* - {String} [cron] - cron expression, see [below](#cron-style-scheduling)
* - {Object} [cronOptions] - cron options, see [cron-parser#options](https://github.com/harrisiirak/cron-parser#options)
* - {String | Number} [interval] - interval expression in millisecond or express explicitly like '1h'. see [below](#interval-style-scheduling)
* - {Boolean} [immediate] - To run a scheduler at startup
* - {Boolean} [disable] - whether to disable a scheduler, usually use in dynamic schedule
*/
Expand Down Expand Up @@ -71,35 +72,32 @@ The rule of thumbs is one job per file.

## Task

Task is a generator function, and accept one parameter, `ctx`. The syntax is, `exports.task = function* (ctx) { ... };`

When the scheduled task runs, the scheduled job information will be logged and written to a local file in a folder called `/logs`. The log file contains many useful information, for example,
Task is a generator/async function, and accept one parameter `ctx` which is an anonymous context with:

- ctx.method: `SCHEDULE`
- ctx.path: `/__schedule/${schedulePath}`. example path: `/__schedule?path=/FULL_PATH_TO/cleandb.js&type=worker&interval=3h`
- ctx.query: `scheule config(type=worker&cron=*%2F5%20*%20*%20*%20*%20*)`
- ctx.path: `/__schedule?path=${schedulePath}&${schedule}`.


To create a task, it is as simple as write a generator function. For example:
To create a task, it is as simple as write a generator / async function. For example:

```javascript
```js
// A simple logger example
exports.task = function* (ctx) {
ctx.logger.info('Info about your task');
};
```

```javascript
```js
// A real world example: wipe out your database.
// Use it with caution. :)
exports.task = function* (ctx) {
yield ctx.service.db.cleandb();
exports.task = async function(ctx) {
await ctx.service.db.cleandb();
};
```

## Scheduling

`schedule` is an object that contains one required property, `type`, four optional properties, `{ cron, interval, immediate, disable }`.
`schedule` is an object that contains one required property, `type`, and optional properties, `{ cron, cronOptions, interval, immediate, disable }`.

### Cron-style Scheduling

Expand All @@ -123,11 +121,14 @@ Use [cron-parser](https://github.com/harrisiirak/cron-parser).

Example:

```javascript
```js
// To execute task every 3 hours
exports.schedule = {
type: 'worker',
cron: '0 0 */3 * * *',
cronOptions: {
// tz: 'Europe/Athens',
}
};
```

Expand All @@ -137,7 +138,7 @@ To use `setInterval`, and support [ms](https://www.npmjs.com/package/ms) convers

Example:

```javascript
```js
// To execute task every 3 hours
exports.schedule = {
type: 'worker',
Expand All @@ -154,28 +155,38 @@ exports.schedule = {

**Custom schedule**

To create a custom schedule, simply create a schedule with a type `custom` and its corresponding method. Inside your custom method, you can schedule the task to be executed by one random worker or all workers with the built-in method `sender.one()` or `sender.all()`.
To create a custom schedule, simply extend `agent.ScheduleStrategy` and register it by `agent.schedule.use(type, clz)`.
You can schedule the task to be executed by one random worker or all workers with the built-in method `this.sendOne()` or `this.sendAll()`.

```javascript
```js
// {app_root}/agent.js
const SCHEDULE_HANDLER = Symbol.for('egg#scheduleHandler');

module.exports = agent => {
// sender.one() - will notify one random worker to execute task
// sender.all() - will notify all workers
agent[SCHEDULE_HANDLER].custom = (schedule, sender) =>
setInterval(sender.one, schedule.interval);
module.exports = function(agent) {
class CustomStrategy extends agent.ScheduleStrategy {
constructor(...args) {
super(...args);
this.interval = setInterval(() => {
this.sendOne();
}, this.schedule.interval);
}
}
agent.schedule.use('custsom', CustomStrategy);
};
```

Then you could use it to defined your job:

```js
// {app_root}/app/schedule/other.js
exports.schedule = {
type: 'custom',
};

exports.task = function* (ctx) {};
```

## Dynamic schedule

```javascript
```js
// {app_root}/app/schedule/sync.js
module.exports = app => {
exports.schedule = {
Expand All @@ -199,7 +210,7 @@ module.exports = app => {

Example:

```javascript
```js
it('test a schedule task', function* () {
// get app instance
yield app.runSchedule('clean_cache');
Expand Down
143 changes: 32 additions & 111 deletions agent.js
Original file line number Diff line number Diff line change
@@ -1,121 +1,42 @@
'use strict';

const loadSchedule = require('./lib/load_schedule');
const parser = require('cron-parser');
const ms = require('humanize-ms');
const safetimers = require('safe-timers');
const SCHEDULE_HANDLER = Symbol.for('egg#scheduleHandler');
const WorkerStrategy = require('./lib/strategy/worker');
const AllStrategy = require('./lib/strategy/all');

module.exports = agent => {
// add handler into `agent[SCHEDULE_HANDLER]` for extend other kind of schedule type.
// worker: will excute in one random worker when schedule excuted.
// all: will excute in all workers when schedule excuted.
const handlers = agent[SCHEDULE_HANDLER] = {
worker: workerHandler,
all: allHander,
};

agent.messenger.once('egg-ready', startSchedule);

function startSchedule() {
agent.disableSchedule = false;
const schedules = loadSchedule(agent);
for (const s in schedules) {
const schedule = schedules[s];
if (schedule.schedule.disable) continue;

const type = schedule.schedule.type;
const handler = handlers[type];
if (!handler) {
const err = new Error(`schedule type [${type}] is not defined`);
err.name = 'EggScheduleError';
throw err;
}
handler(schedule.schedule, {
one() {
sendMessage(agent, 'sendRandom', schedule.key);
},
all() {
sendMessage(agent, 'send', schedule.key);
},
});
}
}

agent.on('close', () => {
agent.disableSchedule = true;
return;
// register built-in strategy
agent.schedule.use('worker', WorkerStrategy);
agent.schedule.use('all', AllStrategy);

// TODO: compatible, will remove at next major
const handlers = {};
Object.defineProperty(agent, SCHEDULE_HANDLER, {
get() {
agent.deprecate('should use `agent.schedule.use()` instead of `agent[Symbol.for(\'egg#scheduleHandler\')]` to register handler.');
return handlers;
},
});
};

function sendMessage(agent, method, key) {
if (agent.disableSchedule) {
agent.coreLogger.info(`[egg-schedule] message ${key} did not sent`);
return;
}
agent.coreLogger.info(`[egg-schedule] send message: ${method} ${key}`);
agent.messenger[method](key);
}

function workerHandler(schedule, sender) {
baseHander(schedule, sender.one);
}

function allHander(schedule, sender) {
baseHander(schedule, sender.all);
}

function baseHander(schedule, send) {
if (!schedule.interval && !schedule.cron) {
throw new Error('[egg-schedule] schedule.interval or schedule.cron must be present');
}

if (schedule.interval) {
const interval = ms(schedule.interval);
safeInterval(send, interval);
}

if (schedule.cron) {
let interval;
try {
interval = parser.parseExpression(schedule.cron);
} catch (err) {
err.message = `[egg-schedule] parse cron instruction(${schedule.cron}) error: ${err.message}`;
throw err;
agent.messenger.once('egg-ready', () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

加一下注释,需要等插件注册 strategy

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

加了,在下一行

// wait for other plugin to register custom strategy
const keys = Object.keys(handlers);
for (const type of keys) {
agent.schedule.use(type, handler2Class(type, handlers[type]));
}
startCron(interval, send);
}

if (schedule.immediate) {
setImmediate(send);
}
}

function startCron(interval, listener) {
const now = Date.now();
let nextTick;
do {
nextTick = interval.next().getTime();
} while (now >= nextTick);

safeTimeout(() => {
listener();
startCron(interval, listener);
}, nextTick - now);
}

function safeTimeout(fn, delay, ...args) {
if (delay < safetimers.maxInterval) {
setTimeout(fn, delay, ...args);
} else {
safetimers.setTimeout(fn, delay, ...args);
}
}
// start schedule after worker ready
agent.schedule.start();
});

function safeInterval(fn, delay, ...args) {
if (delay < safetimers.maxInterval) {
setInterval(fn, delay, ...args);
} else {
safetimers.setInterval(fn, delay, ...args);
function handler2Class(type, fn) {
return class CustomStrategy extends agent.ScheduleStrategy {
constructor(...args) {
super(...args);
fn(this.schedule, {
one: this.sendOne.bind(this),
all: this.sendAll.bind(this),
});
}
};
}
}
};
41 changes: 25 additions & 16 deletions app.js
Original file line number Diff line number Diff line change
Expand Up @@ -33,35 +33,44 @@ module.exports = app => {
return schedule.task(ctx);
};

// log schedule list
for (const s in schedules) {
const schedule = schedules[s];
if (schedule.schedule.disable) continue;
if (!schedule.schedule.disable) app.coreLogger.info('[egg-schedule]: register schedule %s', schedule.key);
}

const task = schedule.task;
const key = schedule.key;
app.coreLogger.info('[egg-schedule]: register schedule %s', key);
app.messenger.on(key, () => {
app.coreLogger.info('[egg-schedule]: get message %s', key);
// register schedule event
app.messenger.on('egg-schedule', data => {
app.coreLogger.info('[egg-schedule]: get message: %j', data);
const key = data.key;
const schedule = schedules[key];

// run with anonymous context
const ctx = app.createAnonymousContext({
method: 'SCHEDULE',
url: `/__schedule?path=${s}&${qs.stringify(schedule.schedule)}`,
});
if (!schedule) {
app.coreLogger.warn(`[egg-schedule] unknown task: ${key}`);
return;
}
/* istanbul ignore next */
if (schedule.schedule.disable) return;

// run with anonymous context
const ctx = app.createAnonymousContext({
method: 'SCHEDULE',
url: `/__schedule?path=${key}&${qs.stringify(schedule.schedule)}`,
});

const start = Date.now();
task(ctx)
const start = Date.now();
const task = schedule.task;
task(ctx)
.then(() => true) // succeed
.catch(err => {
err.message = `[egg-schedule] ${key} excute error. ${err.message}`;
app.logger.error(err);
return false; // failed
return false; // failed
})
.then(success => {
const rt = Date.now() - start;
const status = success ? 'succeed' : 'failed';
app.coreLogger.info(`[egg-schedule] ${key} excute ${status}, used ${rt}ms`);
});
});
}
});
};
Loading