Feed Worker should store Posts in Redis when done

[humphd]

Our feed-worker.js currently parses Posts, puts them into an array, and console.logs them.

We need to use the work in #260 and #259, we need to update it so that a Post gets stored in the database. This depends on #260 for the changes to storage.js.

[c3ho]

I can work on this one!

[humphd]

@c3ho can I get you to do 1 Issue at a time? Right now you have 3 assigned to you, and other students will want a piece of the action, too.

[c3ho]

From @humphd

When we "resolve" in the feed worker (or return if using async), we can get the result back using queue.on('completed', (job, result) => /* result is what you resolved */)

const feedQueue = require('./queue');
const feedParser = require('./parser');
const { logger } = require('../utils/logger');
exports.processor = async function(job) {
  const { url } = job.data;
  logger.info({ url }, 'Processing job');
  let posts;
  try {
    posts = await feedParser(url);
  } catch (err) {
    logger.error({ err, url }, 'feed parser failed to parse url');
    throw err;
  }
  return posts.map(post => ({
    author: post.author,
    date: post.date,
    title: post.title,
    description: post.description,
    postURL: post.link,
  }));
};
exports.start = function() {
  // Start processing jobs from the feed queue...
  logger.info('starting feed queue processing...');
  feedQueue.process(exports.processor);
};

[humphd]

@c3ho I'm going to need this fix to land in the next day or so. Are you able to do it that quickly? If not, I'll need to take it from you. Let me know. We could also work on it in class tomorrow.

[c3ho]

I’ll be able to finish by tomorrow

[humphd]

Awesome. Let's plan to work on reviewing and merging this in class in the morning.