Add bind:true option to getCurrentContext #19

josieusa · 2017-01-03T15:14:00Z

Description

Workaround issue #17 with cujojs/when v3.7.7 and similar packages (mostly Promise libraries) that use queues for async-related stuff.
User code is cleaner and more elegant than initially proposed in #17, because he/she only needs to remember to call LoopBackContext.getCurrentContext() at the first line of his/her middlewares/operation hooks which use the current context feature (if it's a remote hook, the feature should not be needed anymore; if you use it anyway, use it preferably in beforeRemote rather than afterRemote).
P.S. : use it early in the middleware/hook chain for a given route.
Library code, instead, can be improved a lot; this is just my first attempt. Style needs to be checked too.
Also, I only patched the context functions get and set, because I don't use other functions. I didn't patch the other functions yet (~~I deleted them, whoops 😄 This aspect needs to be fixed~~).

Thank you!

Related issues

[BUG] "when" 3.7.7 package makes context get mixed up between concurrent HTTP requests #17

Checklist

New tests added to cover all changes (the new test passes!)
Documentation changes
Code conforms with the style
guide

slnode · 2017-01-03T15:14:01Z

Can one of the admins verify this patch? To accept patch and trigger a build add comment ".ok\W+to\W+test."

slnode · 2017-01-03T15:14:11Z

Can one of the admins verify this patch?

slnode · 2017-01-03T15:14:11Z

Can one of the admins verify this patch?

bajtos · 2017-01-03T16:11:30Z

server/middleware/per-request.js

@@ -31,6 +31,16 @@ function perRequestContextFactory(options) {
  var scope = options.name || name;
  var enableHttpContext = options.enableHttpContext || false;
  var ns = LoopBackContext.createContext(scope);
+  // Monkey-patch LoopBackContext.getCurrentContext in order to fix issue #17


I am wondering - why are we applying this workaround only in the middleware, instead of binding get/set methods directly inside the original getCurrentContext?

Also IIUC, this code is fixating the context object to a single value when a request reaches per-request middleware. How is this going to support the case when there are multiple requests handled in parallel?

Imagine the following handler chain (three different middleware handlers):

per-context

access-token retrieval, finishes in a later tick

store access token in the context

Now consider two requests handled in the following sequence:

Request1 invokes per-context, modifies global LoopBackContext.getCurrentContext to return CTX1

Request1 starts access-token retrieval

Request2 invokes per-context, modifies global LoopBackContext.getCurrentContext to return CTX2

Request2 starts access-token retrieval

Request1 access token is retrieved

The access token is stored in CTX2. Ooops.

I tried to bind inside the original method, but for some reasons it didn't work.
It should work if I monkey-patch the method inside, say, a utility module. This will be my next attempt (less dirty than the first), unless I figure out why it doesn't work in the original method. Please wait for the commit.
I agree that it needs to be moved away from the middleware factory. In fact, if you add the middleware to the app, it works (even outside the middleware); but if you don't add the middleware to the app... whoops.

Regarding the concurrent requests, this is done exactly for them. Even if the code needs to be fixed, tests pass with the workaround (and fail if I comment out the workaround), and it already works in a test app with 10 concurrent curl requests.

I have a possible explanation. Are you sure that points 1 and 3 are correct? In these points, the function that gets invoked is this:
https://github.com/strongloop/loopback-context/blob/v1.0.0/server/middleware/per-request.js#L36
This function (the middleware) does not modify the global LoopBackContext.getCurrentContext (it's the outer function, i.e. the middleware factory, that does, and the latter gets called when adding the middleware to the app).

Fixed, now the workaround is outside the middleware [factory], and it still works (and this time, this is what I expected) in my test app with 10 concurrent curl requests.

And tests pass only with the workaround in place. They fail as soon as you comment the workaround out. 👍

Also, I refactored the original method. I no longer monkey-patch it, which was not beautiful.

I bet it can be done without the proxy, but since I couldn't manage to do it, I'm fine with the proxy.

The proxy makes it clearer, I hope. I delegate the get and set methods of the cls namespace to a local object (boundMethods) which gets created every time you call getCurrentContext() (I used a new object so that the binding of this when you call ctx.get or ctx.set is predictable, and can't get mixed up between requests).

And at the same time, I re-bind the methods to the (globally visible) namespace; more precisely, I bind them to the local variable in the body of createContext()which holds a reference to the (globally visible) namespace.

bajtos · 2017-01-03T16:12:26Z

test/main.test.js

+    var timeout = 50;
+    // Concurrent execution number 1 of 2
+    var execution1 = new Promise(function execution1(outerResolve, reject) {
+      LoopBackContext.runInContext(function pushToContext1() {


AFAICT, these tests are not invoking per-request middleware, therefore they are not verifying the change proposed in that middleware :(

Whoops, you are right, they are not invoking the middleware (I don't need it in the test; I only need parallel execution). The tests pass anyway (and a test app with concurrent requests works), after adding the workaround. But there is one more reason, that is, the middleware factory was called before, somewhere... I'll need to fix that. Tests should pass for only one reason (i.e. the addition of the fix).

Problem solved without changing the tests, since the change is no longer in the middleware [factory] (I moved it inside the original method, instead of patching it in the middleware [factory])

bajtos · 2017-01-03T16:15:42Z

test/main.test.js

@@ -129,4 +130,50 @@ describe('LoopBack Context', function() {
      ], done);
    });
  });
+  it('doesn\'t mix up contexts if using concurrently then() from when 3.7.7',
+  function() {
+    expect(require('when/package.json').version).to.equal('3.7.7');


I am little bit worried about this long-distance coupling between the tests and package.json, and also what will happen when we will want to test two different versions of a same package.

Could we run npm install [email protected] from the test instead? To speed up local npm test and prevent repeated fetching of packages from the registry, we can use https://www.npmjs.com/package/strong-cached-install to cache the results - I have had good experience using this approach in other LoopBack projects. We will need to expose API for installing a single package, but that should be trivial - see the existing installPackage function.

PTAL
Solved in a different way (local modules and stub modules)
Inspiration from here:
Local modules http://stackoverflow.com/a/26028854
Stub modules npm/npm#5499 (comment)
Works for me using two different, conflicting versions of the same package.

slnode · 2017-01-04T08:36:17Z

Can one of the admins verify this patch? To accept patch and trigger a build add comment ".ok\W+to\W+test."

slnode · 2017-01-04T08:36:22Z

Can one of the admins verify this patch?

slnode · 2017-01-04T08:36:22Z

Can one of the admins verify this patch?

josieusa · 2017-01-04T08:54:47Z

Sorry, this is a test comment to see if the pull requests gets re-opened and the disappeared commit re-appears...
EDIT
I deleted the old commits (instead of reverting them), because the code was wrong, then I pushed a new commit, where I address all the line notes (except for the addition of strong-cached-install, which isn't done yet).

jainisenough · 2017-01-04T11:03:48Z

This issue also comes when we use loopback-connector-rest (latest version, 2.1.0) & call any third part api.

josieusa · 2017-01-04T11:12:09Z

@jainisenough Sorry, I made a mistake in issue #17. Fixed. The issue is not that the context gets lost, but rather mixed up between concurrent HTTP requests. Sorry if I confused you.

josieusa · 2017-01-04T16:03:56Z

I just wanted to write a reminder that this PR doesn't seem to fix strongloop/loopback#2519 (I suspect it's because this PR only works for user code, not 3rd-party code invoking "cls-unaware" libraries).

jainisenough · 2017-01-05T08:15:03Z

@josieusa, No. this is context lost issue when we call third party api via loopback-context-rest. i'm aware about [email protected] issue, so I'm using 1.17.0 right now.

josieusa · 2017-01-05T08:41:38Z

@jainisenough the issue #17 is that the context gets mixed up between requests.

bajtos

Thank you @josieusa for the update, the patch looks much better now! I still have few concerns to discuss and/or address, see below.

bajtos · 2017-01-09T15:05:49Z

package.json

@@ -26,13 +26,14 @@
    "cls-hooked": "^4.0.1"
  },
  "devDependencies": {
-    "async": "1.5.2",
+    "async-1.5.2": "file:./test/stub-modules/async-1.5.2",


Nice trick 👍

Will this work on windows?

Using local paths for dependencies is a regular npm feature described in npm docs, see https://docs.npmjs.com/files/package.json#local-paths

I would expect it to work on Windows too.

bajtos · 2017-01-09T15:08:07Z

server/current-context.js

@@ -78,7 +78,28 @@ LoopBackContext.createContext = function(scopeName) {
    process.context[scopeName] = ns;
    // Set up LoopBackContext.getCurrentContext()
    LoopBackContext.getCurrentContext = function() {
-      return ns && ns.active ? ns : null;
+      if (ns && ns.active) {


if (!ns || !ns.active) return null; // the rest of the code is indented one level less

bajtos · 2017-01-09T15:11:16Z

server/current-context.js

+            target[name];
+          },
+        };
+        return new Proxy(ns, handler);


I find using Proxy rather heavy weight. How about using regular prototypal inheritance, would it not work equally well?

var boundContext = Object.create(ns); boundContext.get = ns.bind(ns.get); boundContext.set = ns.bind(ns.set); return boundContext;

Proxy also seems to be not available in Node 4.x, see https://travis-ci.org/strongloop/loopback-context/jobs/189233795#L264

Unhandled error for request GET /TestModels/test: ReferenceError: Proxy is not defined at Object.LoopBackContext.getCurrentContext (/home/travis/build/strongloop/loopback-context/server/current-context.js:99:20)

bajtos · 2017-01-09T15:12:52Z

test/main.test.js

+          setTimeout(resolve, timeout);
+        });
+        whenPromise.then(function pullFromContext2() {
+          var testValue = ctx && ctx.get('test-key', 'test-value-2');


Isn't the second argument of get ignored?

bajtos · 2017-01-09T15:15:19Z

test/main.test.js

+  function() {
+    var timeout = 50;
+    // Concurrent execution number 1 of 2
+    var execution1 = new Promise(function execution1(outerResolve, reject) {


Considering that the code building execution1 and execution2 differ only in one parameter (the value stored in the context), the code would be much easier to read if the creation of execution promise was refactored into a shared function.

Promise.all([ runTestWithValue('test-value-1'), runTestWithValue('test-value-2'), ]); function runTestWithValue(value) { LoopBackContext.runInContext(...); }

bajtos · 2017-01-09T15:16:11Z

test/main.test.js

+        expect(ctx).is.an('object');
+        ctx.set('test-key', 'test-value-1');
+        var whenPromise = whenV377.promise(function(resolve) {
+          setTimeout(resolve, timeout);


Can we simplify this code by using promise.delay? See https://github.com/cujojs/when/blob/master/docs/api.md#promisedelay

bajtos · 2017-01-09T15:16:43Z

test/main.test.js

+        }).then(function verify1(testValue) {
+          expect(testValue).to.equal('test-value-1');
+          outerResolve();
+        }).catch(function(error) {


.catch(reject)

bajtos · 2017-01-09T15:18:41Z

server/current-context.js

+      if (ns && ns.active) {
+        /**
+         * **NOTE**
+         * This only re-binds get and set methods, the most used.


I am worried about breaking applications that expects getCurrentContext to return a value that is not bound yet.

How about introducing a new API for getting the bound context? E.g. loopbackContext.getBoundContext? Alternatively, we can add an options object to getCurrentContext, e.g. loopbackContext.getCurrentContext({ bind: true }).

Thoughts?

josieusa · 2017-01-09T17:03:05Z

@bajtos all done. I chose the getCurrentContext({bind: true}) alternative. PTAL

josieusa · 2017-01-11T16:35:11Z

Just one question: is forEeach blocking in Node (if you don't do I/O inside the callback)? I've read somewhere that it's blocking. The getFailureCount() function in my test relies on the assumption that forEach is blocking.

josieusa · 2017-01-11T23:58:57Z

I read the spec. Polyfills of forEach (not map) in Javascript code that follow the spec and use process.nextTick (so they are non-blocking) are possible. But I think that all native implementations are blocking. So it's ok to use forEach and return immediately after.

josieusa · 2017-01-17T13:01:36Z

Hello,
I think I'm done with this PR.
Comments?

bajtos · 2017-01-17T14:44:13Z

Hi @josieusa, I was thinking about this problem for some time now and before I took a closer look at the code again, I have one concern I'd like to discuss with you.

The solution proposed here "fixes" the lost context for the code inside a function that 1) started with the correct context 2) then called context-breaking code. However, it does not fix the lost context for any code that's called after this function.

Here is an example to illustrate what I mean. It uses a middleware, but a similar example can be written using remote hooks, operation hooks or a remote method.

app.use(function(req, res, next) {
  var ctx = LoopBackContext.getCurrentContext({ bind: true });
  doSomethingThatBreaksContext(function(err) {
    if (err) return next(err);
    // cool, ctx.get() still works
    next();
  });
});

app.use(function(req, res, next) {
  var ctx = LoopBackContext.getCurrentContext({ bind: true });
  // ouch, the context broken :(
});

I think the solution is to bind all callback invocation to the current context too, i.e.

app.use(function(req, res, next) {
  var ctx = LoopBackContext.getCurrentContext({ bind: true });
  next = ctx.bind(next); // <-- IMPORTANT!
  doSomethingThatBreaksContext(function(err) {
    if (err) return next(err);
    // cool, ctx.get() still works
    next();
  });
});

app.use(function(req, res, next) {
  var ctx = LoopBackContext.getCurrentContext({ bind: true });
  // cool, ctx is good
});

Using this principle, perhaps it is better to recommend people to bind all their callbacks, instead of binding the context methods? Using your when-based test case, here is what I mean:

        var ctx = LoopBackContext.getCurrentContext();
        expect(ctx).is.an('object');
        ctx.set('test-key', pushedValue);
        var whenPromise = whenV377().delay(timeout);
        whenPromise.then(ctx.bind(function pullFromContextAndReturn() {
          var pulledValue = ctx && ctx.get('test-key');
          outerResolve({
            pulledValue: pulledValue,
            pushedValue: pushedValue,
          });
        })).catch(ctx.bind(reject));
      });

That way the new bind option is not needed at all.

Another idea to consider in case you prefer to keep the new bind option: perhaps getCurrentContext({ bind: true }) should bind the bind method too, to allow users to fix the CLS context when finishing via the supplied callback?

app.use(function(req, res, next) {
  var ctx = LoopBackContext.getCurrentContext({ bind: true });
  doSomethingThatBreaksContext(function(err) {
    if (err) return ctx.bind(next)(err);
    // cool, ctx.get() still works
    ctx.bind(next)();
  });
});

Thoughts?

josieusa · 2017-01-17T15:01:33Z

Another idea to consider in case you prefer to keep the new bind option: perhaps getCurrentContext({ bind: true }) should bind the bind method too, to allow users to fix the CLS context when finishing via the supplied callback?

I prefer this, if it gives you the freedom to call ctx.bind wherever you want, even after calling context-breaking code. This way, you also avoid reassigning a function argument, contrary to your example:
next = ctx.bind(next); // <-- IMPORTANT!
But I'll think about it, maybe there's an even easier way (or maybe not...)

bajtos · 2017-01-19T13:39:39Z

Another idea to consider in case you prefer to keep the new bind option: perhaps getCurrentContext({ bind: true }) should bind the bind method too, to allow users to fix the CLS context when finishing via the supplied callback?

I prefer this, if it gives you the freedom to call ctx.bind wherever you want, even after calling context-breaking code. This way, you also avoid reassigning a function argument

Sounds good. Would you mind extending this pull request to bind ctx.bind too, including a unit-test and update of README?

josieusa · 2017-01-19T13:43:01Z

sure!

josieusa · 2017-01-23T11:53:02Z

Hello,
I tried binding next as in your example and it didn't work.
So I thought about a completely different solution.
I already have a test implementation and it works.
What if, instead of passing a boolean as the bind option value, we passed the request itself (or the response)? Like this:

var ctx = LoopBackContext.getCurrentContext({ bind: req });

That way, the first time that getCurrentContext() is called (i.e. in the first middleware in your example), its internal implementation could memoize the newly created, "good" context as a value in an ES6 WeakMap (using the passed request as the key for that value).

The second time that getCurrentContext()is called (i.e. in the second middleware in your example), by internally using the request as the WeakMap key again, it could return the memoized value, i.e. the same context as before.
The second time it avoids creating a new context altogether, and the risk of creating a "bad" context.

It's important to note that, in this example, this context instance is returned only for this request, and not for other concurrent requests. There is no mixing up.

Problems:
1.
~~WeakMap is not supported in Node 4. We could use a Symbol or a library that generates a random enough UID.~~

~~Then, in a middleware that must run before the others, we save the ID (or the Symbol) inside the request, like this:~~

~~app.use(function(req, res, next) {~~
~~var id = createId();~~
~~req.id = id;~~
~~});~~

~~Or maybe res.locals.id = id;, which is more idiomatic Express code (in this case you'd need to pass the response to getCurrentContext, and not the request).~~

~~Finally, inside the getCurrentContext() implementation, we use that ID (or Symbol) as the map key, but the map is a plain object, no longer a WeakMap.~~

~~It should delete the context from the map when the response is done.~~

EDIT maybe WeakMap is supported in Node 4, see http://node.green/

In runInContext there is no request. Neither in operation hooks. So you'd need to (generate and) pass an ID (or Symbol) to getCurrentContext, instead of the request.

The developer should delete the context from the map when the task is done.

EDIT Another way, which would be easier to use, would be to attach a "good" context instance to res.locals at the beginning of the middleware chain. Then you could get the context from the res object. But I don't like this option, since not all the routes need a context. If the user wishes to do so anyway, he/she could do it himself/herself in a user-defined middleware.

EDIT 2
This idea is not very different from using directly res.locals to store the values that you would otherwise store in the context. But res.locals is not available in runInContext().

josieusa · 2017-01-25T15:56:52Z

Regarding the solution that uses a WeakMap, in the case of operation hooks, since context.hookState is preserved across "before" and "after" hooks, we could use that object as the key for the WeakMap, couldn't we?
That way, you would write:

MyModel.observe('before save', (context, next) => {
  var clsContext = LoopbackContext.getCurrentContext({bind: context});
  ...
}

Why not use hookState directly? Mostly for legacy code.

bajtos · 2017-01-26T15:56:21Z

I tried binding next as in your example and it didn't work.

Weird. Are you referring to my code snippets from #19 (comment)? They don't show the implementation in getCurrentContext. If simple boundContext.bind = ns.bind(ns.bind) does not work, then how about the following?

boundContext.bind = ns.bind(function(fn, context) {
  return ns.bind.apply(ns, arguments);
});

The first time that getCurrentContext() is called (i.e. in the first middleware in your example), its internal implementation could memoize the newly created, "good" context as a value in an ES6 WeakMap (using the passed request as the key for that value).

The second time that getCurrentContext()is called (i.e. in the second middleware in your example), by internally using the request as the WeakMap key again, it could return the memoized value, i.e. the same context as before.
The second time it avoids creating a new context altogether, and the risk of creating a "bad" context.

I am afraid there is no such thing like a good or a bad context instance, at least as far as my understanding of CLS goes. The ns object (the context) is a constant wrapper that resolves CLS only at the time when get/set/bind methods are called. So if you attach a "good" context to req, and then look it up when the CLS chain is broken, the "good" context will actually become a bad one.

Also code that has access to req can store any extra per-request data on the req object directly and doesn't need CLS.

TBH, I feel like we are over-engineering this. CLS will never work 100% reliably until it's fixed in Node.js core. We need to draw a line where adding extra reliability is not worth the required increase in code/usage complexity.

In that light, I am proposing to land this patch as it is now, because it provides a nice small improvement that's worth it.

Thoughts?

josieusa · 2017-01-26T16:08:54Z

If simple boundContext.bind = ns.bind(ns.bind) does not work

That's exactly the case. At least for me. I'll try your new example instead, and let you know.

code that has access to req can store any extra per-request data on the req object directly and doesn't need CLS

Yes, either in req or in res.locals. But operation hooks don't have direct access to the request, and legacy code can still use this package there. Same for runInContext().

I am afraid there is no such thing like a good or a bad context instance

When I said, "good instance", I meant that the issue was fixed by re-using the same instance. Of course that doesn't mean that the instance can't get broken in different ways; in fact it could (I didn't test the open connection pool issues yet, for example). The practical answer is to write tests on a case-by-case basis.

In that light, I am proposing to land this patch as it is now, because it provides a nice small improvement that's worth it.

If we don't patch the "two middlewares" use case in your example, I feel that something is missing. At least some documentation about that case. I'll take a few days to keep thinking about it, while I try your new example.

bajtos · 2017-01-30T09:33:58Z

In that light, I am proposing to land this patch as it is now, because it provides a nice small improvement that's worth it.

If we don't patch the "two middlewares" use case in your example, I feel that something is missing. At least some documentation about that case. I'll take a few days to keep thinking about it, while I try your new example.

Sounds good. Take your time and let me know when you come up with something.

josieusa · 2017-02-22T16:26:18Z

Hello,
at last, I had the time to test your advice, and it seems to work!

boundContext.bind = ns.bind(function(fn, context) {
  return ns.bind.apply(ns, arguments);
});

I'll take my time to understand the magic, make more tests, and then merge.

josieusa · 2017-02-22T16:46:07Z

Hello again,
I think I understood the magic in your snippet.
In fact, this seems to work too:

// Call to Function.prototype.bind(), not ns.bind()
boundContext.bind = ns.bind.bind(ns);
boundContext.get = boundContext.bind(ns.get);
boundContext.set = boundContext.bind(ns.set);

😎😎😎😎😎😎😎😎
EDIT I'll write a test case for the "2 middlewares" use case and then merge
EDIT 2: in order to make it work, you'll need this line at the beginning of the "bad" middleware:
next = context.bind(next);

josieusa · 2017-02-23T08:42:00Z

@bajtos PTAL
Do you think that the test needs to be made more readable, for example?
Thanks a lot!

josieusa · 2017-02-23T11:17:30Z

This is an alternative snippet to next = ctx.bind(next);:

othiym23/node-continuation-local-storage#29 (comment)

The idea is the same. It uses Namespace.prototype.bind(), too.
But it patches the middleware function object, instead of the function body.
I think the PR is quite ok as it is now.

- fix jsdoc for getCurrentContext - clean up getCurrentContext code - clean up the new test

bajtos · 2017-02-24T09:25:31Z

@josieusa thank you for the update. I did few minor imrovements directly myself - see 442d77f. I you are happy with the current patch, then please squash the commits and we can land this pull request.

crandmck added the #community contribution label Jan 3, 2017

This was referenced Jan 3, 2017

No context available when using express-session with Memory store #6

Closed

[BUG] "when" 3.7.7 package makes context get mixed up between concurrent HTTP requests #17

Closed

bajtos self-assigned this Jan 3, 2017

bajtos requested changes Jan 3, 2017

View reviewed changes

josieusa closed this Jan 4, 2017

josieusa force-pushed the feature/workaround-issue-17 branch from e9ba79a to db9387e Compare January 4, 2017 08:36

crandmck removed the #community contribution label Jan 4, 2017

josieusa reopened this Jan 4, 2017

crandmck added the #community contribution label Jan 4, 2017

josieusa force-pushed the feature/workaround-issue-17 branch from 35d541e to 9f3dfca Compare January 4, 2017 11:59

cgole removed the #community contribution label Jan 7, 2017

bajtos requested changes Jan 9, 2017

View reviewed changes

josieusa added 4 commits January 10, 2017 09:56

Workaround issue strongloop#17 with cujojs/when

cec955c

Fix corner case of missing context

53f68c5

Use local stub module for multi-versions

1216ca4

Remove call to Function.prototype.bind()

dee3e4e

Simplify test using whenPromise.delay()

47daf07

bajtos approved these changes Jan 26, 2017

View reviewed changes

bajtos added feature waiting labels Jan 30, 2017

josieusa added 2 commits February 22, 2017 18:15

Fix mixed up ctx in middlewares after a bad one

22f8a46

Add test for bad middleware chain use case

10df035

josieusa added 4 commits February 23, 2017 10:02

Document case of mixed up ctx in middleware chains

b1ec133

Improve README

688cb9c

Move middleware example into usage section

1d2197f

Explain reason for bug fix

244627a

minor improvements

442d77f

- fix jsdoc for getCurrentContext - clean up getCurrentContext code - clean up the new test

josieusa merged commit d5f977f into strongloop:master Feb 24, 2017

josieusa removed the waiting label Feb 24, 2017

Add bind:true option to getCurrentContext #19

Add bind:true option to getCurrentContext #19

Conversation

josieusa commented Jan 3, 2017 • edited Loading

Description

Related issues

Checklist

slnode commented Jan 3, 2017

slnode commented Jan 3, 2017

slnode commented Jan 3, 2017

Choose a reason for hiding this comment

josieusa Jan 3, 2017 • edited Loading

Choose a reason for hiding this comment

josieusa Jan 3, 2017 • edited Loading

Choose a reason for hiding this comment

josieusa Jan 4, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

josieusa Jan 3, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

slnode commented Jan 4, 2017

slnode commented Jan 4, 2017

slnode commented Jan 4, 2017

josieusa commented Jan 4, 2017 • edited Loading

jainisenough commented Jan 4, 2017 • edited Loading

josieusa commented Jan 4, 2017

josieusa commented Jan 4, 2017 • edited Loading

jainisenough commented Jan 5, 2017

josieusa commented Jan 5, 2017

bajtos left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

josieusa commented Jan 9, 2017

josieusa commented Jan 11, 2017 • edited Loading

josieusa commented Jan 11, 2017

josieusa commented Jan 17, 2017

bajtos commented Jan 17, 2017

josieusa commented Jan 17, 2017

bajtos commented Jan 19, 2017

josieusa commented Jan 19, 2017

josieusa commented Jan 23, 2017 • edited Loading

josieusa commented Jan 25, 2017 • edited Loading

bajtos commented Jan 26, 2017

josieusa commented Jan 26, 2017 • edited Loading

bajtos commented Jan 30, 2017

josieusa commented Feb 22, 2017

josieusa commented Feb 22, 2017 • edited Loading

josieusa commented Feb 23, 2017

josieusa commented Feb 23, 2017

bajtos commented Feb 24, 2017

josieusa commented Jan 3, 2017 •

edited

Loading

josieusa Jan 3, 2017 •

edited

Loading

josieusa Jan 3, 2017 •

edited

Loading

josieusa Jan 4, 2017 •

edited

Loading

josieusa Jan 3, 2017 •

edited

Loading

josieusa commented Jan 4, 2017 •

edited

Loading

jainisenough commented Jan 4, 2017 •

edited

Loading

josieusa commented Jan 4, 2017 •

edited

Loading

josieusa commented Jan 11, 2017 •

edited

Loading

josieusa commented Jan 23, 2017 •

edited

Loading

josieusa commented Jan 25, 2017 •

edited

Loading

josieusa commented Jan 26, 2017 •

edited

Loading

josieusa commented Feb 22, 2017 •

edited

Loading