Heatmaps layer #5253

mourner · 2017-09-07T15:04:57Z

Closes #4756. Still a work in progress, but pretty close.

Technical tasks

Things I need help with

review Refactor fill extrusion texture/fbo cache #5328 to unblock color ramp texture caching
find a less hacky way to set up the non-zoom curve property (for the color ramp) @anandthakker Add ['heatmap-density'] expression #5350
refactor StyleLayer to handle functions as default properties @anandthakker
~~render heatmap intermediate texture in a separate pass (3d?)~~ decided with @lbud @mollymerp to follow up on this after both this and raster DEM PRs are merged
add an official example @ryanbaumann

anandthakker · 2017-09-19T15:17:19Z

mourner · 2017-09-21T16:28:55Z

Performance is looking pretty good at this point — here's a sample JS profiler breakdown (a heavy clustered heatmaps layer takes only 80% more than the background):

kkaefer · 2017-09-22T13:37:04Z

src/style-spec/reference/v8.json

+          "doc": "A heatmap.",
+          "sdk-support": {
+            "basic functionality": {
+            }


All of these in this file need a version number before merging.

Should I include the number of the next would-be release, e.g. 0.41 here for JS?

kkaefer · 2017-09-22T13:39:17Z

src/style-spec/reference/v8.json

+      "property-function": false,
+      "transition": true,
+      "units": "pixels",
+      "doc": "Heatmap radius.",


Can we please improve the documentation for all of the new values in this document?

Yes, that's still in the checklist as you can see.

kkaefer · 2017-09-22T13:55:08Z

src/style/style_layer/heatmap_style_layer.js

+                this.colorRampData[i + 0] = Math.floor(pxColor[0] * 255 / alpha);
+                this.colorRampData[i + 1] = Math.floor(pxColor[1] * 255 / alpha);
+                this.colorRampData[i + 2] = Math.floor(pxColor[2] * 255 / alpha);
+                this.colorRampData[i + 3] = Math.floor(alpha * 255);


This seems bogus. getPaintValue colors have components in the range of 0...1, so we have to multiply them with 255 to add them to the texture buffer. Let's take a pxColor value of (1, 1, 1, 0.2). The result R value would be 1 * 255 / 0.1 == 2550, which is beyond the value range of Uint8Array.

It also seems that you're premultiplying the colors here. However, with WebGL, we're using non-premultiplied colors in our RGBImage data structure:

mapbox-gl-js/src/util/image.js

Lines 98 to 100 in 8fea8d6

// Not premultiplied, because ImageData is not premultiplied.

// UNPACK_PREMULTIPLY_ALPHA_WEBGL must be used when uploading to a texture.

class RGBAImage {

Instead of manually premultiplying here, we should take the advice and use UNPACK_PREMULTIPLY_ALPHA_WEBGL when uploading later on.

E.g. a color ramp of

"stops": [ [0, "rgba(255, 0, 0, 0)"], [1, "rgba(0, 0, 255, 1)"], ]

only shows blue, and doesn't even show a glimpse of red as it's interpolated from red to blue.

It'd also be cool to have unit tests for generating color ramp textures/buffers from a style specification.

Thanks for the explanation — I'll look more closely at the premultiplying stuff.

It'd also be cool to have unit tests for generating color ramp textures/buffers from a style specification.

I think it would be enough to cover this with a render test, since the results are visual, not something where a difference would be useful to see as numbers in the console. What do you think?

Per chat, the code works correctly — leaving it as is as a weird special case because we need to un-premultiply the premultiplied colors returned by getPaintValue because the Texture class later premultiplies them when uploading again.

kkaefer · 2017-09-22T13:55:28Z

src/style/style_layer/heatmap_style_layer.js

+
+    constructor(layer: LayerSpecification) {
+        super(layer);
+        this.colorRampData = new Uint8Array(256 * 4);


Let's use a Uint8ClampedArray here.

Interestingly, WebGL doesn't like Uint8ClampedArray as texture input — it produces texture.js:47 WebGL: INVALID_OPERATION: texImage2D: type UNSIGNED_BYTE but ArrayBufferView not Uint8Array.

kkaefer · 2017-09-22T14:02:38Z

debug/heatmap.html

+            }
+        }
+    });
+});


Can we add the layer below labels please?

Doesn't matter since it's just a debug page. The official example definitely should have labels on top though.

Yeah, but people still look at this, and we should set a good example here.

kkaefer · 2017-09-22T14:14:05Z

src/render/draw_heatmap.js

+    for (let i = 0; i < coords.length; i++) {
+        const coord = coords[i];
+
+        // skip tiles that have uncovered parents to avoid flickering


Can you please expand on the reasoning of this a little more? And potentially add a render test?

Certainly needs a render test, although it will be tricky to make because of the need to manually make 4 new tile fixtures. The situation here is similar to #5004. The difference is that heatmaps don't need such a complex fix (tile masks) — the difference between tiles of different zooms for heatmaps is much more subtle since it's still vector data, so waiting for all 4 children to load before putting them in place of a parent is an acceptable solution to the problem that's also faster and much simpler.

kkaefer · 2017-09-22T14:20:37Z

src/render/draw_heatmap.js

+    // large circles are not clipped to tiles
+    gl.disable(gl.STENCIL_TEST);
+
+    renderToTexture(gl, painter, layer);


Can we please move this call to a different render pass to improve performance? Extrusion fills use a 3d render pass to render the texture before rendering to the main FBO. See mapbox/mapbox-gl-native#9286 for a description of this optimization, and #5101 for @lbud implemented this for extrusion fills.

@kkaefer @lbud should we call this pass differently then? e.g. prerender or something?

Yes, @mollymerp and I were just talking about this yesterday, as hillshades use a pre-render pass as well — it would make sense to rename the 3d pass to prerender and carefully move relevant parts of the current 3d pass (I suppose for now we could move them into draw_fill_extrusion.js, although eventually if we add other 3D types it'll be moved to a file where it can be shared between types).

anandthakker

@mourner some questions and nitpicks inline, but this looks great! I love how simple the code ended up being.

anandthakker · 2017-09-22T17:50:53Z

src/style/style_layer/heatmap_style_layer.js

+    _applyPaintDeclaration(name: any, declaration: any, options: any, globalOptions: any, animationLoop: any, zoomHistory: any) {
+        super._applyPaintDeclaration(name, declaration, options, globalOptions, animationLoop, zoomHistory);
+        if (name === 'heatmap-color') {
+            const len = this.colorRampData.length;


Does this need to happen in _applyPaintDeclaration, or could it happen instead in setPaintProperty()? It looks to me like the latter would work, and I think it'd be preferable to override setPaintProperty rather than the private _applyPaintDeclaration

I actually started with setPaintProperty but later moved to this, I don't remember why — will recheck.

Oh, I think I remember now — the problem is you can't call getPaintValue immediately after setPaintProperty, because it's being set up in a separate "apply paint transitions" stage.

Ah, okay. Could we add a comment about that here, so that no one is tempted to try such a refactor later on?

anandthakker · 2017-09-22T18:14:12Z

src/render/draw_heatmap.js

+        const parentTile = sourceCache.findLoadedParent(coord, 0, {});
+        if (parentTile) {
+            const parentId = parentTile.coord.id;
+            if (sourceCache._tiles[parentId] && !sourceCache._coveredTiles[parentId]) continue;


Can we extract this check into one or more SourceCache methods to avoid accessing its private state?

Yes, good idea.

anandthakker · 2017-09-22T18:18:55Z

src/render/draw_heatmap.js

+        const programConfiguration = bucket.programConfigurations.get(layer.id);
+        const program = painter.useProgram('heatmap', programConfiguration);
+        programConfiguration.setUniforms(gl, program, layer, {zoom: painter.transform.zoom});
+        gl.uniform1f(program.uniforms.u_radius, layer.paint['heatmap-radius']);


layer.getPaintProperty('heatmap-radius') (similar for other uses of layer.paint)

You mean getPaintValue, right? Is there an advantage in case of non-DDS properties? I thought that would be faster.

Yep sorry getPaintValue. No advantage for non-DDS, but at some point I think we want to get rid of StyleLayer.{paint,layout} (or hide it), see #3044. I don't think the performance difference would be enough to matter in code that's executed only once per draw.

anandthakker · 2017-09-22T19:33:10Z

src/render/draw_heatmap.js

+        gl.uniform1f(program.uniforms.u_radius, layer.paint['heatmap-radius']);
+
+        const scale = painter.transform.zoomScale(tile.coord.z - painter.transform.zoom);
+        gl.uniform1f(program.uniforms.u_extrude_scale, EXTENT / tile.tileSize * scale);


I believe this is equivalent to pixelsToTileUnits(tile, 1, painter.transform.zoom)

anandthakker · 2017-09-22T19:47:50Z

src/shaders/heatmap.fragment.glsl

+
+    // Kernel density estimation with a Gaussian kernel of size 5x5
+    float d = -0.5 * 5.0 * 5.0 * dot(v_extrude, v_extrude);
+    float val = weight * u_intensity * GAUSS_COEF * exp(d);


I think having two separate properties for heatmap-intensity and heatmap-weight may be a bit confusing. Is the reason for this in order to decouple the zoom-dependent and feature-dependent dimensions of this multiplier?

Yes, exactly. I thought about this a lot and considered making this one property, but it's just too inconvenient to use — you have to construct complex expressions or zoom+property functions and it's just hard to do intuitively. This separation though fits pretty well into how I think about heatmaps. Another more practical benefit is that you can change heatmap intensity in real time without recomputing all the buffers.

Yeah, I guess the only alternative would be that the expression would have to be of the form: feature-only-expression or zoom-only-expression or ['*', feature-only-expression, zoom-only-expression]... which is probably just as confusing (or more) as having two properties, heh

anandthakker · 2017-09-22T20:08:50Z

src/shaders/heatmap.fragment.glsl

+
+    // Kernel density estimation with a Gaussian kernel of size 5x5
+    float d = -0.5 * 5.0 * 5.0 * dot(v_extrude, v_extrude);
+    float val = weight * u_intensity * GAUSS_COEF * exp(d);


I don't think the 5^2 factor here technically translates to a kernel size of 5x5. Here's my read/analysis to make sure I'm understanding this right:

The "bandwidth" of the Gaussian kernel (which is also its standard deviation) would be the value h in 1/2pi * exp( -0.5 * (u/h)^2 )

Since v_extrude is essentially in units of radii (i.e. at a distance of u_radius from the point, v_extrude is a unit vector), the calculation above is equivalent to weight * u_intensity * GAUSS_COEF * exp( (distance_in_pixels / (radius / 5)^ ), making the bandwidth radius/5.

Three standard deviations is a pretty reasonable definition for the practical "size" of a Gaussian kernel. In this case, that would be: 3/5 * radius.

Yeah, I noted that "radius" is quite a blurry term when referring to kernel density Gaussian kernels, but haven't thought this through further. Given the thinking above, how do you think we should change the formula in the shader?

I think changing 5.0 to 3.0 would be pretty reasonable. Then radius would have a pretty clean meaning:

For advanced users familiar w/ the underlying math, "3 standard deviations of the Gaussian kernel"

For most users, "the distance around each point where it influences the heatmap" (or something like that)

Hmm, experimenting with this a bit, one issue is that with if we use 3.0, then for values of intensity * weight > 0.5, the output for a point ends up getting clipped by the square mesh for that point. Whereas with 5.0, it takes intensity * weight > 5000 to make that happen.

I guess the problem is that, even at x = 3 standard deviations, the gaussian curve evaluates to 0.0018, which is over half of the 1/255 resolution of the color ramp.

Yeah, now I remember that empirically 5.0 is reasonable enough to make clipping artifacts not to be noticeable. Picking more, the performance will suffer because of the number of fragments. Any ideas on how to do better here?

We can't restrict the weight, otherwise you won't be able to use clustering with heatmaps. If there's a cluster with 1000 points, it's weight should be 1000 times the single point weight.

Other option would be to pick another type of kernel (not Gaussian) for the heatmap, or modify the Gaussian to fade out to 0 at the borders regardless of the weight.

Tried Epanechnikov and the heatmap looks pretty bad. Maybe we could both calculate bandwidth and increase the size of the kernel depending on weight in the vertex shader? That way small intensities wouldn't require as many fragments, which would increase performance, and big intensities which cause big kernels wouldn't appear too often.

@mourner Actually if we use a weight-based adjustment to the size of the kernel (i.e., the size of the mesh produced by the vertex shader), then we could go back to just using a simple, fixed value of 3.0 for the bandwidth. I.e., in the vertex shader,

v_extrude = vec2(mod(a_pos, 2.0) * 2.0 - 1.0) * scale_based_on_weight;

And then in fragment

float d = -0.5 * h * h * dot(v_extrude / scale_based_on_weight, v_extrude / scale_based_on_weight);

Where scale_based_on_weight is a scale factor that grows the kernel large enough beyond radius so that the kernel falls effectively to zero at the edge

@anandthakker exactly, that's what I was thinking too! Now we need to determine that scale properly...

anandthakker · 2017-09-24T11:31:11Z

src/render/draw_heatmap.js

+    gl.clearColor(0, 0, 0, 0);
+    gl.clear(gl.COLOR_BUFFER_BIT);
+
+    gl.blendFunc(gl.ONE, gl.ONE);


Would be good to add a comment here to explain the significance of this blend function, since it's essential to how the kernel density method works (and maybe some comments in general on this method)

anandthakker · 2017-09-24T11:34:17Z

src/render/draw_heatmap.js

+        gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
+        gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
+        gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGB, painter.width / 4, painter.height / 4, 0, gl.RGB,
+            painter.extTextureHalfFloat ? painter.extTextureHalfFloat.HALF_FLOAT_OES : gl.UNSIGNED_BYTE, null);


What's the purpose of using HALF_FLOAT_OES when it's supported? Is it to increase the precision we use for kde values?

Yes. Try disabling it and see how the heatmap looks. Better precision allows for a much smoother look.

anandthakker · 2017-09-24T11:37:34Z

src/shaders/heatmap.fragment.glsl

+// Gaussian kernel coefficient: 1 / sqrt(2 * PI)
+#define GAUSS_COEF 0.3989422804014327
+
+void main() {


Suggest renaming this program from heatmap to heatmap_density, and then renaming heatmap_texture to just heatmap

Not sure those names would be clearer. Also, I named them similarly to the extrusion names — fill_extrusion and extrusion_texture.

anandthakker

@mourner this is looking great!

Only one main suggestion: could we change the integration tests to use a simplified geojson source (maybe with, e.g., just 3 or 4 points) rather than a vector tile fixture? I think that would allow the tests to be more easily inspected & evaluate for correctness just by comparing the style definition and the resulting image.

anandthakker · 2017-10-04T18:46:12Z

src/style/style_layer/heatmap_style_layer.js

+    _applyPaintDeclaration(name: any, declaration: any, options: any, globalOptions: any, animationLoop: any, zoomHistory: any) {
+        super._applyPaintDeclaration(name, declaration, options, globalOptions, animationLoop, zoomHistory);
+        if (name === 'heatmap-color') {
+            const len = this.colorRampData.length;


Ah, okay. Could we add a comment about that here, so that no one is tempted to try such a refactor later on?

mourner · 2017-10-04T20:57:56Z

@anandthakker I intentionally used fixtures with a bunch of points instead of a basic GeoJSON to have a good variety of point densities. 3-4 points might not exercise all the edge cases of points overlapping.

BTW, looks like it needs another rebase now :(

1ec5

A couple minor documentation comments. Excited to see this feature come so far along!

1ec5 · 2017-10-05T01:46:54Z

src/style-spec/reference/v8.json

@@ -1972,6 +2001,10 @@
        "doc": "Gets the current zoom level.  Note that in style layout and paint properties, [\"zoom\"] may only appear as the input to a top-level [\"curve\"] expression.",
        "group": "Zoom"
      },
+      "heatmap-density": {
+        "doc": "Gets the density of a pixel in a heatmap layer. Can only be used in `heatmap-color` property.",


Nit: add “the” before heatmap-color.

1ec5 · 2017-10-05T01:48:34Z

src/style-spec/reference/v8.json

@@ -1972,6 +2001,10 @@
        "doc": "Gets the current zoom level.  Note that in style layout and paint properties, [\"zoom\"] may only appear as the input to a top-level [\"curve\"] expression.",
        "group": "Zoom"
      },
+      "heatmap-density": {
+        "doc": "Gets the density of a pixel in a heatmap layer. Can only be used in `heatmap-color` property.",


As a lay reader, I’d have little idea what it means for a pixel to have density – other than pixel density, which is a different concept. Any idea how to explain density in the context of a heatmap?

1ec5 · 2017-10-05T01:51:20Z

src/style-spec/reference/v8.json

+              [1, "red"]
+          ]
+      },
+      "doc": "Defines the color of each pixel based on its density value in a heatmap. Should be either a stop function with input values ranging from `0` to `1`, or a curve expression with a special `[\"heatmap-density\"]` keyword as the input.",


Noting that this explanation is a bit specific to the GL JS function and expression syntaxes, so the iOS/macOS SDK will need to override it with the appropriate syntax.

* Add heatmap-density expression (prepares for #5253) * Accept options for context in which expression is used Prepares for #5193 * Unify creatExpression and createExpressionWithErrorHandling Closes #5409 * declaration => property

Also change kernel bandwidth from 5.0 to 3.0

* Add ['heatmap-density'] expression * Fix lint

kkaefer · 2017-10-06T15:23:30Z

src/render/draw_circle.js

@@ -45,8 +44,6 @@ function drawCircles(painter: Painter, sourceCache: SourceCache, layer: CircleSt
            gl.uniform2fv(program.uniforms.u_extrude_scale, painter.transform.pixelsToGLUnits);
        }

-        gl.uniform1f(program.uniforms.u_devicepixelratio, browser.devicePixelRatio);


Why is this no longer necessary?

It's an unused leftover — we now define DEVICE_PIXEL_RATIO as a shader constant in the shader generation code.

kkaefer · 2017-10-06T15:24:13Z

src/render/draw_heatmap.js

+    gl.activeTexture(gl.TEXTURE1);
+
+    // Use a 4x downscaled screen texture for better performance
+    gl.viewport(0, 0, painter.width / 4, painter.height / 4);


Do we need to account for the device pixel ratio here? E.g. do we need a "retina" version of the downscaled texture?

I think not because painter.width and painter.height already have "retina" values.

kkaefer · 2017-10-06T15:25:31Z

src/render/draw_heatmap.js

+    gl.bindFramebuffer(gl.FRAMEBUFFER, fbo);
+    gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, texture, 0);
+
+    // If using half-float texture as a render target is not supported, fall back to a low precision texture


Is there any way we can test unavailability of this extension? For VAOs, I think we have code that specifically prevents reporting that this extension is available so that we can run the test with it enabled and disabled.

Unfortunately the gl module which we use for tests doesn't seem to support the extension, so we're forced to only test the "fallback" state.

Also, this check is not about availability of the halfFloat extension, it's about being able to use halfFloat textures as a render target (which some GPUs simply don't support while they support the extension). So the only way to know that is to try, and then check the framebuffer attachment status (this is even officially documented somewhere).

Found this thread on the matter https://groups.google.com/forum/#!topic/webgl-dev-list/ms21hnWk0tg

mourner added the under development label Sep 7, 2017

mourner requested a review from kkaefer September 7, 2017 15:04

mourner mentioned this pull request Sep 7, 2017

Proposal: Heatmap layer type #4756

Closed

mourner force-pushed the heatmaps branch 2 times, most recently from c50308e to 000975c Compare September 14, 2017 12:49

mourner mentioned this pull request Sep 19, 2017

Refactor fill extrusion texture/fbo cache #5328

Merged

5 tasks

mourner force-pushed the heatmaps branch from 60130cb to a76e4a7 Compare September 21, 2017 14:43

mourner requested a review from anandthakker September 21, 2017 17:17

kkaefer requested changes Sep 22, 2017

View reviewed changes

anandthakker reviewed Sep 22, 2017

View reviewed changes

anandthakker suggested changes Sep 24, 2017

View reviewed changes

mourner force-pushed the heatmaps branch from 77fe319 to 5059738 Compare September 29, 2017 09:18

anandthakker force-pushed the heatmaps branch from 2f60811 to 6571615 Compare October 2, 2017 16:06

mourner added release blocker ⛔ and removed under development labels Oct 4, 2017

anandthakker reviewed Oct 4, 2017

View reviewed changes

1ec5 reviewed Oct 5, 2017

View reviewed changes

anandthakker pushed a commit that referenced this pull request Oct 5, 2017

Add heatmap-density expression (prepares for #5253)

4bc29ed

anandthakker mentioned this pull request Oct 5, 2017

Preparatory expression refactors #5407

Merged

anandthakker pushed a commit that referenced this pull request Oct 5, 2017

Add heatmap-density expression (prepares for #5253)

d70a7db

anandthakker pushed a commit that referenced this pull request Oct 5, 2017

Add heatmap-density expression (prepares for #5253)

04109a0

mourner added 5 commits October 6, 2017 18:03

WIP dump heatmap layer

0516123

remove redundant heatmap attributes

2d361be

cleanup heatmap style reference

2e9a43e

stop-based heatmap color ramps

338127e

simplify heatmap debug page; remove weight scale

40892b7

mourner and others added 19 commits October 6, 2017 18:03

use texture class for heatmaps

77afbf4

fix bug with heatmap resizing

02bf664

cache color ramp texture

7bdca0c

hack to get rid of heatmap flicker

fcb2886

cache viewport buffer/vao

1be3441

heatmap fallback when no half-floats available

36064b9

fix flow errors and some caught bugs

03fe9f8

update error message fixtures

bc22520

Scale mesh to ensure it's large enough for kernel to drop to zero

709cc7b

Also change kernel bandwidth from 5.0 to 3.0

use a smaller heatmap zero for less artifacts

7ffca2e

nicer heatmap debug page

7be0d3a

add more heatmap code comments

390f851

more heatmap code cleanup

8e379b9

gracefully fallback on bad half-float framebuffer support

aa4b02f

fix infinite loop in heatmap code

66fbdce

Add ['heatmap-density'] expression (#5350)

23e3ad3

* Add ['heatmap-density'] expression * Fix lint

better heatmap docs

9f7c829

fix heatmap docs generation

2331136

add heatmap render tests

f50affa

mourner force-pushed the heatmaps branch from 6571615 to f50affa Compare October 6, 2017 15:04

clarify heatmap-density description

fd9431f

anandthakker approved these changes Oct 6, 2017

View reviewed changes

add a comment about overriding setPaintProperty

049e9bf

kkaefer approved these changes Oct 6, 2017

View reviewed changes

Special case handling for heatmap-color expression default values

a9ef898

mourner mentioned this pull request Oct 6, 2017

Heatmap layer mapbox/mapbox-gl-native#10146

Closed

add a test for heatmap color expression

13bd39a

mourner merged commit 15a0db7 into master Oct 6, 2017

mourner deleted the heatmaps branch October 6, 2017 16:49

mourner mentioned this pull request Oct 6, 2017

An official example for heatmaps #5420

Closed

	// Not premultiplied, because ImageData is not premultiplied.
	// UNPACK_PREMULTIPLY_ALPHA_WEBGL must be used when uploading to a texture.
	class RGBAImage {

Heatmaps layer #5253

Heatmaps layer #5253

Conversation

mourner commented Sep 7, 2017 • edited Loading

Technical tasks

Things I need help with

anandthakker commented Sep 19, 2017

mourner commented Sep 21, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anandthakker left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anandthakker Sep 22, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anandthakker Sep 24, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mourner Sep 24, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anandthakker left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mourner commented Oct 4, 2017 • edited Loading

1ec5 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mourner commented Sep 7, 2017 •

edited

Loading

mourner commented Sep 21, 2017 •

edited

Loading

anandthakker Sep 22, 2017 •

edited

Loading

anandthakker Sep 24, 2017 •

edited

Loading

mourner Sep 24, 2017 •

edited

Loading

mourner commented Oct 4, 2017 •

edited

Loading