Introduce Capped distances to evaluate the indices of pairs within a fixed radius #1941

ayushsuhane · 2018-06-15T01:27:20Z

As discussed with @richardjgowers and @jbarnoud , the main idea is to create a functionality which opens up the method to evaluate the distances, as an optional keyword, whereas the default way would be to switch to the best method automatically based on the performance rules obtained from the benchmarks. Here, an initial functionality of capped distances is included using brute force method. The rules could be as simple as if len(selection) > 1000 or more complicated based on the cutoff radius.

PR Checklist

Tests?
Docs?
CHANGELOG updated?
Issue raised/referenced?

coveralls · 2018-06-15T01:50:07Z

Coverage decreased (-0.07%) to 89.884% when pulling 1068578 on ayushsuhane:cappedfeature into 1a1b2f1 on MDAnalysis:develop.

jbarnoud · 2018-06-15T06:26:04Z

If all your capped distance methods have the same signature, you can have them in a dictionary which would avoid a long if...elif...elif...:

distance_methods = {
    'bruteforce': _capped_brutefore,
    'pkdt': _capped_pkdt,
    'octree': _octree,
}
if method is None:
    distance_function = _determine_method(...)
elif method.lower() in distance_methods:
    distance_function = distance_methods[method.lower()]
else:
    raise ValueError('The is no known method called "{}".'.format(method))

An other benefit is that the method argument can take a callable with the same signature which makes your function extremely easy to play with: if I find an awesome method in an other library, I can write a small wrapper function that use the agreed signature, and I can use my alien function as a method without modifying MDAnalysis code.

richardjgowers · 2018-06-16T17:09:35Z

package/MDAnalysis/lib/distances.py

+
+    for i, coords in enumerate(reference):
+        dist = distance_array(coords[None, :], configuration, box=box)[0]
+        idx = np.where((dist <= max_cutoff) & (dist > min_cutoff))[0]


Min cutoff here means that it will never return a distance of 0, which sometimes it should. I think it'd be better to have min_cutoff=None as default and only apply this condition if it is defined.

Should it be None, or we can also put a 0 as a default and dist >= min_cutoff, which will catch the distance of 0 as well.

that would also work. it's also a needless comparison too though (performance)

Because of floating point precision, 0 could very much be -1e-16 which would be lesser than 0.

richardjgowers · 2018-06-16T17:10:39Z

package/MDAnalysis/lib/distances.py

+    _check_array(reference, 'reference')
+    _check_array(configuration, 'configuration')
+
+    for i, coords in enumerate(reference):


So this way of iterating over one coordinate set is probably slower than distance_array(ref, conf). I think guess_bonds does it this way to avoid memory problems

Yes exactly. The distance_array cannot go beyond 5k particles. So unless we are explicitly limiting the brute force algorithm to 5k, shouldn't the more robust way be a part of the calculation. I agree once we put in the rules, the iteration over each particle would be about half as fast as the distance_array and should be replaced.

richardjgowers · 2018-06-16T17:11:01Z

package/MDAnalysis/lib/distances.py

+        for j in idx:
+            pairs.append((i,j))
+
+    return pairs


These functions need to also return the distance values too, it's important

richardjgowers · 2018-06-16T17:13:26Z

Something which might be interesting, which follows @jbarnoud 's idea of a dict is to register distance calculation methods in a similar way to how we do Readers. So something like:

@distance_method('brute_force')
def brute_force():
    pass

@distance_method('pkd_tree')
def pkd_tree():
    pass

Where having the same signature becomes essential. This would let new methods be dropped in without having to mess inside MDA. Not sure how this would work with the guessing methods....

jbarnoud · 2018-06-17T08:46:08Z

Not sure how this would work with the guessing methods....

The guessing method an only work on the methods that are known as the method is written. The only risk there is if a distance method name is overwritten by the decorator and the guessing method refer to the distance method by shortcut name rather than by callback. If the guessing method refers to the functions directly, I see no problem with the decorator except maybe the shadow of over-engineering flying nearby.

ayushsuhane · 2018-06-17T21:34:54Z

It is little tricky to work with decorators. I will propose what I understood based on the comments. I can change accordingly in the files. I am also not sure whether it is sensible to place every capped functions in the distances.py or should they be seperated.
First, we need to define the decorator with arguments as:

def distance_method(*outer_args, **outer_kwargs):
    def decorator(function):
        def new_function(*args, **kwargs):
            pairs, distance = function(*args, **kwargs)
            return pairs, distance
        return new_function
    return decorator

The brute force for capped distances can be directly wrapped using the decorator as

@distance_method('brute_force')
def capped_bruteforce(reference, configuration, max_cutoff, min_cutoff = 0, box = None):
    pairs, distance = [], []
    if reference.shape == (3,):
        reference = reference[None, :]
    if configuration.shape == (3,):
        configuration = configuration[None, :]

    _check_array(reference, 'reference')
    _check_array(configuration, 'configuration')
    for i, coords in enumerate(reference):
        dist = distance_array(coords[None, :], configuration, box=box)[0]
        idx = np.where((dist <= max_cutoff) & (dist >= min_cutoff))[0]
        for j in idx:
            pairs.append((i,j))
            distance.append(dist[j])
    return pairs, distance

@distance_method('pkd_tree')
def pkd_tree():
    pass

and then the higher level function to calculate the pairs and distances, which will be called by the user.

def capped_distance(reference, configuration, max_cutoff, min_cutoff = 0, box = None, method = None):
    if box is not None:
        boxtype = _box_check(box)
        # Convert [A,B,C,alpha,beta,gamma] to [[A],[B],[C]]
        if (boxtype == 'tri_box'):
            box = triclinic_vectors(box)
        if (boxtype == 'tri_vecs_bad'):
            box = triclinic_vectors(triclinic_box(box[0], box[1], box[2]))
    
    distance_methods = {
    'brute_force': capped_bruteforce,
        'pkdtree': capped_pkdtree
    }
    
    if method is None:
        distance_function = distance_methods[_determine_method(reference, configuration, max_cutoff, box = box)]
    elif method.lower() in distance_methods:
        distance_function = distance_methods[method.lower()]
    else:
        raise ValueError('The is no known method called "{}".'.format(method))
    
    pairs, distance = distance_function(reference, configuration, max_cutoff, min_cutoff = 0,  box = box)
    return pairs, distance

richardjgowers · 2018-06-18T13:36:53Z

~~@utkbansal~~ @ayushsuhane on second thought, I think the decorators are just overcomplicating it for now, we can leave them out

richardjgowers · 2018-06-18T13:37:42Z

package/MDAnalysis/lib/distances.py

@@ -389,6 +389,118 @@ def self_distance_array(reference, box=None, result=None, backend="serial"):

    return distances

+def capped_distance(reference, configuration, max_cutoff, min_cutoff = 0, box = None, method = None):


the spaces for the kwargs don't follow pep8 here. something like flake8 etc can sort this out

utkbansal · 2018-06-18T13:42:54Z

@richardjgowers Did you mean to reply to someone else? 😄

richardjgowers · 2018-06-18T13:43:35Z

package/MDAnalysis/lib/distances.py

+    max_cutoff : float32
+        The maximum cutoff distance. Only the pair of indices which 
+        have smaller distance than this cutoff distance will be returned.
+    min_cutoff : float32


this needs to be documented as optional

richardjgowers

looking better but needs some fixes

richardjgowers · 2018-06-19T15:06:02Z

package/MDAnalysis/lib/distances.py

+
+    Returns
+    -------
+    indices : list


these should be returned as numpy arrays

richardjgowers · 2018-06-19T15:06:15Z

package/MDAnalysis/lib/distances.py

+        provided as ``cutoff`` in arguments.
+    distances : list
+        Distances corresponding to each pair of indices.
+


simple usage example here

richardjgowers · 2018-06-19T15:07:00Z

package/MDAnalysis/lib/distances.py

+    configuration : numpy.array of type numpy.float32
+        All the attributes are similar to reference array. Essentially
+        second group of atoms.
+    max_cutoff : float32


this is just a float, it doesn't have to be float32

richardjgowers · 2018-06-19T15:07:41Z

package/MDAnalysis/lib/distances.py

+
+    Parameters
+    -----------
+    reference : numpy.array of type numpy.float32


these two arrays don't need to be float32, it's just bruteforce which needs that. You can take care of the types in _bruteforce where it's necessary

richardjgowers · 2018-06-19T15:08:14Z

package/MDAnalysis/lib/distances.py

+
+    Note
+    -----
+    Currently only supports brute force. Similar methods


seealso distance_array

richardjgowers · 2018-06-19T15:08:46Z

package/MDAnalysis/lib/distances.py

@@ -390,6 +390,131 @@ def self_distance_array(reference, box=None, result=None, backend="serial"):
    return distances


+def capped_distance(reference, configuration, max_cutoff, min_cutoff=None, box=None, method=None):
+    """Returns the list of indices of reference and configuration


This should be a clearer one line description

richardjgowers · 2018-06-19T15:09:35Z

package/MDAnalysis/lib/distances.py

+    An optional keyword for the method is provided to allow usage of other
+    implemented methods. Currently pkdtree and bruteforce
+    can be used. Depending on the number of particles
+    and distribution, the times may vary significantly.


it's not clear what's meant by the times may vary. Make it clear that there's some automatic guessing of the best method, but users can also override this if they think they know better

richardjgowers · 2018-06-19T15:12:14Z

testsuite/MDAnalysisTests/lib/test_distances.py

+
+def test_capped_distance_withpbc():
+	box = np.array([1., 1., 1., 90., 90., 90.], dtype=np.float32)
+	point1 = np.array([0.1, 0.1, 0.1], dtype=np.float32)


these tests aren't suitable. the coordinates are nearly always going to be arrays of coords (ie shape (n, 3)) not single points. If you need to, generate ~100 random coords with a fixed seed, do it the slow way with distance_array then check that this new method agrees. Use a seed which ensures that the pbc option makes a difference and the results have meaningful size (ie not always 0/all)

richardjgowers · 2018-06-19T15:12:29Z

testsuite/MDAnalysisTests/lib/test_distances.py

+
+	pairs, distances = mda.lib.distances.capped_distance(point1, point2, max_cutoff=0.2)
+
+	assert_equal(len(pairs), 0)


it is worth testing the corner case of no results though

richardjgowers · 2018-06-19T15:12:41Z

testsuite/MDAnalysisTests/lib/test_distances.py

+
+	assert_equal(len(pairs), 0)
+
+def test_capped_distance_type():


shouldn't be required/forced

richardjgowers · 2018-06-22T16:07:54Z

package/MDAnalysis/lib/distances.py

+
+    Note
+    -----
+    Currently only supports brute force (see also distance_array).


the ..seealso:: :func:distance_array (or something like that..) is a documentation markup that creates a link to the other function, sorry I wasn't clearer

richardjgowers · 2018-06-22T16:08:29Z

package/MDAnalysis/lib/distances.py

+    """
+    pairs, distance = [], []
+    # Check whether reference and configuration are numpy arrays
+    if not isinstance(reference, np.ndarray):


np.asarray does all the required checking in one line

richardjgowers

More tests required!

richardjgowers · 2018-06-22T16:13:11Z

testsuite/MDAnalysisTests/lib/test_distances.py

+
+
+def test_capped_distance_checkbrute():
+    box = np.array([1., 1., 1., 90., 90., 90.], dtype=np.float32)


there's still more tests required here, eg checking array of points vs array of points, checking triclinic vs orthogonal boxes (can use a pytest.parametrize for this)

if we're using np.random we also need to seed the random generator so that the exact same test is ran each time. (otherwise maybe there's a 1 in 1,000,00 chance that the test fails with certain random conditions)

ayushsuhane · 2018-06-23T09:09:34Z

Where should we put the individual capped distances functions for their respective methods. For instance capped_pkdt should be placed in distances.py or in pkdtree.py.

jbarnoud · 2018-06-23T09:23:42Z

@ayushsuhane Both options could be argued for. I'll go with 'distances.py' to keep the feature contained.

…dfeature

ayushsuhane · 2018-06-25T01:33:03Z

How can I get around the circular import i.e. imports of _check_box in pkdtree and importing PeriodicKDTree in pdkt_capped function?

richardjgowers · 2018-06-25T01:34:19Z

@ayushsuhane you can import inside the function if it resolves a circular import

ayushsuhane · 2018-06-25T06:37:13Z

These functions are all for any arbitrary set of reference and configuration particles. For PKDTree and/or other methods, how can I check whether the preprocessing data structure (tree with configuration particles is built or not) without using it specifically as an argument.

jbarnoud · 2018-06-25T07:49:57Z

I think I see your problem, but I am not completely sure. Could you be more specific?

ayushsuhane · 2018-06-25T08:20:52Z

So, for instance, the desired attribute is to evaluate the bonds between particles. Every atom has a different radius. Ideally, neighbors of every particles should be first evaluated and then every neighbor should be checked against the sum of radius of particles. Since, the capped distance is written for arbitrary set of reference and configuration of particles, guessing bonds would require repeated calling of capped function with a one query at a time.
While doing so, we don't want to spend time in building the tree again but just checking if a tree exists should be enough. Additionally, since another requirement for the function is to be method independent, passing any arguments would destroy the purpose.
One way could be to define a class object and create the tree in __init__ method, but I am not sure whether that would serve the purpose or if there is any traditional way to handle this.

jbarnoud · 2018-06-25T08:45:35Z

Good, I understood the correct problem from your previous explanation. Assuming that you were not limited in the signatures, how would you solve the problem?

richardjgowers · 2018-06-26T15:48:58Z

package/MDAnalysis/lib/distances.py

+        if (boxtype == 'tri_vecs_bad'):
+            box = triclinic_vectors(triclinic_box(box[0], box[1], box[2]))
+    if method is None:
+        method = _determine_method(reference, configuration,


we could actually just return a function from _determine_method rather than a string which we later compare. So

method = _determine_method() pairs, dist = method(reference, configuration, etc)

This is preferable because they you don't have to type out the signature over and over again for each different method.

wrt specifying method, you would then have to change _determine_method to accept a hint:

def _determine_method(ref, conf, max_cutoff, box=box, method=None): if method is not None: if method == 'bruteforce': return _bruteforce_method elif: etc etc

richardjgowers · 2018-06-26T15:49:21Z

package/MDAnalysis/lib/distances.py

+        The shape of the reference array should be ``(N, 3)``
+        or a (3, ) coordinate.
+    configuration : array
+        Similar to reference array and  second group of coordinates


this doc line doesn't seem to make sense?

richardjgowers · 2018-06-26T15:56:44Z

package/MDAnalysis/lib/distances.py

+    return np.array(pairs), np.array(dist)
+
+
+def _determine_method(reference, configuration, cutoff, box):


we should add min_cutoff to this now, in case it becomes useful later

richardjgowers · 2018-06-26T15:57:25Z

package/MDAnalysis/lib/distances.py

+    .. SeeAlso:: :func:'MDAnalysis.lib.distances.distance_array'
+    .. SeeAlso:: :func:'MDAnalysis.lib.pkdtree.PeriodicKDTree'
+
+    Similar methods can be defined and can be directly linked to this function.


this level of documentation is made for users, so this line isn't useful and maybe confusing

ayushsuhane · 2018-06-26T21:14:38Z

The added rules in _determine_method are obtained from here
https://github.com/ayushsuhane/Benchmarks_Distance/blob/master/Notebooks/Capped_Distance.ipynb.
To list them down, the method brute force is chosen if any of the reference or configuration has a length smaller than 5000. Additionally, if the cutoff distance is larger than 10% of the box size, brute force is the preferred method. PKDTree is used in all other cases.
The current tests check whether method = 'pkdtree' and method = 'bruteforce' work independently. Is there a need to have a test case which checks whether method is selected automatically?

richardjgowers · 2018-06-29T15:20:27Z

package/MDAnalysis/lib/distances.py

+    pairs, dist = method(reference, configuration, max_cutoff,
+                         min_cutoff=min_cutoff, box=box)
+
+    return np.array(pairs), np.array(dist)


should be np.asarray in case the methods already return an array

ayushsuhane · 2018-06-30T09:45:01Z

@richardjgowers Did you get a chance to look into pkdtree and kdtree?

Also, should the self_capped_distance needs to be added in this PR or maybe around selection using capped distance OR should it be left for a different PR, since it is more of an application of the library function?

…dfeature

ayushsuhane · 2018-06-30T23:06:08Z

Incase we want it here, I quickly changed the method definition in AroundSelection with capped_distance

 def _apply_KDTree(self, group):
        self.method = 'pkdtree'
        sel = self.sel.apply(group)
        # All atoms in group that aren't in sel
        sys = group[~np.in1d(group.indices, sel.indices)]
        box = self.validate_dimensions(group.dimensions)
        indx, dist = distances.capped_distance(sel.positions, sys.positions ,max_cutoff=self.cutoff, box=box, method=self.method)
        return sys[np.unique(indx[:,1])]

I ran the tests after this modification, and it is not breaking any of the tests. I was wondering how to deal with the general definition in the __init__ function of DistanceSelection class

if flags['use_KDTree_routines'] in (True, 'fast', 'always'):
            self.apply = self._apply_KDTree
        else:
            self.apply = self._apply_distmat

        self.periodic = flags['use_periodic_selections']
        # KDTree doesn't support periodic
        if self.periodic:
            self.apply = self._apply_distmat

One possibility is to replace the 'if flags ..' condition directly with self.apply = self._capped, and define the respective distance treatements in these functions, but all the modifications needs to be done in a single PR.

jbarnoud · 2018-07-02T07:53:16Z

testsuite/MDAnalysisTests/lib/test_distances.py

@@ -43,14 +43,6 @@ def test_transform_StoR_pass(coord_dtype):
    assert_equal(original_r, test_r)


-def test_transform_StoR_fail():


Why removing this test?

The test is included in test_transform_StoR_pass using parametrize. I guess someone modified it recently which got reflected here after I performed a pull from the develop branch.

richardjgowers

nearly done

richardjgowers · 2018-07-02T21:48:14Z

testsuite/MDAnalysisTests/lib/test_distances.py

+method_1 = ('bruteforce', 'pkdtree')
+
+np.random.seed(90003)
+points = (np.random.uniform(low=0, high=1.0,


these two lines should be inside the test function

and now deleted

Oh, didn't see it. Sorry about that.

richardjgowers · 2018-07-02T21:49:16Z

testsuite/MDAnalysisTests/lib/test_distances.py

+                        size=(100, 3))*(boxes_1[0][:3])).astype(np.float32)
+
+
+@pytest.mark.parametrize('box, query , method',


I think if you stack the decorators, you don't need the product call, it does it itself, ie

@pytest.mark.parametrize('box', ...) @pytest.mark.parametrize('query', ...) @pytest.mark.parametrize('method', ...) def test_thing(box, query, method):

richardjgowers · 2018-07-02T21:49:40Z

package/MDAnalysis/lib/distances.py

@@ -1,4 +1,4 @@
-# -*- Mode: python; tab-width: 4; indent-tabs-mode:nil; -*-
+ # -*- Mode: python; tab-width: 4; indent-tabs-mode:nil; -*-


this space might even break the emacs format line thing here?

richardjgowers · 2018-07-10T23:37:28Z

Ok looks good, but needs documenting in the CHANGELOG then we're done

…dfeature

richardjgowers

this needs tests for min_cutoff

codecov · 2018-07-11T21:03:21Z

Codecov Report

Merging #1941 into develop will increase coverage by <.01%.
The diff coverage is 88%.

@@             Coverage Diff             @@
##           develop    #1941      +/-   ##
===========================================
+ Coverage    88.42%   88.43%   +<.01%     
===========================================
  Files          142      142              
  Lines        17021    17096      +75     
  Branches      2595     2617      +22     
===========================================
+ Hits         15051    15119      +68     
- Misses        1377     1381       +4     
- Partials       593      596       +3

Impacted Files	Coverage Δ
package/MDAnalysis/lib/distances.py	`87.17% <88%> (+1.1%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1a1b2f1...9d3a2fc. Read the comment docs.

richardjgowers · 2018-07-11T21:26:08Z

@ayushsuhane the code report thing above has a link to distances.py showing which lines aren't covered by tests. Give it a look and see if you can add some more tests that cover those lines, eg min_distance isn't covered, maybe it doesn't work?

richardjgowers · 2018-07-12T19:39:44Z

@ayushsuhane awesome, PR merged!

Clean initial Capped distance feature

e47ed4d

richardjgowers requested changes Jun 16, 2018

View reviewed changes

richardjgowers reviewed Jun 18, 2018

View reviewed changes

richardjgowers requested changes Jun 19, 2018

View reviewed changes

richardjgowers reviewed Jun 22, 2018

View reviewed changes

richardjgowers requested changes Jun 22, 2018

View reviewed changes

Merge branch 'develop' of github.com:MDAnalysis/mdanalysis into cappe…

cd3d29c

…dfeature

ayushsuhane force-pushed the cappedfeature branch from 52743f1 to cd3d29c Compare June 24, 2018 22:19

ayushsuhane added 3 commits June 24, 2018 15:36

Modification as per the review

23e9868

modified zip with itertools.product

dd12343

random seed

9833a69

ayushsuhane added 2 commits June 24, 2018 19:56

Capped PKDTree

e1dce21

PEP8 and added test

8cc25fe

richardjgowers requested changes Jun 26, 2018

View reviewed changes

Added rules for method selection and modifications

f02868c

richardjgowers reviewed Jun 29, 2018

View reviewed changes

ayushsuhane added 2 commits June 30, 2018 02:15

Changes the return type

61d9c6b

Updated with current repository and merged to remove conflicts

15e40ae

ayushsuhane added 2 commits June 30, 2018 11:43

Merge branch 'develop' of github.com:MDAnalysis/mdanalysis into cappe…

1d0ba7c

…dfeature

fixed the testcase

57363bb

jbarnoud reviewed Jul 2, 2018

View reviewed changes

richardjgowers requested changes Jul 2, 2018

View reviewed changes

ayushsuhane added 2 commits July 3, 2018 11:58

Stacked decorators in tests

ae6c2d9

Removed the seed

32b2f06

ayushsuhane and others added 3 commits July 10, 2018 18:07

Merge branch 'develop' of github.com:MDAnalysis/mdanalysis into cappe…

6197b5a

…dfeature

Modified Changelog

1068578

Update CHANGELOG

2218686

richardjgowers approved these changes Jul 11, 2018

View reviewed changes

richardjgowers requested changes Jul 11, 2018

View reviewed changes

ayushsuhane and others added 4 commits July 11, 2018 19:35

Added tests for minimum cutoff

cea2c37

added few more tests

7e25ea3

more tests

edc5b2d

Update test_distances.py

9d3a2fc

richardjgowers approved these changes Jul 12, 2018

View reviewed changes

richardjgowers merged commit 4be325c into MDAnalysis:develop Jul 12, 2018

ayushsuhane deleted the cappedfeature branch July 15, 2018 21:06

		@@ -389,6 +389,118 @@ def self_distance_array(reference, box=None, result=None, backend="serial"):

		return distances

		def capped_distance(reference, configuration, max_cutoff, min_cutoff = 0, box = None, method = None):


		pairs, distances = mda.lib.distances.capped_distance(point1, point2, max_cutoff=0.2)

		assert_equal(len(pairs), 0)



		def test_capped_distance_checkbrute():
		box = np.array([1., 1., 1., 90., 90., 90.], dtype=np.float32)

		return np.array(pairs), np.array(dist)


		def _determine_method(reference, configuration, cutoff, box):

		@@ -43,14 +43,6 @@ def test_transform_StoR_pass(coord_dtype):
		assert_equal(original_r, test_r)


		def test_transform_StoR_fail():

		size=(100, 3))*(boxes_1[0][:3])).astype(np.float32)


		@pytest.mark.parametrize('box, query , method',

		@@ -1,4 +1,4 @@
		# -- Mode: python; tab-width: 4; indent-tabs-mode:nil; --
		# -- Mode: python; tab-width: 4; indent-tabs-mode:nil; --

Introduce Capped distances to evaluate the indices of pairs within a fixed radius #1941

Introduce Capped distances to evaluate the indices of pairs within a fixed radius #1941

Conversation

ayushsuhane commented Jun 15, 2018 • edited Loading

PR Checklist

coveralls commented Jun 15, 2018 • edited Loading

jbarnoud commented Jun 15, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

richardjgowers commented Jun 16, 2018

jbarnoud commented Jun 17, 2018

ayushsuhane commented Jun 17, 2018 • edited Loading

richardjgowers commented Jun 18, 2018 • edited Loading

Choose a reason for hiding this comment

utkbansal commented Jun 18, 2018

Choose a reason for hiding this comment

richardjgowers left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

richardjgowers left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ayushsuhane commented Jun 23, 2018

jbarnoud commented Jun 23, 2018

ayushsuhane commented Jun 25, 2018

richardjgowers commented Jun 25, 2018

ayushsuhane commented Jun 25, 2018

jbarnoud commented Jun 25, 2018

ayushsuhane commented Jun 25, 2018

jbarnoud commented Jun 25, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ayushsuhane commented Jun 26, 2018

Choose a reason for hiding this comment

ayushsuhane commented Jun 30, 2018

ayushsuhane commented Jun 30, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

richardjgowers left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

richardjgowers commented Jul 10, 2018

richardjgowers left a comment

Choose a reason for hiding this comment

codecov bot commented Jul 11, 2018 • edited Loading

Codecov Report

richardjgowers commented Jul 11, 2018

richardjgowers commented Jul 12, 2018

ayushsuhane commented Jun 15, 2018 •

edited

Loading

coveralls commented Jun 15, 2018 •

edited

Loading

ayushsuhane commented Jun 17, 2018 •

edited

Loading

richardjgowers commented Jun 18, 2018 •

edited

Loading

ayushsuhane commented Jun 30, 2018 •

edited

Loading

codecov bot commented Jul 11, 2018 •

edited

Loading