Initial changes to refactor snapshot at the beginning barrier. #2298

ebadyano · 2018-06-28T21:15:41Z

SATB barrier is used in metronome gc policy. We would like to enable it
for other gc polices under option -Xgc:snapshotAtTheBeginningBarrier.
These are initial changes to add the option and start decoupling barrier
code from metronome gc.

Signed-off-by: Evgenia Badyanova [email protected]

ebadyano · 2018-06-28T21:16:44Z

Work in progress. @amicic @dmitripivkine Aleks, Dmitri could you please provide your feedback and comments. Thank you! omr pull request: eclipse-omr/omr#2717

amicic · 2018-06-29T18:13:25Z

runtime/gc_glue_java/ConfigurationDelegate.hpp

@@ -88,6 +88,9 @@ class MM_ConfigurationDelegate
 		if (extensions->alwaysCallWriteBarrier) {
 			writeBarrierType = gc_modron_wrtbar_always;
 		}
+		if (extensions->isSnapshotAtTheBeginningBarrierEnabled()) {
+			writeBarrierType = gc_modron_wrtbar_realtime;


gc_modron_wrtbar_realtime should change into gc_modron_wrtbar_satb

renamed to barrier to gc_modron_wrtbar_satb. I'm keeping gc_modron_wrtbar_realtime in omr until the initial opnej9 changes get committed.

amicic · 2018-06-29T18:17:23Z

runtime/gc_glue_java/GlobalCollectorDelegate.cpp

+			if (NULL == _extensions->sATBBarrierRememberedSet) {
+				return false;
+			}
+		}


hm, I have a feeling this should be in OMR part

If you make MM_RememberedSetWorkPackets to take workPackets as a specific MM_WorkPacketsSATB type, it's safe to upcast markingScheme->getWorkPackets() to (MM_WorkPacketsSATB* ), since it's under isSATBEnabled check.

I moved this code to OMR.

amicic · 2018-06-29T18:54:04Z

runtime/gc_modron_standard/StandardAccessBarrier.cpp

+ * is in progress is to be remembered for later marking and scanning.
+ */
+void
+MM_StandardAccessBarrier::rememberObject(MM_EnvironmentBase *env, J9Object *object)


we will now have 2 completely different remember actions in barrierr world:

as part of a generational barrier, we need to remember an old object (referencing a new object); the action could be more specifically called remember-object-as-root (in subsequent GCs)

as part SATB concurrent marking, we need to remember old reference point being overwritten; the action could be more specifically called remember-object-to-rescan (in this GC cycle)

Therefore, the name of methods should be appropriately renamed to avoid confusion. Renaming this method should be sufficent, for now. Creative suggestions are welcome.

renamed the method to rememberObjectToRescan

amicic · 2018-06-29T18:55:43Z

runtime/gc_modron_standard/StandardAccessBarrier.cpp

+	}
+	MM_EnvironmentBase* env = MM_EnvironmentBase::getEnvironment(vmThread->omrVMThread);
+
+	if (isBarrierActive(env)) {


these two isBarrierActive method names should be more specific (refer to SATB)

renamed the helper to isSATBBarrierActive.

amicic · 2018-06-29T18:57:45Z

runtime/gc_modron_standard/StandardAccessBarrier.cpp

+MM_StandardAccessBarrier::preObjectStoreInternal(J9VMThread *vmThread, J9Object *destObject, fj9object_t *destAddress, J9Object *value, bool isVolatile)
+{
+	J9JavaVM *javaVM = vmThread->javaVM;
+	if (javaVM->gcWriteBarrierType != gc_modron_wrtbar_realtime) {


not sure, but this check should be part of isBarrierActive(). the point is minimize the amount of barrier type (in this case SATB) specific code

moved the check to isSATBBarrierActive helper.

amicic · 2018-06-29T18:59:25Z

runtime/gc_modron_standard/StandardAccessBarrier.cpp

+	J9JavaVM *javaVM = vmThread->javaVM;
+	if (javaVM->gcWriteBarrierType == gc_modron_wrtbar_realtime) {
+		return retValue;
+	}


note to other reviewers: I'm ok with this hack for now (it forces slow path array copying).

runtime/gc_modron_standard/StandardAccessBarrier.hpp

amicic · 2018-09-14T15:40:40Z

runtime/compiler/control/J9Options.cpp

@@ -1106,7 +1106,7 @@ J9::Options::fePreProcess(void * base)
         case j9gc_modron_wrtbar_cardmark:               wrtbarMode = TR_WrtbarCardMark; break;
         case j9gc_modron_wrtbar_cardmark_and_oldcheck:  wrtbarMode = TR_WrtbarCardMarkAndOldCheck; break;
         case j9gc_modron_wrtbar_cardmark_incremental:   wrtbarMode = TR_WrtbarCardMarkIncremental; break;
-         case j9gc_modron_wrtbar_realtime:               wrtbarMode = TR_WrtbarRealTime; break;
+         case j9gc_modron_wrtbar_satb:                   wrtbarMode = TR_WrtbarRealTime; break;


we should have j9gc_modron_wrtbar_satb_and_oldcheck defined and checked at various spots where j9gc_modron_wrtbar_satb is checked

amicic · 2018-09-14T15:45:19Z

runtime/gc_modron_standard/StandardAccessBarrier.cpp

+MM_StandardAccessBarrier::rememberObjectToRescan(MM_EnvironmentBase *env, J9Object *object)
+{
+	//TODO SATB re-enable opt?
+	//if (_markingScheme->markObject(env, object, true)) {


this should be in. otherwise, it will create way too many unnecessary/duplicate remembered objects (and overstress marking work stack)

amicic · 2018-09-14T15:46:45Z

runtime/gc_modron_standard/StandardAccessBarrier.cpp

@@ -481,6 +604,10 @@ I_32
 MM_StandardAccessBarrier::backwardReferenceArrayCopyIndex(J9VMThread *vmThread, J9IndexableObject *srcObject, J9IndexableObject *destObject, I_32 srcIndex, I_32 destIndex, I_32 lengthInSlots)
 {
 	I_32 retValue = ARRAY_COPY_NOT_DONE;
+	//TODO SATB re-enable opt?
+	if (_extensions->configuration->isSnapshotAtTheBeginningBarrierEnabled()) {


this (opt for ref array copying) is ok to disable, for the initial contribution

amicic · 2018-09-14T15:51:26Z

runtime/oti/j9accessbarrier.h

@@ -171,13 +171,13 @@ typedef struct J9IndexableObject* mm_j9array_t;
 	(void)0)
 #define J9OBJECT__PRE_OBJECT_STORE_ADDRESS(vmThread, object, address, value) \
 	((void)0, \
-	(OMR_GC_ALLOCATION_TYPE_SEGREGATED == (J9VMTHREAD_JAVAVM((vmThread)))->gcAllocationType) ? \
+	(OMR_GC_WRITE_BARRIER_TYPE_REALTIME == (J9VMTHREAD_JAVAVM((vmThread)))->gcWriteBarrierType) ? \


this should check for OMR_GC_WRITE_BARRIER_TYPE_SATB || OMR_GC_WRITE_BARRIER_TYPE_SATB_AND_OLDCHECK, but I guess you did not introduce them yet?

I did introduce them in OMR but I missed this spot, thank you for catching it. I will update the pull request.

amicic · 2018-09-14T15:59:18Z

runtime/gc_modron_standard/StandardAccessBarrier.hpp

+	MMINLINE bool isSATBBarrierActive(MM_EnvironmentBase* env)
+	{
+		return _extensions->configuration->isSnapshotAtTheBeginningBarrierEnabled() &&
+				!_extensions->sATBBarrierRememberedSet->isGlobalFragmentIndexPreserved(env);


the first part might not be necessary, since the second part would never be satisfied if SATB is not enabled

amicic · 2018-09-14T16:12:36Z

runtime/gc_modron_standard/StandardAccessBarrier.cpp

+	if (_extensions->configuration->isSnapshotAtTheBeginningBarrierEnabled()) {
+
+		_extensions->sATBBarrierRememberedSet->initializeFragment(env, &(((J9VMThread *)env->getLanguageVMThread())->sATBBarrierRememberedSetFragment));
+//TODO: SATB re-enable later


keep this in, but we'll worry about double barrier in more details later.
for example, we don't have yet the mechanism to enable/disable double barrier.

uncommented it.

amicic · 2018-09-14T16:37:26Z

runtime/gc_modron_standard/StandardAccessBarrier.cpp

 /**
 * Called after an object is stored into another object.
 */
 void
 MM_StandardAccessBarrier::postObjectStore(J9VMThread *vmThread, J9Object *destObject, fj9object_t *destAddress, J9Object *value, bool isVolatile)
 {
-	postObjectStoreImpl(vmThread, destObject, value);
+	if (_extensions->configuration->isIncrementalUpdateBarrierEnabled()) {


similarly to isSATBBarrierActive, there should be a helper isIncrementalUpdateBarrierActive within this class.
then, that helper probably does not need to go as far as to configuration, but just check in thread local data if the barrier is active. it should not ever be really active on thread level, if in configuration it's not enabled

isIncrementalUpdateBarrierEnabled checks for barrier type which I can only get from configuration at this point. Since I don't want to execute postObjectStoreImpl for SATB barriers. I think we talked that eventually we might move the generational barrier code to preObjectStoreImpl

I'm realizing it's a bit more complex because we also have generational barrier. If we decide to leave it as a post barrier , than we should check if either of Incremental update is active or generational barrier is enabled.
I believe that generational barrier does not have to be post barrier. So we could have it either pre or post, depending if we run SATB or IncrementalUpdate for Concurrent Mark.
It's probably safest not to change runtime behaviour in existing configuration (and leave it as a post), although it might mean not the simplest/optimal code.
Lastly, do we really need this explict check here? We already have vmThread->privateFlags & J9_PRIVATE_FLAGS_CONCURRENT_MARK_ACTIVE check inside postObjectStoreImpl, which should turn into something that is IncurementalUpdate specific, if already not the case.

for now moved it to a helper with isIncrementalUpdateBarrierEnabled check.

amicic · 2018-09-14T18:11:34Z

runtime/gc_modron_standard/StandardAccessBarrier.hpp

@@ -92,6 +99,28 @@ class MM_StandardAccessBarrier : public MM_ObjectAccessBarrier
 	virtual bool preMonitorTableSlotRead(J9JavaVM *vm, j9object_t *srcAddress);	
 #endif	

+	/* New methods */
+	bool preObjectStoreInternal(J9VMThread *vmThread, J9Object *destObject, fj9object_t *destAddress, J9Object *value, bool isVolatile);


align the naming... there is already postObjectStoreImpl, so this should be probably preObjectStoreImpl? although throughout GC code we tend more often to use ...Internal than ...Impl. do a quick check what is more common before changing.

renamed methods to preObjectStoreImpl since all other methods in this file follow Impl convention.

amicic · 2018-09-14T19:45:18Z

runtime/gc_include/ObjectAccessBarrierAPI.hpp

@@ -2625,7 +2625,8 @@ class MM_ObjectAccessBarrierAPI
 	VMINLINE void
 	internalPreStoreObject(J9VMThread *vmThread, j9object_t object, fj9object_t *destAddress, j9object_t value)
 	{
-		if (j9gc_modron_wrtbar_realtime == _writeBarrierType) {
+		if ((j9gc_modron_wrtbar_satb == _writeBarrierType) ||
+				(j9gc_modron_wrtbar_satb_and_oldcheck == _writeBarrierType)) {
 			internalPreStoreObjectRealtime(vmThread, object, destAddress, value);


rename internalPreStoreObjectRealtime to internalPreStoreObjectSATB

no it's not, you did it just for static

oh right, renamed both now

amicic · 2018-09-14T19:45:32Z

runtime/gc_include/ObjectAccessBarrierAPI.hpp

@@ -2644,7 +2645,8 @@ class MM_ObjectAccessBarrierAPI
 	VMINLINE void
 	internalStaticPreStoreObject(J9VMThread *vmThread, j9object_t object, j9object_t *destAddress, j9object_t value)
 	{
-		if (j9gc_modron_wrtbar_realtime == _writeBarrierType) {
+		if (j9gc_modron_wrtbar_satb == _writeBarrierType ||
+				j9gc_modron_wrtbar_satb_and_oldcheck == _writeBarrierType) {
 			internalStaticPreStoreObjectRealtime(vmThread, object, destAddress, value);


rename to SATB

amicic · 2018-09-14T19:48:04Z

runtime/oti/j9accessbarrier.h

@@ -171,13 +171,13 @@ typedef struct J9IndexableObject* mm_j9array_t;
 	(void)0)
 #define J9OBJECT__PRE_OBJECT_STORE_ADDRESS(vmThread, object, address, value) \
 	((void)0, \
-	(OMR_GC_ALLOCATION_TYPE_SEGREGATED == (J9VMTHREAD_JAVAVM((vmThread)))->gcAllocationType) ? \
+	((OMR_GC_WRITE_BARRIER_TYPE_SATB || OMR_GC_WRITE_BARRIER_TYPE_SATB_AND_OLDCHECK) == (J9VMTHREAD_JAVAVM((vmThread)))->gcWriteBarrierType) ? \


this is actually not correct. the 'or' condition should be between two equality comparisons

amicic · 2018-09-17T18:06:15Z

runtime/oti/j9accessbarrier.h

 	J9VMTHREAD_JAVAVM((vmThread))->memoryManagerFunctions->J9MetronomeWriteBarrierStore((vmThread), (j9object_t)(object), (fj9object_t*)(address), (j9object_t)(value)) : \
 	(void)0 \
 	)
 #define J9OBJECT__PRE_OBJECT_STORE_ADDRESS_VM(javaVM, object, address, value) \
 	((void)0, \
-	(OMR_GC_ALLOCATION_TYPE_SEGREGATED == javaVM->gcAllocationType) ? \
+	((OMR_GC_WRITE_BARRIER_TYPE_SATB == (J9VMTHREAD_JAVAVM((vmThread)))->gcWriteBarrierType) || \
+	(OMR_GC_WRITE_BARRIER_TYPE_SATB_AND_OLDCHECK == (J9VMTHREAD_JAVAVM((vmThread)))->gcWriteBarrierType)) ? \


this API provides javaVM, not vmThread

amicic · 2018-09-17T18:08:11Z

runtime/gc_include/ObjectAccessBarrierAPI.hpp

@@ -2644,8 +2645,9 @@ class MM_ObjectAccessBarrierAPI
 	VMINLINE void
 	internalStaticPreStoreObject(J9VMThread *vmThread, j9object_t object, j9object_t *destAddress, j9object_t value)
 	{
-		if (j9gc_modron_wrtbar_realtime == _writeBarrierType) {
-			internalStaticPreStoreObjectRealtime(vmThread, object, destAddress, value);
+		if (j9gc_modron_wrtbar_satb == _writeBarrierType ||


add extra brackets

amicic · 2018-09-17T18:29:36Z

runtime/oti/j9accessbarrier.h

@@ -195,13 +197,15 @@ typedef struct J9IndexableObject* mm_j9array_t;
 	)
 #define J9STATIC__PRE_OBJECT_STORE(vmThread, clazz, address, value) \
 	((void)0, \
-	(OMR_GC_ALLOCATION_TYPE_SEGREGATED == (J9VMTHREAD_JAVAVM((vmThread)))->gcAllocationType) ? \
+	((OMR_GC_WRITE_BARRIER_TYPE_SATB == javaVM->gcWriteBarrierType) || \


there is no javaVM directly available here

amicic · 2018-09-17T18:30:13Z

runtime/oti/j9accessbarrier.h

 	J9VMTHREAD_JAVAVM((vmThread))->memoryManagerFunctions->J9MetronomeWriteBarrierJ9ClassStore((vmThread), J9VM_J9CLASS_TO_HEAPCLASS((clazz)), (j9object_t*)(address), (j9object_t)(value)) : \
 	(void)0 \
 	)
 #define J9STATIC__PRE_OBJECT_STORE_VM(javaVM, object, address, value) \
 	((void)0, \
-	(OMR_GC_ALLOCATION_TYPE_SEGREGATED == javaVM->gcAllocationType) ? \
+	((OMR_GC_WRITE_BARRIER_TYPE_SATB == (J9VMTHREAD_JAVAVM((vmThread)))->gcWriteBarrierType) || \


there is not vmThread available here, but javamVM, what is exactly you need

I think I finally fixed all the typos in this file! thank you!

dmitripivkine · 2018-09-17T22:15:04Z

The failure in travis-ci build is unrelated and has been fixed by #2649 , so the only way to pass it rebase forward

dmitripivkine

lgtm

ebadyano · 2018-09-17T22:21:35Z

I have a few failures in my personal build, investigating. will update once it's fixed. thanks!

andrewcraik · 2018-09-17T22:26:44Z

I just marked this WIP pending further updates from @ebadyano to help prevent an accidental merge.

amicic · 2018-09-18T03:34:28Z

Jenkins test sanity xLinux jdk8

amicic · 2018-09-18T15:59:07Z

Jenkins test sanity xLinux jdk8

SATB barrier is used in metronome gc policy. We would like to enable it for other gc polices under option -Xgc:snapshotAtTheBeginningBarrier. These are initial changes to add the option and start decoupling barrier code from metronome gc. Signed-off-by: Evgenia Badyanova <[email protected]>

amicic · 2018-09-18T18:33:37Z

Jenkins test sanity xLinux jdk8

amicic · 2018-09-18T18:40:41Z

runtime/gc_modron_standard/StandardAccessBarrier.hpp

+	{
+		return ((_extensions->configuration->isSnapshotAtTheBeginningBarrierEnabled()) &&
+				(!_extensions->sATBBarrierRememberedSet->isGlobalFragmentIndexPreserved(env)));
+	}


Extra brackets are really needed when it's not so obvious which operator has priority. I believe that for most people there is no dilemma that -> operator is among the highest. But you can leave it as is, unless you end up changing the code again.

amicic · 2018-09-18T18:44:55Z

runtime/gc_modron_standard/StandardAccessBarrier.hpp

@@ -47,7 +48,9 @@ class MM_StandardAccessBarrier : public MM_ObjectAccessBarrier
 #if defined(J9VM_GC_GENERATIONAL)
 	MM_GenerationalAccessBarrierComponent _generationalAccessBarrierComponent;	/**< Generational Component of Access Barrier */
 #endif /* J9VM_GC_GENERATIONAL */
-
+#if defined(OMR_GC_REALTIME)
+	bool _doubleBarrierActive; /**< Global indicator that the double barrier is active. New threads will be set to double barrier mode if this falg is true. */


is it initialized?

not yet, I was going to see where to initialize it once we start working on the code for doublebarrier (in the follow up change), the code is guarded by isSnapshotAtTheBeginningBarrierEnabled option so should cause any issues for the existing gc policies

amicic · 2018-09-18T20:45:54Z

jenkins test sanity zlinux jdk8

amicic · 2018-09-18T23:45:04Z

runtime/gc_modron_startup/arrayCopy.cpp

@@ -274,7 +274,7 @@ initializeReferenceArrayCopyTable(J9ReferenceArrayCopyTable *table)
 	table->forwardReferenceArrayCopyWithCheckIndex[j9gc_modron_wrtbar_cardmark] = forwardReferenceArrayCopyWithCheckAndAlwaysWrtbarIndex;
 	table->forwardReferenceArrayCopyWithCheckIndex[j9gc_modron_wrtbar_cardmark_incremental] = forwardReferenceArrayCopyWithCheckAndAlwaysWrtbarIndex;
 	table->forwardReferenceArrayCopyWithCheckIndex[j9gc_modron_wrtbar_cardmark_and_oldcheck] = forwardReferenceArrayCopyWithCheckAndAlwaysWrtbarIndex;
-	table->forwardReferenceArrayCopyWithCheckIndex[j9gc_modron_wrtbar_realtime] = forwardReferenceArrayCopyWithCheckAndAlwaysWrtbarIndex;
+	table->forwardReferenceArrayCopyWithCheckIndex[j9gc_modron_wrtbar_satb] = forwardReferenceArrayCopyWithCheckAndAlwaysWrtbarIndex;


this table should be expanded for satb and oldcheck too (fine for future changes)

dmitripivkine changed the title ~~Initial changes to refactor snapshot at the beginning barrier.~~ WIP:Initial changes to refactor snapshot at the beginning barrier. Jun 28, 2018

dmitripivkine added comp:gc depends:omr Pull request is dependent on a corresponding change in OMR labels Jun 28, 2018

amicic reviewed Jun 29, 2018

View reviewed changes

runtime/gc_modron_standard/StandardAccessBarrier.hpp Outdated Show resolved Hide resolved

ebadyano force-pushed the master branch from 9d10f0f to 251e478 Compare July 19, 2018 19:01

ebadyano force-pushed the master branch from 251e478 to 2d9e94c Compare September 6, 2018 14:22

ebadyano force-pushed the master branch 4 times, most recently from 25f847a to 7249204 Compare September 14, 2018 15:09

amicic reviewed Sep 14, 2018

View reviewed changes

ebadyano force-pushed the master branch from 7249204 to 0653ae0 Compare September 14, 2018 19:32

amicic reviewed Sep 14, 2018

View reviewed changes

ebadyano force-pushed the master branch from 0653ae0 to 88f4a42 Compare September 14, 2018 20:06

ebadyano force-pushed the master branch 2 times, most recently from 2ff130e to be8ddc6 Compare September 17, 2018 14:45

amicic reviewed Sep 17, 2018

View reviewed changes

ebadyano force-pushed the master branch from be8ddc6 to 0095bf7 Compare September 17, 2018 18:11

amicic reviewed Sep 17, 2018

View reviewed changes

ebadyano force-pushed the master branch from 0095bf7 to e24d4d2 Compare September 17, 2018 18:37

dmitripivkine changed the title ~~WIP:Initial changes to refactor snapshot at the beginning barrier.~~ Initial changes to refactor snapshot at the beginning barrier. Sep 17, 2018

dmitripivkine approved these changes Sep 17, 2018

View reviewed changes

andrewcraik changed the title ~~Initial changes to refactor snapshot at the beginning barrier.~~ WIP: Initial changes to refactor snapshot at the beginning barrier. Sep 17, 2018

ebadyano force-pushed the master branch 2 times, most recently from b7f48e2 to ed394d6 Compare September 18, 2018 15:01

ebadyano force-pushed the master branch from ed394d6 to 2abca9b Compare September 18, 2018 18:31

amicic reviewed Sep 18, 2018

View reviewed changes

amicic changed the title ~~WIP: Initial changes to refactor snapshot at the beginning barrier.~~ Initial changes to refactor snapshot at the beginning barrier. Sep 18, 2018

amicic reviewed Sep 18, 2018

View reviewed changes

amicic approved these changes Sep 18, 2018

View reviewed changes

amicic merged commit 95c966c into eclipse-openj9:master Sep 19, 2018

This was referenced Sep 19, 2018

GC: Enable snapshot at the beginning barrier for standard gc polices eclipse-omr/omr#2987

Closed

Follow up for initial SATB barrier changes #2988

Merged

Initial changes to refactor snapshot at the beginning barrier. #2298

Initial changes to refactor snapshot at the beginning barrier. #2298

Conversation

ebadyano commented Jun 28, 2018

ebadyano commented Jun 28, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

amicic Sep 14, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

amicic Sep 14, 2018 • edited Loading

Choose a reason for hiding this comment

ebadyano Sep 14, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

amicic Sep 14, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dmitripivkine commented Sep 17, 2018 • edited Loading

dmitripivkine left a comment

Choose a reason for hiding this comment

ebadyano commented Sep 17, 2018

andrewcraik commented Sep 17, 2018

amicic commented Sep 18, 2018

amicic commented Sep 18, 2018

amicic commented Sep 18, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

amicic commented Sep 18, 2018

Choose a reason for hiding this comment

amicic Sep 14, 2018 •

edited

Loading

amicic Sep 14, 2018 •

edited

Loading

ebadyano Sep 14, 2018 •

edited

Loading

amicic Sep 14, 2018 •

edited

Loading

dmitripivkine commented Sep 17, 2018 •

edited

Loading