Inconsistent soft-dirty against mprotect dirty bits

[ivmai]

Source: master (b4e1ce5)
Build: https://app.travis-ci.com/github/ivmai/bdwgc/jobs/602918435
Host: Linux gcc203 6.0.0-6-powerpc64
Occurrence: ~1/5
How to reproduce: ./configure --enable-werror --enable-gc-assertions --disable-threads --enable-cplusplus && make -j check CFLAGS_EXTRA="-O3 -mcpu=native -D CHECK_SOFT_VDB" && ./gctest
Switched to incremental mode
Emulating dirty bits with mprotect/signals
Inconsistent soft-dirty against mprotect dirty bits
Aborted

Related issue #544.

[ivmai]

Also, observed on Ubuntu 22.04 x64
Occurrence: ~1/70
How to reproduce: ./configure --enable-gc-assertions --disable-thread-local-alloc && make -j check CFLAGS_EXTRA="-D CHECK_SOFT_VDB" && ./gctest

[ivmai]

Additional temp patch to check if re-read of pagemap provides correct result:

diff --git a/os_dep.c b/os_dep.c
index fbbb301e..d0f11b47 100644
--- a/os_dep.c
+++ b/os_dep.c
@@ -3335,6 +3335,9 @@ GC_API GC_push_other_roots_proc GC_CALL GC_get_push_other_roots(void)
         /* have just unprotected a page in the GC's thread structure,   */
         /* and then to have the thread stopping code set the dirty      */
         /* flag, if necessary.                                          */
+#       ifdef DEBUG_DIRTY_BITS
+          GC_log_printf("dirty page at: %p (mprotect)\n", (void *)h);
+#       endif
         for (i = 0; i < divHBLKSZ(GC_page_size); i++) {
             word index = PHT_HASH(h+i);
 
@@ -4054,6 +4058,17 @@ GC_INLINE void GC_proc_read_dirty(GC_bool output_unneeded)
             /* Ensure that each clean page according to the soft-dirty  */
             /* VDB is also identified such by the mprotect-based one.   */
             if (get_pht_entry_from_index(GC_dirty_pages, PHT_HASH(vaddr))) {
+              pagemap_elem_t buf1[1];//??
+              off_t fpos = (off_t)(((word)vaddr >> GC_log_pagesize)
+                            * sizeof(pagemap_elem_t));
+
+              if (lseek(pagemap_fd, fpos, SEEK_SET) == (off_t)(-1))
+                ABORT("lseek(pagemap)");
+              if (PROC_READ(pagemap_fd, buf1, sizeof(buf1))
+                    != (int)sizeof(buf1))
+                ABORT("read(pagemap)");
+              if ((buf1[0] & PM_SOFTDIRTY_MASK) != 0)
+                ABORT("Inconsistent soft/mprotect-dirty but OK after re-read");
               ABORT("Inconsistent soft-dirty against mprotect dirty bits");
             }
 #         endif

[ivmai]

Additional temp patch to check if re-read of pagemap provides correct result

Both on powerpc64 and x64 the result of re-read was the same -> the issue is not in the implementation of GC_soft_read_dirty.

[ivmai]

Source: 242a3a7
Arch: ppc64le
Build: https://app.travis-ci.com/github/ivmai/bdwgc/jobs/611522393

[ivmai]

Source: 57be926
Arch: ppc64le
Build: https://app.travis-ci.com/github/ivmai/bdwgc/jobs/612950934