xsnap not deterministic? (weakrefs, SES shim)

[dckc]

Build environment: Linux
Target device: xsnap

Description

I'm trying to get reproducible snapshot hashes a packaged version of our SES shim (see bootstrap.umd.js ) but they seem to be non-deterministic.

I reproduced the problem using just the weakref example that comes with xsnap:

Steps to Reproduce

1. Build xsnap as usual (cd makefiles/lin; make)
2. run xsnap examples/weaks/before.js -w out.xss
3. note the sha256sum of out.xss
4. run xsnap examples/weaks/before.js -w out.xss again
5. see a different sha256sum of out.xss

Expected behavior
same snapshot

Other information

xsnap version 8b127bc

transcript:

connolly@jambox:~/projects/xs-snap/xsnap/examples$ ../makefiles/lin/bin/lin/release/xsnap weaks/before.js -w out.xss 
connolly@jambox:~/projects/xs-snap/xsnap/examples$ sha256sum out.xss 
3ce3e751036fafd9b0dfdd914319d5ff0e85470400541d4341199340f6204427  out.xss
connolly@jambox:~/projects/xs-snap/xsnap/examples$ ../makefiles/lin/bin/lin/release/xsnap weaks/before.js -w out.xss 
connolly@jambox:~/projects/xs-snap/xsnap/examples$ sha256sum out.xss 
75342d99a4d1d96ac3ddd865a36819056ce9788599b61845c4f4fc66729a3397  out.xss
connolly@jambox:~/projects/xs-snap/xsnap/examples$ ../makefiles/lin/bin/lin/release/xsnap weaks/before.js -w out.xss 
connolly@jambox:~/projects/xs-snap/xsnap/examples$ sha256sum out.xss 
c1644281efd59c06c671b46ed8841860fd29ce2f3f46c6690bee2eb11517664b  out.xss

cc @warner @dtribble @erights

[phoddie]

Have you looked beyond the hash to characterize the differences?

[dckc]

No... any suggestions on where to look?

The outputs are the same size (~275k). They differ at around 125k. Occasionally two outputs match:

connolly@jambox:~/projects/xs-snap/xsnap/examples$ ../makefiles/lin/bin/lin/release/xsnap weaks/before.js -w out.xss 
connolly@jambox:~/projects/xs-snap/xsnap/examples$ ls -l out1.xss out.xss 
-rw-rw-r-- 1 connolly connolly 275115 Feb 16 16:52 out1.xss
-rw-rw-r-- 1 connolly connolly 275115 Feb 16 16:52 out.xss
connolly@jambox:~/projects/xs-snap/xsnap/examples$ cmp out1.xss out.xss 
out1.xss out.xss differ: byte 125624, line 2
connolly@jambox:~/projects/xs-snap/xsnap/examples$ mv out.xss out2.xss
connolly@jambox:~/projects/xs-snap/xsnap/examples$ ../makefiles/lin/bin/lin/release/xsnap weaks/before.js -w out.xss 
connolly@jambox:~/projects/xs-snap/xsnap/examples$ cmp out2.xss out.xss 
connolly@jambox:~/projects/xs-snap/xsnap/examples$ ../makefiles/lin/bin/lin/release/xsnap weaks/before.js -w out4.xss 
connolly@jambox:~/projects/xs-snap/xsnap/examples$ mv out.xss out3.xss
connolly@jambox:~/projects/xs-snap/xsnap/examples$ cmp out2.xss out4.xss 
out2.xss out4.xss differ: byte 125920, line 2
connolly@jambox:~/projects/xs-snap/xsnap/examples$ cmp out2.xss out3.xss 
connolly@jambox:~/projects/xs-snap/xsnap/examples$ 

[phoddie]

Running cmp with the default settings gives about the same information as comparing signatures -- either there is a difference or not. I was hoping to characterize the number of differences. Running cmp with -l -b appears like it might give a useful count.

You note that sometimes there are zero, so maybe the other times it is a small number too?

[dckc]

looks like around 24 to 26 bytes differ for the weakref case. For the SES bootstrap bundle, a couple thousand bytes seem to differ.

connolly@jambox:~/projects/xs-snap/xsnap/examples$ cmp -l -b out1.xss out2.xss 
125624 264 M-4    0 ^@
125625  16 ^N     0 ^@
126048   0 ^@   264 M-4
126049   0 ^@    16 ^N
126136 267 M-7    0 ^@
126137  16 ^N     0 ^@
126560   0 ^@   267 M-7
126561   0 ^@    16 ^N
126712 273 M-;    0 ^@
126713  16 ^N     0 ^@
127136   0 ^@   273 M-;
127137   0 ^@    16 ^N
127224 275 M-=    0 ^@
127225  16 ^N     0 ^@
127648   0 ^@   275 M-=
127649   0 ^@    16 ^N
248324 162 r     62 2
248325   5 ^E     0 ^@
248420 162 r     62 2
248421  45 %     40  
248548 162 r     62 2
248549   5 ^E     0 ^@
248612 162 r     62 2
248613  45 %     40  
connolly@jambox:~/projects/xs-snap/xsnap/examples$ cmp -l -b out1.xss out2.xss |wc -l
24
connolly@jambox:~/projects/xs-snap/xsnap/examples$ cmp -l -b out2.xss out3.xss |wc -l
0
connolly@jambox:~/projects/xs-snap/xsnap/examples$ cmp -l -b out3.xss out4.xss |wc -l
26

connolly@jambox:~/projects/xs-snap/xsnap/examples$ cmp -l -b boot1.xss boot2.xss |wc -l
2030
connolly@jambox:~/projects/xs-snap/xsnap/examples$ cmp -l -b boot2.xss boot3.xss |wc -l
1686
connolly@jambox:~/projects/xs-snap/xsnap/examples$ cmp -l -b boot3.xss boot4.xss |wc -l
1831

[dckc]

generators/before.js also differs by 6 or 8 bytes.

The others show 0. (I commented out new Date() in values/before.js)

xsnap/examples$ for js in */before.js; do echo $js; xsnap $js -w out1.xss; xsnap $js -w out2.xss; cmp -l -b out1.xss out2.xss | wc -l ; done

xsnap/examples$ xsnap generators/before.js -w out1.xss
xsnap/examples$ xsnap generators/before.js -w out2.xss
xsnap/examples$ cmp -l -b out1.xss out2.xss 
126057 137 _     17 ^O
126058 335 M-]  327 M-W
126059 105 E    154 l
126060 151 i    174 |
126585 136 ^     16 ^N
126586 335 M-]  327 M-W
126587 105 E    154 l
126588 151 i    174 |

[phoddie]

Thanks for the additional data. The two runs may be separate issues -- in the first, the differences are all in two byte groups whereas in h second they are in four byte groups. The two byte groups suggest that it could be the slot ID value (it is a two byte value).

We anticipated that there could be challenges on the road deterministic snapshots. Here we are. ;)

[phoddie]

I looked into this a bit. In the weals/before.js case, the mismatched offsets are in the snapshots BLOC atom, which means they are in the chunk heap not a slot. So, my guess about slot ID values looks wrong.

[phoddie]

The differences are in the hash tables used internally by WeakMap and WeakSet. The hash used for objects depends on the address of the object's slot. When it occasionally works for you, either the addresses or their hashes are the same.

I did a quick hack to eliminate the address dependency. The resulting snapshots match. But, it is a hack so the snapshots don't completely work on reload. Not sure yet about the best way to solve this.

[phoddie]

We pushed some changes to XS that give deterministic results for both the weak and generator tests. Those were separate issues.

[dckc]

yay! It works here too.

e0bdcf7 seems to be the fix.