docs: add snapshot instructions #224
Conversation
|
Visit the preview URL for this PR (updated for commit b46ba52): https://fetch-docs-preview--pr224-docs-snapshots-r0y1j9js.web.app (expires Tue, 03 May 2022 14:27:10 GMT) 🔥 via Firebase Hosting GitHub Action 🌎 |
docs/docs/snapshots.md
Outdated
| curl -v https://storage.googleapis.com/fetch-ai-mainnet-snapshots/fetchhub-4-pruned.tgz -o- 2>headers.out | tee >(md5sum > md5sum.out) | gunzip -c | tar -xvf - --directory=~/.fetchd | ||
|
|
||
| # (optional, but recommended) compare source md5 checksum provided in the headers by google, with the one calculated locally | ||
| [[ $(awk -F\" '/etag:/{ print $2 }' headers.out) == $(awk '{ print $1 }' md5sum.out) ]] && echo "OK - md5sum match" || echo "ERROR - md5sum MISMATCH" |
There was a problem hiding this comment.
that's some cool stuff here!
To double-check, is using the etag fine here? Seeing on https://cloud.google.com/storage/docs/hashes-etags#xml-api it might not guarantee to be the md5 - maybe x-goog-hash to be preferred (they actually have the same value when I tested, but not sure it's meant to stay like this)
Also wondered if md5 was enough here given how easy is to tamper with it nowadays. But I guess it's ok, the consensus is supposed to deal with the extra validation that this state is pristine
There was a problem hiding this comment.
Well spotted, thanks! I will fix...
There was a problem hiding this comment.
As for md5 collisions... something they do (in a slightly different context) on Osmosis is to publish their hashes on the blockchain itself. I'll have a think about whether I can push a memo with the (assumed immutable) GCS x-goog-generation version number vs md5sum, that should give a very good confidence that a downloaded snapshot was created by my k8s pod, even in the scenario where an adversary has write access to our bucket.
There was a problem hiding this comment.
No description provided.