Code

Encrypted blob store repository#50846

Code

Closed

albertzaharovits wants to merge 160 commits intoelastic:repository-encrypted-client-sideelastic/elasticsearch:repository-encrypted-client-sidefrom albertzaharovits:encrypted-blob-storealbertzaharovits/elasticsearch:encrypted-blob-storeCopy head branch name to clipboard

+2,162-85

Commits

Commits on Nov 30, 2019

• Polished main

  albertzaharovits

  committed
  92a3b34

Commits on Dec 1, 2019

• First successful tests

  albertzaharovits

  committed
  d07c05f

• More tests

  albertzaharovits

  committed
  de603a7

• More tests

  albertzaharovits

  committed
  7cc62e0

• BufferOnMarkInputStreamBug

  albertzaharovits

  committed
  cbd3c50

Commits on Dec 2, 2019

• A few bugs...

  albertzaharovits

  committed
  9eb9bcf

• BufferOnMark bug

  albertzaharovits

  committed
  5919a11

• More more more bugs!

  albertzaharovits

  committed
  47f6aea

Commits on Dec 3, 2019

• Mad tests

  albertzaharovits

  committed
  8263062

• Manic testing

  albertzaharovits

  committed
  cf97ba2

Commits on Dec 4, 2019

• BufferOnMarkInputStreamTests completed

  albertzaharovits

  committed
  3c82ba9

• Checkstyle

  albertzaharovits

  committed
  76d8271

• Merge branch 'repository-encrypted-client-side' into packet-based-cipherstream-2

  albertzaharovits

  committed
  6b30902

Commits on Dec 5, 2019

• BufferOnMarkInputStream javadocs

  albertzaharovits

  committed
  4e9778e

• merge fallout

  albertzaharovits

  committed
  24d6d27

• PrefixInputStream tests

  albertzaharovits

  committed
  3cd79bd

• WIP

  albertzaharovits

  committed
  c610fe8

• CountingInputStreamTests

  albertzaharovits

  committed
  e4f8564

• Renaming and more javadocs

  albertzaharovits

  committed
  c816c45

• Refactor ChainingInputStream

  albertzaharovits

  committed
  29c484b

Commits on Dec 6, 2019

• Scarce EncryptionPacketsInputStream javadocs

  albertzaharovits

  committed
  db5f58e

Commits on Dec 8, 2019

• ChainingInputStream polishing and tests

  albertzaharovits

  committed
  d6dc875

• ChainingInputStreamTests

  albertzaharovits

  committed
  7cb48f6

Commits on Dec 9, 2019

• ChainingInputStreamTests without mark/reset

  albertzaharovits

  committed
  83e028b

• ChainingInputStreamTests mark/reset

  albertzaharovits

  committed
  26a624f

• More tests

  albertzaharovits

  committed
  aeb6698

Commits on Dec 10, 2019

• WIP

  albertzaharovits

  committed
  2939289

• DecryptionPacketsInputStream tests

  albertzaharovits

  committed
  76678a6

• Tests done!

  albertzaharovits

  committed
  f44b97c

• More javadocs

  albertzaharovits

  committed
  016164a

Commits on Dec 18, 2019

• Merge branch 'repository-encrypted-client-side' into packet-based-cipherstream-2

  albertzaharovits

  committed
  5b26ff6

Commits on Dec 19, 2019

• Repository and plugin, WIP

  albertzaharovits

  committed
  bae9a8d

• Mhm, needs bc-jar

  albertzaharovits

  committed
  80a079b

Commits on Dec 20, 2019

• WIP how to prevent multiple repositories with different passwords

  albertzaharovits

  committed
  0a4cc77

Commits on Dec 23, 2019

• Update x-pack/plugin/repository-encrypted/src/main/java/org/elasticsearch/repositories/encrypted/ChainingInputStream.java

  Show description for a0751c6
  

  albertzaharovits

  and

  tvernum

  authored
  a0751c6

• Update x-pack/plugin/repository-encrypted/src/main/java/org/elasticsearch/repositories/encrypted/ChainingInputStream.java

  Show description for 92e177f
  

  albertzaharovits

  and

  tvernum

  authored
  92e177f

• Update x-pack/plugin/repository-encrypted/src/main/java/org/elasticsearch/repositories/encrypted/EncryptionPacketsInputStream.java

  Show description for 5d9321a
  

  albertzaharovits

  and

  tvernum

  authored
  5d9321a

Commits on Dec 25, 2019

• Tim's review WIP

  albertzaharovits

  committed
  c231486

• ChainingInputStream javadocs

  albertzaharovits

  committed
  bda96b6

• Logging on component close

  albertzaharovits

  committed
  cb6006d

• Merge branch 'repository-encrypted-client-side' into packet-based-cipherstream-2

  albertzaharovits

  committed
  fad9eb4

• Nit

  albertzaharovits

  committed
  65f0adb

Commits on Dec 26, 2019

• No Randomness in ChainingInputStreamTests

  albertzaharovits

  committed
  b40d999

• Update x-pack/plugin/repository-encrypted/src/test/java/org/elasticsearch/repositories/encrypted/ChainingInputStreamTests.java

  Show description for cb7bc1c
  

  albertzaharovits

  and

  tvernum

  authored
  cb7bc1c

Commits on Dec 27, 2019

• Tim's review WIP before mark/reset review

  albertzaharovits

  committed
  7acaf54

• Almost WIP

  albertzaharovits

  committed
  dfeea83

Commits on Dec 28, 2019

• Review complete

  albertzaharovits

  committed
  da29e2f

Commits on Dec 29, 2019

• Merge branch 'packet-based-cipherstream-2' into encrypted-blob-store

  albertzaharovits

  committed
  8c465be

• Merge fallout

  albertzaharovits

  committed
  bed1e51

Commits on Dec 30, 2019

• Update x-pack/plugin/repository-encrypted/src/main/java/org/elasticsearch/repositories/encrypted/EncryptionPacketsInputStream.java

  Show description for 5e7269b
  

  albertzaharovits

  and

  tvernum

  authored
  5e7269b

• IV position

  albertzaharovits

  committed
  14ee4aa

• Update x-pack/plugin/repository-encrypted/src/main/java/org/elasticsearch/repositories/encrypted/PrefixInputStream.java

  Show description for 7ee63c0
  

  albertzaharovits

  and

  tvernum

  authored
  7ee63c0

• move mark supported before mark/reset implementations

  albertzaharovits

  committed
  74f38b2

• Package-protected instead of protected for final classes

  albertzaharovits

  committed
  fee2d79

• RemainingPrefixByteCount

  albertzaharovits

  committed
  eda1fe0

• Update x-pack/plugin/repository-encrypted/src/main/java/org/elasticsearch/repositories/encrypted/DecryptionPacketsInputStream.java

  Show description for e85aefe
  

  albertzaharovits

  and

  tvernum

  authored
  e85aefe

• no iv instance variable

  albertzaharovits

  committed
  8a0773a

• Nit

  albertzaharovits

  committed
  07d7ac8

• Exception messages

  albertzaharovits

  committed
  fd10914

• Fix tests with exception names

  albertzaharovits

  committed
  2e41d4f

• Test for reader of fewer bytes

  albertzaharovits

  committed
  97f5917

• Adjust counting input stream docs

  albertzaharovits

  committed
  4fd6dcc

Commits on Dec 31, 2019

• RingBuffer

  albertzaharovits

  committed
  3d1daf4

• WIP

  albertzaharovits

  committed
  4fcd49d

• WIP

  albertzaharovits

  committed
  9ef136e

Commits on Jan 1, 2020

• More javadoc to the ring buffer inner

  albertzaharovits

  committed
  cb966b2

• Small test polishing

  albertzaharovits

  committed
  0f9f77c

• Merge branch 'packet-based-cipherstream-2' into encrypted-blob-store

  albertzaharovits

  committed
  5924356

Commits on Jan 5, 2020

• Encryption without CMS

  albertzaharovits

  committed
  3cf85b4

• Javadoc

  albertzaharovits

  committed
  151b248

• Snapshot works

  albertzaharovits

  committed
  7d1b3cc

Commits on Jan 6, 2020

• Remove bouncy castle dependency

  albertzaharovits

  committed
  31a2b30

Commits on Jan 7, 2020

• Password based metadata encryptor

  albertzaharovits

  committed
  591e48c

• Proper reordering of the two writes

  albertzaharovits

  committed
  1ee490a

• License check

  albertzaharovits

  committed
  5a1eee5

• Ooops metadata decrypt logic flaw

  albertzaharovits

  committed
  e51a304

Commits on Jan 8, 2020

• License and consistent

  albertzaharovits

  committed
  1087294

• Consistent Secure Settings reshuffle

  albertzaharovits

  committed
  b788e98

• Consistency check

  albertzaharovits

  committed
  4e204a3

Commits on Jan 9, 2020

• Cleanup WIP

  albertzaharovits

  committed
  3248163

Commits on Jan 10, 2020

• Renames and nits

  albertzaharovits

  committed
  f1406ff

• Merge branch 'repository-encrypted-client-side' into encrypted-blob-store

  albertzaharovits

  committed
  78c1984

• Checkstyle

  albertzaharovits

  committed
  2565d6e

Commits on Jan 13, 2020

• Consistency check on create snapshot

  albertzaharovits

  committed
  2c40aad

Commits on Jan 14, 2020

• Merge branch 'repository-encrypted-client-side' into encrypted-blob-store

  albertzaharovits

  committed
  db374fb

• More javadocs for PasswordBasedEncryptor

  albertzaharovits

  committed
  f73b54c

Commits on Jan 15, 2020

• EncryptedRepository#restoreShard logging

  albertzaharovits

  committed
  36062db

• Verify check

  albertzaharovits

  committed
  677bbb7

Commits on Jan 20, 2020

• Err msg nits

  albertzaharovits

  committed
  d2133e0

• Merge branch 'repository-encrypted-client-side' into encrypted-blob-store

  albertzaharovits

  committed
  6353f7a

• Revert SnapshotsService

  albertzaharovits

  committed
  a26b2b9

• PasswordBasedEncryptor javadoc

  albertzaharovits

  committed
  948b3c6

Commits on Jan 22, 2020

• Revert Consistent Settings check changes

  albertzaharovits

  committed
  83c0e44

Commits on Jan 28, 2020

• Merge branch 'repository-encrypted-client-side' into encrypted-blob-store

  albertzaharovits

  committed
  da10d0a

Commits on Jan 29, 2020

• Rename xpack license test method to reflect that only encr snapshot are

  Show description for 0a14765
  albertzaharovits

  committed
  0a14765

• Nits

  albertzaharovits

  committed
  6d562fc

Commits on Jan 30, 2020

• Authenticated metadata

  albertzaharovits

  committed
  bceda55

• Merge branch 'repository-encrypted-client-side' into encrypted-blob-store

  albertzaharovits

  committed
  eeac0b5

Commits on Jan 31, 2020

• Fix all compilation errors

  albertzaharovits

  committed
  29fa2aa

Commits on Feb 2, 2020

• Prepend name of the metadata to the encrypted blob

  albertzaharovits

  committed
  bb33045

Commits on Feb 3, 2020

• Nits

  albertzaharovits

  committed
  0308140

• Before rewriting cleanup

  albertzaharovits

  committed
  d7e400c

• Merge branch 'repository-encrypted-client-side' into encrypted-blob-store

  albertzaharovits

  committed
  8751c7d

Commits on Feb 4, 2020

• Refactor delete + cleanup

  albertzaharovits

  committed
  90bcaab

• BlobEncryptionMetadata getInt bug

  albertzaharovits

  committed
  57ab5ad

• Fix delete and cleanup for unknown metadata

  albertzaharovits

  committed
  5d9d4c1

• Try make custom metadata not display to user

  albertzaharovits

  committed
  03a58ed

• Fix children

  albertzaharovits

  committed
  3ee957d

Commits on Feb 5, 2020

• missing encryption metadata handling

  albertzaharovits

  committed
  82e8a89

Commits on Feb 6, 2020

• Checkstyle

  albertzaharovits

  committed
  b994d28

• Checkstyle

  albertzaharovits

  committed
  2e14453

Commits on Feb 11, 2020

• Merge branch 'repository-encrypted-client-side' into encrypted-blob-store

  albertzaharovits

  committed
  19fe147

• writeSnapshotIndexLatestBlob

  albertzaharovits

  committed
  4aaf895

• Write index.latest un-encrypted

  albertzaharovits

  committed
  8a9fe5d

• Keep all metadata blobs until the data blob is deleted

  albertzaharovits

  committed
  80e536c

Commits on Feb 12, 2020

• IntegTests in progress

  albertzaharovits

  committed
  60c4ad9

Commits on Feb 13, 2020

• Yey! EncryptedFS tests!!!

  albertzaharovits

  committed
  7953174

Commits on Feb 14, 2020

• Trimmings

  albertzaharovits

  committed
  9f7f4b4

• checkstyle

  albertzaharovits

  committed
  19a7828

Commits on Feb 15, 2020

• invert stale-surviving semantics

  albertzaharovits

  committed
  63c6855

Commits on Feb 16, 2020

• Cleanup WIP

  albertzaharovits

  committed
  b593d66

• Revert "invert stale-surviving semantics"

  Show description for 30d4f58
  albertzaharovits

  committed
  30d4f58

• BlobStoreRepository reshuffle stale root blobs and indices

  albertzaharovits

  committed
  1745345

Commits on Feb 17, 2020

• Revert "BlobStoreRepository reshuffle stale root blobs and indices"

  Show description for 68b9235
  albertzaharovits

  committed
  68b9235

• Revert "Cleanup WIP"

  Show description for 74c3d58
  albertzaharovits

  committed
  74c3d58

• Append repository generation to metadata blob name

  albertzaharovits

  committed
  1889aba

Commits on Feb 18, 2020

• deleteBlobsIgnoringIfNotExists refactor for new metadata blob name

  albertzaharovits

  committed
  3d82aa4

• nothing

  albertzaharovits

  committed
  b085c12

• cleanUnreferencedEncryptionMetadata

  albertzaharovits

  committed
  901b04a

• Refactor cleanup for the new metadata name

  albertzaharovits

  committed
  0ba402a

Commits on Feb 19, 2020

• Cleanup verify readonly

  albertzaharovits

  committed
  0d7077d

• Merge branch 'repository-encrypted-client-side' into encrypted-blob-store

  albertzaharovits

  committed
  1b0d732

• RepositoryMetaVersion from merge

  albertzaharovits

  committed
  6cf5812

• checkstyle

  albertzaharovits

  committed
  9e8001b

Commits on Feb 20, 2020

• Test nit

  albertzaharovits

  committed
  7ffaefb

• Encrypted GCS

  albertzaharovits

  committed
  950686f

• Encrypted Azure WIP

  albertzaharovits

  committed
  710ef33

• Prepending meta id is WRONG!

  albertzaharovits

  committed
  b88ec6d

Commits on Feb 21, 2020

• ChainingInputStream#chain fixes EncryptedAzureBlobStoreRepository

  albertzaharovits

  committed
  426bd05

• Merge branch 'repository-encrypted-client-side' into encrypted-blob-store

  albertzaharovits

  committed
  6936339

• License code after refactor

  albertzaharovits

  committed
  10e84d3

Commits on Feb 23, 2020

• EncryptedS3BlobStoreRepositoryIntegTests and MockHttpServer

  albertzaharovits

  committed
  94b3847

• Nit

  albertzaharovits

  committed
  6708354

• bad logger

  albertzaharovits

  committed
  12a074b

Commits on Feb 24, 2020

• ESMockAPIBasedRepositoryIntegTestCase Executor Service

  albertzaharovits

  committed
  a8c931c

• Explain that readBlob requires two concurrent connections

  albertzaharovits

  committed
  ffe0b63

Commits on Feb 25, 2020

• Add EncryptedRepository#getEncryptedBlobByteLength

  albertzaharovits

  committed
  28dd9e1

• Log unrecognized blobs during cleanup

  albertzaharovits

  committed
  b9ba309

• Fix EncryptedRepository#cleanup bug

  albertzaharovits

  committed
  ef4c860

• Add EncryptedGoogleCloudStorageThirdPartyTests

  albertzaharovits

  committed
  40fa0eb

• use ThreadPool#terminate in ESMockAPIBasedRepository...

  albertzaharovits

  committed
  b7e140c

• Revert to wrapping the HTTP IO executor executor in

  Show description for fe505e5
  albertzaharovits

  committed
  fe505e5

• Use IOUtils.close

  albertzaharovits

  committed
  8e036f0

• Refactor validateRepositoryPasswordHash

  albertzaharovits

  committed
  262b3e9

• Log password validation exceptions

  albertzaharovits

  committed
  a7e2dac

Commits on Mar 5, 2020

• Merge branch 'repository-encrypted-client-side' into encrypted-blob-store

  albertzaharovits

  committed
  62c1566

• Merge branch 'repository-encrypted-client-side' into encrypted-blob-store

  albertzaharovits

  committed
  83031ae

• Merge branch 'repository-encrypted-client-side' into encrypted-blob-store

  albertzaharovits

  committed
  d81d20a

• BlobPathTests

  albertzaharovits

  committed
  856cfe3

Commits on Mar 10, 2020

• Merge branch 'repository-encrypted-client-side' into encrypted-blob-store

  albertzaharovits

  committed
  bcc42df