Skip to content

Commit

Permalink
Update S2 compression
Browse files Browse the repository at this point in the history
Relevant updates:

* Less allocations on decode: klauspost/compress#322
* Fixed rare out-of-bounds write on amd64.
* ARM64 decompression assembly. Around 2x output speed. klauspost/compress#324
* Speed up decompression on non-assembly platforms. klauspost/compress#328

Upgrade cpuid to match simdjson.
  • Loading branch information
klauspost committed Mar 10, 2021
1 parent 2f29719 commit 6c8f004
Show file tree
Hide file tree
Showing 3 changed files with 12 additions and 9 deletions.
4 changes: 2 additions & 2 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,8 @@ require (
github.com/hashicorp/vault/api v1.0.4
github.com/jcmturner/gokrb5/v8 v8.4.2
github.com/json-iterator/go v1.1.10
github.com/klauspost/compress v1.11.7
github.com/klauspost/cpuid v1.3.1
github.com/klauspost/compress v1.11.12
github.com/klauspost/cpuid/v2 v2.0.4
github.com/klauspost/pgzip v1.2.5
github.com/klauspost/readahead v1.3.1
github.com/klauspost/reedsolomon v1.9.11
Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -333,6 +333,8 @@ github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+o
github.com/klauspost/compress v1.11.0/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
github.com/klauspost/compress v1.11.7 h1:0hzRabrMN4tSTvMfnL3SCv1ZGeAP23ynzodBgaHeMeg=
github.com/klauspost/compress v1.11.7/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
github.com/klauspost/compress v1.11.12 h1:famVnQVu7QwryBN4jNseQdUKES71ZAOnB6UQQJPZvqk=
github.com/klauspost/compress v1.11.12/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
github.com/klauspost/cpuid v1.2.3/go.mod h1:Pj4uuM528wm8OyEC2QMXAi2YiTZ96dNQPGgoMS4s3ek=
github.com/klauspost/cpuid v1.3.1 h1:5JNjFYYQrZeKRJ0734q51WCEEn2huer72Dc7K+R/b6s=
github.com/klauspost/cpuid v1.3.1/go.mod h1:bYW4mA6ZgKPob1/Dlai2LviZJO7KGI3uoWLd42rAQw4=
Expand Down
15 changes: 8 additions & 7 deletions pkg/s3select/select_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,13 @@ import (
"fmt"
"io"
"io/ioutil"
"math"
"net/http"
"os"
"reflect"
"strings"
"testing"

"github.com/klauspost/cpuid"
"github.com/klauspost/cpuid/v2"
"github.com/minio/minio-go/v7"
"github.com/minio/simdjson-go"
)
Expand Down Expand Up @@ -363,11 +362,13 @@ func TestJSONQueries(t *testing.T) {
t.Run(testCase.name, func(t *testing.T) {
// Hack cpuid to the CPU doesn't appear to support AVX2.
// Restore whatever happens.
defer func(f cpuid.Flags) {
cpuid.CPU.Features = f
}(cpuid.CPU.Features)
cpuid.CPU.Features &= math.MaxUint64 - cpuid.AVX2

if cpuid.CPU.Supports(cpuid.AVX2) {
cpuid.CPU.Disable(cpuid.AVX2)
defer cpuid.CPU.Enable(cpuid.AVX2)
}
if simdjson.SupportedCPU() {
t.Fatal("setup error: expected cpu to be unsupported")
}
testReq := testCase.requestXML
if len(testReq) == 0 {
var escaped bytes.Buffer
Expand Down

0 comments on commit 6c8f004

Please sign in to comment.