Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update libs dependency #2775

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
Open

Update libs dependency #2775

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
b4abdb4
[create-pull-request] automated change
vdaas-ci Dec 13, 2024
004df91
Merge branch 'main' into create-pull-request/patch
vankichi Dec 17, 2024
9f97f75
:wrench: Fix build error
vankichi Dec 19, 2024
2513650
Added output for Go Module with fixed Version (#2784)
kpango Dec 19, 2024
bd80f2b
:wrench: Update go deps
vankichi Dec 19, 2024
4622c52
:wrench: Update deps
vankichi Dec 19, 2024
eb049c4
:white_check_mark: Fix race condition
vankichi Dec 19, 2024
98d83e7
Revert ":wrench: Update deps"
vankichi Dec 19, 2024
1d74d13
:wrench: Update deps
vankichi Dec 19, 2024
fe700ed
:wrench: Update deps
vankichi Dec 19, 2024
d2102b2
Revert ":white_check_mark: Fix race condition"
vankichi Dec 19, 2024
589be29
:wrench: Update deps
vankichi Dec 19, 2024
b9a4264
:white_check_mark: Fix unit test
vankichi Dec 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug_report.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ assignees: ""
- Go Version: v1.23.4
- Rust Version: v1.83.0
- Docker Version: v27.4.0
- Kubernetes Version: v1.31.4
- Kubernetes Version: v1.32.0
- Helm Version: v3.16.3
- NGT Version: v2.3.5
- Faiss Version: v1.9.0
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/security_issue_report.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ assignees: ""
- Go Version: v1.23.4
- Rust Version: v1.83.0
- Docker Version: v27.4.0
- Kubernetes Version: v1.31.4
- Kubernetes Version: v1.32.0
- Helm Version: v3.16.3
- NGT Version: v2.3.5
- Faiss Version: v1.9.0
2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
- Go Version: v1.23.4
- Rust Version: v1.83.0
- Docker Version: v27.4.0
- Kubernetes Version: v1.31.4
- Kubernetes Version: v1.32.0
- Helm Version: v3.16.3
- NGT Version: v2.3.5
- Faiss Version: v1.9.0
Expand Down
2 changes: 1 addition & 1 deletion apis/grpc/v1/agent/sidecar/sidecar_vtproto.pb.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ const _ = grpc.SupportPackageIsVersion7
// SidecarClient is the client API for Sidecar service.
//
// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.
type SidecarClient any
type SidecarClient interface{}

type sidecarClient struct {
cc grpc.ClientConnInterface
Expand Down
15 changes: 13 additions & 2 deletions charts/vald-benchmark-operator/values.schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,10 @@
"interceptors": {
"type": "array",
"description": "gRPC client interceptors",
"items": { "type": "string", "enum": ["TraceInterceptor"] }
"items": {
"type": "string",
"enum": ["TraceInterceptor", "MetricInterceptor"]
}
},
"keepalive": {
"type": "object",
Expand Down Expand Up @@ -698,7 +701,15 @@
"initial_window_size": { "type": "integer" },
"interceptors": {
"type": "array",
"items": { "type": "string" }
"items": {
"type": "string",
"enum": [
"RecoverInterceptor",
"AccessLogInterceptor",
"TraceInterceptor",
"MetricInterceptor"
]
}
},
"keepalive": {
"type": "object",
Expand Down
73 changes: 56 additions & 17 deletions charts/vald/values.schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3717,7 +3717,7 @@
"description": "gRPC client interceptors",
"items": {
"type": "string",
"enum": ["TraceInterceptor"]
"enum": ["TraceInterceptor", "MetricInterceptor"]
}
},
"keepalive": {
Expand Down Expand Up @@ -7053,7 +7053,10 @@
"description": "gRPC client interceptors",
"items": {
"type": "string",
"enum": ["TraceInterceptor"]
"enum": [
"TraceInterceptor",
"MetricInterceptor"
]
}
},
"keepalive": {
Expand Down Expand Up @@ -7413,7 +7416,7 @@
"description": "gRPC client interceptors",
"items": {
"type": "string",
"enum": ["TraceInterceptor"]
"enum": ["TraceInterceptor", "MetricInterceptor"]
}
},
"keepalive": {
Expand Down Expand Up @@ -7765,7 +7768,10 @@
"description": "gRPC client interceptors",
"items": {
"type": "string",
"enum": ["TraceInterceptor"]
"enum": [
"TraceInterceptor",
"MetricInterceptor"
]
}
},
"keepalive": {
Expand Down Expand Up @@ -9642,7 +9648,10 @@
"description": "gRPC client interceptors",
"items": {
"type": "string",
"enum": ["TraceInterceptor"]
"enum": [
"TraceInterceptor",
"MetricInterceptor"
]
}
},
"keepalive": {
Expand Down Expand Up @@ -9990,7 +9999,10 @@
"description": "gRPC client interceptors",
"items": {
"type": "string",
"enum": ["TraceInterceptor"]
"enum": [
"TraceInterceptor",
"MetricInterceptor"
]
}
},
"keepalive": {
Expand Down Expand Up @@ -10339,7 +10351,10 @@
"description": "gRPC client interceptors",
"items": {
"type": "string",
"enum": ["TraceInterceptor"]
"enum": [
"TraceInterceptor",
"MetricInterceptor"
]
}
},
"keepalive": {
Expand Down Expand Up @@ -12222,7 +12237,7 @@
"description": "gRPC client interceptors",
"items": {
"type": "string",
"enum": ["TraceInterceptor"]
"enum": ["TraceInterceptor", "MetricInterceptor"]
}
},
"keepalive": {
Expand Down Expand Up @@ -14278,7 +14293,10 @@
"description": "gRPC client interceptors",
"items": {
"type": "string",
"enum": ["TraceInterceptor"]
"enum": [
"TraceInterceptor",
"MetricInterceptor"
]
}
},
"keepalive": {
Expand Down Expand Up @@ -14626,7 +14644,10 @@
"description": "gRPC client interceptors",
"items": {
"type": "string",
"enum": ["TraceInterceptor"]
"enum": [
"TraceInterceptor",
"MetricInterceptor"
]
}
},
"keepalive": {
Expand Down Expand Up @@ -14989,7 +15010,7 @@
"description": "gRPC client interceptors",
"items": {
"type": "string",
"enum": ["TraceInterceptor"]
"enum": ["TraceInterceptor", "MetricInterceptor"]
}
},
"keepalive": {
Expand Down Expand Up @@ -16684,7 +16705,10 @@
"description": "gRPC client interceptors",
"items": {
"type": "string",
"enum": ["TraceInterceptor"]
"enum": [
"TraceInterceptor",
"MetricInterceptor"
]
}
},
"keepalive": {
Expand Down Expand Up @@ -17032,7 +17056,10 @@
"description": "gRPC client interceptors",
"items": {
"type": "string",
"enum": ["TraceInterceptor"]
"enum": [
"TraceInterceptor",
"MetricInterceptor"
]
}
},
"keepalive": {
Expand Down Expand Up @@ -18730,7 +18757,10 @@
"description": "gRPC client interceptors",
"items": {
"type": "string",
"enum": ["TraceInterceptor"]
"enum": [
"TraceInterceptor",
"MetricInterceptor"
]
}
},
"keepalive": {
Expand Down Expand Up @@ -19078,7 +19108,10 @@
"description": "gRPC client interceptors",
"items": {
"type": "string",
"enum": ["TraceInterceptor"]
"enum": [
"TraceInterceptor",
"MetricInterceptor"
]
}
},
"keepalive": {
Expand Down Expand Up @@ -22455,7 +22488,10 @@
"description": "gRPC client interceptors",
"items": {
"type": "string",
"enum": ["TraceInterceptor"]
"enum": [
"TraceInterceptor",
"MetricInterceptor"
]
}
},
"keepalive": {
Expand Down Expand Up @@ -22803,7 +22839,10 @@
"description": "gRPC client interceptors",
"items": {
"type": "string",
"enum": ["TraceInterceptor"]
"enum": [
"TraceInterceptor",
"MetricInterceptor"
]
}
},
"keepalive": {
Expand Down
10 changes: 5 additions & 5 deletions example/client/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,17 @@ go 1.23.4

replace (
github.com/envoyproxy/protoc-gen-validate => github.com/envoyproxy/protoc-gen-validate v1.1.0
github.com/goccy/go-json => github.com/goccy/go-json v0.10.3
github.com/goccy/go-json => github.com/goccy/go-json v0.10.4
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Based on the analysis of the shell script outputs, I can now generate the final response:

Review dependency updates with security considerations

The dependency updates require attention in the following areas:

  • The codebase has proper TLS security configurations in place, with explicit handling of secure connections in multiple components (redis, mysql, cassandra, grpc)
  • There are several test files still using the deprecated grpc.WithInsecure() that should be updated:
    • hack/benchmark/e2e/agent/core/ngt/ngt_bench_test.go
    • pkg/agent/core/ngt/service/ngt_test.go
    • tests/e2e/performance/max_vector_dim_test.go

Please update the test files to use the newer insecure.NewCredentials() instead of the deprecated grpc.WithInsecure() when upgrading to gRPC v1.69.0.

🔗 Analysis chain

Verify breaking changes in critical dependency updates

Several security-critical dependencies have been updated:

  • golang.org/x/crypto: v0.31.0
  • google.golang.org/grpc: v1.69.0
  • google.golang.org/protobuf: v1.36.0

Also applies to: 11-11, 14-18

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for breaking changes in release notes and security advisories

# Check for any Go files using deprecated APIs
echo "Checking for potential breaking changes in gRPC usage..."
rg -l 'grpc\.' | while read -r file; do
  if [[ "$file" =~ \.go$ ]]; then
    echo "Analyzing $file for gRPC API usage..."
    ast-grep --pattern 'grpc.$_' "$file"
  fi
done

# Check for security advisories
echo "Checking security advisories..."
gh api graphql -f query='
{
  securityVulnerabilities(first: 5, ecosystem: GO, package: "google.golang.org/grpc") {
    nodes {
      advisory {
        summary
        severity
        publishedAt
      }
      vulnerableVersionRange
      firstPatchedVersion {
        identifier
      }
    }
  }
}'

Length of output: 112332

Script:

#!/bin/bash
# Let's check for breaking changes in gRPC v1.69.0 release notes and verify version compatibility

# Check gRPC version in go.mod
echo "Checking gRPC version in go.mod..."
rg "google.golang.org/grpc" -A 1 -B 1 example/client/go.mod

# Check for any deprecated API usage
echo "Checking for potential deprecated API usage..."
rg -l "grpc\." | while read -r file; do
  if [[ "$file" =~ \.go$ ]]; then
    echo "Analyzing $file for gRPC API usage..."
    # Look for specific deprecated patterns
    rg -A 2 -B 2 "grpc\.(WithInsecure|WithCompressor|WithDecompressor|WithUserAgent|WithTimeout)" "$file" || true
  fi
done

# Check for crypto package usage
echo "Checking crypto package usage..."
rg "golang.org/x/crypto" -A 1 -B 1 example/client/go.mod

# Check for protobuf version
echo "Checking protobuf version..."
rg "google.golang.org/protobuf" -A 1 -B 1 example/client/go.mod

Length of output: 67525

Script:

#!/bin/bash
# Let's check for any security advisories and breaking changes in the release notes

# Check for any security advisories for grpc v1.69.0
echo "Checking gRPC security advisories..."
gh api graphql -f query='
{
  securityVulnerabilities(first: 5, ecosystem: GO, package: "google.golang.org/grpc", package: "golang.org/x/crypto", package: "google.golang.org/protobuf") {
    nodes {
      advisory {
        summary
        severity
        publishedAt
      }
      vulnerableVersionRange
      firstPatchedVersion {
        identifier
      }
    }
  }
}'

# Check for any TLS/security configuration in the codebase
echo "Checking TLS/security configurations..."
rg -A 3 -B 3 "WithTLS|WithTransportCredentials|WithInsecure\(false\)" internal/config/

# Check for any deprecated API usage in tests
echo "Checking test files for deprecated API usage..."
find . -name "*_test.go" -type f -exec grep -l "grpc\.WithInsecure" {} \;

Length of output: 3235

github.com/golang/protobuf => github.com/golang/protobuf v1.5.4
github.com/kpango/glg => github.com/kpango/glg v1.6.15
github.com/pkg/sftp => github.com/pkg/sftp v1.13.7
golang.org/x/crypto => golang.org/x/crypto v0.30.0
golang.org/x/crypto => golang.org/x/crypto v0.31.0
golang.org/x/net => golang.org/x/net v0.32.0
golang.org/x/text => golang.org/x/text v0.21.0
google.golang.org/genproto => google.golang.org/genproto v0.0.0-20241209162323-e6fa225c2576
google.golang.org/genproto/googleapis/api => google.golang.org/genproto/googleapis/api v0.0.0-20241209162323-e6fa225c2576
google.golang.org/genproto/googleapis/rpc => google.golang.org/genproto/googleapis/rpc v0.0.0-20241209162323-e6fa225c2576
google.golang.org/grpc => google.golang.org/grpc v1.68.1
google.golang.org/grpc => google.golang.org/grpc v1.69.0
google.golang.org/protobuf => google.golang.org/protobuf v1.35.2
gopkg.in/yaml.v3 => gopkg.in/yaml.v3 v3.0.1
sigs.k8s.io/yaml => sigs.k8s.io/yaml v1.4.0
Expand All @@ -33,10 +33,10 @@ require (
github.com/goccy/go-json v0.10.2 // indirect
github.com/kpango/fastime v1.1.9 // indirect
github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 // indirect
golang.org/x/net v0.29.0 // indirect
golang.org/x/net v0.30.0 // indirect
golang.org/x/sys v0.28.0 // indirect
golang.org/x/text v0.21.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20240903143218-8af14fe29dc1 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20241015192408-796eee8c2d53 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20241206012308-a4fef0638583 // indirect
google.golang.org/protobuf v1.35.2 // indirect
)
24 changes: 20 additions & 4 deletions example/client/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,17 @@
buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go v1.35.2-20241127180247-a33202765966.1 h1:jLd96rDDNJ+zIJxvV/L855VEtrjR0G4aePVDlCpf6kw=
buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go v1.35.2-20241127180247-a33202765966.1/go.mod h1:mnHCFccv4HwuIAOHNGdiIc5ZYbBCvbTWZcodLN5wITI=
github.com/goccy/go-json v0.10.3 h1:KZ5WoDbxAIgm2HNbYckL0se1fHD6rz5j4ywS6ebzDqA=
github.com/goccy/go-json v0.10.3/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY=
github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/goccy/go-json v0.10.4 h1:JSwxQzIqKfmFX1swYPpUThQZp/Ka4wzJdK0LWVytLPM=
github.com/goccy/go-json v0.10.4/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/kpango/fastime v1.1.9 h1:xVQHcqyPt5M69DyFH7g1EPRns1YQNap9d5eLhl/Jy84=
github.com/kpango/fastime v1.1.9/go.mod h1:vyD7FnUn08zxY4b/QFBZVG+9EWMYsNl+QF0uE46urD4=
github.com/kpango/fuid v0.0.0-20221203053508-503b5ad89aa1 h1:rxyM+7uaZQ35P9fbixdnld/h4AgEhODoubuy6A4nDdk=
Expand All @@ -18,6 +24,16 @@ github.com/sirupsen/logrus v1.9.0 h1:trlNQbNUG3OdDrDil03MCb1H2o9nJ1x4/5LYw7byDE0
github.com/sirupsen/logrus v1.9.0/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/vdaas/vald-client-go v1.7.15 h1:uOUlmRh7aJx2nyT+9Iv28uNzqdc9xlXyWbN5nLNUALM=
github.com/vdaas/vald-client-go v1.7.15/go.mod h1:5PYD1Cf1UqgevuzofZNIEtcXatjQSaXwmn8xHvY74jA=
go.opentelemetry.io/otel v1.31.0 h1:NsJcKPIW0D0H3NgzPDHmo0WW6SptzPdqg/L1zsIm2hY=
go.opentelemetry.io/otel v1.31.0/go.mod h1:O0C14Yl9FgkjqcCZAsE053C13OaddMYr/hz6clDkEJE=
go.opentelemetry.io/otel/metric v1.31.0 h1:FSErL0ATQAmYHUIzSezZibnyVlft1ybhy4ozRPcF2fE=
go.opentelemetry.io/otel/metric v1.31.0/go.mod h1:C3dEloVbLuYoX41KpmAhOqNriGbA+qqH6PQ5E5mUfnY=
go.opentelemetry.io/otel/sdk v1.31.0 h1:xLY3abVHYZ5HSfOg3l2E5LUj2Cwva5Y7yGxnSW9H5Gk=
go.opentelemetry.io/otel/sdk v1.31.0/go.mod h1:TfRbMdhvxIIr/B2N2LQW2S5v9m3gOQ/08KsbbO5BPT0=
go.opentelemetry.io/otel/sdk/metric v1.31.0 h1:i9hxxLJF/9kkvfHppyLL55aW7iIJz4JjxTeYusH7zMc=
go.opentelemetry.io/otel/sdk/metric v1.31.0/go.mod h1:CRInTMVvNhUKgSAMbKyTMxqOBC0zgyxzW55lZzX43Y8=
go.opentelemetry.io/otel/trace v1.31.0 h1:ffjsj1aRouKewfr85U2aGagJ46+MvodynlQ1HYdmJys=
go.opentelemetry.io/otel/trace v1.31.0/go.mod h1:TXZkRk7SM2ZQLtR6eoAWQFIHPvzQ06FJAsO1tJg480A=
go.uber.org/atomic v1.7.0 h1:ADUqmZGgLDDfbSL9ZmPxKTybcoEYHgpYfELNoN+7hsw=
go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
go.uber.org/multierr v1.6.0 h1:y6IPFStTAIT5Ytl7/XYmHvzXQ7S3g/IeZW9hyZ5thw4=
Expand All @@ -36,7 +52,7 @@ google.golang.org/genproto/googleapis/api v0.0.0-20241209162323-e6fa225c2576 h1:
google.golang.org/genproto/googleapis/api v0.0.0-20241209162323-e6fa225c2576/go.mod h1:1R3kvZ1dtP3+4p4d3G8uJ8rFk/fWlScl38vanWACI08=
google.golang.org/genproto/googleapis/rpc v0.0.0-20241209162323-e6fa225c2576 h1:8ZmaLZE4XWrtU3MyClkYqqtl6Oegr3235h7jxsDyqCY=
google.golang.org/genproto/googleapis/rpc v0.0.0-20241209162323-e6fa225c2576/go.mod h1:5uTbfoYQed2U9p3KIj2/Zzm02PYhndfdmML0qC3q3FU=
google.golang.org/grpc v1.68.1 h1:oI5oTa11+ng8r8XMMN7jAOmWfPZWbYpCFaMUTACxkM0=
google.golang.org/grpc v1.68.1/go.mod h1:+q1XYFJjShcqn0QZHvCyeR4CXPA+llXIeUIfIe00waw=
google.golang.org/grpc v1.69.0 h1:quSiOM1GJPmPH5XtU+BCoVXcDVJJAzNcoyfC2cCjGkI=
google.golang.org/grpc v1.69.0/go.mod h1:vyjdE6jLBI76dgpDojsFGNaHlxdjXN9ghpnd2o7JGZ4=
google.golang.org/protobuf v1.35.2 h1:8Ar7bF+apOIoThw1EdZl0p1oWvMqTHmpA2fRTyZO8io=
google.golang.org/protobuf v1.35.2/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE=
Loading
Loading