Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merge pending] Add mirror gateway definition #2237

Closed
wants to merge 69 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
0fc1949
add mirror definition and update proto dependencies
hlts2 Feb 9, 2023
e0472c2
Add mirror gateway component (#1949)
hlts2 Apr 25, 2023
b0aef58
Rename ip to host of proto payload field for mirror gateway (#2031)
hlts2 May 18, 2023
c3f987a
make proto/add & make format
hlts2 Jun 16, 2023
6347736
fix docker build for mirror
hlts2 Jun 19, 2023
2c0d146
Revert "fix docker build for mirror"
hlts2 Jun 19, 2023
e905008
fix docker build error for mirror gateway
hlts2 Jun 19, 2023
48ad760
Add Custom Resource for mirror target (#2023)
hlts2 Jul 4, 2023
1f24460
fix mirror image tag
hlts2 Jul 4, 2023
8bbe931
Add mirror gateway document (#2105)
hlts2 Jul 25, 2023
9d40824
merge ingress configuration for lb and filter
hlts2 Aug 15, 2023
7e4c2c7
fix duplicate definition error
hlts2 Aug 15, 2023
bd35400
make proto/all & make schema update & make format
hlts2 Aug 15, 2023
9fd7fa2
fix test execution error for mirror handler
hlts2 Aug 15, 2023
91ea275
make proto/all format and update schema and fails build error
hlts2 Sep 21, 2023
c26ac88
use resuable workflow to build mirror image
hlts2 Sep 21, 2023
8a172d9
use internal sync package and refactor
hlts2 Sep 21, 2023
65af965
fix error handling
hlts2 Sep 21, 2023
19558ab
add network policy settings and update deploy command
hlts2 Sep 21, 2023
3924e8f
add default setting logic
hlts2 Sep 21, 2023
8fabca9
add new fileld to crd schema and update crd
hlts2 Sep 25, 2023
00b9e01
fix make command to deploy vald with mirror components
hlts2 Sep 25, 2023
6eaba03
add e2e test for mirror
hlts2 Sep 26, 2023
4fea273
add debug command to get pod status
hlts2 Sep 26, 2023
b32c705
bugfix command
hlts2 Sep 26, 2023
37923f3
disable sidecar e2e and add mirror e2e command
hlts2 Sep 26, 2023
b410364
update cluster role
hlts2 Sep 26, 2023
7ece16f
add cluster-info dump command
hlts2 Sep 26, 2023
2ed49ca
Revert "add cluster-info dump command"
hlts2 Sep 26, 2023
454eaf3
debug: add logs command
hlts2 Sep 27, 2023
388f8ef
Revert "debug: add logs command"
hlts2 Sep 27, 2023
91deb87
add RemoveByTimestamp for mirror
hlts2 Sep 27, 2023
0b96823
fix: update sleep time for operator deploy
hlts2 Sep 28, 2023
86f4211
update wait time for vmt
hlts2 Sep 28, 2023
5ea8d4f
Returns all connection information in the registration rpc (#2199)
hlts2 Oct 2, 2023
192ae24
Add handler logic to continue processing (#2204)
hlts2 Oct 11, 2023
9661fc3
Update Mirror Gateway document (#2207)
hlts2 Nov 8, 2023
746e01c
Merge branch 'main' into feature/mirror-gateway-definition
hlts2 Nov 16, 2023
08c44fb
feat: schema and protobuf update
hlts2 Nov 16, 2023
7d7a690
fix: reviewdog warning and add comment
hlts2 Nov 16, 2023
c9007ed
style: format code with Gofumpt and Prettier
deepsource-autofix[bot] Nov 16, 2023
78734df
fix: reviewdog warning
hlts2 Nov 16, 2023
88dad2e
fix: reviewdog warning
hlts2 Nov 16, 2023
ee0f48a
fix: move mirror client mock definition to internal package
hlts2 Nov 16, 2023
374dc57
feat: add new handler base logic
hlts2 Nov 21, 2023
9e87530
fix: refactor stream list object rpc
hlts2 Nov 21, 2023
e62a657
fix: divide logic to other function
hlts2 Nov 22, 2023
6c6b744
style: format code with Gofumpt and Prettier
deepsource-autofix[bot] Nov 22, 2023
c825ad8
Merge branch 'main' into feature/mirror-gateway-definition
hlts2 Nov 29, 2023
104c22b
make proto/all format
hlts2 Nov 29, 2023
a36f1a8
fix: return grpc error
hlts2 Nov 29, 2023
1512b78
Merge branch 'main' into feature/mirror-gateway-definition
hlts2 Dec 4, 2023
23b1f72
fix: make proto/all format
hlts2 Dec 4, 2023
68a8072
feat: add mirror service test
hlts2 Dec 4, 2023
d6580ec
fix: add mock definition code
hlts2 Dec 4, 2023
2b1cac5
style: format code with Gofumpt and Prettier
deepsource-autofix[bot] Dec 4, 2023
19ee272
fix: use internal method
hlts2 Dec 5, 2023
f5afe4c
fix: delete Exist method because it is not needed
hlts2 Dec 5, 2023
6c2789c
fix: use address cache insted of connectedAddr method
hlts2 Dec 5, 2023
70e2e17
Merge branch 'main' into feature/mirror-gateway-definition
hlts2 Dec 5, 2023
b032917
fix: make format
hlts2 Dec 5, 2023
f34b142
feat: improve connection management and add new test
hlts2 Dec 5, 2023
7c502f0
fix: make format
hlts2 Dec 5, 2023
1cb6662
fix: lint warnning
hlts2 Dec 5, 2023
ffa4eb6
fix: refactor error handling
hlts2 Dec 6, 2023
21b781f
feat: add test for syncronizee based on address and small refactor
hlts2 Dec 6, 2023
1aef5da
Merge branch 'main' into feature/mirror-gateway-definition
hlts2 Dec 6, 2023
08a6bc1
fix: improve error handling and example image tag
hlts2 Dec 7, 2023
5bee190
fix: schema update and format
hlts2 Dec 7, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix: return grpc error
Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com>
hlts2 committed Nov 29, 2023

Verified

This commit was signed with the committer’s verified signature.
hlts2 Hiroto Funakoshi
commit a36f1a87955de7c6ad72778d23fdd2c518a0b693
2 changes: 1 addition & 1 deletion pkg/gateway/mirror/handler/grpc/handler.go
Original file line number Diff line number Diff line change
@@ -3136,8 +3136,8 @@ func (s *server) StreamListObject(req *payload.Object_List_Request, stream vald.
return obj, s.streamListObject(ctx, client, stream)
})
if err != nil {
st, msg, err := status.ParseError(err, codes.Internal, "failed to parse "+vald.StreamListObjectRPCName+" gRPC error response")
if span != nil {
st, msg, err := status.ParseError(err, codes.Internal, "failed to parse "+vald.StreamListObjectRPCName+" gRPC error response")
span.RecordError(err)
span.SetAttributes(trace.FromGRPCStatus(st.Code(), msg)...)
span.SetStatus(trace.StatusError, err.Error())

Unchanged files with check annotations Beta

/*
#cgo LDFLAGS: -lngt
#include <NGT/Capi.h>

Check failure on line 22 in internal/core/algorithm/ngt/ngt.go

GitHub Actions / Analyze (go)

fatal error: NGT/Capi.h: No such file or directory
#include <stdlib.h>
*/
import "C"
if err != nil {
switch st {
case StateClosed:
cerr := errors.Wrapf(err, "circuitbreaker state is %s, this error is not caused by circuitbreaker", st.String())
log.Debug(cerr)

Check warning on line 90 in internal/circuitbreaker/manager.go

Codecov / codecov/patch

internal/circuitbreaker/manager.go#L89-L90

Added lines #L89 - L90 were not covered by tests
case StateOpen:
if !errors.Is(err, errors.ErrCircuitBreakerOpenState) {
err = errors.Join(err, errors.ErrCircuitBreakerOpenState)
}
// Bind binds the actual data from the Mirror receiver fields.
func (m *Mirror) Bind() *Mirror {
m.SelfMirrorAddr = GetActualValue(m.SelfMirrorAddr)
m.GatewayAddr = GetActualValue(m.GatewayAddr)
m.PodName = GetActualValue(m.PodName)
m.RegisterDuration = GetActualValue(m.RegisterDuration)
m.Namespace = GetActualValue(m.Namespace)
m.DiscoveryDuration = GetActualValue(m.DiscoveryDuration)
m.Colocation = GetActualValue(m.Colocation)
m.Group = GetActualValue(m.Group)
if m.Net != nil {
m.Net = m.Net.Bind()
} else {
m.Net = new(Net).Bind()
}
if m.Client != nil {
m.Client = m.Client.Bind()
} else {
m.Client = new(GRPCClient).Bind()
}
return m

Check warning on line 71 in internal/config/mirror.go

Codecov / codecov/patch

internal/config/mirror.go#L51-L71

Added lines #L51 - L71 were not covered by tests
}
}
// TODO delete <<<---
func (c *controller) GetManager() Manager {
return c.mgr

Check warning on line 166 in internal/k8s/reconciler.go

Codecov / codecov/patch

internal/k8s/reconciler.go#L165-L166

Added lines #L165 - L166 were not covered by tests
}
// TODO delete --->>>
elapsedTime := time.Since(now)
record(ctx, info.FullMethod, err, float64(elapsedTime)/float64(time.Millisecond))
return resp, err
}, func(srv interface{}, ss grpc.ServerStream, info *grpc.StreamServerInfo, handler grpc.StreamHandler) (err error) {

Check warning on line 68 in internal/net/grpc/interceptor/server/metric/metric.go

Codecov / codecov/patch

internal/net/grpc/interceptor/server/metric/metric.go#L68

Added line #L68 was not covered by tests
now := time.Now()
err = handler(srv, ss)
elapsedTime := time.Since(now)
type MD = metadata.MD
func NewOutgoingContext(ctx context.Context, md MD) context.Context {
return metadata.NewOutgoingContext(ctx, md)

Check warning on line 29 in internal/net/grpc/metadata.go

Codecov / codecov/patch

internal/net/grpc/metadata.go#L28-L29

Added lines #L28 - L29 were not covered by tests
}
func FromIncomingContext(ctx context.Context) (metadata.MD, bool) {
return metadata.FromIncomingContext(ctx)

Check warning on line 33 in internal/net/grpc/metadata.go

Codecov / codecov/patch

internal/net/grpc/metadata.go#L32-L33

Added lines #L32 - L33 were not covered by tests
}
WithName(func() string {
name, err := os.Hostname()
if err != nil {
log.Warn(err)
}

Check warning on line 36 in pkg/gateway/mirror/handler/grpc/option.go

Codecov / codecov/patch

pkg/gateway/mirror/handler/grpc/option.go#L35-L36

Added lines #L35 - L36 were not covered by tests
return name
}()),
WithIP(net.LoadLocalIP()),
// WithIP returns the option to set the IP for server.
func WithIP(ip string) Option {
return func(s *server) error {
if len(ip) != 0 {
s.ip = ip
}
return nil

Check warning on line 48 in pkg/gateway/mirror/handler/grpc/option.go

Codecov / codecov/patch

pkg/gateway/mirror/handler/grpc/option.go#L45-L48

Added lines #L45 - L48 were not covered by tests
}
}
// WithName returns the option to set the name for server.
func WithName(name string) Option {
return func(s *server) error {
if len(name) != 0 {
s.name = name
}
return nil

Check warning on line 58 in pkg/gateway/mirror/handler/grpc/option.go

Codecov / codecov/patch

pkg/gateway/mirror/handler/grpc/option.go#L55-L58

Added lines #L55 - L58 were not covered by tests
}
}
func WithGateway(g service.Gateway) Option {
return func(s *server) error {
if g != nil {
s.gateway = g
}
return nil

Check warning on line 67 in pkg/gateway/mirror/handler/grpc/option.go

Codecov / codecov/patch

pkg/gateway/mirror/handler/grpc/option.go#L62-L67

Added lines #L62 - L67 were not covered by tests
}
}
func WithMirror(m service.Mirror) Option {
return func(s *server) error {
if m != nil {
s.mirror = m
}
return nil

Check warning on line 76 in pkg/gateway/mirror/handler/grpc/option.go

Codecov / codecov/patch

pkg/gateway/mirror/handler/grpc/option.go#L71-L76

Added lines #L71 - L76 were not covered by tests
}
}
func WithErrGroup(eg errgroup.Group) Option {
return func(s *server) error {
if eg != nil {
s.eg = eg
}
return nil

Check warning on line 85 in pkg/gateway/mirror/handler/grpc/option.go

Codecov / codecov/patch

pkg/gateway/mirror/handler/grpc/option.go#L82-L85

Added lines #L82 - L85 were not covered by tests
}
}
func WithStreamConcurrency(c int) Option {
return func(s *server) error {
if c > 0 {
s.streamConcurrency = c
}
return nil

Check warning on line 94 in pkg/gateway/mirror/handler/grpc/option.go

Codecov / codecov/patch

pkg/gateway/mirror/handler/grpc/option.go#L91-L94

Added lines #L91 - L94 were not covered by tests
}
}
func WithValdAddr(addr string) Option {
return func(s *server) error {
if len(addr) == 0 {
return errors.NewErrCriticalOption("valdAddr", addr)
}
s.vAddr = addr
return nil

Check warning on line 104 in pkg/gateway/mirror/handler/grpc/option.go

Codecov / codecov/patch

pkg/gateway/mirror/handler/grpc/option.go#L98-L104

Added lines #L98 - L104 were not covered by tests
}
}