Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spin app with Redis trigger stuck in crash loop with no useful logs #201

Open
calebschoepp opened this issue Jan 19, 2024 · 5 comments
Open
Assignees
Labels
enhancement New feature or request

Comments

@calebschoepp
Copy link

I'm having trouble running a Spin app with the Redis trigger. The app is stuck in a crash loop. No Spin logs are being produced. Using the steps in #197 the last containerd-shim-spin log line I see is >>> running spin trigger.

My app source code is as follows:

//go:build ignore

package main

import (
	"fmt"

	"github.com/fermyon/spin/sdk/go/v2/redis"
)

func init() {
	// redis.Handle() must be called in the init() function.
	redis.Handle(func(payload []byte) error {
		fmt.Println("Payload::::")
		fmt.Println(string(payload))
		return nil
	})
}

// main functiion must be included for the compiler but is not executed.
func main() {}
spin_manifest_version = 2

[application]
name = "order-processor"
version = "0.1.0"
authors = ["Caleb Schoepp <[email protected]>"]
description = "Process orders off of redis queue"

[application.trigger.redis]
address = "redis://redis-master.default.svc.cluster.local:6379"

[[trigger.redis]]
channel = "orders"
component = "order-processor"

[component.order-processor]
source = "main.wasm"
allowed_outbound_hosts = []
[component.order-processor.build]
command = "tinygo build -target=wasi -gc=leaking -no-debug -o main.wasm main.go"
  • I installed Redis into my cluster via helm install redis bitnami/redis.
  • I'm running on a local k3d cluster.

My hunch is that I'm configuring the Redis trigger address incorrectly, but I'm not unsure how to debug further because I'm not getting any useful logs. Is there a way to turn on more verbose tracing/logging?

@Mossaka
Copy link
Member

Mossaka commented Jan 19, 2024

Could you please provide containerd logs?

@calebschoepp
Copy link
Author

Well this is embarrassing. I went back to grab containerd logs for you and found the solution. Turns out it there was a log line I had missed:

time="2024-01-19T19:32:57.498518805Z" level=info msg="error running start function: Redis trigger failed to connect to redis://localhost:6379"

This made me realize I hadn't been rerunning spin build && spin registry push when I was messing around with the Redis address. After re-building and pushing the app I ran into:

time="2024-01-19T19:33:25.090823054Z" level=info msg="error running start function: NOAUTH: Authentication required."

I resolved this by adding the password of the Redis server to the URL and then it worked as expected.

I guess that I just didn't notice the relevant log line the first time around. Related to #198 though, I wonder if there is any way that the containerd logs can be made more accessible. Execing into the node isn't very convenient or obvious.

@Mossaka it'd be great to hear if you have any thoughts on improving the findability of the containerd logs but otherwise I'm closing this issue.

@Mossaka
Copy link
Member

Mossaka commented Jan 22, 2024

Oh thanks!

Related to #198 though, I wonder if there is any way that the containerd logs can be made more accessible. Execing into the node isn't very convenient or obvious.

yes, stuff like this should go into the pod log.

@calebschoepp
Copy link
Author

yes, stuff like this should go into the pod log.

Is there a clear path to getting these containerd logs into the pod logs?

Do you want me to write something up in #198 tracking this?

@Mossaka
Copy link
Member

Mossaka commented Jan 22, 2024

Do you want me to write something up in #198 tracking this?

That's okay. I can just re-open this issue to track.

Is there a clear path to getting these containerd logs into the pod logs?

The trigger hook provided by spin runtime may help? I am not entirely clear though.

@Mossaka Mossaka reopened this Jan 23, 2024
@Mossaka Mossaka self-assigned this Jan 23, 2024
@Mossaka Mossaka added the enhancement New feature or request label Jan 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants