Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Execution in progress" hangs for 10 minutes and then "Error encountered running Starlark code." #1769

Closed
thib-wien opened this issue Nov 13, 2023 · 10 comments
Labels
bug Something isn't working painful Painful bug

Comments

@thib-wien
Copy link

thib-wien commented Nov 13, 2023

What's your CLI version?

0.85.18

Description & steps to reproduce

My environment with Kurtosis was working last week. I restarted the computer and now I face the following problem.

When starting Kurtosis, the screen freezes with this for 10 minutes:
Screenshot from 2023-11-13 10-25-58

After having been frozen for 10 minutes, Kurtosis throws this error:
Screenshot from 2023-11-12 18-12-40

The problem might be between the screen and the chair.

Desired behavior

An error message sooner maybe.
But I am not sure what the origin of the problem is: me, the scripts or Kurtosis?

What is the severity of this bug?

Painful; this is causing significant friction in my workflow.

What area of the product does this pertain to?

Other: anything not covered by the above

@thib-wien thib-wien added the bug Something isn't working label Nov 13, 2023
@github-actions github-actions bot added the painful Painful bug label Nov 13, 2023
@mieubrisse
Copy link
Collaborator

Hey @thib-wien , thanks for filing this and apologies in the response delay (the team is at a conference, so we're going to be slower than usual to respond this week)! This does indeed look like a problem with Kurtosis; the specific error that's happening is that Kurtosis is checking to see if our centralized logs collector is available before starting the service, and it's failing.

Would you mind running a kurtosis enclave dump my-testnet and sending the output directory to us? That will contain logs on the enclave, so we can see what's going on. You can send it to [email protected] if it contains sensitive data that you're not comfortable posting on this issue.

@thib-wien
Copy link
Author

Thanks a lot for considering the problem.
I sent you data by email.

@tedim52
Copy link
Contributor

tedim52 commented Nov 14, 2023

Hey @thib-wien thank you for sending, taking a look at the data now! Also, just wanted to confirm, is this issue happening when you try and run any kurtosis package or were you seeing this issue only when running this specific package?

@thib-wien
Copy link
Author

Thanks!
I haven't tried with another package. I am given only this one at my workplace.

@thib-wien
Copy link
Author

PS: you write "were you seeing this issue" but I am still seeing it. I tried various things and nothing fixes the problem unfortunately.

@tedim52
Copy link
Contributor

tedim52 commented Nov 14, 2023

Gotcha! It seems like you're trying to run the full beacon node in the ethereum package. Any chance you can send the network params yaml so I can try and reproduce on 0.85.18?

@thib-wien
Copy link
Author

thib-wien commented Nov 14, 2023

Gladly!
Do you find in this folder the parameters that you need to see?

@tedim52
Copy link
Contributor

tedim52 commented Nov 14, 2023

yes! Thanks you.

@thib-wien
Copy link
Author

thib-wien commented Nov 15, 2023

Dear Kurtosis team,

We fixed our problem this morning by uninstalling Docker, deleting all data that it left on my computer and reinstalling it.

We suspect that sudo docker system prune --volumes -a -f would be sufficient (it might avoid the removal of dozens of folders by hand). If someone encounters my problem one day, try this command.

We hope that you will figure out the problem that I experienced with your log collector. We keep an eye on this page and are ready to help you at our level.

@tedim52
Copy link
Contributor

tedim52 commented Nov 22, 2023

Hey @thib-wien ! Thanks for getting back to us and letting us know how you solved the issue and apologies for the late reply (our team is returning from a long week at a conference) We believe we may have some resource leakage around our log collector containers and your problem/solution may back that hypothesis. We're tracking this issue here: #1833 so I'll close out this issue for now. If you have any more info regarding this one your end, don't hesitate to reach us there. Thanks again for your support!

@tedim52 tedim52 closed this as completed Nov 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working painful Painful bug
Projects
None yet
Development

No branches or pull requests

3 participants