This repository has been archived by the owner on Nov 6, 2020. It is now read-only.
Parity Light Client Sync Stuck #6319
Labels
Milestone
Comments
5chdn
added
F2-bug 🐞
|
Can reproduce this, at a different block number though. Sync simply stalls. |
|
Same here, different block number.(macOS 10.12.6) |
|
could be simply that request credits are running out. "pip=trace" logs would be helpful, although it would produce a lot. the last 30 seconds would be enough in any case. |
|
Having the exact same problem. Constantly stalls and will remain stalled until I kill and restart the client. |
|
FWIW I had the same problem but it went away when compiling and running the latest master. |
|
@Robzz I just tried with the latest commit on master running on Ubuntu and it still stalls during sync. |
|
+1 for |
|
+1 for Parity//v1.7.2-beta-9f47909-20170918/x86_64-linux-gnu/rustc1.19.0 laptop went to sleep. came back and its stuck at 1938838. restarting doesn't change anything. how should I enable better logging to help debug? |
|
Ran parity inside docker (which uses the 1.7.2 deb) with https://gist.github.com/WyseNynja/30fd6df9ca726604043a452e5bc525fc If I remove I have grafana setup for my archival node and so have a graph of upload bandwidth for the parity container: |
|
Started a light client from scratch with pip=trace logging and --reserved-only pointed at my archival node: https://gist.github.com/WyseNynja/4a956ca5e3a5a8b24be3aa0376eff030 EDIT 1 (12:15 PM PST): It stalled a couple times so far, but it resumes after a few minutes. EDIT 2 (4:15 PM PST): Sigh. Of course with logging on, it it was able to sync fully |
|
My node got stuck again. Pretty useless logs. After a few minutes of that it started again. What logging should I enable on the other side? My guess is @rphmeier is right about request credits simply running out. I would have expected a reserved peer to not be able to run out of credits though. And without --reserved-peers it still seems to stall sometimes. |
|
@wysenynja Currently there isn't any logic that gives reserved peers infinite credits, although that would definitely be a useful addition! In general, it can be pretty hard to find peers to serve light client data, because many do not. An upgraded peer discovery protocol could alleviate that for sure. |
|
I was able to light-sync musicoin with latest master and the sync was also stuck at some point. I was connected to my own archive node (nobody else is running a light-serving node on musicoin), and after shutting down the light client, restarting the archival node, and restarting the light client, it was able to finish the sync. Minor annoyance: after completing the sync, the logs do not show the latest block number, so I can only verify it's finished via RPC/WS or UI. Edit: Just finished a light sync on expanse without any issues. |
|
Very interesting, now being aware of the credits, I just managed to get a stuck foundation light-sync to continue after changing the Edit, might have been a coincidence, it's stuck again. |
|
I think we should add more verbose logging when credits run out along with a countdown for when we will retry. Then users like me will know to just be patient instead of thinking something is actually broken. Hopefully more peers that serve light clients will join the network. Is there an open issue tracking better peer discovery? On a related note (maybe this is what #6010 is about), given that I already have a fully synced node, I don't really want to waste time validating any headers on my laptop. Is there some way to skip to a known good header? For example, my archival node has #4314162 8649…1a5e so I might as well start as close to there as I can. If I could do that, the light node wouldn't need to use so many credits fetching old headers and would be a much lighter load on the network. |
|
Providing my own archive node via reserved peers helps to get the light client synced and gives me some control over "credits" (at least it feels like it does, not sure how they work). I'm now at block 3.7m with the light client on foundation. |
|
hello ! same problem here +1 |
|
same here v1.7.7, managed to get up to block 2.8mil ... |
This was referenced Oct 29, 2017
|
Same problem here - I'm using I'm running |
|
+1 here. Stuck at 2023711. Parity version is 1.8.3 |
|
I'm having the same issue on foundation: Is there a workaround for this problem ? |
|
@ilhanu the workaround for mainnet is to use v1.11 that will be released soon (here). Right now you can build it yourself from the source. The v1.11 uses hardcoded block headers that prevents you from having to download them. |
|
There are also binaries for 1.11 if you want to test light sync before it is actually released. |
|
v1.11 doesn't change much, I deleted the cache folder in 1.10 and started anew. Here you can see the output with 1.11 - binary I downloaded from sources of @5chdn |
|
@ilhanu you need to |
|
I'm having the same issue, but also whenever this happens my I/O is going absolutely through the roof. Can anyone confirm the same? Using iotop my I/O is at 100%, often 'more', constantly once I stop receiving headers. |
|
This should be solved by now. |
|
i can confirm i have not had any syncing problems with at least 6 recent light nodes/sync |
|
Still observing this. |
|
The code base has changed a bit lately, can you open a new issue or alternatively provide some more details such as which version, how many peers you got connected and so on? Would be good if you could run Thanks |
|
I am experiencing this issue with light client on classic chain 2019-04-15 16:20:23 IO Worker #2 DEBUG sync Maintaining sync (AncestorSearch(Awaiting(ReqId(89), 7856016, Complete { start: Number(7856016), skip: 0, max: 64, reverse: true }))) |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
and others
Before filing a new issue, please provide the following information.
Your issue description goes here below. Try to include actual vs. expected behavior and steps to reproduce the issue.
I tried running
parity --light --scale-verifiers --num-verifiers 4 --fast-and-loose(andparity --light) but both times I get stuck at:I would expect it to sync all the way to the current tip, which is currently 4168389.
The text was updated successfully, but these errors were encountered: