upload-bin fails to upload index files
#3658
Comments
AnnaShaleva
added
bug
|
Within the scope of this issue let's move |
|
Block 91326 (FS testnet) exisits.: |
|
89052 (mainnet) - not found. |
Can you GET/HEAD it?
What if try to request more times? |
i dont know OID, so I cant |
Yet nothing |
You have it, it's HwkGuEX5bra2et7pm2CD2fs4vi9oqfrSrKjkX4GBc6m5. The question is about existing object. |
with this one everything is okay, its correct and returned. GET/HEAD several times from different nodes and with/without --ttl 1 |
|
For |
Refs #3658 Signed-off-by: Ekaterina Pavlova <[email protected]>
Refs #3658 Signed-off-by: Ekaterina Pavlova <[email protected]>
Ref #3658 Signed-off-by: Ekaterina Pavlova <[email protected]>
Ref #3658 Signed-off-by: Ekaterina Pavlova <[email protected]>
Ref #3658 Signed-off-by: Ekaterina Pavlova <[email protected]>
After 3 times all blocks were found. Depends on nspcc-dev/neofs-node#2721. |
Ref #3658 Signed-off-by: Ekaterina Pavlova <[email protected]>
Ref #3658 Signed-off-by: Ekaterina Pavlova <[email protected]>
|
For the record: some blocks are really missing from NeoFS. We have 18ed3bb that should solve this issue, but we're not 100% sure if it helps, hence let's keep this issue open, test the updated uploading script on mainnet one more time and check if it solves the problem with gaps in blocks uploading. |
|
It seems to be still relevant, based on new uploader runs: |
|
How does it really happen? You're uploading an object for block 1618 and... get an error? get a successful reply? don't get any reply? Can we get more data from logs in this case? |
From the logs, it was successfully uploaded, we didn't have any related errors, that could explain it. |
|
And we don't know its OID? I think we need some verbose mode with more data in this case to trace the problem. Seems like it happens often enough and we don't need to wait for full upload cycle to complete. |
No OID. #3655 will be as a verbose mode - it will put OIDs directly into the index file. Or we can try to write everything in logs to see it faster -200k blocks will be enough to catch. |
|
Log line is much easier to add, let's add it and then try a new upload, see where it fails and then take a look at it from the NeoFS side. Either it loses an object that was accepted (bad), or it reports an incorrect status for failed upload (bad). Or something else. Maybe related to nspcc-dev/neofs-node#2975, maybe not. |
Logs for testing purpose.Refs. https://github .com//issues/3658#issuecomment-2468210667 Signed-off-by: Ekaterina Pavlova <[email protected]>
|
The problem is on NeoFS side, some objects are silently failed to be uploaded. Fixed by nspcc-dev/neofs-node#3014, hence I consider this issue as not planned, no actions required from NeoGo side. |
Blocks
91326(FS testnet),89052(mainnet) and27475(testnet) can't be found in the storage. Check what's wrong with them or is it just a small number of retries.The text was updated successfully, but these errors were encountered: