Documents tree build fails every time

[fe-dox]

Fresh install of ubuntu, internet connectivity present.

1. ./rmapi
2. Enter one-time code (go to https://my.remarkable.com/device/desktop/connect): <code>
3. ERROR: 2023/03/07 16:33:28 main.go:77: failed to build documents tree, last error: cant parse line '', wrong number of fields 0

It happens every time I try, on different devices too

[amildahl]

@fe-dox Running rmapi with RMAPI_TRACE=1 ./rmapi may provide some additional info to help the maintainers debug your issue.

I'm not familiar with the reMarkable interface (I just stumbled upon this project today), but I'm assuming the blank line is a bug on reMarkable's side, so I would expect something like the following to be a fix (albeit hacky). I'll let one of the maintainers comment on that assumption though.

diff --git a/api/sync15/tree.go b/api/sync15/tree.go
index 3889c43..f443489 100644
--- a/api/sync15/tree.go
+++ b/api/sync15/tree.go
@@ -39,6 +39,9 @@ func parseEntry(line string) (*Entry, error) {
        entry := Entry{}
        rdr := NewFieldReader(line)
        numFields := len(rdr.fields)
+       if numFields == 0 {
+               return nil, nil
+       }
        if numFields != 5 {
                return nil, fmt.Errorf("wrong number of fields %d", numFields)

@@ -90,6 +93,9 @@ func parseIndex(f io.Reader) ([]*Entry, error) {
                if err != nil {
                        return nil, fmt.Errorf("cant parse line '%s', %w", line, err)
                }
+               if entry == nil {
+                       continue
+               }

                entries = append(entries, entry)
        }

[ddvk]

seems that the index file is empty or the format changed. what version of the sw are you running

[Azeirah]

To me it does look like there could be a bug in the file sent by ReMarkable.

Here's the end of a trace with RMAPI_TRACE=1

Filenames and URLs edited for privacy. Metadata left untouched

... MANY MANY such calls succesfully executing before
{"relative_path":"{PATH1}","url":"{URL1}","expires":"2023-04-09T06:42:07.953701761Z","method":"GET"} <nil>
Trace: 2023/04/09 06:41:07 blobstorage.go:71: get url: {URL1}
Trace: 2023/04/09 06:41:07 transport.go:231: got generation header:  1677134650525175
Trace: 2023/04/09 06:41:07 blobdoc.go:131: name from metadata:  {actual notebook 1}
Trace: 2023/04/09 06:41:08 transport.go:231: got generation header:  1670624874403315
Trace: 2023/04/09 06:41:08 blobdoc.go:131: name from metadata:  {actual notebook 2}
Trace: 2023/04/09 06:41:08 transport.go:231: got generation header:  1671441008477970
Trace: 2023/04/09 06:41:08 blobdoc.go:131: name from metadata:  {actual notebook 3}
Trace: 2023/04/09 06:41:08 main.go:70: cant parse line '', wrong number of fields 0
ERROR: 2023/04/09 06:41:08 main.go:77: failed to build documents tree, last error: cant parse line '', wrong number of fields 0

Looks like it can get notebooks, but at some point it fails to parse one. Either RM is sending nonsensical data sometimes, or there's an unhandled case here.

[ddvk]

without the acutal index file, can't really say, either there is an empty line at the end, which would be ok and can be handled, or an empty line somewhere between which would be strange.

[Azeirah]

What exact file do you mean by the index file? I got this JSON back from the google storage API after POSTing this

{"Method":"GET", "relative_path":"a valid hash removed for privacy reasons"}

https://internal.cloud.remarkable.com/sync/v2/signed-urls/downloads

Edited away details again

{
    "deleted": false,
    "lastModified": -not relevant-,
    "lastOpened": -not relevant-,
    "lastOpenedPage": -not relevant-,
    "metadatamodified": true,
    "modified": true,
    "parent": -not relevant-,
    "pinned": false,
    "synced": false,
    "type": "DocumentType",
    "version": 0,
    "visibleName": -not relevant-
}

Do note, the last line is empty, if that's what you mean.

[ddvk]

the index file is something like

3
hash:uuid:foo:bar

one of the other requests

[klreeher]

I'm also seeing this, running on Ubuntu 22.04.2, having downloaded v0.0.25.
When I run ./rmapi, I get asked for the one time code from remarkable, and enter that. The console sits and thinks for a second, but after the remarkable webpage eventually redirects from the code to the main logged in landing, then I start seeing the trace happening.

Some of the last errors I see with the RMAPI_TRACE=1 are :

Trace: 2023/04/30 10:16:24 transport.go:231: got generation header:  1659648347695747
Trace: 2023/04/30 10:16:24 blobdoc.go:131: name from metadata:  duty, given chance
Trace: 2023/04/30 10:16:25 transport.go:231: got generation header:  1666357492489097
Trace: 2023/04/30 10:16:25 blobdoc.go:131: name from metadata:  All Things Untied
Trace: 2023/04/30 10:16:25 main.go:70: cant parse line '', wrong number of fields 0
ERROR: 2023/04/30 10:16:25 main.go:77: failed to build documents tree, last error: cant parse line '', wrong number of fields 0

If there's any other info that would be helpful to debug this issue, please let me know -- happy to recreate.