Configuring the debugger with multiple source roots

[aserra54]

Hello,

I have a TypeScript project with the below directory structure (stripped down for brevity):

<root>
|__samples
|  |__sample1
|  |  |__src
|  |  |  |__... (TypeScript source files and directories)
|  |  |__tsconfig.json
|  |__sample2
|  |  |__... (similar layout to 'sample1')
|  |__sample3
|     |__... (similar layout to 'sample1')
|__src
|  |__... (TypeScript source files and directories)
|__tsconfig.json

The rationale behind the layout is that everything under the base src directory corresponds to the API of the project, while everything under samples correspond to very specific examples indicating how to use the API. Each sample has a src directory containing TypeScript source files, and a corresponding tsconfig.json. Those tsconfig.json files look as follows (again, stripped down for brevity):

{
  "compilerOptions": {
    "sourceMap": true,
    "outDir": "../../build/out-tsc/sample1",
    "baseUrl": "src",
    "paths": {
      "*": [
        "./*",
        "../../../src/*"
      ]
    }
  }
}

Specifically, we use the baseUrl and paths compiler options to reference the source code both in the sample (./*) and the API (../../../src/*) directories. Using this, the output directory (../../build/out-tsc/sample1) winds up looking something:

<root>/build/out-tsc/sample1
|__samples
|  |__sample1
|     |__src
|        |__... (JavaScript sources and source maps)
|__src
   |__... (JavaScript sources and source maps)

In light of this, what's the proper way to configure this debugger to work with this setup? I've tried a number of things to no avail, and I guess it boils down to my misunderstanding of some of the properties. Specifically, I have this in my launch.json:

{
  "name": "Duk Attach (Sample1)",
  "type": "duk",
  "request": "attach",
  "stopOnEntry": false,
  "address": "localhost",
  "port": 9091,
  "localRoot": "${workspaceRoot}",
  "sourceMaps": true,
  "outDir": "${workspaceRoot}/build/out-tsc/sample1",
  "debugLog": false
}

But I don't see how this can make sense. outDir points to the root of the sample1 output, but the source files in that directory are spread out in two different sub-directories (at odd locations no less).

I've hacked the debugger to get this setup to work by using multiple search paths for searching for files, but it isn't production quality (or really sustainable). I'm also wondering if I even need to hack the debugger, or if this is something that could be supported with the current debugger implementation.

So... any ideas? I'm also open (to a certain degree) to changing my directory structures if necessary. Any help would be appreciated.

(Please consider that I'm not really a Node developer, so if any of the above looks odd or non-standard, it wasn't intentional.)

[aserra54]

To clarify as to what the issue is: the debugger actually attaches, but breakpoints never trigger. Below is the output of setting debugLog to true:

Attached to duktape debugger.
Protocol ID: 2 20200 v2.2.0 unknown
Status Notification: PAUSE
Finalized Initialization.
removeAllTargetBreakpoints
OUT -> <REQ: 1> <0x17: LISTBREAK>
[FE] threadsRequest
[FE] stackTraceRequest
IN <- <REP: 2> <EOM: 0> 
OUT -> <REQ: 1> <0x1c: GETCALLSTACK>
IN <- <REP: 2> <str: app/Zux.js> <str: > <int: 57> <int: 0> <EOM: 0> 
OUT -> <REQ: 1> <0x13: RESUME>
IN <- <REP: 2> <EOM: 0> 
OUT -> <REQ: 1> <0x1e: EVAL>
[FE] setBreakPointsRequest
Unknown source file: C:\proj\samples\sample1\src\Qux.ts
[FE] setBreakPointsRequest
[FE] setBreakPointsRequest
[FE] setBreakPointsRequest
[FE] configurationDoneRequest
Unknown source file: C:\proj\src\app\Foo.ts
Unknown source file: C:\proj\src\app\Bar.ts
Unknown source file: C:\proj\src\app\Baz.ts
[FE] threadsRequest
IN <- <ERR: 3> <int: 3> <str: invalid callstack index> <EOM: 0> 
Request undefined returned error: Not found : invalid callstack index
OUT -> <REQ: 1> <0x1e: EVAL>
IN <- <ERR: 3> <int: 3> <str: invalid callstack index> <EOM: 0> 
Request undefined returned error: Not found : invalid callstack index
Disconnected: Connection error: Error: read ECONNRESET
[FE] disconnectRequest

The strange is that the files referenced in the Unknown source file messages actually exist; I can copy-paste that path and open the file in Notepad and it exists.

[andrewrch]

Hi.

I'm a fairly new maintainer to this repo, but its my intention in the coming weeks to have a look in to the code for source maps etc, since I have a similarly complex project which currently I'm struggling to get working just though debugger config (I use the debugger with source maps disabled).

Would you be able to share a zip with an example so I can reproduce and I might be able to help.

Also, I merged today a PR for multiple source roots (Using the debugger config localRoots). I wonder if you can pull master and try that to see if it fixes your issue.

[andrewrch]

Also, I'd be interested to see your debugger 'hacks' - it might help my own debugging with these issues and give some ideas for changes, so if you don't mind please push to a fork, it doesn't matter how ugly the changes are :)

[aserra54]

I'm going to try and take some time to clean up the changes before getting them to you... I just realized they were made against a really old version of this repository (0.4.1).

[andrewrch]

I pushed out an extension update to the marketplace the other day so maybe give that a try as well and see if it helps for your use case.

[benjymous]

I'm seeing the same issue as the OP

This is my launch json

{
    "version": "0.2.0",
    "configurations": [
        {
            "name"        : "Attach to Prism",
            "type"        : "duk",
            "request"     : "attach",
            "stopOnEntry" : false,

            "address"     : "localhost",
            "port"        : 9091,

            "localRoots"   : ["C:\\Users\\Rich\\AppData\\Roaming\\Prism-384\\","c:\\Users\\Rich\\prism\\"],

            "debugLog"    : true
        }
    ]
}

and this is the sort of log I see when I try and enable a breakpoint

Attached to duktape debugger.
Protocol ID: 2 20300 v2.3.0 Prism-384
Finalized Initialization.
removeAllTargetBreakpoints
OUT -> <REQ: 1> <0x17: LISTBREAK>
Status Notification: PAUSE
[FE] threadsRequest
[FE] stackTraceRequest
[FE] stackTraceRequest
IN <- <REP: 2> <EOM: 0> 
OUT -> <REQ: 1> <0x1c: GETCALLSTACK>
IN <- <REP: 2> <str: packages\Tools\paint.js> <str: > <int: 448> <int: 0> <EOM: 0> 
OUT -> <REQ: 1> <0x1c: GETCALLSTACK>
IN <- <REP: 2> <str: packages\Tools\paint.js> <str: > <int: 448> <int: 0> <EOM: 0> 
OUT -> <REQ: 1> <0x13: RESUME>
IN <- <REP: 2> <EOM: 0> 
OUT -> <REQ: 1> <0x1e: EVAL>
[FE] setBreakPointsRequest
Unknown source file: c:\Users\Rich\prism\packages\Tools\paint.js
[FE] configurationDoneRequest
[FE] threadsRequest
IN <- <REP: 2> <int: 0> <str: [object global]> <EOM: 0> 
OUT -> <REQ: 1> <0x1e: EVAL>
IN <- <REP: 2> <int: 0> <str: [object global]> <EOM: 0> 

My debug code is pretty vanilla, so it should't be any local changes - I've put as many breakpoints inside the c++ code as I can (inside duktape itself and the transport code), and it never hits - it looks like it's failing here:

https://github.com/harold-b/vscode-duktape-debug/blob/master/src/DukDebugger.ts#L817

It looks like it's trying to match up the true filename c:\Users\Rich\prism\packages\Tools\paint.js against the roots (which should match into [c:\Users\Rich\prism\][packages\Tools\paint.js] and send just packages\Tools\paint.js over to the debug API.

I think the option for multiple roots is fairly recent (it wasn't in the 0.4 release) so I'm guessing the new code added for that is incomplete, so isn't properly looping over all the roots trying to resolve the relative path

[aserra54]

I had a coworker take a look at our issue, and she noticed a few things:

• The Unknown source file: ... error message seems to contain the wrong path in the error message, or at the very least is misleading. The path it points to exists, but the path it was comparing it to did not, leading to the confusion.
• I needed to move my sample code to the same level as my API code.
• For some odd reason, I need to have the mappings.wasm module in the root of my machine (under C:\) in order to get the source mapping to work. I tried moving it to other location (e.g. the root of my repository), but to no avail.
• I needed to make one additional to the code in order to get the source map to work correctly. At the end of the unmapSourceFile method of DukDebugger.ts, I had to change:

                for (const candidateFile of candidate.srcMap._sources) {
                    if (candidateFile && Path.resolve(this._outDir, candidateFile) === path)
                    {
                        return candidate;
                    }
                }

...to (only the 'if' condition changed)...

                for (const candidateFile of candidate.srcMap._sources) {
                    if (candidateFile && Path.resolve(Path.dirname(this.normPath(Path.join(this._outDir, candidate.name))), candidateFile) === path)
                    {
                        return candidate;
                    }
                }

These were the minimal set of changes required in order for the debugger to work with my build.

I don't really know how to proceed, as I'm not really a real TypeScript nor source mapping expert, but I figured I'd share this information with you.

Does anyone know how that mappings.wasm file works, or why it's even needed?

[andrewrch]

I've pushed out an update to the market place with @chsimp's fix merged. Please let me know if this fixes the issue and I'll close this, but if not I plan to push a new update to the marketplace with much more logging of file paths and filenames within the week which will hopefully help to diagnose these issues.

[aserra54]

I'll have him test it out as soon as possible (I work with him), but I don't believe that solves the .wasm issue I mentioned earlier. Though increased logging for paths is always useful, as the path resolution is almost always what seems to break when VS Code updates.

[andrewrch]

Thanks @aserra54. I've just pushed out an update (0.5.4) which is an attempt to make the current path resolution logic a bit more robust. Please let me know if this fixes anything for you.

[andrewrch]

Also, I pushed out v0.5.5 which I hope also fixes the issue with mappings.wasm

[patrick-janeiro]

Hi @andrewrch , I'm working on the same team as @aserra54 and @chsimp . I've attempted to use the new version of the debugger with our current repository structure. In fact since this ticket was opened we re-organized our repository structure to have all the folders under the same root as follows:

-src
---- core
---- sample

I seem to still be struggling to get the debugger to hit breakpoints:

{
  "name": "Sample",
  "type": "duk",
  "request": "attach",
  "stopOnEntry": false,
  "address": "localhost",
  "port": 9091,
  "localRoots": [
    "${workspaceRoot}/src"
  ],
  "sourceMaps": true,
  "outDir": "${workspaceRoot}/build/out-tsc/sample",
  "debugLog": false
},

I've tested this with vs-code 1.39.2 and 1.40.2.

Do you have any suggestions?

[andrewrch]

How do you pass the paths to duktape? I'm assuming it is embedded in a framework, so check how the paths are being passed to duktape when you compile the source (check the filename and directory prefix).

I'd recommend turning on debug logs for the debugger and seeing what paths are printed with the latest version. Duktape will only hit the breakpoints if the paths passed from vscode match the paths you've passed to duktape.

When using source maps, it'll try to read the source map and get the file path... I would probably try just using ${workspaceRoot} as your localRoot, assuming your tsconfig/webpack or whatever are in ${workspaceRoot}

If your duktape is running on a remote with a different path prefix you can use the 'dukRemoteRoot' option, or use this option if you're running from a different location and there is a directory prefix you need to prepend.