Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Heap snapshot does not seem to work anymore on 1.11 (in chrome profile loader) #54414

Closed
KristofferC opened this issue May 8, 2024 · 16 comments · Fixed by #55890
Closed

Heap snapshot does not seem to work anymore on 1.11 (in chrome profile loader) #54414

KristofferC opened this issue May 8, 2024 · 16 comments · Fixed by #55890
Labels
profiler regression Regression in behavior compared to a previous version

Comments

@KristofferC
Copy link
Member

julia> using Profile

julia> Profile.take_heap_snapshot("1.11.heapsnapshot")
"1.11.heapsnapshot"

Trying to load this in Chrome leads to

An error occurred when a call to method 'buildSnapshot' was requested
TypeError: Cannot read properties of undefined (reading '1')
    at F.name (devtools://devtools/bundled/devtools-frontend/front_end/entrypoints/heap_snapshot_worker/heap_snapshot_worker.js:1:8216)
    at F.rawName (devtools://devtools/bundled/devtools-frontend/front_end/entrypoints/heap_snapshot_worker/heap_snapshot_worker.js:1:38825)
    at F.name (devtools://devtools/bundled/devtools-frontend/front_end/entrypoints/heap_snapshot_worker/heap_snapshot_worker.js:1:39038)
    at F.isDocumentDOMTreesRoot (devtools://devtools/bundled/devtools-frontend/front_end/entrypoints/heap_snapshot_worker/heap_snapshot_worker.js:1:40431)
    at b.isUserRoot (devtools://devtools/bundled/devtools-frontend/front_end/entrypoints/heap_snapshot_worker/heap_snapshot_worker.js:1:35720)
    at b.calculateDistances (devtools://devtools/bundled/devtools-frontend/front_end/entrypoints/heap_snapshot_worker/heap_snapshot_worker.js:1:18912)
    at b.calculateDistances (devtools://devtools/bundled/devtools-frontend/front_end/entrypoints/heap_snapshot_worker/heap_snapshot_worker.js:1:35160)
    at b.initialize (devtools://devtools/bundled/devtools-frontend/front_end/entrypoints/heap_snapshot_worker/heap_snapshot_worker.js:1:14141)
    at new b (devtools://devtools/bundled/devtools-frontend/front_end/entrypoints/heap_snapshot_worker/heap_snapshot_worker.js:1:33684)
    at A.buildSnapshot (devtools://devtools/bundled/devtools-frontend/front_end/entrypoints/heap_snapshot_worker/heap_snapshot_worker.js:1:43578)

On 1.10 the same procedure works fine.

@KristofferC KristofferC added regression Regression in behavior compared to a previous version profiler labels May 8, 2024
@IanButterworth
Copy link
Member

Does the snapshot load in vscode's viewer?

@KristofferC
Copy link
Member Author

In VSCode I can load the 1.11 snapshot but not the 1.10 one...

@KristofferC KristofferC changed the title Heap snapshot does not seem to work anymore on 1.11 Heap snapshot does not seem to work anymore on 1.11 (in chrome profile loader) May 8, 2024
@IanButterworth
Copy link
Member

Yeah these aren't backported to 1.10 yet
#53833
#53984

@nsajko
Copy link
Contributor

nsajko commented Sep 11, 2024

AFAIK the VS Code heap snapshot viewer doesn't allow diffing two heap snapshots, unlike the Chromium Dev Tools? So I guess there's currently no way to diff heap snapshots 🫤.

nsajko added a commit to nsajko/julia that referenced this issue Sep 11, 2024
Say that:
* VS Code may be used for viewing heap snapshots. Important because
  Chromium currently may *not* be used for this on Julia v1.11 and up:
  JuliaLang#54414
* Firefox can't be used.
@IanButterworth
Copy link
Member

Any help figuring out why would be appreciated. They seem to expect different metadata. See the PRs that have changed it

@IanButterworth
Copy link
Member

Trying to get to the bottom of this.

Supposedly this is a valid V8 structure

{
  "snapshot": {
    "title": "Heap Snapshot",
    "uid": 1,
    "meta": {
      "node_fields": ["type", "name", "id", "self_size", "edge_count"],
      "node_types": [
        ["object", "closure", "regexp", "string", "number", "native", "synthetic", "concatenated string", "sliced string"],
        "string",
        "number",
        "number",
        "number"
      ],
      "edge_fields": ["type", "name_or_index", "to_node"],
      "edge_types": [
        ["context", "element", "property", "internal", "hidden", "shortcut", "weak"],
        "string_or_number",
        "node"
      ]
    },
    "node_count": 1000,
    "edge_count": 5000
  },
  "strings": [
    "",
    "(root)",
    "Object",
    "Array",
    "foo",
    "bar",
    /* ... other strings ... */
  ],
  "nodes": [
    0, 1, 2, 3, 4, /* ... node data ... */
  ],
  "edges": [
    0, 1, 2, /* ... edge data ... */
  ],
  /* ... other sections ... */
}

A julia example I just collected

{
  "snapshot": {
    "meta": {
      "node_fields": ["type", "name", "id", "self_size", "edge_count", "trace_node_id", "detachedness"],
      "node_types": [
        ["synthetic", "jl_task_t", "jl_module_t", "jl_array_t", "object", "jl_datatype_t", "String", "jl_svec_t", "jl_sym_t"],
        "string",
        "number",
        "number",
        "number",
        "number",
        "number"
      ],
      "edge_fields": ["type", "name_or_index", "to_node"],
      "edge_types": [
        ["internal", "element", "hidden", "property", "binding"],
        "string_or_number",
        "from_node"
      ],
      "trace_function_info_fields": ["function_id", "name", "script_name", "script_id", "line", "column"],
      "trace_node_fields": ["id", "function_info_index", "count", "size", "children"],
      "sample_fields": ["timestamp_us", "last_assigned_id"],
      "location_fields": ["object_index", "script_id", "line", "column"]
    },
    "node_count": 1759206,
    "edge_count": 5447107,
    "trace_function_count": 0
  },
  "trace_function_infos": [],
  "trace_tree": [],
  "samples": [],
  "locations": [],
  "nodes": [
    0, 0, 0, 0, 4, 0, 0,
    0, 1, 1, 0, 6, 0, 0,
...
  "edges":[
    0,1,7,
    0,2,14,
...
  "strings":[
    "",
    "GC roots",
    ...
}

#53833 which fixed opening in VSCode's viewer, and likely broke chrome devtools made these changes

Screenshot 2024-09-24 at 9 29 21 PM

References:
https://learn.microsoft.com/en-us/microsoft-edge/devtools-guide-chromium/memory-problems/heap-snapshot-schema#schema-of-the-heapsnapshot-data
https://v8docs.nodesource.com/node-0.8/d8/deb/classv8_1_1_heap_snapshot.html#a159532596beaf59473bf3189703c39a1

@IanButterworth
Copy link
Member

The problem appears to be the inclusion of

  "sample_fields":["timestamp_us","last_assigned_id"],
  "location_fields":["object_index","script_id","line","column"]

without it vscode cannot load the heapsnapshot, but chrome devtools can.
with it chrome devtools cannot, but vscode can.

@IanButterworth
Copy link
Member

Dev tools parsing for those fields (which fails if they are there, but the line numbers don't correspond to here)
https://github.com/ChromeDevTools/devtools-frontend/blob/75970e0fc2a9be009fa4d946cedc0e0018c16a88/front_end/entrypoints/heap_snapshot_worker/HeapSnapshotLoader.ts#L241-L251

vscode
https://github.com/microsoft/vscode-v8-heap-tools/blob/c5b34396392397925ecbb4ecb904a27a2754f2c1/v8-heap-parser/src/decoder.rs#L48-L50

It'd be helpful to find the exact files & lines these are coming from in devtools, but I guess these js files were minified

TypeError: Cannot read properties of undefined (reading 'length')
    at new O (devtools://devtools/bundled/devtools-frontend/front_end/entrypoints/heap_snapshot_worker/heap_snapshot_worker.js:1:13338)
    at new A (devtools://devtools/bundled/devtools-frontend/front_end/entrypoints/heap_snapshot_worker/heap_snapshot_worker.js:1:39902)
    at R.buildSnapshot (devtools://devtools/bundled/devtools-frontend/front_end/entrypoints/heap_snapshot_worker/heap_snapshot_worker.js:1:50296)
    at HeapSnapshotWorkerDispatcher.dispatchMessage (devtools://devtools/bundled/devtools-frontend/front_end/entrypoints/heap_snapshot_worker/heap_snapshot_worker.js:1:54089)

@IanButterworth
Copy link
Member

IanButterworth commented Sep 25, 2024

The error above is to this (in an unminified version of heap_snapshot_worker,js) https://gist.github.com/IanButterworth/9fb742180ee6e682e04d34819fd9d500#file-heap_snapshot_worker-js-L693

So somehow the inclusion of

  "sample_fields":["timestamp_us","last_assigned_id"],
  "location_fields":["object_index","script_id","line","column"]

is preventing strings from being populated.


It's this raw line https://github.com/ChromeDevTools/devtools-frontend/blob/2709fa744749f851721829cc218698729f74bd24/front_end/entrypoints/heap_snapshot_worker/HeapSnapshot.ts#L828

@gbaraldi
Copy link
Member

@IanButterworth
Copy link
Member

That's linked above. I think this is a bug in what each expects to be there.

@nsajko

This comment was marked as outdated.

@IanButterworth
Copy link
Member

Yep. That's at the bottom of that comment

@IanButterworth
Copy link
Member

I see that a vscode-generated heapsnapshot opens in chrome dev tools, and visa-versa. It's just julia ones that don't open in dev tools.

Some summaries

Julia

julia> json_string = read("/Users/ian/Documents/GitHub/julia/96449_180407234248750.heapsnapshot", String);

julia> js = JSON3.read(json_string)
JSON3.Object{Base.CodeUnits{UInt8, String}, Vector{UInt64}} with 8 entries:
  :snapshot             => {…
  :trace_function_infos => Union{}[]
  :trace_tree           => Union{}[]
  :samples              => Union{}[]
  :locations            => Union{}[]
  :nodes                => [0, 0, 0, 0, 4, 0, 0, 0, 1, 1  …  0, 0, 0, 4, 1747986, 4660449792, 16, 1, 0, 0]
  :edges                => Union{Float64, Int64}[0, 1, 7, 0, 2, 14, 0, 4, 21, 0  …  43428, 3, 29907, 43428, 3, 539187, 43428, 3, 1528671, 10617334]
  :strings              => ["", "GC roots", "GC finalizer list roots", "Task", "root task", "current task", "Main", "main_module", "Array{Any, 1}", "…

julia> js.snapshot
JSON3.Object{Base.CodeUnits{UInt8, String}, SubArray{UInt64, 1, Vector{UInt64}, Tuple{UnitRange{Int64}}, true}} with 4 entries:
  :meta                 => {…
  :node_count           => 1747684
  :edge_count           => 5408593
  :trace_function_count => 0

julia> js.snapshot.meta
JSON3.Object{Base.CodeUnits{UInt8, String}, SubArray{UInt64, 1, Vector{UInt64}, Tuple{UnitRange{Int64}}, true}} with 8 entries:
  :node_fields                => ["type", "name", "id", "self_size", "edge_count", "trace_node_id", "detachedness"]
  :node_types                 => Any[["synthetic", "jl_task_t", "jl_module_t", "jl_array_t", "object", "jl_datatype_t", "String", "jl_svec_t", "jl_sy…
  :edge_fields                => ["type", "name_or_index", "to_node"]
  :edge_types                 => Any[["internal", "element", "hidden", "property", "binding"], "string_or_number", "from_node"]
  :trace_function_info_fields => ["function_id", "name", "script_name", "script_id", "line", "column"]
  :trace_node_fields          => ["id", "function_info_index", "count", "size", "children"]
  :sample_fields              => ["timestamp_us", "last_assigned_id"]
  :location_fields            => ["object_index", "script_id", "line", "column"]

julia> for (k,v) in js
           @show k,length(v)
       end
(k, length(v)) = (:snapshot, 4)
(k, length(v)) = (:trace_function_infos, 0)
(k, length(v)) = (:trace_tree, 0)
(k, length(v)) = (:samples, 0)
(k, length(v)) = (:locations, 0)
(k, length(v)) = (:nodes, 12233788)
(k, length(v)) = (:edges, 16225779)
(k, length(v)) = (:strings, 1748010)

chrome

julia> json_string = read("/Users/ian/Documents/GitHub/julia/chrome_Heap-20240926T080701.heapsnapshot", String);

julia> js = JSON3.read(json_string)
JSON3.Object{Base.CodeUnits{UInt8, String}, Vector{UInt64}} with 8 entries:
  :snapshot             => {…
  :nodes                => [9, 1, 1, 0, 14, 0, 0, 9, 2, 3  …  87, 0, 0, 9, 73578, 252940, 0, 0, 0, 0]
  :edges                => [1, 1, 7, 5, 3695, 23807, 5, 3696, 24038, 5  …  5824616, 1, 85, 5667606, 1, 86, 5296452, 1, 87, 5583277]
  :trace_function_infos => Union{}[]
  :trace_tree           => Union{}[]
  :samples              => Union{}[]
  :locations            => [522795, 513, 55, 461709, 522760, 513, 55, 461709, 525546, 513  …  55, 894511, 4174961, 513, 55, 1402055, 4174968, 513, 55, 1402…
  :strings              => ["<dummy>", "", "(GC roots)", "(Bootstrapper)", "(Builtins)", "(Client heap)", "(Code flusher)", "(Compilation cache)", "(Debugg…

julia> js.snapshot
JSON3.Object{Base.CodeUnits{UInt8, String}, SubArray{UInt64, 1, Vector{UInt64}, Tuple{UnitRange{Int64}}, true}} with 4 entries:
  :meta                 => {…
  :node_count           => 841614
  :edge_count           => 3659995
  :trace_function_count => 0

julia> js.snapshot.meta
JSON3.Object{Base.CodeUnits{UInt8, String}, SubArray{UInt64, 1, Vector{UInt64}, Tuple{UnitRange{Int64}}, true}} with 8 entries:
  :node_fields                => ["type", "name", "id", "self_size", "edge_count", "trace_node_id", "detachedness"]
  :node_types                 => Any[["hidden", "array", "string", "object", "code", "closure", "regexp", "number", "native", "synthetic", "concatenated st…
  :edge_fields                => ["type", "name_or_index", "to_node"]
  :edge_types                 => Any[["context", "element", "property", "internal", "hidden", "shortcut", "weak"], "string_or_number", "node"]
  :trace_function_info_fields => ["function_id", "name", "script_name", "script_id", "line", "column"]
  :trace_node_fields          => ["id", "function_info_index", "count", "size", "children"]
  :sample_fields              => ["timestamp_us", "last_assigned_id"]
  :location_fields            => ["object_index", "script_id", "line", "column"]

julia> for (k,v) in js
           @show k,length(v)
       end
(k, length(v)) = (:snapshot, 4)
(k, length(v)) = (:nodes, 5891298)
(k, length(v)) = (:edges, 10979985)
(k, length(v)) = (:trace_function_infos, 0)
(k, length(v)) = (:trace_tree, 0)
(k, length(v)) = (:samples, 0)
(k, length(v)) = (:locations, 241380)
(k, length(v)) = (:strings, 136268)

vscode

julia> json_string = read("/Users/ian/Documents/GitHub/julia/vscode-profile-2024-09-25-21-49-23.heapsnapshot", String);

julia> js = JSON3.read(json_string)
JSON3.Object{Base.CodeUnits{UInt8, String}, Vector{UInt64}} with 8 entries:
  :snapshot             => {…
  :nodes                => [9, 1, 1, 0, 8, 0, 0, 9, 2, 3  …  0, 0, 0, 8, 15102, 316, 14, 0, 0, 0]
  :edges                => [1, 1, 7, 5, 3432, 22477, 5, 3433, 22540, 1  …  427007, 1, 109, 427014, 1, 110, 427021, 1, 111, 427028]
  :trace_function_infos => Union{}[]
  :trace_tree           => Union{}[]
  :samples              => Union{}[]
  :locations            => [134162, 7, 261, 28, 134204, 7, 273, 36, 134225, 7  …  330, 24, 418257, 7, 346, 14, 418264, 7, 370, 13]
  :strings              => ["<dummy>", "", "(GC roots)", "(Bootstrapper)", "(Builtins)", "(Client heap)", "(Code flusher)", "(Compilation cache)", "(Debugg…

julia> js.snapshot
JSON3.Object{Base.CodeUnits{UInt8, String}, SubArray{UInt64, 1, Vector{UInt64}, Tuple{UnitRange{Int64}}, true}} with 4 entries:
  :meta                 => {…
  :node_count           => 61005
  :edge_count           => 257826
  :trace_function_count => 0

julia> js.snapshot.meta
JSON3.Object{Base.CodeUnits{UInt8, String}, SubArray{UInt64, 1, Vector{UInt64}, Tuple{UnitRange{Int64}}, true}} with 8 entries:
  :node_fields                => ["type", "name", "id", "self_size", "edge_count", "trace_node_id", "detachedness"]
  :node_types                 => Any[["hidden", "array", "string", "object", "code", "closure", "regexp", "number", "native", "synthetic", "concatenated st…
  :edge_fields                => ["type", "name_or_index", "to_node"]
  :edge_types                 => Any[["context", "element", "property", "internal", "hidden", "shortcut", "weak"], "string_or_number", "node"]
  :trace_function_info_fields => ["function_id", "name", "script_name", "script_id", "line", "column"]
  :trace_node_fields          => ["id", "function_info_index", "count", "size", "children"]
  :sample_fields              => ["timestamp_us", "last_assigned_id"]
  :location_fields            => ["object_index", "script_id", "line", "column"]

julia> for (k,v) in js
           @show k,length(v)
       end
(k, length(v)) = (:snapshot, 4)
(k, length(v)) = (:nodes, 427035)
(k, length(v)) = (:edges, 773478)
(k, length(v)) = (:trace_function_infos, 0)
(k, length(v)) = (:trace_tree, 0)
(k, length(v)) = (:samples, 0)
(k, length(v)) = (:locations, 22168)
(k, length(v)) = (:strings, 29769)

@IanButterworth
Copy link
Member

IanButterworth commented Sep 26, 2024

We don't populate "locations".

According to https://learn.microsoft.com/en-us/microsoft-edge/devtools-guide-chromium/memory-problems/heap-snapshot-schema

"locations" : Contains information about the script location of nodes. To parse this data, use snapshot.meta.location_fields with the nodes array. | Array

But if you take the number of node entries in the vscode one as an example: 427035 divide by the length of node_fields (7) gives 61005 nodes.
Multiply that by the length of location_fields (4) gives 244020 which is much bigger than the actual 22168.

If we can figure out the right size to zero pad to, it'd be worth trying I think.

@IanButterworth
Copy link
Member

Found the issue! The order of the fields in the json object is critical for dev tools! I guess their JSON parser doesn't just load the whole json object in then process it, but does some processing on the fly..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
profiler regression Regression in behavior compared to a previous version
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants