-
Notifications
You must be signed in to change notification settings - Fork 29.8k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
doc: initial cut at support tiers for diag tools
PR-URL: #21870 Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Joyee Cheung <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Matheus Marchini <[email protected]>
- Loading branch information
Showing
1 changed file
with
146 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,146 @@ | ||
| # Diagnostic Tooling Support Tiers | ||
|
|
||
| Diagnostic tooling is important to the consumers of Node.js. It is used both | ||
| in development and in production in order to investigate problems. The failure | ||
| of one of these tools may be as big a problem for an end user as a bug within | ||
| the runtime itself. | ||
|
|
||
| The Node.js project has assessed the tools and the APIs which support those | ||
| tools. Each of the tools and APIs has been put into one of | ||
| the following tiers. | ||
|
|
||
| * Tier 1 - Must always be working(CI tests passing) for all | ||
| Current and LTS Node.js releases. A release will not be shipped if the test | ||
| suite for the tool/API is not green. To be considered for inclusion | ||
| in this tier it must have a good test suite and that test suite and a job | ||
| must exist in the Node.js CI so that it can be run as part of the release | ||
| process. Tests on master will be run nightly when possible to provide | ||
| early warning of potential issues. No commit to the current and LTS | ||
| release branches should break this tool/API if the next major release | ||
| is within 1 month. In addition: | ||
| * The maintainers of the tool must remain responsive when there | ||
| are problems; | ||
| * The tool must be actively used by the ecosystem; | ||
| * The tool must be heavily depended on; | ||
| * The tool must have a guide or other documentation in the Node.js GitHub | ||
| organization or website; | ||
| * The tool must be working on all supported platforms; | ||
| * The tool must only be using APIs exposed by Nodejs as opposed to | ||
| its dependencies; and | ||
| * The tool must be open source. | ||
|
|
||
| * Tier 2 - Must be working(CI tests passing) for all | ||
| LTS releases. An LTS release will not be shipped if the test | ||
| suite for the tool/API is not green. To be considered for inclusion | ||
| in this tier it must have a good test suite and that test suite and a job | ||
| must exist in the Node.js CI so that it can be run as part of the release | ||
| process. In addition: | ||
| * The maintainers of the tool must remain responsive when | ||
| there are problems; | ||
| * The tool must be actively used by the ecosystem; | ||
| * The tool must be heavily depended on; | ||
| * The tool must have a guide or other documentation in the Node.js GitHub | ||
| organization or website; | ||
| * The tool must be open source. | ||
|
|
||
| * Tier 3 - If possible its test suite | ||
| will be run at least nightly in the Node.js CI and issues opened for | ||
| failures. Does not block shipping a release. | ||
|
|
||
| * Tier 4 - Does not block shipping a release. | ||
|
|
||
| * Unclassified - tool/API is new or does not have the required testing in the | ||
| Node.js CI in order to qualify for a higher tier. | ||
|
|
||
| The choice of which tier a particular tool will be assigned to, will be a | ||
| collaborative decision between Diagnostics WG and Release WG. Some of the | ||
| criteria considered might be: | ||
|
|
||
| * If the tool fits into a key category as listed below. | ||
| * Whether the tool is actively used by the ecosystem. | ||
| * The availability of alternatives. | ||
| * Impact to the overall ecosystem if the tool is not working. | ||
| * The availability of reliable test suite that can be integrated into our CI. | ||
| * The availability of maintainer or community collaborator who will help | ||
| resolve issues when there are CI failures. | ||
| * If the tool is maintained by the Node.js Foundation GitHub organization. | ||
|
|
||
| The current categories of tools/APIs that fall under these Tiers are: | ||
|
|
||
| * FFDC (F) - First failure data capture, easy to consume initial diagnostic | ||
| information. | ||
| * Tracing (T) - use of logging to provide information about execution flow. | ||
| * Memory (M) - tools that provide additional information about memory | ||
| used in the Heap or by native code. | ||
| * Profiling (P) - tools that provide additional information about where | ||
| CPU cycles are being spent. | ||
| * AsyncFlow (A) - tools that provide additional insight into asynchronous | ||
| execution flow. | ||
|
|
||
| ## Adding a tool to this list | ||
|
|
||
| Any tool that might be used to investigate issues when running Node.js can | ||
| be added to the list. If there is a new tool that should be added to the | ||
| list, it should start by being added to the "Not yet classified" or | ||
| "Tier 4" lists. Once it has been added to the list "promotion" to Tier 3 | ||
| through Tier 1 requires that the requirements listed above be met AND | ||
| have agreement from Diagnostics WG and Release WG based on the criteria | ||
| listed above. | ||
|
|
||
| ## Tiers | ||
|
|
||
| The tools are currently assigned to Tiers as follows: | ||
|
|
||
| ## Tier 1 | ||
|
|
||
| | Tool Type | Tool/API Name | Regular Testing in Node.js CI | Integrated with Node.js | Target Tier | | ||
| |-----------|---------------------------|-------------------------------|-------------------------|-------------| | ||
| | | | | | | | ||
|
|
||
| ## Tier 2 | ||
|
|
||
| | Tool Type | Tool/API Name | Regular Testing in Node.js CI | Integrated with Node.js | Target Tier | | ||
| |-----------|---------------------------|-------------------------------|-------------------------|-------------| | ||
| | | | | | | | ||
|
|
||
|
|
||
| ## Tier 3 | ||
|
|
||
| | Tool Type | Tool/API Name | Regular Testing in Node.js CI | Integrated with Node.js | Target Tier | | ||
| |-----------|---------------------------|-------------------------------|-------------------------|-------------| | ||
| | | | | | | | ||
|
|
||
| ## Tier 4 | ||
|
|
||
| | Tool Type | Tool/API Name | Regular Testing in Node.js CI | Integrated with Node.js | Target Tier | | ||
| |-----------|---------------------------|-------------------------------|-------------------------|-------------| | ||
| | | | | | | | ||
|
|
||
| ## Not yet classified | ||
|
|
||
| | Tool Type | Tool/API Name | Regular Testing in Node.js CI | Integrated with Node.js | Target Tier | | ||
| |-----------|---------------------------|-------------------------------|-------------------------|-------------| | ||
| | FFDC | node-report | No | No | 1 | | ||
| | Memory | mdb_V8 | No | No | 4 | | ||
| | Memory | node-heapdump | No | No | 2 | | ||
| | Memory | V8 heap profiler | No | Yes | 1 | | ||
| | Memory | V8 sampling heap profiler | No | Yes | 1 | | ||
| | AsyncFlow | Async Hooks (API) | ? | Yes | 1 | | ||
| | Debugger | V8 Debug protocol (API) | No | Yes | 1 | | ||
| | Debugger | Command line Debug Client | ? | Yes | 1 | | ||
| | Debugger | llnode | ? | No | 2 | | ||
| | Debugger | Chrome Dev tools | ? | No | 3 | | ||
| | Debugger | Chakracore - time-travel | No | Data source only | too early | | ||
| | Tracing | trace_events (API) | No | Yes | 1 | | ||
| | Tracing | DTrace | No | Partial | 3 | | ||
| | Tracing | LTTng | No | Removed? | N/A | | ||
| | Tracing | ETW | No | Partial | 3 | | ||
| | Tracing | Systemtap | No | Partial | ? | | ||
| | Profiling | V8 CPU profiler (--prof) | No | Yes | 1 | | ||
| | Profiling | Linux perf | No | Partial | ? | | ||
| | Profiling | DTrace | No | Partial | 3 | | ||
| | Profiling | Windows Xperf | No | ? | ? | | ||
| | Profiling | 0x | No | No | 4 | | ||
| | Profiling | node-clinic | No | No | to early | | ||
| | F/P/T | appmetrics | No | No | ? | | ||
| | M/T | eBPF tracing tool | No | No | ? | |