discuss: node.js contributor survey

[gireeshpunathil]

Discuss the relevant sections from the contributor survey result that falls in the purview of diagnostics WG, such as:

debugging test failures: 
 - a bit annoying: 40%
 - an occasional blocker: 15%

how important diagnostics is for you:
 - 1 (high priority): 35%
 - 2: 35%
 - 3: 30%

how do you think the project is doing in terms of diagnostics:
 - doing well: 40%
 - needs improvement: 35%
 - no opinion: 25%

and come up with actionable items that address these.

ref: https://github.com/nodejs/TSC/blob/b42c37e8d7e6815177ca198b5a59a23e19750998/surveys/2020%20Node.js%20Contributors%20Survey%20Results.pdf

[jasnell]

It's been a while since we've had any kind of comprehensive survey on the state of diagnostics in node.js so perhaps a reasonable next step would be to devise a survey that could help shed more light on what the current issues are?

For my part, there are several pain points:

1. Inconsistency in the various mechanisms. We have to look across multiple components and apply multiple models to get the information out that we need.

2. Performance cost of collecting diagnostic information

3. Incomplete diagnostic information

4. Continued lack of documented diagnostic use cases to help drive continued development

5. Instability / bugs in the diagnostic tooling (assuming I get the time, I want to rip out the entire trace event mechanism and replace it with something that works better)

[gireeshpunathil]

We have this devised last year https://docs.google.com/document/d/1T2hGtYcUxLiBZD29daDNvZkgRZi4YPNHwYqZ8YtT0yw/edit#

that was driven through nodejs/user-feedback#165 , but I am not 100% sure where we went from there.

Agree that it is worthwhile to float a survey. Already tagged `diag-agenda, so we can discuss in the next sitting.

1. Inconsistency in the various mechanisms. - agree, this I guess Proposal to drive Diagnostics WG initiatives through user journeys #295 is trying to address

2. Performance cost of collecting diagnostic information - I guess this issue is scoped within profiling tools only?

3. Continued lack of documented diagnostic use cases to help drive continued development - I am not ure why this is happening. Defenitely adoption and usage scenarios are wide, yet we don't seem to receive RFEs! restartPolicy architecture of kubernetes can be a reason, but I may be wrong.

[naugtur]

Re 2, heap snapshots and core dumps have costs too, but I don't know what we'd get from the survey on performance cost. People are barely equipped to tell what their apps performance is, they're unlikely to know anything about the impact of tools.

Am I misunderstanding?

[naugtur]

As for 3 I can say I've been teaching people how to analyze heap snapshots since 2012 and it keeps being news to everyone. Ranging from 2yr experienced devs to CTOs

[mmarchini]

The survey (especially the first questioned mentioned by OP) was focused on core development experience, which I believe is different than the DX of Node.js projects? If that's the case, the Users Journeys we've been working on wouldn't be quite that helpful.

It's also worth noting that a bit annoying and an occasional blocker could be a result of constant flaky tests in CI and the time it takes just to build the project to try out something (IIRC I answered one of those two for this question, and the reason was more because of the huge overhead just to get started and not because of the debugging tools).

I agree a follow up survey to dive deeper into this area would help shed light into what are the issues folks are facing.

[mmarchini]

Should we keep it on the agenda?

[mmarchini]

Removing from agenda, let's re-add if discussion gets traction again

[github-actions]

This issue is stale because it has been open many days with no activity. It will be closed soon unless the stale label is removed or a comment is made.