Introduction: I'm John. How can I help?

[jkleinsc]

Hi! I'm John Kleinschmidt and I work for Microsoft on Electron. My coworker Charles Kerr and I are planning on pairing together 1 day a week to help out the Node.js Build Working Group.

While I would consider myself primarily a coder, I have been running Electron’s CI for about 2 years and am hoping that my experience there will be helpful here. In particular here are the areas I think I can be helpful:

• Jenkins - Electron initially used Jenkins so I have experience on that platform and am comfortable helping out there.
• CI on diverse platforms - Electron doesn’t run on as many platforms as Node, but we do test on Mac, Linux, and Windows. Additionally, we have special setups for armv7, arm64, and Windows on Arm which have required some unique CI solutions.
• Maturing CI offerings - Electron started off on Jenkins and other in-house CI and for a while this met our needs but over time it became clear that we needed to provide a more mature solution. I’m happy to say that today Electron has a comprehensive and robust CI setup that is scalable and for the most part just lets developers get their work done. I’m hoping my experience here will be helpful to this group in maturing Node’s build experience.

[rvagg]

Welcome @jkleinsc & @ckerr!

First I'll be honest that we still haven't figured out how best to onboard people smoothly. The challenges of balancing the access & trust required to access most of our resources creates a bit of a speed bump in getting people up to speed. So I hope you'll be patient with us as we work on this.

We also have @davidstanke, from Google, who is keen to contribute too (#2171) we're all very keen to find a way for you all to become productive contributors because we have a large load that we'd like to share! It is considerably easier to establish trust relationships with people who work for large companies that already contribute to Node (the contractual employment arrangement means we get a legal basis for trust). So I think we should be able to get you all up to our basic access level fairly quickly.

I think the first thing is to establish your presence around here so that there is a sense among the @nodejs/build team that there's a basic level of commitment. Some suggestions for that:

• Watch this repo, it's not high traffic
• Participate in our meetings (every 3 weeks-ish, it's on the Node.js gcal, which I think you can add from here: https://calendar.google.com/calendar/embed?src=nodejs.org_nr77ama8p7d7f9ajrpnu506c98%40group.calendar.google.com, there's an issue opened on this repo prior to each meeting as a reminder and we collect agenda items -- questions and requests for deep-dives are welcome).
• Comment, review PRs, put up your hand if you think you can contribute something
• Scan nodejs/node issues and PRs tagged with build: https://github.com/nodejs/node/issues?q=is%3Aissue+is%3Aopen+label%3Abuild (the ones where @nodejs/build are mentioned are particularly interesting to us, some others just touch toolchain items).
• There's a list of open items being discussed in Introduction: I'm Dave. How can I help? #2171, feel free to pick some things off and ask if you can contribute (in here, that issue, or a new issue).
• Open new issues here if you want to make suggestions or want to dig in to things that are not clear

In terms of scope of potential work (/cc @davidstanke, @AshCripps and anyone else newish too), here's my thoughts from a historical perspective:

Aside from the Node.js codebase itself, the Build Working Group is the oldest formal part of what is now the OpenJS Foundation. We started it while we were still negotiating with Joyent for more open governance of Node.js (under the "node-forward" banner). That process eventually escalated with the fork (io.js) but even by that time we already had our Jenkins infrastructure which was farm more sophisticated than what Joyent had, and were accumulating donor infrastructure companies that were keen to contribute to Node and we were the only ones offering a way to do that.

After the "merge" and the formation of the Foundation, our infrastructure and the complexity of what we do ramped up, in accordance with the skill and time of the people we had contributing and the amount of donated infrastructure we were able to accumulate. A lot of that was ad-hoc, as these things often are, but most of it has persisted. We maintain most of what can be called "infrastructure" that the project relies on, not just Jenkins.

We're now 6 years removed from that, we've gone through a period of growth and the last few years have been more characterised by decline, mainly in terms of people-time. Thankfully IBM have stepped up in recent years and has been our main people-contributor and this has kept us going fairly strong.

We have a relatively sophisticated setup, a significant amount of resources (all still gratis!), but it constantly shows its age. We've talked many times about rearchitecting it to properly iron out all the wrinkles (flaws) that we see but it's hard to do that while still continuing to serve nodejs/node, libuv and some other associated projects. Most of our time is spent on fixing things and patching over older things, but we still have the basic architecture we've always had. Access keeps on being a problem we have to find ways around. Some of the tasks can only be done by a small core group who have access to key resources--it would be nice if our architecture allowed for more granularity than it currently does.

So in summary, there are two parallel ways to think about contributions here: (1) help us maintain and slowly improve what we have to serve Node.js (and associated projects), (2) help us build some next-generation pieces of the puzzle to replace what we're currently doing (CI, www asset serving, release pipeline, test/release/infra server maintenance, access granularity, relationships with donor companies, relationship with client projects, management of this repo, management of the team, etc. etc.). There's a lot of scope for doing things in entirely new ways, you'll just have to apply some skill in navigating such things in a well-established open source project with a team of opinionated (and sometimes grumpy) people.

[Trott]

I have nothing concrete to add, but want to comment anyway to say, OMG, welcome, hooray, can't wait to talk more at a meeting or something like that!

[AshCripps]

I think i'm a bit of a special case when comes to new joiners as I had IBM platform specific tasks prepped for me before I joined IBM as a grad, but I think a good start would be to look through the docs with fresh eyes as weve just had a sort out in #2151 so would be good to have an outside opinion on anything thats not clear. (Feel free to PR any improvements)

Dont think I have anything else extra to add to Rod's point besides feel free to attend the next meeting -> #2198

[mhdawson]

Great to see you get engaged @jkleinsc & @ckerr, as mentioned when we touched base if you need help/are stuck etc. Feel free to reach out to me for an in-person meeting in addition to opening issues etc. in the repo.

[sam-github]

Hello you all. Just a quick ping, have you had time to look at anything here?

[jkleinsc]

Hi @sam-github! I started working on #1931 via nodejs/node#32129 and I've run into something I'd like verification on - where are the Jenkins configs for CI stored? As far as I can tell they are not in GitHub anywhere, so I'm guessing they are just stored server side? The reason I'm asking is that in nodejs/node#32129 I added testing from the tarball builds but the tests aren't passing and because I can't see the configs used for testing elsewhere, I'm not sure if there is an actual issue with the tarball builds or if I'm just running the tests differently than they are running in CI.

[jkleinsc]

I should mention I also looked into #2205 with a POC of running an arm64 self hosted GitHub action runner.

[sam-github]

@jkleinsc answered the specific CI config question over in your PR, but wrt. the larger problem,
https://ci.nodejs.org/job/node-test-commit-linux/configure <-- you can't see that, I guess?

@nodejs/build Why isn't the config viewable by anybody? There are no secrets in any config I've seen, is it a jenkins limitation in how permissions can be setup?

[jkleinsc]

@sam-github if I go to that URL, all I see is:
jkleinsc is missing the Job/ExtendedRead permission

[sam-github]

I support giving everyone that access. Lets give time for the build-wg to comment on why it isn't already there.

[richardlau]

It's because of this:

i.e. we've given extended read to collaborators but not to general authenticated users. AFAIK there's nothing secret in the job configs.

[rvagg]

I'm not super confident that we're secrets-free across the board. I can't think of any instances where we might have thrown things into job config that shouldn't be but I'm always surprised at the strange things I find in ci.nodejs.org.

https://github.com/nodejs/jenkins-config-test is up and running and contains up to date config XML from the server. We've just not opened it up yet to the public. I've suggested it would be good for someone to spend some time looking through these configs to make sure that we're not publishing anything we probably shouldn't.

[sam-github]

jenkins-config-test/jobs (master u=) % ls *.xml | wc -l
145
jenkins-config-test/jobs (master u=) % cat *.xml | wc -l
61388

We might need a new issue, with a long checklist, and to partition the work on that.

[github-actions]

This issue is stale because it has been open many days with no activity. It will be closed soon unless the stale label is removed or a comment is made.

[Trott]

Have we lost our opportunity to collaborate here? Or can we pick this up again?

[jkleinsc]

While I would love to participate in the Build Working Group, I don't think I currently have capacity to do so. If that changes, I'll open up an new issue.