✈️ Give to package.json

[chadwhitacre]

---

✈️ This is the flight deck for the Give to package.json project. ✈️

---

🗻 That project is part of the Make it easier for companies to fund open source epic. 🗻

---

The definition of done is when we can say "Give us $10,000 and a package.json, and we'll distribute it for you."

Todo

• make room on the /on/npm/foo/ page for the claiming workflow—move the name to the banner h1, and replace the wombat with the description: Make room for package claiming workflow #4304
• new email subflooring: Refactor mixins #4343 Improve email subsystem #4344 Start an Application object #4345 Fix regression in deployment #4365 Use Application in Payday #4350 Further Application improvements #4355 Move some email API #4346 Limit email address length #4347 Simplify verification landing page #4348 Throttle better #4349 Convert Throttle to a 400 #4383
• pull emails from maintainers only, not author (Integrate initial package claiming PRs #4305 (comment)): Rebuild npm syncing #4438
  • also dedupe email addresses: Rebuild npm syncing #4438
• set up npm syncing—besides just needing to stay in sync, people are going to want to update the email address on file in their package.json release a new version take some action at npm and then "immediately" see the new email show up to continue the flow: Rebuild npm syncing #4438
  • add sync lag to health dashboard
• Implement claim/unclaim/reclaim packages: Integrate initial package claiming PRs #4305.
• bulk add from /on/npm/: Add bulk claiming at /on/npm/ #4488
• discovery via package.json: Provide discovery via package.json #4507
• fix bugs
• write blog post justifying amount recommendation
• "Pay for Open Source" on the homepage: "Pay for Open Source" form on homepage #4598
• update payment processing howtos: Document new payment process inside.gratipay.com#1154
• write howto for managing mailing list of company payers
• Takeoff on the project board

---

✈️ This is the flight deck for the Give to package.json project. ✈️

---

[chadwhitacre]

#4305 is awesome ... but how does one get to a package page in the first place? Search? Browse? Edit URL? Eventually #4416 will be a nice way to get there based on email, but even then we'll still a way to get to a particular package based on its name as well.

[chadwhitacre]

Kicking out a bunch of new tickets in Takeoff. Need to stay focused on #4305.

[chadwhitacre]

Alright, #4305 is topped up, consolidated, w/ three remaining pretty manageable PRs.

Looking beyond that, we need to get syncing going again. That's a pre-req and a not insignificant one, though we had some implementation of that before and there's code around somewhere towards it. I guess that's the primary focus next week (but don't forget gratipay/inside.gratipay.com#1051!).

Once syncing is out and we deploy #4305, there are a few directions we can go:

• Cleanup—There are lots of little inefficiencies piled up on the runway ready for takeoff. Which are highest value?
• Discovery—There is no way right now to get from gratipay.com to any given npm package page. Search? Browse? Bulk-add?
• Outreach—We can start emailing individual package maintainers inviting them to apply to join, with links to their specific packages.
• Announcement—At some point we should write that blog post and see if there's anything interesting here.

[chadwhitacre]

Alright, scope clarified. This is about the "Give us $10,000 and a package.json" MVP. What's left for that is syncing, and then pledging (#4195), and then a UI for uploading a package.json like we had started looking at in #4135. How far can we get this week?

[chadwhitacre]

I'm reading through our existing npm syncing code and retracing why we turned it off (we turned it off, didn't we?) and thinking through how to turn it back on.

[chadwhitacre]

I'm finding an initial attempt at #4148 (comment) (same thread as our tech debt discussion, btw).

[chadwhitacre]

Alright, the way I'm reading #4148 (and jogging my memory) is that the initial plan of running a worker dyno to continually update from npm did not work out because it was too buggy. So I gave up on running the syncer continuously and instead focused on getting it to run once for an initial load. We did fix a number of issues in the syncer after that point but never got back around to setting it up to run continuously. The first successful load took two days though so I think we should look into some better parallelization.

[chadwhitacre]

Moving to PR in #4438 ...

[chadwhitacre]

Alright, we're out of time to deploy npm claiming (#4305) this week. I fly out in 18 hours and then go straight into a weekend conference. It's too risky to try to deploy #4305 before then. I'd be fine to deploy npm syncing (#4438), though. I'm going to try to get that ready for review in the next few hours.

@dowski @rohitpaulk If either of you have any cycles in the next 12 hours to review #4438, that would be awesome. Then we can deploy tomorrow morning and let it run over the weekend. Should be pretty low risk since it'll be in a separate worker process operating on a barely-used table (packages), and it'll be good to have a couple days experience with how it behaves before deploying #4305 on top of it.

Let's aim for Monday for the big #4305 rollout. @clone1018 and I will be together in Austin and we will have Monday and Tuesday free. #4305 and #4441 are the two top priorities for those days.

This is happening, people!

!m * 💸 💸 💸 💸 💃 🌻

[chadwhitacre]

We are live with package claiming!

[chadwhitacre]

lol, duped existing aspen. :)

[chadwhitacre]

HEY @mattbk COULD WE PLEASE LINK NPM PACKAGE TO EXISTING?!?!??! :P

[chadwhitacre]

Alright, we've landed #4305 and squished a few regressions. I find ten remaining tickets on the project board. I've split them into "Heavy" (4) and "Light" (6) columns. I guess the reality is that we've got another three to six months of work here on this project.

[chadwhitacre]

I might prioritize #4476 over #4195. It should be simpler and it is something interesting for the giver side vs. the receiver side which is what we've been focusing on so far with #4305 and #4488. (slack)

[chadwhitacre]

Yeah, going for it. With #4476 we'll be able to minimally say, "give to package.json." Everything else from there—and there's plenty!—is icing on the cake. Hope you like icing. 😁 🍰

[chadwhitacre]

#4476 landed in #4507. Now for some icing ... and bugfixes.

[chadwhitacre]

I folded the two Takeoff columns together.

[chadwhitacre]

So do we fix bugs and improve incrementally? Or do we try to get the whole dream workflow implemented barely with chicken wire first?

[chadwhitacre]

Incremental. Steady on, as we've been. #4305 took us four months. I started thinking through the Whole Dream Workflow and it's feeling like it will take about that again, three to six months as estimated above at #4427 (comment). Then we'll see if anyone wants it! 😅

[chadwhitacre]

I don't want to wait that long. 😞

[chadwhitacre]

Alright, well, I've kicked off a PR in #4569 for the Whole Dream Workflow. #4548 is a prerequisite for that as well as for #4539. Kick the can down the road! :-)

[chadwhitacre]

Turns out we need some significant improvements to our email infrastructure before we can move forward with the Whole Dream Workflow (WDW). Priorities I see there:

• Once we land the UI changes in Limit pending verifications in emails UI #4579 "Ready to verify" #4584 that will mitigate the worst of our present phishing risk (Mitigate npm spam/phishing #4557). There'll be more to do on that as we scale the WDW, but for now I think the next step is to ...
• Allow for sending emails to non-participants (Support queueing emails without participants #4548), which requires an unsubscribe/blacklist.
• Automated bounce/marked-as-spam handling (catch mail bounces and mark as such in db/ui #2993) is close behind.
• Insta-send (Send email messages immediately #4545) is a nice-to-have but not critical for getting the WDW out the door initially.

[chadwhitacre]

Alright, I've kicked off #4598 to try to close the loop on giving to package.json. I am hoping we can move forward with a semi-manual process, because I don't want to spend six months building this out. It's so risky—we don't even know yet if anyone actually wants this enough to pay for it! Let's find out! :-)

In addition to the development work on #4598, we are going to need a ToS review, a HOWTO for handling the payments that come through #4598, and a blog post justifying a recommended dollar amount.

[chadwhitacre]

slack

[chadwhitacre]

Looks like we'll also need an admin dashboard for reviewing and distributing payments.

[chadwhitacre]

Might need an integration branch to bring together #4598 #4600 ... etc.?

[chadwhitacre]

Brag page could double as dashboard?

[chadwhitacre]

Blog post draft: https://gratipay.news/9205443e209d

[chadwhitacre]

Got some feedback from @jdorfman on the blog post draft, and made some updates to it. 👍

[chadwhitacre]

The definition of done is when we can say "Give us $10,000 and a package.json, and we'll distribute it for you."

Kind of amazing. This is getting stripped down, essentialized, bare-metaled in #4598:

The definition of done is when we can say "Give us $10,000 and a package.json, and we'll distribute it for you."

[mattbk]

Not to add any more work tonight, but I mean they could email us with suggestions in the interim before any uploading/parsing/ecosystems are added to the form.

[chadwhitacre]

This has evolved into gratipay/inside.gratipay.com#1160. I'm shedding weight from this project so we can bring this in for a landing.