[server] Don't skip prebuilds if .gitpod.yml has a 'before' task but no 'init' task #10352

jankeromnes · 2022-05-30T13:21:02Z

Description

Don't skip prebuilds if .gitpod.yml has a before task but no init task.

Related Issue(s)

Fixes #10336

Drive-by fix:

When GitHub (Enterprise) sends webhooks without a host header, fall back to the hostname from the repository URL (fixes Unsupported GitHub Enterprise host: undefined error preventing prebuild trigger with webhooks from github.com and various GitHub Enterprise versions)

How to test

Release Notes

Fix: Don't skip prebuilds if .gitpod.yml has a 'before' task but no 'init' task

Documentation

jldec · 2022-06-02T06:04:48Z

/werft run

👍 started the job as gitpod-build-jx-fix-before.1
(with .werft/ from main)

jldec · 2022-06-02T06:47:00Z

I tested this and it works for me with a GitLab repo ☑️

However I do not see the prebuilds being triggered for a project using the test-before GitHub repo in this preview env, so I think something is still not right in that scenario (not using GitHub app, just OAuth afaict)

I have invited @jankeromnes as collaborator on https://github.com/jldec/test-before and sent you invite to the project team.

jankeromnes · 2022-06-02T07:33:26Z

Many thanks @jldec for testing this PR!

And also for inviting me to your test assets:

I see in https://jx-fix-before.preview.gitpod-dev.com/t/test-before/projects that you have added only your GitHub repo as a Project
I'm invited to both https://github.com/jldec/test-before and https://gitlab.com/jldec/test-before, but not as an admin, so I cannot install webhooks (i.e. I cannot add them as Projects or test them) -- that's not a problem though, because I can fork both repos

Looking into the database, I see that:

you've added and deleted both GitLab and GitHub repos
you've then added only the GitHub repo
the usage database shows that a workspace was started for the GitHub repo, but no webhook was ever received 🤔

mysql> select * from d_b_project_usage;
+--------------------------------------+---------------------+--------------------------+---------+----------------------------+
| projectId                            | lastWebhookReceived | lastWorkspaceStart       | deleted | _lastModified              |
+--------------------------------------+---------------------+--------------------------+---------+----------------------------+
| dad0fbef-b9ba-45c0-8e42-20e78d61b435 |                     | 2022-06-02T06:54:10.488Z |       0 | 2022-06-02 06:54:10.496795 |
+--------------------------------------+---------------------+--------------------------+---------+----------------------------+
1 row in set (0.02 sec)

Will now try to reproduce by forking your repos, adding them as projects, and pushing to them. 👀

EDIT: Results:

Only the GitLab push triggered a new prebuild
The GitHub push did not trigger a new prebuild -- looking into why 🔍

jankeromnes · 2022-06-02T07:49:25Z

A few suspicious error messages from server, making it look like it's trying (and failing) to use GitHub Enterprise code for github.com 🤔

Note: This looks like a real bug, but unrelated to this Pull Request. Will dive into it anyway since I'm here. 🔍

When I add the GitHub project and it tries to install a webhook:

    {
        component: "server",
        severity: "INFO",
        time: "2022-06-02T07:35:24.056Z",
        message: "Update prebuild installation for project.",
        payload: {
            cloneUrl: "https://github.com/jankeromnes/test-before.git",
            teamId: "0380f06c-322c-4345-988c-0ab64688f173",
            installerId: "4b2615de-396c-4c49-9f94-a42b57a157e0",
        },
    },
    {
        component: "server",
        severity: "INFO",
        time: "2022-06-02T07:35:25.433Z",
        message: "Unknown GitHub Enterprise event received",
        payload: { event: "ping" },
    },

When I push a commit to GitHub:

    {
        "@type": "type.googleapis.com/google.devtools.clouderrorreporting.v1beta1.ReportedErrorEvent",
        serviceContext: { service: "server", version: "jx-fix-before.1" },
        stack_trace:
            "Error: Unsupported GitHub Enterprise host: undefined\n    at GitHubEnterpriseApp.<anonymous> (/app/node_modules/@gitpod/server/dist/ee/src/prebuilds/github-enterprise-app.js:90:27)\n    at Generator.next (<anonymous>)\n    at /app/node_modules/@gitpod/server/dist/ee/src/prebuilds/github-enterprise-app.js:22:71\n    at new Promise (<anonymous>)\n    at __awaiter (/app/node_modules/@gitpod/server/dist/ee/src/prebuilds/github-enterprise-app.js:18:12)\n    at GitHubEnterpriseApp.findUser (/app/node_modules/@gitpod/server/dist/ee/src/prebuilds/github-enterprise-app.js:84:16)\n    at GitHubEnterpriseApp.<anonymous> (/app/node_modules/@gitpod/server/dist/ee/src/prebuilds/github-enterprise-app.js:54:39)\n    at Generator.next (<anonymous>)\n    at /app/node_modules/@gitpod/server/dist/ee/src/prebuilds/github-enterprise-app.js:22:71\n    at new Promise (<anonymous>)\n    at __awaiter (/app/node_modules/@gitpod/server/dist/ee/src/prebuilds/github-enterprise-app.js:18:12)\n    at /app/node_modules/@gitpod/server/dist/ee/src/prebuilds/github-enterprise-app.js:46:46\n    at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5)\n    at next (/app/node_modules/express/lib/router/route.js:137:13)\n    at Route.dispatch (/app/node_modules/express/lib/router/route.js:112:3)\n    at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5)",
        component: "server",
        severity: "ERROR",
        time: "2022-06-02T07:37:01.517Z",
        message: "Cannot find user.",
        error: "Error: Unsupported GitHub Enterprise host: undefined\n    at GitHubEnterpriseApp.<anonymous> (/app/node_modules/@gitpod/server/dist/ee/src/prebuilds/github-enterprise-app.js:90:27)\n    at Generator.next (<anonymous>)\n    at /app/node_modules/@gitpod/server/dist/ee/src/prebuilds/github-enterprise-app.js:22:71\n    at new Promise (<anonymous>)\n    at __awaiter (/app/node_modules/@gitpod/server/dist/ee/src/prebuilds/github-enterprise-app.js:18:12)\n    at GitHubEnterpriseApp.findUser (/app/node_modules/@gitpod/server/dist/ee/src/prebuilds/github-enterprise-app.js:84:16)\n    at GitHubEnterpriseApp.<anonymous> (/app/node_modules/@gitpod/server/dist/ee/src/prebuilds/github-enterprise-app.js:54:39)\n    at Generator.next (<anonymous>)\n    at /app/node_modules/@gitpod/server/dist/ee/src/prebuilds/github-enterprise-app.js:22:71\n    at new Promise (<anonymous>)\n    at __awaiter (/app/node_modules/@gitpod/server/dist/ee/src/prebuilds/github-enterprise-app.js:18:12)\n    at /app/node_modules/@gitpod/server/dist/ee/src/prebuilds/github-enterprise-app.js:46:46\n    at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5)\n    at next (/app/node_modules/express/lib/router/route.js:137:13)\n    at Route.dispatch (/app/node_modules/express/lib/router/route.js:112:3)\n    at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5)",
        payload: {},
    },

jankeromnes · 2022-06-02T09:17:35Z

Looks like github.com does not send / no longer sends its hostname in the webhook headers 🤔 how can we know that it comes from github.com?

      {
        "host": "jx-fix-before.preview.gitpod-dev.com",
        "user-agent": "GitHub-Hookshot/b257cdf",
        "content-length": "7424",
        "accept": "*/*",
        "content-type": "application/json",
        "x-forwarded-for": "10.0.2.2",
        "x-forwarded-host": "jx-fix-before.preview.gitpod-dev.com",
        "x-forwarded-proto": "https",
        "x-github-delivery": "062e6fb4-e254-11ec-9eab-e106b31fee52",
        "x-github-event": "push",
        "x-github-hook-id": "361307289",
        "x-github-hook-installation-target-id": "499037053",
        "x-github-hook-installation-target-type": "repository",
        "x-hub-signature": "sha1=91f7211e2b5258dce6754c7a702836e812b8bed2",
        "x-hub-signature-256": "sha256=6589db1af6a201ff3db0b0ceabe95df722703be5490ed9bd2851ed01178a15f1",
        "x-real-ip": "10.0.2.2",
        "accept-encoding": "gzip"
      }

…no 'init' task

…r, fall back to the hostname from the repository URL Fixes "Unsupported GitHub Enterprise host: undefined"

jankeromnes · 2022-06-02T10:02:32Z

Update: I've added a drive-by fix which makes us use the hostname from the repository URL when GitHub sends webhooks without a host header. ✅

This should fix triggering prebuilds on GitHub pushes here (when using webhooks instead of the app), and also generally fix prebuild webhooks for various other GitHub Enterprise versions (I had noticed the same error message in prod a few times, which means a few GHE servers were affected by this bug too).

jldec · 2022-06-05T12:15:05Z

/werft run

👍 started the job as gitpod-build-jx-fix-before.5
(with .werft/ from main)

jldec

Works for me now on both GitLab and GitHub
Thank you @jankeromnes

jankeromnes requested a review from a team May 30, 2022 13:21

roboquat added release-note size/XS labels May 30, 2022

github-actions bot added the team: webapp Issue belongs to the WebApp team label May 30, 2022

jankeromnes mentioned this pull request May 30, 2022

[GitHub.com only] Prebuilds no longer triggering if gitpod.yml uses a 'before' task without an 'init' #10336

Closed

geropl requested a review from jldec May 30, 2022 15:29

geropl assigned jldec May 30, 2022

jankeromnes force-pushed the jx/fix-before branch from 2595bae to 556b206 Compare June 2, 2022 09:21

jankeromnes and others added 2 commits June 2, 2022 09:57

[server] Don't skip prebuilds if .gitpod.yml has a 'before' task but …

8a59838

…no 'init' task

[server] When GitHub (Enterprise) sends webhooks without a host heade…

72846e8

…r, fall back to the hostname from the repository URL Fixes "Unsupported GitHub Enterprise host: undefined"

jankeromnes force-pushed the jx/fix-before branch from 556b206 to 72846e8 Compare June 2, 2022 09:59

roboquat added size/S and removed size/XS labels Jun 2, 2022

jldec approved these changes Jun 7, 2022

View reviewed changes

roboquat merged commit f0b3ea6 into main Jun 7, 2022

roboquat deleted the jx/fix-before branch June 7, 2022 05:25

laushinka mentioned this pull request Jun 9, 2022

Prebuild only run when first time of project add into self-host Gitpod #10024

Closed

AlexTugarev mentioned this pull request Jun 9, 2022

Epic: Prebuilds stability #10361

Closed

11 tasks

roboquat added deployed: webapp Meta team change is running in production deployed Change is completely running in production labels Jun 9, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[server] Don't skip prebuilds if .gitpod.yml has a 'before' task but no 'init' task #10352

[server] Don't skip prebuilds if .gitpod.yml has a 'before' task but no 'init' task #10352

jankeromnes commented May 30, 2022 •

edited

Loading

jldec commented Jun 2, 2022 •

edited by werft-gitpod-dev-com bot

Loading

jldec commented Jun 2, 2022 •

edited

Loading

jankeromnes commented Jun 2, 2022 •

edited

Loading

jankeromnes commented Jun 2, 2022 •

edited

Loading

jankeromnes commented Jun 2, 2022 •

edited by werft-gitpod-dev-com bot

Loading

jankeromnes commented Jun 2, 2022 •

edited

Loading

jldec commented Jun 5, 2022 •

edited by werft-gitpod-dev-com bot

Loading

jldec left a comment

[server] Don't skip prebuilds if .gitpod.yml has a 'before' task but no 'init' task #10352

[server] Don't skip prebuilds if .gitpod.yml has a 'before' task but no 'init' task #10352

Conversation

jankeromnes commented May 30, 2022 • edited Loading

Description

Related Issue(s)

How to test

Release Notes

Documentation

jldec commented Jun 2, 2022 • edited by werft-gitpod-dev-com bot Loading

jldec commented Jun 2, 2022 • edited Loading

jankeromnes commented Jun 2, 2022 • edited Loading

jankeromnes commented Jun 2, 2022 • edited Loading

jankeromnes commented Jun 2, 2022 • edited by werft-gitpod-dev-com bot Loading

jankeromnes commented Jun 2, 2022 • edited Loading

jldec commented Jun 5, 2022 • edited by werft-gitpod-dev-com bot Loading

jldec left a comment

Choose a reason for hiding this comment

jankeromnes commented May 30, 2022 •

edited

Loading

jldec commented Jun 2, 2022 •

edited by werft-gitpod-dev-com bot

Loading

jldec commented Jun 2, 2022 •

edited

Loading

jankeromnes commented Jun 2, 2022 •

edited

Loading

jankeromnes commented Jun 2, 2022 •

edited

Loading

jankeromnes commented Jun 2, 2022 •

edited by werft-gitpod-dev-com bot

Loading

jankeromnes commented Jun 2, 2022 •

edited

Loading

jldec commented Jun 5, 2022 •

edited by werft-gitpod-dev-com bot

Loading