Support Forgejo / Gitea webhooks

[mvdkleijn]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

---

• I'd be willing to implement this feature (contributing guide)

Describe the user story

As a devops engineer maintaining a fully Open Source stack for an Open Source service, I would like to use Atlantis & Terraform to manage our infrastructure, but I cannot do so because Atlantis does not support Forgejo / Gitea.

Describe the solution you'd like
Support the Forgejo / Gitea webhooks.

Compared to the current Github, Bitbucket, Gitlab & Azure solutions, Forgejo / Gitea is the only full, free and true Open Source product made by & for the community.

Since Atlantis says it is for the benefit of the community and that is the reason why it wants to be, and remain, an Open Source tool... I would like to request that Forgejo / Gitea webhooks be supported.

Forgejo / Gitea is a stable and feature rich, free open source alternative to the big four. It is increasingly popular and well known.

Describe the drawbacks of your solution
Maintenance overhead for a 5th forge. (possibly 4th, see comment below)

Describe alternatives you've considered

• Github:
  • not free,
  • expensive,
  • not open source,
  • legal concerns
• Gitlab:
  • not free,
  • open source,
  • roadmap concerns
• Bitbucket:
  • not free,
  • expensive,
  • not open source,
  • not a common platform for Open Source devs
• Azure DevOps:
  • not free,
  • expensive,
  • not open source,
  • not a common platform for Open Source devs,
  • legal concerns,
  • roadmap / continued existence concerns (considering remarks from MS rep stating GH is probably the way to go instead when asked about Azure DevOps future)

(edit: layout)

[kirstus]

I managed to get Atlantis to work with Gitea by changing two lines, by forcing atlantis to use Gitea as if it were a Github Enterprise instance, although the changes were pretty botchy and i haven't tested it at all.

Gitea aims to be 100% compatible with Github, although Gitea api V1 is compatible with Github api v3. So this is the first point of change: https://github.com/runatlantis/atlantis/blob/main/server/events/vcs/github_credentials.go#L196

But even with that change into v1, the api call still adds another api/v3/ in front of it. That's because the go-github library adds that suffix, so i had to change this line too: https://github.com/google/go-github/blob/v53.2.0/github/github.go#L389

To make this simple integration work officially, there could be at least three possible solutions:

• Up the Gitea api version to v3;
• Fork the go-github repo and make changes on it and on Atlantis;
• Write the Gitea handler in Atlantis, completely separate of the implementation for Github.

The first one is clearly the one with least effort, and the one which impacts Atlantis the less, but depends on a lot of discussion and major decisions over on the Gitea project.

Also, I could make it work using basic auth, as Gitea's and Github's token auth is slightly different.

Either way, depending on the chosen solution this integration could spill over into Gitea backlog more than on Atlantis.

PR used for tests:
https://try.gitea.io/kirstus/tst/pulls/3

⚠️I haven't actually applied anything, so tests are definitely still required and very welcome using Gitea, Gogs, Forgego and other open alternatives.

[jamengual]

I agree, I think the first approach is the best, we do not want to use a forked go-library in the project so if Gitea go library gets updated then you can add the new VCS for Gitea to atlantis.

[jseiser]

We are back looking at this, since we are wanting to move on from gitlab since it no longer will support kubernetes fully. If go-github lib was able to get a PR in place to allow setting "api/v3/" via some sort of env var or something, would atlantis be OK with doing the same thing here: https://github.com/runatlantis/atlantis/blob/main/server/events/vcs/github_credentials.go#L196

Trying to avoid having to fork both repo's.

[jamengual]

as long as we default it to the current value, we will be open to do that

[mvdkleijn]

So... As far as I can tell, nobody is actively working on this right? How likely do we think it is that Google / the maintainers of go-github will make the necessary changes?

(I'm submitting an issue anyway)

[mvdkleijn]

Ooookay... I was about to hit submit to an issue on go-github when I noticed this line in the doc:

// Note that NewEnterpriseClient is a convenience helper only;
// its behavior is equivalent to using NewClient, followed by setting
// the BaseURL and UploadURL fields.

Translation: we don't need a change in the go-github library if the Atlantis project would be willing to create a single convenience function itself instead of relying on the one provided by go-github.

It would be equivalent to using the NewClient function instead of the NewEnterpriseClient function but with the added benefit of allowing easy support for Gitea. Its not a complex function anyway so maintenance costs would be low. (see code snippet at the bottom)

What's your take on this @jamengual ?

// code taken from go-github

func NewEnterpriseClient(baseURL, uploadURL string, httpClient *http.Client) (*Client, error) {
	baseEndpoint, err := url.Parse(baseURL)
	if err != nil {
		return nil, err
	}

	if !strings.HasSuffix(baseEndpoint.Path, "/") {
		baseEndpoint.Path += "/"
	}
	if !strings.HasSuffix(baseEndpoint.Path, "/api/v3/") &&
		!strings.HasPrefix(baseEndpoint.Host, "api.") &&
		!strings.Contains(baseEndpoint.Host, ".api.") {
		baseEndpoint.Path += "api/v3/"
	}

	uploadEndpoint, err := url.Parse(uploadURL)
	if err != nil {
		return nil, err
	}

	if !strings.HasSuffix(uploadEndpoint.Path, "/") {
		uploadEndpoint.Path += "/"
	}
	if !strings.HasSuffix(uploadEndpoint.Path, "/api/uploads/") &&
		!strings.HasPrefix(uploadEndpoint.Host, "api.") &&
		!strings.Contains(uploadEndpoint.Host, ".api.") {
		uploadEndpoint.Path += "api/uploads/"
	}

	c := NewClient(httpClient)
	c.BaseURL = baseEndpoint
	c.UploadURL = uploadEndpoint
	return c, nil
}

[jamengual]

I think we need @GenPage @lukemassa @nitrocode @chenrui333 @X-Guardian eyes here

[GenPage]

@kirstus @jseiser @mvdkleijn

After reviewing, it makes sense to me that we can get away with supporting Gitea without overloading the existing GHE client interface. We can always refactor down the road if there is an issue to track in the upstream go-github library when official Gitea support becomes available.

[mvdkleijn]

So if I understand you correctly @GenPage:

a PR that adds a wrapper function to atlantis a la go-github's NewEnterpriseClient() would be an acceptable way to work around the hardcoded versions?

If so, I'll try to add something next week-ish.

[GenPage]

@mvdkleijn Correct, I wouldn't be opposed to additional logic in a wrapper function to compensate for the lack of support for Gitea in go-github. I would envision an entirely separate client. (gitea_client.go). The function can be in there or abstracted in a higher interface, whatever makes sense.

[mvdkleijn]

@mvdkleijn Correct, I wouldn't be opposed to additional logic in a wrapper function to compensate for the lack of support for Gitea in go-github. I would envision an entirely separate client. (gitea_client.go). The function can be in there or abstracted in a higher interface, whatever makes sense.

An entirely separate client makes sense and would indicate the project's intention to support more than proprietary SaaS solutions, which would be a nice bonus.

I'll see if I can scrounge some time in the next couple of weeks to work on this. Though I would definitely not mind if someone beats me to it.

[bencurio]

Despite the similarities, I would not recommend using GitHub's SDK to communicate with Gitea instance.

Instead, here is the official SDK from Gitea: https://gitea.com/gitea/go-sdk
Documentation: https://pkg.go.dev/code.gitea.io/sdk/gitea

[mvdkleijn]

Despite the similarities, I would not recommend using GitHub's SDK to communicate with Gitea instance.

Instead, here is the official SDK from Gitea: https://gitea.com/gitea/go-sdk
Documentation: https://pkg.go.dev/code.gitea.io/sdk/gitea

Honestly, I didn't even realize that there was a SDK for Gitea. It'll probably take a little more work to do a full Gitea client but that's obviously the best.

[mvdkleijn]

FYI, I've started a branch on my fork based on the Gitea SDK. It might take a while for me to get a draft PR in due to family life but we'll see. Planning to create a draft PR as soon as I've implemented the interface, regardless of whether I had time to add tests / try it out.

[mvdkleijn]

I managed to find some time this evening to ram out an initial client implementation. In the hopes that others can help test.

Be forewarned: I did not (yet) have the time to test it in any capacity. I.e. it's probably still full of holes. Feel free to comment on PR #4229

[mvdkleijn]

Small update for those keeping track: @florianbeisel found my PR and had also started work on his own version for this issue. After contacting me, we decided to work together on this PR.

[mvdkleijn]

For the users keeping track of this issue but not the PR, a small update.

We, @mvdkleijn and @florianbeisel, have brought our PR into a state where we feel it is ready for consideration by maintainers and possible testing by users who'd be so inclined.

If you are reading this and are inclined to help test out the PR for Gitea support, please leave your feedback on the actual PR over at #4229

[mvdkleijn]

Just an update for those following along... we have resolved all of the comments and issues that popped up from reviews so far and are now waiting for two final approvals. So unless something pops up from the final two reviews, I'm hopeful we'll get this in for the next release.

[mvdkleijn]

It's merged! Cool, thanks again @florianbeisel @GenPage @nitrocode @X-Guardian and everonone else! :)

[silopolis]

Awesome work guys! 👍😎