Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add self-hosted Gitlab migration support #9096

Closed
wants to merge 12 commits into from

Conversation

aqtrans
Copy link
Contributor

@aqtrans aqtrans commented Nov 20, 2019

This extends PR#9084 to allow migrating from self-hosted Gitlab instances.

Detection is based on adding a '#gitlab' fragment to the end of the given Clone Address.

@techknowlogick techknowlogick added the type/feature Completely new functionality. Can only be merged if feature freeze is not active. label Nov 21, 2019
@6543 6543 mentioned this pull request Nov 21, 2019
24 tasks
@maicWorkGithub
Copy link

would you add an option ignore certificate?

@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Dec 4, 2019
@stale
Copy link

stale bot commented Feb 2, 2020

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs during the next 2 months. Thank you for your contributions.

@stale stale bot added the issue/stale label Feb 2, 2020
@stale stale bot removed the issue/stale label Apr 1, 2020
@6543

This comment has been minimized.

@6543

This comment has been minimized.

@6543 6543 force-pushed the gitlab-migrate_selfhosted branch from 4e09879 to 7c2dba1 Compare April 19, 2020 20:00
@6543

This comment has been minimized.

modules/migrations/gitlab.go Outdated Show resolved Hide resolved
Just add a '#gitlab' fragment to the end of your Clone Address.
@6543 6543 force-pushed the gitlab-migrate_selfhosted branch from 8aa129a to 21b7783 Compare April 19, 2020 20:43
modules/migrations/gitlab.go Outdated Show resolved Hide resolved
Copy link
Member

@6543 6543 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

found a way to test it - worked just fine 🎉

@GiteaBot GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Apr 19, 2020
@codecov-io
Copy link

Codecov Report

Merging #9096 into master will increase coverage by 0.00%.
The diff coverage is 0.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #9096   +/-   ##
=======================================
  Coverage   43.26%   43.26%           
=======================================
  Files         601      601           
  Lines       85531    85534    +3     
=======================================
+ Hits        37001    37010    +9     
+ Misses      43958    43951    -7     
- Partials     4572     4573    +1     
Impacted Files Coverage Δ
modules/migrations/gitlab.go 1.75% <0.00%> (-0.02%) ⬇️
services/pull/pull.go 33.96% <0.00%> (ø)
routers/repo/view.go 36.82% <0.00%> (+0.67%) ⬆️
modules/git/repo.go 51.04% <0.00%> (+1.25%) ⬆️
models/unit.go 44.44% <0.00%> (+2.46%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 842bba9...8a450df. Read the comment docs.

@mrsdizzie
Copy link
Member

I think adding #gitlab to the URL for detection feels like a confusing interface -- we should just add a drop down menu or something even if gitlab is the only option for now. I know there should be gitea/gogs at some point too (there is a PR for it) and maybe/hopefully more in the future.

Copy link
Member

@6543 6543 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

x

@6543
Copy link
Member

6543 commented Apr 25, 2020

@aqtrans are you on this?

@6543
Copy link
Member

6543 commented Apr 29, 2020

@techknowlogick can we add this to v1.12 since it works only UI has change ...

@guillep2k
Copy link
Member

I agree with @mrsdizzie here. There should be either a drop-down or a list of radio buttons (auto-detect, GitHub, GitLab, ... others to come). Slight problem here is that GitHub doesn't allow for a custom host address, which would be interesting for consistency (but it can't be helped since the api is provided by GitHub itself). Either way, IMHO it should not be a fragment of the URL. 😁

@techknowlogick
Copy link
Member

another +1 for what @mrsdizzie said. In GitLab they have a migration page where you first select the forge type (gitlab, github, gitea, etc..) and then it will have different settings based on what you've selected (for example github we may want to have different help text about personal tokens needed for migration), and we could also split mirror and migration as it has lead to confusion in past.

@guillep2k our GitHub migration logic would also in theory support github enterprise too at a custom URL.

I am thankful for this PR as it has brought up these discussions. If it goes in I wouldn't be upset, and so I won't be a blocker, but I think the high-level needs to be reviewed first. Maybe it goes in temporarily until we can get improve the migration experience?

@@ -43,6 +44,9 @@ func (f *GitlabDownloaderFactory) Match(opts base.MigrateOptions) (bool, error)
if strings.EqualFold(u.Host, "gitlab.com") && opts.AuthUsername != "" {
matched = true
}
if strings.EqualFold(u.Fragment, "gitlab") && opts.AuthUsername != "" {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For self-hosted gitlab, we need a new option on the UI to let user select.

@lunny
Copy link
Member

lunny commented May 2, 2020

A new wizard UI is better. I propose we have a migration type selection page before current.
There are four migration types you can chose:

  • Mirror from a git service
  • Migrate from a github service
  • Migrate from a gitlab service
  • Migrate from other git service

The extra page just give a query param to the current page with a migration type.

@stale
Copy link

stale bot commented Jul 18, 2020

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs during the next 2 months. Thank you for your contributions.

@stale stale bot added the issue/stale label Jul 18, 2020
@6543
Copy link
Member

6543 commented Jul 18, 2020

no stale just nobody wants to contribute the frontend 😅

@stale stale bot removed the issue/stale label Jul 18, 2020
@lunny lunny added the issue/confirmed Issue has been reviewed and confirmed to be present or accepted to be implemented label Jul 19, 2020
@capi
Copy link

capi commented Aug 25, 2020

It would be great if this could be merged with the #gitlab fragment. I agree it's not "good UI" but at least it would work and I'm eagerly waiting to migrate some projects from my self-hosted Gitlab to Gitea. Thanks in advance!

@6543
Copy link
Member

6543 commented Aug 25, 2020

@aqtrans do you have time, I wont for in the next weeks ...

@codecov-commenter
Copy link

Codecov Report

Merging #9096 into master will decrease coverage by 0.01%.
The diff coverage is 0.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #9096      +/-   ##
==========================================
- Coverage   43.41%   43.39%   -0.02%     
==========================================
  Files         645      645              
  Lines       71288    71290       +2     
==========================================
- Hits        30947    30939       -8     
- Misses      35327    35333       +6     
- Partials     5014     5018       +4     
Impacted Files Coverage Δ
modules/migrations/gitlab.go 1.72% <0.00%> (-0.02%) ⬇️
modules/indexer/stats/db.go 52.17% <0.00%> (-8.70%) ⬇️
modules/git/utils.go 73.77% <0.00%> (-3.28%) ⬇️
modules/queue/unique_queue_disk_channel.go 53.84% <0.00%> (-1.54%) ⬇️
modules/log/event.go 56.60% <0.00%> (-0.95%) ⬇️
modules/git/repo.go 49.23% <0.00%> (-0.51%) ⬇️
services/pull/pull.go 41.57% <0.00%> (-0.47%) ⬇️
modules/util/timer.go 85.71% <0.00%> (+42.85%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update eb4db04...570c612. Read the comment docs.

@lunny lunny closed this in #12619 Aug 28, 2020
@go-gitea go-gitea locked and limited conversation to collaborators Nov 24, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
issue/confirmed Issue has been reviewed and confirmed to be present or accepted to be implemented lgtm/need 1 This PR needs approval from one additional maintainer to be merged. type/feature Completely new functionality. Can only be merged if feature freeze is not active.
Projects
None yet
Development

Successfully merging this pull request may close these issues.