Migration

[GuillaumeGomez]

1. find out how to keep each history

   git filter-branch for creating a new branch of each repo where the files are moved into a subdirectory

2. how the final repository should look like (folder wise)
   (both folders and repositories:) sys/..., gir-files, atk, cairo, gdk, gdk-pixbuf, gdkx11, gio, glib, graphene, gtk, examples, pango, pangocairo
3. migrating the CI to github-actions
4. checking how to keep the "release branches" from the other repositories

   by renaming: glib-0.10, et

5. update release script
6. update the crate versions
7. what about the repositories' issues?

   very likely using a python script to migrate them all

8. update the crates' git repositories URL
9. archive old repositories and add a link to the new repository
10. Add a python script (like the one in gstreamer-rs) to handle regenerations

[bilelmoussaoui]

Does it make sense to keep the gtk4 ones separate ? there are quiet few of them actually: gsk4, gdk4, gdk4-x11, gdk4-wayland, graphene, gtk4 & probably sourceview5

[GuillaumeGomez]

We wait for their official stabilization to bring them in. Also, some crates that aren't "core" will remain outside of this repository (like sourceview).

[jplatte]

Oh, you want to create a monorepo? 🙂

I recommend using git subtree to merge repos. It's as simple as git subtree add -P <prefix> <repository> <ref> to merge an existing repository with its whole history into another one.

[GuillaumeGomez]

I'm almost done already. ;) The issue is mostly how the repository should look like in the end.

[sdroege]

One problem with this is that all commit/PR/issue references in the commit messages are broken now.

[jplatte]

Actually, if you're using git subtree or an equivalent, commit references won't break.

The rest would be true too if you moved to GitLab or anything else. I strongly encourage everyone to not put anything in commit messages that only works if the repository is at a specific location.

You could also always write your own custom script based on git filter-branch that rewrites the commit messages to contain permalinks, but that would probably be way too much effort to be worth it.

[GuillaumeGomez]

No, no subtree in here.

[jplatte]

Still my point stands. If you're already preserving history, commit hashes (and thus references in other commits) are not necessarily going to break – depending on whether you rewrite all of the commits you integrate.

[sdroege]

Yeah I agree and it's not really a problem, I just wanted to mention it :)

However I think something broke when merging the repos. The files don't have any history anymore.

Compare for example https://github.com/gtk-rs/gtk-rs/blame/ongoing/cairo/src/surface.rs vs https://github.com/gtk-rs/gtk-rs/commits/ongoing/cairo/src/surface.rs . git blame works, git log doesn't.

That seems a bit suboptimal and can be prevented. See for example GStreamer's merged repos: https://gitlab.freedesktop.org/thiblahute/gstreamer/-/commits/monorepo_simple/subprojects/gst-plugins-base/gst/audiomixer/gstaudiomixer.c

I think that is my only concern right now.

Bugfix releases we would do from the old repos so that's fine (don't archive them yet!), old tags stay in the old repo also not a problem.

[jplatte]

@sdroege This may even be a difference of GitHub vs. GitLab, rather than git log not working. I'm seeing the same thing in a subtree-merged repo on GitHub. How was merging done in GStreamer?

[sdroege]

@sdroege This may even be a difference of GitHub vs. GitLab, rather than git log not working. I'm seeing the same thing in a subtree-merged repo on GitHub.

It's the same effect when using git directly on the commandline though. Something's not correct there :)

How was merging done in GStreamer?

Taking each repo individually, git mv to a new subdirectory inside each, then git merge them all into one.

[sdroege]

As we found out, this happened because @GuillaumeGomez updated the commit messages of all commits after the merge commit. So this has to be redone, and also we decided to not update the commit messages anymore as this will also cause chaos with the existing merge commits inside the repositories.

[sdroege]

Turns out this is actually a bug in the git UI and github just replicates that, while gitlab handles it better. So I guess we'll have to live with this if we want to merge the repos.

[sdroege]

The only way around that is to use git filter-branch to move all the files inside the individual commits instead of doing a git mv later.

[ids1024]

Turns out this is actually a bug in the git UI and github just replicates that, while gitlab handles it better.

Not a bug, just a design decision, and only the default.

In the CLI, git log --follow cairo/src/surface.rs will show you the full history. I don't know if GitHub provides any way to do that in the web interface.

[sdroege]

In the CLI, git log --follow cairo/src/surface.rs will show you the full history

This only works for actual files though, not for directories. Not much of a problem here, I guess.

[bilelmoussaoui]

I'm wondering what's still left to be done as part of the migration?

[sdroege]

Moving the remaining issues and PRs

[GuillaumeGomez]

I think we can now close this issue.