Multi, Mono, Meta, Manifest โ€“ Composite Repository?

Mono or Multiple-Repositories? Meta-Repos!

There was this discussion about whether to do Mono or Multiple-Repositories? I wonโ€™t pick it up again.

Some cool guys that suggested: why not use the best of the two world and use meta-repos?!

I was interested in โ€œmeta-repoโ€ tooling

I asked myself how they would solve problems and gave soem of them a try:

  • there are multi-repository management tools that work with by tagging or grouping repos like gr or mu-repo
  • tools that just do subdirectory iteration over your repos like gitbatch and โ€ฆ
  • many git-extras repisitories with subdirectory iteration scripts that do the same like git multi – the one i am using ๐Ÿ˜‰
  • tools which combine multi-repo manifests with different VCS systems like myrepos โ€“ old and mature, within your linux system package management
  • tools that try to standardize the directory layout for managing your reositories like ghq โ€“ definitly you should use such layout!
  • tools which basically reassemble what git submodule does like mu-repo or git-metarepo or meta (see below)
  • tools that just checkout all your orgโ€™s repositories from GitHub, GitLab or BitBucket like ghorg
  • and git itself, who was not lazy and added git for-each-repo which also does simple iteration (experimental status!)
  • and then there was git-slave where itโ€™s master deceided it was an anachronistic project and decided to give it up

In the end most tools mentioned here do not focus on workflow problems rather than introducing new manifests or config files ๐Ÿ™

But basicallygit submodule is a meta-repo since we have .submodules!

Yes, thatโ€™s aboslutly valid. .submodules basically behaves like a multiple repository manifest (and you can set submodules also to active:false)

  • You can use git submodule foreach to create/commit/manage multiple branches
  • You could just setup a workspace or manifest repository that reassembles what other tools do

But isnโ€™t there more to it?

  • There-repo are tools like repo which is used by Android ecosystem or the Alibaba Golang rewrite git-repo-go

  • They are using a manifest repository as well but you donโ€™t need you to check out everything and it has an efficient way of managing dependencies
    (Android OS has 2000 repositories with repos different devices and drivers!)

  • They add pre-/post-process like symlinking or comamnd execution over mutliple repisitories (just like myrepos )
  • They introduce the concept of a topic (your current task) and solve workflow problems like having to change several sources in different repositories for one feature, diffing over mutliple repisitories and automatically creating a code review (using gerrit)

Letโ€™s redefine our problem!

We want git submodules and an umbrella to add some workflow features wouldnโ€™t be that wrong?

I did a search and came up with git-meta (not to be mistaken by other projects of the same name). They also have a nyce presention that bring up the mist important thoughts (WATCH IT!)

Unfortunatly GitLab docs did not return results on how to better solve where to put merge requests on meta-repos etc.

And then there is the more established meta which has 1.4K stars (only?) . But they try to reinvent .submodules and add a ton of (un)related features with like mono-meta-repo project migration. And they even use .metain their own repository to defined plugin-dependencies #facepalm.

The main issues on meta-repos are still GitLab and how to integrate your workflow (like multiple merge requests; microservices should be decoupled and should not break!).

I donโ€™t know what tool is better – try it out! I just say i would use

So .submodules alone make a repository a meta-repository?

Yes and no โ€“ when you treat donโ€™t treat dependencies like 3rd party libraries but an actual composite of multi-repos you need to work on.

Meta-Repos are a workaround for big too-big-to-be-fun mono-repositories and a half-baked solution for composite workspaces (like needed for microservices) since needed umbrella workflows are not integrated into GitHub or GitLab. If the problem doesnโ€™t hurt donโ€™t try to solve it since you create new problems. You can still use Virtual Workspaces like linking repo folders in your IDE or via symlink. Just try to stay focused on one task before having multiple workspaces open at the same time.

  • Multi-Pipeline setup can be simplified if using a meta-repo (or submodules)
  • Each affected service (repository) on your working topic will have a merge-request
  • .submodulesshould have CI checks (like is every repository-branch correctly set in .submodules ?)

Is โ€˜meta-repoโ€™ a buzzword?

Yes. Basically itโ€™s multi-repo and not much new except a new view on the workflow issues of having a composite repos.

Still you can use git subtreein your project but keep in mind when a repository is shared among many projects/workspaces it may diverge since upstreams are treated as second-class citizen and itโ€™s harder to maintain to branch from dev on new features.

But then again: git-meta sayโ€™s in their project description:

Repository for the git-meta project — build your own monorepo using Git submodules

Multi, Mono, Meta – Composite Repo?

What now?

Further Readings

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.