Gitlab
Download an LFS backed file from GitLab.com without git and git-lfs installed
It is possible to download a Git LFS-backed file from GitLab.com without having git or git-lfs installed by using the GitLab API directly. This article provides two shell scripts that demonstrate how to do this.
Well, the API is there and you can do it already!
Just dig into what git is doing by a test-clone with any LFS repo:
export GIT_CURL_VERBOSE=1
export GIT_TRACE_CURL=1
git clone <my-repo> 2>&1 | tee git-clone.log
From there you are able to figure out what is happening on SSH.
GitLab: checkout all available repositories
This guide provides a set of shell commands to automate the process of checking out all available repositories from one or more GitLab instances. It leverages the GitLab API, jq, and parallel to efficiently clone projects.
Generate a private token
https://<GITLAB-SERVER1>/profile/personal_access_tokens
https://<GITLAB-SERVER2>/profile/personal_access_tokens
Checkout a list of all available repositories
QUERY=".[] | .path_with_namespace + "\t" + .ssh_url_to_repo" # JQ Query
curl --request GET --header "PRIVATE-TOKEN: <PRIVATE-TOKEN>" "<GITLAB-SERVER1>/api/v4/projects?simple=true&per_page=65536" | jq -r "$QUERY" > repo.list
curl --request GET --header "PRIVATE-TOKEN: <PRIVATE-TOKEN>" "<GITLAB-SERVER2>/api/v3/projects?simple=true&per_page=65536" | jq -r "$QUERY" >> repo.list
Create directories for repositories
cat repo.list | cut -f1 | xargs mkdir -p
Checkout projects (with GNU parallel)
parallel --colsep '\t' --jobs 4 -a repo.list git clone {2} {1}
Build list of git repositories
find . -type d -name ".git" | xargs realpath | xargs dirname > path.list
Report repository branch or checkout branch
cat path.list | xargs -I{} sh -c "cd {}; echo {}; git branch"
cat path.list | xargs -I{} sh -c "cd {}; echo {}; git checkout master"
cat path.list | xargs -I{} sh -c "cd {}; echo {}; git checkout develop"
Note: when you are migrating repositories you should use git clone --mirror.
BASH in Gitlab CICD and Developer Projects (2023)
