Download an LFS backed file from GitLab.com without `git` and `git-lfs` installed

Well, the API is there and you can do it already!

Just dig into the what git is doing by a test-clone with any LFS repo:

export GIT_CURL_VERBOSE=1                                                                               
export GIT_TRACE_CURL=1                                 
git clone <my-repo> 2>&1 | tee git-clone.log                                                

From there you are able to figure what is happing on SSH:

Now: download an LFS backed file from GitLab.com without git and git-lfs installed!

#!/usr/bin/env bash

PROJECT="${1}" # your-company/or-repo/whatever
FILEPATH="${2}" # foo/bar.tar.gz
REF="${3:-master}" # branch
SSHID="${4:-$HOME/.ssh/ecdsa_id}" # like $HOME/.ssh/mycompany

OID="$(set -o pipefail; curl -fSL -H "PRIVATE-TOKEN: ${GITLAB_TOKEN}" "https://gitlab.com/api/v4/projects/${PROJECT//\//%2F}/repository/files/${FILEPATH//\//%2F}/raw?ref=${REF}" -o - | grep '^oid' | cut -d: -f2)"
AUTH="$(set -o pipefail; ssh -o IdentitiesOnly=yes -o IdentityFile="${SSHID}" git@gitlab.com git-lfs-authenticate "${PROJECT}.git" download | jq -er '.header.Authorization')"
mkdir -p "${FILEPATH%%/*}"
curl -H "Authorization: $AUTH" "https://gitlab.com/${PROJECT}.git/gitlab-lfs/objects/${OID}" -o "${FILEPATH}"
file "${FILEPATH}"

And you can even do it without using SSH!

#!/usr/bin/env bash

PROJECT="${1}" # your-company/or-repo/whatever
FILEPATH="${2}" # foo/bar.tar.gz
BRANCH="${3:-master}" # branch

# get LFS file info
IFS=$'\n' LFS_OBJECT_INFO=($(curl -fsSL -H "PRIVATE-TOKEN: ${GITLAB_TOKEN}" "https://gitlab.com/api/v4/projects/${PROJECT//\//%2F}/repository/files/${FILEPATH//\//%2F}/raw?ref=${BRANCH}" -o -))
OID="$(echo "${LFS_OBJECT_INFO[@]}" | grep -oP "oid sha256:\K[^\s]+")"
SIZE="$(echo "${LFS_OBJECT_INFO[@]}" | grep -oP "size \K[^\s]+")"
# request actions for given lfs objects
LFS_OBJECT_REQUEST_JSON="$(jq -rnc --arg oid $OID --arg size $SIZE --arg ref "refs/heads/$BRANCH" '{"operation":"download","objects":[{"oid":$oid,"size":$size}],"transfers":["lfs-standalone-file","basic"],"ref":{"name": $ref }}')"
LFS_OBJECT_ACTION_DOWNLOAD="$(set -o pipefail; curl -sSlf -H "Content-Type: application/json" -X POST -d "$JSON" "https://oauth2:${GITLAB_TOKEN}@gitlab.com/${PROJECT}.git/info/lfs/objects/batch" | jq -er '.objects[0].actions.download')"
AUTH="$(echo "$LFS_OBJECT_ACTION_DOWNLOAD" | jq -er '.header.Authorization')"
DOWNLOAD_URL="$(echo "$LFS_OBJECT_ACTION_DOWNLOAD" | jq -er '.href')"
mkdir -p "${FILEPATH%%/*}"
curl -H "Authorization: ${AUTH}" "${DOWNLOAD_URL}" -o "${FILEPATH}"

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.