Help! We've ran into a DockerHub rate limit!

About

Yes, it is still happining. In 2025! Here you will find:

• Podman Dockerhub Mirror Configuration
• K8s Quickfix: Rewriting Existing K8s Resources
• Permanent Mirror Configuration for containerd
• K8s Admission Webhook to do the same

Podman Dockerhub Mirror Configuration

~/.config/containers/registries.conf.d/dockerhub-mirror.conf:

[[registry]]                                                                                              
prefix = "docker.io"                                                      
insecure = false                                                                              
blocked = false
location = "public.ecr.aws/docker"  

[[registry.mirror]]
location = "mirror.gcr.io"

[[registry.mirror]]
location = "gitlab.com/acme-org/dependency_proxy/containers"

[[registry.mirror]]
location = "registry-1.docker.io"                                                              

[[registry.mirror]]
location = …

Fix for Amazon SSM Login on rootfs (no disk space left)

Sometimes AWS’s SSM StartInteractiveCommand doesn’t work once an EC2 instance’s root fs has run out of disk space.

aws ssm start-session \
  --target i-0ab4e6dce100a0f58 \
  --document-name AWS-StartInteractiveCommand 

/etc/systemd/system/var-lib-amazon-ssm.mount

# SSM agent won't be able to login when disk is full
# so we reserve some space in-memory

[Unit]
Description=Mount /var/lib/amazon/ssm as tmpfs for SSM Agent
Documentation=man:systemd.mount(5)
Before=amazon-ssm-agent.service

[Mount]
What=tmpfs
Where=/var/lib/amazon/ssm
Type=tmpfs
# Mount options:
#   defaults: Standard options
#   noatime: Do not update inode access times for performance
#   nosuid: Do not allow set-user-identifier or set-group-identifier bits to take effect
#   nodev: Do not interpret character or block special devices
#   noexec: Do not allow execution of binaries
#   mode=1777: Set directory permissions to rwxrwxrwt (sticky bit, world-writable)
#   size=64M: Limit the size of the …

Amazon Linux 2025 has been officially canceled

(Some people didn’t take note of that, so let’s make it clear)

Already in 2024(!) AWS re:Invent had news on that topic that did not get much attention: There will be NO AL2025! *

AWS cited customer feedback requesting more stability and longer support cycles rather than frequent major version changes. Many organizations found the biannual major releases challenging for “enterprise deployment cycles”. – maybe 20th century organizations? 😄

Therefore:

• AL2 EOL has been extended to 2026
• and AL2023 EOL until 2029!

Instead of Amazon Linux 2025, AWS will focus on AL2023 Enhancement:

Craftsmanship And The Right Tools for Your Job

I wonder, why I my hardware was always superior than the one my companies provided me with. Shouldn’t they be interested in getting best quality? Would I order an electrician, and then forbid him to use his tools and give him my IKEA toolbox? – I wouldn’t!

A Software Craftsman’s tools are not mere instruments; they are his accumulated skill, capital, and tradition made tangible. I believe you have to care for your tools. Also, ★nix craftsmen often tend to solve problems with the capabilities of their systems.

DwarFS vs. SquashFS

I couldn’t see if DwarFS was comparing against SquashFS with LZMA. So I did my own tests.

Results

Multi, Mono, Meta, Manifest – Composite Repository?

There was this discussion about whether to use Mono- or Multi-Repositories? I won’t pick it up again.

Some cool people suggested: why not use the best of both worlds and use meta-repos?!

I Was Interested in “Meta-Repo” Tooling

I asked myself how these tools would solve problems and gave some of them a try:

• There are multi-repository management tools that work by tagging or grouping repos, like gr or mu-repo.
• Tools that just do subdirectory iteration over your repos, like gitbatch.
• Many git-extras repositories with subdirectory iteration scripts that do the same, like git multi (the one I am using ;)).
• Tools which combine multi-repo manifests with different VCS systems, like myrepos (old and mature, often found within your Linux system package management).
• Tools that try to standardize the directory layout for managing your repositories, like ghq (you should definitely use such a layout!).
• Tools which basically reassemble what git submodule does, like mu-repo, git-metarepo, or meta …

How to get AWS-CLI v2 down from 127M to 67M

Follow these steps:

 0
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

FROM alpine:3.12

# 1. Install glibc compatibility for Alpine
RUN apk --no-cache add binutils \
    && echo "Getting libc libraries" \
    && curl -sL https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub -o /etc/apk/keys/sgerrand.rsa.pub \
    && curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VER}/glibc-${GLIBC_VER}.apk \
    && curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VER}/glibc-bin-${GLIBC_VER}.apk \
    && echo "Installing libc libraries" \
    && apk add --no-cache \
        glibc-${GLIBC_VER}.apk \
        glibc-bin-${GLIBC_VER}.apk

# 2. Install rush parallel runner (temporary)
# This is used for optimizing the botocore data later.
RUN echo "Installing rush parallel runner …

Microsoft WSL2 kernel modifications

If you want to dig into: for now it seems all to be HyperV related. I maybe wrong since I haven’t reviewed the code itself.

It’s based on the next 5.4.x kernel - probably since Ubuntu Focal also has 5.4 on LTS.

git clone --depth 1 --branch v5.4.51 https://github.com/gregkh/linux.git upstream & pid1=$!
git clone --depth 1 --branch linux-msft-5.4.51  https://github.com/microsoft/WSL2-Linux-Kernel.git wsl2 & pid2=$!
wait $pid1 $pid2

File differences

diff -qr --exclude=.git upstream wsl2 | tee diff.txt
cat  \
    <(cat diff.txt | grep -oP '(Files upstream/)\K[^ ]+') \
    <(cat diff.txt | grep -oP '(Only in wsl2/)\K.+' | sed 's|: |/|g') \
    | sort -u \
    | (while read f; do if [[ -f "wsl2/$f" ]]; then echo "$f"; fi; done;) \
| tee files.txt

Get all commits on diffed files

Improvement: Diff the original commit list with WSL source

Techstack n - 1 is dead!

TL;DR TechStack n-1 is dead. It ended with the rise of the clouds and software release cycles going down to weeks due to containerized CIs.

Against ‘it’s stable and mature so let it run’

Beeing OpenSource-based, Ubuntu already had the concept of point releases every 6 months when the Docker and K8s hit the world and gave automated CIs a big boost in making system containers. Some years after Docker itself switched to a 3-month release cycle. So did the Linux Kernel with 2-3 months. Firefox 4-weeks.

Download an LFS backed file from GitLab.com without `git` and `git-lfs` installed

It is possible to download a Git LFS-backed file from GitLab.com without having git or git-lfs installed by using the GitLab API directly. This article provides two shell scripts that demonstrate how to do this.

Well, the API is there and you can do it already!

Just dig into what git is doing by a test-clone with any LFS repo:

export GIT_CURL_VERBOSE=1
export GIT_TRACE_CURL=1
git clone <my-repo> 2>&1 | tee git-clone.log

From there you are able to figure out what is happening on SSH.