Aws-Cli

How to get AWS-CLI v2 down from 127M to 67M

Follow these steps:

 0
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48


FROM alpine:3.12

# 1. Install glibc compatibility for Alpine
RUN apk --no-cache add binutils \
    && echo "Getting libc libraries" \
    && curl -sL https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub -o /etc/apk/keys/sgerrand.rsa.pub \
    && curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VER}/glibc-${GLIBC_VER}.apk \
    && curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VER}/glibc-bin-${GLIBC_VER}.apk \
    && echo "Installing libc libraries" \
    && apk add --no-cache \
        glibc-${GLIBC_VER}.apk \
        glibc-bin-${GLIBC_VER}.apk

# 2. Install rush parallel runner (temporary)
# This is used for optimizing the botocore data later.
RUN echo "Installing rush parallel runner (temporary)" \
    && curl -sSfL https://github.com/shenwei356/rush/releases/download/v0.4.2/rush_linux_amd64.tar.gz -o - \
    | tar -C /tmp/ -zxf -

# 3. Install AWS CLI v2
RUN echo "Installing AWS CLI" \
    && curl -sL https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip -o awscliv2.zip \
    && unzip awscliv2.zip \
        -x 'aws/dist/awscli/examples/**' 'aws/dist/docutils/**' \
    && aws/install

# 4. Clean and Slim the Installation
RUN echo "Cleaning and slimming AWS CLI installation" \
    # Remove installation files and unnecessary components
    && rm -rvf \
        awscliv2.zip \
        aws \
        /usr/local/aws-cli/v2/*/dist/aws_completer \
        /usr/local/aws-cli/v2/*/dist/awscli/data/ac.index \
        /usr/local/aws-cli/v2/*/dist/awscli/examples \
    # Remove example JSON files from botocore data
    && find /usr/local/aws-cli/v2/current/dist/botocore/data/ -name "*.json" -name "examples*" -exec rm {} \; \
    # Optimize remaining JSON files by removing 'documentation' field using jq/rush
    && find /usr/local/aws-cli/v2/current/dist/botocore/data/ -name "*.json" \
        | /tmp/rush 'echo "optimizing {}"; jq -cer "del(.. | .documentation?)" "{}" > "{}.tmp" && mv "{}.tmp" "{}"'

# 5. Final Cleanup
RUN echo "Removing artifacts" \
    && rm /tmp/rush \
    && apk --no-cache del \
        binutils \
    && rm glibc-${GLIBC_VER}.apk \
    && rm glibc-bin-${GLIBC_VER}.apk \
    && rm -rf /var/cache/apk/*

Notes

I do think there can be much more improved in botocore (like compressing assets) or just rewritting AWS CLI to Golang.

AWS CLI V2 is generally available since February 2020. BUT…

The AWS CLI V2 is distributed as a binary package, but this new distribution method comes with a few caveats.

It is distributed as a binary package (built on Python with PyInstaller that has bundled native libs) BUT…

Problem #1 this requires a decent GLIBC and probably won’t run on older Red Hat distributions.
Problem #2 as it turns out, many people use the official docker images to build their images and push to AWS ECR. And since these images are based on Alpine and therefore use MUSL and NOT GLIBC, it simply won’t run (but well, people have seen Alpine pipelines in AWS builds now, although they do NOT support any other Linux than their own; probably THEY HAVE TO).
Problem #3 you now have to download the application to update it (hey, the way to install software on Windows or Mac? But hey, an own updater with aws/install --update).
Problem #4 forget installing and updating from pip for now; well, you can directly pip from the repo, BUT it’s tricky. Although the version says 2.0.38 (no beta or alpha!) it still requires a botocore==2.0.0dev42. I guess they should suffix a “ga” for “General Availability”.
Problem #5 If you belong to the majority (or the people with broken python installations) please refrain from saying “I have no problems! It’s better now!”

Anyway, they have some nice features now, and it seems they packed aws-shell right into the CLI: https://aws.amazon.com/blogs/developer/aws-cli-v2-is-now-generally-available/ https://www.youtube.com/watch?v=U5y7JI_mHk8