How to get AWS-CLI v2 down from 127M to 67M

Take these steps:

FROM alpine:3.12

# install glibc compatibility for alpine
RUN apk --no-cache add binutils \
    && echo "getting libc libraries" \
    && curl -sL https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub -o /etc/apk/keys/sgerrand.rsa.pub \
    && curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VER}/glibc-${GLIBC_VER}.apk \
    && curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VER}/glibc-bin-${GLIBC_VER}.apk \
    && echo "installing libc libraries" \
    && apk add --no-cache \
        glibc-${GLIBC_VER}.apk \
        glibc-bin-${GLIBC_VER}.apk \
    && echo "installing of rush parallel runner (temporary)" \
    && curl -sSfL https://github.com/shenwei356/rush/releases/download/v0.4.2/rush_linux_amd64.tar.gz  -o - \
        | tar -C /tmp/ -zxf - \
    && echo "installing AWS CLI" \
    && curl -sL https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip -o awscliv2.zip \
    && unzip awscliv2.zip \
        -x 'aws/dist/awscli/examples/**' 'aws/dist/docutils/**'\
    && aws/install  \
    && echo "Cleaning and sliming AWS CLI installation" \
    && rm -rvf \
        awscliv2.zip \
        aws \
        /usr/local/aws-cli/v2/*/dist/aws_completer \
        /usr/local/aws-cli/v2/*/dist/awscli/data/ac.index \
        /usr/local/aws-cli/v2/*/dist/awscli/examples \
    && find /usr/local/aws-cli/v2/current/dist/botocore/data/ -name "*.json" -name "examples*" -exec rm {} \; \
    && find /usr/local/aws-cli/v2/current/dist/botocore/data/ -name "*.json" \
        | /tmp/rush 'echo "optimizing {}"; jq -cer "del(.. | .documentation?)" "{}" > "{}.tmp" && mv "{}.tmp" "{}"' \
    && echo "Removing artificats" \
    && rm /tmp/rush \
    && apk --no-cache del \
        binutils \
    && rm glibc-${GLIBC_VER}.apk \
    && rm glibc-bin-${GLIBC_VER}.apk \
    && rm -rf /var/cache/apk/*

I do think there can be much more improved in botocore (like compressing assets) or just rewritting AWS CLI to Golang. I hope if an v3 hits the floor it is in Golang (for the easy of use; go sdk is already very powerful and maybe there is an alternative client – but would be fine if that is 100% compatible to AWS CLI).

“Ternary operator” with JQ: Checking for empty values

As you noticed I use JQ commonly like RegExp. I recently used select(.!=null) to filter for non-null values. Turns out I missed something in docs:

jq -nc '{a:1},[1,23],true,null,42|values'

OUTPUT

{"a":1}
[1,23]
true
42
jq -nc '{a:1},[1,23],true,null,42|iterables'

OUTPUT

{"a":1}
[1,23]
jq -nc '{a:1},[1,23],true,null,42|nulls'

OUTPUT

null

Now that can be used to set defaults:

jq -nc '.notset?|values // "default"'

OUTPUT

"default"
$ jq -nc '.notset|values // "default"'

OUTPUT

"default"

Or just use it to do some little validation on expected input:

jq -nc '.notset|error("has no value")'

OUTPUT

jq: error (at <unknown>): has no value
jq -nc '{foo:"bar"}|(.baz|values // error("baz not set")|.'

OUTPUT

jq: error (at <unknown>): has no value
jq -nc '{foo:"bar"}|(.baz|values // error("baz not set")) as $e|.'
jq: error (at <unknown>): baz not set
jq -nc '{foo:"bar"}|(.foo|values // error("foo not set")) as $e|.'

OUTPUT

{"foo":"bar"}
jq -nc 'def chkkey(k): .[k]|values // error(k + "not set"), {foo:"bar"}|(.foo|values // error("foo is empty")) as $e|.'

OUTPUT

{"foo":"bar"}
jq -nc 'def notempty(k): .[k]|values // error(k + " is empty or not set"); {foo:"bar"}|notempty("foo")'

OUTPUT

"bar"
jq -nc 'def notempty(k): .[k]|values // error(k + " is empty or not set"); {baz:"bar"}|notempty("foo")'

OUTPUT

jq: error (at <unknown>): foo is empty or not set

Let’s turn it into a helper function:

jq -nc 'def notempty(k):
        . as $in
        |(
            (k|arrays // [k])[] as $k
            |$in[$k]|values // error($k + " is empty or not set")
        ) as $x
        |$in;

OUTPUT

{"foo":"bar","baz":"mar"}
jq -nc 'def notempty(k):
        . as $in
        |(
            (k|arrays // [k])[] as $k
            |$in[$k]|values // error($k + " is empty or not set")
        ) as $x
        |$in;
        {foo:"bar", baz: "mar"}|notempty(["meer"])'

OUTPUT

jq: error (at <unknown>): meer is empty or not set