≡

Python

AWS CLI V2 is generally available since February 2020. BUT…

The AWS CLI V2 is distributed as a binary package, but this new distribution method comes with a few caveats.

It is distributed as a binary package (built on Python with PyInstaller that has bundled native libs) BUT…

Anyway, they have some nice features now, and it seems they packed aws-shell right into the CLI: https://aws.amazon.com/blogs/developer/aws-cli-v2-is-now-generally-available/ https://www.youtube.com/watch?v=U5y7JI_mHk8

Infojunk November 2018

This is a collection of interesting links and resources I came across in November 2018, covering topics such as security, Linux, AWS, and development.

Hacking / MITM-API-Testing

Linux

Windows

Python

KataCode

Spectre/Meltdown

Project

Security

Tools

AWS

Development

Other

bash: shell table output to json

This post presents a Python script that converts tabular command-line output into a more versatile JSON format. This allows for easier data manipulation using tools like jq, as an alternative to complex text-processing pipelines in bash.

You know that sometimes it would be really great to format a shell output to a more versatile format like JSON or YAML that you can process with jq instead of writing long pipes with text-processing.

Infojunk October 2018

This is a collection of interesting links and resources I came across in October 2018, covering a wide range of topics including browser extensions, collaborative coding, Linux, AWS, and more.

Browser Extensions

Collaborative Coding

Focusing on IDEs. Web-based solutions are mostly ignored.

Linux

NodeJS

DevOps

AI/MachineLearning

AWS

JmesPath is not as powerful as jq, but Amazon AWS probably chose it since it might be faster and the query-selectors are a bit more sophisticated (?).

AWS sync is not reliable!

While migrating from s3cmd to the AWS S3 CLI, I noticed that files did not reliably sync when using the AWS CLI.

I tested this behavior with different versions, and they all exhibited the same issue:

Test Setup

  1. Set up the AWS CLI utility and configure your credentials.

  2. Create a testing S3 bucket.

  3. Set up some random files:

    # Create 10 random files of 10MB each
for i in {1..10}; do dd if=/dev/urandom of=multi/part-$i.out bs=1MB count=10; done;
# Then copy the first 5 files over
mkdir multi-changed
cp -r multi/part-{1,2,3,4,5}.out multi-changed
# And replace the content in the remaining 5 files (6-10)
for i in {6..10}; do dd if=/dev/urandom of=multi-changed/part-$i.out bs=1MB count=10; done;

Testing S3 sync with AWS CLI

Cleanup

$ aws s3 rm s3://l3testing/multi --recursive

Inital sync

$ aws s3 sync multi s3://l3testing/multi
upload: multi/part-1.out to s3://l3testing/multi/part-1.out       
upload: multi/part-3.out to s3://l3testing/multi/part-3.out     
upload: multi/part-2.out to s3://l3testing/multi/part-2.out     
upload: multi/part-4.out to s3://l3testing/multi/part-4.out     
upload: multi/part-10.out to s3://l3testing/multi/part-10.out   
upload: multi/part-5.out to s3://l3testing/multi/part-5.out     
upload: multi/part-6.out to s3://l3testing/multi/part-6.out     
upload: multi/part-8.out to s3://l3testing/multi/part-8.out     
upload: multi/part-7.out to s3://l3testing/multi/part-7.out     
upload: multi/part-9.out to s3://l3testing/multi/part-9.out

Update files

Only 5 files should now be uploaded. Timestamps for all 10 files should be changed.

Infojunk September 2018

This is a collection of interesting links and resources I came across in September 2018, covering topics like Kotlin, Python, Markdown, note-taking apps, web development, Linux, and more.

Kotlin

Python

Markdown Notetaking

Some notetaking apps you should give a try. At least Notion is very promising (yet you have to pay).

Infojunk August 2018

This is a collection of interesting links and resources I came across in August 2018, covering topics like Linux, Apache, hardware, coding, Python, and math.

Linux

It’s about responsiveness - not about the best performance!

Apache

Hardware

Coding

Python

Yes, a deep dive into Python. And I don’t like it. As well as PHP. Do Rust. Trust me!

Albert Launcher 0.14: Switch Application Window Plugin

This post introduces a Python extension for the Albert Launcher that allows you to switch between application windows.

Since I really don’t like the Switcher Plugin for GNOME and I’m a fan of Albert Launcher, I created this extension. Maybe it will get accepted into their python extensions.

Drop the following code to ~/.local/share/albert/org.albert.extension.python/switch-app-window.py or other provided locations and activate it within the extensions menu:

Albert Extension Menu
import re
import subprocess

from albertv0 import *

__iid__ = "PythonInterface/v0.1"
__prettyname__ = "Switch App Window"
__version__ = "1.0"
__trigger__ = "w "
__author__ = "Markus Geiger <mg@evolution515.net>"
__id__ = "window"
__dependencies__ = []

iconPath = iconLookup("go-next")

def handleQuery(query):
    stripped = query.string.strip()
    if not query.isTriggered and not stripped:
        return

    results = []
    process = subprocess.Popen(['wmctrl', '-l'], stdout=subprocess.PIPE, encoding='utf8')

    output, error = process.communicate()

    patt = re.compile(r'^(\w+)\s+(\d+)\s+([^\s]+)\s+(.+)$')
    window_re = re.compile(r'^(.+)\s+-\s+(.+)$')

    for line in output.split('\n'):
        match = patt.match(line)
        if not match:
            continue

        window_id = match.group(1)
        fulltitle = match.group(4)
        if not query.string.lower() in fulltitle.lower():
            continue

        titlematch = window_re.match(fulltitle)

        if titlematch:
            windowtitle = titlematch.group(1)
            program_title = titlematch.group(2)
        else:
            program_title = fulltitle
            windowtitle = fulltitle

        results.append(
            Item(
                id="%s_%s" % (__id__, window_id),
                icon=iconPath,
                text=program_title,
                subtext=windowtitle,
                completion=query.rawString,
                actions=[
                    ProcAction("Focus", ["wmctrl", "-ia", window_id]),
                    ProcAction("Close", ["wmctrl", "-ic", window_id])
                ]
            )
        )
    return results

Update Confluence Page by API

You can create you own API token here: https://id.atlassian.com/manage/api-tokens and live-update any information you want. The script basicaly creates a HTML file, pumps it by JQ into a JSON-file and uploads it.

#!/bin/bash
# Update Confluence page by API

# Strict mode
set -euo pipefail

# Some informations
PAGEID=602767382
SPACE="EL3"
AUTH="user@example.com:GETYOUROWNTOKENORNEVERKNOW"
API_URL="https://mycompany.atlassian.net/wiki/rest/api"

# Create temp dir
TMP=$( mktemp -d )

# Shutdown handler
shutdown() {
# Cleanup temp directory
if [ -e "$TMP" ]; then
rm -fr "$TMP"
fi
}
trap shutdown TERM EXIT

# We first need current page version for update with next-page version
curl --silent --user ${AUTH} ${API_URL}/content/${PAGEID} &gt; ${TMP}/current.json
VERSION=$( cat ${TMP}/current.json | jq '.version.number' )
NEXTVERSION=$( expr 1 + ${VERSION} )
echo Got Version: ${VERSION}

# Get information
create page.txt

# Create HTML file
echo "

Date of creation: $( date --utc )
&lt;pre&gt;$( cat ${TMP}/page.txt | sed 's/$/&lt;br\&gt;/g' | tr -d '\n' )&lt;/br\&gt;&lt;/pre&gt;
" &gt; ${TMP}/page.html

# Prepare upload JSON with JQ
cat ${TMP}/page.html | jq -sR "@text | {\"id\":\"$PAGEID\",\"type\":\"page\",\"title\":\"Information Gathering\",\"space\":{\"key\":\"${SPACE}\"},\"body\":{\"storage\":{\"value\": . ,\"representation\":\"storage\"}},\"version\":{\"number\":${NEXTVERSION}}}"  &gt; ${TMP}/upload.json

# Upload
curl \
--silent \
--user ${AUTH} \
-X PUT -H 'Content-Type: application/json' \
-T ${TMP}/upload.json \
${API_URL}/content/${PAGEID} \
1&gt;/dev/null

echo Updated Version: ${NEXTVERSION}