git: encrypt credentials within repository

Especially in the microservices era you should use a config server and never store your credentials in repository!

You should not use git smudge/clean filters for encryption. Why? Example!

Let’s create a example repository

% TMP=$(mktemp -d)
% cd $TMP
% git init
% echo 'Hello world!' > credentials

Add .gitattributes

/credentials filter=crypto

Add .git/config

[filter "crypto"]
smudge = openssl enc -aes-256-cbc -salt 
clean = openssl enc -aes-256-cbc -salt 

Note: require indicates that these commands need exit code 0 otherwise it could happen that these files are added though without any encryption. You can test this by using smudge = gpg -d -q โ€“batch โ€“no-tty -r <SIGNATURE> and clean = gpg -ea -q โ€“batch -no-tty -r <SIGNATURE> filters.

Add file to to stage and commit

% git add .gitattributes credentials
% git add credentials
enter aes-256-cbc encryption password:
Verifying - enter aes-256-cbc encryption password:
% git status
Changes to be committed:
    new file:   .gitattributes
    new file:   credentials
% git commit -m "Inital commit"                                                               [master (Basis-Commit) 860cedc] Initial commit
 2 files changed, 2 insertions(+)
 create mode 100644 .gitattributes
 create mode 100644 credentials

Note: The .git/config for filters will not be added to the commit!

Clone the example repository

% TMP2=$(mktemp -d)
% git clone $TMP .
% cat credentials

We now have an encrypted file without our repository. To decode it we need the clean command. On the other side anyone still can commit this file and break configuration.


  • Yes, you can automate credentials encoding or decoding with filters.
  • But you could also encode or decode manualy and check-in or -out. This makes not much difference!
  • GitLab provides file locks. This probably makes sense as long as you use GitLab premium. Otherwise you might use git commit hooks.
  • If you wanna use git repos to store credentuals and have them fully crypted you may have a look at [Git Annnex GCrypt]( Yet this is not based on git but a own project written in Haskell! It just uses the git-format!
  • You’re doomed if you store your production credentials in clear text so every developer can read it.
  • Credentials should never be stored in repositories!
  • Use config servers and tools like consul for creating configs or directly pull the configs from the servers!

So, why all that circumstances?! This easily can get messed up. This is just a badly designed ยปsolutionยซ . Therefore I would call it only a ยปworkaroundยซ and no real solution!

Docker on Windows: CIFS v3.02 mounts failing with big file count

Oh, well I love Docker and Windows โ€“ NOT! Another issue:


We use containers for our developer environments. The projects are built with tools shipped within the containers. While that works like a charm for OSX and Linux, we face problems with some containers on Windows which have a heavy load of source files. The project directory is mounted to the container by CIFS 3.02.

As workaround we have found out, that builds do work with CIFS 2.0, but not with 2.1 or 3.02 mounts. If the build is failing it complains about “File not found”. Yet the file is there and can be read!

I am certain the bug did not occur with Docker for Windows 1.13.0 (2017-01-19), yet already 1.12.1 (2016-09-16) introduced CIFS version 3.02. Maybe it is CIFS kernel module related or relies on communciation with windows CIFS server? I guess LinuxKit v0.4 or Microsoft’s Red Stone update introduced the problem.

Expected behavior

On long running builds files are accessible and and build does not fail since a file-not-found or similar is encountered.

Actual behavior

Build fails since it encounters ‘file-not-found-error’. Yet, the file can be read in the terminal by cat or other tools.


Microsoft Windows [Version 10.0.17134.165]
Version 18.06.0-ce-win70 (19075)
Channel: stable

Running Linux Containers only.

Steps to reproduce the behavior

0. Setup

I tried following test setup. Please replace CIFS vers=2.0 with 3.02 or 2.1 and you’ll see it fails. These commands are run by Windows WSL Ubuntu18.04 with DOCKER_HOST=tcp://localhost:2375.

1. Build test image

docker build -t dockerbug .

2. Create test volumes with CIFS 2.0 and 3.02 protocol.

Please adjust host, username, password and domain to your Windows environment so the shares can be mounted.

The problem is NOT symlink related. I also tried switching mount options like mfsymlinks and so on.

docker volume create \
   --driver local \
   --opt type=cifs \
   --opt device=// \
   --opt o=username=Markus,password=XXXXXXXX,relatime,vers=3.02,sec=ntlmsspi,cache=strict,username=Markus,domain=REFDESK-01,uid=0,noforceuid,gid=0,noforcegid,addr=,file_mode=0777,dir_mode=0777,iocharset=utf8,nounix,serverino,mapposix,mfsymlinks,noperm,rsize=1048576,wsize=1048576,echo_interval=60,actimeo=1,nobrl \
docker volume create \
   --driver local \
   --opt type=cifs \
   --opt device=// \
   --opt o=username=Markus,password=XXXXXXXX,relatime,vers=2.0,sec=ntlmsspi,cache=strict,username=Markus,domain=REFDESK-01,uid=0,noforceuid,gid=0,noforcegid,addr=,file_mode=0777,dir_mode=0777,iocharset=utf8,nounix,serverino,mapposix,mfsymlinks,noperm,rsize=1048576,wsize=1048576,echo_interval=60,actimeo=1,nobrl \

3. Try a containered npm install

Attention! This may take a while to run! It also may rollback some files. This is not the error!

Failing with CIFS 3.02

docker run -it --rm -v testcifs3.02:/mnt/c --workdir $(pwd) dockerbug /bin/sh -c '$(getent passwd $(whoami) | cut -d: -f7)'
bash-4.2# mount | grep cifs
// on /mnt type cifs (rw,relatime,vers=3.02,sec=ntlmsspi,cache=strict,username=Markus,domain=REFDESK-01,uid=0,noforceuid,gid=0,noforcegid,addr=,file_mode=0777,dir_mode=0777,iocharset=utf8,nounix,serverino,mapposix,nobrl,mfsymlinks,noperm,rsize=1048576,wsize=1048576,echo_interval=60,actimeo=1)
npm install node-sass
Error: EINVAL: invalid argument, open '/mnt/c/Users/Markus/Desktop/docker-containered/node_modules/node-sass/package.json'
    at Error (native)
    at Object.fs.openSync (fs.js:642:18)
    at Object.fs.readFileSync (fs.js:510:33)
    at Object.Module._extensions..json (module.js:592:20)
    at Module.load (module.js:494:32)
    at tryModuleLoad (module.js:453:12)
    at Function.Module._load (module.js:445:3)
    at Module.require (module.js:504:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (/mnt/c/Users/Markus/Desktop/docker-containered/node_modules/node-sass/lib/extensions.js:7:9)
npm ERR! errno 1
npm ERR! node-sass@4.9.2 install: `node scripts/install.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the node-sass@4.9.2 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2018-07-26T13_49_04_149Z-debug.log

exit code 1
33833 verbose stack Error: node-sass@4.9.0 install: `node scripts/install.js`
33833 verbose stack Exit status 1
33833 verbose stack     at EventEmitter.<anonymous> (/usr/local/share/.config/yarn/global/node_modules/npm/node_modules/npm-lifecycle/index.js:304:16)
33833 verbose stack     at emitTwo (events.js:106:13)
33833 verbose stack     at EventEmitter.emit (events.js:191:7)
33833 verbose stack     at ChildProcess.<anonymous> (/usr/local/share/.config/yarn/global/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
33833 verbose stack     at emitTwo (events.js:106:13)
33833 verbose stack     at ChildProcess.emit (events.js:191:7)
33833 verbose stack     at maybeClose (internal/child_process.js:920:16)
33833 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:230:5)
33834 verbose pkgid node-sass@4.9.0
33835 verbose cwd /mnt/c/Users/Markus/Projects/bugreport
33836 verbose Linux 4.9.93-linuxkit-aufs
33837 verbose argv "/usr/bin/node" "/usr/local/bin/npm" "install"
33838 verbose node v6.14.3
33839 verbose npm  v6.2.0
33840 error code ELIFECYCLE
33841 error errno 1
33842 error node-sass@4.9.0 install: `node scripts/install.js`
33842 error Exit status 1
33843 error Failed at the node-sass@4.9.0 install script.
33843 error This is probably not a problem with npm. There is likely additional logging output above.
33844 verbose exit [ 1, true ]

From a run within a different container


[ 2828.299719] 0000 4800 53fe 424d 0040 0001 0034 c000  ...H.SMB@...4...
[ 2828.299720] 0005 0001 0009 0000 0000 0000 b470 0001  ............p...
[ 2828.299721] 0000 0000 4238 0000 0001 0000 0015 0000  ....8B..........
[ 2828.299723] 4c00 0000 3377 4dfd 4a86 50e0 64f8 2bbe  .L..w3.M.J.P.d.+
[ 2828.299723] 1d92 7f70 0009 0000 0000 0000            ..p.........
[ 2828.299757] 0000 4800 53fe 424d 0040 0001 0034 c000  ...H.SMB@...4...
[ 2828.299759] 0005 0001 0009 0000 0000 0000 b470 0001  ............p...
[ 2828.299760] 0000 0000 4238 0000 0001 0000 0015 0000  ....8B..........
[ 2828.299761] 4c00 0000 3377 4dfd 4a86 50e0 64f8 2bbe  .L..w3.M.J.P.d.+
[ 2828.299762] 1d92 7f70 0009 0000                      ..p.....
[ 2828.299766] Status code returned 0xc0000034 STATUS_OBJECT_NAME_NOT_FOUND

Successful with CIFS 2.0

docker run -it --rm -v testcifs2.0:/mnt/c --workdir $(pwd) dockerbug /bin/sh -c '$(getent passwd $(whoami) | cut -d: -f7)'
bash-4.2# mount | grep cifs
// on /mnt/c type cifs (rw,relatime,vers=2.0,sec=ntlmsspi,cache=strict,username=Markus,domain=REFDESK-01,uid=0,noforceuid,gid=0,noforcegid,addr=,file_mode=0777,dir_mode=0777,iocharset=utf8,nounix,serverino,mapposix,nobrl,mfsymlinks,noperm,rsize=65536,wsize=65536,echo_interval=60,actimeo=1)
npm install
+ node-sass@4.9.2
added 199 packages from 134 contributors and audited 531 packages in 45.896s
found 4 moderate severity vulnerabilities
  run `npm audit fix` to fix them, or `npm audit` for details

real    0m46.728s
user    0m13.020s
sys 0m6.630s

exit code 0

Ubuntu Bionic: HD Graphics 520 i915 configuration



GRUB_CMDLINE_LINUX_DEFAULT="noplymouth intel_pstate=skylake_hwp i915.enable_rc6=1 i915.enable_guc=3 i915.enable_fbc=1 i915.semaphores=1 nvme_load=YES intel_pstate=enable i915.enable_psr=1 i915.disable_power_well=0"
# GRUB_CMDLINE_LINUX="elevator=deadline"

# Uncomment to disable graphical terminal (grub-pc only)

# you can see them in real GRUB with the command `vbeinfo'
# GRUB_GFXMODE=1024x768x16


Section "Device"
Identifier  "Intel Graphics"
Driver      "intel"
Option      "DRI" "3"
Option      "HWRotation" "true"
Option      "Tiling" "true"
Option      "SwapBuffersWait" "0"
Option      "ReprobeOutputs" "true"
Option      "VSync" "true"
# Option      "TearFree" "true"
# Option      "AccelMode" "sna"
Section "dri"
Mode 0666

List supported kernel mod options

modinfo -p i915

To check which options are currently enabled, run

systool -m i915 -av

If editing /etc/modprobe.d/* don’t forget to udpate ramdisk! Or use kernel params in GRUB.

Kernel Housekeeper Update Script

I use that script for Kernel Housekeeping since I’m mostly on mainline kernel. The script is currently used with Ubuntu Bionic Beaver.


function version_gt() { test "$(echo "$@" | tr " " "\n" | sort -V | head -n 1)" != "$1"; }
function version_le() { test "$(echo "$@" | tr " " "\n" | sort -V | head -n 1)" == "$1"; }
function version_lt() { test "$(echo "$@" | tr " " "\n" | sort -rV | head -n 1)" != "$1"; }
function version_eq() { test "$(echo "$@" | tr " " "\n" | sort -rV | head -n 1)" == "$1"; }
# fi


DUMP="$( curl -s ";O=A" )"
if [ "$?" -eq "0" ] &amp;&amp; [ -n "$DUMP" ]; then
LATEST_KERNEL_VERSION="$( echo "$DUMP" | sed 's/&lt;\/*[^&gt;]*&gt;//g' | grep -E -o "^v[^/]+" |  sort -V  | grep -v '\-rc' | tail -1 | sed -r 's/^v//g' )"
LATEST_KERNEL_VERSION_SHORT="$( echo $LATEST_KERNEL_VERSION | tr '.-' '.' | cut -d. -f1-3 )"
LATEST_KERNEL_VERSION="Unable to resolve"

# Test if we have a generic kernel version
GENERIC_KERNEL_VERSION=$( dpkg-query -W -f'${db:Status-Abbrev} ${Package} ${Version}\n'  | grep -e '^ii' | grep 'linux-image-generic' | awk '{ print $3 }'  )
echo "Generic Kernel Version"
echo "======================"
if [ -n "$GENERIC_KERNEL_VERSION" ]; then
GENERIC_KERNEL_VERSION_SHORT="$( echo $GENERIC_KERNEL_VERSION | tr '.-' '.' | cut -d. -f1-3 )"
echo "Not installed".

CURRENT_KERNEL_VERSION=$( uname -r | cut -d- -f1,2 )
CURRENT_KERNEL_VERSION_SHORT="$( echo $CURRENT_KERNEL_VERSION | tr '.-' '.' | cut -d. -f1-3 )"
echo Current Kernel
echo ==============

echo "Latest Kernel (stable)"
echo "======================"


# TODO add /boot/efi support
echo "Partitions "
echo ==========
BOOT_PARTITIONS="$( df -h --output=target | grep "/boot" | tr '\n' ':' )"
if [ -z "$BOOT_PARTITIONS" ]; then
echo "No special partitions."
for boot_partition in $BOOT_PARTITIONS; do
boot_part_total=$( df $boot_partition -h --output=size | tail -n 1 | tr -d '[:space:]' )
boot_part_avail=$( df $boot_partition -h --output=avail | tail -n 1 | tr -d '[:space:]' )
echo "$boot_partition size=$boot_part_total, free=$boot_part_avail"
# Installed kernels

if [ ! "$( echo $boot_partition | grep efi &gt; /dev/null 2&gt;&amp;1; echo $? )" -eq 0 ]; then
INSTALLED="$( test -r ${boot_partition} &amp;&amp; ls -tr ${boot_partition}/{vmlinuz,initrd.img}-* | cut -d- -f2 | sort | uniq | tr '\n' ':'  )"
for version in $INSTALLED; do
echo -n " $version taking "
SIZE=$( cd $boot_partition; ls -1  | grep "\-${version}-" | xargs du -shc | tail -1 | cut -f 1 )
echo $SIZE

echo "Installed packages"
echo "=================="

ALL_INSTALLED_PACKAGES=$( dpkg-query -W -f'${db:Status-Abbrev} ${Package} ${Version}\n'  | grep -e '^ii' | grep -e 'linux-image\|linux-signed-image\|linux-headers' | awk '{print $2}' | sort -V  )

echo "Removeable packages"
echo "==================="

# config files
REMOVEABLE_PACKAGES="$( dpkg-query -W -f'${db:Status-Abbrev} ${Package} ${Version}\n'  | grep -e '^rc' | grep -e 'linux-image\|linux-signed-image\|linux-headers' | awk '{print $2}' | sort | grep -v -E "$IGNORE_PACKAGES" )"

for tag in "linux-image" "linux-headers" "linux-signed-image" "linux-image-extra"; do
# echo c="$tag-$CURRENT_KERNEL_VERSION-";
packages_to_be_removed="$( echo $ALL_INSTALLED_PACKAGES | grep $tag | sort -V | awk 'index($0,c){exit} //' c="$tag-$CURRENT_KERNEL_VERSION" | grep -v -E "$IGNORE_PACKAGES" )"
REMOVEABLE_PACKAGES="$( echo -e "$REMOVEABLE_PACKAGES\n$packages_to_be_removed" | sed '/^$/d' | sort -V | uniq )"

if ! [ $(id -u) = 0 ]; then
echo "You need to be root! Aborting..."
exit 1

if [ -z "$REMOVEABLE_PACKAGES" ]; then
echo "No packages to remove found!"
read -p "Remove (y/n)?" CHOICE

case "$CHOICE" in
y|Y )
CMD="apt-get remove --purge $( echo $REMOVEABLE_PACKAGES | tr '\n' ' ' )"
# echo $CMD
eval "$CMD"
apt-get autoremove
* ) ;;

# Install packages from latest kernel
if [ $HAS_INTERNET == 1 ]; then

echo "Kernel upgrade"
echo "=============="

if [ -n "$LATEST_INSTALLED" ]; then
echo "$LATEST_KERNEL_VERSION already installed. No need to update."
PACKAGES="$( curl -s "$LATEST_KERNEL_VERSION_SHORT/" |  grep -o -E 'linux-[_[:alnum:]\.\-]+.deb' | sort | uniq )"
# echo $PACKAGES
MACHINE_TYPE=`uname -m`
if [ ${MACHINE_TYPE} == 'x86_64' ]; then
INSTALL_PACKAGES="$( echo $PACKAGES | grep -E "_($ARCH|all)" |  grep -v lowlatency )"
if [ -z "$INSTALL_PACKAGES" ]; then
echo "No packages to install found. Check your script for URL errors!" 1&gt;&amp;2
exit 1

echo "Following packages will be installed:"
read -p "Continue (y/n)?" CHOICE

# exit
TMP=$( mktemp -d --suffix=.kernel-$LATEST_KERNEL_VERSION )
cd $TMP

for package in $INSTALL_PACKAGES; do
echo -n Downloading $package...
wget  -q "$url"
if [ "$?" -eq 0 ]; then
echo done.
echo failed. aborting.
exit 1
if [ -n "$INSTALL_PACKAGES" ]; then
dpkg -i *.deb

Away with grep! Use ripgrep!

I used ack for some time now grepping source code, yet I moved on to ripgrep since offers quiet more than than Grep und SourceCode-greps like SilverSearcher and Co. Plus it can mostly replace Grep and is mostly about 6x times faster (written In Rust). I am still astonished when I really get results instantly.

See here:

This should be really installed on every server where you have to deal with source code.

Update Confluence Page by API

You can create you own API token here: and live-update any information you want. The script basicaly creates a HTML file, pumps it by JQ into a JSON-file and uploads it.

# Update Confluence page by API

# Strict mode
set -euo pipefail

# Some informations

# Create temp dir
TMP=$( mktemp -d )

# Shutdown handler
shutdown() {
# Cleanup temp directory
if [ -e "$TMP" ]; then
rm -fr "$TMP"
trap shutdown TERM EXIT

# We first need current page version for update with next-page version
curl --silent --user ${AUTH} ${API_URL}/content/${PAGEID} > ${TMP}/current.json
VERSION=$( cat ${TMP}/current.json | jq '.version.number' )
NEXTVERSION=$( expr 1 + ${VERSION} )
echo Got Version: ${VERSION}

# Get information
create page.txt

# Create HTML file
echo "

Date of creation: $( date --utc )
<pre>$( cat ${TMP}/page.txt | sed 's/$/<br\>/g' | tr -d '\n' )</br\></pre>
" > ${TMP}/page.html

# Prepare upload JSON with JQ
cat ${TMP}/page.html | jq -sR "@text | {\"id\":\"$PAGEID\",\"type\":\"page\",\"title\":\"Information Gathering\",\"space\":{\"key\":\"${SPACE}\"},\"body\":{\"storage\":{\"value\": . ,\"representation\":\"storage\"}},\"version\":{\"number\":${NEXTVERSION}}}"  > ${TMP}/upload.json

# Upload
curl \
--silent \
--user ${AUTH} \
-X PUT -H 'Content-Type: application/json' \
-T ${TMP}/upload.json \
${API_URL}/content/${PAGEID} \

echo Updated Version: ${NEXTVERSION}

IP in VPN vs. LAN: alias IP address by iptables

Sceneria: while your are at work you are on LAN and you use 192.168.x.x. But once you do home office you connect by VPN to the same DB and the IP changes to 10.x.x.x. And you don’t wanna change configs for your app ๐Ÿ™

Using IP tables that can be worked around easily:

# Enable IP forwarding
sudo sh -c 'echo "1" > /proc/sys/net/ipv4/ip_forward'



ping -c 1 -W 1 $IP_VIRTUAL                                                 
PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=63 time=124 ms

ping -c 1 -W 1 $IP_LAN                                                          PING ( 56(84) bytes of data.

--- ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

sudo iptables -t nat -A PREROUTING -d $IP_LAN -j DNAT --to-destination $IP_VIRTUAL
sudo iptables -t nat -A POSTROUTING -j MASQUERADE

ping -c 1 -W 1 $IP_LAN
PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=63 time=125 ms

Laptop Perfomance: irqbalancer vs. intel_pstate

Today I uninstalled irqbalancer and I had some more performance gain on my GNOME desktop.

The CPUfreq control panel showed me IRQBALANCE DETECTED and they say:

Why I should not use a single core for power saving

  • Moderns OS/kernel work better on multi-core architectures.
  • You need at least 1 core for a foreground application and 1 for background system services.
  • Linux Kernel switches between CPU cores to avoid overheating, CPU thermal throttling and to balance system load.
  • Many CPUs have Hyper-Threading (HT) technology enabled by default. So there is no reason to run half of a physical CPU core.

Very simply said. I miss some contradictions.

I’m not certain how laptop drains battery running without intel_pstate=skylake_hwp set to disable. The pstate does make sense to me. Mainly I think it was irqbalancer. I have do do some stress testing laterโ€ฆ

Infojunk July 2018

Better than VNC and TeamViewer – NoMachine and the NX protocol

The NX protocol is basicaly a successor to the X protocol and very nyce for Streaming. NoMachine implements that and is a better VNC with video streaming and a nice alternative to TeamViewer with good cross-platform capabilities. Also keyboard and mouse works cross-plattform without any problems. This is an issue with most alternatives to TeamViewer. Yet TeamViewer still is number #1 for me.


In 2001, the compression and transport protocol NX was created to improve on the performance of the native X display protocol to the point that it could be usable over a slow link such as a dial-up modem. It wrapped remote connections inย SSHย sessions for encryption.

The NX scheme was derived from that of DXPC โ€“ the Differential X Protocol Compressor project. NX 1.x was released to the general public on February 14, 2003, the final version of ‘NX’ being 3.5. The last update of 3.5 was in 2012.

The core compression technology up until NX 3.5 was made available to the community under the GNUย GPL2ย license whilst other components such as the NX Server and NX Client programs were proprietary.

In 2009, Google made a freely available Open Source GPL2 version of the server called Neatx. Other open source variants of NoMachine’s NX are also available (see below).

Starting in 2013, with the release of version 4.0, NX technology became closed source.