≡

Cli

CSV tools

Here are some CSV utilities that may come in handy since most teams deal with this format.

Pure BASH interactive CLI/TUI menu (single-/multi-select/checkboxes)

This post presents a first version of a pure bash script for creating interactive command-line menus with single-select, multi-select, and checkbox functionality.

First version. To be refactored.

Asciinema recording

Gist Link

Inspired by

Notes

Permission to copy and modify is granted under the Creative Commons Attribution 4.0 license.

PHP: just in case posix_isatty() is missing

This article provides a couple of PHP code snippets from StackOverflow that can be used to determine if a PHP script is running in an interactive terminal (TTY). These are useful workarounds for environments where the posix_isatty() function is not available.

Short function

function is_a_tty()
{
    static $result;
    if (is_null($result)) {
        $fp = fopen('php://stdin', 'r');
        $stat = fstat($fp);
        $mode = $stat['mode'] & 0170000; // S_IFMT
        $result = $mode == 0020000; // S_IFCHR
        fclose($fp);
    }
    return $result;
}

Info Class from StackOverflow by leigh

<?php

class IOMode
{
    public $stdin;
    public $stdout;
    public $stderr;

    private function getMode(&$dev, $fp)
    {
        $stat = fstat($fp);
        $mode = $stat['mode'] & 0170000; // S_IFMT

        $dev = new StdClass;

        $dev->isFifo = $mode == 0010000; // S_IFIFO
        $dev->isChr  = $mode == 0020000; // S_IFCHR
        $dev->isDir  = $mode == 0040000; // S_IFDIR
        $dev->isBlk  = $mode == 0060000; // S_IFBLK
        $dev->isReg  = $mode == 0100000; // S_IFREG
        $dev->isLnk  = $mode == 0120000; // S_IFLNK
        $dev->isSock = $mode == 0140000; // S_IFSOCK
    }

    public function __construct()
    {
        $this->getMode($this->stdin, fopen('php://stdin', 'r'));
        $this->getMode($this->stdout, fopen('php://stdout', 'w'));
        $this->getMode($this->stderr, fopen('php://stderr', 'w'));
    }
}

$io = new IOMode;

Input

$ php io.php
// Character device as input
// $io->stdin->isChr  == true

$ echo | php io.php
// Input piped from another command
// $io->stdin->isFifo == true

$ php io.php < infile
// Input from a regular file (name taken verbatim from C headers)
// $io->stdin->isReg  == true

$ mkdir test
$ php io.php < test
// Directory used as input
// $io->stdin->isDir  == true
Output:

$ php io.php
// $io->stdout->isChr  == true

$ php io.php | cat
// $io->stdout->isFifo == true

$ php io.php > outfile
// $io->stdout->isReg  == true
Error:

$ php io.php
// $io->stderr->isChr  == true

$ php io.php 2>&1 | cat
// stderr redirected to stdout AND piped to another command
// $io->stderr->isFifo == true

$ php io.php 2>error
// $io->stderr->isReg  == true

I hope this PHP stuff finally dies.

bash: shell table output to json

This post presents a Python script that converts tabular command-line output into a more versatile JSON format. This allows for easier data manipulation using tools like jq, as an alternative to complex text-processing pipelines in bash.

You know that sometimes it would be really great to format a shell output to a more versatile format like JSON or YAML that you can process with jq instead of writing long pipes with text-processing.

AWS S3 Sync is Not Reliable and Slow!

This article explores reliability issues with AWS CLI’s S3 sync functionality and provides alternative solutions for better file synchronization.

While migrating from s3cmd to AWS S3 CLI, I noticed that files don’t sync properly when using AWS CLI.

I tested with different versions and they all revealed the same behavior:

Test Setup

  1. Setup AWS CLI utility and configure your credentials
  2. Create a testing S3 bucket
  3. Setup some random files
# Create 10 random files of 10MB each
for i in {1..10}; do dd if=/dev/urandom of=multi/part-$i.out bs=1MB count=10; done;
# Then copy the first 5 files over
mkdir multi-changed
cp -r multi/part-{1,2,3,4,5}.out multi-changed
# And replace the content in 5 files
for i in {6..10}; do dd if=/dev/urandom of=multi-changed/part-$i.out bs=1MB count=10; done;

Testing S3 Sync with AWS CLI

Cleanup

$ aws s3 rm s3://testbucket/multi --recursive

Initial Sync

$ aws s3 sync multi s3://testbucket/multi
upload: multi/part-1.out to s3://testbucket/multi/part-1.out         
upload: multi/part-3.out to s3://testbucket/multi/part-3.out      
upload: multi/part-2.out to s3://testbucket/multi/part-2.out      
upload: multi/part-4.out to s3://testbucket/multi/part-4.out      
upload: multi/part-10.out to s3://testbucket/multi/part-10.out    
upload: multi/part-5.out to s3://testbucket/multi/part-5.out      
upload: multi/part-6.out to s3://testbucket/multi/part-6.out      
upload: multi/part-8.out to s3://testbucket/multi/part-8.out      
upload: multi/part-7.out to s3://testbucket/multi/part-7.out      
upload: multi/part-9.out to s3://testbucket/multi/part-9.out

Update Files

Only 5 files should now be uploaded. Timestamps for all 10 files should be changed.

Away with grep! Use ripgrep!

Ripgrep is a tool that recursively searches your current directory for a regex pattern. This article provides a brief overview of ripgrep and why it is a good alternative to grep.

I used ack for some time for grepping source code, yet I moved on to ripgrep since it offers quite a bit more than grep and source-code-greps like The Silver Searcher. Plus it can mostly replace grep and is about 6 times faster (it’s written in Rust). I am still astonished when I get results instantly.