Useful command line snippets

Fri 4 May 2018
Mon 20 Apr 2020

This is a collection of great commands to try and might prove usefull in some situations. This is a ever growing page as I will keep updating this when finding new great command.

Base64 Encoding

The Base64 encoding has friendly characters, for this reason, it is very versatile and useful. In a number of situations, mainly in transport or storage, this will be a great way for compatibility between systems. While I don't have any serious cases I want to enlight I do have 2 snippets that are worthwhile and easy to remember.

# Encoding piped string
rob@Rathalos ~ $ cat myexamplefile.txt | base64

# Decoding piped string
rob@Rathalos ~ $ cat mybase64encodedfile.txt | base64 -d

Dig

A very powerful CLI DNS tool is dig. You can test everything related to DNS with it. Here are a few commands to get you going with this tool.

# Query a domain
rob@Rathalos ~ $ dig A google.com
# Get a short answer back
rob@Rathalos ~ $ dig A google.com +short

# Get the PTR record
rob@Rathalos ~ $ dig -x 8.8.8.8

# Get DNS from a specific DNS server
rob@Rathalos ~ $ dig @1.1.1.1 A google.com
# Trace the request
rob@Rathalos ~ $ dig @1.1.1.1 A google.com +trace
# Lookup SOA from domain
rob@Rathalos ~ $ dig google.com +nssearch

Generate passwords

I've used a script that I wrote in Python for years to generate my passwords. We need strong and random enough passwords to secure accounts every day. This command will run on virtually any distro without the need for programs.

You want to strengthen your password even more? Sure, try the second command! I always try to get my password as strong as possible. In this day and age it suprises me how many services limit the amount and range of characters. Not to mention password policies that don't make any sense and actually make your password less secure.

# date +%s           displays the current date in POSIX time format
# sha256             gets the checksum of this string
# base64             converts this to a base64 string
# head -c 64         gets the first 64 bytes of the piped content
# echo               displays the manipulated string
rob@Rathalos ~ $ date +%s | sha256sum | base64 | head -c 64 ; echo
# tr -cd             will act as a filter for what characters we want to allow
# < /dev/urandom     will echo the contents of the file to stdin
# head -c 64         gets the first 64 bytes of the piped content
# echo               displays the manipulated string
rob@Rathalos ~ $ tr -cd [:graph:] < /dev/urandom | head -c 64 ; echo

Gource

Gource is a cool way to visualize your Git commit history. What better way to show off cool work like this?

# Start a gource that automatically skips to the next commit when nothing happens and 1 day = 1 second
rob@Rathalos ~ $ gource -a 1 -s 1

GPG / Paperkey

GNU Private Guard. The standard encryption tool in the open software world. It's great. It's fantastic, and here are a few snippet that might help you out.

# Create a key
rob@Rathalos ~ $ gpg --gen-full-key 
# List keys
rob@Rathalos ~ $ gpg --list-keys
# Export secret key
rob@Rathalos ~ $ gpg --export-secret-keys B54K853A > my-private-key.asc
# Export public key
rob@Rathalos ~ $ gpg --armor --export B54K853A > my-public-key.asc
# Do a basic encrypt
rob@Rathalos ~ $ gpg -e <file>
# Do a basic decrypt
rob@Rathalos ~ $ gpg -d <file>
# Import a keyfile
rob@Rathalos ~ $ gpg --import <file>
# Modify a key
rob@Rathalos ~ $ gpg --edit-key B54K853A

# -- 
# -- From here we are entering the GPG tool!
# --

# Trust the selected key for edit
gpg> trust
# Add a new uid to the selected key for edit
gpg> adduid
# Select one of the uid's
gpg> uid <id>
# Delete one of the uid's
# Note: You would logically only use this if the key is not distributed yet
gpg> deluid <id>
# Revoke selected uid
# Note: Public keyserver only allow revoking of uid's
gpg> revuid
# Make selected uid primary
gpg> primary

File commands

File commands are always great to know. In the GNU/Linux universe you've got most of the following always under your fingertips in commandline interfaces.

# See size of files in current directory
rob@Rathalos ~ $ du -sh -- *
# See system-side usage of disk space
rob@Rathalos ~ $ df

# See all files with rights, including dots
rob@Rathalos ~ $ ls -al
# Create a new file
rob@Rathalos ~ $ touch testfile
# Empty file
rob@Rathalos ~ $ echo '' > testfile
# Show all non-printable characters in a file
rob@Rathalos ~ $ cat -A testfile

# Find files with regex in current dir
rob@Rathalos ~ $ find . -name "*-avatar.jpg" -type f
# First example, including subdirs
rob@Rathalos ~ $ find . -name "**-avatar.jpg" -type f
# Delete whatever you find
rob@Rathalos ~ $ find . -name "**-avatar.jpg" -type f -delete

# See the last 50 lines of log
rob@Rathalos ~ $ tail -n 50 /var/log/nginx
# Keep tail in foreground and give you realtime feedback
rob@Rathalos ~ $ tail -f /var/log/nginx

# Go to your Home folder (nothing after cd) :)
rob@Rathalos ~ $ cd

# Check sum of file to check integrity
# You get the point with algo's prefix
rob@Rathalos ~ $ sha1sum testfile
rob@Rathalos ~ $ sha512sum testfile
rob@Rathalos ~ $ md5sum testfile

Then there are file commands that change rights and ownership. They might be basic but very usefull! Also, be warned when using the -Rf flag. The R part means recursive. The f part means by force. It might seem obvious yet these commands, though simple, can ruin your day quickly :)

# Change rights of file
rob@Rathalos ~ $ chmod 0644 test
# Change rights of all files in current directory
rob@Rathalos ~ $ chmod 0644 *
# Change rights of all files in current and subdirectories
rob@Rathalos ~ $ chmod 0644 * -Rf

# Change ownership of file
rob@Rathalos ~ $ chown rob:rob test
# Change rights of all files in current directory
rob@Rathalos ~ $ chown rob:rob *
# Change rights of all files in current and subdirectories
rob@Rathalos ~ $ chown rob:rob * -Rf

SSH / SCP / Putty

SSH is a pre for anyone working with remote these days. Insecure connections are not acceptable anymore. Laws like GDPR ensure that IT companies take it up a notch. Here is a collection of handy commands to know.

Yes, there are also some putty commands when you need them. Filezilla for instance still needs a putty formated key instead of an OpenSSH key formatted ssh key.

# Login with SSH with an alternative port number
rob@Rathalos ~ $ ssh user@domain.tld -p {port-number}

# Add a quoted commando to login, run the command and close the session
rob@Rathalos ~ $ ssh user@domain.tld -p {port-number} 'ls -al'

# Copy a local file to the remote server with scp
rob@Rathalos ~ $ scp -P {port-number} {local-file} user@domain.tld:~/{destination-file}

# Copy a remote file to local with scp
rob@Rathalos ~ $ scp -P {port-number} user@domain.tld:~/{remote-file} {local-directory}

# Convert an existing OpenSSH key to the Putty format
rob@Rathalos ~ $ puttygen -t rsa -C ~./.ssh/id_rsa -o ~/.ssh/id_rsa.ppk

Stress Commands

Stressing systems can easily be done with stress or stress-ng. However here are also a few other handy commands that you can run with everyday programs.

# Utilize CPU
sha1sum /dev/zero

# Load bytes in memory (3GB in this case)
yes | tr \\n x | head -c $((1024*1024*3000)) | grep n

Systemctl

This tool manages all running services on modern systems. There are some handy commands to know when you want to check a service.

# Start|Stop|Status|Restart
rob@Rathalos ~ $ systemctl restart nginx.service

# Enable|Disable
rob@Rathalos ~ $ systemctl disable nginx.service
rob@Rathalos ~ $ systemctl enable nginx.service

# See whats inside the service file
rob@Rathalos ~ $ systemctl cat nginx.service
# Edit a service file
rob@Rathalos ~ $ systemctl edit --full nginx.service
# Do a daemon-reload for reloading changes
rob@Rathalos ~ $ systemctl daemon-reload

# List all active services
rob@Rathalos ~ $ systemctl list-units
# List all configured services
rob@Rathalos ~ $ systemctl list-units --all

User and groupadd + mod

Basic usermod commands to change certain aspects of the user on your system. Always use the programs in place to manipulate the user accounts the correct way.

# Add a new user
rob@Rathalos ~ $ useradd rob2
# Set the password for the new user
rob@Rathalos ~ $ passwd rob2

# Add a user and set the home folder different
rob@Rathalos ~ $ useradd -d /data/rob2home rob2
# Assign user to many groups at once during the add command
rob@Rathalos ~ $ useradd -G admins,docker,games rob2
# Move the home folder of an existing user
rob@Rathalos ~ $ usermod -m -d /data/rob2 rob2

# Add a user that is set to expire (YYYY-MM-DD)
rob@Rathalos ~ $ useradd -e 2020-12-05 rob2
# Verify the expiry information
rob@Rathalos ~ $ chage -l rob2
# Add a user and: 
# Set an expiry period of 10 days for the password (-f is expressed in days)
rob@Rathalos ~ $ useradd -e 2020-12-05 -f 10 rob2

# Assign a nologin shell to an existing user
rob@Rathalos ~ $ usermod -s /sbin/nologin rob2
# Change the shell to `bash` of an existing user
rob@Rathalos ~ $ usermod -s /bin/bash rob2

# Lock an existing user
rob@Rathalos ~ $ usermod -L rob2
# Unlock an existing user
rob@Rathalos ~ $ usermod -L rob2

# Change login name
rob@Rathalos ~ $ usermod -l somethingdifferent rob2

FFmpeg

I love FFmpeg as a cli tool. It is so versatile! No need for shady converter tools -- and Windows at that :wink:, those days are over!

# Let's start easy, get some info
rob@Rathalos ~ $ ffmpeg -i video.mp4
# Without configuration banner
rob@Rathalos ~ $ ffmpeg -i video.mp4 -hide_banner
# Removing audio from a video file
rob@Rathalos ~ $ ffmpeg -i input.mp4 -an output.mp4
# Removing video from a video file
rob@Rathalos ~ $ ffmpeg -i input.mp4 -vn output.mp3
# Now some conversion! Let's check the formats
rob@Rathalos ~ $ ffmpeg -formats
# For example Flash Video to MP4
rob@Rathalos ~ $ ffmpeg -i video.flv -vn output.mp4
# Compress file and make video codec h264
rob@Rathalos ~ $ ffmpeg -i input.mp4 -vcodec h264 -acodec aac output.mp4
# Speed up videos twice as fast
ffmpeg -i input.mp4 -filter:v "setpts=0.5*PTS" output.mp4
# Convert movies affected by the VFS (Vertical Filming Syndrome)
rob@Rathalos ~ $ ffmpeg -i compressed.mp4 -filter_complex '[0:v]scale=ih*16/9:-1,boxblur=luma_radius=min(h\,w)/20:luma_power=1:chroma_radius=min(cw\,ch)/20:chroma_power=1[bg];[bg][0:v]overlay=(W-w)/2:(H-h)/2,crop=h=iw*9/16' output.mp4
# Create a gif from a video file where
#   -ss is the starting point in seconds
#   -t how many seconds of video the gif should be
#   scale the resolution of the gif is
rob@Rathalos ~ $ ffmpeg -ss 10 -t 2 -i video.webm -vf "fps=10,scale=800:-1:flags=lanczos,split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse" -loop 0 sample.gif
# Use ffplay to preview video/audio files
rob@Rathalos ~ $ ffplay video.mp4

OpenSSL

# Check the certificate alternative names and other entries
rob@Rathalos ~ $ openssl x509 -in server.cert -text -noout
# Check a remote server certificate, handy for debugging remotes
rob@Rathalos ~ $ openssl s_client -connect google.com:443
# Show certificate details
rob@Rathalos ~ $ openssl s_client -connect google.com:443 | openssl x509 -noout -text
# Show only the Subject Alternative Names (SAN)
rob@Rathalos ~ $ openssl s_client -connect google.com:443 | openssl x509 -noout -text | grep DNS:

Nginx

Nginx is a modern webserver. These are common but good commands to know when debugging config.

# Let Nginx test the configuration
rob@Rathalos ~ $ nginx -it

Nmap

Nmap is a great program. It allows you to do many great things like looking up open ports, looking up devices based on MAC. Discovering devices that allow this. Nmap can tell you a great deal about a network in a few commands.

# Find raspberries on the network
rob@Rathalos ~ $ nmap -sP 192.168.1.0/24 | awk '/^Nmap/{ip=$NF}/B8:27:EB/{print ip}'

# Scan a single ip
rob@Rathalos ~ $ nmap 192.168.1.1
# Scan a range of ip's
rob@Rathalos ~ $ nmap 192.168.1.1-20
# Scan a subnet
rob@Rathalos ~ $ nmap 192.168.1.0/24

# Scan a single port
rob@Rathalos ~ $ nmap -p 22 192.168.1.1
# Scan a range of ports
rob@Rathalos ~ $ nmap -p 1-100 192.168.1.1
# Quickly scan common ports, i.e. 22, 443, 365 ect.
rob@Rathalos ~ $ nmap -F 192.168.1.1

# Scan a host
rob@Rathalos ~ $ nmap www.testhostname.com

QR Code Commands

I like the following few QR generating commando's. Why? They can run locally! I don't trust online tools to create my QR codes. QR Codes are a solid way of creating a physical medium to save any type of file. Store them in a vault or other safe place around the house as a means of last resort backup.

# Create a simple QR Code
rob@Rathalos ~ $ qrencode -o qr-anything.png '#spreaditlikebutter'

# Create a "Setup my network QR Code"
# This will prompt for the SSID and WPA password
rob@Rathalos ~ $ qrencode -s 7 -o qr-wifi.png "WIFI:S:$(zenity --entry --text="Network name (SSID)" --title="Create WiFi QR");T:WPA;P:$(zenity --password --title="Wifi Password");;"

(MY)SQL Snippets

Sometimes you just need to know a tiny bit of information. The only thing you've got is a commandline window. Here are some good queries you can easily execute via the commandline on a connection.

# Give back the collation of a schema
rob> SELECT * FROM information_schema.SCHEMATA WHERE schema_name = 'DB_NAME';
# Select all tables from a specific schema
rob> SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE = 'BASE TABLE' AND TABLE_SCHEMA = 'DB_NAME';

PHP

There are some handy commands when working with the binaries. Here are some that might help you in a pinch.

# Shows the current version, alternativly use long notation --version
rob@Rathalos ~ $ php -v
# Shows the current configuration
rob@Rathalos ~ $ php -i
# Shows the current version for extensions
rob@Rathalos ~ $ php -r 'foreach (get_loaded_extensions() as $extension) echo "$extension: " . phpversion($extension) . "\n";'
# Give PHP process more memory access, alternativly use short notation (2G)
rob@Rathalos ~ $ php -d  memory_limit=2048M 
# Open a interactive PHP shell session
rob@Rathalos ~ $ php -a

Lynx

The lightweight, commandline, text-based browser! It will help you in times of need. Here a small collection that might help you. You can also read my article about Lynx.

# Run in default mode
rob@Rathalos ~ $ lynx https://thoughtforcoding.com
# Accept all cookies if you trust the website
rob@Rathalos ~ $ lynx https://thoughtforcoding.com --accept-all-cookies
# Run in "secure mode" to ensure everything over SSL
rob@Rathalos ~ $ lynx https://thoughtforcoding.com --force-secure
# Use Realm to stay within the given domain to prevent outbound links
rob@Rathalos ~ $ lynx https://thoughtforcoding.com --realm
# Keep track of your session
rob@Rathalos ~ $ lynx https://thoughtforcoding.com --session=./lynx_session

Docker Commands

Docker is a very versatile that you can use for many projects. Here are a few commands that you can use.

# Check the running docker processes
rob@Rathalos ~ $ docker ps
# Check dockers internal resources monitor
rob@Rathalos ~ $ docker stats

# This will start a interactive shell session with an image
# The container will destroy itself after the session is done
rob@Rathalos ~ $ docker run -it -P busybox /bin/sh

# Check how the image gets build
rob@Rathalos ~ $ docker history busybox

# Checkout a lot of stats around a container
rob@Rathalos ~ $ docker inspect 

# Docker run daemonized
rob@Rathalos ~ $ docker run --name my-custom-image -d busybox
# Open specific ports
rob@Rathalos ~ $ docker run --name my-custom-image -p 30000:80 busybox
# Bind on ip -- replace with public ip :)
rob@Rathalos ~ $ docker run --name my-custom-image -p 127.0.0.1:30000:80 busybox

# Manually create a network
rob@Rathalos ~ $ docker network create network-name

# Manually connect two containers with an existing network
rob@Rathalos ~ $ docker network connect network-name my-own-image

# Making a backup of some content in a running container
rob@Rathalos ~ $ docker cp myprojectname:/var/www/html/config ~/backups/backup-`date +%Y-%m-%d`-myprojectname

# Copy local content into a container
rob@Rathalos ~ $ docker cp ./ myprojectname:/var/www/html/wp-content

# Inspecting an existing volume in a temporary container
rob@Rathalos ~ $ docker container create --name temp -v myprojectname-data:/var/www/html/wp-content busybox

# Inspecting an existing volume in a temporary container
# Yet now we only do `ls -al`, handy if you want to check some dirs
rob@Rathalos ~ $ docker run --rm -i -v=myprojectname-data:/tmp/myvolume busybox:latest ls -al /tmp/myvolume

Then there are some cleaning commands that I sometimes use to clean my cache up. However, use these will caution! They utilise flags that you should know about before using.

The -q flag will return only the container id's of those commands and thus should be used with care! Then there is the -a flag. This will return all containers, running or not.

docker kill $(docker ps -q) will kill all running containers for instance. The command below will kill all even if the container is not in a running state.

The -f flag should always raise some flags... if you catch my meaning. This is force. Always check --force commands before running them.

docker rmi is for removing docker images.

# Cleaning up the running containers, removing them in the process
rob@Rathalos ~ $ docker kill $(docker ps -aq)
rob@Rathalos ~ $ docker rm $(docker ps -aq)
rob@Rathalos ~ $ docker network prune -y

# This deletes all images including intermediate images
rob@Rathalos ~ $ docker rmi $(docker images -aq) -f