Useful command line snippets

Fri 4 May 2018
Fri 25 Oct 2019

This is a collection of great commands to try and might prove usefull in some situations. This is a ever growing page as I will keep updating this when finding new great command.

Base64 Encoding

The Base64 encoding has friendly characters, for this reason, it is very versatile and useful. In a number of situations, mainly in transport or storage, this will be a great way for compatibility between systems. While I don't have any serious cases I want to enlight I do have 2 snippets that are worthwhile and easy to remember.

# Encoding piped string
rob@Rathalos ~ $ cat myexamplefile.txt | base64

# Decoding piped string
rob@Rathalos ~ $ cat mybase64encodedfile.txt | base64 -d

Dig

A very powerful CLI DNS tool is dig. You can test everything related to DNS with it. Here are a few commands to get you going with this tool.

# Query a domain
rob@Rathalos ~ $ dig A google.com
# Get a short answer back
rob@Rathalos ~ $ dig A google.com +short

# Get the PTR record
rob@Rathalos ~ $ dig -x 8.8.8.8

# Get DNS from a specific DNS server
rob@Rathalos ~ $ dig @1.1.1.1 A google.com
# Trace the request
rob@Rathalos ~ $ dig @1.1.1.1 A google.com +trace
# Lookup SOA from domain
rob@Rathalos ~ $ dig google.com +nssearch

Generate passwords

I've used a script that I wrote in Python for years to generate my passwords. We need strong and random enough passwords to secure accounts every day. This command will run on virtually any distro without the need for programs.

You want to strengthen your password even more? Sure, try the second command! I always try to get my password as strong as possible. In this day and age it suprises me how many services limit the amount and range of characters. Not to mention password policies that don't make any sense and actually make your password less secure.

# date +%s           displays the current date in POSIX time format
# sha256             gets the checksum of this string
# base64             converts this to a base64 string
# head -c 64         gets the first 64 bytes of the piped content
# echo               displays the manipulated string
rob@Rathalos ~ $ date +%s | sha256sum | base64 | head -c 64 ; echo
# tr -cd             will act as a filter for what characters we want to allow
# < /dev/urandom     will echo the contents of the file to stdin
# head -c 64         gets the first 64 bytes of the piped content
# echo               displays the manipulated string
rob@Rathalos ~ $ tr -cd [:graph:] < /dev/urandom | head -c 64 ; echo

GPG / Paperkey

GNU Private Guard. The standard encryption tool in the open software world. It's great. It's fantastic, and here are a few snippet that might help you out.

# Create a key
rob@Rathalos ~ $ gpg --gen-full-key 
# List keys
rob@Rathalos ~ $ gpg --list-keys
# Export secret key
rob@Rathalos ~ $ gpg --export-secret-keys B54K853A > my-private-key.asc
# Export public key
rob@Rathalos ~ $ gpg --armor --export B54K853A > my-public-key.asc
# Do a basic encrypt
rob@Rathalos ~ $ gpg -e <file>
# Do a basic decrypt
rob@Rathalos ~ $ gpg -d <file>
# Import a keyfile
rob@Rathalos ~ $ gpg --import <file>
# Modify a key
rob@Rathalos ~ $ gpg --edit-key B54K853A

# -- 
# -- From here we are entering the GPG tool!
# --

# Trust the selected key for edit
gpg> trust
# Add a new uid to the selected key for edit
gpg> adduid
# Select one of the uid's
gpg> uid <id>
# Delete one of the uid's
# Note: You would logically only use this if the key is not distributed yet
gpg> deluid <id>
# Revoke selected uid
# Note: Public keyserver only allow revoking of uid's
gpg> revuid
# Make selected uid primary
gpg> primary

File commands

File commands are always great to know. In the GNU/Linux universe you've got most of the following always under your fingertips in commandline interfaces.

# See size of files in current directory
rob@Rathalos ~ $ du -sh -- *
# See system-side usage of disk space
rob@Rathalos ~ $ df

# See all files with rights, including dots
rob@Rathalos ~ $ ls -al
# Create a new file
rob@Rathalos ~ $ touch testfile
# Empty file
rob@Rathalos ~ $ echo '' > testfile
# Show all non-printable characters in a file
rob@Rathalos ~ $ cat -A testfile

# Find files with regex in current dir
rob@Rathalos ~ $ find . -name "*-avatar.jpg" -type f
# First example, including subdirs
rob@Rathalos ~ $ find . -name "**-avatar.jpg" -type f
# Delete whatever you find
rob@Rathalos ~ $ find . -name "**-avatar.jpg" -type f -delete

# See the last 50 lines of log
rob@Rathalos ~ $ tail -n 50 /var/log/nginx
# Keep tail in foreground and give you realtime feedback
rob@Rathalos ~ $ tail -f /var/log/nginx

# Check sum of file to check integrity
# You get the point with algo's prefix
rob@Rathalos ~ $ sha1sum testfile
rob@Rathalos ~ $ sha512sum testfile
rob@Rathalos ~ $ md5sum testfile

Then there are file commands that change rights and ownership. They might be basic but very usefull! Also, be warned when using the -Rf flag. The R part means recursive. The f part means by force. It might seem obvious yet these commands, though simple, can ruin your day quickly :)

# Change rights of file
rob@Rathalos ~ $ chmod 0644 test
# Change rights of all files in current directory
rob@Rathalos ~ $ chmod 0644 *
# Change rights of all files in current and subdirectories
rob@Rathalos ~ $ chmod 0644 * -Rf

# Change ownership of file
rob@Rathalos ~ $ chown rob:rob test
# Change rights of all files in current directory
rob@Rathalos ~ $ chown rob:rob *
# Change rights of all files in current and subdirectories
rob@Rathalos ~ $ chown rob:rob * -Rf

SSH / SCP

SSH is a pre for anyone working with remote these days. Insecure connections are not acceptable anymore. Laws like GDPR ensure that IT companies take it up a notch. Here is a collection of handy commands to know.

# Login with SSH with an alternative port number
rob@Rathalos ~ $ ssh user@domain.tld -p {port-number}

# Add a quoted commando to login, run the command and close the session
rob@Rathalos ~ $ ssh user@domain.tld -p {port-number} 'ls -al'

# Copy a local file to the remote server with scp
rob@Rathalos ~ $ scp -P {port-number} {local-file} user@domain.tld:~/{destination-file}

# Copy a remote file to local with scp
rob@Rathalos ~ $ scp -P {port-number} user@domain.tld:~/{remote-file} {local-directory}

Stress Commands

Stressing systems can easily be done with stress or stress-ng. However here are also a few other handy commands that you can run with everyday programs.

# Utilize CPU
sha1sum /dev/zero

# Load bytes in memory (3GB in this case)
yes | tr \\n x | head -c $((1024*1024*3000)) | grep n

Systemctl

This tool manages all running services on modern systems. There are some handy commands to know when you want to check a service.

# Start|Stop|Status|Restart
rob@Rathalos ~ $ systemctl restart nginx.service

# Enable|Disable
rob@Rathalos ~ $ systemctl disable nginx.service
rob@Rathalos ~ $ systemctl enable nginx.service

# See whats inside the service file
rob@Rathalos ~ $ systemctl cat nginx.service
# Edit a service file
rob@Rathalos ~ $ systemctl edit --full nginx.service
# Do a daemon-reload for reloading changes
rob@Rathalos ~ $ systemctl daemon-reload

# List all active services
rob@Rathalos ~ $ systemctl list-units
# List all configured services
rob@Rathalos ~ $ systemctl list-units --all

FFmpeg

I love FFmpeg as a cli tool. It is so versatile! No need for shady converter tools -- and Windows at that :wink:, those days are over!

# Let's start easy, get some info
rob@Rathalos ~ $ ffmpeg -i video.mp4
# Without configuration banner
rob@Rathalos ~ $ ffmpeg -i video.mp4 -hide_banner
# Removing audio from a video file
rob@Rathalos ~ $ ffmpeg -i input.mp4 -an output.mp4
# Removing video from a video file
rob@Rathalos ~ $ ffmpeg -i input.mp4 -vn output.mp3
# Now some conversion! Let's check the formats
rob@Rathalos ~ $ ffmpeg -formats
# For example Flash Video to MP4
rob@Rathalos ~ $ ffmpeg -i video.flv -vn output.mp4
# Compress file and make video codec h264
rob@Rathalos ~ $ ffmpeg -i input.mp4 -vcodec h264 -acodec aac output.mp4
# Speed up videos twice as fast
ffmpeg -i input.mp4 -filter:v "setpts=0.5*PTS" output.mp4
# Convert movies affected by the VFS (Vertical Filming Syndrome)
rob@Rathalos ~ $ ffmpeg -i compressed.mp4 -filter_complex '[0:v]scale=ih*16/9:-1,boxblur=luma_radius=min(h\,w)/20:luma_power=1:chroma_radius=min(cw\,ch)/20:chroma_power=1[bg];[bg][0:v]overlay=(W-w)/2:(H-h)/2,crop=h=iw*9/16' output.mp4
# Create a gif from a video file where
#   -ss is the starting point in seconds
#   -t how many seconds of video the gif should be
#   scale the resolution of the gif is
rob@Rathalos ~ $ ffmpeg -ss 10 -t 2 -i video.webm -vf "fps=10,scale=800:-1:flags=lanczos,split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse" -loop 0 sample.gif
# Use ffplay to preview video/audio files
rob@Rathalos ~ $ ffplay video.mp4

OpenSSL

# Check the certificate alternative names and other entries
rob@Rathalos ~ $ openssl x509 -in server.cert -text -noout
# Check a remote server certificate, handy for debugging remotes
rob@Rathalos ~ $ openssl s_client -connect google.com:443
# Show certificate details
rob@Rathalos ~ $ openssl s_client -connect google.com:443 | openssl x509 -noout -text
# Show only the Subject Alternative Names (SAN)
rob@Rathalos ~ $ openssl s_client -connect google.com:443 | openssl x509 -noout -text | grep DNS:

Nmap

Nmap is a great program. It allows you to do many great things like looking up open ports, looking up devices based on MAC. Discovering devices that allow this. Nmap can tell you a great deal about a network in a few commands.

# Find raspberries on the network
rob@Rathalos ~ $ nmap -sP 192.168.1.0/24 | awk '/^Nmap/{ip=$NF}/B8:27:EB/{print ip}'

# Scan a single ip
rob@Rathalos ~ $ nmap 192.168.1.1
# Scan a range of ip's
rob@Rathalos ~ $ nmap 192.168.1.1-20
# Scan a subnet
rob@Rathalos ~ $ nmap 192.168.1.0/24

# Scan a single port
rob@Rathalos ~ $ nmap -p 22 192.168.1.1
# Scan a range of ports
rob@Rathalos ~ $ nmap -p 1-100 192.168.1.1
# Quickly scan common ports, i.e. 22, 443, 365 ect.
rob@Rathalos ~ $ nmap -F 192.168.1.1

# Scan a host
rob@Rathalos ~ $ nmap www.testhostname.com

QR Code Commands

I like the following few QR generating commando's. Why? They can run locally! I don't trust online tools to create my QR codes. QR Codes are a solid way of creating a physical medium to save any type of file. Store them in a vault or other safe place around the house as a means of last resort backup.

# Create a simple QR Code
rob@Rathalos ~ $ qrencode -o qr-anything.png '#spreaditlikebutter'

# Create a "Setup my network QR Code"
# This will prompt for the SSID and WPA password
rob@Rathalos ~ $ qrencode -s 7 -o qr-wifi.png "WIFI:S:$(zenity --entry --text="Network name (SSID)" --title="Create WiFi QR");T:WPA;P:$(zenity --password --title="Wifi Password");;"

Lynx

The lightweight, commandline, text-based browser! It will help you in times of need. Here a small collection that might help you. You can also read my article about Lynx.

# Run in default mode
rob@Rathalos ~ $ lynx https://thoughtforcoding.com
# Accept all cookies if you trust the website
rob@Rathalos ~ $ lynx https://thoughtforcoding.com --accept-all-cookies
# Run in "secure mode" to ensure everything over SSL
rob@Rathalos ~ $ lynx https://thoughtforcoding.com --force-secure
# Use Realm to stay within the given domain to prevent outbound links
rob@Rathalos ~ $ lynx https://thoughtforcoding.com --realm
# Keep track of your session
rob@Rathalos ~ $ lynx https://thoughtforcoding.com --session=./lynx_session

Docker Commands

Docker is a very versatile that you can use for many projects. Here are a few commands that you can use.

# Check the running docker processes
rob@Rathalos ~ $ docker ps
# Check dockers internal resources monitor
rob@Rathalos ~ $ docker stats

# This will start a interactive shell session with an image
# The container will destroy itself after the session is done
rob@Rathalos ~ $ docker run -it -P busybox /bin/sh

# Check how the image gets build
rob@Rathalos ~ $ docker history busybox

# Checkout a lot of stats around a container
rob@Rathalos ~ $ docker inspect 

# Docker run daemonized
rob@Rathalos ~ $ docker run --name my-custom-image -d busybox
# Open specific ports
rob@Rathalos ~ $ docker run --name my-custom-image -p 30000:80 busybox
# Bind on ip -- replace with public ip :)
rob@Rathalos ~ $ docker run --name my-custom-image -p 127.0.0.1:30000:80 busybox

# Manually create a network
rob@Rathalos ~ $ docker network create network-name

# Manually connect two containers with an existing network
rob@Rathalos ~ $ docker network connect network-name my-own-image

# Making a backup of some content in a running container
rob@Rathalos ~ $ docker cp myprojectname:/var/www/html/config ~/backups/backup-`date +%Y-%m-%d`-myprojectname

# Copy local content into a container
rob@Rathalos ~ $ docker cp ./ myprojectname:/var/www/html/wp-content

# Inspecting an existing volume in a temporary container
rob@Rathalos ~ $ docker container create --name temp -v myprojectname-data:/var/www/html/wp-content busybox

# Inspecting an existing volume in a temporary container
# Yet now we only do `ls -al`, handy if you want to check some dirs
rob@Rathalos ~ $ docker run --rm -i -v=myprojectname-data:/tmp/myvolume busybox:latest ls -al /tmp/myvolume

Then there are some cleaning commands that I sometimes use to clean my cache up. However, use these will caution! They utilise flags that you should know about before using.

The -q flag will return only the container id's of those commands and thus should be used with care! Then there is the -a flag. This will return all containers, running or not.

docker kill $(docker ps -q) will kill all running containers for instance. The command below will kill all even if the container is not in a running state.

The -f flag should always raise some flags... if you catch my meaning. This is force. Always check --force commands before running them.

docker rmi is for removing docker images.

# Cleaning up the running containers, removing them in the process
rob@Rathalos ~ $ docker kill $(docker ps -aq)
rob@Rathalos ~ $ docker rm $(docker ps -aq)
rob@Rathalos ~ $ docker network prune -y

# This deletes all images including intermediate images
rob@Rathalos ~ $ docker rmi $(docker images -aq) -f