Bash one-liners every sysadmin should know

Efficient command-line operation is a cornerstone of system administration. Bash one-liners, by chaining together standard Unix utilities, can…

Efficient command-line operation is a cornerstone of system administration. Bash one-liners, by chaining together standard Unix utilities, can dramatically reduce the time spent on routine tasks, diagnostics, and data manipulation. This article explores a selection of powerful and practical Bash one-liners that every sysadmin should integrate into their daily toolkit, focusing on real-world use cases, command flags, and pipeline mechanics.

File System Navigation and Manipulation

Finding and Filtering Files

The find command is indispensable for locating files based on various criteria. Combining it with other tools via pipes enables powerful filtering and action.

• Find files modified in the last 24 hours:

find . -type f -mtime -1 -print0 | xargs -0 ls -l

-type f restricts results to regular files. -mtime -1 finds files modified within the last day (24 hours). -print0 outputs filenames separated by null characters, which xargs -0 safely handles, preventing issues with spaces or special characters in filenames.

• Find and delete empty directories:

find /path/to/search -type d -empty -delete

This command finds all empty directories within /path/to/search and removes them. Use with caution.

Searching File Contents with grep

grep is fundamental for text pattern matching. Its various flags enhance its utility for log analysis and code inspection.

• Recursively search for case-insensitive string, show line numbers, and ignore binary files:

grep -RIn --exclude-from=.gitignore "search_string" .

-R (or -r) performs a recursive search. -I ignores binary files. -n shows line numbers. -i makes the search case-insensitive. --exclude-from=.gitignore is a powerful addition to prevent searching in version-controlled ignored files (e.g., node_modules, build directories).

• Find files containing "error" but not "critical":

grep -r "error" . | grep -v "critical"

This pipes the output of the first grep to a second grep -v, which inverts the match, showing lines that do NOT contain "critical".

In-place Editing with sed

sed is a stream editor often used for simple text transformations, particularly useful for configuration file adjustments.

• Replace all occurrences of "old_string" with "new_string" in a file:

sed -i 's/old_string/new_string/g' /path/to/file.conf

-i performs an in-place edit, modifying the file directly. The s command performs substitution, g ensures all occurrences on a line are replaced.

• Remove lines matching a pattern from multiple files:

find . -name "*.log" -print0 | xargs -0 sed -i '/^DEBUG/d'

This finds all .log files and, for each, deletes lines starting with "DEBUG".

Process Management and System Monitoring

Listing and Filtering Processes with ss

ss is a modern utility to investigate sockets, replacing netstat in many scenarios, offering faster and more detailed output.

• List all listening TCP sockets with process info, numerical addresses, and PID:

ss -tulpn

-t for TCP, -u for UDP, -l for listening sockets, -p to show process using socket, -n for numerical port numbers/addresses (no DNS lookup).

• Find processes listening on a specific port (e.g., 8080):

ss -tulpn | grep ":8080"

Real-time Log Monitoring with journalctl

For systems using Systemd, journalctl is the primary tool for querying and displaying logs from the journal.

• Follow logs from a specific unit (e.g., nginx) in real-time, showing boot-specific messages:

journalctl -fu nginx.service

-f (follow) streams new messages. -u filters by unit. For the current boot, it will show logs from the beginning of the boot. To see all logs from the beginning of time for that unit, remove -b or specify -b all if available.

• Show kernel messages since last boot:

journalctl -kb

-k shows kernel messages, -b (boot) shows messages from the current boot.

Data Processing and Transformation

Powerful Text Manipulation with awk

awk is a programming language designed for text processing, often used for columnar data. It processes text line by line.

• Extract the 5th column from a CSV file, skipping the header:

awk -F',' 'NR > 1 {print $5}' data.csv

-F',' sets the field separator to a comma. NR > 1 is a condition that ensures processing starts from the second line (skipping the header). $5 refers to the fifth field.

• Sum the values of a column (e.g., the 3rd column):

cat numbers.txt | awk '{sum += $3} END {print sum}'

This accumulates the sum of the third column for all lines and prints the total at the end.

JSON Parsing with jq

jq is a lightweight and flexible command-line JSON processor. It's essential when dealing with API responses or configuration files in JSON format.

• Extract a specific field from a JSON object and output as raw string:

curl -s "https://api.github.com/repos/stedolan/jq/releases/latest" | jq -r '.tag_name'

-r (raw output) prints the string without JSON quotation marks. This command fetches the latest jq release tag from GitHub and extracts its name.

• Filter an array of JSON objects based on a condition:

cat users.json | jq '.[] | select(.isActive == true) | .email'

This processes an array of user objects, selects only active users, and outputs their email addresses.

Archiving and Synchronization

Efficient Archiving with tar

tar is the standard Unix utility for creating and extracting archive files. Its combination with compression utilities is very common.

• Create a gzipped tar archive of a directory:

tar czf archive_name.tar.gz /path/to/directory

c (create), z (gzip compression), f (specify archive filename). This is a foundational command for backups.

• Extract a gzipped tar archive:

tar xzf archive_name.tar.gz -C /destination/path

x (extract), z (gzip decompression), f (specify archive filename). -C changes to the specified directory before performing the operation.

Synchronizing Files with rsync

rsync is a fast, versatile, remote (and local) file-copying tool. It is renowned for its delta-transfer algorithm, which minimizes data transfer by only sending the differences between files.

• Recursively synchronize a local directory to a remote server, show progress, and preserve permissions/timestamps:

rsync -aP --delete /local/path/ user@remote:/remote/path/

-a (archive mode) is a shortcut for -rlptgoD (recursive, links, permissions, times, group, owner, devices). -P is a combination of --partial and --progress, useful for large transfers. --delete removes files from the destination that are not present in the source, ensuring an exact mirror. The trailing slash on /local/path/ is crucial: it means "the contents of this directory," not the directory itself.

• Dry run an rsync operation to see what would change:

rsync -aPnv /local/path/ user@remote:/remote/path/

The -n (or --dry-run) flag is invaluable for previewing changes before execution.

Common Pitfalls and Troubleshooting

• Shell Expansion vs. xargs: Understand when to use find ... -exec, find ... | xargs, or direct shell expansion (e.g., ls *.txt). xargs -0 with find -print0 is crucial for handling filenames with spaces or special characters safely.
• sed -i Backups: On macOS, sed -i requires an argument for a backup suffix (e.g., sed -i '' for no backup). Linux sed allows -i without an argument. Be aware of this platform difference.
• Regular Expressions: Different tools (grep, sed, awk, find's -regex) might use slightly different regex dialects (basic, extended, Perl-compatible). Always test complex regex patterns.
• Quoting: Always use single quotes ('...') for arguments that contain special characters or spaces to prevent shell expansion, especially with awk, sed, and grep patterns. Double quotes ("...") allow variable expansion.
• Testing with --dry-run: For commands like rsync, rm, or sed -i, always perform a dry run or operate on copies of files first if there's any doubt about the outcome.