Shell Scripting: Automate Tasks from the CLI

Shell scripting turns a sequence of CLI commands into a reusable program. Write the script once, run it anywhere on Linux or macOS, and your terminal does the repetitive work while you focus on the actual problem. This guide walks through Bash from the first command to error handling and real-world automation, with working examples at every step.

If you are stuck on a scripting assignment, have your script done covers Bash, Python automation, and 30+ other languages with verified, runnable deliverables.

Basic Shell Commands for Automation

Shell scripting strings together a sequence of commands to complete a task. Master these 5 before writing your first script.

echo prints text or variable values to the terminal:

message="Hello, World!"
echo $message

cd changes the working directory:

cd /path/to/directory

ls lists the contents of the current directory:

ls

cp and mv copy or move files:

cp source_file destination_directory
mv source_file destination_directory

rm removes files or directories:

rm filename
rm -r directory_name

Organizing files with a script

Put those 5 commands together and you can sort a cluttered folder by file type in under 10 lines:

#!/bin/bash

# Create target directories
mkdir images documents videos

# Move files by extension
mv *.jpg images/
mv *.pdf documents/
mv *.mp4 videos/

The * wildcard matches any filename that ends with the given extension, so mv *.jpg images/ moves every JPEG in one call instead of naming each file individually.

Variables and User Inputs

Variables store data; the read command pulls input from the user at runtime.

Declare a variable by assigning a value. Access it with $:

name="John"
echo "Hello, $name"

Command-line arguments

Pass arguments when calling a script and access them by position:

./myscript.sh arg1 arg2

Inside the script, $1 holds the first argument and $2 the second:

echo "First argument: $1"
echo "Second argument: $2"

Reading user input with read

The read command pauses execution and waits for keyboard input:

read -p "Enter your name: " username
echo "Hello, $username!"

Example: calculating rectangle area

#!/bin/bash

read -p "Enter the length: " length
read -p "Enter the width: " width

area=$((length * width))
echo "The area of the rectangle is: $area square units"

Conditional Statements

Conditionals let a script take different actions depending on data. The 3 building blocks are if, else, and elif.

if [ condition ]; then
    # runs when condition is true
else
    # runs when condition is false
fi

Comparison operators

| Operator | Meaning | | -------- | --------------------- | | -eq | equal | | -ne | not equal | | -lt | less than | | -le | less than or equal | | -gt | greater than | | -ge | greater than or equal |

if [ $num -eq 0 ]; then
    echo "The number is zero"
elif [ $num -lt 0 ]; then
    echo "The number is negative"
else
    echo "The number is positive"
fi

Combining conditions with && and ||

&& (AND) requires both conditions to be true. || (OR) requires at least one:

if [ $age -ge 18 ] && [ "$country" == "USA" ]; then
    echo "Eligible to vote in the USA"
fi

Example: odd or even

#!/bin/bash

read -p "Enter a number: " num

if [ $((num % 2)) -eq 0 ]; then
    echo "$num is even"
else
    echo "$num is odd"
fi

Loops for Automation

Loops execute a block of commands repeatedly. Shell scripting provides 2 primary loop types.

The for loop iterates over a fixed sequence:

for variable in sequence
do
    # commands to execute
done

Print numbers 1 through 5:

for i in 1 2 3 4 5
do
    echo $i
done

The while loop

The while loop runs as long as a condition holds true:

while [ condition ]
do
    # commands to execute
done

Count down from 10 to 1:

count=10
while [ $count -gt 0 ]
do
    echo $count
    count=$((count - 1))
done

Example: multiplication table

#!/bin/bash

read -p "Enter a number: " num

for i in {1..10}
do
    result=$((num * i))
    echo "$num * $i = $result"
done

File Manipulation and Text Processing

3 commands cover most file and text processing work in Bash: grep, sed, and awk.

grep searches for a pattern inside a file:

grep "error" logfile.txt

Transforming text with sed

sed edits text without opening a file in an editor. Replace every occurrence of "old" with "new":

sed 's/old/new/g' input.txt > output.txt

Extracting columns with awk

awk splits each line into fields and lets you operate on individual columns. Pull the third column from a CSV:

awk -F ',' '{print $3}' data.csv

Example: extracting unique IP addresses from a log

#!/bin/bash

log_file="system.log"

grep -E -o "([0-9]{1,3}\.){3}[0-9]{1,3}" "$log_file" | sort | uniq > ips.txt

grep isolates every IP address, sort puts them in order, and uniq discards duplicates before writing the list to ips.txt.

Redirection and Piping

Redirection controls where a command sends its output or reads its input. Piping chains commands so the output of one becomes the input of the next.

> writes stdout to a file (overwrites):

ls > file_list.txt

>> appends stdout to an existing file:

echo "Appended content" >> existing_file.txt

< reads stdin from a file instead of the keyboard:

sort < unsorted_data.txt > sorted_data.txt

| (pipe) passes stdout from one command as stdin to the next:

cat log.txt | grep "error" | sort > error_log.txt

Example: counting errors and warnings by frequency

cat system.log | grep -E "error|warning" | sort | uniq -c | sort -nr > error_warning_summary.txt

Each stage in the pipeline adds one transformation: filter, deduplicate, count, sort by frequency. The final file is ready to read or send as a report.

Functions for Modularity

Functions wrap a block of commands under a name. Call the name to run the block again without rewriting it.

Define and call a simple function:

function greet {
    echo "Hello, World!"
}
greet

Passing parameters

Use $1, $2, and so on to access arguments passed to the function:

function greet {
    echo "Hello, $1!"
}
greet "Alice"

Returning values

Bash functions return an integer exit code via return. Capture computed output with command substitution instead:

function multiply {
    local result=$(( $1 * $2 ))
    echo $result
}

result=$(multiply 5 3)
echo "The result is: $result"

Example: string manipulation toolkit

#!/bin/bash

function get_length {
    echo "${#1}"
}

function get_substring {
    echo "${1:$2:$3}"
}

function concatenate {
    echo "$1$2"
}

input="Hello, World!"

length=$(get_length "$input")
substring=$(get_substring "$input" 0 5)
concatenated=$(concatenate "Hi, " "there!")

echo "Length: $length"
echo "Substring: $substring"
echo "Concatenated: $concatenated"

Error Handling and Exit Codes

Every command exits with a status code. 0 means success; any non-zero value signals a failure. The special variable $? holds the last exit status.

Exiting on failure

Use exit with a non-zero code to stop the script when something goes wrong:

if [ $# -eq 0 ]; then
    echo "No arguments provided. Exiting."
    exit 1
fi

Trapping signals with trap

The trap command registers a handler that runs when the script receives a signal. This keeps scripts from dying silently:

trap 'echo "Script interrupted. Exiting."; exit 2' INT TERM

Example: validating numeric input

#!/bin/bash

read -p "Enter a number: " num

if ! [[ "$num" =~ ^[0-9]+$ ]]; then
    echo "Invalid input. Please enter a valid number."
    exit 1
fi

result=$((num * 2))
echo "Double of $num is: $result"

Advanced Scripting Techniques

These 3 techniques handle the tasks that basic commands cannot.

Regular expressions for pattern matching

Bash supports regex matching with =~ inside [[ ]]. Validate an email address format:

pattern="^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,4}$"
if [[ "$email" =~ $pattern ]]; then
    echo "Valid email address"
else
    echo "Invalid email address"
fi

Command substitution with $(...)

Capture the output of a command and assign it to a variable:

file_count=$(ls | wc -l)
echo "Number of files in the directory: $file_count"

Arithmetic with (( ))

The (( )) construct evaluates arithmetic expressions and comparisons:

total=$((num1 + num2))
if (( total > 100 )); then
    echo "Total is greater than 100"
fi

Example: validating an email address

#!/bin/bash

read -p "Enter an email address: " email

pattern="^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,4}$"

if [[ "$email" =~ $pattern ]]; then
    echo "Valid email address"
else
    echo "Invalid email address"
fi

Script Execution and Permissions

A script file is just text until you mark it executable. The chmod command sets that permission:

chmod +x script.sh

Running a script by path

Call the script using its absolute path or a relative path from the current directory:

# Absolute path
/home/user/scripts/script.sh

# Relative path
./scripts/script.sh

Adding scripts to PATH

Add your scripts directory to PATH so you can call them by name from anywhere:

# Temporary (current session only)
export PATH=$PATH:/home/user/scripts

# Permanent (add to ~/.bashrc or ~/.profile)
echo 'export PATH=$PATH:/home/user/scripts' >> ~/.bashrc

Example: automated timestamped backup

#!/bin/bash

backup_dir="/backup"
source_dir="/important_files"

timestamp=$(date +"%Y%m%d_%H%M%S")
backup_filename="backup_$timestamp.tar.gz"

tar -czvf "$backup_dir/$backup_filename" "$source_dir"
echo "Backup completed: $backup_filename"

The date command with the +"%Y%m%d_%H%M%S" format string produces a unique timestamp for every run, so backups never overwrite each other.

Real-world Automation Examples

Installing and updating packages across systems

#!/bin/bash

packages=("package1" "package2" "package3")

for package in "${packages[@]}"; do
    sudo apt-get install -y "$package"
done

Counting error levels in a log file

#!/bin/bash

log_file="system.log"
error_levels=("error" "warning" "info")

for level in "${error_levels[@]}"; do
    count=$(grep -c "$level" "$log_file")
    echo "$level: $count occurrences"
done

Cleaning up temporary files via cron

Schedule this script to run every night at 2:00 AM by adding 0 2 * * * /path/to/cleanup.sh to your crontab:

#!/bin/bash

find /tmp -name "*.tmp" -delete

Monitoring CPU usage and sending alerts

#!/bin/bash

threshold=80

while true; do
    cpu_usage=$(top -bn1 | grep "Cpu(s)" | awk '{print $2}')

    if (( $(bc <<< "$cpu_usage > $threshold") )); then
        echo "High CPU usage detected: $cpu_usage%"
        echo "Subject: High CPU Usage Alert" | mail -s "High CPU Usage Alert" admin@example.com
    fi

    sleep 300
done

Best Practices

Keep scripts modular. Break tasks into functions, each handling one job. Functions you write today become building blocks for the next script.

Comment your code. A comment before each section saves debugging time when you return to a script months later:

#!/bin/bash

# Calculate the factorial of the number passed as the first argument

number=$1
factorial=1

for (( i=1; i<=number; i++ )); do
    factorial=$(( factorial * i ))
done

echo "The factorial of $number is: $factorial"

Never hardcode secrets. Passwords and API keys belong in environment variables or a config file excluded from version control. Hardcoding them in a script that ends up in a shared repo exposes them permanently.

Use version control. Track your scripts in Git the same way you track application code. Every change is reversible and the history tells you what changed and when. See Git Branch and Merge: A Practical Guide for a solid starting workflow.

Related Resources

Shell scripting pairs directly with Linux system work. For assignment help across scripting, Python automation, and 30+ other languages, see Programming Homework Help.