Introduction

The shell script programming (aka shell scripting), is a powerful skill for automating tasks in a command-line environment. It's great for streamlining repetitive tasks and managing system processes.

Various shell

There are various types of shells, each with its own features and capabilities. Here are some of the commonly used types of shells:
  • Bourne Shell (sh): The original Bourne shell, developed by Stephen Bourne. It is a simple and efficient shell, but it lacks some of the features found in more modern shells. The Bourne shell is a command-line interpreter or shell that was the original Unix shell.
  • Bash (Bourne Again SHell): Bash is one of the most popular and widely used shells. It is an enhanced version of the original Bourne shell, providing additional features like command history, job control, and improved scripting capabilities. Bash is the default shell for many Unix-like operating systems.
  • Zsh (Z Shell): Zsh is an extended Bourne shell with features from Bash, Korn shell, and others. It includes advanced tab completion, spelling correction, and improved scripting capabilities. Zsh is highly customizable and user-friendly.
  • PowerShell: Developed by Microsoft, PowerShell is a task automation framework and shell. It is especially prevalent in Windows environments but is also available for Linux and macOS.
  • Korn Shell (ksh): The Korn shell was developed by David Korn as a superset of the Bourne shell with additional features. It includes features like command-line editing, history, and associative arrays. There are two main versions: ksh88 and ksh93.
  • Dash: Dash is a minimalistic POSIX-compliant shell, designed to be lightweight and fast. It is often used as the default system shell on some Unix-like systems for scripts and system utilities.
  • Ash (Almquist Shell): Ash is a lightweight shell that aims to be POSIX-compliant. It is often used in embedded systems and is known for its small memory footprint.
  • Fish (Friendly Interactive SHell): Fish is designed to be user-friendly and interactive. It provides features like syntax highlighting, autosuggestions, and a consistent command syntax. While not POSIX-compliant, it offers a unique and intuitive user experience.
These shells vary in terms of features, syntax, and use cases. The choice of which shell to use often depends on personal preference, system requirements, and specific features needed for scripting or interactive use. If you have a specific context or use case in mind, feel free to ask for more tailored advice!

Some common terminologies

Scripting involves a variety of terminology that is commonly used across different programming and scripting languages. Here are some key terms:
  • Script: A script is a series of commands or instructions written in a scripting language to be executed by a runtime environment.
  • Interpreter: An interpreter is a program that reads and executes scripts line by line. It interprets the script's instructions without the need for compilation.
  • Syntax: Syntax refers to the rules governing the structure of statements in a script. Correct syntax is essential for the script to be properly interpreted and executed.
  • Variable: A variable is a named storage location in which data can be stored and retrieved. Variables are used to store information that may change during the script's execution.
  • Data Types: Data types define the type of data that a variable can hold, such as integers, strings, or booleans.
  • Conditional Statements: Conditional statements (e.g., if, else, elif) allow the script to make decisions based on certain conditions.
  • Loop: A loop is a control structure that allows a set of instructions to be repeated multiple times. Common loop types include for and while loops.
  • Function: A function is a reusable block of code that performs a specific task. Functions help in organizing and modularizing code.
  • Arguments/Parameters: Arguments or parameters are values that can be passed to a function or a script. They provide input to the code.
  • Comments: Comments are non-executable lines in a script used to provide explanations, notes, or documentation for the code. They are ignored during execution.
  • Debugging: Debugging is the process of identifying and fixing errors (bugs) in a script. This often involves using tools to trace and analyze the script's execution.
  • Shebag (#!)The shebang is a special character sequence (e.g., `#!/bin/bash` ) at the beginning of a script that specifies the interpreter to be used for executing the script.
  • Scripting Languages: Scripting languages are programming languages designed for scripting, often with a focus on ease of use and rapid development. Examples include Bash, Python, Perl, and Ruby.
  • Environment Variables: Environment variables are variables outside the script that store configuration settings, paths, or other information relevant to the execution environment.
These terms are foundational to understanding and working with scripts in various programming and scripting languages. Depending on the language, some terms may have language-specific nuances.

Execution:

  • Create a file using command ===> `vi filename.sh`
  • To make the text file executable ===> `chmod 755 filename.sh`
  • To run the script ===> `./filename.sh`
  • A tab (to create a space) in between the script like: echo "Hello `tab` World" will show a space as Hello `tab` World
  • But when there is just space placed, but not tab between `Hello World`, it will not show any space.
The `chmod u+rx myscript.sh` command is used to change the permissions of a file in a Unix-like operating system, such as Linux or macOS. Let's break down the components of this command:
  • chmod`: Stands for "change mode" and is the command used to change file permissions.
  • u+rx: This specifies the changes to be made to the permissions of the file. Here's what each part means:
    • u: Refers to the user (owner) of the file.
    • +rx: Adds read (`r`) and execute (`x`) permissions.
  • `myscript.sh`This is the name of the file for which you want to change the permissions. Replace it with the actual name of your script.
Putting it all together, `chmod u+rx myscript.sh` means "change the file permissions of `myscript.sh` to give the owner (user) read and execute permissions."
  • read(r): Allows the owner to read the contents of the file.
  • Execute (`x`): Allows the owner to execute the file as a program.
After running this command, the owner of the file `myscript.sh` will have the necessary permissions to read and execute the script. Adjusting permissions in this way is common when you want to make a script executable by its owner.

Example scripts:

Action Description Script
echo Outputs text or variables to the terminal.
#!/bin/sh
        echo "Hello, World!"
read Reads input from the user and assigns it to a variable (The `-p` option is used to specify a prompt that will be displayed to the user. In this case, the prompt is "Enter your name: ")
#!/bin/sh
        read -p "Enter your name: " username
if Conditional statement for branching based on a condition.
#!/bin/sh
        if [ "$var" -gt 10 ]; then
            echo "Greater than 10"
        fi
for Looping construct for iterating over a sequence of values.
#!/bin/sh
        for i in {1..5}; do
            echo $i
        done
while Looping construct that continues as long as a specified condition is true.
#!/bin/sh
        counter=0
        while [ $counter -lt 5 ]; do
            echo $counter
            ((counter++))
        done
case Case statement for conditional branching based on pattern matching.
#!/bin/sh
        case "$option" in
            "a") echo "Option A";;
            "b") echo "Option B";;
            *)   echo "Default Option";;
        esac
function Declares and defines a function for code modularity.
#!/bin/sh
        function greet {
            echo "Hello, $1!"
        }
        
        greet "John"
ls Lists files and directories in a directory.
#!/bin/sh
        ls -l
cp Copies files or directories.
#!/bin/sh
        cp file1.txt file2.txt
mv Moves or renames files or directories.
#!/bin/sh
        mv oldfile.txt newfile.txt
rm Removes files or directories.
#!/bin/sh
        rm file.txt
touch Creates an empty file or updates the access and modification times of a file.
#!/bin/sh
        touch newfile.txt
grep Searches for a pattern in a file or input stream.
#!/bin/sh
        grep "pattern" file.txt
sed Stream editor for filtering and transforming text.
#!/bin/sh
        sed 's/old/new/g' input.txt
awk Text processing tool for pattern scanning and processing.
#!/bin/sh
        awk '{print $2}' data.txt
Here
  • Unix system: `#!/bin/sh`: The first line tells Unix that the file is to be executed by /bin/sh.
  • Perl: script may start with the line `#!/usr/bin/perl` to tell your interactive shell that the program which follows should be executed by perl. For Bourne shell programming, we shall stick to `#!/bin/sh`.

Variables

In shell scripting, variables are used to store and manipulate data. The concept of variables - a symbolic name for a chunk of memory to which we can assign values, read and manipulate its contents. Here are the key aspects of variables in shell scripting:

  1. Variable Declaration and Assignment:

    Syntax:

    variable_name=value

    Example:

    greeting="Hello, World!"
  2. Variable Naming Rules:
    • Variable names are case-sensitive.
    • They can consist of letters, numbers, and underscores.
    • The first character must be a letter or an underscore.
  3. Accessing Variable Values:
    #!/bin/sh
        echo $greeting
  4. Special Variables:
    • Positional Parameters: $0, $1, $2, ..., represent the script name and its arguments.
    • Number of Arguments: $# gives the number of command-line arguments.
    • All Arguments: $@ or $* represents all command-line arguments.
    • Exit Status: $? holds the exit status of the last executed command.
  5. Read Input into a Variable:
    read -p "Enter your name: " username
  6. Variable Concatenation:
    fullname="$firstname $lastname"
  7. Variable Scope:
    export myvar="some value"
  8. Unsetting Variables:
    unset myvar
  9. Using Variables in Commands:
    count=5
        echo "There are $count apples."

These are fundamental concepts when working with variables in shell scripting. They provide a way to store, retrieve, and manipulate data within your scripts, making them dynamic and adaptable.

Wildcards:

  • Copy all the files from /tmp/a into /tmp/b

    $ cp /tmp/a/* /tmp/b/
    $ cp /tmp/a/*.txt /tmp/b/
    $ cp /tmp/a/*.html /tmp/b/

  • rename all .txt files to .bak

    $ mv *.txt *.bak

Escape Characters:

Escape characters in bash scripting are used to represent characters that are difficult to type or might have a special meaning in the context of a script. They are preceded by a backslash (\) to indicate that the following character should be treated differently. Here are some common escape characters in bash scripting along with examples:
  • Newline (`\n`): Represents a new line.

    echo -e "Line 1\nLine 2"

    Output:
    Line 1
    Line 2
  • Tab (`\t): Represents a horizontal tab.

    $ echo -e "Column 1\tColumn 2"

    Output: Column 1    Column 2
  • Backspace (`\b`): Moves the cursor one character to the left.

    $echo -e "Hello\bWorld"

    Output: HellWorld
  • Carriage Return (`\r`): Moves the cursor to the beginning of the line.

    $echo -e "123\rABC"

    Output: ABC
  • Single Quote ( `\'`): Escapes a single quote within a single-quoted string.

    $echo 'It '\''s a single quote'

    Output: It 's a single quote
  • Double Quote (`\"`): Escapes a double quote within a double-quoted string.

    $echo "She said, \"Hello!\""

    Output: She said, "Hello!"
  • Dollar Sign (`\$`): Escapes the dollar sign to prevent variable expansion.

    $echo "The cost is \$10"

    Output: The cost is $10
  • Backslash (`\\`): Escapes a backslash.

    $echo "This is a backslash: \\"

    Output: This is a backslash: \
  • $ echo Hello    World

    Output: Hello World
  • $ echo "Hello     World"

    Output: Hello     World
  • $ echo "Hello      "World""

    Output: Hello      "World"
  • Characters (*, ', etc):

    $ echo *

    Output: LICENSE README.md Shell-scripting-tutorials.ipynb all_args.sh count_args.sh exit_status.sh my-script-0.sh my-script-1.sh my-script-2.sh my-script-3.sh my-script-4.sh myvar2.sh myvar3.sh var.sh var2.sh var3.sh var4.sh test.txt
    (basically all files in the current directory)

    $ echo *txt

    Output: test.txt
    (*txt means all files ending in txt)

    $ echo "*"

    Output: *
    ( * in double quotes, is interpreted literally.)

    $ echo *txt

    Output: test.txt

    $ echo "*txt"

    Output: *txt
    (the same applies, but we have appended txt to the string)
  • $echo "A quote is \", backslash is \\, backtick is \`."

    Output: A quote is ", backslash is \, backtick is `.
  • $echo "A few spaces are and dollar is \$. \$X is ${X}."

    Output: A few spaces are and dollar is $. $X is 5.
  • Some other examples:

    echo "The cost is $10" ==> The cost is 0

    echo 'The cost is $10' ==> The cost is $10

    echo "The cost is \$10" ==> The cost is $10

Loop

Most languages have the concept of loops: If we want to repeat a task twenty times, we don't want to have to type in the code twenty times, with maybe a slight change each time. In Bash scripting, you can use various types of loops to iterate through a set of commands or statements. The two primary types of loops are the for loop and the while loop. Here are examples of both:

1. 'for' loop:

The `for` loop is used when you know in advance how many times you want to execute a set of statements.
  • Numeric Range:
    
                #!/bin/bash 
    for i in {1..5}
    do
      echo "Iteration $i"
    done
    Output:
    Iteration 1
    Iteration 2
    Iteration 3
    Iteration 4
    Iteration 5
  • Iterate Over Items in an Array:
    
                #!/bin/bash
                
                fruits=("Apple" "Banana" "Orange")
                
                for fruit in "${fruits[@]}"
                do
                  echo "Fruit: $fruit"
                done
                
    Output:
    Fruit: Apple
    Fruit: Banana
    Fruit: Orange

2. `while` Loop:

The while loop is used when you want to execute a set of statements as long as a condition is true.

        #!/bin/bash

        counter=5

        while [ $counter -gt 0 ]
        do
          echo "Countdown: $counter"
          ((counter--))
        done
      
  • `-gt`: This is a comparison operator that stands for "greater than."
  • `0`: This is the value that $counter is being compared to.
Output:
Countdown: 5
Countdown: 4
Countdown: 3
Countdown: 2
Countdown: 1
Remember to make your script executable using chmod +x script.sh and then execute it with ./script.sh assuming the script is named script.sh.

Test

In shell scripting, the `test` command is often used to evaluate conditional expressions. Alternatively, you can use square brackets `[ ]` as a synonym for `test`. Test is most often invoked indirectly via the if and while statements. It is also the reason you will come into difficulties if you create a program called test and try to run it, as this shell builtin will be called instead of your program! The syntax for if...then...else... is:

      #!/bin/bash
      if [ ... ]
      then
        # if-code
      else
        # else-code
      fi
    
Note that fi is if backwards! This is used again later with case and esac. Also, be aware of the syntax - the "if [ ... ]" and the "then" commands must be on different lines. Alternatively, the semicolon ";" can separate them:

      #!/bin/bash
      if [ ... ]; then
        # do something
      fi
    
You can also use the elif, like this

      #!/bin/bash
      if  [ something ]; then
      echo "Something"
      elif [ something_else ]; then
        echo "Something else"
      else
        echo "None of the above"
      fi
    
Examples:
  • String Comparison:
    
              #!/bin/bash
        
              str1="hello"
              str2="world"
              
              if [ "$str1" = "$str2" ]; then
                echo "Strings are equal"
              else
                echo "Strings are not equal"
              fi
            
  • Numeric Comparison:
    
              #!/bin/bash
    
              num1=5
              num2=10
              
              if [ "$num1" -eq "$num2" ]; then
                echo "Numbers are equal"
              else
                echo "Numbers are not equal"
              fi
            
  • File Existence:
    
              #!/bin/bash
    
              file_path="/path/to/some/file.txt"
              
              if [ -e "$file_path" ]; then
                echo "File exists"
              else
                echo "File does not exist"
              fi
            
  • Logical AND:
    
              #!/bin/bash
    
              num1=5
              num2=10
              
              if [ "$num1" -gt 0 ] && [ "$num2" -lt 20 ]; then
                echo "Both conditions are true"
              else
                echo "At least one condition is false"
              fi
            

The case statement

A `case` statement in Bash scripting is used for conditional branching based on pattern matching. It is often used as an alternative to multiple `if-elif-else` statements when you have multiple conditions to check. Its syntax is really quite simple:

Example:


      #!/bin/bash

      echo "Enter a fruit: "
      read fruit
      
      case "$fruit" in
        "apple")
          echo "Selected fruit: Apple"
          ;;
        "banana" | "plantain")
          echo "Selected fruit: Banana or Plantain"
          ;;
        "orange")
          echo "Selected fruit: Orange"
          ;;
        *)
          echo "Unknown fruit"
          ;;
      esac
    

Bash command references

Command Description Example
& Run the previous command in the background ls &
&& Logical AND if [ "$foo" -ge "0" ] && [ "$foo" -le "9" ]
|| Logical OR if [ "$foo" -lt "0" ] || [ "$foo" -gt "9" ]
^ Start of line grep "^foo"
$ End of line grep "foo$"
= String equality (cf. -eq) if [ "$foo" = "bar" ]
! Logical NOT if [ "$foo" != "bar" ]
$$ PID of current shell echo "my PID = $$"
$! PID of last background command ls & echo "PID of ls = $!"
$? Exit status of last command ls ; echo "ls returned code $?"
$0 Name of current command (as called) echo "I am $0"
$1 Name of current command's first parameter echo "My first argument is $1"
$9 Name of current command's ninth parameter echo "My ninth argument is $9"
$@ All of current command's parameters (preserving whitespace and quoting) echo "My arguments are $@"
$* All of current command's parameters (not preserving whitespace and quoting) echo "My arguments are $*"
-eq Numeric Equality if [ "$foo" -eq "9" ]
-ne Numeric Inequality if [ "$foo" -ne "9" ]
-lt Less Than if [ "$foo" -lt "9" ]
-le Less Than or Equal if [ "$foo" -le "9" ]
-gt Greater Than if [ "$foo" -gt "9" ]
-ge Greater Than or Equal if [ "$foo" -ge "9" ]
-z String is zero length if [ -z "$foo" ]
-n String is not zero length if [ -n "$foo" ]
-nt Newer Than if [ "$file1" -nt "$file2" ]
-d Is a Directory if [ -d /bin ]
-f Is a File if [ -f /bin/ls ]
-r Is a readable file if [ -r /bin/ls ]
-w Is a writable file if [ -w /bin/ls ]
-x Is an executable file if [ -x /bin/ls ]
( ... ) Function definition function myfunc() { echo hello }

References


Some other interesting things to know: