Bash scripting Guide

Example

1#!/usr/bin/env bash
2
3NAME="John"
4echo "Hello $NAME!"
5

Variables

1NAME="John"
2echo $NAME
3echo "$NAME"
4echo "${NAME}!"
5

String quotes

1NAME="John"
2echo "Hi $NAME"  #=> Hi John
3echo 'Hi $NAME'  #=> Hi $NAME
4

Shell execution

1echo "I'm in $(pwd)"
2echo "I'm in `pwd`"
3# Same
4

See Command substitution

Conditional execution

1git commit && git push
2git commit || echo "Commit failed"
3

Functions

1get_name() {
2  echo "John"
3}
4
5echo "You are $(get_name)"
6

See: Functions

Conditionals

1if [[ -z "$string" ]]; then
2  echo "String is empty"
3elif [[ -n "$string" ]]; then
4  echo "String is not empty"
5fi
6

Strict mode

1set -euo pipefail
2IFS=$'\n\t'
3

Brace expansion

1echo {A,B}.js
2

{A,B}
Same as A B
{A,B}.js
Same as A.js B.js
{1..5}
Same as 1 2 3 4 5

#Parameter expansions

Basics

1name="John"
2echo ${name}
3echo ${name/J/j}    #=> "john" (substitution)
4echo ${name:0:2}    #=> "Jo" (slicing)
5echo ${name::2}     #=> "Jo" (slicing)
6echo ${name::-1}    #=> "Joh" (slicing)
7echo ${name:(-1)}   #=> "n" (slicing from right)
8echo ${name:(-2):1} #=> "h" (slicing from right)
9echo ${food:-Cake}  #=> $food or "Cake"
10
11length=2
12echo ${name:0:length}  #=> "Jo"
13

1STR="/path/to/foo.cpp"
2echo ${STR%.cpp}    # /path/to/foo
3echo ${STR%.cpp}.o  # /path/to/foo.o
4
5echo ${STR##*.}     # cpp (extension)
6echo ${STR##*/}     # foo.cpp (basepath)
7
8echo ${STR#*/}      # path/to/foo.cpp
9echo ${STR##*/}     # foo.cpp
10
11echo ${STR/foo/bar} # /path/to/bar.cpp
12
13STR="Hello world"
14echo ${STR:6:5}   # "world"
15echo ${STR:-5:5}  # "world"
16
17SRC="/path/to/foo.cpp"
18BASE=${SRC##*/}   #=> "foo.cpp" (basepath)
19DIR=${SRC%$BASE}  #=> "/path/to/" (dirpath)
20

Substitution

${FOO%suffix}
Remove suffix
${FOO#prefix}
Remove prefix
${FOO%%suffix}
Remove long suffix
${FOO##prefix}
Remove long prefix
${FOO/from/to}
Replace first match
${FOO//from/to}
Replace all
${FOO/%from/to}
Replace suffix
${FOO/#from/to}
Replace prefix

Comments

1# Single line comment
2
3: '
4This is a
5multi line
6comment
7'
8

Substrings

${FOO:0:3}
Substring (position, length)
${FOO:-3:3}
Substring from the right

Length

${#FOO}
Length of $FOO

Manipulation

1STR="HELLO WORLD!"
2echo ${STR,}   #=> "hELLO WORLD!" (lowercase 1st letter)
3echo ${STR,,}  #=> "hello world!" (all lowercase)
4
5STR="hello world!"
6echo ${STR^}   #=> "Hello world!" (uppercase 1st letter)
7echo ${STR^^}  #=> "HELLO WORLD!" (all uppercase)
8

Default values

${FOO:-val}
$FOO, or val if not set
${FOO:=val}
Set $FOO to val if not set
${FOO:+val}
val if $FOO is set
${FOO:?message}
Show error message and exit if $FOO is not set

The : is optional (eg, ${FOO=word} works)

#Loops

Basic for loop

1for i in /etc/rc.*; do
2  echo $i
3done
4

C-like for loop

1for ((i = 0 ; i < 100 ; i++)); do
2  echo $i
3done
4

Ranges

1for i in {1..5}; do
2    echo "Welcome $i"
3done
4

With step size

1for i in {5..50..5}; do
2    echo "Welcome $i"
3done
4

Reading lines

1cat file.txt | while read line; do
2  echo $line
3done
4

Forever

1while true; do
2  ···
3done
4

#Functions

Defining functions

1myfunc() {
2    echo "hello $1"
3}
4
5# Same as above (alternate syntax)
6function myfunc() {
7    echo "hello $1"
8}
9
10myfunc "John"
11

Returning values

1myfunc() {
2    local myresult='some value'
3    echo $myresult
4}
5
6result="$(myfunc)"
7

Raising errors

1myfunc() {
2  return 1
3}
4
5if myfunc; then
6  echo "success"
7else
8  echo "failure"
9fi
10

Arguments

$#
Number of arguments
$*
All arguments
$@
All arguments, starting from first
$1
First argument
$_
Last argument of the previous command

#Conditionals

Conditions

Note that [[ is actually a command/program that returns either 0 (true) or 1 (false). Any program that obeys the same logic (like all base utils, such as grep(1) or ping(1)) can be used as condition, see examples.

[[ -z STRING ]]
Empty string
[[ -n STRING ]]
Not empty string
[[ STRING == STRING ]]
Equal
[[ STRING != STRING ]]
Not Equal
[[ NUM -eq NUM ]]
Equal
[[ NUM -ne NUM ]]
Not equal
[[ NUM -lt NUM ]]
Less than
[[ NUM -le NUM ]]
Less than or equal
[[ NUM -gt NUM ]]
Greater than
[[ NUM -ge NUM ]]
Greater than or equal
[[ STRING =~ STRING ]]
Regexp
(( NUM < NUM ))
Numeric conditions

[[ -o noclobber ]]
If OPTIONNAME is enabled
[[ ! EXPR ]]
Not
[[ X ]] && [[ Y ]]
And
[[ X ]] || [[ Y ]]
Or

File conditions

[[ -e FILE ]]
Exists
[[ -r FILE ]]
Readable
[[ -h FILE ]]
Symlink
[[ -d FILE ]]
Directory
[[ -w FILE ]]
Writable
[[ -s FILE ]]
Size is > 0 bytes
[[ -f FILE ]]
File
[[ -x FILE ]]
Executable
[[ FILE1 -nt FILE2 ]]
1 is more recent than 2
[[ FILE1 -ot FILE2 ]]
2 is more recent than 1
[[ FILE1 -ef FILE2 ]]
Same files

Example

1# String
2if [[ -z "$string" ]]; then
3  echo "String is empty"
4elif [[ -n "$string" ]]; then
5  echo "String is not empty"
6fi
7
8# Combinations
9if [[ X ]] && [[ Y ]]; then
10  ...
11fi
12
13# Equal
14if [[ "$A" == "$B" ]]
15
16# Regex
17if [[ "A" =~ . ]]
18
19if (( $a < $b )); then
20   echo "$a is smaller than $b"
21fi
22
23if [[ -e "file.txt" ]]; then
24  echo "file exists"
25fi
26

#Arrays

Defining arrays

1Fruits=('Apple' 'Banana' 'Orange')
2
3Fruits[0]="Apple"
4Fruits[1]="Banana"
5Fruits[2]="Orange"
6

Working with arrays

1echo ${Fruits[0]}           # Element #0
2echo ${Fruits[@]}           # All elements, space-separated
3echo ${#Fruits[@]}          # Number of elements
4echo ${#Fruits}             # String length of the 1st element
5echo ${#Fruits[3]}          # String length of the Nth element
6echo ${Fruits[@]:3:2}       # Range (from position 3, length 2)
7

Operations

1Fruits=("${Fruits[@]}" "Watermelon")    # Push
2Fruits+=('Watermelon')                  # Also Push
3Fruits=( ${Fruits[@]/Ap*/} )            # Remove by regex match
4unset Fruits[2]                         # Remove one item
5Fruits=("${Fruits[@]}")                 # Duplicate
6Fruits=("${Fruits[@]}" "${Veggies[@]}") # Concatenate
7lines=(`cat "logfile"`)                 # Read from file
8

Iteration

1for i in "${arrayName[@]}"; do
2  echo $i
3done
4

#Dictionaries

Defining

1declare -A sounds
2
3sounds[dog]="bark"
4sounds[cow]="moo"
5sounds[bird]="tweet"
6sounds[wolf]="howl"
7

Declares sound as a Dictionary object (aka associative array).

Working with dictionaries

1echo ${sounds[dog]} # Dog's sound
2echo ${sounds[@]}   # All values
3echo ${!sounds[@]}  # All keys
4echo ${#sounds[@]}  # Number of elements
5unset sounds[dog]   # Delete dog
6

Iteration

Iterate over values

1for val in "${sounds[@]}"; do
2  echo $val
3done
4

Iterate over keys

1for key in "${!sounds[@]}"; do
2  echo $key
3done
4

#Options

Options

1set -o noclobber  # Avoid overlay files (echo "hi" > foo)
2set -o errexit    # Used to exit upon error, avoiding cascading errors
3set -o pipefail   # Unveils hidden failures
4set -o nounset    # Exposes unset variables
5

Glob options

1shopt -s nullglob    # Non-matching globs are removed  ('*.foo' => '')
2shopt -s failglob    # Non-matching globs throw errors
3shopt -s nocaseglob  # Case insensitive globs
4shopt -s dotglob     # Wildcards match dotfiles ("*.sh" => ".foo.sh")
5shopt -s globstar    # Allow ** for recursive matches ('lib/**/*.rb' => 'lib/a/b/c.rb')
6

Set GLOBIGNORE as a colon-separated list of patterns to be removed from glob matches.

#History

Commands

history
Show history
shopt -s histverify
Don’t execute expanded result immediately

Expansions

!$
Expand last parameter of most recent command
!*
Expand all parameters of most recent command
!-n
Expand nth most recent command
!n
Expand nth command in history
!<command>
Expand most recent invocation of command <command>

Operations

!!
Execute last command again
!!:s/<FROM>/<TO>/
Replace first occurrence of <FROM> to <TO> in most recent command
!!:gs/<FROM>/<TO>/
Replace all occurrences of <FROM> to <TO> in most recent command
!$:t
Expand only basename from last parameter of most recent command
!$:h
Expand only directory from last parameter of most recent command

!! and !$ can be replaced with any valid expansion.

Slices

!!:n
Expand only nth token from most recent command (command is 0; first argument is 1)
!^
Expand first argument from most recent command
!$
Expand last token from most recent command
!!:n-m
Expand range of tokens from most recent command
!!:n-$
Expand nth token to last from most recent command

!! can be replaced with any valid expansion i.e. !cat, !-2, !42, etc.

#Miscellaneous

Numeric calculations

1$((a + 200))      # Add 200 to $a
2
3$((RANDOM%=200))  # Random number 0..200
4

Subshells

1(cd somedir; echo "I'm now in $PWD")
2pwd # still in first directory
3

Redirection

1python hello.py > output.txt   # stdout to (file)
2python hello.py >> output.txt  # stdout to (file), append
3python hello.py 2> error.log   # stderr to (file)
4python hello.py 2>&1           # stderr to stdout
5python hello.py 2>/dev/null    # stderr to (null)
6python hello.py &>/dev/null    # stdout and stderr to (null)
7
8python hello.py < foo.txt      # feed foo.txt to stdin for python
9

Inspecting commands

1command -V cd
2#=> "cd is a function/alias/whatever"
3

Trap errors

1trap 'echo Error at about $LINENO' ERR
2

1traperr() {
2  echo "ERROR: ${BASH_SOURCE[1]} at about ${BASH_LINENO[0]}"
3}
4
5set -o errtrace
6trap traperr ERR
7

Case/switch

1case "$1" in
2  start | up)
3    vagrant up
4    ;;
5
6  *)
7    echo "Usage: $0 {start|stop|ssh}"
8    ;;
9esac
10

Source relative

1source "${0%/*}/../share/foo.sh"
2

printf

1printf "Hello %s, I'm %s" Sven Olga
2#=> "Hello Sven, I'm Olga
3
4printf "1 + 1 = %d" 2
5#=> "1 + 1 = 2"
6
7printf "This is how you print a float: %f" 2
8#=> "This is how you print a float: 2.000000"
9

Directory of script

1DIR="${0%/*}"
2

Getting options

1while [[ "$1" =~ ^- && ! "$1" == "--" ]]; do case $1 in
2  -V | --version )
3    echo $version
4    exit
5    ;;
6  -s | --string )
7    shift; string=$1
8    ;;
9  -f | --flag )
10    flag=1
11    ;;
12esac; shift; done
13if [[ "$1" == '--' ]]; then shift; fi
14

Heredoc

1cat <<END
2hello world
3END
4

Reading input

1echo -n "Proceed? [y/n]: "
2read ans
3echo $ans
4
5read -n 1 ans    # Just one character
6

Special variables

$?
Exit status of last task
$!
PID of last background task
$$
PID of shell
$0
Filename of the shell script

Go to previous directory

1pwd # /home/user/foo
2cd bar/
3pwd # /home/user/foo/bar
4cd -
5pwd # /home/user/foo
6

Check for command’s result

1if ping -c 1 google.com; then
2  echo "It appears you have a working internet connection"
3fi
4

Grep check

1if grep -q 'foo' ~/.bash_history; then
2  echo "You appear to have typed 'foo' in the past"
3fi

`{A,B}`	Same as `A B`
`{A,B}.js`	Same as `A.js B.js`
`{1..5}`	Same as `1 2 3 4 5`

`${FOO:0:3}`	Substring (position, length)
`${FOO:-3:3}`	Substring from the right

`${FOO:-val}`	`$FOO`, or `val` if not set
`${FOO:=val}`	Set `$FOO` to `val` if not set
`${FOO:+val}`	`val` if `$FOO` is set
`${FOO:?message}`	Show error message and exit if `$FOO` is not set

`!$`	Expand last parameter of most recent command
`!*`	Expand all parameters of most recent command
`!-n`	Expand `n`th most recent command
`!n`	Expand `n`th command in history
`!<command>`	Expand most recent invocation of command `<command>`

`!!`	Execute last command again
`!!:s/<FROM>/<TO>/`	Replace first occurrence of `<FROM>` to `<TO>` in most recent command
`!!:gs/<FROM>/<TO>/`	Replace all occurrences of `<FROM>` to `<TO>` in most recent command
`!$:t`	Expand only basename from last parameter of most recent command
`!$:h`	Expand only directory from last parameter of most recent command

`${FOO%suffix}`	Remove suffix
`${FOO#prefix}`	Remove prefix
`${FOO%%suffix}`	Remove long suffix
`${FOO##prefix}`	Remove long prefix
`${FOO/from/to}`	Replace first match
`${FOO//from/to}`	Replace all
`${FOO/%from/to}`	Replace suffix
`${FOO/#from/to}`	Replace prefix

`$#`	Number of arguments
`$*`	All arguments
`$@`	All arguments, starting from first
`$1`	First argument
`$_`	Last argument of the previous command

`[[ -z STRING ]]`	Empty string
`[[ -n STRING ]]`	Not empty string
`[[ STRING == STRING ]]`	Equal
`[[ STRING != STRING ]]`	Not Equal
`[[ NUM -eq NUM ]]`	Equal
`[[ NUM -ne NUM ]]`	Not equal
`[[ NUM -lt NUM ]]`	Less than
`[[ NUM -le NUM ]]`	Less than or equal
`[[ NUM -gt NUM ]]`	Greater than
`[[ NUM -ge NUM ]]`	Greater than or equal
`[[ STRING =~ STRING ]]`	Regexp
`(( NUM < NUM ))`	Numeric conditions

`[[ -o noclobber ]]`	If OPTIONNAME is enabled
`[[ ! EXPR ]]`	Not
`[[ X ]] && [[ Y ]]`	And
`[[ X ]] \|\| [[ Y ]]`	Or

`[[ -e FILE ]]`	Exists
`[[ -r FILE ]]`	Readable
`[[ -h FILE ]]`	Symlink
`[[ -d FILE ]]`	Directory
`[[ -w FILE ]]`	Writable
`[[ -s FILE ]]`	Size is > 0 bytes
`[[ -f FILE ]]`	File
`[[ -x FILE ]]`	Executable
`[[ FILE1 -nt FILE2 ]]`	1 is more recent than 2
`[[ FILE1 -ot FILE2 ]]`	2 is more recent than 1
`[[ FILE1 -ef FILE2 ]]`	Same files

`history`	Show history
`shopt -s histverify`	Don’t execute expanded result immediately

`!!:n`	Expand only `n`th token from most recent command (command is `0`; first argument is `1`)
`!^`	Expand first argument from most recent command
`!$`	Expand last token from most recent command
`!!:n-m`	Expand range of tokens from most recent command
`!!:n-$`	Expand `n`th token to last from most recent command

`$?`	Exit status of last task
`$!`	PID of last background task
`$$`	PID of shell
`$0`	Filename of the shell script