Skip to content
LumenaryAcademy
Practice 55 min

Functions, arguments, arrays and script structure

Writing functions that behave, handling arguments and options like a real command, storing lists and tables in arrays, and laying out a script professionals would accept.

Code tested · 5AI-draftedAwaiting review by a computer scientist
MotivationH

When a one-liner becomes a program

Most shell scripts start life as a command you typed twice. Then it grows a variable, an if, a loop, an option, and one day it is two hundred lines long, run by other people, from a scheduler, with arguments you never thought of. The difference between a script that survives that growth and one that becomes everyone's nightmare is structure: functions with clear inputs and outputs, careful handling of arguments, data kept in proper arrays rather than space-separated strings, and a predictable layout with a usage message and meaningful exit statuses.

Bash's functions and arrays are less powerful than those of a general-purpose language, and their rules are a little odd: functions cannot really return values, and variables are global unless you say otherwise. This lesson shows how experienced shell programmers work within those rules. Its last section is also about knowing when not to: when a script has outgrown Bash and should be rewritten in a language like Python.

DefinitionsH

Functions

A function is defined with name() { commands; } and called like any command, with arguments. Inside it, the arguments are $1, $2, … and "$@", just like a script's. (The alternative syntax function name { ... } comes from the Korn shell; the name() form is the portable one.)

bash
#!/usr/bin/env bash
greet() {
    local name=${1:-world}            # local: invisible outside the function
    local mark=${2:-!}
    printf 'Hello, %s%s\n' "$name" "$mark"
}
greet
greet Ama "?"

leaky() { temp="I escaped"; }         # no local: the variable is global
leaky
echo "after leaky: $temp"

is_even() { (( $1 % 2 == 0 )); }      # "returns" a status: its last command's
for n in 3 4; do
    if is_even "$n"; then echo "$n is even"; else echo "$n is odd"; fi
done

square() { echo $(( $1 * $1 )); }     # "returns" data: by printing it
result=$(square 7)
echo "7 squared is $result"

outer() { local x="outer's x"; inner; }
inner() { echo "inner sees: $x"; }    # dynamic scope: sees its caller's locals
x="global x"
outer
inner
text
Hello, world!
Hello, Ama?
after leaky: I escaped
3 is odd
4 is even
7 squared is 49
inner sees: outer's x
inner sees: global x

Four rules explain everything above:

  • Variables are global by default. leaky set temp for the whole script. Declare every variable a function uses with local, unless you mean to change a global.
  • A function's status is its last command's status, or the number given to return (0 to 255). That is how is_even works as a condition: its body is a single (( )) test. return is for success or failure, never for data.
  • To return data, print it and let the caller capture it with $( ), as square does. (This runs the function in a subshell, so it cannot change the caller's variables; when that matters, a function can assign to a global, or to a variable whose name it is given, using a nameref, local -n result=$1, available since Bash 4.3.)
  • Scope is dynamic. A function sees the local variables of the function that called it, which is why inner saw outer's x. This is different from almost every modern language, and another reason to pass values as arguments rather than relying on what happens to be in scope.
ComputationC

Arguments and the environment

Arguments arrive as positional parameters, and the same rules apply to scripts and functions:

bash
#!/usr/bin/env bash
show() { printf '%d args:' "$#"; printf ' <%s>' "$@"; echo; }

forward() {
    show "$@"                         # exactly the arguments received
    show $*                           # re-split at spaces: usually a bug
    show "$*"                         # joined into one argument
}
forward "a b" c

consume() {
    local first=$1
    shift                             # drop $1; the rest move down
    echo "first=$first, then $# more: $*"
}
consume x y z

set -- one two three                  # replace this shell's own positional parameters
echo "now \$# is $#, \$2 is $2"

plain="not exported"
export SHARED="exported"
bash -c 'echo "a child process sees: [${plain-}] [${SHARED-}]"'
GREETING=hej bash -c 'echo "set for one command only: $GREETING"'
echo "and afterwards: [${GREETING-}]"
text
2 args: <a b> <c>
3 args: <a> <b> <c>
1 args: <a b c>
first=x, then 2 more: y z
now $# is 3, $2 is two
a child process sees: [] [exported]
set for one command only: hej
and afterwards: []
  • "$@" forwards arguments exactly. Use it whenever one function or script passes its arguments on. $* unquoted re-splits them, and "$*" joins them into one string, which is right only for messages.
  • shift drops the first argument and moves the rest down, the usual way to consume a leading subcommand or value. shift 2 drops two.
  • set -- ... replaces the current positional parameters, handy for supplying defaults (lesson 1's greet.sh used it).
  • Only exported variables reach child processes. A child process (another program, or another bash) inherits the environment: exported variables only. VAR=value command sets a variable in the environment of that one command, which is the idiomatic way to configure a program for a single run, for example LC_ALL=C sort or TZ=UTC date.

The builtin source file (or . file) runs another file's commands in the current shell, so its functions and variables become available; that is how ~/.bashrc is loaded and how scripts share a library of functions. Running the file as a program instead would execute it in a child process, and its definitions would vanish when it exited.

Worked examplesH

Arrays

A string is a poor container for a list: you saw in lesson 2 how word splitting destroys elements that contain spaces. Bash has two kinds of array for real lists and tables. Indexed arrays are numbered from 0; associative arrays (Bash 4.0 and later, so not in the old Bash that macOS ships) map string keys to values and must be declared with declare -A.

bash
#!/usr/bin/env bash
fruits=(mango "passion fruit" guava)
fruits+=(lychee)                                  # append
echo "${#fruits[@]} fruits; first: ${fruits[0]}; last: ${fruits[-1]}"
for f in "${fruits[@]}"; do printf '<%s> ' "$f"; done; echo
echo "indices: ${!fruits[*]}; slice: ${fruits[*]:1:2}"
unset 'fruits[1]'                                 # arrays can have gaps
echo "after unset: indices ${!fruits[*]}, values ${fruits[*]}"

declare -A capital=([Kenya]=Nairobi [Peru]=Lima [Japan]=Tokyo)   # Bash 4.0+
capital[Ghana]=Accra
mapfile -t countries < <(printf '%s\n' "${!capital[@]}" | LC_ALL=C sort)
for c in "${countries[@]}"; do
    printf '%-6s -> %s\n' "$c" "${capital[$c]}"
done
[[ -v capital[Chile] ]] || echo "no entry for Chile"

declare -A count=()
for word in the cat and the hat and the bat; do
    (( ++count[$word] ))
done
for word in the and cat; do echo "$word: ${count[$word]}"; done
text
4 fruits; first: mango; last: lychee
<mango> <passion fruit> <guava> <lychee> 
indices: 0 1 2 3; slice: passion fruit guava
after unset: indices 0 2 3, values mango guava lychee
Ghana  -> Accra
Japan  -> Tokyo
Kenya  -> Nairobi
Peru   -> Lima
no entry for Chile
the: 3
and: 2
cat: 1

The syntax is dense, so here it is in one place:

SyntaxMeaning
a=(x "y z"), a+=(w)create; append
"${a[@]}"every element as a separate word: the form to use in loops and arguments
${a[i]}, ${a[-1]}one element; the last element
${#a[@]}number of elements
${!a[@]}the indices (or, for associative arrays, the keys)
${a[@]:1:2}a slice
unset 'a[i]'remove an element (the indices are not renumbered)
[[ -v a[key] ]]does the key exist?
mapfile -t lines < fileread a file (or < <(command)) into an array, one line per element

Associative arrays do not remember insertion order, so the example sorted the keys before printing them. And (( ++count[$word] )) is the shell's version of a word counter: in arithmetic, an unset element counts as 0.

ComputationC

Handing results back

Since a function cannot return data the way a Python or JavaScript function does, Bash programmers use three patterns, each with its place:

bash
#!/usr/bin/env bash
# Three ways for a function to hand back results.

# 1. Print it; the caller captures it. Simple, but runs in a subshell.
basename_of() { printf '%s\n' "${1##*/}"; }
name=$(basename_of /var/log/nginx/access.log)
echo "1. $name"

# 2. Fill a variable the caller names: a nameref (Bash 4.3+). No subshell.
split_version() {
    local -n parts=$1                 # parts is now another name for the caller's array
    IFS=. read -r -a parts <<< "$2"
}
split_version ver "5.1.16"
echo "2. major=${ver[0]} minor=${ver[1]} patch=${ver[2]}"

# 3. Return a status, and print details only on failure.
require_number() {
    [[ $1 =~ ^-?[0-9]+$ ]] && return 0
    echo "not a number: $1" >&2
    return 1
}
for v in 42 4x2; do
    if require_number "$v" 2>/dev/null; then echo "3. $v ok"; else echo "3. $v rejected (status $?)"; fi
done

# A trap: local hides the status of a command substitution
f() { local out=$(false); echo "  with local on the same line: $?"; }
g() { local out; out=$(false); echo "  declared first, then assigned: $?"; }
f; g
text
1. access.log
2. major=5 minor=1 patch=16
3. 42 ok
3. 4x2 rejected (status 1)
  with local on the same line: 0
  declared first, then assigned: 1
  1. Print and capture is the default: clear, composable ($(basename_of "$f") works anywhere), and the function cannot accidentally change the caller's variables. Its costs are a subshell per call, which matters in a loop over thousands of items, and the loss of trailing newlines.
  2. A nameref (local -n) lets a function fill an array or several variables for the caller, without a subshell. Choose local variable names that the caller is unlikely to pass in, because a nameref pointing at a variable of the same name refers to itself.
  3. A status plus a message on stderr suits validation and checks: the caller writes if require_number "$v"; then, and the message is there for a human when it fails.

The last two lines show a trap worth remembering. local out=$(false) reports status 0, because the status you see is that of the local command, which succeeded, not of the command substitution. Declare first, then assign, whenever you need the status (and with set -e, which lesson 7 covers, whenever you want a failure to stop the script). ShellCheck warns about this pattern.

Worked examplesH

Anatomy of a command-line script

A script that other people use should behave like any other command: accept options in the usual way, explain itself with -h, report mistakes on stderr, and exit with 0 on success and a non-zero status on misuse (2 is the convention for bad usage). The builtin getopts parses short options, including combined ones like -un3, and stops at the first non-option argument or at --.

bash
#!/usr/bin/env bash
usage() {
    cat <<'EOT'
Usage: shout [-n COUNT] [-u] [-h] WORD...
  -n COUNT  repeat each word COUNT times (default 1)
  -u        upper-case the words
  -h        show this help
EOT
}

main() {
    local count=1 upper=false opt OPTIND        # OPTIND must be local to reuse getopts
    while getopts ':n:uh' opt; do
        case $opt in
            n) count=$OPTARG ;;
            u) upper=true ;;
            h) usage; return 0 ;;
            :) echo "shout: -$OPTARG needs a value" >&2; return 2 ;;
            \?) echo "shout: unknown option -$OPTARG" >&2; return 2 ;;
        esac
    done
    shift $(( OPTIND - 1 ))                       # what remains are the WORDs
    if [[ ! $count =~ ^[1-9][0-9]*$ ]]; then
        echo "shout: COUNT must be a positive whole number, not '$count'" >&2
        return 2
    fi
    if (( $# == 0 )); then
        echo "shout: give at least one WORD (try -h)" >&2
        return 2
    fi
    local word i
    for word in "$@"; do
        if $upper; then word=${word^^}; fi
        for (( i = 0; i < count; i++ )); do printf '%s ' "$word"; done
    done
    echo
}

# A real script ends with:  main "$@"
# Here we call main several times, showing errors (stderr) and statuses too.
for args in "-n 2 -u hola mundo" "-h" "-un3 hi" "-- -u" "-n" "-x hi" "-n zero hi" "-u"; do
    printf '$ shout %s\n' "$args"
    read -r -a argv <<< "$args"
    main "${argv[@]}" 2>&1
    echo "  (status $?)"
done
text
$ shout -n 2 -u hola mundo
HOLA HOLA MUNDO MUNDO 
  (status 0)
$ shout -h
Usage: shout [-n COUNT] [-u] [-h] WORD...
  -n COUNT  repeat each word COUNT times (default 1)
  -u        upper-case the words
  -h        show this help
  (status 0)
$ shout -un3 hi
HI HI HI 
  (status 0)
$ shout -- -u
-u 
  (status 0)
$ shout -n
shout: -n needs a value
  (status 2)
$ shout -x hi
shout: unknown option -x
  (status 2)
$ shout -n zero hi
shout: COUNT must be a positive whole number, not 'zero'
  (status 2)
$ shout -u
shout: give at least one WORD (try -h)
  (status 2)

How it works:

  • The option string ':n:uh' lists the options; a colon after a letter means it takes a value, delivered in OPTARG. The leading colon puts getopts in silent mode, so it reports problems by setting opt to : (missing value) or ? (unknown option) instead of printing its own message, and the script can phrase the error itself.
  • shift $(( OPTIND - 1 )) discards the options once they are parsed, leaving only the operands in "$@".
  • Validate everything that came from outside: the count must be a positive integer, and at least one word is required. Each failure prints a message naming the program, on stderr, and returns 2.
  • local OPTIND matters only because the demonstration calls main several times; getopts keeps its position in OPTIND between calls.

getopts handles only short options. For long options (--count 3), scripts typically loop over the arguments themselves with while (( $# )); do case $1 in ...; esac; shift; done, or use the external getopt program from util-linux, which is not available everywhere. Lesson 8's project shows the loop.

ExplanationH

How professionals lay out a script

A layout that you will recognise in well-maintained repositories, and that the Google Shell Style Guide and others broadly agree on:

bash
#!/usr/bin/env bash
# backup-photos: copy new photos to the archive disk.
set -euo pipefail

readonly ARCHIVE_DIR=${ARCHIVE_DIR:-/mnt/archive}    # settings, overridable from the environment

log()  { printf '%s %s\n' "$(date +%T)" "$*" >&2; }
die()  { log "error: $*"; exit 1; }
usage() { ... }

copy_new_photos() {
    local source=$1 count=0
    ...
}

main() {
    # parse options, validate, then call the functions above
    ...
}

main "$@"

The habits it encodes: a shebang and a one-line description; strict mode (lesson 7); constants and settings at the top, in capitals, marked readonly; small functions that use local for everything; log and die helpers that write to stderr; all real work inside functions; and a single call, main "$@", on the last line. That last line also protects you from a half-downloaded script: Bash reads scripts as it runs them, and if the file is cut off, nothing runs because main is never reached. If the file is meant to be both run and sourced as a library, guard the call with [[ ${BASH_SOURCE[0]} == "$0" ]] && main "$@".

Structured this way, a script can also be tested. Because the work lives in functions and main is called only on the last line, a test file can source the script and call its functions directly with prepared inputs. The most widely used framework for this is bats-core, in which each test is a small block of shell that runs a command and asserts on its status and output. Even without a framework, a tests.sh that calls your functions and compares results with expected values catches most regressions, and it runs anywhere Bash does.

When to stop using Bash. Shell is excellent at running programs and connecting their input and output. It is poor at data structures, arithmetic beyond integers, error handling and anything that needs to be tested in depth. A common rule of thumb among engineers is to switch to Python (or another general-purpose language) when a script passes a couple of hundred lines, needs nested data structures, or does more computing than orchestrating. Keeping a small Bash wrapper that calls a well-tested program is often the best of both worlds.

Common misconceptionsH

Common misconceptions

Misconception

"return 42 returns 42 to the caller, like other languages." It sets the function's exit status to 42, which the caller can only inspect through $? or an if. Statuses are limited to 0 to 255 and meant for success or failure. Return data by printing it.

Misconception

"Variables inside a function are local." They are global unless declared with local (or declare inside the function). Forgetting local in a function called from a loop is a classic way to clobber the loop's own variable.

Misconception

"Inside a function, $1 is the script's first argument." Each function call has its own positional parameters. To use the script's arguments in a function, pass them in: main "$@".

Misconception

"Associative arrays work in any Bash." They need Bash 4.0 or later. The /bin/bash on macOS is version 3.2, so a script using declare -A fails there unless it is run with a newer Bash, which is one more reason to use #!/usr/bin/env bash and to state the minimum version in your README.

Misconception

"A space-separated string is a good enough list." It breaks as soon as an element contains a space or a wildcard. Use an array and "${array[@]}".

Exercises

Exercises

Run the functions in a terminal to check your predictions.

1. Functions, arguments, arrays and script structure

Warm-upComputational

What does this print?

total=10
add() { total=$(( total + $1 )); }
add 5
add 7
echo "$total"

2. Functions, arguments, arrays and script structure

StretchComputational

What does this print?

f() { return 300; }
f
echo $?

3. Functions, arguments, arrays and script structure

CoreComputational

What does this print?

count() { echo "$#"; }
count "a b" "" c

4. Functions, arguments, arrays and script structure

CoreComputational

What does the last line print?

list=(x "y z" w)
unset 'list[0]'
echo "${!list[@]}"

5. Functions, arguments, arrays and script structure

CoreComputational

What does this print?

g() { local v="in"; }
h() { v="changed"; }
v=out
g; echo -n "$v "
h; echo "$v"

6. Functions, arguments, arrays and script structure

CoreComputational

A script calls getopts 'vo:f' opt. Which of these command lines are parsed without an error? Choose all that apply.

Choose all that apply

7. Functions, arguments, arrays and script structure

Warm-upComputational

A wrapper script should pass all of its arguments, unchanged, to rsync. Which line does that?

Choose one

Further reading and resources